2023-08-27T10:59:42.836 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc 2023-08-27T10:59:42.837 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --name rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi --verbose --archive /home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381530 --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381530/orig.config.yaml 2023-08-27T10:59:42.887 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-27T10:59:42.964 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381530 branch: wip-rishabh-xiubo-wip-57154 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: null first_in_suite: false job_id: '7381530' kernel: client: sha1: distro flavor: default kdb: true sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-xiubo-wip-57154 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-xiubo-wip-57154 sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-xiubo-wip-57154 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1da267a3afa9ab8f9b1d164359066f8bd682ffd8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi135.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 449a1bc2027504e7b3c3d7b30fa4178906581da7 timestamp: 2023-08-26_02:16:12 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1553 2023-08-27T10:59:42.965 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1da267a3afa9ab8f9b1d164359066f8bd682ffd8/qa; will attempt to use it 2023-08-27T10:59:42.966 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1da267a3afa9ab8f9b1d164359066f8bd682ffd8/qa/tasks 2023-08-27T10:59:42.966 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-27T10:59:42.971 INFO:teuthology.task.internal:Checking packages... 2023-08-27T10:59:42.995 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1da267a3afa9ab8f9b1d164359066f8bd682ffd8' 2023-08-27T10:59:42.995 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-27T10:59:42.996 INFO:teuthology.packaging:ref: None 2023-08-27T10:59:42.996 INFO:teuthology.packaging:tag: None 2023-08-27T10:59:42.996 INFO:teuthology.packaging:branch: wip-rishabh-xiubo-wip-57154 2023-08-27T10:59:42.996 INFO:teuthology.packaging:sha1: 1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T10:59:42.997 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-xiubo-wip-57154 2023-08-27T10:59:43.214 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5707.g1da267a3 2023-08-27T10:59:43.215 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-27T10:59:43.221 INFO:teuthology.task.internal:no buildpackages task found 2023-08-27T10:59:43.222 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-27T10:59:43.260 INFO:teuthology.task.internal:Saving configuration 2023-08-27T10:59:43.280 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-27T10:59:43.285 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-27T10:59:43.310 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381530', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-27 10:54:36.260079', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-27T10:59:43.331 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381530', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-27 10:54:36.261401', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-27T10:59:43.332 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-27T10:59:43.338 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-08-27T10:59:43.338 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-08-27T10:59:43.338 INFO:teuthology.run_tasks:Running task console_log... 2023-08-27T10:59:43.428 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3f81f228b0>, signals=[15]) 2023-08-27T10:59:43.429 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-27T10:59:43.435 INFO:teuthology.task.internal:Opening connections... 2023-08-27T10:59:43.435 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T10:59:43.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-27T10:59:43.510 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2023-08-27T10:59:43.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-27T10:59:43.578 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-27T10:59:43.585 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2023-08-27T10:59:43.607 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2023-08-27T10:59:43.608 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2023-08-27T10:59:43.667 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Red Hat Enterprise Linux" 2023-08-27T10:59:43.668 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8.6 (Ootpa)" 2023-08-27T10:59:43.668 INFO:teuthology.orchestra.run.smithi045.stdout:ID="rhel" 2023-08-27T10:59:43.668 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="fedora" 2023-08-27T10:59:43.668 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8.6" 2023-08-27T10:59:43.669 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2023-08-27T10:59:43.669 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-27T10:59:43.669 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2023-08-27T10:59:43.669 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-27T10:59:43.670 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.redhat.com/" 2023-08-27T10:59:43.670 INFO:teuthology.orchestra.run.smithi045.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-27T10:59:43.670 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-27T10:59:43.670 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T10:59:43.671 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-27T10:59:43.671 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-27T10:59:43.671 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-27T10:59:43.672 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-27T10:59:43.672 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-08-27T10:59:43.696 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2023-08-27T10:59:43.716 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2023-08-27T10:59:43.717 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2023-08-27T10:59:43.777 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="Red Hat Enterprise Linux" 2023-08-27T10:59:43.777 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="8.6 (Ootpa)" 2023-08-27T10:59:43.778 INFO:teuthology.orchestra.run.smithi135.stdout:ID="rhel" 2023-08-27T10:59:43.778 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="fedora" 2023-08-27T10:59:43.778 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="8.6" 2023-08-27T10:59:43.779 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el8" 2023-08-27T10:59:43.779 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-27T10:59:43.779 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2023-08-27T10:59:43.779 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-27T10:59:43.780 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://www.redhat.com/" 2023-08-27T10:59:43.780 INFO:teuthology.orchestra.run.smithi135.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-27T10:59:43.780 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-27T10:59:43.780 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T10:59:43.780 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-27T10:59:43.781 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-27T10:59:43.781 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-27T10:59:43.781 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-27T10:59:43.782 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2023-08-27T10:59:43.804 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-27T10:59:43.814 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-27T10:59:43.852 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-27T10:59:43.853 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2023-08-27T10:59:43.855 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2023-08-27T10:59:43.875 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-27T10:59:43.881 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-27T10:59:43.881 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2023-08-27T10:59:43.916 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2023-08-27T10:59:43.945 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-27T10:59:44.001 INFO:teuthology.run_tasks:Running task kernel... 2023-08-27T10:59:44.018 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'} 2023-08-27T10:59:44.019 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-08-27T10:59:44.019 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-08-27T10:59:44.020 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-27T10:59:44.021 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-27T10:59:44.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T10:59:44.040 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2023-08-27T10:59:44.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T10:59:44.042 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-08-27T10:59:44.099 INFO:teuthology.orchestra.run.smithi135.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-27T10:59:44.100 INFO:teuthology.task.kernel:Running kernel on smithi135: 4.18.0-372.9.1.el8.x86_64 2023-08-27T10:59:44.100 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum install -y kernel 2023-08-27T10:59:44.102 INFO:teuthology.orchestra.run.smithi045.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-27T10:59:44.103 INFO:teuthology.task.kernel:Running kernel on smithi045: 4.18.0-372.9.1.el8.x86_64 2023-08-27T10:59:44.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2023-08-27T10:59:45.305 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-08-27T10:59:45.306 INFO:teuthology.orchestra.run.smithi045.stdout:Unable to read consumer identity 2023-08-27T10:59:45.315 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-08-27T10:59:45.316 INFO:teuthology.orchestra.run.smithi135.stdout:Unable to read consumer identity 2023-08-27T10:59:46.732 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-08-27T10:59:46.802 INFO:teuthology.orchestra.run.smithi135.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-08-27T10:59:47.424 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-08-27T10:59:48.003 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-08-27T10:59:51.066 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 338 kB/s | 24 kB 00:00 2023-08-27T10:59:51.511 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 403 kB/s | 24 kB 00:00 2023-08-27T10:59:51.744 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-08-27T10:59:51.805 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-08-27T10:59:51.806 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2023-08-27T10:59:51.806 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-08-27T10:59:51.930 DEBUG:teuthology.orchestra.run.smithi045:> echo no | sudo yum reinstall kernel || true 2023-08-27T10:59:52.205 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-08-27T10:59:52.258 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-27T10:59:52.259 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-27T10:59:52.259 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-27T10:59:52.283 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-08-27T10:59:52.283 INFO:teuthology.orchestra.run.smithi045.stdout:Unable to read consumer identity 2023-08-27T10:59:52.393 DEBUG:teuthology.orchestra.run.smithi135:> echo no | sudo yum reinstall kernel || true 2023-08-27T10:59:52.560 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Aug 2023 10:59:50 AM UTC. 2023-08-27T10:59:52.606 INFO:teuthology.orchestra.run.smithi045.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-27T10:59:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:Error: No packages marked for reinstall. 2023-08-27T10:59:52.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum reinstall -y kernel || true 2023-08-27T10:59:52.724 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-08-27T10:59:52.724 INFO:teuthology.orchestra.run.smithi135.stdout:Unable to read consumer identity 2023-08-27T10:59:52.982 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:01 ago on Sun 27 Aug 2023 10:59:51 AM UTC. 2023-08-27T10:59:53.025 INFO:teuthology.orchestra.run.smithi135.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-27T10:59:53.054 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-08-27T10:59:53.055 INFO:teuthology.orchestra.run.smithi045.stdout:Unable to read consumer identity 2023-08-27T10:59:53.064 INFO:teuthology.orchestra.run.smithi135.stderr:Error: No packages marked for reinstall. 2023-08-27T10:59:53.111 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum reinstall -y kernel || true 2023-08-27T10:59:53.328 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:03 ago on Sun 27 Aug 2023 10:59:50 AM UTC. 2023-08-27T10:59:53.372 INFO:teuthology.orchestra.run.smithi045.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-27T10:59:53.406 INFO:teuthology.orchestra.run.smithi045.stderr:Error: No packages marked for reinstall. 2023-08-27T10:59:53.438 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-08-27T10:59:53.439 INFO:teuthology.orchestra.run.smithi135.stdout:Unable to read consumer identity 2023-08-27T10:59:53.452 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2023-08-27T10:59:53.535 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-08-27T10:59:53.536 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-08-27T10:59:53.536 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-08-27T10:59:53.537 DEBUG:teuthology.parallel:result is None 2023-08-27T10:59:53.693 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Aug 2023 10:59:51 AM UTC. 2023-08-27T10:59:53.737 INFO:teuthology.orchestra.run.smithi135.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-27T10:59:53.765 INFO:teuthology.orchestra.run.smithi135.stderr:Error: No packages marked for reinstall. 2023-08-27T10:59:53.817 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2023-08-27T10:59:53.899 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-08-27T10:59:53.900 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-08-27T10:59:53.900 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-08-27T10:59:53.900 DEBUG:teuthology.parallel:result is None 2023-08-27T10:59:53.901 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-27T10:59:53.909 INFO:teuthology.task.internal:Creating test directory... 2023-08-27T10:59:53.912 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-27T10:59:53.914 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-27T10:59:53.931 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-27T10:59:53.943 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-27T10:59:53.972 INFO:teuthology.task.internal:Creating archive directory... 2023-08-27T10:59:53.973 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-27T10:59:53.975 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-27T10:59:53.997 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-27T10:59:54.007 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-27T10:59:54.007 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-27T10:59:54.034 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-27T10:59:54.061 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-27T10:59:54.073 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-27T10:59:54.082 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-27T10:59:54.090 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-27T10:59:54.091 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-27T10:59:54.098 INFO:teuthology.task.internal:Configuring sudo... 2023-08-27T10:59:54.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-27T10:59:54.126 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-27T10:59:54.158 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-27T10:59:54.167 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-27T10:59:54.168 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-27T10:59:54.194 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-27T10:59:54.214 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-27T10:59:54.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-27T10:59:54.341 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-27T10:59:54.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-27T10:59:54.468 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T10:59:54.469 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-27T10:59:54.536 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-27T10:59:54.579 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-27T10:59:54.633 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-27T10:59:54.716 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-27T10:59:54.757 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T10:59:54.757 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-27T10:59:54.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2023-08-27T10:59:54.826 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2023-08-27T10:59:54.884 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-27T10:59:54.924 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-27T10:59:55.251 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-27T10:59:55.260 INFO:teuthology.task.internal:Starting timer... 2023-08-27T10:59:55.261 INFO:teuthology.run_tasks:Running task pcp... 2023-08-27T10:59:55.306 INFO:teuthology.run_tasks:Running task selinux... 2023-08-27T10:59:55.330 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-08-27T10:59:55.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2023-08-27T10:59:55.393 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2023-08-27T10:59:55.395 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2023-08-27T10:59:55.457 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2023-08-27T10:59:55.459 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-27T10:59:55.459 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2023-08-27T10:59:55.484 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2023-08-27T10:59:55.485 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2023-08-27T10:59:55.505 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2023-08-27T10:59:55.506 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive'} 2023-08-27T10:59:55.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-27T10:59:55.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T10:59:55.554 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-27T10:59:55.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T10:59:55.583 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-27T10:59:55.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2023-08-27T10:59:55.622 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2023-08-27T10:59:55.653 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-27T10:59:55.663 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-08-27T10:59:55.669 INFO:teuthology.repo_utils:Fetching main from origin 2023-08-27T10:59:55.822 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-27T10:59:55.867 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-27T10:59:55.868 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi135.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-27T11:08:42.824 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com')] 2023-08-27T11:08:42.825 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-08-27T11:08:42.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-27T11:08:42.903 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-08-27T11:08:42.973 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-08-27T11:08:42.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-08-27T11:08:42.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-27T11:08:43.091 DEBUG:teuthology.orchestra.run.smithi135:> true 2023-08-27T11:08:43.160 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-08-27T11:08:43.161 INFO:teuthology.run_tasks:Running task clock... 2023-08-27T11:08:43.175 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-27T11:08:43.176 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-27T11:08:43.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-27T11:08:43.180 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-27T11:08:43.180 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-27T11:08:43.210 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-27T11:08:43.225 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-27T11:08:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-27T11:08:43.255 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-27T11:08:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2023-08-27T11:08:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2023-08-27T11:08:43.318 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2023-08-27T11:08:43.320 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-27T11:08:43.330 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2023-08-27T11:08:43.335 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-27T11:08:43.345 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-27T11:08:43.359 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-27T11:08:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:bash: ntpq: command not found 2023-08-27T11:08:43.412 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-27T11:08:43.412 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-08-27T11:08:43.413 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:08:43.413 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:08:43.413 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:08:43.413 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:08:43.441 INFO:teuthology.orchestra.run.smithi135.stderr:bash: ntpq: command not found 2023-08-27T11:08:43.444 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-27T11:08:43.445 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2023-08-27T11:08:43.445 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:08:43.445 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:08:43.445 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:08:43.445 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-27T11:08:43.446 INFO:teuthology.run_tasks:Running task install... 2023-08-27T11:08:43.456 DEBUG:teuthology.task.install:project ceph 2023-08-27T11:08:43.456 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}} 2023-08-27T11:08:43.457 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'} 2023-08-27T11:08:43.457 INFO:teuthology.task.install:Using flavor: default 2023-08-27T11:08:43.480 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-27T11:08:43.481 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-27T11:08:43.482 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8', 'tag': None, 'wait_for_package': False} 2023-08-27T11:08:43.482 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T11:08:43.484 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8', 'tag': None, 'wait_for_package': False} 2023-08-27T11:08:43.484 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T11:08:43.625 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/ 2023-08-27T11:08:43.625 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5707.g1da267a3 2023-08-27T11:08:43.761 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/ 2023-08-27T11:08:43.762 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5707.g1da267a3 2023-08-27T11:08:43.810 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-27T11:08:43.811 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:08:43.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-27T11:08:43.840 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-27T11:08:43.840 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-27T11:08:43.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-27T11:08:43.931 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-xiubo-wip-57154/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-27T11:08:43.932 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:08:43.932 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-27T11:08:43.963 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-27T11:08:43.964 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1da267a3afa9ab8f9b1d164359066f8bd682ffd8/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-27T11:08:43.989 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-27T11:08:44.033 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-27T11:08:44.056 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2023-08-27T11:08:44.057 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2023-08-27T11:08:44.111 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-27T11:08:44.178 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2023-08-27T11:08:44.180 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2023-08-27T11:08:44.448 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-08-27T11:08:44.549 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-08-27T11:08:44.722 INFO:teuthology.orchestra.run.smithi045.stdout:44 files removed 2023-08-27T11:08:44.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-27T11:08:44.786 INFO:teuthology.orchestra.run.smithi135.stdout:44 files removed 2023-08-27T11:08:44.832 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-27T11:08:45.119 INFO:teuthology.orchestra.run.smithi045.stdout:Updating Subscription Management repositories. 2023-08-27T11:08:45.162 INFO:teuthology.orchestra.run.smithi135.stdout:Updating Subscription Management repositories. 2023-08-27T11:08:45.772 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 272 kB/s | 81 kB 00:00 2023-08-27T11:08:45.894 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 207 kB/s | 81 kB 00:00 2023-08-27T11:08:46.215 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-08-27T11:08:46.278 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-08-27T11:08:46.532 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 5.1 kB/s | 1.5 kB 00:00 2023-08-27T11:08:46.571 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2023-08-27T11:08:46.692 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-08-27T11:08:46.722 INFO:teuthology.orchestra.run.smithi135.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-08-27T11:08:47.675 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-08-27T11:08:48.644 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 16 MB 00:01 2023-08-27T11:08:51.325 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 341 kB/s | 24 kB 00:00 2023-08-27T11:08:52.265 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 390 kB/s | 24 kB 00:00 2023-08-27T11:08:52.327 INFO:teuthology.orchestra.run.smithi045.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 53 MB 00:00 2023-08-27T11:08:53.176 INFO:teuthology.orchestra.run.smithi135.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-08-27T11:09:01.003 INFO:teuthology.orchestra.run.smithi045.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-08-27T11:09:01.486 INFO:teuthology.orchestra.run.smithi135.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 MB/s | 47 MB 00:00 2023-08-27T11:09:12.599 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-27T11:09:12.600 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-27T11:09:12.615 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-27T11:09:12.616 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-27T11:09:12.617 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-27T11:09:12.619 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-27T11:09:12.621 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-27T11:09:12.624 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-27T11:09:12.626 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-27T11:09:12.635 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-27T11:09:12.636 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-27T11:09:12.651 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-27T11:09:12.652 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-27T11:09:12.653 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-27T11:09:12.654 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-27T11:09:12.656 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-27T11:09:12.660 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-27T11:09:12.661 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-27T11:09:12.755 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-27T11:09:12.768 INFO:teuthology.orchestra.run.smithi135.stdout:=========================================================================================================================== 2023-08-27T11:09:12.768 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-27T11:09:12.768 INFO:teuthology.orchestra.run.smithi135.stdout:=========================================================================================================================== 2023-08-27T11:09:12.769 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-27T11:09:12.769 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-27T11:09:12.769 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-27T11:09:12.769 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 6.4 k 2023-08-27T11:09:12.769 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 5.4 M 2023-08-27T11:09:12.770 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 867 k 2023-08-27T11:09:12.770 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 167 k 2023-08-27T11:09:12.770 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 1.6 M 2023-08-27T11:09:12.770 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 132 k 2023-08-27T11:09:12.770 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 1.7 M 2023-08-27T11:09:12.771 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 7.4 M 2023-08-27T11:09:12.771 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 47 k 2023-08-27T11:09:12.771 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 14 M 2023-08-27T11:09:12.771 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 48 M 2023-08-27T11:09:12.772 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 261 k 2023-08-27T11:09:12.772 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 171 k 2023-08-27T11:09:12.772 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 260 k 2023-08-27T11:09:12.772 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 18 k 2023-08-27T11:09:12.772 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-27T11:09:12.773 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-27T11:09:12.773 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-27T11:09:12.773 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 31 k 2023-08-27T11:09:12.773 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 789 k 2023-08-27T11:09:12.773 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 139 k 2023-08-27T11:09:12.774 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-27T11:09:12.774 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-27T11:09:12.774 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-27T11:09:12.774 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-27T11:09:12.774 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-27T11:09:12.775 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-27T11:09:12.775 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 197 k 2023-08-27T11:09:12.775 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 371 k 2023-08-27T11:09:12.775 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 364 k 2023-08-27T11:09:12.775 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 107 k 2023-08-27T11:09:12.775 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 94 k 2023-08-27T11:09:12.776 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.5 M 2023-08-27T11:09:12.776 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 179 k 2023-08-27T11:09:12.776 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-27T11:09:12.776 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2023-08-27T11:09:12.776 INFO:teuthology.orchestra.run.smithi135.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-27T11:09:12.777 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.7 M 2023-08-27T11:09:12.777 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.6 M 2023-08-27T11:09:12.777 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-27T11:09:12.777 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-27T11:09:12.778 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 22 M 2023-08-27T11:09:12.778 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 23 k 2023-08-27T11:09:12.778 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 2.3 M 2023-08-27T11:09:12.778 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 242 k 2023-08-27T11:09:12.778 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 4.6 M 2023-08-27T11:09:12.779 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 18 M 2023-08-27T11:09:12.779 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 15 k 2023-08-27T11:09:12.779 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 24 k 2023-08-27T11:09:12.779 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-27T11:09:12.779 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-27T11:09:12.780 INFO:teuthology.orchestra.run.smithi135.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-27T11:09:12.780 INFO:teuthology.orchestra.run.smithi135.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-27T11:09:12.780 INFO:teuthology.orchestra.run.smithi135.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-27T11:09:12.780 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 139 k 2023-08-27T11:09:12.780 INFO:teuthology.orchestra.run.smithi135.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-27T11:09:12.781 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-27T11:09:12.781 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-27T11:09:12.781 INFO:teuthology.orchestra.run.smithi135.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-27T11:09:12.781 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-27T11:09:12.781 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-27T11:09:12.782 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 473 k 2023-08-27T11:09:12.782 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-27T11:09:12.782 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 6.8 M 2023-08-27T11:09:12.782 INFO:teuthology.orchestra.run.smithi135.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-27T11:09:12.782 INFO:teuthology.orchestra.run.smithi135.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-27T11:09:12.783 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-27T11:09:12.783 INFO:teuthology.orchestra.run.smithi135.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-27T11:09:12.783 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-27T11:09:12.783 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-27T11:09:12.783 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-27T11:09:12.784 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-27T11:09:12.784 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-27T11:09:12.784 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-27T11:09:12.784 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-27T11:09:12.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-27T11:09:12.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-27T11:09:12.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-27T11:09:12.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-27T11:09:12.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-27T11:09:12.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 45 k 2023-08-27T11:09:12.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 126 k 2023-08-27T11:09:12.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-27T11:09:12.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-27T11:09:12.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-27T11:09:12.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-27T11:09:12.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-27T11:09:12.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-27T11:09:12.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-27T11:09:12.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-27T11:09:12.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-27T11:09:12.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-27T11:09:12.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-27T11:09:12.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-27T11:09:12.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-27T11:09:12.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-27T11:09:12.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-27T11:09:12.789 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-27T11:09:12.789 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-27T11:09:12.789 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-27T11:09:12.789 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-27T11:09:12.789 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-27T11:09:12.790 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-27T11:09:12.790 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-27T11:09:12.790 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-27T11:09:12.790 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-27T11:09:12.790 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-27T11:09:12.791 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-27T11:09:12.791 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-27T11:09:12.791 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-27T11:09:12.791 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-27T11:09:12.791 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-27T11:09:12.792 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-27T11:09:12.792 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-27T11:09:12.792 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-27T11:09:12.792 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-27T11:09:12.792 INFO:teuthology.orchestra.run.smithi135.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-27T11:09:12.793 INFO:teuthology.orchestra.run.smithi135.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-27T11:09:12.793 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-27T11:09:12.793 INFO:teuthology.orchestra.run.smithi135.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-27T11:09:12.793 INFO:teuthology.orchestra.run.smithi135.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-27T11:09:12.793 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-27T11:09:12.793 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-27T11:09:12.794 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-27T11:09:12.794 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-27T11:09:12.794 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-27T11:09:12.794 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-27T11:09:12.794 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-27T11:09:12.795 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-27T11:09:12.795 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-27T11:09:12.795 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-27T11:09:12.795 INFO:teuthology.orchestra.run.smithi135.stdout:Enabling module streams: 2023-08-27T11:09:12.795 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql 10 2023-08-27T11:09:12.796 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:09:12.796 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-27T11:09:12.796 INFO:teuthology.orchestra.run.smithi135.stdout:=========================================================================================================================== 2023-08-27T11:09:12.796 INFO:teuthology.orchestra.run.smithi135.stdout:Install 128 Packages 2023-08-27T11:09:12.796 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 3 Packages 2023-08-27T11:09:12.797 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:09:12.797 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 206 M 2023-08-27T11:09:12.797 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-27T11:09:12.803 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-08-27T11:09:12.816 INFO:teuthology.orchestra.run.smithi045.stdout:=========================================================================================================================== 2023-08-27T11:09:12.816 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2023-08-27T11:09:12.817 INFO:teuthology.orchestra.run.smithi045.stdout:=========================================================================================================================== 2023-08-27T11:09:12.817 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-08-27T11:09:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-27T11:09:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-27T11:09:12.817 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 6.4 k 2023-08-27T11:09:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 5.4 M 2023-08-27T11:09:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 867 k 2023-08-27T11:09:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 167 k 2023-08-27T11:09:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 1.6 M 2023-08-27T11:09:12.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 132 k 2023-08-27T11:09:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 1.7 M 2023-08-27T11:09:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 7.4 M 2023-08-27T11:09:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 47 k 2023-08-27T11:09:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 14 M 2023-08-27T11:09:12.819 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 48 M 2023-08-27T11:09:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 261 k 2023-08-27T11:09:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 171 k 2023-08-27T11:09:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 260 k 2023-08-27T11:09:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 18 k 2023-08-27T11:09:12.820 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-27T11:09:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-27T11:09:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-27T11:09:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 31 k 2023-08-27T11:09:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 789 k 2023-08-27T11:09:12.821 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 139 k 2023-08-27T11:09:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-27T11:09:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-27T11:09:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-27T11:09:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-27T11:09:12.822 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-27T11:09:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-27T11:09:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 197 k 2023-08-27T11:09:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 371 k 2023-08-27T11:09:12.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 364 k 2023-08-27T11:09:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 107 k 2023-08-27T11:09:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 94 k 2023-08-27T11:09:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.5 M 2023-08-27T11:09:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 179 k 2023-08-27T11:09:12.824 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-27T11:09:12.825 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2023-08-27T11:09:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-27T11:09:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.7 M 2023-08-27T11:09:12.825 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 3.6 M 2023-08-27T11:09:12.825 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-08-27T11:09:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-27T11:09:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 22 M 2023-08-27T11:09:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 23 k 2023-08-27T11:09:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 2.3 M 2023-08-27T11:09:12.826 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 242 k 2023-08-27T11:09:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 4.6 M 2023-08-27T11:09:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 18 M 2023-08-27T11:09:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5707.g1da267a3.el8 ceph-noarch 15 k 2023-08-27T11:09:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 24 k 2023-08-27T11:09:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-27T11:09:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-27T11:09:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-27T11:09:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-27T11:09:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-27T11:09:12.828 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 139 k 2023-08-27T11:09:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-27T11:09:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-27T11:09:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-27T11:09:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-27T11:09:12.829 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-27T11:09:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-27T11:09:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 473 k 2023-08-27T11:09:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-27T11:09:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 6.8 M 2023-08-27T11:09:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-27T11:09:12.830 INFO:teuthology.orchestra.run.smithi045.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-27T11:09:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-27T11:09:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-27T11:09:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-27T11:09:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-27T11:09:12.831 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-27T11:09:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-27T11:09:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-27T11:09:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-27T11:09:12.832 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-27T11:09:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-27T11:09:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-27T11:09:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-27T11:09:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-27T11:09:12.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-27T11:09:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 45 k 2023-08-27T11:09:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:18.0.0-5707.g1da267a3.el8 ceph 126 k 2023-08-27T11:09:12.834 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-27T11:09:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-27T11:09:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-27T11:09:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-27T11:09:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-27T11:09:12.835 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-27T11:09:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-27T11:09:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-27T11:09:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-27T11:09:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-27T11:09:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-27T11:09:12.836 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-27T11:09:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-27T11:09:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-27T11:09:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-27T11:09:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-27T11:09:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-27T11:09:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-27T11:09:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-27T11:09:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-27T11:09:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-27T11:09:12.838 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-27T11:09:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-27T11:09:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-27T11:09:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-27T11:09:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-27T11:09:12.839 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-27T11:09:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-27T11:09:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-27T11:09:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-27T11:09:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-27T11:09:12.840 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-27T11:09:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-27T11:09:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-27T11:09:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-27T11:09:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-27T11:09:12.841 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-27T11:09:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-27T11:09:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-27T11:09:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-27T11:09:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-27T11:09:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-27T11:09:12.842 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-27T11:09:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-27T11:09:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-27T11:09:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-27T11:09:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-27T11:09:12.843 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-27T11:09:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-27T11:09:12.844 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling module streams: 2023-08-27T11:09:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql 10 2023-08-27T11:09:12.844 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:09:12.844 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-08-27T11:09:12.845 INFO:teuthology.orchestra.run.smithi045.stdout:=========================================================================================================================== 2023-08-27T11:09:12.845 INFO:teuthology.orchestra.run.smithi045.stdout:Install 128 Packages 2023-08-27T11:09:12.845 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 3 Packages 2023-08-27T11:09:12.845 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:09:12.846 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 206 M 2023-08-27T11:09:12.846 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-08-27T11:09:13.103 INFO:teuthology.orchestra.run.smithi045.stdout:(1/131): ceph-18.0.0-5707.g1da267a3.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-08-27T11:09:13.307 INFO:teuthology.orchestra.run.smithi135.stdout:(1/131): ceph-18.0.0-5707.g1da267a3.el8.x86_64. 31 kB/s | 6.4 kB 00:00 2023-08-27T11:09:13.487 INFO:teuthology.orchestra.run.smithi045.stdout:(2/131): ceph-base-18.0.0-5707.g1da267a3.el8.x8 9.4 MB/s | 5.4 MB 00:00 2023-08-27T11:09:13.562 INFO:teuthology.orchestra.run.smithi045.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 2.2 MB/s | 167 kB 00:00 2023-08-27T11:09:13.824 INFO:teuthology.orchestra.run.smithi135.stdout:(2/131): ceph-fuse-18.0.0-5707.g1da267a3.el8.x8 1.6 MB/s | 867 kB 00:00 2023-08-27T11:09:13.871 INFO:teuthology.orchestra.run.smithi045.stdout:(4/131): ceph-mds-18.0.0-5707.g1da267a3.el8.x86 7.4 MB/s | 2.3 MB 00:00 2023-08-27T11:09:13.925 INFO:teuthology.orchestra.run.smithi135.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 1.6 MB/s | 167 kB 00:00 2023-08-27T11:09:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:(5/131): ceph-mgr-18.0.0-5707.g1da267a3.el8.x86 6.7 MB/s | 1.6 MB 00:00 2023-08-27T11:09:14.296 INFO:teuthology.orchestra.run.smithi045.stdout:(6/131): ceph-fuse-18.0.0-5707.g1da267a3.el8.x8 727 kB/s | 867 kB 00:01 2023-08-27T11:09:14.663 INFO:teuthology.orchestra.run.smithi045.stdout:(7/131): ceph-mon-18.0.0-5707.g1da267a3.el8.x86 8.2 MB/s | 4.6 MB 00:00 2023-08-27T11:09:14.800 INFO:teuthology.orchestra.run.smithi135.stdout:(4/131): ceph-mds-18.0.0-5707.g1da267a3.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-08-27T11:09:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:(8/131): ceph-common-18.0.0-5707.g1da267a3.el8. 11 MB/s | 22 MB 00:01 2023-08-27T11:09:14.972 INFO:teuthology.orchestra.run.smithi045.stdout:(9/131): ceph-selinux-18.0.0-5707.g1da267a3.el8 326 kB/s | 24 kB 00:00 2023-08-27T11:09:15.059 INFO:teuthology.orchestra.run.smithi135.stdout:(5/131): ceph-base-18.0.0-5707.g1da267a3.el8.x8 2.7 MB/s | 5.4 MB 00:01 2023-08-27T11:09:15.335 INFO:teuthology.orchestra.run.smithi135.stdout:(6/131): ceph-mgr-18.0.0-5707.g1da267a3.el8.x86 2.9 MB/s | 1.6 MB 00:00 2023-08-27T11:09:16.097 INFO:teuthology.orchestra.run.smithi045.stdout:(10/131): ceph-radosgw-18.0.0-5707.g1da267a3.el 9.5 MB/s | 14 MB 00:01 2023-08-27T11:09:16.172 INFO:teuthology.orchestra.run.smithi045.stdout:(11/131): cephfs-mirror-18.0.0-5707.g1da267a3.e 3.4 MB/s | 260 kB 00:00 2023-08-27T11:09:16.213 INFO:teuthology.orchestra.run.smithi135.stdout:(7/131): ceph-mon-18.0.0-5707.g1da267a3.el8.x86 4.0 MB/s | 4.6 MB 00:01 2023-08-27T11:09:16.231 INFO:teuthology.orchestra.run.smithi045.stdout:(12/131): libcephfs-devel-18.0.0-5707.g1da267a3 534 kB/s | 31 kB 00:00 2023-08-27T11:09:16.381 INFO:teuthology.orchestra.run.smithi045.stdout:(13/131): libcephfs2-18.0.0-5707.g1da267a3.el8. 5.1 MB/s | 789 kB 00:00 2023-08-27T11:09:16.465 INFO:teuthology.orchestra.run.smithi045.stdout:(14/131): libcephsqlite-18.0.0-5707.g1da267a3.e 1.6 MB/s | 139 kB 00:00 2023-08-27T11:09:16.540 INFO:teuthology.orchestra.run.smithi045.stdout:(15/131): librados-devel-18.0.0-5707.g1da267a3. 1.8 MB/s | 139 kB 00:00 2023-08-27T11:09:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:(16/131): libradosstriper1-18.0.0-5707.g1da267a 5.6 MB/s | 473 kB 00:00 2023-08-27T11:09:17.574 INFO:teuthology.orchestra.run.smithi045.stdout:(17/131): librgw2-18.0.0-5707.g1da267a3.el8.x86 7.2 MB/s | 6.8 MB 00:00 2023-08-27T11:09:17.649 INFO:teuthology.orchestra.run.smithi045.stdout:(18/131): python3-ceph-argparse-18.0.0-5707.g1d 599 kB/s | 45 kB 00:00 2023-08-27T11:09:17.716 INFO:teuthology.orchestra.run.smithi045.stdout:(19/131): python3-ceph-common-18.0.0-5707.g1da2 1.8 MB/s | 126 kB 00:00 2023-08-27T11:09:17.800 INFO:teuthology.orchestra.run.smithi045.stdout:(20/131): python3-cephfs-18.0.0-5707.g1da267a3. 2.3 MB/s | 197 kB 00:00 2023-08-27T11:09:17.900 INFO:teuthology.orchestra.run.smithi045.stdout:(21/131): python3-rados-18.0.0-5707.g1da267a3.e 3.6 MB/s | 371 kB 00:00 2023-08-27T11:09:17.984 INFO:teuthology.orchestra.run.smithi045.stdout:(22/131): python3-rbd-18.0.0-5707.g1da267a3.el8 4.3 MB/s | 364 kB 00:00 2023-08-27T11:09:18.059 INFO:teuthology.orchestra.run.smithi045.stdout:(23/131): python3-rgw-18.0.0-5707.g1da267a3.el8 1.4 MB/s | 107 kB 00:00 2023-08-27T11:09:18.126 INFO:teuthology.orchestra.run.smithi045.stdout:(24/131): rbd-fuse-18.0.0-5707.g1da267a3.el8.x8 1.4 MB/s | 94 kB 00:00 2023-08-27T11:09:18.651 INFO:teuthology.orchestra.run.smithi045.stdout:(25/131): rbd-mirror-18.0.0-5707.g1da267a3.el8. 6.7 MB/s | 3.5 MB 00:00 2023-08-27T11:09:18.726 INFO:teuthology.orchestra.run.smithi045.stdout:(26/131): rbd-nbd-18.0.0-5707.g1da267a3.el8.x86 2.3 MB/s | 179 kB 00:00 2023-08-27T11:09:18.810 INFO:teuthology.orchestra.run.smithi045.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5707.g 278 kB/s | 23 kB 00:00 2023-08-27T11:09:18.877 INFO:teuthology.orchestra.run.smithi045.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5707.g1da267a 1.9 MB/s | 132 kB 00:00 2023-08-27T11:09:19.001 INFO:teuthology.orchestra.run.smithi135.stdout:(8/131): ceph-radosgw-18.0.0-5707.g1da267a3.el8 4.9 MB/s | 14 MB 00:02 2023-08-27T11:09:19.126 INFO:teuthology.orchestra.run.smithi135.stdout:(9/131): ceph-selinux-18.0.0-5707.g1da267a3.el8 194 kB/s | 24 kB 00:00 2023-08-27T11:09:19.486 INFO:teuthology.orchestra.run.smithi045.stdout:(29/131): ceph-test-18.0.0-5707.g1da267a3.el8.x 11 MB/s | 48 MB 00:04 2023-08-27T11:09:19.535 INFO:teuthology.orchestra.run.smithi045.stdout:(30/131): ceph-mgr-dashboard-18.0.0-5707.g1da26 2.7 MB/s | 1.7 MB 00:00 2023-08-27T11:09:19.627 INFO:teuthology.orchestra.run.smithi045.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5707.g1d 2.6 MB/s | 242 kB 00:00 2023-08-27T11:09:19.677 INFO:teuthology.orchestra.run.smithi135.stdout:(10/131): ceph-osd-18.0.0-5707.g1da267a3.el8.x8 4.1 MB/s | 18 MB 00:04 2023-08-27T11:09:19.686 INFO:teuthology.orchestra.run.smithi045.stdout:(32/131): ceph-mgr-rook-18.0.0-5707.g1da267a3.e 814 kB/s | 47 kB 00:00 2023-08-27T11:09:19.753 INFO:teuthology.orchestra.run.smithi045.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5707.g1 218 kB/s | 15 kB 00:00 2023-08-27T11:09:19.795 INFO:teuthology.orchestra.run.smithi135.stdout:(11/131): cephfs-mirror-18.0.0-5707.g1da267a3.e 2.2 MB/s | 260 kB 00:00 2023-08-27T11:09:19.861 INFO:teuthology.orchestra.run.smithi135.stdout:(12/131): libcephfs-devel-18.0.0-5707.g1da267a3 467 kB/s | 31 kB 00:00 2023-08-27T11:09:19.945 INFO:teuthology.orchestra.run.smithi045.stdout:(34/131): ceph-volume-18.0.0-5707.g1da267a3.el8 1.3 MB/s | 261 kB 00:00 2023-08-27T11:09:20.021 INFO:teuthology.orchestra.run.smithi045.stdout:(35/131): cephadm-18.0.0-5707.g1da267a3.el8.noa 2.2 MB/s | 171 kB 00:00 2023-08-27T11:09:20.045 INFO:teuthology.orchestra.run.smithi135.stdout:(13/131): libcephfs2-18.0.0-5707.g1da267a3.el8. 4.2 MB/s | 789 kB 00:00 2023-08-27T11:09:20.096 INFO:teuthology.orchestra.run.smithi135.stdout:(14/131): libcephsqlite-18.0.0-5707.g1da267a3.e 2.7 MB/s | 139 kB 00:00 2023-08-27T11:09:20.112 INFO:teuthology.orchestra.run.smithi045.stdout:(36/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-08-27T11:09:20.129 INFO:teuthology.orchestra.run.smithi045.stdout:(37/131): cephfs-top-18.0.0-5707.g1da267a3.el8. 166 kB/s | 18 kB 00:00 2023-08-27T11:09:20.146 INFO:teuthology.orchestra.run.smithi135.stdout:(15/131): librados-devel-18.0.0-5707.g1da267a3. 2.7 MB/s | 139 kB 00:00 2023-08-27T11:09:20.246 INFO:teuthology.orchestra.run.smithi135.stdout:(16/131): libradosstriper1-18.0.0-5707.g1da267a 4.6 MB/s | 473 kB 00:00 2023-08-27T11:09:20.313 INFO:teuthology.orchestra.run.smithi045.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 321 kB/s | 59 kB 00:00 2023-08-27T11:09:20.413 INFO:teuthology.orchestra.run.smithi045.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 751 kB/s | 75 kB 00:00 2023-08-27T11:09:20.440 INFO:teuthology.orchestra.run.smithi045.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 939 kB/s | 306 kB 00:00 2023-08-27T11:09:20.506 INFO:teuthology.orchestra.run.smithi045.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-08-27T11:09:20.564 INFO:teuthology.orchestra.run.smithi045.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.1 MB/s | 476 kB 00:00 2023-08-27T11:09:20.592 INFO:teuthology.orchestra.run.smithi045.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-08-27T11:09:20.623 INFO:teuthology.orchestra.run.smithi045.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 564 kB/s | 33 kB 00:00 2023-08-27T11:09:20.657 INFO:teuthology.orchestra.run.smithi045.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 244 kB/s | 16 kB 00:00 2023-08-27T11:09:20.690 INFO:teuthology.orchestra.run.smithi045.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-08-27T11:09:20.757 INFO:teuthology.orchestra.run.smithi045.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-08-27T11:09:20.782 INFO:teuthology.orchestra.run.smithi045.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2023-08-27T11:09:20.825 INFO:teuthology.orchestra.run.smithi045.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-08-27T11:09:20.850 INFO:teuthology.orchestra.run.smithi045.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2023-08-27T11:09:20.917 INFO:teuthology.orchestra.run.smithi045.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 732 kB/s | 49 kB 00:00 2023-08-27T11:09:20.975 INFO:teuthology.orchestra.run.smithi045.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 7.8 MB/s | 1.2 MB 00:00 2023-08-27T11:09:20.980 INFO:teuthology.orchestra.run.smithi135.stdout:(17/131): librgw2-18.0.0-5707.g1da267a3.el8.x86 9.3 MB/s | 6.8 MB 00:00 2023-08-27T11:09:21.000 INFO:teuthology.orchestra.run.smithi045.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 704 kB/s | 59 kB 00:00 2023-08-27T11:09:21.031 INFO:teuthology.orchestra.run.smithi135.stdout:(18/131): python3-ceph-argparse-18.0.0-5707.g1d 897 kB/s | 45 kB 00:00 2023-08-27T11:09:21.043 INFO:teuthology.orchestra.run.smithi045.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 901 kB/s | 60 kB 00:00 2023-08-27T11:09:21.076 INFO:teuthology.orchestra.run.smithi045.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-08-27T11:09:21.089 INFO:teuthology.orchestra.run.smithi135.stdout:(19/131): python3-ceph-common-18.0.0-5707.g1da2 2.1 MB/s | 126 kB 00:00 2023-08-27T11:09:21.126 INFO:teuthology.orchestra.run.smithi045.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2023-08-27T11:09:21.152 INFO:teuthology.orchestra.run.smithi045.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 446 kB/s | 34 kB 00:00 2023-08-27T11:09:21.173 INFO:teuthology.orchestra.run.smithi135.stdout:(20/131): python3-cephfs-18.0.0-5707.g1da267a3. 2.3 MB/s | 197 kB 00:00 2023-08-27T11:09:21.194 INFO:teuthology.orchestra.run.smithi045.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-08-27T11:09:21.219 INFO:teuthology.orchestra.run.smithi045.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 961 kB/s | 64 kB 00:00 2023-08-27T11:09:21.257 INFO:teuthology.orchestra.run.smithi135.stdout:(21/131): python3-rados-18.0.0-5707.g1da267a3.e 4.4 MB/s | 371 kB 00:00 2023-08-27T11:09:21.261 INFO:teuthology.orchestra.run.smithi045.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 280 kB/s | 19 kB 00:00 2023-08-27T11:09:21.286 INFO:teuthology.orchestra.run.smithi045.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 358 kB/s | 24 kB 00:00 2023-08-27T11:09:21.328 INFO:teuthology.orchestra.run.smithi045.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 437 kB/s | 29 kB 00:00 2023-08-27T11:09:21.340 INFO:teuthology.orchestra.run.smithi135.stdout:(22/131): python3-rbd-18.0.0-5707.g1da267a3.el8 4.3 MB/s | 364 kB 00:00 2023-08-27T11:09:21.345 INFO:teuthology.orchestra.run.smithi045.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 469 kB/s | 27 kB 00:00 2023-08-27T11:09:21.391 INFO:teuthology.orchestra.run.smithi135.stdout:(23/131): python3-rgw-18.0.0-5707.g1da267a3.el8 2.1 MB/s | 107 kB 00:00 2023-08-27T11:09:21.395 INFO:teuthology.orchestra.run.smithi045.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-08-27T11:09:21.420 INFO:teuthology.orchestra.run.smithi045.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-08-27T11:09:21.454 INFO:teuthology.orchestra.run.smithi045.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-08-27T11:09:21.474 INFO:teuthology.orchestra.run.smithi135.stdout:(24/131): rbd-fuse-18.0.0-5707.g1da267a3.el8.x8 1.1 MB/s | 94 kB 00:00 2023-08-27T11:09:21.488 INFO:teuthology.orchestra.run.smithi045.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-08-27T11:09:21.513 INFO:teuthology.orchestra.run.smithi045.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 391 kB/s | 23 kB 00:00 2023-08-27T11:09:21.538 INFO:teuthology.orchestra.run.smithi045.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-08-27T11:09:21.599 INFO:teuthology.orchestra.run.smithi045.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 779 kB/s | 47 kB 00:00 2023-08-27T11:09:21.649 INFO:teuthology.orchestra.run.smithi045.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-08-27T11:09:21.674 INFO:teuthology.orchestra.run.smithi045.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.6 MB/s | 200 kB 00:00 2023-08-27T11:09:21.742 INFO:teuthology.orchestra.run.smithi045.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 463 kB/s | 43 kB 00:00 2023-08-27T11:09:21.767 INFO:teuthology.orchestra.run.smithi045.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2023-08-27T11:09:21.792 INFO:teuthology.orchestra.run.smithi045.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 963 kB/s | 48 kB 00:00 2023-08-27T11:09:21.820 INFO:teuthology.orchestra.run.smithi045.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-08-27T11:09:21.843 INFO:teuthology.orchestra.run.smithi045.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 849 kB/s | 43 kB 00:00 2023-08-27T11:09:21.850 INFO:teuthology.orchestra.run.smithi135.stdout:(25/131): rbd-mirror-18.0.0-5707.g1da267a3.el8. 9.4 MB/s | 3.5 MB 00:00 2023-08-27T11:09:21.859 INFO:teuthology.orchestra.run.smithi045.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-08-27T11:09:21.893 INFO:teuthology.orchestra.run.smithi045.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-08-27T11:09:21.900 INFO:teuthology.orchestra.run.smithi135.stdout:(26/131): rbd-nbd-18.0.0-5707.g1da267a3.el8.x86 3.5 MB/s | 179 kB 00:00 2023-08-27T11:09:21.927 INFO:teuthology.orchestra.run.smithi045.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-08-27T11:09:21.942 INFO:teuthology.orchestra.run.smithi135.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5707.g 552 kB/s | 23 kB 00:00 2023-08-27T11:09:21.952 INFO:teuthology.orchestra.run.smithi045.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-08-27T11:09:22.004 INFO:teuthology.orchestra.run.smithi045.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2023-08-27T11:09:22.025 INFO:teuthology.orchestra.run.smithi135.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5707.g1da267a 1.6 MB/s | 132 kB 00:00 2023-08-27T11:09:22.044 INFO:teuthology.orchestra.run.smithi045.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 5.9 MB/s | 559 kB 00:00 2023-08-27T11:09:22.069 INFO:teuthology.orchestra.run.smithi045.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 277 kB/s | 18 kB 00:00 2023-08-27T11:09:22.111 INFO:teuthology.orchestra.run.smithi045.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-08-27T11:09:22.136 INFO:teuthology.orchestra.run.smithi045.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-08-27T11:09:22.187 INFO:teuthology.orchestra.run.smithi045.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-08-27T11:09:22.226 INFO:teuthology.orchestra.run.smithi135.stdout:(29/131): ceph-mgr-dashboard-18.0.0-5707.g1da26 8.7 MB/s | 1.7 MB 00:00 2023-08-27T11:09:22.237 INFO:teuthology.orchestra.run.smithi045.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 19 MB/s | 2.3 MB 00:00 2023-08-27T11:09:22.262 INFO:teuthology.orchestra.run.smithi045.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-08-27T11:09:22.287 INFO:teuthology.orchestra.run.smithi045.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-08-27T11:09:22.313 INFO:teuthology.orchestra.run.smithi045.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-08-27T11:09:22.338 INFO:teuthology.orchestra.run.smithi045.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-08-27T11:09:22.363 INFO:teuthology.orchestra.run.smithi045.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 772 kB/s | 39 kB 00:00 2023-08-27T11:09:22.397 INFO:teuthology.orchestra.run.smithi045.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-08-27T11:09:22.414 INFO:teuthology.orchestra.run.smithi045.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 3.3 MB/s | 171 kB 00:00 2023-08-27T11:09:22.447 INFO:teuthology.orchestra.run.smithi045.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-08-27T11:09:22.498 INFO:teuthology.orchestra.run.smithi045.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.1 MB/s | 110 kB 00:00 2023-08-27T11:09:22.539 INFO:teuthology.orchestra.run.smithi045.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-08-27T11:09:22.590 INFO:teuthology.orchestra.run.smithi045.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-08-27T11:09:22.640 INFO:teuthology.orchestra.run.smithi045.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-08-27T11:09:22.682 INFO:teuthology.orchestra.run.smithi045.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 928 kB/s | 39 kB 00:00 2023-08-27T11:09:22.741 INFO:teuthology.orchestra.run.smithi045.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-08-27T11:09:22.827 INFO:teuthology.orchestra.run.smithi135.stdout:(30/131): ceph-common-18.0.0-5707.g1da267a3.el8 2.3 MB/s | 22 MB 00:09 2023-08-27T11:09:22.898 INFO:teuthology.orchestra.run.smithi045.stdout:(103/131): python3-scipy-1.0.0-21.module+el8.5. 28 MB/s | 14 MB 00:00 2023-08-27T11:09:22.941 INFO:teuthology.orchestra.run.smithi045.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 2.6 MB/s | 538 kB 00:00 2023-08-27T11:09:22.967 INFO:teuthology.orchestra.run.smithi045.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.2 MB/s | 157 kB 00:00 2023-08-27T11:09:22.986 INFO:teuthology.orchestra.run.smithi135.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5707.g1d 1.5 MB/s | 242 kB 00:00 2023-08-27T11:09:22.992 INFO:teuthology.orchestra.run.smithi045.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-08-27T11:09:23.025 INFO:teuthology.orchestra.run.smithi045.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-08-27T11:09:23.068 INFO:teuthology.orchestra.run.smithi045.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 637 kB/s | 27 kB 00:00 2023-08-27T11:09:23.086 INFO:teuthology.orchestra.run.smithi135.stdout:(32/131): ceph-mgr-diskprediction-local-18.0.0- 8.6 MB/s | 7.4 MB 00:00 2023-08-27T11:09:23.111 INFO:teuthology.orchestra.run.smithi135.stdout:(33/131): ceph-mgr-rook-18.0.0-5707.g1da267a3.e 379 kB/s | 47 kB 00:00 2023-08-27T11:09:23.137 INFO:teuthology.orchestra.run.smithi135.stdout:(34/131): ceph-prometheus-alerts-18.0.0-5707.g1 288 kB/s | 15 kB 00:00 2023-08-27T11:09:23.159 INFO:teuthology.orchestra.run.smithi045.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-08-27T11:09:23.184 INFO:teuthology.orchestra.run.smithi045.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 2.7 MB/s | 320 kB 00:00 2023-08-27T11:09:23.187 INFO:teuthology.orchestra.run.smithi135.stdout:(35/131): ceph-volume-18.0.0-5707.g1da267a3.el8 3.4 MB/s | 261 kB 00:00 2023-08-27T11:09:23.210 INFO:teuthology.orchestra.run.smithi045.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 878 kB/s | 44 kB 00:00 2023-08-27T11:09:23.229 INFO:teuthology.orchestra.run.smithi135.stdout:(36/131): cephfs-top-18.0.0-5707.g1da267a3.el8. 432 kB/s | 18 kB 00:00 2023-08-27T11:09:23.263 INFO:teuthology.orchestra.run.smithi135.stdout:(37/131): cephadm-18.0.0-5707.g1da267a3.el8.noa 1.3 MB/s | 171 kB 00:00 2023-08-27T11:09:23.268 INFO:teuthology.orchestra.run.smithi045.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-08-27T11:09:23.336 INFO:teuthology.orchestra.run.smithi045.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 25 MB/s | 3.7 MB 00:00 2023-08-27T11:09:23.361 INFO:teuthology.orchestra.run.smithi045.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-08-27T11:09:23.386 INFO:teuthology.orchestra.run.smithi045.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-08-27T11:09:23.403 INFO:teuthology.orchestra.run.smithi045.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 3.0 MB/s | 126 kB 00:00 2023-08-27T11:09:23.428 INFO:teuthology.orchestra.run.smithi045.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 478 kB/s | 20 kB 00:00 2023-08-27T11:09:23.453 INFO:teuthology.orchestra.run.smithi045.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-08-27T11:09:23.488 INFO:teuthology.orchestra.run.smithi045.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-08-27T11:09:23.537 INFO:teuthology.orchestra.run.smithi045.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-08-27T11:09:23.621 INFO:teuthology.orchestra.run.smithi045.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 29 MB/s | 4.9 MB 00:00 2023-08-27T11:09:23.671 INFO:teuthology.orchestra.run.smithi045.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-08-27T11:09:23.746 INFO:teuthology.orchestra.run.smithi045.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-08-27T11:09:23.789 INFO:teuthology.orchestra.run.smithi135.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 112 kB/s | 59 kB 00:00 2023-08-27T11:09:23.792 INFO:teuthology.orchestra.run.smithi045.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.9 MB/s | 354 kB 00:00 2023-08-27T11:09:23.855 INFO:teuthology.orchestra.run.smithi045.stdout:(125/131): postgresql-contrib-10.21-2.module+el 12 MB/s | 808 kB 00:00 2023-08-27T11:09:23.930 INFO:teuthology.orchestra.run.smithi045.stdout:(126/131): postgresql-server-10.21-2.module+el8 28 MB/s | 5.1 MB 00:00 2023-08-27T11:09:23.972 INFO:teuthology.orchestra.run.smithi135.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 410 kB/s | 75 kB 00:00 2023-08-27T11:09:23.981 INFO:teuthology.orchestra.run.smithi045.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-08-27T11:09:23.997 INFO:teuthology.orchestra.run.smithi135.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 399 kB/s | 306 kB 00:00 2023-08-27T11:09:24.123 INFO:teuthology.orchestra.run.smithi135.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 349 kB/s | 44 kB 00:00 2023-08-27T11:09:24.290 INFO:teuthology.orchestra.run.smithi135.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-08-27T11:09:24.348 INFO:teuthology.orchestra.run.smithi135.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-08-27T11:09:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:(128/131): librados2-18.0.0-5707.g1da267a3.el8. 8.7 MB/s | 3.7 MB 00:00 2023-08-27T11:09:24.416 INFO:teuthology.orchestra.run.smithi135.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 263 kB/s | 33 kB 00:00 2023-08-27T11:09:24.448 INFO:teuthology.orchestra.run.smithi045.stdout:(129/131): librbd1-18.0.0-5707.g1da267a3.el8.x8 7.8 MB/s | 3.6 MB 00:00 2023-08-27T11:09:24.474 INFO:teuthology.orchestra.run.smithi135.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-08-27T11:09:24.482 INFO:teuthology.orchestra.run.smithi045.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 6.2 MB/s | 798 kB 00:00 2023-08-27T11:09:24.733 INFO:teuthology.orchestra.run.smithi135.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 544 kB/s | 173 kB 00:00 2023-08-27T11:09:24.759 INFO:teuthology.orchestra.run.smithi135.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.3 MB/s | 384 kB 00:00 2023-08-27T11:09:24.876 INFO:teuthology.orchestra.run.smithi135.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 573 kB/s | 81 kB 00:00 2023-08-27T11:09:24.892 INFO:teuthology.orchestra.run.smithi135.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-08-27T11:09:25.003 INFO:teuthology.orchestra.run.smithi135.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 138 kB/s | 18 kB 00:00 2023-08-27T11:09:25.137 INFO:teuthology.orchestra.run.smithi135.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 365 kB/s | 49 kB 00:00 2023-08-27T11:09:25.171 INFO:teuthology.orchestra.run.smithi135.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 4.2 MB/s | 1.2 MB 00:00 2023-08-27T11:09:25.224 INFO:teuthology.orchestra.run.smithi045.stdout:(131/131): ceph-osd-18.0.0-5707.g1da267a3.el8.x 1.6 MB/s | 18 MB 00:10 2023-08-27T11:09:25.227 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-08-27T11:09:25.227 INFO:teuthology.orchestra.run.smithi045.stdout:Total 17 MB/s | 206 MB 00:12 2023-08-27T11:09:25.279 INFO:teuthology.orchestra.run.smithi135.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 413 kB/s | 59 kB 00:00 2023-08-27T11:09:25.305 INFO:teuthology.orchestra.run.smithi135.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 451 kB/s | 60 kB 00:00 2023-08-27T11:09:25.814 INFO:teuthology.orchestra.run.smithi135.stdout:(55/131): ceph-test-18.0.0-5707.g1da267a3.el8.x 7.2 MB/s | 48 MB 00:06 2023-08-27T11:09:25.839 INFO:teuthology.orchestra.run.smithi135.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 29 kB/s | 16 kB 00:00 2023-08-27T11:09:25.864 INFO:teuthology.orchestra.run.smithi135.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 484 kB/s | 283 kB 00:00 2023-08-27T11:09:25.940 INFO:teuthology.orchestra.run.smithi135.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 268 kB/s | 34 kB 00:00 2023-08-27T11:09:26.026 INFO:teuthology.orchestra.run.smithi135.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 399 kB/s | 64 kB 00:00 2023-08-27T11:09:26.082 INFO:teuthology.orchestra.run.smithi135.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 136 kB/s | 19 kB 00:00 2023-08-27T11:09:26.094 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-08-27T11:09:26.149 INFO:teuthology.orchestra.run.smithi135.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 194 kB/s | 24 kB 00:00 2023-08-27T11:09:26.208 INFO:teuthology.orchestra.run.smithi135.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 233 kB/s | 29 kB 00:00 2023-08-27T11:09:26.226 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-08-27T11:09:26.227 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-08-27T11:09:26.275 INFO:teuthology.orchestra.run.smithi135.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2023-08-27T11:09:26.342 INFO:teuthology.orchestra.run.smithi135.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-08-27T11:09:26.442 INFO:teuthology.orchestra.run.smithi135.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-08-27T11:09:26.468 INFO:teuthology.orchestra.run.smithi135.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 488 kB/s | 61 kB 00:00 2023-08-27T11:09:26.534 INFO:teuthology.orchestra.run.smithi135.stdout:(67/131): python3-routes-2.4.1-12.el8.noarch.rp 283 kB/s | 196 kB 00:00 2023-08-27T11:09:26.577 INFO:teuthology.orchestra.run.smithi135.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 637 kB/s | 85 kB 00:00 2023-08-27T11:09:26.644 INFO:teuthology.orchestra.run.smithi135.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 131 kB/s | 23 kB 00:00 2023-08-27T11:09:26.669 INFO:teuthology.orchestra.run.smithi135.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 758 kB/s | 70 kB 00:00 2023-08-27T11:09:26.711 INFO:teuthology.orchestra.run.smithi135.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 701 kB/s | 47 kB 00:00 2023-08-27T11:09:26.736 INFO:teuthology.orchestra.run.smithi135.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.2 MB/s | 200 kB 00:00 2023-08-27T11:09:26.770 INFO:teuthology.orchestra.run.smithi135.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 725 kB/s | 43 kB 00:00 2023-08-27T11:09:26.796 INFO:teuthology.orchestra.run.smithi135.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-08-27T11:09:26.821 INFO:teuthology.orchestra.run.smithi135.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 959 kB/s | 48 kB 00:00 2023-08-27T11:09:26.846 INFO:teuthology.orchestra.run.smithi135.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-08-27T11:09:26.905 INFO:teuthology.orchestra.run.smithi135.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 509 kB/s | 43 kB 00:00 2023-08-27T11:09:26.931 INFO:teuthology.orchestra.run.smithi135.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.3 MB/s | 109 kB 00:00 2023-08-27T11:09:26.956 INFO:teuthology.orchestra.run.smithi135.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-08-27T11:09:26.989 INFO:teuthology.orchestra.run.smithi135.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-08-27T11:09:27.015 INFO:teuthology.orchestra.run.smithi135.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 318 kB/s | 19 kB 00:00 2023-08-27T11:09:27.048 INFO:teuthology.orchestra.run.smithi135.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 3.4 MB/s | 1.7 MB 00:00 2023-08-27T11:09:27.066 INFO:teuthology.orchestra.run.smithi135.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2023-08-27T11:09:27.099 INFO:teuthology.orchestra.run.smithi135.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 6.5 MB/s | 559 kB 00:00 2023-08-27T11:09:27.125 INFO:teuthology.orchestra.run.smithi135.stdout:(85/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 237 kB/s | 18 kB 00:00 2023-08-27T11:09:27.150 INFO:teuthology.orchestra.run.smithi135.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2023-08-27T11:09:27.209 INFO:teuthology.orchestra.run.smithi135.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.3 MB/s | 605 kB 00:00 2023-08-27T11:09:27.234 INFO:teuthology.orchestra.run.smithi135.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 863 kB/s | 72 kB 00:00 2023-08-27T11:09:27.259 INFO:teuthology.orchestra.run.smithi135.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-08-27T11:09:27.285 INFO:teuthology.orchestra.run.smithi135.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-08-27T11:09:27.302 INFO:teuthology.orchestra.run.smithi135.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-08-27T11:09:27.327 INFO:teuthology.orchestra.run.smithi135.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-08-27T11:09:27.352 INFO:teuthology.orchestra.run.smithi135.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 767 kB/s | 39 kB 00:00 2023-08-27T11:09:27.386 INFO:teuthology.orchestra.run.smithi135.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-08-27T11:09:27.412 INFO:teuthology.orchestra.run.smithi135.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.8 MB/s | 171 kB 00:00 2023-08-27T11:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-08-27T11:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-08-27T11:09:27.437 INFO:teuthology.orchestra.run.smithi135.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-08-27T11:09:27.504 INFO:teuthology.orchestra.run.smithi135.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.1 MB/s | 2.3 MB 00:00 2023-08-27T11:09:27.530 INFO:teuthology.orchestra.run.smithi135.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2023-08-27T11:09:27.547 INFO:teuthology.orchestra.run.smithi135.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-08-27T11:09:27.580 INFO:teuthology.orchestra.run.smithi135.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-08-27T11:09:27.614 INFO:teuthology.orchestra.run.smithi135.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-08-27T11:09:27.639 INFO:teuthology.orchestra.run.smithi135.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 663 kB/s | 39 kB 00:00 2023-08-27T11:09:27.673 INFO:teuthology.orchestra.run.smithi135.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-08-27T11:09:27.707 INFO:teuthology.orchestra.run.smithi135.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-08-27T11:09:27.732 INFO:teuthology.orchestra.run.smithi135.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-08-27T11:09:27.774 INFO:teuthology.orchestra.run.smithi135.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2023-08-27T11:09:27.808 INFO:teuthology.orchestra.run.smithi135.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-08-27T11:09:27.850 INFO:teuthology.orchestra.run.smithi135.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 637 kB/s | 27 kB 00:00 2023-08-27T11:09:27.892 INFO:teuthology.orchestra.run.smithi135.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-08-27T11:09:27.926 INFO:teuthology.orchestra.run.smithi135.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-08-27T11:09:28.018 INFO:teuthology.orchestra.run.smithi135.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-08-27T11:09:28.085 INFO:teuthology.orchestra.run.smithi135.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 6.7 MB/s | 457 kB 00:00 2023-08-27T11:09:28.127 INFO:teuthology.orchestra.run.smithi135.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-08-27T11:09:28.202 INFO:teuthology.orchestra.run.smithi135.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-08-27T11:09:28.227 INFO:teuthology.orchestra.run.smithi135.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 638 kB/s | 64 kB 00:00 2023-08-27T11:09:28.252 INFO:teuthology.orchestra.run.smithi135.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-08-27T11:09:28.278 INFO:teuthology.orchestra.run.smithi135.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-08-27T11:09:28.295 INFO:teuthology.orchestra.run.smithi135.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-08-27T11:09:28.354 INFO:teuthology.orchestra.run.smithi135.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-08-27T11:09:28.396 INFO:teuthology.orchestra.run.smithi135.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-08-27T11:09:28.538 INFO:teuthology.orchestra.run.smithi135.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 12 MB/s | 14 MB 00:01 2023-08-27T11:09:28.580 INFO:teuthology.orchestra.run.smithi135.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-08-27T11:09:28.649 INFO:teuthology.orchestra.run.smithi135.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.1 MB/s | 354 kB 00:00 2023-08-27T11:09:28.747 INFO:teuthology.orchestra.run.smithi135.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 11 MB/s | 4.9 MB 00:00 2023-08-27T11:09:28.823 INFO:teuthology.orchestra.run.smithi135.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2023-08-27T11:09:28.865 INFO:teuthology.orchestra.run.smithi135.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.7 MB/s | 808 kB 00:00 2023-08-27T11:09:28.943 INFO:teuthology.orchestra.run.smithi135.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-08-27T11:09:29.024 INFO:teuthology.orchestra.run.smithi135.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-08-27T11:09:29.074 INFO:teuthology.orchestra.run.smithi135.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-08-27T11:09:29.366 INFO:teuthology.orchestra.run.smithi135.stdout:(130/131): librados2-18.0.0-5707.g1da267a3.el8. 7.4 MB/s | 3.7 MB 00:00 2023-08-27T11:09:29.499 INFO:teuthology.orchestra.run.smithi135.stdout:(131/131): librbd1-18.0.0-5707.g1da267a3.el8.x8 6.5 MB/s | 3.6 MB 00:00 2023-08-27T11:09:29.504 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-27T11:09:29.504 INFO:teuthology.orchestra.run.smithi135.stdout:Total 12 MB/s | 206 MB 00:16 2023-08-27T11:09:30.347 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-27T11:09:30.466 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-27T11:09:30.467 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-27T11:09:31.585 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-27T11:09:31.589 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-27T11:09:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-08-27T11:09:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-27T11:09:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-27T11:09:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-27T11:09:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-27T11:09:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-27T11:09:33.855 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-27T11:09:34.080 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 5/134 2023-08-27T11:09:34.507 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 5/134 2023-08-27T11:09:34.705 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:09:34.837 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:09:35.005 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:09:35.105 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:09:35.252 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 8/134 2023-08-27T11:09:35.420 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-08-27T11:09:35.654 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-27T11:09:35.934 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-27T11:09:36.344 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-27T11:09:36.558 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-27T11:09:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 14/134 2023-08-27T11:09:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 14/134 2023-08-27T11:09:36.831 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-27T11:09:36.971 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 15/134 2023-08-27T11:09:37.043 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-27T11:09:37.125 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 15/134 2023-08-27T11:09:37.260 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-27T11:09:37.326 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-08-27T11:09:37.428 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-27T11:09:37.487 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-08-27T11:09:37.576 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-27T11:09:37.642 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-08-27T11:09:37.762 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-27T11:09:37.829 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-27T11:09:37.913 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-27T11:09:38.046 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-27T11:09:38.235 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-27T11:09:38.487 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 5/134 2023-08-27T11:09:38.688 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-27T11:09:38.894 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 5/134 2023-08-27T11:09:38.979 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-27T11:09:39.120 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:09:39.237 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:09:39.436 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:09:39.516 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:09:39.612 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-27T11:09:39.733 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 8/134 2023-08-27T11:09:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-08-27T11:09:39.970 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-08-27T11:09:40.257 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-27T11:09:40.534 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-08-27T11:09:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-08-27T11:09:40.849 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-27T11:09:41.267 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-27T11:09:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-08-27T11:09:41.565 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-27T11:09:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-27T11:09:41.753 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 14/134 2023-08-27T11:09:41.786 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:18.0.0-5707.g1da267a3.el8. 26/134 2023-08-27T11:09:41.858 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 14/134 2023-08-27T11:09:41.933 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el 27/134 2023-08-27T11:09:42.045 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 15/134 2023-08-27T11:09:42.170 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_6 28/134 2023-08-27T11:09:42.192 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 15/134 2023-08-27T11:09:42.362 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-27T11:09:42.415 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-08-27T11:09:42.576 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-08-27T11:09:42.779 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-08-27T11:09:42.952 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-27T11:09:43.097 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-27T11:09:43.261 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-27T11:09:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-27T11:09:44.092 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-27T11:09:44.267 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-27T11:09:44.694 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-27T11:09:44.877 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-08-27T11:09:45.464 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-08-27T11:09:45.619 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-08-27T11:09:45.786 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-08-27T11:09:45.907 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-08-27T11:09:46.086 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-08-27T11:09:46.225 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-08-27T11:09:46.264 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-08-27T11:09:46.562 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-27T11:09:46.602 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-27T11:09:46.775 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:18.0.0-5707.g1da267a3.el8. 26/134 2023-08-27T11:09:46.891 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-27T11:09:46.896 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el 27/134 2023-08-27T11:09:47.060 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 36/134 2023-08-27T11:09:47.139 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_6 28/134 2023-08-27T11:09:47.211 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_6 37/134 2023-08-27T11:09:47.368 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-27T11:09:47.370 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-08-27T11:09:47.518 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-08-27T11:09:47.711 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-08-27T11:09:47.785 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-08-27T11:09:47.997 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-08-27T11:09:48.233 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-08-27T11:09:48.361 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-27T11:09:48.446 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-27T11:09:48.646 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-08-27T11:09:48.791 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-08-27T11:09:48.930 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-08-27T11:09:49.123 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-08-27T11:09:49.130 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-27T11:09:49.289 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-08-27T11:09:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-08-27T11:09:49.597 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-08-27T11:09:49.797 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-08-27T11:09:50.173 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-08-27T11:09:50.175 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-08-27T11:09:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-08-27T11:09:50.526 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-08-27T11:09:50.656 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-08-27T11:09:50.804 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-08-27T11:09:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-08-27T11:09:51.037 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-08-27T11:09:51.110 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-08-27T11:09:51.272 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-27T11:09:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-08-27T11:09:51.503 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-27T11:09:51.586 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-08-27T11:09:51.657 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 36/134 2023-08-27T11:09:51.782 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-08-27T11:09:51.832 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_6 37/134 2023-08-27T11:09:51.976 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-08-27T11:09:52.025 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-08-27T11:09:52.125 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-08-27T11:09:52.177 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-08-27T11:09:52.273 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-08-27T11:09:52.393 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-08-27T11:09:52.453 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-08-27T11:09:52.565 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-08-27T11:09:52.731 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-08-27T11:09:52.843 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-08-27T11:09:52.996 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-08-27T11:09:53.053 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-08-27T11:09:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-27T11:09:53.254 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-08-27T11:09:53.265 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-27T11:09:53.393 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-27T11:09:53.472 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-08-27T11:09:53.598 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-27T11:09:53.647 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-08-27T11:09:53.841 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-27T11:09:53.862 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-08-27T11:09:54.041 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-08-27T11:09:54.050 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-27T11:09:54.173 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-27T11:09:54.228 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-08-27T11:09:54.321 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-27T11:09:54.412 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-08-27T11:09:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-27T11:09:54.614 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-08-27T11:09:54.715 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-27T11:09:54.854 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-08-27T11:09:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-27T11:09:55.046 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-27T11:09:55.174 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-27T11:09:55.233 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-08-27T11:09:55.428 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-27T11:09:55.536 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-08-27T11:09:55.607 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-27T11:09:55.783 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-27T11:09:55.804 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-08-27T11:09:56.016 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-27T11:09:56.072 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-08-27T11:09:56.202 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-27T11:09:56.331 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-08-27T11:09:56.386 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-27T11:09:56.590 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-27T11:09:56.694 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-08-27T11:09:56.942 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-08-27T11:09:57.163 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-08-27T11:09:57.397 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-08-27T11:09:57.583 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-08-27T11:09:57.737 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-27T11:09:57.787 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-08-27T11:09:57.966 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-27T11:09:58.121 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-08-27T11:09:58.211 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-27T11:09:58.405 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-08-27T11:09:58.496 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-27T11:09:58.680 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-08-27T11:09:58.732 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-27T11:09:58.878 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-27T11:09:58.968 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-08-27T11:09:59.158 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-27T11:09:59.346 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-27T11:09:59.645 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-27T11:09:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-27T11:09:59.899 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-27T11:09:59.917 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 92/134 2023-08-27T11:09:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 92/134 2023-08-27T11:10:00.079 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-27T11:10:00.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 93/134 2023-08-27T11:10:00.228 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-27T11:10:00.444 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-27T11:10:00.671 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-27T11:10:00.862 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-27T11:10:01.045 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-27T11:10:01.223 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-27T11:10:01.463 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-27T11:10:01.679 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-27T11:10:01.898 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-27T11:10:02.231 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-27T11:10:02.499 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-27T11:10:02.716 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-27T11:10:02.928 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-27T11:10:03.980 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:10:04.052 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-27T11:10:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:10:04.345 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-27T11:10:04.607 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-27T11:10:04.815 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:10:04.916 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-27T11:10:05.067 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 95/134 2023-08-27T11:10:05.146 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 95/134 2023-08-27T11:10:05.220 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-27T11:10:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 96/134 2023-08-27T11:10:05.458 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-27T11:10:06.287 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-27T11:10:06.441 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 92/134 2023-08-27T11:10:06.543 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 92/134 2023-08-27T11:10:06.716 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 93/134 2023-08-27T11:10:10.407 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:10:10.641 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:10:11.241 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 94/134 2023-08-27T11:10:11.566 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 95/134 2023-08-27T11:10:11.639 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 95/134 2023-08-27T11:10:11.807 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 96/134 2023-08-27T11:10:24.741 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 96/134 2023-08-27T11:10:24.741 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2023-08-27T11:10:24.742 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2023-08-27T11:10:24.742 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /dev 2023-08-27T11:10:24.742 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /run 2023-08-27T11:10:24.742 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2023-08-27T11:10:24.742 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2023-08-27T11:10:24.743 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2023-08-27T11:10:24.743 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2023-08-27T11:10:24.743 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /dev 2023-08-27T11:10:24.743 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:10:25.064 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 97/134 2023-08-27T11:10:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 97/134 2023-08-27T11:10:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 98/134 2023-08-27T11:10:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 98/134 2023-08-27T11:10:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 99/134 2023-08-27T11:10:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 99/134 2023-08-27T11:10:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 100/134 2023-08-27T11:10:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 100/134 2023-08-27T11:10:29.703 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 101/134 2023-08-27T11:10:29.858 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 101/134 2023-08-27T11:10:29.986 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.e 102/134 2023-08-27T11:10:30.182 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3. 103/134 2023-08-27T11:10:30.346 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 104/134 2023-08-27T11:10:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 104/134 2023-08-27T11:10:30.871 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 105/134 2023-08-27T11:10:30.876 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 96/134 2023-08-27T11:10:30.876 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2023-08-27T11:10:30.876 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2023-08-27T11:10:30.877 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /dev 2023-08-27T11:10:30.877 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /run 2023-08-27T11:10:30.877 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2023-08-27T11:10:30.877 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2023-08-27T11:10:30.878 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2023-08-27T11:10:30.878 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2023-08-27T11:10:30.878 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /dev 2023-08-27T11:10:30.878 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:10:31.262 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 97/134 2023-08-27T11:10:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 105/134 2023-08-27T11:10:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 106/134 2023-08-27T11:10:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 106/134 2023-08-27T11:10:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el 107/134 2023-08-27T11:10:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 108/134 2023-08-27T11:10:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 108/134 2023-08-27T11:10:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 109/134 2023-08-27T11:10:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 109/134 2023-08-27T11:10:32.891 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 97/134 2023-08-27T11:10:33.080 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 98/134 2023-08-27T11:10:33.298 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 98/134 2023-08-27T11:10:33.479 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 99/134 2023-08-27T11:10:33.933 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 99/134 2023-08-27T11:10:34.145 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 100/134 2023-08-27T11:10:34.162 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 100/134 2023-08-27T11:10:34.246 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 110/134 2023-08-27T11:10:34.448 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 111/134 2023-08-27T11:10:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 111/134 2023-08-27T11:10:34.654 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5707.g1da26 112/134 2023-08-27T11:10:34.698 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5707.g1da26 112/134 2023-08-27T11:10:34.830 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 113/134 2023-08-27T11:10:35.203 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 113/134 2023-08-27T11:10:35.346 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 114/134 2023-08-27T11:10:35.880 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 101/134 2023-08-27T11:10:36.124 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 101/134 2023-08-27T11:10:36.242 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.e 102/134 2023-08-27T11:10:36.444 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3. 103/134 2023-08-27T11:10:36.620 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 104/134 2023-08-27T11:10:36.840 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 104/134 2023-08-27T11:10:37.146 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 105/134 2023-08-27T11:10:38.108 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 105/134 2023-08-27T11:10:38.294 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 106/134 2023-08-27T11:10:38.386 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 106/134 2023-08-27T11:10:38.776 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el 107/134 2023-08-27T11:10:39.004 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 108/134 2023-08-27T11:10:39.040 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 108/134 2023-08-27T11:10:39.219 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 109/134 2023-08-27T11:10:39.231 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 109/134 2023-08-27T11:10:40.197 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 114/134 2023-08-27T11:10:40.564 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 110/134 2023-08-27T11:10:40.821 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 111/134 2023-08-27T11:10:40.855 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 111/134 2023-08-27T11:10:41.019 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5707.g1da26 112/134 2023-08-27T11:10:41.062 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5707.g1da26 112/134 2023-08-27T11:10:41.112 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 115/134 2023-08-27T11:10:41.211 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 113/134 2023-08-27T11:10:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-08-27T11:10:41.563 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 113/134 2023-08-27T11:10:41.618 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 117/134 2023-08-27T11:10:41.770 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 114/134 2023-08-27T11:10:41.891 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-08-27T11:10:42.036 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-08-27T11:10:42.242 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-08-27T11:10:42.474 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-08-27T11:10:42.657 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_ 121/134 2023-08-27T11:10:42.788 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-27T11:10:45.101 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:10:45.336 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:10:45.356 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:10:45.503 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 124/134 2023-08-27T11:10:45.779 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 125/134 2023-08-27T11:10:46.072 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 126/134 2023-08-27T11:10:46.344 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 114/134 2023-08-27T11:10:46.410 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-27T11:10:46.585 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-08-27T11:10:46.678 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-08-27T11:10:46.869 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-08-27T11:10:46.882 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-08-27T11:10:47.019 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-27T11:10:47.028 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-27T11:10:47.163 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-08-27T11:10:47.163 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:10:47.264 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 115/134 2023-08-27T11:10:47.348 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:10:47.348 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-27T11:10:47.576 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-27T11:10:47.618 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-08-27T11:10:47.752 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:10:47.841 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 117/134 2023-08-27T11:10:48.126 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-08-27T11:10:48.270 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:10:48.270 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 1/134 2023-08-27T11:10:48.271 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 2/134 2023-08-27T11:10:48.271 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 3/134 2023-08-27T11:10:48.271 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 4/134 2023-08-27T11:10:48.271 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5707.g1da26 5/134 2023-08-27T11:10:48.271 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:10:48.272 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:10:48.272 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 8/134 2023-08-27T11:10:48.272 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 9/134 2023-08-27T11:10:48.272 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 10/134 2023-08-27T11:10:48.272 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 11/134 2023-08-27T11:10:48.272 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 12/134 2023-08-27T11:10:48.273 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 13/134 2023-08-27T11:10:48.273 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_ 14/134 2023-08-27T11:10:48.273 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 15/134 2023-08-27T11:10:48.273 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 16/134 2023-08-27T11:10:48.273 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_6 17/134 2023-08-27T11:10:48.274 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 18/134 2023-08-27T11:10:48.274 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 19/134 2023-08-27T11:10:48.274 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el 20/134 2023-08-27T11:10:48.274 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:18.0.0-5707.g1da267a3.el8. 21/134 2023-08-27T11:10:48.274 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_6 22/134 2023-08-27T11:10:48.275 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 23/134 2023-08-27T11:10:48.275 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 24/134 2023-08-27T11:10:48.275 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 25/134 2023-08-27T11:10:48.275 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 26/134 2023-08-27T11:10:48.275 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 27/134 2023-08-27T11:10:48.276 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 28/134 2023-08-27T11:10:48.276 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3. 29/134 2023-08-27T11:10:48.276 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 30/134 2023-08-27T11:10:48.277 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-08-27T11:10:48.285 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 31/134 2023-08-27T11:10:48.286 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 32/134 2023-08-27T11:10:48.286 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el 33/134 2023-08-27T11:10:48.286 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 34/134 2023-08-27T11:10:48.286 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.e 35/134 2023-08-27T11:10:48.286 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 36/134 2023-08-27T11:10:48.286 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 37/134 2023-08-27T11:10:48.288 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 38/134 2023-08-27T11:10:48.288 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-27T11:10:48.289 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-27T11:10:48.289 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-27T11:10:48.289 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-27T11:10:48.289 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-27T11:10:48.289 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-27T11:10:48.290 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-27T11:10:48.290 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-27T11:10:48.290 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-27T11:10:48.290 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-27T11:10:48.290 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-27T11:10:48.291 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-27T11:10:48.291 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-27T11:10:48.291 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-27T11:10:48.291 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-27T11:10:48.291 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-27T11:10:48.291 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-27T11:10:48.292 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-27T11:10:48.292 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-27T11:10:48.292 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-27T11:10:48.292 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-27T11:10:48.293 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-27T11:10:48.293 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-27T11:10:48.293 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-27T11:10:48.293 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-27T11:10:48.293 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-27T11:10:48.294 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-27T11:10:48.294 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-27T11:10:48.294 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-27T11:10:48.294 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-27T11:10:48.294 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-27T11:10:48.295 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-27T11:10:48.295 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-27T11:10:48.295 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-08-27T11:10:48.295 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-08-27T11:10:48.295 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-08-27T11:10:48.296 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-08-27T11:10:48.296 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-08-27T11:10:48.296 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-08-27T11:10:48.296 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-08-27T11:10:48.296 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-08-27T11:10:48.297 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-08-27T11:10:48.297 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-08-27T11:10:48.297 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-08-27T11:10:48.297 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-08-27T11:10:48.297 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-08-27T11:10:48.298 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-08-27T11:10:48.298 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-08-27T11:10:48.298 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-08-27T11:10:48.298 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-08-27T11:10:48.298 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-08-27T11:10:48.298 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-08-27T11:10:48.299 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-08-27T11:10:48.299 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-08-27T11:10:48.299 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-08-27T11:10:48.299 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-08-27T11:10:48.299 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-08-27T11:10:48.300 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-08-27T11:10:48.300 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-08-27T11:10:48.300 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-08-27T11:10:48.300 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-08-27T11:10:48.300 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-08-27T11:10:48.301 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-08-27T11:10:48.301 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-08-27T11:10:48.301 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-08-27T11:10:48.301 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-08-27T11:10:48.301 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-08-27T11:10:48.302 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-08-27T11:10:48.302 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-08-27T11:10:48.302 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-08-27T11:10:48.302 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-08-27T11:10:48.302 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-08-27T11:10:48.302 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-08-27T11:10:48.303 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-08-27T11:10:48.303 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-08-27T11:10:48.303 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-08-27T11:10:48.303 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-08-27T11:10:48.303 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-08-27T11:10:48.304 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-08-27T11:10:48.304 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-08-27T11:10:48.304 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-08-27T11:10:48.304 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-08-27T11:10:48.304 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-08-27T11:10:48.305 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-08-27T11:10:48.305 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-08-27T11:10:48.305 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-08-27T11:10:48.305 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-08-27T11:10:48.305 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-08-27T11:10:48.306 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-08-27T11:10:48.306 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-08-27T11:10:48.306 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 129/134 2023-08-27T11:10:48.306 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-27T11:10:48.306 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 131/134 2023-08-27T11:10:48.307 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:10:48.307 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-27T11:10:48.468 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-08-27T11:10:48.739 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-08-27T11:10:48.897 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_ 121/134 2023-08-27T11:10:49.077 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-27T11:10:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:10:50.978 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:50 ago on Sun 27 Aug 2023 11:09:00 AM UTC. 2023-08-27T11:10:51.302 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:10:51.564 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:10:51.581 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-27T11:10:51.768 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 124/134 2023-08-27T11:10:52.085 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 125/134 2023-08-27T11:10:52.413 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 126/134 2023-08-27T11:10:52.883 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-27T11:10:53.050 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-08-27T11:10:53.142 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-08-27T11:10:53.350 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-08-27T11:10:53.363 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-08-27T11:10:53.551 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-27T11:10:53.559 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-27T11:10:53.571 INFO:teuthology.orchestra.run.smithi045.stdout:Installed products updated. 2023-08-27T11:10:53.736 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-08-27T11:10:53.736 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:10:53.810 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:10:53.811 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2023-08-27T11:10:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-27T11:10:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.811 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.811 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-08-27T11:10:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-27T11:10:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.1-7.el8.noarch 2023-08-27T11:10:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.0.4-10.el8.x86_64 2023-08-27T11:10:53.812 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.813 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.814 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.815 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.816 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:10:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-27T11:10:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.1-9.el8.x86_64 2023-08-27T11:10:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-27T11:10:53.817 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-27T11:10:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: jq-1.6-3.el8.x86_64 2023-08-27T11:10:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-27T11:10:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-27T11:10:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-27T11:10:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.818 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-27T11:10:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-27T11:10:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-27T11:10:53.819 INFO:teuthology.orchestra.run.smithi045.stdout: libpq-13.5-1.el8.x86_64 2023-08-27T11:10:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-27T11:10:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-27T11:10:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.820 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-27T11:10:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-27T11:10:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-27T11:10:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-27T11:10:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-27T11:10:53.821 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-27T11:10:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-27T11:10:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-27T11:10:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-27T11:10:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-27T11:10:53.822 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-27T11:10:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-27T11:10:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-27T11:10:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-27T11:10:53.823 INFO:teuthology.orchestra.run.smithi045.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-27T11:10:53.824 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:10:53.824 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:10:53.824 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:10:53.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-27T11:10:53.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-27T11:10:53.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-27T11:10:53.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-27T11:10:53.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-27T11:10:53.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-27T11:10:53.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-27T11:10:53.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-27T11:10:53.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-27T11:10:53.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-27T11:10:53.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-27T11:10:53.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-27T11:10:53.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-27T11:10:53.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-27T11:10:53.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-27T11:10:53.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-27T11:10:53.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-27T11:10:53.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-27T11:10:53.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-27T11:10:53.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-27T11:10:53.829 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-27T11:10:53.829 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-27T11:10:53.829 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-27T11:10:53.829 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-27T11:10:53.829 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-27T11:10:53.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-27T11:10:53.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-27T11:10:53.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-2.6-1.el8.noarch 2023-08-27T11:10:53.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-27T11:10:53.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-27T11:10:53.830 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-27T11:10:53.831 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-27T11:10:53.831 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-27T11:10:53.831 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-27T11:10:53.831 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.832 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.832 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-27T11:10:53.832 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-27T11:10:53.832 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.832 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-27T11:10:53.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-27T11:10:53.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-27T11:10:53.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-27T11:10:53.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-27T11:10:53.833 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-27T11:10:53.834 INFO:teuthology.orchestra.run.smithi045.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-27T11:10:53.834 INFO:teuthology.orchestra.run.smithi045.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-27T11:10:53.834 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-27T11:10:53.834 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-27T11:10:53.834 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-27T11:10:53.834 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-27T11:10:53.835 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-27T11:10:53.835 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.835 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.835 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:10:53.835 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-27T11:10:53.836 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-27T11:10:53.836 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-27T11:10:53.836 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-27T11:10:53.836 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-27T11:10:53.837 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-27T11:10:53.837 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:10:53.837 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-08-27T11:10:53.937 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:10:53.938 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-27T11:10:54.015 DEBUG:teuthology.parallel:result is None 2023-08-27T11:10:54.194 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-27T11:10:54.391 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:10:54.900 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:10:54.901 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 1/134 2023-08-27T11:10:54.901 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 2/134 2023-08-27T11:10:54.901 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 3/134 2023-08-27T11:10:54.901 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 4/134 2023-08-27T11:10:54.902 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5707.g1da26 5/134 2023-08-27T11:10:54.902 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 6/134 2023-08-27T11:10:54.902 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 7/134 2023-08-27T11:10:54.902 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 8/134 2023-08-27T11:10:54.902 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 9/134 2023-08-27T11:10:54.903 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 10/134 2023-08-27T11:10:54.903 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 11/134 2023-08-27T11:10:54.903 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 12/134 2023-08-27T11:10:54.903 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 13/134 2023-08-27T11:10:54.903 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_ 14/134 2023-08-27T11:10:54.904 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 15/134 2023-08-27T11:10:54.904 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 16/134 2023-08-27T11:10:54.904 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_6 17/134 2023-08-27T11:10:54.904 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86 18/134 2023-08-27T11:10:54.904 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 19/134 2023-08-27T11:10:54.905 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el 20/134 2023-08-27T11:10:54.905 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:18.0.0-5707.g1da267a3.el8. 21/134 2023-08-27T11:10:54.905 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_6 22/134 2023-08-27T11:10:54.905 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 23/134 2023-08-27T11:10:54.905 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 24/134 2023-08-27T11:10:54.906 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 25/134 2023-08-27T11:10:54.906 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 26/134 2023-08-27T11:10:54.906 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 27/134 2023-08-27T11:10:54.906 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 28/134 2023-08-27T11:10:54.906 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3. 29/134 2023-08-27T11:10:54.906 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noa 30/134 2023-08-27T11:10:54.915 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.n 31/134 2023-08-27T11:10:54.915 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da 32/134 2023-08-27T11:10:54.916 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el 33/134 2023-08-27T11:10:54.916 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 34/134 2023-08-27T11:10:54.916 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.e 35/134 2023-08-27T11:10:54.916 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 36/134 2023-08-27T11:10:54.916 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 37/134 2023-08-27T11:10:54.917 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 38/134 2023-08-27T11:10:54.917 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-27T11:10:54.917 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-27T11:10:54.917 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-27T11:10:54.918 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-27T11:10:54.918 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-27T11:10:54.918 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-27T11:10:54.918 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-27T11:10:54.918 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-27T11:10:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-27T11:10:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-27T11:10:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-27T11:10:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-27T11:10:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-27T11:10:54.919 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-27T11:10:54.920 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-27T11:10:54.920 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-27T11:10:54.920 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-27T11:10:54.920 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-27T11:10:54.920 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-27T11:10:54.921 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-27T11:10:54.921 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-27T11:10:54.921 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-27T11:10:54.921 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-27T11:10:54.921 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-27T11:10:54.922 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-27T11:10:54.922 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-27T11:10:54.922 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-27T11:10:54.922 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-27T11:10:54.922 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-27T11:10:54.922 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-27T11:10:54.923 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-27T11:10:54.923 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-27T11:10:54.923 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-27T11:10:54.923 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-08-27T11:10:54.923 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-08-27T11:10:54.924 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-08-27T11:10:54.924 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-08-27T11:10:54.924 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-08-27T11:10:54.924 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-08-27T11:10:54.924 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-08-27T11:10:54.925 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-08-27T11:10:54.925 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-08-27T11:10:54.925 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-08-27T11:10:54.925 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-08-27T11:10:54.925 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-08-27T11:10:54.926 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-08-27T11:10:54.926 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-08-27T11:10:54.926 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-08-27T11:10:54.926 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-08-27T11:10:54.926 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-08-27T11:10:54.927 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-08-27T11:10:54.927 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-08-27T11:10:54.927 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-08-27T11:10:54.927 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-08-27T11:10:54.927 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-08-27T11:10:54.927 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-08-27T11:10:54.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-08-27T11:10:54.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-08-27T11:10:54.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-08-27T11:10:54.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-08-27T11:10:54.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-08-27T11:10:54.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-08-27T11:10:54.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-08-27T11:10:54.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-08-27T11:10:54.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-08-27T11:10:54.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-08-27T11:10:54.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-08-27T11:10:54.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-08-27T11:10:54.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-08-27T11:10:54.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-08-27T11:10:54.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-08-27T11:10:54.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-08-27T11:10:54.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-08-27T11:10:54.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-08-27T11:10:54.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-08-27T11:10:54.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-08-27T11:10:54.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-08-27T11:10:54.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-08-27T11:10:54.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-08-27T11:10:54.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-08-27T11:10:54.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-08-27T11:10:54.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-08-27T11:10:54.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-08-27T11:10:54.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-08-27T11:10:54.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-08-27T11:10:54.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-08-27T11:10:54.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-08-27T11:10:54.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-08-27T11:10:54.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-08-27T11:10:54.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-08-27T11:10:54.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 129/134 2023-08-27T11:10:54.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-27T11:10:54.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 131/134 2023-08-27T11:10:54.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-27T11:10:54.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-27T11:10:57.865 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-27T11:10:57.865 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:57 ago on Sun 27 Aug 2023 11:09:00 AM UTC. 2023-08-27T11:11:00.419 INFO:teuthology.orchestra.run.smithi135.stdout:Installed products updated. 2023-08-27T11:11:00.653 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:00.653 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2023-08-27T11:11:00.653 INFO:teuthology.orchestra.run.smithi135.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-27T11:11:00.654 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.654 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.654 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-27T11:11:00.654 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-27T11:11:00.654 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.1-7.el8.noarch 2023-08-27T11:11:00.655 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.0.4-10.el8.x86_64 2023-08-27T11:11:00.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.656 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.656 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.656 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.656 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.656 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.656 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.657 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.657 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.657 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.657 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.658 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.658 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.658 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.658 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.658 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.658 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.659 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.659 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.659 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:18.0.0-5707.g1da267a3.el8.noarch 2023-08-27T11:11:00.659 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-27T11:11:00.659 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.1-9.el8.x86_64 2023-08-27T11:11:00.660 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-27T11:11:00.660 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-27T11:11:00.660 INFO:teuthology.orchestra.run.smithi135.stdout: jq-1.6-3.el8.x86_64 2023-08-27T11:11:00.660 INFO:teuthology.orchestra.run.smithi135.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-27T11:11:00.660 INFO:teuthology.orchestra.run.smithi135.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-27T11:11:00.661 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-27T11:11:00.661 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.661 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.661 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.661 INFO:teuthology.orchestra.run.smithi135.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-27T11:11:00.662 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-27T11:11:00.662 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-27T11:11:00.662 INFO:teuthology.orchestra.run.smithi135.stdout: libpq-13.5-1.el8.x86_64 2023-08-27T11:11:00.662 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-27T11:11:00.662 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-27T11:11:00.663 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.663 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.663 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-27T11:11:00.663 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.663 INFO:teuthology.orchestra.run.smithi135.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-27T11:11:00.664 INFO:teuthology.orchestra.run.smithi135.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-27T11:11:00.664 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-27T11:11:00.664 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-27T11:11:00.664 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-27T11:11:00.664 INFO:teuthology.orchestra.run.smithi135.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-27T11:11:00.664 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-27T11:11:00.665 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-27T11:11:00.665 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-27T11:11:00.665 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-27T11:11:00.665 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-27T11:11:00.665 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-27T11:11:00.666 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-27T11:11:00.666 INFO:teuthology.orchestra.run.smithi135.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-27T11:11:00.666 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:11:00.666 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:11:00.666 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-27T11:11:00.667 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-27T11:11:00.667 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-27T11:11:00.667 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-27T11:11:00.667 INFO:teuthology.orchestra.run.smithi135.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-27T11:11:00.667 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-27T11:11:00.668 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.668 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.668 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.668 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-27T11:11:00.668 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-27T11:11:00.669 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-27T11:11:00.669 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-27T11:11:00.669 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-27T11:11:00.669 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-27T11:11:00.669 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-27T11:11:00.670 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-27T11:11:00.670 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-27T11:11:00.670 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-27T11:11:00.670 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-27T11:11:00.670 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-27T11:11:00.670 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-27T11:11:00.671 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-27T11:11:00.671 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-27T11:11:00.671 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-27T11:11:00.671 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-27T11:11:00.671 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-27T11:11:00.672 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-27T11:11:00.672 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-27T11:11:00.672 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-27T11:11:00.672 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-27T11:11:00.672 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-2.6-1.el8.noarch 2023-08-27T11:11:00.673 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-27T11:11:00.673 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-27T11:11:00.673 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-27T11:11:00.673 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-27T11:11:00.673 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-27T11:11:00.674 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-27T11:11:00.674 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.674 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.674 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-27T11:11:00.674 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-27T11:11:00.675 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.675 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-27T11:11:00.675 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-27T11:11:00.675 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-27T11:11:00.675 INFO:teuthology.orchestra.run.smithi135.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-27T11:11:00.675 INFO:teuthology.orchestra.run.smithi135.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-27T11:11:00.676 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-27T11:11:00.676 INFO:teuthology.orchestra.run.smithi135.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-27T11:11:00.676 INFO:teuthology.orchestra.run.smithi135.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-27T11:11:00.676 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-27T11:11:00.676 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-27T11:11:00.677 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-27T11:11:00.677 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-27T11:11:00.677 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-27T11:11:00.677 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.678 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.678 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:18.0.0-5707.g1da267a3.el8.x86_64 2023-08-27T11:11:00.678 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-27T11:11:00.678 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-27T11:11:00.678 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-27T11:11:00.678 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-27T11:11:00.679 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-27T11:11:00.679 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-27T11:11:00.680 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:00.680 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-27T11:11:00.903 DEBUG:teuthology.parallel:result is None 2023-08-27T11:11:00.904 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T11:11:01.155 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-27T11:11:01.184 INFO:teuthology.orchestra.run.smithi045.stdout:18.0.0-5707.g1da267a3.el8 2023-08-27T11:11:01.185 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5707.g1da267a3.el8 2023-08-27T11:11:01.186 INFO:teuthology.task.install:The correct ceph version 18.0.0-5707.g1da267a3 is installed. 2023-08-27T11:11:01.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1da267a3afa9ab8f9b1d164359066f8bd682ffd8 2023-08-27T11:11:01.475 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-27T11:11:01.501 INFO:teuthology.orchestra.run.smithi135.stdout:18.0.0-5707.g1da267a3.el8 2023-08-27T11:11:01.502 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5707.g1da267a3.el8 2023-08-27T11:11:01.502 INFO:teuthology.task.install:The correct ceph version 18.0.0-5707.g1da267a3 is installed. 2023-08-27T11:11:01.502 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-27T11:11:01.520 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:01.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-27T11:11:01.588 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:01.588 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-27T11:11:01.630 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-27T11:11:01.631 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:01.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2023-08-27T11:11:01.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-27T11:11:01.732 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:01.733 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2023-08-27T11:11:01.761 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-27T11:11:01.828 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-27T11:11:01.828 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:01.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-27T11:11:01.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-27T11:11:01.930 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:01.930 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-27T11:11:01.957 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-27T11:11:02.026 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-27T11:11:02.027 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:02.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2023-08-27T11:11:02.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-27T11:11:02.126 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:02.127 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2023-08-27T11:11:02.153 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-27T11:11:02.219 INFO:teuthology.run_tasks:Running task ceph... 2023-08-27T11:11:02.641 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-08-27T11:11:02.643 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /var/log/ceph 2023-08-27T11:11:02.645 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /var/log/ceph 2023-08-27T11:11:02.673 INFO:tasks.ceph:Disabling ceph logrotate... 2023-08-27T11:11:02.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-27T11:11:02.717 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-27T11:11:02.744 INFO:tasks.ceph:Creating extra log directories... 2023-08-27T11:11:02.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-27T11:11:02.785 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-27T11:11:02.818 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-08-27T11:11:02.826 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-08-27T11:11:02.840 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-08-27T11:11:02.841 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:02.841 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-27T11:11:02.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-27T11:11:03.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-27T11:11:03.036 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:03.036 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-27T11:11:03.053 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-27T11:11:03.186 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-27T11:11:03.218 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-08-27T11:11:03.218 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-08-27T11:11:03.218 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381530', 'branch': 'wip-rishabh-xiubo-wip-57154', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': None, 'first_in_suite': False, 'job_id': '7381530', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-xiubo-wip-57154'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-xiubo-wip-57154', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-xiubo-wip-57154', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1da267a3afa9ab8f9b1d164359066f8bd682ffd8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi045.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi135.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1da267a3afa9ab8f9b1d164359066f8bd682ffd8'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '449a1bc2027504e7b3c3d7b30fa4178906581da7', 'timestamp': '2023-08-26_02:16:12', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1553'} 2023-08-27T11:11:03.219 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-27T11:11:03.222 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-27T11:11:03.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/run/ceph 2023-08-27T11:11:03.280 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/run/ceph 2023-08-27T11:11:03.335 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:03.336 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2023-08-27T11:11:03.364 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-27T11:11:03.364 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2023-08-27T11:11:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-27T11:11:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:11:03.423 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 387530 Links: 1 2023-08-27T11:11:03.423 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:11:03.423 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:11:03.423 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-08-27 11:10:47.961590566 +0000 2023-08-27T11:11:03.423 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-08-27 11:08:14.879385118 +0000 2023-08-27T11:11:03.423 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-08-27 11:08:14.879385118 +0000 2023-08-27T11:11:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-08-27T11:11:03.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-27T11:11:03.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-08-27T11:11:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-08-27T11:11:03.490 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000124236 s, 4.1 MB/s 2023-08-27T11:11:03.491 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-27T11:11:03.549 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2023-08-27T11:11:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-27T11:11:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:11:03.607 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 385985 Links: 1 2023-08-27T11:11:03.607 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:11:03.607 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:11:03.608 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-08-27 11:10:47.961590566 +0000 2023-08-27T11:11:03.608 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-08-27 11:08:15.387377343 +0000 2023-08-27T11:11:03.608 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-08-27 11:08:15.387377343 +0000 2023-08-27T11:11:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-08-27T11:11:03.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-27T11:11:03.673 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-08-27T11:11:03.673 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-08-27T11:11:03.673 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.00010708 s, 4.8 MB/s 2023-08-27T11:11:03.675 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-27T11:11:03.732 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2023-08-27T11:11:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-27T11:11:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:11:03.790 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 386905 Links: 1 2023-08-27T11:11:03.790 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:11:03.791 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:11:03.791 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-08-27 11:10:47.961590566 +0000 2023-08-27T11:11:03.791 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-08-27 11:08:15.896369553 +0000 2023-08-27T11:11:03.791 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-08-27 11:08:15.896369553 +0000 2023-08-27T11:11:03.791 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-08-27T11:11:03.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-27T11:11:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-08-27T11:11:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-08-27T11:11:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000117888 s, 4.3 MB/s 2023-08-27T11:11:03.858 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-27T11:11:03.916 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2023-08-27T11:11:03.974 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-27T11:11:03.974 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:11:03.974 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 388683 Links: 1 2023-08-27T11:11:03.975 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:11:03.975 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:11:03.975 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-08-27 11:10:47.961590566 +0000 2023-08-27T11:11:03.975 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-08-27 11:08:16.408361717 +0000 2023-08-27T11:11:03.976 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-08-27 11:08:16.408361717 +0000 2023-08-27T11:11:03.976 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-08-27T11:11:03.977 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-27T11:11:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-08-27T11:11:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-08-27T11:11:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000111909 s, 4.6 MB/s 2023-08-27T11:11:04.040 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-27T11:11:04.098 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:04.099 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:04.099 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2023-08-27T11:11:04.120 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-27T11:11:04.120 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2023-08-27T11:11:04.180 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-27T11:11:04.180 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:11:04.180 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 390309 Links: 1 2023-08-27T11:11:04.181 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:11:04.181 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:11:04.181 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-08-27 11:10:54.602974438 +0000 2023-08-27T11:11:04.181 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-08-27 11:08:04.287025062 +0000 2023-08-27T11:11:04.181 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-08-27 11:08:04.287025062 +0000 2023-08-27T11:11:04.182 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-08-27T11:11:04.182 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-27T11:11:04.248 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-08-27T11:11:04.249 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-08-27T11:11:04.249 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000267775 s, 1.9 MB/s 2023-08-27T11:11:04.250 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-27T11:11:04.309 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2023-08-27T11:11:04.369 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-27T11:11:04.369 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:11:04.369 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 388360 Links: 1 2023-08-27T11:11:04.370 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:11:04.370 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:11:04.370 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-08-27 11:10:54.602974438 +0000 2023-08-27T11:11:04.371 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-08-27 11:08:04.797014336 +0000 2023-08-27T11:11:04.371 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-08-27 11:08:04.797014336 +0000 2023-08-27T11:11:04.371 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-08-27T11:11:04.372 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-27T11:11:04.436 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-08-27T11:11:04.437 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-08-27T11:11:04.437 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000255451 s, 2.0 MB/s 2023-08-27T11:11:04.438 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-27T11:11:04.498 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2023-08-27T11:11:04.556 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-27T11:11:04.557 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:11:04.557 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 389479 Links: 1 2023-08-27T11:11:04.557 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:11:04.557 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:11:04.558 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-08-27 11:10:54.602974438 +0000 2023-08-27T11:11:04.558 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-08-27 11:08:05.296003842 +0000 2023-08-27T11:11:04.558 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-08-27 11:08:05.296003842 +0000 2023-08-27T11:11:04.558 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-08-27T11:11:04.559 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-27T11:11:04.624 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-08-27T11:11:04.625 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-08-27T11:11:04.625 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000245232 s, 2.1 MB/s 2023-08-27T11:11:04.626 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-27T11:11:04.684 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2023-08-27T11:11:04.742 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-27T11:11:04.743 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-27T11:11:04.743 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 6h/6d Inode: 394261 Links: 1 2023-08-27T11:11:04.743 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-27T11:11:04.743 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-08-27T11:11:04.744 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-08-27 11:10:54.602974438 +0000 2023-08-27T11:11:04.744 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-08-27 11:08:05.820992800 +0000 2023-08-27T11:11:04.744 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-08-27 11:08:05.820992800 +0000 2023-08-27T11:11:04.744 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: - 2023-08-27T11:11:04.745 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-27T11:11:04.809 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-08-27T11:11:04.810 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-08-27T11:11:04.810 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000202537 s, 2.5 MB/s 2023-08-27T11:11:04.811 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-27T11:11:04.870 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:04.870 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-27T11:11:04.870 INFO:tasks.ceph:Generating config... 2023-08-27T11:11:04.890 INFO:tasks.ceph:[client] client mount timeout = 600 2023-08-27T11:11:04.890 INFO:tasks.ceph:[client] debug client = 20 2023-08-27T11:11:04.890 INFO:tasks.ceph:[client] debug ms = 1 2023-08-27T11:11:04.891 INFO:tasks.ceph:[client] pid file = 2023-08-27T11:11:04.891 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-08-27T11:11:04.891 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-08-27T11:11:04.892 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-08-27T11:11:04.892 INFO:tasks.ceph:[mds] debug mds = 20 2023-08-27T11:11:04.892 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-08-27T11:11:04.892 INFO:tasks.ceph:[mds] debug ms = 1 2023-08-27T11:11:04.893 INFO:tasks.ceph:[mds] mds debug frag = True 2023-08-27T11:11:04.893 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-08-27T11:11:04.893 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-08-27T11:11:04.894 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-08-27T11:11:04.894 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-08-27T11:11:04.894 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-08-27T11:11:04.894 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-08-27T11:11:04.895 INFO:tasks.ceph:[mgr] debug client = 20 2023-08-27T11:11:04.895 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-08-27T11:11:04.895 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-08-27T11:11:04.895 INFO:tasks.ceph:[mgr] debug ms = 1 2023-08-27T11:11:04.896 INFO:tasks.ceph:[mon] debug mon = 20 2023-08-27T11:11:04.896 INFO:tasks.ceph:[mon] debug ms = 1 2023-08-27T11:11:04.896 INFO:tasks.ceph:[mon] debug paxos = 20 2023-08-27T11:11:04.896 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-08-27T11:11:04.897 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-08-27T11:11:04.897 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-08-27T11:11:04.897 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-08-27T11:11:04.898 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-08-27T11:11:04.898 INFO:tasks.ceph:[osd] debug ms = 1 2023-08-27T11:11:04.899 INFO:tasks.ceph:[osd] debug osd = 20 2023-08-27T11:11:04.899 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-08-27T11:11:04.899 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-08-27T11:11:04.900 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-08-27T11:11:04.900 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-08-27T11:11:04.900 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-08-27T11:11:04.900 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-08-27T11:11:04.901 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-08-27T11:11:04.901 INFO:tasks.ceph:Setting up mon.a... 2023-08-27T11:11:04.902 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-08-27T11:11:04.972 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.keyring 2023-08-27T11:11:04.975 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-08-27T11:11:05.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-27T11:11:05.050 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.45'), ('mon.b', '172.21.15.135'), ('mon.c', '[v2:172.21.15.135:3301,v1:172.21.15.135:6790]')] 2023-08-27T11:11:05.050 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.45,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-08-27T11:11:05.051 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:05.052 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-08-27T11:11:05.107 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.45 --add b 172.21.15.135 --addv c '[v2:172.21.15.135:3301,v1:172.21.15.135:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:05.196 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:05.196 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: generated fsid 8ea1ae34-1697-4b79-b2ce-3f741cd3c828 2023-08-27T11:11:05.197 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = pacific 2023-08-27T11:11:05.197 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 0 2023-08-27T11:11:05.197 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 8ea1ae34-1697-4b79-b2ce-3f741cd3c828 2023-08-27T11:11:05.197 INFO:teuthology.orchestra.run.smithi045.stdout:last_changed 2023-08-27T11:11:05.195766+0000 2023-08-27T11:11:05.197 INFO:teuthology.orchestra.run.smithi045.stdout:created 2023-08-27T11:11:05.195766+0000 2023-08-27T11:11:05.198 INFO:teuthology.orchestra.run.smithi045.stdout:min_mon_release 16 (pacific) 2023-08-27T11:11:05.198 INFO:teuthology.orchestra.run.smithi045.stdout:election_strategy: 1 2023-08-27T11:11:05.198 INFO:teuthology.orchestra.run.smithi045.stdout:0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2023-08-27T11:11:05.198 INFO:teuthology.orchestra.run.smithi045.stdout:1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2023-08-27T11:11:05.198 INFO:teuthology.orchestra.run.smithi045.stdout:2: [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] mon.c 2023-08-27T11:11:05.199 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-08-27T11:11:05.200 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-08-27T11:11:05.217 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8ea1ae34-1697-4b79-b2ce-3f741cd3c828... 2023-08-27T11:11:05.218 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-27T11:11:05.260 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-27T11:11:05.304 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2023-08-27T11:11:05.304 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2023-08-27T11:11:05.304 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-27T11:11:05.304 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2023-08-27T11:11:05.305 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.305 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2023-08-27T11:11:05.305 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.305 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2023-08-27T11:11:05.306 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2023-08-27T11:11:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2023-08-27T11:11:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.307 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2023-08-27T11:11:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2023-08-27T11:11:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-27T11:11:05.308 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2023-08-27T11:11:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2023-08-27T11:11:05.309 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-27T11:11:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2023-08-27T11:11:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.310 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2023-08-27T11:11:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2023-08-27T11:11:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2023-08-27T11:11:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2023-08-27T11:11:05.311 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2023-08-27T11:11:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2023-08-27T11:11:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2023-08-27T11:11:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-27T11:11:05.312 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-27T11:11:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2023-08-27T11:11:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-27T11:11:05.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2023-08-27T11:11:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2023-08-27T11:11:05.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2023-08-27T11:11:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2023-08-27T11:11:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2023-08-27T11:11:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2023-08-27T11:11:05.315 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2023-08-27T11:11:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 8ea1ae34-1697-4b79-b2ce-3f741cd3c828 2023-08-27T11:11:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.316 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2023-08-27T11:11:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2023-08-27T11:11:05.316 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2023-08-27T11:11:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2023-08-27T11:11:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2023-08-27T11:11:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.317 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2023-08-27T11:11:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2023-08-27T11:11:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2023-08-27T11:11:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2023-08-27T11:11:05.318 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2023-08-27T11:11:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2023-08-27T11:11:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2023-08-27T11:11:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.319 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2023-08-27T11:11:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2023-08-27T11:11:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2023-08-27T11:11:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.320 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2023-08-27T11:11:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2023-08-27T11:11:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2023-08-27T11:11:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2023-08-27T11:11:05.321 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2023-08-27T11:11:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2023-08-27T11:11:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 20 2023-08-27T11:11:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 20 2023-08-27T11:11:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-08-27T11:11:05.322 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2023-08-27T11:11:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 10 2023-08-27T11:11:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2023-08-27T11:11:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2023-08-27T11:11:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2023-08-27T11:11:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2023-08-27T11:11:05.323 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2023-08-27T11:11:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2023-08-27T11:11:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.324 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2023-08-27T11:11:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-08-27T11:11:05.324 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2023-08-27T11:11:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2023-08-27T11:11:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2023-08-27T11:11:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2023-08-27T11:11:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2023-08-27T11:11:05.325 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2023-08-27T11:11:05.326 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2023-08-27T11:11:05.326 INFO:teuthology.orchestra.run.smithi045.stdout: debug finisher = 20 2023-08-27T11:11:05.326 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.326 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2023-08-27T11:11:05.326 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-08-27T11:11:05.327 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2023-08-27T11:11:05.327 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2023-08-27T11:11:05.327 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2023-08-27T11:11:05.327 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2023-08-27T11:11:05.327 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2023-08-27T11:11:05.327 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2023-08-27T11:11:05.328 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2023-08-27T11:11:05.328 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2023-08-27T11:11:05.328 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2023-08-27T11:11:05.328 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.328 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-27T11:11:05.329 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2023-08-27T11:11:05.329 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2023-08-27T11:11:05.329 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.329 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2023-08-27T11:11:05.329 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-27T11:11:05.330 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-27T11:11:05.330 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.330 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2023-08-27T11:11:05.330 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2023-08-27T11:11:05.330 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.330 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2023-08-27T11:11:05.331 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2023-08-27T11:11:05.331 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:05.331 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2023-08-27T11:11:05.331 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2023-08-27T11:11:05.331 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2023-08-27T11:11:05.332 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2023-08-27T11:11:05.332 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-27T11:11:05.332 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-27T11:11:05.332 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2023-08-27T11:11:05.332 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2023-08-27T11:11:05.333 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-08-27T11:11:05.333 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2023-08-27T11:11:05.333 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2023-08-27T11:11:05.333 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2023-08-27T11:11:05.333 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2023-08-27T11:11:05.334 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2023-08-27T11:11:05.334 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2023-08-27T11:11:05.334 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2023-08-27T11:11:05.334 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2023-08-27T11:11:05.334 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2023-08-27T11:11:05.334 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2023-08-27T11:11:05.335 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2023-08-27T11:11:05.335 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2023-08-27T11:11:05.335 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2023-08-27T11:11:05.335 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2023-08-27T11:11:05.335 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2023-08-27T11:11:05.336 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = True 2023-08-27T11:11:05.336 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2023-08-27T11:11:05.336 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = True 2023-08-27T11:11:05.336 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2023-08-27T11:11:05.337 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2023-08-27T11:11:05.337 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2023-08-27T11:11:05.338 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2023-08-27T11:11:05.339 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2023-08-27T11:11:05.339 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-27T11:11:05.339 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2023-08-27T11:11:05.339 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.340 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2023-08-27T11:11:05.340 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.340 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2023-08-27T11:11:05.340 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.340 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2023-08-27T11:11:05.341 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2023-08-27T11:11:05.341 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.341 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2023-08-27T11:11:05.341 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2023-08-27T11:11:05.341 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.342 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-27T11:11:05.342 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2023-08-27T11:11:05.342 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.342 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2023-08-27T11:11:05.343 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-27T11:11:05.343 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.343 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2023-08-27T11:11:05.343 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.343 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2023-08-27T11:11:05.344 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2023-08-27T11:11:05.344 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2023-08-27T11:11:05.344 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2023-08-27T11:11:05.344 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2023-08-27T11:11:05.344 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2023-08-27T11:11:05.345 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2023-08-27T11:11:05.345 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-27T11:11:05.345 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-27T11:11:05.345 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2023-08-27T11:11:05.345 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.346 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-27T11:11:05.346 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.346 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2023-08-27T11:11:05.346 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.346 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2023-08-27T11:11:05.347 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.347 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2023-08-27T11:11:05.347 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2023-08-27T11:11:05.347 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2023-08-27T11:11:05.347 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.45,172.21.15.135,[v2:172.21.15.135:3301,v1:172.21.15.135:6790]" 2023-08-27T11:11:05.348 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on skipped message = False 2023-08-27T11:11:05.348 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = 8ea1ae34-1697-4b79-b2ce-3f741cd3c828 2023-08-27T11:11:05.348 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.348 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2023-08-27T11:11:05.348 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2023-08-27T11:11:05.349 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.349 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2023-08-27T11:11:05.349 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2023-08-27T11:11:05.349 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2023-08-27T11:11:05.349 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.350 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2023-08-27T11:11:05.350 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2023-08-27T11:11:05.350 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.350 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2023-08-27T11:11:05.350 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2023-08-27T11:11:05.351 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2023-08-27T11:11:05.351 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.351 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2023-08-27T11:11:05.351 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2023-08-27T11:11:05.351 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.351 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2023-08-27T11:11:05.352 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.352 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2023-08-27T11:11:05.352 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.352 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2023-08-27T11:11:05.352 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.353 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2023-08-27T11:11:05.353 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2023-08-27T11:11:05.353 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.353 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2023-08-27T11:11:05.354 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2023-08-27T11:11:05.354 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore block size = 96636764160 2023-08-27T11:11:05.354 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore fsck on mount = True 2023-08-27T11:11:05.354 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluefs = 20 2023-08-27T11:11:05.354 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluestore = 20 2023-08-27T11:11:05.354 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-08-27T11:11:05.355 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2023-08-27T11:11:05.355 INFO:teuthology.orchestra.run.smithi135.stdout: debug rocksdb = 10 2023-08-27T11:11:05.355 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd backfillfull_ratio = 0.85 2023-08-27T11:11:05.355 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd full ratio = 0.9 2023-08-27T11:11:05.355 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd nearfull ratio = 0.8 2023-08-27T11:11:05.356 INFO:teuthology.orchestra.run.smithi135.stdout: osd failsafe full ratio = 0.95 2023-08-27T11:11:05.356 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = bluestore 2023-08-27T11:11:05.356 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2023-08-27T11:11:05.356 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.356 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2023-08-27T11:11:05.357 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-08-27T11:11:05.357 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2023-08-27T11:11:05.357 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2023-08-27T11:11:05.357 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2023-08-27T11:11:05.357 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2023-08-27T11:11:05.358 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2023-08-27T11:11:05.358 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2023-08-27T11:11:05.358 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2023-08-27T11:11:05.358 INFO:teuthology.orchestra.run.smithi135.stdout: debug finisher = 20 2023-08-27T11:11:05.358 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.358 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2023-08-27T11:11:05.359 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-08-27T11:11:05.359 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2023-08-27T11:11:05.359 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2023-08-27T11:11:05.359 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2023-08-27T11:11:05.359 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2023-08-27T11:11:05.360 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2023-08-27T11:11:05.360 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2023-08-27T11:11:05.360 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2023-08-27T11:11:05.360 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2023-08-27T11:11:05.360 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2023-08-27T11:11:05.361 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.361 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-27T11:11:05.361 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2023-08-27T11:11:05.361 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2023-08-27T11:11:05.361 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.362 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2023-08-27T11:11:05.362 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-27T11:11:05.362 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-27T11:11:05.362 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.362 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2023-08-27T11:11:05.363 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2023-08-27T11:11:05.363 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.363 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2023-08-27T11:11:05.363 INFO:teuthology.orchestra.run.smithi135.stdout: mon op complaint time = 120 2023-08-27T11:11:05.363 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-27T11:11:05.364 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2023-08-27T11:11:05.364 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2023-08-27T11:11:05.364 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2023-08-27T11:11:05.364 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2023-08-27T11:11:05.364 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-27T11:11:05.364 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-27T11:11:05.365 INFO:teuthology.orchestra.run.smithi135.stdout: client mount timeout = 600 2023-08-27T11:11:05.365 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2023-08-27T11:11:05.365 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-08-27T11:11:05.365 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = "" 2023-08-27T11:11:05.365 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2023-08-27T11:11:05.366 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2023-08-27T11:11:05.366 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2023-08-27T11:11:05.366 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2023-08-27T11:11:05.366 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2023-08-27T11:11:05.366 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2023-08-27T11:11:05.367 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2023-08-27T11:11:05.367 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2023-08-27T11:11:05.367 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2023-08-27T11:11:05.367 INFO:teuthology.orchestra.run.smithi135.stdout:[mds] 2023-08-27T11:11:05.368 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds = 20 2023-08-27T11:11:05.368 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds balancer = 20 2023-08-27T11:11:05.368 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-08-27T11:11:05.368 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug frag = True 2023-08-27T11:11:05.368 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug scatterstat = True 2023-08-27T11:11:05.368 INFO:teuthology.orchestra.run.smithi135.stdout: mds op complaint time = 180 2023-08-27T11:11:05.369 INFO:teuthology.orchestra.run.smithi135.stdout: mds verify scatter = True 2023-08-27T11:11:05.369 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2023-08-27T11:11:05.369 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2023-08-27T11:11:05.369 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2023-08-27T11:11:05.370 INFO:tasks.ceph:Creating admin key on mon.a... 2023-08-27T11:11:05.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-08-27T11:11:05.463 INFO:tasks.ceph:Copying monmap to all nodes... 2023-08-27T11:11:05.464 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:05.464 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-08-27T11:11:05.481 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:05.482 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-08-27T11:11:05.538 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:11:05.538 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:05.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-27T11:11:05.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-27T11:11:05.660 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:05.660 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:05.732 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi135.front.sepia.ceph.com 2023-08-27T11:11:05.733 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:05.733 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-27T11:11:05.733 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-27T11:11:05.777 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:05.778 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:05.837 INFO:tasks.ceph:Setting up mon nodes... 2023-08-27T11:11:05.837 INFO:tasks.ceph:Setting up mgr nodes... 2023-08-27T11:11:05.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-08-27T11:11:05.897 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-08-27T11:11:05.900 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-08-27T11:11:06.047 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-08-27T11:11:06.050 INFO:tasks.ceph:Setting up mds nodes... 2023-08-27T11:11:06.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-08-27T11:11:06.111 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-08-27T11:11:06.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-08-27T11:11:06.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-08-27T11:11:06.239 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-08-27T11:11:06.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-08-27T11:11:06.270 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-08-27T11:11:06.332 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-08-27T11:11:06.335 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-08-27T11:11:06.364 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-08-27T11:11:06.460 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-08-27T11:11:06.463 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-08-27T11:11:06.489 INFO:tasks.ceph_client:Setting up client nodes... 2023-08-27T11:11:06.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-27T11:11:06.536 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-08-27T11:11:06.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-08-27T11:11:06.595 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-08-27T11:11:06.608 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-08-27T11:11:06.652 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-08-27T11:11:06.665 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-08-27T11:11:06.748 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-08-27T11:11:06.761 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-08-27T11:11:06.761 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-27T11:11:06.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-08-27T11:11:06.790 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:06.790 INFO:tasks.ceph:role: osd.0 2023-08-27T11:11:06.791 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:11:06.791 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-27T11:11:06.858 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:11:06.858 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:11:06.859 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:11:06.859 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:11:06.859 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:11:06.859 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:11:06.859 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:11:06.860 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:11:06.860 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:11:06.860 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:11:06.880 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-08-27T11:11:06.884 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-08-27T11:11:06.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-08-27T11:11:06.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-08-27T11:11:07.037 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-08-27T11:11:07.105 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:07.105 INFO:tasks.ceph:role: osd.1 2023-08-27T11:11:07.105 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:11:07.106 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-27T11:11:07.173 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:11:07.174 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:11:07.174 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:11:07.174 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:11:07.174 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:11:07.175 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:11:07.175 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:11:07.175 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:11:07.175 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:11:07.175 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:11:07.197 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-08-27T11:11:07.198 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-08-27T11:11:07.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-08-27T11:11:07.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-08-27T11:11:07.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-08-27T11:11:07.420 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:07.420 INFO:tasks.ceph:role: osd.2 2023-08-27T11:11:07.421 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:11:07.421 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-27T11:11:07.489 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:11:07.489 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:11:07.489 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:11:07.490 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:11:07.490 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:11:07.490 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:11:07.490 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:11:07.491 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:11:07.491 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:11:07.491 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:11:07.510 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-08-27T11:11:07.512 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-08-27T11:11:07.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-08-27T11:11:07.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-08-27T11:11:07.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-08-27T11:11:07.731 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:07.731 INFO:tasks.ceph:role: osd.3 2023-08-27T11:11:07.732 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:11:07.732 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-27T11:11:07.798 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:11:07.798 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:11:07.799 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:11:07.799 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:11:07.799 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:11:07.799 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:11:07.799 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:11:07.800 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:11:07.800 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:11:07.800 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:11:07.821 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2023-08-27T11:11:07.822 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2023-08-27T11:11:07.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-08-27T11:11:07.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-08-27T11:11:07.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:08.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:08.079+0000 7f8c6df54640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-08-27T11:11:08.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:08.079+0000 7f8c6df54640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-08-27T11:11:08.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:08.079+0000 7f8c6df54640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-27T11:11:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:08.079+0000 7f8c6df54640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-27T11:11:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:08.079+0000 7f8c6df54640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-27T11:11:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:08.079+0000 7f8c6df54640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-08-27T11:11:10.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-08-27T11:11:10.985 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:11.078+0000 7fd7819cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-08-27T11:11:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:11.078+0000 7fd7819cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-08-27T11:11:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:11.078+0000 7fd7819cb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-27T11:11:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:11.078+0000 7fd7819cb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-27T11:11:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:11.078+0000 7fd7819cb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-27T11:11:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:11.078+0000 7fd7819cb640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-08-27T11:11:13.943 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-08-27T11:11:13.977 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:14.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:14.072+0000 7f2a7c29b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-08-27T11:11:14.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:14.073+0000 7f2a7c29b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-08-27T11:11:14.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:14.073+0000 7f2a7c29b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-27T11:11:14.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:14.073+0000 7f2a7c29b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-27T11:11:14.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:14.073+0000 7f2a7c29b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-27T11:11:14.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:14.073+0000 7f2a7c29b640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-08-27T11:11:16.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-08-27T11:11:16.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:17.064+0000 7f888b7ba640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-08-27T11:11:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:17.064+0000 7f888b7ba640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-08-27T11:11:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:17.064+0000 7f888b7ba640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-27T11:11:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:17.064+0000 7f888b7ba640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-27T11:11:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:17.064+0000 7f888b7ba640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-27T11:11:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:17.065+0000 7f888b7ba640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-08-27T11:11:19.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-08-27T11:11:19.968 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-08-27T11:11:20.001 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:20.001 INFO:tasks.ceph:role: osd.4 2023-08-27T11:11:20.001 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com 2023-08-27T11:11:20.002 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-27T11:11:20.078 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:11:20.078 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:11:20.078 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:11:20.079 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:11:20.079 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:11:20.079 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:11:20.079 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:11:20.079 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:11:20.080 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:11:20.080 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:11:20.513 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-08-27T11:11:20.515 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-08-27T11:11:20.516 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-08-27T11:11:20.678 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-08-27T11:11:20.715 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-08-27T11:11:20.784 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:20.784 INFO:tasks.ceph:role: osd.5 2023-08-27T11:11:20.785 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com 2023-08-27T11:11:20.785 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-27T11:11:20.860 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:11:20.861 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:11:20.861 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:11:20.861 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:11:20.862 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:11:20.862 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:11:20.862 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:11:20.862 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:11:20.862 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:11:20.863 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:11:21.279 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-08-27T11:11:21.280 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-08-27T11:11:21.280 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-08-27T11:11:21.387 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-08-27T11:11:21.419 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-08-27T11:11:21.486 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:21.486 INFO:tasks.ceph:role: osd.6 2023-08-27T11:11:21.486 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com 2023-08-27T11:11:21.487 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-27T11:11:21.560 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:11:21.561 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:11:21.561 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:11:21.562 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:11:21.562 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:11:21.562 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:11:21.562 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:11:21.563 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:11:21.563 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:11:21.563 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:11:22.025 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-08-27T11:11:22.027 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-08-27T11:11:22.028 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-08-27T11:11:22.186 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-08-27T11:11:22.219 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-08-27T11:11:22.287 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-27T11:11:22.287 INFO:tasks.ceph:role: osd.7 2023-08-27T11:11:22.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com 2023-08-27T11:11:22.288 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-27T11:11:22.361 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-27T11:11:22.361 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-27T11:11:22.362 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-27T11:11:22.362 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-27T11:11:22.362 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-27T11:11:22.362 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-08-27T11:11:22.363 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-27T11:11:22.363 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-27T11:11:22.363 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-27T11:11:22.363 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-27T11:11:22.804 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-08-27T11:11:22.806 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-08-27T11:11:22.807 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-08-27T11:11:22.983 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-08-27T11:11:23.020 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:23.123 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:23.122+0000 7fcd1200c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-08-27T11:11:23.124 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:23.122+0000 7fcd1200c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-08-27T11:11:23.124 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:23.122+0000 7fcd1200c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-27T11:11:23.124 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:23.122+0000 7fcd1200c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-27T11:11:23.124 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:23.122+0000 7fcd1200c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-27T11:11:23.124 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:23.122+0000 7fcd1200c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-08-27T11:11:25.990 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-08-27T11:11:26.021 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:26.114 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:26.112+0000 7fd99f8b4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-08-27T11:11:26.114 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:26.112+0000 7fd99f8b4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-08-27T11:11:26.114 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:26.112+0000 7fd99f8b4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-27T11:11:26.115 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:26.112+0000 7fd99f8b4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-27T11:11:26.115 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:26.112+0000 7fd99f8b4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-27T11:11:26.115 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:26.113+0000 7fd99f8b4640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-08-27T11:11:28.978 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-08-27T11:11:29.014 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:29.111 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:29.109+0000 7f64d03d7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-08-27T11:11:29.111 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:29.109+0000 7f64d03d7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-08-27T11:11:29.111 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:29.109+0000 7f64d03d7640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-27T11:11:29.112 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:29.109+0000 7f64d03d7640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-27T11:11:29.112 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:29.109+0000 7f64d03d7640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-27T11:11:29.112 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:29.110+0000 7f64d03d7640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-08-27T11:11:31.977 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-08-27T11:11:32.008 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:32.099 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:32.098+0000 7fce8b185640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-08-27T11:11:32.100 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:32.098+0000 7fce8b185640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-08-27T11:11:32.100 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:32.098+0000 7fce8b185640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-27T11:11:32.100 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:32.098+0000 7fce8b185640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-27T11:11:32.100 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:32.098+0000 7fce8b185640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-27T11:11:32.101 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:32.099+0000 7fce8b185640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-08-27T11:11:33.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:11:33.224 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:11:34.956 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-08-27T11:11:34.988 INFO:tasks.ceph:Reading keys from all nodes... 2023-08-27T11:11:34.988 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:34.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-08-27T11:11:35.022 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:35.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-08-27T11:11:35.095 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:35.096 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-08-27T11:11:35.167 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:35.167 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-08-27T11:11:35.237 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:35.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-08-27T11:11:35.308 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:35.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-08-27T11:11:35.377 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:35.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-08-27T11:11:35.447 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:35.448 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-08-27T11:11:35.479 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:35.480 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-08-27T11:11:35.549 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:35.550 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-08-27T11:11:35.619 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:35.619 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-08-27T11:11:35.687 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:35.688 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-08-27T11:11:35.756 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:35.756 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-08-27T11:11:35.824 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:35.824 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-08-27T11:11:35.892 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:35.892 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-08-27T11:11:35.913 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:35.913 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-08-27T11:11:35.974 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:35.975 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-08-27T11:11:35.992 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:35.993 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-08-27T11:11:36.049 INFO:tasks.ceph:Adding keys to all mons... 2023-08-27T11:11:36.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-27T11:11:36.052 DEBUG:teuthology.orchestra.run.smithi135:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-27T11:11:36.092 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.y] 2023-08-27T11:11:36.092 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBJL+tkW6B8NRAAF2hsNVRi3Ji4M8anuYwpDg== 2023-08-27T11:11:36.092 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2023-08-27T11:11:36.093 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tknzGlBhAABVGsUvHGyOdTgH9miKO0Cg== 2023-08-27T11:11:36.093 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2023-08-27T11:11:36.093 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tk6qRCDhAAGaj+NY32IDjdRcuSNbASpw== 2023-08-27T11:11:36.093 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.0] 2023-08-27T11:11:36.094 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBML+tkOzvBBBAA196QdZiLzcsm5hLBGDl0Qw== 2023-08-27T11:11:36.094 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.1] 2023-08-27T11:11:36.094 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBPL+tkIXaxBBAAfHwZ2SsmOENhnNxZsG230Q== 2023-08-27T11:11:36.094 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.2] 2023-08-27T11:11:36.094 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBSL+tkpzphBBAA37UzwlmuLdSWDIrkBu/8Rw== 2023-08-27T11:11:36.095 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.3] 2023-08-27T11:11:36.095 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBVL+tkt63hAxAAJReGReJHqyggSbdFEhm3+A== 2023-08-27T11:11:36.095 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.x] 2023-08-27T11:11:36.095 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkNxfXAhAA/K5r8Mx/C7nRitNuzXXvKg== 2023-08-27T11:11:36.095 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2023-08-27T11:11:36.096 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkEWbPExAAgt/9BpPD1vK9QoG8l3HeFw== 2023-08-27T11:11:36.096 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2023-08-27T11:11:36.096 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkE9NxGxAAnWum05rSbYQSLZ9IjjqDpQ== 2023-08-27T11:11:36.096 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.4] 2023-08-27T11:11:36.097 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBbL+tkTOFXBxAAZ1I44e50IrNzgj7mjKyuGA== 2023-08-27T11:11:36.097 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.5] 2023-08-27T11:11:36.097 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBeL+tkdezFBhAANof2C80WiBZ6hWm3oB3kfg== 2023-08-27T11:11:36.097 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.6] 2023-08-27T11:11:36.097 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBhL+tk7auXBhAAJPwKwTVIUzao5D3t4iXYcw== 2023-08-27T11:11:36.098 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.7] 2023-08-27T11:11:36.098 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBkL+tkY+ztBRAAvM2oxELLN8WV0i0spqfY6Q== 2023-08-27T11:11:36.098 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-08-27T11:11:36.098 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ== 2023-08-27T11:11:36.098 INFO:teuthology.orchestra.run.smithi045.stdout:[client.2] 2023-08-27T11:11:36.099 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tk1xh+IxAAswoUpd+D70hUwfeKI2gc0w== 2023-08-27T11:11:36.099 INFO:teuthology.orchestra.run.smithi045.stdout:[client.1] 2023-08-27T11:11:36.099 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkMCDoJhAAevee0PlTshlWJcPzX8fpYg== 2023-08-27T11:11:36.099 INFO:teuthology.orchestra.run.smithi045.stdout:[client.3] 2023-08-27T11:11:36.099 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkGWuaLBAAYM2t+TMEs59u635jL9f9kg== 2023-08-27T11:11:36.114 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.y] 2023-08-27T11:11:36.114 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBJL+tkW6B8NRAAF2hsNVRi3Ji4M8anuYwpDg== 2023-08-27T11:11:36.114 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2023-08-27T11:11:36.114 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBKL+tknzGlBhAABVGsUvHGyOdTgH9miKO0Cg== 2023-08-27T11:11:36.115 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2023-08-27T11:11:36.115 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBKL+tk6qRCDhAAGaj+NY32IDjdRcuSNbASpw== 2023-08-27T11:11:36.115 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.0] 2023-08-27T11:11:36.115 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBML+tkOzvBBBAA196QdZiLzcsm5hLBGDl0Qw== 2023-08-27T11:11:36.116 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.1] 2023-08-27T11:11:36.116 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBPL+tkIXaxBBAAfHwZ2SsmOENhnNxZsG230Q== 2023-08-27T11:11:36.116 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.2] 2023-08-27T11:11:36.116 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBSL+tkpzphBBAA37UzwlmuLdSWDIrkBu/8Rw== 2023-08-27T11:11:36.116 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.3] 2023-08-27T11:11:36.117 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBVL+tkt63hAxAAJReGReJHqyggSbdFEhm3+A== 2023-08-27T11:11:36.117 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.x] 2023-08-27T11:11:36.117 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBKL+tkNxfXAhAA/K5r8Mx/C7nRitNuzXXvKg== 2023-08-27T11:11:36.117 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2023-08-27T11:11:36.117 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBKL+tkEWbPExAAgt/9BpPD1vK9QoG8l3HeFw== 2023-08-27T11:11:36.118 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2023-08-27T11:11:36.118 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBKL+tkE9NxGxAAnWum05rSbYQSLZ9IjjqDpQ== 2023-08-27T11:11:36.118 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.4] 2023-08-27T11:11:36.118 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBbL+tkTOFXBxAAZ1I44e50IrNzgj7mjKyuGA== 2023-08-27T11:11:36.118 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.5] 2023-08-27T11:11:36.119 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBeL+tkdezFBhAANof2C80WiBZ6hWm3oB3kfg== 2023-08-27T11:11:36.119 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.6] 2023-08-27T11:11:36.120 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBhL+tk7auXBhAAJPwKwTVIUzao5D3t4iXYcw== 2023-08-27T11:11:36.120 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.7] 2023-08-27T11:11:36.120 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBkL+tkY+ztBRAAvM2oxELLN8WV0i0spqfY6Q== 2023-08-27T11:11:36.120 INFO:teuthology.orchestra.run.smithi135.stdout:[client.0] 2023-08-27T11:11:36.120 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ== 2023-08-27T11:11:36.121 INFO:teuthology.orchestra.run.smithi135.stdout:[client.2] 2023-08-27T11:11:36.121 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBKL+tk1xh+IxAAswoUpd+D70hUwfeKI2gc0w== 2023-08-27T11:11:36.121 INFO:teuthology.orchestra.run.smithi135.stdout:[client.1] 2023-08-27T11:11:36.121 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBKL+tkMCDoJhAAevee0PlTshlWJcPzX8fpYg== 2023-08-27T11:11:36.121 INFO:teuthology.orchestra.run.smithi135.stdout:[client.3] 2023-08-27T11:11:36.122 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBKL+tkGWuaLBAAYM2t+TMEs59u635jL9f9kg== 2023-08-27T11:11:36.123 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-27T11:11:36.143 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-27T11:11:36.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:11:36.218 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:11:36.276 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:11:36.278 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:11:36.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.338 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.393 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.449 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.502 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.504 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.557 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-27T11:11:36.559 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-27T11:11:36.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:11:36.615 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:11:36.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:11:36.673 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-27T11:11:36.729 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.731 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.787 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.840 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.842 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.898 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-27T11:11:36.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:11:36.954 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:11:37.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:11:37.014 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:11:37.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:11:37.073 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:11:37.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:11:37.132 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-27T11:11:37.188 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-08-27T11:11:37.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-08-27T11:11:37.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-27T11:11:37.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-08-27T11:11:37.344 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-08-27T11:11:37.371 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-27T11:11:37.469 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-08-27T11:11:37.496 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-08-27T11:11:37.562 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-27T11:11:37.661 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-08-27T11:11:37.687 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:37.690 DEBUG:teuthology.orchestra.run.smithi135:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-27T11:11:37.744 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-08-27T11:11:37.745 INFO:tasks.ceph.mon.a:Restarting daemon 2023-08-27T11:11:37.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-08-27T11:11:37.747 INFO:tasks.ceph.mon.a:Started 2023-08-27T11:11:37.748 INFO:tasks.ceph.mon.b:Restarting daemon 2023-08-27T11:11:37.748 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-08-27T11:11:37.786 INFO:tasks.ceph.mon.b:Started 2023-08-27T11:11:37.786 INFO:tasks.ceph.mon.c:Restarting daemon 2023-08-27T11:11:37.787 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-08-27T11:11:37.789 INFO:tasks.ceph.mon.c:Started 2023-08-27T11:11:37.789 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-08-27T11:11:37.790 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-08-27T11:11:37.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-08-27T11:11:37.792 INFO:tasks.ceph.mgr.y:Started 2023-08-27T11:11:37.792 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-08-27T11:11:37.793 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-08-27T11:11:37.795 INFO:tasks.ceph.mgr.x:Started 2023-08-27T11:11:37.795 INFO:tasks.ceph:Setting crush tunables to default 2023-08-27T11:11:37.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd crush tunables default 2023-08-27T11:11:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.920+0000 7fbe0a3cd700 1 Processor -- start 2023-08-27T11:11:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.920+0000 7fbe0a3cd700 1 -- start start 2023-08-27T11:11:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe0a3cd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe0a3cd700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe0a3cd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0415b460 con 0x7fbe040b17c0 2023-08-27T11:11:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe0a3cd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe04157fc0 con 0x7fbe04157be0 2023-08-27T11:11:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe0a3cd700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe040b2280 con 0x7fbe0406d5e0 2023-08-27T11:11:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe03fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe037fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbe040b17c0 0x7fbe0406ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37532/0 (socket says 172.21.15.45:37532) 2023-08-27T11:11:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe037fe700 1 -- 172.21.15.45:0/629621463 learned_addr learned my addr 172.21.15.45:0/629621463 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:11:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe03fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50754/0 (socket says 172.21.15.45:50754) 2023-08-27T11:11:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 msgr2=0x7fbe040716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 msgr2=0x7fbe040716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:11:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.921+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.922+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 msgr2=0x7fbe040b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-08-27T11:11:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.922+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 msgr2=0x7fbe040b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.922+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:37.922+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:11:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.122+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.122+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 msgr2=0x7fbe040716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.122+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 msgr2=0x7fbe040716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.122+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.122+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:11:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.123+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.123+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 msgr2=0x7fbe040b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.123+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 msgr2=0x7fbe040b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.123+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.123+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:11:38.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.523+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:38.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.523+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 msgr2=0x7fbe040716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:38.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.523+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 msgr2=0x7fbe040716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:38.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.523+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:38.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.523+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:11:38.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.524+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:38.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.524+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 msgr2=0x7fbe040b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:38.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.524+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 msgr2=0x7fbe040b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:38.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.524+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:38.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:38.524+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:11:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.324+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.324+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 msgr2=0x7fbe040716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.324+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 msgr2=0x7fbe040716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.324+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.324+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:11:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.325+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.326+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 msgr2=0x7fbe040b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.326+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 msgr2=0x7fbe040b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.326+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:39.326+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:11:40.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.921+0000 7fbe01ffb700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 msgr2=0x7fbe040b1c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:11:40.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.921+0000 7fbe01ffb700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.135:3300/0 conn(0x7fbe0406d5e0 0x7fbe040b1c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:40.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.921+0000 7fbe01ffb700 1 -- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 msgr2=0x7fbe040716b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:11:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.921+0000 7fbe01ffb700 1 --2- 172.21.15.45:0/629621463 >> v2:172.21.15.45:3300/0 conn(0x7fbe04157be0 0x7fbe040716b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.921+0000 7fbe01ffb700 1 -- 172.21.15.45:0/629621463 >> v1:172.21.15.45:6789/0 conn(0x7fbe040b17c0 legacy=0x7fbe0406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.921+0000 7fbe01ffb700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.921+0000 7fbe01ffb700 1 -- 172.21.15.45:0/629621463 --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe040b2560 con 0x7fbdf00061e0 2023-08-27T11:11:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.921+0000 7fbe01ffb700 1 -- 172.21.15.45:0/629621463 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe04157fc0 con 0x7fbdf0003740 2023-08-27T11:11:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.922+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.922+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdf0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.922+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdf0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.922+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:40.922+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:11:41.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.123+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:41.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.123+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdf0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.123+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdf0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.123+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.123+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:11:41.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.524+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:41.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.524+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdf0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:41.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.524+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdf0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:41.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.524+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:41.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:41.524+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:11:42.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:42.326+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:42.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:42.326+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdf0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-27T11:11:42.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:42.326+0000 7fbe03fff700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdf0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-27T11:11:42.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:42.326+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-27T11:11:42.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:42.326+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:11:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.113+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032872573 0 0) 0x7fbe040b2560 con 0x7fbdf00061e0 2023-08-27T11:11:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.113+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdec0035b0 con 0x7fbdf00061e0 2023-08-27T11:11:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 <== mon.1 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4217309034 0 0) 0x7fbdec0035b0 con 0x7fbdf00061e0 2023-08-27T11:11:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdf0005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:11:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdf0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf000b030 con 0x7fbdf00061e0 2023-08-27T11:11:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 <== mon.1 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fbdf80016f0 con 0x7fbdf00061e0 2023-08-27T11:11:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 >> v1:172.21.15.135:6789/0 conn(0x7fbdf00061e0 legacy=0x7fbdf00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 reap_dead start 2023-08-27T11:11:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec003a00 0x7fbdec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec004de0 0x7fbdec005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe04157fc0 con 0x7fbdec004de0 2023-08-27T11:11:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec00bc90 con 0x7fbdec003a00 2023-08-27T11:11:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec00be50 con 0x7fbdf0003740 2023-08-27T11:11:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec004de0 0x7fbdec005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec003a00 0x7fbdec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.117+0000 7fbe037fe700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbdec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec003a00 msgr2=0x7fbdec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec003a00 0x7fbdec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe0896a700 1 -- 172.21.15.45:0/629621463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec0035b0 con 0x7fbdec004de0 2023-08-27T11:11:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe03fff700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec003a00 0x7fbdec003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe037fe700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdec004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe0896a700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec004de0 0x7fbdec005260 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe0406c380 tx=0x7fbdec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38dd6d1fbf05412 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc0042c0 con 0x7fbdec004de0 2023-08-27T11:11:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc00f030 con 0x7fbdec004de0 2023-08-27T11:11:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.118+0000 7fbe02ffd700 1 -- 172.21.15.45:0/629621463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc004660 con 0x7fbdec004de0 2023-08-27T11:11:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.119+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/629621463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec004de0 msgr2=0x7fbdec005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.119+0000 7fbe0a3cd700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec004de0 0x7fbdec005260 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe0406c380 tx=0x7fbdec005880 comp rx=0 tx=0).stop 2023-08-27T11:11:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.119+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/629621463 shutdown_connections 2023-08-27T11:11:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.119+0000 7fbe0a3cd700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec004de0 0x7fbdec005260 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.119+0000 7fbe0a3cd700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec003a00 0x7fbdec003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.119+0000 7fbe0a3cd700 1 --2- 172.21.15.45:0/629621463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbdec004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.119+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/629621463 >> 172.21.15.45:0/629621463 conn(0x7fbe040cc7c0 msgr2=0x7fbe040ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.119+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/629621463 shutdown_connections 2023-08-27T11:11:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.119+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/629621463 wait complete. 2023-08-27T11:11:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.121+0000 7fbe0a3cd700 1 Processor -- start 2023-08-27T11:11:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.122+0000 7fbe0a3cd700 1 -- start start 2023-08-27T11:11:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe0a3cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec003a00 0x7fbe041f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe0a3cd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec004de0 0x7fbe040ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe0a3cd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbe040ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe0a3cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040b38b0 con 0x7fbdec003a00 2023-08-27T11:11:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe0a3cd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040b2c30 con 0x7fbdec004de0 2023-08-27T11:11:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe0a3cd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040b2f10 con 0x7fbdf0003740 2023-08-27T11:11:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec003a00 0x7fbe041f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec003a00 0x7fbe041f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50814/0 (socket says 172.21.15.45:50814) 2023-08-27T11:11:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe03fff700 1 -- 172.21.15.45:0/3993496264 learned_addr learned my addr 172.21.15.45:0/3993496264 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:11:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe0896a700 1 --2- 172.21.15.45:0/3993496264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbe040ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe037fe700 1 --2- 172.21.15.45:0/3993496264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec004de0 0x7fbe040ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe03fff700 1 -- 172.21.15.45:0/3993496264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 msgr2=0x7fbe040ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe03fff700 1 --2- 172.21.15.45:0/3993496264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbe040ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe03fff700 1 -- 172.21.15.45:0/3993496264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec004de0 msgr2=0x7fbe040ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe03fff700 1 --2- 172.21.15.45:0/3993496264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec004de0 0x7fbe040ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.123+0000 7fbe03fff700 1 -- 172.21.15.45:0/3993496264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe040b31f0 con 0x7fbdec003a00 2023-08-27T11:11:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.124+0000 7fbe03fff700 1 --2- 172.21.15.45:0/3993496264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec003a00 0x7fbe041f6000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4019760 tx=0x7fbdf4003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.124+0000 7fbe00ff9700 1 -- 172.21.15.45:0/3993496264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf401e040 con 0x7fbdec003a00 2023-08-27T11:11:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.124+0000 7fbe00ff9700 1 -- 172.21.15.45:0/3993496264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf400c040 con 0x7fbdec003a00 2023-08-27T11:11:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.124+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/3993496264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe040b34d0 con 0x7fbdec003a00 2023-08-27T11:11:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.124+0000 7fbe00ff9700 1 -- 172.21.15.45:0/3993496264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4017a10 con 0x7fbdec003a00 2023-08-27T11:11:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.124+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/3993496264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe040c9250 con 0x7fbdec003a00 2023-08-27T11:11:43.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.124+0000 7fbe00ff9700 1 -- 172.21.15.45:0/3993496264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbdf4005a20 con 0x7fbdec003a00 2023-08-27T11:11:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.124+0000 7fbe00ff9700 1 -- 172.21.15.45:0/3993496264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbdf40065e0 con 0x7fbdec003a00 2023-08-27T11:11:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.125+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/3993496264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe04072690 con 0x7fbdec003a00 2023-08-27T11:11:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.128+0000 7fbe00ff9700 1 -- 172.21.15.45:0/3993496264 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbdf4006000 con 0x7fbdec003a00 2023-08-27T11:11:43.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.209+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/3993496264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbe040734a0 con 0x7fbdec003a00 2023-08-27T11:11:43.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.211+0000 7fbe00ff9700 1 -- 172.21.15.45:0/3993496264 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbdf40067f0 con 0x7fbdec003a00 2023-08-27T11:11:43.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.212+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/3993496264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec003a00 msgr2=0x7fbe041f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:43.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.212+0000 7fbe0a3cd700 1 --2- 172.21.15.45:0/3993496264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec003a00 0x7fbe041f6000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4019760 tx=0x7fbdf4003980 comp rx=0 tx=0).stop 2023-08-27T11:11:43.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.212+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/3993496264 shutdown_connections 2023-08-27T11:11:43.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.212+0000 7fbe0a3cd700 1 --2- 172.21.15.45:0/3993496264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdec003a00 0x7fbe041f6000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.212+0000 7fbe0a3cd700 1 --2- 172.21.15.45:0/3993496264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdf0003740 0x7fbe040ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.212+0000 7fbe0a3cd700 1 --2- 172.21.15.45:0/3993496264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec004de0 0x7fbe040ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.212+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/3993496264 >> 172.21.15.45:0/3993496264 conn(0x7fbe040cc7c0 msgr2=0x7fbe040cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:43.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.212+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/3993496264 shutdown_connections 2023-08-27T11:11:43.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:43.212+0000 7fbe0a3cd700 1 -- 172.21.15.45:0/3993496264 wait complete. 2023-08-27T11:11:43.216 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2023-08-27T11:11:43.226 INFO:tasks.ceph:check_enable_crimson: False 2023-08-27T11:11:43.227 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-08-27T11:11:43.227 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:43.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-08-27T11:11:43.239 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:43.237+0000 7f22b76ca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:11:43.256 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:43.257 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-08-27T11:11:43.305 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:43.303+0000 7f22b76ca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:11:43.324 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:43.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-08-27T11:11:43.380 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:43.378+0000 7f22b76ca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:11:43.392 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:11:43.392 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-08-27T11:11:43.458 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:43.456+0000 7f22b76ca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:11:43.461 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:43.461 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-08-27T11:11:43.487 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:43.487 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-08-27T11:11:43.552 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:43.553 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-08-27T11:11:43.608 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:43.606+0000 7f22b76ca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:11:43.619 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:11:43.620 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-08-27T11:11:43.687 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa 0 2023-08-27T11:11:43.847 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.845+0000 7f7c0eb6d700 1 Processor -- start 2023-08-27T11:11:43.848 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.845+0000 7f7c0eb6d700 1 -- start start 2023-08-27T11:11:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.846+0000 7f7c0eb6d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c0806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.846+0000 7f7c0eb6d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7c0806a7c0 0x7f7c080b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.851 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.846+0000 7f7c0eb6d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0806b160 con 0x7f7c0815d340 2023-08-27T11:11:43.851 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.846+0000 7f7c0eb6d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c08072010 con 0x7f7c0806a7c0 2023-08-27T11:11:43.851 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.846+0000 7f7c0eb6d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0806ac50 con 0x7f7c08160d00 2023-08-27T11:11:43.851 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.846+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c0806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.846+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c0806a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:59760/0 (socket says 172.21.15.135:59760) 2023-08-27T11:11:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.846+0000 7f7c07fff700 1 -- 172.21.15.135:0/3410059583 learned_addr learned my addr 172.21.15.135:0/3410059583 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.846+0000 7f7c0d10a700 1 --2- 172.21.15.135:0/3410059583 >> v2:172.21.15.45:3300/0 conn(0x7f7c0806a7c0 0x7f7c080b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c07fff700 1 -- 172.21.15.135:0/3410059583 >> v2:172.21.15.45:3300/0 conn(0x7f7c0806a7c0 msgr2=0x7f7c080b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c07fff700 1 --2- 172.21.15.135:0/3410059583 >> v2:172.21.15.45:3300/0 conn(0x7f7c0806a7c0 0x7f7c080b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c07fff700 1 -- 172.21.15.135:0/3410059583 >> v1:172.21.15.45:6789/0 conn(0x7f7c0815d340 legacy=0x7f7c080cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c07fff700 1 -- 172.21.15.135:0/3410059583 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c080cdf90 con 0x7f7c08160d00 2023-08-27T11:11:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c07fff700 1 --2- 172.21.15.135:0/3410059583 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c0806a280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098f0 tx=0x7f7bf4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb78fc339a318cc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c077fe700 1 -- 172.21.15.135:0/3410059583 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400b8a0 con 0x7f7c08160d00 2023-08-27T11:11:43.854 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c077fe700 1 -- 172.21.15.135:0/3410059583 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf400ba00 con 0x7f7c08160d00 2023-08-27T11:11:43.854 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c077fe700 1 -- 172.21.15.135:0/3410059583 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400bd90 con 0x7f7c08160d00 2023-08-27T11:11:43.854 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3410059583 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 msgr2=0x7f7c0806a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:43.854 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c0eb6d700 1 --2- 172.21.15.135:0/3410059583 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c0806a280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40098f0 tx=0x7f7bf4009c00 comp rx=0 tx=0).stop 2023-08-27T11:11:43.854 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3410059583 shutdown_connections 2023-08-27T11:11:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c0eb6d700 1 --2- 172.21.15.135:0/3410059583 >> v2:172.21.15.45:3300/0 conn(0x7f7c0806a7c0 0x7f7c080b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c0eb6d700 1 --2- 172.21.15.135:0/3410059583 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c0806a280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.847+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3410059583 >> 172.21.15.135:0/3410059583 conn(0x7f7c080cc7c0 msgr2=0x7f7c080ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.848+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3410059583 shutdown_connections 2023-08-27T11:11:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.848+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3410059583 wait complete. 2023-08-27T11:11:43.858 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.849+0000 7f7c0eb6d700 1 Processor -- start 2023-08-27T11:11:43.859 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.849+0000 7f7c0eb6d700 1 -- start start 2023-08-27T11:11:43.859 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0eb6d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0806a7c0 0x7f7c0814c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.859 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0eb6d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0815d340 0x7f7c080c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.859 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0eb6d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c080b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:43.860 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0eb6d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080acc30 con 0x7f7c0806a7c0 2023-08-27T11:11:43.860 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0eb6d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080ada90 con 0x7f7c0815d340 2023-08-27T11:11:43.860 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0eb6d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080b2b90 con 0x7f7c08160d00 2023-08-27T11:11:43.860 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0d10a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c080b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.861 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0815d340 0x7f7c080c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.861 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0d10a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c080b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:59768/0 (socket says 172.21.15.135:59768) 2023-08-27T11:11:43.861 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0d10a700 1 -- 172.21.15.135:0/3447392242 learned_addr learned my addr 172.21.15.135:0/3447392242 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:43.861 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0c909700 1 --2- 172.21.15.135:0/3447392242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0806a7c0 0x7f7c0814c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:43.862 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0d10a700 1 -- 172.21.15.135:0/3447392242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0815d340 msgr2=0x7f7c080c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:43.862 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0d10a700 1 --2- 172.21.15.135:0/3447392242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0815d340 0x7f7c080c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.862 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0d10a700 1 -- 172.21.15.135:0/3447392242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0806a7c0 msgr2=0x7f7c0814c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:43.862 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0d10a700 1 --2- 172.21.15.135:0/3447392242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0806a7c0 0x7f7c0814c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.863 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0d10a700 1 -- 172.21.15.135:0/3447392242 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4009550 con 0x7f7c08160d00 2023-08-27T11:11:43.863 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c07fff700 1 --2- 172.21.15.135:0/3447392242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0815d340 0x7f7c080c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:43.863 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0d10a700 1 --2- 172.21.15.135:0/3447392242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c080b06c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c00007e90 tx=0x7f7c0000e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:43.863 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c057fa700 1 -- 172.21.15.135:0/3447392242 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0000ed30 con 0x7f7c08160d00 2023-08-27T11:11:43.864 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c057fa700 1 -- 172.21.15.135:0/3447392242 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0000ee90 con 0x7f7c08160d00 2023-08-27T11:11:43.864 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3447392242 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c080ace80 con 0x7f7c08160d00 2023-08-27T11:11:43.864 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c057fa700 1 -- 172.21.15.135:0/3447392242 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00014660 con 0x7f7c08160d00 2023-08-27T11:11:43.864 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.850+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3447392242 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c080ad440 con 0x7f7c08160d00 2023-08-27T11:11:43.864 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.851+0000 7f7c057fa700 1 -- 172.21.15.135:0/3447392242 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7c00014ac0 con 0x7f7c08160d00 2023-08-27T11:11:43.865 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.851+0000 7f7c057fa700 1 -- 172.21.15.135:0/3447392242 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7c0001ec30 con 0x7f7c08160d00 2023-08-27T11:11:43.865 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.851+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3447392242 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c08072690 con 0x7f7c08160d00 2023-08-27T11:11:43.865 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.854+0000 7f7c057fa700 1 -- 172.21.15.135:0/3447392242 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f7c0001e460 con 0x7f7c08160d00 2023-08-27T11:11:43.940 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.938+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3447392242 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa", "id": 0} v 0) v1 -- 0x7f7c080adbd0 con 0x7f7c08160d00 2023-08-27T11:11:43.943 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.941+0000 7f7c057fa700 1 -- 172.21.15.135:0/3447392242 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7c0001e600 con 0x7f7c08160d00 2023-08-27T11:11:43.943 INFO:teuthology.orchestra.run.smithi135.stdout:0 2023-08-27T11:11:43.944 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.942+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3447392242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 msgr2=0x7f7c080b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:43.944 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.942+0000 7f7c0eb6d700 1 --2- 172.21.15.135:0/3447392242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c080b06c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c00007e90 tx=0x7f7c0000e3c0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.945 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.942+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3447392242 shutdown_connections 2023-08-27T11:11:43.945 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.942+0000 7f7c0eb6d700 1 --2- 172.21.15.135:0/3447392242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0806a7c0 0x7f7c0814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.945 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.942+0000 7f7c0eb6d700 1 --2- 172.21.15.135:0/3447392242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c08160d00 0x7f7c080b06c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.945 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.942+0000 7f7c0eb6d700 1 --2- 172.21.15.135:0/3447392242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c0815d340 0x7f7c080c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:43.946 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.942+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3447392242 >> 172.21.15.135:0/3447392242 conn(0x7f7c080cc7c0 msgr2=0x7f7c080cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:43.946 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.942+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3447392242 shutdown_connections 2023-08-27T11:11:43.946 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:43.942+0000 7f7c0eb6d700 1 -- 172.21.15.135:0/3447392242 wait complete. 2023-08-27T11:11:43.956 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 1d89c17a-9f89-47a3-a818-ad5c27e9b2e0 1 2023-08-27T11:11:44.003 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:44.001+0000 7f74d6019200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:11:44.007 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:44.005+0000 7f22b76ca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:11:44.071 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:44.070+0000 7f74d6019200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:11:44.078 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.075+0000 7ff83614b700 1 Processor -- start 2023-08-27T11:11:44.078 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff83614b700 1 -- start start 2023-08-27T11:11:44.081 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff83614b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff83015d340 0x7ff830160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.081 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff83614b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff83006a990 0x7ff8300b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.081 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff83614b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83006b160 con 0x7ff8300cd880 2023-08-27T11:11:44.081 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff83614b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff830072010 con 0x7ff83015d340 2023-08-27T11:11:44.081 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff83614b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8300af590 con 0x7ff83006a990 2023-08-27T11:11:44.082 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff82ffff700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff83006a990 0x7ff8300b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.082 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff82f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff83015d340 0x7ff830160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.082 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff82effd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff8300cd880 0x7ff83006a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.135:47960/0 (socket says 172.21.15.135:47960) 2023-08-27T11:11:44.082 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82effd700 1 -- 172.21.15.135:0/3002756685 learned_addr learned my addr 172.21.15.135:0/3002756685 (peer_addr_for_me v1:172.21.15.135:0/0) 2023-08-27T11:11:44.082 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.076+0000 7ff82f7fe700 1 --2- 172.21.15.135:0/3002756685 >> v2:172.21.15.45:3300/0 conn(0x7ff83015d340 0x7ff830160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.135:50580/0 (socket says 172.21.15.135:50580) 2023-08-27T11:11:44.083 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82f7fe700 1 -- 172.21.15.135:0/3002756685 >> v2:172.21.15.135:3300/0 conn(0x7ff83006a990 msgr2=0x7ff8300b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:44.083 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82f7fe700 1 --2- 172.21.15.135:0/3002756685 >> v2:172.21.15.135:3300/0 conn(0x7ff83006a990 0x7ff8300b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.083 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82f7fe700 1 -- 172.21.15.135:0/3002756685 >> v1:172.21.15.45:6789/0 conn(0x7ff8300cd880 legacy=0x7ff83006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.083 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82f7fe700 1 -- 172.21.15.135:0/3002756685 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8300af2b0 con 0x7ff83015d340 2023-08-27T11:11:44.083 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82e7fc700 1 -- 172.21.15.135:0/3002756685 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762172889 0 0) 0x7ff83006b160 con 0x7ff8300cd880 2023-08-27T11:11:44.084 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82ffff700 1 --2- 172.21.15.135:0/3002756685 >> v2:172.21.15.135:3300/0 conn(0x7ff83006a990 0x7ff8300b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:44.084 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82f7fe700 1 --2- 172.21.15.135:0/3002756685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 0x7ff830160d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff830072af0 tx=0x7ff820009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dbbe0de3c88fb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:44.084 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82e7fc700 1 -- 172.21.15.135:0/3002756685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820011830 con 0x7ff83015d340 2023-08-27T11:11:44.084 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82e7fc700 1 -- 172.21.15.135:0/3002756685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff820011990 con 0x7ff83015d340 2023-08-27T11:11:44.084 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.077+0000 7ff82e7fc700 1 -- 172.21.15.135:0/3002756685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820011d20 con 0x7ff83015d340 2023-08-27T11:11:44.085 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.078+0000 7ff83614b700 1 -- 172.21.15.135:0/3002756685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 msgr2=0x7ff830160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.085 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.078+0000 7ff83614b700 1 --2- 172.21.15.135:0/3002756685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 0x7ff830160d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff830072af0 tx=0x7ff820009b20 comp rx=0 tx=0).stop 2023-08-27T11:11:44.085 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.078+0000 7ff83614b700 1 -- 172.21.15.135:0/3002756685 shutdown_connections 2023-08-27T11:11:44.085 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.078+0000 7ff83614b700 1 --2- 172.21.15.135:0/3002756685 >> v2:172.21.15.135:3300/0 conn(0x7ff83006a990 0x7ff8300b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.085 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.078+0000 7ff83614b700 1 --2- 172.21.15.135:0/3002756685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 0x7ff830160d00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.086 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.078+0000 7ff83614b700 1 -- 172.21.15.135:0/3002756685 >> 172.21.15.135:0/3002756685 conn(0x7ff8300cc7c0 msgr2=0x7ff8300ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:44.086 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.078+0000 7ff83614b700 1 -- 172.21.15.135:0/3002756685 shutdown_connections 2023-08-27T11:11:44.086 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.078+0000 7ff83614b700 1 -- 172.21.15.135:0/3002756685 wait complete. 2023-08-27T11:11:44.088 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.079+0000 7ff83614b700 1 Processor -- start 2023-08-27T11:11:44.089 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.079+0000 7ff83614b700 1 -- start start 2023-08-27T11:11:44.089 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff83614b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83006a990 0x7ff830156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.089 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff83614b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8300cd880 0x7ff830151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.089 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff83614b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 0x7ff830159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.090 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff83614b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8301584d0 con 0x7ff83015d340 2023-08-27T11:11:44.090 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff83614b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8301569b0 con 0x7ff83006a990 2023-08-27T11:11:44.090 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff83614b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8301581f0 con 0x7ff8300cd880 2023-08-27T11:11:44.090 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff82f7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83006a990 0x7ff830156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.090 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff82effd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8300cd880 0x7ff830151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.091 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff82f7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83006a990 0x7ff830156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47920/0 (socket says 172.21.15.135:47920) 2023-08-27T11:11:44.091 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff82ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 0x7ff830159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.091 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.080+0000 7ff82f7fe700 1 -- 172.21.15.135:0/3001907957 learned_addr learned my addr 172.21.15.135:0/3001907957 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:44.091 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff82ffff700 1 -- 172.21.15.135:0/3001907957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8300cd880 msgr2=0x7ff830151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.091 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff82ffff700 1 --2- 172.21.15.135:0/3001907957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8300cd880 0x7ff830151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.092 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff82ffff700 1 -- 172.21.15.135:0/3001907957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83006a990 msgr2=0x7ff830156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.092 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff82ffff700 1 --2- 172.21.15.135:0/3001907957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83006a990 0x7ff830156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.092 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff82ffff700 1 -- 172.21.15.135:0/3001907957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828009a70 con 0x7ff83015d340 2023-08-27T11:11:44.092 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff82f7fe700 1 --2- 172.21.15.135:0/3001907957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83006a990 0x7ff830156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:44.092 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff82ffff700 1 --2- 172.21.15.135:0/3001907957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 0x7ff830159e20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff82800ef90 tx=0x7ff82800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:44.092 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff827fff700 1 -- 172.21.15.135:0/3001907957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828009e80 con 0x7ff83015d340 2023-08-27T11:11:44.093 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff827fff700 1 -- 172.21.15.135:0/3001907957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82800f040 con 0x7ff83015d340 2023-08-27T11:11:44.093 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff827fff700 1 -- 172.21.15.135:0/3001907957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800ccd0 con 0x7ff83015d340 2023-08-27T11:11:44.093 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff83614b700 1 -- 172.21.15.135:0/3001907957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8200094a0 con 0x7ff83015d340 2023-08-27T11:11:44.093 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff83614b700 1 -- 172.21.15.135:0/3001907957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff830156640 con 0x7ff83015d340 2023-08-27T11:11:44.094 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff827fff700 1 -- 172.21.15.135:0/3001907957 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff828003d20 con 0x7ff83015d340 2023-08-27T11:11:44.094 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.081+0000 7ff827fff700 1 -- 172.21.15.135:0/3001907957 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff828014830 con 0x7ff83015d340 2023-08-27T11:11:44.094 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.082+0000 7ff83614b700 1 -- 172.21.15.135:0/3001907957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8301561c0 con 0x7ff83015d340 2023-08-27T11:11:44.094 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.084+0000 7ff827fff700 1 -- 172.21.15.135:0/3001907957 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff8301561c0 con 0x7ff83015d340 2023-08-27T11:11:44.138 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:44.136+0000 7f22b76ca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:11:44.152 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:44.150+0000 7f74d6019200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:11:44.183 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.181+0000 7ff83614b700 1 -- 172.21.15.135:0/3001907957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1d89c17a-9f89-47a3-a818-ad5c27e9b2e0", "id": 1} v 0) v1 -- 0x7ff830072690 con 0x7ff83015d340 2023-08-27T11:11:44.185 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.183+0000 7ff827fff700 1 -- 172.21.15.135:0/3001907957 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d89c17a-9f89-47a3-a818-ad5c27e9b2e0", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff82802b0b0 con 0x7ff83015d340 2023-08-27T11:11:44.185 INFO:teuthology.orchestra.run.smithi135.stdout:1 2023-08-27T11:11:44.187 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.184+0000 7ff83614b700 1 -- 172.21.15.135:0/3001907957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 msgr2=0x7ff830159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.187 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.184+0000 7ff83614b700 1 --2- 172.21.15.135:0/3001907957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 0x7ff830159e20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff82800ef90 tx=0x7ff82800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.187 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.184+0000 7ff83614b700 1 -- 172.21.15.135:0/3001907957 shutdown_connections 2023-08-27T11:11:44.187 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.184+0000 7ff83614b700 1 --2- 172.21.15.135:0/3001907957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff83015d340 0x7ff830159e20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.187 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.184+0000 7ff83614b700 1 --2- 172.21.15.135:0/3001907957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff83006a990 0x7ff830156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.188 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.184+0000 7ff83614b700 1 --2- 172.21.15.135:0/3001907957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8300cd880 0x7ff830151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.188 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.184+0000 7ff83614b700 1 -- 172.21.15.135:0/3001907957 >> 172.21.15.135:0/3001907957 conn(0x7ff8300cc7c0 msgr2=0x7ff8300afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:44.188 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.184+0000 7ff83614b700 1 -- 172.21.15.135:0/3001907957 shutdown_connections 2023-08-27T11:11:44.188 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.184+0000 7ff83614b700 1 -- 172.21.15.135:0/3001907957 wait complete. 2023-08-27T11:11:44.199 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new b98fcbad-8b7b-4487-bbc8-ba2548e82f19 2 2023-08-27T11:11:44.226 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:44.223+0000 7f22b76ca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:11:44.234 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:44.233+0000 7f74d6019200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:11:44.310 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:44.308+0000 7f22b76ca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:11:44.319 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.317+0000 7f7c566f6700 1 Processor -- start 2023-08-27T11:11:44.319 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.317+0000 7f7c566f6700 1 -- start start 2023-08-27T11:11:44.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c566f6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7c500d0520 0x7f7c500ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c566f6700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7c500cb5c0 0x7f7c5015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c566f6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c5015ec50 con 0x7f7c500b65d0 2023-08-27T11:11:44.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c566f6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c5015c620 con 0x7f7c500d0520 2023-08-27T11:11:44.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c566f6700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c5015eea0 con 0x7f7c500cb5c0 2023-08-27T11:11:44.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c54c93700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7c500cb5c0 0x7f7c5015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c54c93700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7c500cb5c0 0x7f7c5015c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47930/0 (socket says 172.21.15.135:47930) 2023-08-27T11:11:44.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c54c93700 1 -- 172.21.15.135:0/2140674839 learned_addr learned my addr 172.21.15.135:0/2140674839 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:44.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c4f7fe700 1 --2- 172.21.15.135:0/2140674839 >> v2:172.21.15.45:3300/0 conn(0x7f7c500d0520 0x7f7c500ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c54c93700 1 -- 172.21.15.135:0/2140674839 >> v2:172.21.15.45:3300/0 conn(0x7f7c500d0520 msgr2=0x7f7c500ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:44.324 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c54c93700 1 --2- 172.21.15.135:0/2140674839 >> v2:172.21.15.45:3300/0 conn(0x7f7c500d0520 0x7f7c500ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.324 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c54c93700 1 -- 172.21.15.135:0/2140674839 >> v1:172.21.15.45:6789/0 conn(0x7f7c500b65d0 legacy=0x7f7c500cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.324 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.318+0000 7f7c54c93700 1 -- 172.21.15.135:0/2140674839 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5015f180 con 0x7f7c500cb5c0 2023-08-27T11:11:44.324 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c54c93700 1 --2- 172.21.15.135:0/2140674839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500cb5c0 0x7f7c5015c0e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7c500d0d80 tx=0x7f7c4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fe54f94d035481 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:44.324 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c4effd700 1 -- 172.21.15.135:0/2140674839 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4800e070 con 0x7f7c500cb5c0 2023-08-27T11:11:44.325 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c4effd700 1 -- 172.21.15.135:0/2140674839 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4800b910 con 0x7f7c500cb5c0 2023-08-27T11:11:44.325 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c4effd700 1 -- 172.21.15.135:0/2140674839 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4800bcc0 con 0x7f7c500cb5c0 2023-08-27T11:11:44.325 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c566f6700 1 -- 172.21.15.135:0/2140674839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500cb5c0 msgr2=0x7f7c5015c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.325 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c566f6700 1 --2- 172.21.15.135:0/2140674839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500cb5c0 0x7f7c5015c0e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7c500d0d80 tx=0x7f7c4800b330 comp rx=0 tx=0).stop 2023-08-27T11:11:44.325 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c566f6700 1 -- 172.21.15.135:0/2140674839 shutdown_connections 2023-08-27T11:11:44.326 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c566f6700 1 --2- 172.21.15.135:0/2140674839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500cb5c0 0x7f7c5015c0e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.326 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c566f6700 1 --2- 172.21.15.135:0/2140674839 >> v2:172.21.15.45:3300/0 conn(0x7f7c500d0520 0x7f7c500ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.326 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c566f6700 1 -- 172.21.15.135:0/2140674839 >> 172.21.15.135:0/2140674839 conn(0x7f7c5006f960 msgr2=0x7f7c5006fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:44.326 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c566f6700 1 -- 172.21.15.135:0/2140674839 shutdown_connections 2023-08-27T11:11:44.327 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.319+0000 7f7c566f6700 1 -- 172.21.15.135:0/2140674839 wait complete. 2023-08-27T11:11:44.329 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.321+0000 7f7c566f6700 1 Processor -- start 2023-08-27T11:11:44.330 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.321+0000 7f7c566f6700 1 -- start start 2023-08-27T11:11:44.330 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c566f6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500b65d0 0x7f7c500c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.330 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c566f6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c500cb5c0 0x7f7c501f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.330 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c566f6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c500d0520 0x7f7c500c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.330 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c566f6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c501f4f40 con 0x7f7c500cb5c0 2023-08-27T11:11:44.331 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c566f6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500c4d90 con 0x7f7c500b65d0 2023-08-27T11:11:44.331 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c566f6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500c5070 con 0x7f7c500d0520 2023-08-27T11:11:44.331 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500b65d0 0x7f7c500c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.331 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500b65d0 0x7f7c500c47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47932/0 (socket says 172.21.15.135:47932) 2023-08-27T11:11:44.331 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c4ffff700 1 -- 172.21.15.135:0/4088301956 learned_addr learned my addr 172.21.15.135:0/4088301956 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:44.332 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c54c93700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c500d0520 0x7f7c500c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.332 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c4f7fe700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c500cb5c0 0x7f7c501f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.332 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c54c93700 1 -- 172.21.15.135:0/4088301956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500b65d0 msgr2=0x7f7c500c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.332 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c54c93700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500b65d0 0x7f7c500c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.332 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c54c93700 1 -- 172.21.15.135:0/4088301956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c500cb5c0 msgr2=0x7f7c501f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.333 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c54c93700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c500cb5c0 0x7f7c501f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.333 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c54c93700 1 -- 172.21.15.135:0/4088301956 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c48009b30 con 0x7f7c500d0520 2023-08-27T11:11:44.333 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c4ffff700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500b65d0 0x7f7c500c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:44.333 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c54c93700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c500d0520 0x7f7c500c6440 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7c48015000 tx=0x7f7c48004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:44.333 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.323+0000 7f7c4cff9700 1 -- 172.21.15.135:0/4088301956 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48004880 con 0x7f7c500d0520 2023-08-27T11:11:44.334 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.324+0000 7f7c4cff9700 1 -- 172.21.15.135:0/4088301956 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c480099b0 con 0x7f7c500d0520 2023-08-27T11:11:44.334 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.324+0000 7f7c566f6700 1 -- 172.21.15.135:0/4088301956 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c500c5350 con 0x7f7c500d0520 2023-08-27T11:11:44.334 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.324+0000 7f7c4cff9700 1 -- 172.21.15.135:0/4088301956 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4801c680 con 0x7f7c500d0520 2023-08-27T11:11:44.334 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.324+0000 7f7c566f6700 1 -- 172.21.15.135:0/4088301956 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c500c8860 con 0x7f7c500d0520 2023-08-27T11:11:44.334 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.324+0000 7f7c4cff9700 1 -- 172.21.15.135:0/4088301956 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7c4801cae0 con 0x7f7c500d0520 2023-08-27T11:11:44.334 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.324+0000 7f7c4cff9700 1 -- 172.21.15.135:0/4088301956 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f7c48025b90 con 0x7f7c500d0520 2023-08-27T11:11:44.335 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.325+0000 7f7c566f6700 1 -- 172.21.15.135:0/4088301956 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c500734a0 con 0x7f7c500d0520 2023-08-27T11:11:44.335 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.327+0000 7f7c4cff9700 1 -- 172.21.15.135:0/4088301956 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f7c48043b00 con 0x7f7c500d0520 2023-08-27T11:11:44.383 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:44.382+0000 7f74d6019200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:11:44.413 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.410+0000 7f7c566f6700 1 -- 172.21.15.135:0/4088301956 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b98fcbad-8b7b-4487-bbc8-ba2548e82f19", "id": 2} v 0) v1 -- 0x7f7c500ce4f0 con 0x7f7c500d0520 2023-08-27T11:11:44.415 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.413+0000 7f7c4cff9700 1 -- 172.21.15.135:0/4088301956 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b98fcbad-8b7b-4487-bbc8-ba2548e82f19", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7c48032030 con 0x7f7c500d0520 2023-08-27T11:11:44.416 INFO:teuthology.orchestra.run.smithi135.stdout:2 2023-08-27T11:11:44.417 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.414+0000 7f7c566f6700 1 -- 172.21.15.135:0/4088301956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c500d0520 msgr2=0x7f7c500c6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.417 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.414+0000 7f7c566f6700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c500d0520 0x7f7c500c6440 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7c48015000 tx=0x7f7c48004ad0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.417 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.415+0000 7f7c566f6700 1 -- 172.21.15.135:0/4088301956 shutdown_connections 2023-08-27T11:11:44.417 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.415+0000 7f7c566f6700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7c500d0520 0x7f7c500c6440 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.418 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.415+0000 7f7c566f6700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c500b65d0 0x7f7c500c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.418 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.415+0000 7f7c566f6700 1 --2- 172.21.15.135:0/4088301956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c500cb5c0 0x7f7c501f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.418 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.415+0000 7f7c566f6700 1 -- 172.21.15.135:0/4088301956 >> 172.21.15.135:0/4088301956 conn(0x7f7c5006f960 msgr2=0x7f7c50153ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:44.418 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.415+0000 7f7c566f6700 1 -- 172.21.15.135:0/4088301956 shutdown_connections 2023-08-27T11:11:44.418 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.415+0000 7f7c566f6700 1 -- 172.21.15.135:0/4088301956 wait complete. 2023-08-27T11:11:44.430 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 279ba826-637d-47c4-ac4f-1b31f3185aae 3 2023-08-27T11:11:44.592 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f65548700 1 Processor -- start 2023-08-27T11:11:44.592 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f65548700 1 -- start start 2023-08-27T11:11:44.594 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f65548700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8f600d0520 0x7f8f600ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.595 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f65548700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f6015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.595 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f65548700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6015ec50 con 0x7f8f600b65d0 2023-08-27T11:11:44.595 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f65548700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6015c620 con 0x7f8f600d0520 2023-08-27T11:11:44.595 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f65548700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6015eea0 con 0x7f8f600cb5c0 2023-08-27T11:11:44.595 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5f7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f6015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.596 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5f7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f6015c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:59798/0 (socket says 172.21.15.135:59798) 2023-08-27T11:11:44.596 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5f7fe700 1 -- 172.21.15.135:0/1330168594 learned_addr learned my addr 172.21.15.135:0/1330168594 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:44.596 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5e7fc700 1 --2- 172.21.15.135:0/1330168594 >> v2:172.21.15.45:3300/0 conn(0x7f8f600d0520 0x7f8f600ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.596 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5f7fe700 1 -- 172.21.15.135:0/1330168594 >> v2:172.21.15.45:3300/0 conn(0x7f8f600d0520 msgr2=0x7f8f600ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:44.597 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5f7fe700 1 --2- 172.21.15.135:0/1330168594 >> v2:172.21.15.45:3300/0 conn(0x7f8f600d0520 0x7f8f600ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.597 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5f7fe700 1 -- 172.21.15.135:0/1330168594 >> v1:172.21.15.45:6789/0 conn(0x7f8f600b65d0 legacy=0x7f8f600cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.597 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5f7fe700 1 -- 172.21.15.135:0/1330168594 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6015f180 con 0x7f8f600cb5c0 2023-08-27T11:11:44.597 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5f7fe700 1 --2- 172.21.15.135:0/1330168594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f6015c0e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009e50 tx=0x7f8f5800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=967aca695514e4e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:44.597 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5dffb700 1 -- 172.21.15.135:0/1330168594 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800e070 con 0x7f8f600cb5c0 2023-08-27T11:11:44.598 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5dffb700 1 -- 172.21.15.135:0/1330168594 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800b910 con 0x7f8f600cb5c0 2023-08-27T11:11:44.598 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.591+0000 7f8f5dffb700 1 -- 172.21.15.135:0/1330168594 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800bcc0 con 0x7f8f600cb5c0 2023-08-27T11:11:44.598 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.592+0000 7f8f65548700 1 -- 172.21.15.135:0/1330168594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 msgr2=0x7f8f6015c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.598 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.592+0000 7f8f65548700 1 --2- 172.21.15.135:0/1330168594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f6015c0e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009e50 tx=0x7f8f5800b330 comp rx=0 tx=0).stop 2023-08-27T11:11:44.598 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.592+0000 7f8f65548700 1 -- 172.21.15.135:0/1330168594 shutdown_connections 2023-08-27T11:11:44.599 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.592+0000 7f8f65548700 1 --2- 172.21.15.135:0/1330168594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f6015c0e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.599 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.592+0000 7f8f65548700 1 --2- 172.21.15.135:0/1330168594 >> v2:172.21.15.45:3300/0 conn(0x7f8f600d0520 0x7f8f600ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.599 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.592+0000 7f8f65548700 1 -- 172.21.15.135:0/1330168594 >> 172.21.15.135:0/1330168594 conn(0x7f8f6006f960 msgr2=0x7f8f6006fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:44.599 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.592+0000 7f8f65548700 1 -- 172.21.15.135:0/1330168594 shutdown_connections 2023-08-27T11:11:44.599 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.592+0000 7f8f65548700 1 -- 172.21.15.135:0/1330168594 wait complete. 2023-08-27T11:11:44.601 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.593+0000 7f8f65548700 1 Processor -- start 2023-08-27T11:11:44.602 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.593+0000 7f8f65548700 1 -- start start 2023-08-27T11:11:44.602 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f65548700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600b65d0 0x7f8f6015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.602 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f65548700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f600c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.602 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f65548700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f600d0520 0x7f8f600c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.602 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f65548700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c83b0 con 0x7f8f600d0520 2023-08-27T11:11:44.603 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f65548700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c8690 con 0x7f8f600b65d0 2023-08-27T11:11:44.603 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f65548700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c8970 con 0x7f8f600cb5c0 2023-08-27T11:11:44.603 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5e7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f600c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.603 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5effd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600b65d0 0x7f8f6015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.603 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5e7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f600c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:59804/0 (socket says 172.21.15.135:59804) 2023-08-27T11:11:44.604 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5e7fc700 1 -- 172.21.15.135:0/3241524375 learned_addr learned my addr 172.21.15.135:0/3241524375 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:44.604 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5effd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600b65d0 0x7f8f6015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47944/0 (socket says 172.21.15.135:47944) 2023-08-27T11:11:44.604 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5f7fe700 1 --2- 172.21.15.135:0/3241524375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f600d0520 0x7f8f600c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.604 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5e7fc700 1 -- 172.21.15.135:0/3241524375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600b65d0 msgr2=0x7f8f6015a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.604 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5e7fc700 1 --2- 172.21.15.135:0/3241524375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600b65d0 0x7f8f6015a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.605 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5e7fc700 1 -- 172.21.15.135:0/3241524375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f600d0520 msgr2=0x7f8f600c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.605 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5e7fc700 1 --2- 172.21.15.135:0/3241524375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f600d0520 0x7f8f600c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.605 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5e7fc700 1 -- 172.21.15.135:0/3241524375 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009b30 con 0x7f8f600cb5c0 2023-08-27T11:11:44.605 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5effd700 1 --2- 172.21.15.135:0/3241524375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600b65d0 0x7f8f6015a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:44.606 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.594+0000 7f8f5e7fc700 1 --2- 172.21.15.135:0/3241524375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f600c9450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400e840 tx=0x7f8f5400eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:44.606 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.595+0000 7f8f477fe700 1 -- 172.21.15.135:0/3241524375 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54009590 con 0x7f8f600cb5c0 2023-08-27T11:11:44.606 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.595+0000 7f8f477fe700 1 -- 172.21.15.135:0/3241524375 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f54013680 con 0x7f8f600cb5c0 2023-08-27T11:11:44.606 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.595+0000 7f8f65548700 1 -- 172.21.15.135:0/3241524375 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f600c8c50 con 0x7f8f600cb5c0 2023-08-27T11:11:44.606 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.595+0000 7f8f477fe700 1 -- 172.21.15.135:0/3241524375 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54013a30 con 0x7f8f600cb5c0 2023-08-27T11:11:44.607 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.595+0000 7f8f65548700 1 -- 172.21.15.135:0/3241524375 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6015a850 con 0x7f8f600cb5c0 2023-08-27T11:11:44.607 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.595+0000 7f8f477fe700 1 -- 172.21.15.135:0/3241524375 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8f54005a20 con 0x7f8f600cb5c0 2023-08-27T11:11:44.607 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.595+0000 7f8f477fe700 1 -- 172.21.15.135:0/3241524375 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f8f54006700 con 0x7f8f600cb5c0 2023-08-27T11:11:44.607 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.596+0000 7f8f65548700 1 -- 172.21.15.135:0/3241524375 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6015aa70 con 0x7f8f600cb5c0 2023-08-27T11:11:44.607 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.599+0000 7f8f477fe700 1 -- 172.21.15.135:0/3241524375 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8f54021060 con 0x7f8f600cb5c0 2023-08-27T11:11:44.683 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.681+0000 7f8f65548700 1 -- 172.21.15.135:0/3241524375 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd new", "uuid": "279ba826-637d-47c4-ac4f-1b31f3185aae", "id": 3} v 0) v1 -- 0x7f8f6015f6f0 con 0x7f8f600cb5c0 2023-08-27T11:11:44.686 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.684+0000 7f8f477fe700 1 -- 172.21.15.135:0/3241524375 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "279ba826-637d-47c4-ac4f-1b31f3185aae", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8f54017070 con 0x7f8f600cb5c0 2023-08-27T11:11:44.687 INFO:teuthology.orchestra.run.smithi135.stdout:3 2023-08-27T11:11:44.688 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.685+0000 7f8f65548700 1 -- 172.21.15.135:0/3241524375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 msgr2=0x7f8f600c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.688 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.685+0000 7f8f65548700 1 --2- 172.21.15.135:0/3241524375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f600c9450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400e840 tx=0x7f8f5400eb50 comp rx=0 tx=0).stop 2023-08-27T11:11:44.688 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.685+0000 7f8f65548700 1 -- 172.21.15.135:0/3241524375 shutdown_connections 2023-08-27T11:11:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.685+0000 7f8f65548700 1 --2- 172.21.15.135:0/3241524375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f600d0520 0x7f8f600c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.685+0000 7f8f65548700 1 --2- 172.21.15.135:0/3241524375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600b65d0 0x7f8f6015a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.685+0000 7f8f65548700 1 --2- 172.21.15.135:0/3241524375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cb5c0 0x7f8f600c9450 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.685+0000 7f8f65548700 1 -- 172.21.15.135:0/3241524375 >> 172.21.15.135:0/3241524375 conn(0x7f8f6006f960 msgr2=0x7f8f600b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:44.690 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.685+0000 7f8f65548700 1 -- 172.21.15.135:0/3241524375 shutdown_connections 2023-08-27T11:11:44.690 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.685+0000 7f8f65548700 1 -- 172.21.15.135:0/3241524375 wait complete. 2023-08-27T11:11:44.699 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 23ee4406-9cb4-4559-acc4-d2f25a487f37 4 2023-08-27T11:11:44.763 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:44.761+0000 7f22b76ca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:11:44.799 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:44.797+0000 7f74d6019200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:11:44.823 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.820+0000 7fd3304b6700 1 Processor -- start 2023-08-27T11:11:44.823 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.820+0000 7fd3304b6700 1 -- start start 2023-08-27T11:11:44.826 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.821+0000 7fd3304b6700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd32806cf40 0x7fd32806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.827 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.821+0000 7fd3304b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd32815b270 0x7fd32806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.827 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.821+0000 7fd3304b6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd328071e80 0x7fd32815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.827 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.821+0000 7fd3304b6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3280ca650 con 0x7fd32815b270 2023-08-27T11:11:44.827 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.821+0000 7fd3304b6700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32806f8b0 con 0x7fd32806cf40 2023-08-27T11:11:44.827 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.821+0000 7fd3304b6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32806d860 con 0x7fd328071e80 2023-08-27T11:11:44.828 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.821+0000 7fd32ea53700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd328071e80 0x7fd32815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.828 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.821+0000 7fd32e252700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd32806cf40 0x7fd32806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.828 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32ea53700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd328071e80 0x7fd32815ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:59810/0 (socket says 172.21.15.135:59810) 2023-08-27T11:11:44.828 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32e252700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd32806cf40 0x7fd32806ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47950/0 (socket says 172.21.15.135:47950) 2023-08-27T11:11:44.828 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32ea53700 1 -- 172.21.15.135:0/3000532129 learned_addr learned my addr 172.21.15.135:0/3000532129 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:44.829 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32da51700 1 --2- 172.21.15.135:0/3000532129 >> v2:172.21.15.45:3300/0 conn(0x7fd32815b270 0x7fd32806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.829 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32e252700 1 -- 172.21.15.135:0/3000532129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd328071e80 msgr2=0x7fd32815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:44.829 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32e252700 1 --2- 172.21.15.135:0/3000532129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd328071e80 0x7fd32815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.829 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32e252700 1 -- 172.21.15.135:0/3000532129 >> v2:172.21.15.45:3300/0 conn(0x7fd32815b270 msgr2=0x7fd32806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:44.833 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32e252700 1 --2- 172.21.15.135:0/3000532129 >> v2:172.21.15.45:3300/0 conn(0x7fd32815b270 0x7fd32806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.833 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32e252700 1 -- 172.21.15.135:0/3000532129 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32806f010 con 0x7fd32806cf40 2023-08-27T11:11:44.833 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32ea53700 1 --2- 172.21.15.135:0/3000532129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd328071e80 0x7fd32815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:44.833 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32e252700 1 --2- 172.21.15.135:0/3000532129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd32806cf40 0x7fd32806ac90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3280cfd30 tx=0x7fd324009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2cb0669734f2c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:44.834 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32d250700 1 -- 172.21.15.135:0/3000532129 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324011830 con 0x7fd32806cf40 2023-08-27T11:11:44.834 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32d250700 1 -- 172.21.15.135:0/3000532129 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd324011990 con 0x7fd32806cf40 2023-08-27T11:11:44.834 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.822+0000 7fd32d250700 1 -- 172.21.15.135:0/3000532129 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324011d20 con 0x7fd32806cf40 2023-08-27T11:11:44.834 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.823+0000 7fd3304b6700 1 -- 172.21.15.135:0/3000532129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd32806cf40 msgr2=0x7fd32806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.834 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.823+0000 7fd3304b6700 1 --2- 172.21.15.135:0/3000532129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd32806cf40 0x7fd32806ac90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3280cfd30 tx=0x7fd324009b20 comp rx=0 tx=0).stop 2023-08-27T11:11:44.835 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.823+0000 7fd3304b6700 1 -- 172.21.15.135:0/3000532129 shutdown_connections 2023-08-27T11:11:44.835 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.823+0000 7fd3304b6700 1 --2- 172.21.15.135:0/3000532129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd328071e80 0x7fd32815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.835 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.823+0000 7fd3304b6700 1 --2- 172.21.15.135:0/3000532129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd32806cf40 0x7fd32806ac90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.835 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.823+0000 7fd3304b6700 1 --2- 172.21.15.135:0/3000532129 >> v2:172.21.15.45:3300/0 conn(0x7fd32815b270 0x7fd32806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.835 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.823+0000 7fd3304b6700 1 -- 172.21.15.135:0/3000532129 >> 172.21.15.135:0/3000532129 conn(0x7fd3280cc7e0 msgr2=0x7fd3280ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:44.836 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.823+0000 7fd3304b6700 1 -- 172.21.15.135:0/3000532129 shutdown_connections 2023-08-27T11:11:44.836 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.823+0000 7fd3304b6700 1 -- 172.21.15.135:0/3000532129 wait complete. 2023-08-27T11:11:44.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.824+0000 7fd3304b6700 1 Processor -- start 2023-08-27T11:11:44.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.824+0000 7fd3304b6700 1 -- start start 2023-08-27T11:11:44.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd3304b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32806cf40 0x7fd3280b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd3304b6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd328071e80 0x7fd328155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd3304b6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd32815b270 0x7fd32814a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:44.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd3304b6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280af3b0 con 0x7fd32806cf40 2023-08-27T11:11:44.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd3304b6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd328156450 con 0x7fd328071e80 2023-08-27T11:11:44.840 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd3304b6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280b4bc0 con 0x7fd32815b270 2023-08-27T11:11:44.840 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32da51700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd328071e80 0x7fd328155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.840 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32da51700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd328071e80 0x7fd328155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47958/0 (socket says 172.21.15.135:47958) 2023-08-27T11:11:44.840 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32da51700 1 -- 172.21.15.135:0/3293253751 learned_addr learned my addr 172.21.15.135:0/3293253751 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:44.840 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32ea53700 1 --2- 172.21.15.135:0/3293253751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd32815b270 0x7fd32814a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.841 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32e252700 1 --2- 172.21.15.135:0/3293253751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32806cf40 0x7fd3280b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:44.841 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32ea53700 1 -- 172.21.15.135:0/3293253751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd328071e80 msgr2=0x7fd328155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.841 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32ea53700 1 --2- 172.21.15.135:0/3293253751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd328071e80 0x7fd328155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.841 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32ea53700 1 -- 172.21.15.135:0/3293253751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32806cf40 msgr2=0x7fd3280b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.841 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32ea53700 1 --2- 172.21.15.135:0/3293253751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32806cf40 0x7fd3280b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.842 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.825+0000 7fd32ea53700 1 -- 172.21.15.135:0/3293253751 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320009b30 con 0x7fd32815b270 2023-08-27T11:11:44.842 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.826+0000 7fd32ea53700 1 --2- 172.21.15.135:0/3293253751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd32815b270 0x7fd32814a280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd32000eed0 tx=0x7fd32000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:44.842 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.826+0000 7fd31effd700 1 -- 172.21.15.135:0/3293253751 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320018070 con 0x7fd32815b270 2023-08-27T11:11:44.842 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.826+0000 7fd31effd700 1 -- 172.21.15.135:0/3293253751 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd320003a20 con 0x7fd32815b270 2023-08-27T11:11:44.842 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.826+0000 7fd3304b6700 1 -- 172.21.15.135:0/3293253751 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3240094a0 con 0x7fd32815b270 2023-08-27T11:11:44.842 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.826+0000 7fd31effd700 1 -- 172.21.15.135:0/3293253751 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320010040 con 0x7fd32815b270 2023-08-27T11:11:44.843 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.826+0000 7fd3304b6700 1 -- 172.21.15.135:0/3293253751 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3280aefd0 con 0x7fd32815b270 2023-08-27T11:11:44.843 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.826+0000 7fd31effd700 1 -- 172.21.15.135:0/3293253751 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd32000bb70 con 0x7fd32815b270 2023-08-27T11:11:44.843 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.826+0000 7fd31effd700 1 -- 172.21.15.135:0/3293253751 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fd32000be20 con 0x7fd32815b270 2023-08-27T11:11:44.843 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.827+0000 7fd3304b6700 1 -- 172.21.15.135:0/3293253751 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32806dca0 con 0x7fd32815b270 2023-08-27T11:11:44.843 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.829+0000 7fd31effd700 1 -- 172.21.15.135:0/3293253751 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fd32001d410 con 0x7fd32815b270 2023-08-27T11:11:44.915 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.914+0000 7fd3304b6700 1 -- 172.21.15.135:0/3293253751 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd new", "uuid": "23ee4406-9cb4-4559-acc4-d2f25a487f37", "id": 4} v 0) v1 -- 0x7fd3280726d0 con 0x7fd32815b270 2023-08-27T11:11:44.918 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.917+0000 7fd31effd700 1 -- 172.21.15.135:0/3293253751 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "23ee4406-9cb4-4559-acc4-d2f25a487f37", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd32002c0b0 con 0x7fd32815b270 2023-08-27T11:11:44.919 INFO:teuthology.orchestra.run.smithi135.stdout:4 2023-08-27T11:11:44.920 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.918+0000 7fd3304b6700 1 -- 172.21.15.135:0/3293253751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd32815b270 msgr2=0x7fd32814a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:44.920 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.918+0000 7fd3304b6700 1 --2- 172.21.15.135:0/3293253751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd32815b270 0x7fd32814a280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd32000eed0 tx=0x7fd32000b390 comp rx=0 tx=0).stop 2023-08-27T11:11:44.920 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.918+0000 7fd3304b6700 1 -- 172.21.15.135:0/3293253751 shutdown_connections 2023-08-27T11:11:44.920 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.918+0000 7fd3304b6700 1 --2- 172.21.15.135:0/3293253751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32806cf40 0x7fd3280b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.920 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.918+0000 7fd3304b6700 1 --2- 172.21.15.135:0/3293253751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd32815b270 0x7fd32814a280 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.921 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.918+0000 7fd3304b6700 1 --2- 172.21.15.135:0/3293253751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd328071e80 0x7fd328155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:44.921 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.918+0000 7fd3304b6700 1 -- 172.21.15.135:0/3293253751 >> 172.21.15.135:0/3293253751 conn(0x7fd3280cc7e0 msgr2=0x7fd3280c73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:44.921 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.918+0000 7fd3304b6700 1 -- 172.21.15.135:0/3293253751 shutdown_connections 2023-08-27T11:11:44.921 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:44.918+0000 7fd3304b6700 1 -- 172.21.15.135:0/3293253751 wait complete. 2023-08-27T11:11:44.922 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:44.919+0000 7f74d6019200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:11:44.931 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 996b9bc1-3eab-4765-b323-f2fe8edbd353 5 2023-08-27T11:11:45.008 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:45.006+0000 7f74d6019200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:11:45.058 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.051+0000 7ff505910700 1 Processor -- start 2023-08-27T11:11:45.058 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.051+0000 7ff505910700 1 -- start start 2023-08-27T11:11:45.059 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff505910700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000b8c20 0x7ff5000cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.059 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff505910700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5000ce2e0 0x7ff5000b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.059 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff505910700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50015d6a0 con 0x7ff50015d2c0 2023-08-27T11:11:45.059 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff505910700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5000ce770 con 0x7ff5000ce2e0 2023-08-27T11:11:45.059 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff505910700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000d2c90 con 0x7ff5000b8c20 2023-08-27T11:11:45.060 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff4fe7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000b8c20 0x7ff5000cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.060 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff4fe7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000b8c20 0x7ff5000cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:59822/0 (socket says 172.21.15.135:59822) 2023-08-27T11:11:45.060 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff4fe7fc700 1 -- 172.21.15.135:0/3876545917 learned_addr learned my addr 172.21.15.135:0/3876545917 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:45.060 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff4ff7fe700 1 --2- 172.21.15.135:0/3876545917 >> v2:172.21.15.45:3300/0 conn(0x7ff5000ce2e0 0x7ff5000b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.060 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff4fe7fc700 1 -- 172.21.15.135:0/3876545917 >> v2:172.21.15.45:3300/0 conn(0x7ff5000ce2e0 msgr2=0x7ff5000b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:45.061 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff4fe7fc700 1 --2- 172.21.15.135:0/3876545917 >> v2:172.21.15.45:3300/0 conn(0x7ff5000ce2e0 0x7ff5000b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.061 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff4fe7fc700 1 -- 172.21.15.135:0/3876545917 >> v1:172.21.15.45:6789/0 conn(0x7ff50015d2c0 legacy=0x7ff5000d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.061 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff4fe7fc700 1 -- 172.21.15.135:0/3876545917 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5000d1bf0 con 0x7ff5000b8c20 2023-08-27T11:11:45.061 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.052+0000 7ff4fe7fc700 1 --2- 172.21.15.135:0/3876545917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000b8c20 0x7ff5000cdda0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00098f0 tx=0x7ff4f0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a119074662249c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:45.061 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff4fdffb700 1 -- 172.21.15.135:0/3876545917 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0011830 con 0x7ff5000b8c20 2023-08-27T11:11:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff4fdffb700 1 -- 172.21.15.135:0/3876545917 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0011990 con 0x7ff5000b8c20 2023-08-27T11:11:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff4fdffb700 1 -- 172.21.15.135:0/3876545917 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0011d20 con 0x7ff5000b8c20 2023-08-27T11:11:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff505910700 1 -- 172.21.15.135:0/3876545917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000b8c20 msgr2=0x7ff5000cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff505910700 1 --2- 172.21.15.135:0/3876545917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000b8c20 0x7ff5000cdda0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00098f0 tx=0x7ff4f0009c00 comp rx=0 tx=0).stop 2023-08-27T11:11:45.062 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff505910700 1 -- 172.21.15.135:0/3876545917 shutdown_connections 2023-08-27T11:11:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff505910700 1 --2- 172.21.15.135:0/3876545917 >> v2:172.21.15.45:3300/0 conn(0x7ff5000ce2e0 0x7ff5000b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff505910700 1 --2- 172.21.15.135:0/3876545917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000b8c20 0x7ff5000cdda0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff505910700 1 -- 172.21.15.135:0/3876545917 >> 172.21.15.135:0/3876545917 conn(0x7ff5000d2fd0 msgr2=0x7ff5000d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff505910700 1 -- 172.21.15.135:0/3876545917 shutdown_connections 2023-08-27T11:11:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.053+0000 7ff505910700 1 -- 172.21.15.135:0/3876545917 wait complete. 2023-08-27T11:11:45.063 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.054+0000 7ff505910700 1 Processor -- start 2023-08-27T11:11:45.064 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff505910700 1 -- start start 2023-08-27T11:11:45.064 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff505910700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5000b8c20 0x7ff500151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.064 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff505910700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000ce2e0 0x7ff5001e5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.064 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff505910700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50015d2c0 0x7ff5001e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.064 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff505910700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff500151970 con 0x7ff5000b8c20 2023-08-27T11:11:45.065 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff505910700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5001e8900 con 0x7ff50015d2c0 2023-08-27T11:11:45.065 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff505910700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5001e8be0 con 0x7ff5000ce2e0 2023-08-27T11:11:45.065 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff4fe7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000ce2e0 0x7ff5001e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.065 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff4ff7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50015d2c0 0x7ff5001e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.065 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff4fe7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000ce2e0 0x7ff5001e5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:59826/0 (socket says 172.21.15.135:59826) 2023-08-27T11:11:45.065 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff4fe7fc700 1 -- 172.21.15.135:0/2798627592 learned_addr learned my addr 172.21.15.135:0/2798627592 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:45.066 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.055+0000 7ff4feffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5000b8c20 0x7ff500151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.066 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4fe7fc700 1 -- 172.21.15.135:0/2798627592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50015d2c0 msgr2=0x7ff5001e5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.066 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4fe7fc700 1 --2- 172.21.15.135:0/2798627592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50015d2c0 0x7ff5001e5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.066 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4fe7fc700 1 -- 172.21.15.135:0/2798627592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5000b8c20 msgr2=0x7ff500151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.067 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4fe7fc700 1 --2- 172.21.15.135:0/2798627592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5000b8c20 0x7ff500151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.067 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4fe7fc700 1 -- 172.21.15.135:0/2798627592 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f0009550 con 0x7ff5000ce2e0 2023-08-27T11:11:45.067 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4ff7fe700 1 --2- 172.21.15.135:0/2798627592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50015d2c0 0x7ff5001e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:45.067 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4feffd700 1 --2- 172.21.15.135:0/2798627592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5000b8c20 0x7ff500151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:45.067 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4fe7fc700 1 --2- 172.21.15.135:0/2798627592 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000ce2e0 0x7ff5001e5790 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0014040 tx=0x7ff4f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:45.068 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4e7fff700 1 -- 172.21.15.135:0/2798627592 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0004510 con 0x7ff5000ce2e0 2023-08-27T11:11:45.068 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4e7fff700 1 -- 172.21.15.135:0/2798627592 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f001e070 con 0x7ff5000ce2e0 2023-08-27T11:11:45.068 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4e7fff700 1 -- 172.21.15.135:0/2798627592 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00038e0 con 0x7ff5000ce2e0 2023-08-27T11:11:45.068 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff505910700 1 -- 172.21.15.135:0/2798627592 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5001e8ec0 con 0x7ff5000ce2e0 2023-08-27T11:11:45.068 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff505910700 1 -- 172.21.15.135:0/2798627592 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5001e9480 con 0x7ff5000ce2e0 2023-08-27T11:11:45.068 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4e7fff700 1 -- 172.21.15.135:0/2798627592 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff4f0003d40 con 0x7ff5000ce2e0 2023-08-27T11:11:45.069 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.056+0000 7ff4e7fff700 1 -- 172.21.15.135:0/2798627592 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7ff4f001a9a0 con 0x7ff5000ce2e0 2023-08-27T11:11:45.069 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.057+0000 7ff505910700 1 -- 172.21.15.135:0/2798627592 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50004e620 con 0x7ff5000ce2e0 2023-08-27T11:11:45.069 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.059+0000 7ff4e7fff700 1 -- 172.21.15.135:0/2798627592 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff4f0020070 con 0x7ff5000ce2e0 2023-08-27T11:11:45.092 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:45.091+0000 7f74d6019200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:11:45.143 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.142+0000 7ff505910700 1 -- 172.21.15.135:0/2798627592 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd new", "uuid": "996b9bc1-3eab-4765-b323-f2fe8edbd353", "id": 5} v 0) v1 -- 0x7ff5000c68a0 con 0x7ff5000ce2e0 2023-08-27T11:11:45.151 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.150+0000 7ff4e7fff700 1 -- 172.21.15.135:0/2798627592 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "996b9bc1-3eab-4765-b323-f2fe8edbd353", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff4f0015020 con 0x7ff5000ce2e0 2023-08-27T11:11:45.152 INFO:teuthology.orchestra.run.smithi135.stdout:5 2023-08-27T11:11:45.153 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.151+0000 7ff505910700 1 -- 172.21.15.135:0/2798627592 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000ce2e0 msgr2=0x7ff5001e5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.153 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.151+0000 7ff505910700 1 --2- 172.21.15.135:0/2798627592 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000ce2e0 0x7ff5001e5790 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0014040 tx=0x7ff4f0004750 comp rx=0 tx=0).stop 2023-08-27T11:11:45.153 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.151+0000 7ff505910700 1 -- 172.21.15.135:0/2798627592 shutdown_connections 2023-08-27T11:11:45.153 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.151+0000 7ff505910700 1 --2- 172.21.15.135:0/2798627592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5000b8c20 0x7ff500151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.154 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.151+0000 7ff505910700 1 --2- 172.21.15.135:0/2798627592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff50015d2c0 0x7ff5001e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.154 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.151+0000 7ff505910700 1 --2- 172.21.15.135:0/2798627592 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5000ce2e0 0x7ff5001e5790 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.154 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.151+0000 7ff505910700 1 -- 172.21.15.135:0/2798627592 >> 172.21.15.135:0/2798627592 conn(0x7ff5000d2fd0 msgr2=0x7ff5000b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:45.154 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.151+0000 7ff505910700 1 -- 172.21.15.135:0/2798627592 shutdown_connections 2023-08-27T11:11:45.155 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.152+0000 7ff505910700 1 -- 172.21.15.135:0/2798627592 wait complete. 2023-08-27T11:11:45.165 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new d9bd54f7-ae70-4fab-9b76-afbf0e040d17 6 2023-08-27T11:11:45.307 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.306+0000 7fa775613700 1 Processor -- start 2023-08-27T11:11:45.307 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.306+0000 7fa775613700 1 -- start start 2023-08-27T11:11:45.310 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa775613700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7700b2f60 0x7fa77006b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.310 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa775613700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa77006b890 0x7fa77006bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.310 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa775613700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7700ceda0 con 0x7fa77015a2c0 2023-08-27T11:11:45.310 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa775613700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7700cf360 con 0x7fa7700b2f60 2023-08-27T11:11:45.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa775613700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77015a6a0 con 0x7fa77006b890 2023-08-27T11:11:45.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76f7fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa77006b890 0x7fa77006bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76effd700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa77015a2c0 0x7fa770073800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:59218/0 (socket says 172.21.15.135:59218) 2023-08-27T11:11:45.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76f7fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa77006b890 0x7fa77006bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47978/0 (socket says 172.21.15.135:47978) 2023-08-27T11:11:45.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76effd700 1 -- 172.21.15.135:0/1315225825 learned_addr learned my addr 172.21.15.135:0/1315225825 (peer_addr_for_me v1:172.21.15.135:0/0) 2023-08-27T11:11:45.312 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76e7fc700 1 --2- 172.21.15.135:0/1315225825 >> v2:172.21.15.45:3300/0 conn(0x7fa7700b2f60 0x7fa77006b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.312 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76f7fe700 1 -- 172.21.15.135:0/1315225825 >> v2:172.21.15.45:3300/0 conn(0x7fa7700b2f60 msgr2=0x7fa77006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:45.312 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76f7fe700 1 --2- 172.21.15.135:0/1315225825 >> v2:172.21.15.45:3300/0 conn(0x7fa7700b2f60 0x7fa77006b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.312 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76f7fe700 1 -- 172.21.15.135:0/1315225825 >> v1:172.21.15.135:6789/0 conn(0x7fa77015a2c0 legacy=0x7fa770073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76f7fe700 1 -- 172.21.15.135:0/1315225825 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700cf080 con 0x7fa77006b890 2023-08-27T11:11:45.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76dffb700 1 -- 172.21.15.135:0/1315225825 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390420062 0 0) 0x7fa7700ceda0 con 0x7fa77015a2c0 2023-08-27T11:11:45.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76f7fe700 1 --2- 172.21.15.135:0/1315225825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77006b890 0x7fa77006bec0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa7700aef00 tx=0x7fa76800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=145e3d029c7640c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:45.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76dffb700 1 -- 172.21.15.135:0/1315225825 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800e070 con 0x7fa77006b890 2023-08-27T11:11:45.313 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76dffb700 1 -- 172.21.15.135:0/1315225825 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76800bb20 con 0x7fa77006b890 2023-08-27T11:11:45.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.307+0000 7fa76dffb700 1 -- 172.21.15.135:0/1315225825 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768012430 con 0x7fa77006b890 2023-08-27T11:11:45.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.308+0000 7fa775613700 1 -- 172.21.15.135:0/1315225825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77006b890 msgr2=0x7fa77006bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.308+0000 7fa775613700 1 --2- 172.21.15.135:0/1315225825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77006b890 0x7fa77006bec0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa7700aef00 tx=0x7fa76800b330 comp rx=0 tx=0).stop 2023-08-27T11:11:45.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.308+0000 7fa775613700 1 -- 172.21.15.135:0/1315225825 shutdown_connections 2023-08-27T11:11:45.314 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.308+0000 7fa775613700 1 --2- 172.21.15.135:0/1315225825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77006b890 0x7fa77006bec0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.315 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.308+0000 7fa775613700 1 --2- 172.21.15.135:0/1315225825 >> v2:172.21.15.45:3300/0 conn(0x7fa7700b2f60 0x7fa77006b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.315 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.308+0000 7fa775613700 1 -- 172.21.15.135:0/1315225825 >> 172.21.15.135:0/1315225825 conn(0x7fa7700b2440 msgr2=0x7fa7700b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:45.315 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.308+0000 7fa775613700 1 -- 172.21.15.135:0/1315225825 shutdown_connections 2023-08-27T11:11:45.315 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.308+0000 7fa775613700 1 -- 172.21.15.135:0/1315225825 wait complete. 2023-08-27T11:11:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.309+0000 7fa775613700 1 Processor -- start 2023-08-27T11:11:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.309+0000 7fa775613700 1 -- start start 2023-08-27T11:11:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa775613700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77006b890 0x7fa7701f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa775613700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7700b2f60 0x7fa7700cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.318 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa775613700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77015a2c0 0x7fa7700cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa775613700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700ad480 con 0x7fa77006b890 2023-08-27T11:11:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa775613700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700ad720 con 0x7fa77015a2c0 2023-08-27T11:11:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa775613700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700ada00 con 0x7fa7700b2f60 2023-08-27T11:11:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa76e7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7700b2f60 0x7fa7700cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.319 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa76f7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77015a2c0 0x7fa7700cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.320 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa76e7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7700b2f60 0x7fa7700cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.135:59838/0 (socket says 172.21.15.135:59838) 2023-08-27T11:11:45.320 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa76e7fc700 1 -- 172.21.15.135:0/4180495352 learned_addr learned my addr 172.21.15.135:0/4180495352 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:45.320 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.310+0000 7fa76effd700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77006b890 0x7fa7701f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.320 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa76f7fe700 1 -- 172.21.15.135:0/4180495352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7700b2f60 msgr2=0x7fa7700cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.320 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa76f7fe700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7700b2f60 0x7fa7700cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.321 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa76f7fe700 1 -- 172.21.15.135:0/4180495352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77006b890 msgr2=0x7fa7701f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.321 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa76f7fe700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77006b890 0x7fa7701f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.321 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa76f7fe700 1 -- 172.21.15.135:0/4180495352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009a70 con 0x7fa77015a2c0 2023-08-27T11:11:45.321 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa76effd700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77006b890 0x7fa7701f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa76e7fc700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7700b2f60 0x7fa7700cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:11:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa76f7fe700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77015a2c0 0x7fa7700cf9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa768016000 tx=0x7fa768004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa7577fe700 1 -- 172.21.15.135:0/4180495352 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768012430 con 0x7fa77015a2c0 2023-08-27T11:11:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa7577fe700 1 -- 172.21.15.135:0/4180495352 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768020070 con 0x7fa77015a2c0 2023-08-27T11:11:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa7577fe700 1 -- 172.21.15.135:0/4180495352 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768004280 con 0x7fa77015a2c0 2023-08-27T11:11:45.322 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa775613700 1 -- 172.21.15.135:0/4180495352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7700adce0 con 0x7fa77015a2c0 2023-08-27T11:11:45.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa775613700 1 -- 172.21.15.135:0/4180495352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7700ae4e0 con 0x7fa77015a2c0 2023-08-27T11:11:45.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa7577fe700 1 -- 172.21.15.135:0/4180495352 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa7680046e0 con 0x7fa77015a2c0 2023-08-27T11:11:45.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.311+0000 7fa7577fe700 1 -- 172.21.15.135:0/4180495352 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fa768022070 con 0x7fa77015a2c0 2023-08-27T11:11:45.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.312+0000 7fa775613700 1 -- 172.21.15.135:0/4180495352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7700adb40 con 0x7fa77015a2c0 2023-08-27T11:11:45.323 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.314+0000 7fa7577fe700 1 -- 172.21.15.135:0/4180495352 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa7700adb40 con 0x7fa77015a2c0 2023-08-27T11:11:45.400 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.399+0000 7fa775613700 1 -- 172.21.15.135:0/4180495352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d9bd54f7-ae70-4fab-9b76-afbf0e040d17", "id": 6} v 0) v1 -- 0x7fa7700ae200 con 0x7fa77015a2c0 2023-08-27T11:11:45.408 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.407+0000 7fa7577fe700 1 -- 172.21.15.135:0/4180495352 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d9bd54f7-ae70-4fab-9b76-afbf0e040d17", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa7700ae200 con 0x7fa77015a2c0 2023-08-27T11:11:45.408 INFO:teuthology.orchestra.run.smithi135.stdout:6 2023-08-27T11:11:45.410 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.408+0000 7fa775613700 1 -- 172.21.15.135:0/4180495352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77015a2c0 msgr2=0x7fa7700cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.410 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.408+0000 7fa775613700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77015a2c0 0x7fa7700cf9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa768016000 tx=0x7fa768004ad0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.410 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.408+0000 7fa775613700 1 -- 172.21.15.135:0/4180495352 shutdown_connections 2023-08-27T11:11:45.410 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.408+0000 7fa775613700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77006b890 0x7fa7701f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.410 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.408+0000 7fa775613700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa77015a2c0 0x7fa7700cf9e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.411 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.408+0000 7fa775613700 1 --2- 172.21.15.135:0/4180495352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7700b2f60 0x7fa7700cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.411 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.408+0000 7fa775613700 1 -- 172.21.15.135:0/4180495352 >> 172.21.15.135:0/4180495352 conn(0x7fa7700b2440 msgr2=0x7fa7700c9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:45.411 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.408+0000 7fa775613700 1 -- 172.21.15.135:0/4180495352 shutdown_connections 2023-08-27T11:11:45.411 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.408+0000 7fa775613700 1 -- 172.21.15.135:0/4180495352 wait complete. 2023-08-27T11:11:45.421 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new d1119d04-f9a6-40e1-a79b-bab5eb998f2b 7 2023-08-27T11:11:45.448 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:45.447+0000 7f22b76ca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:11:45.520 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:45.519+0000 7f22b76ca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:11:45.536 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:45.535+0000 7f74d6019200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:11:45.543 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.542+0000 7f8c9a905700 1 Processor -- start 2023-08-27T11:11:45.543 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.542+0000 7f8c9a905700 1 -- start start 2023-08-27T11:11:45.546 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c9a905700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c940caa00 0x7f8c940cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.546 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c9a905700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c940ce340 con 0x7f8c940ca620 2023-08-27T11:11:45.547 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c9a905700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c940ceec0 con 0x7f8c940af200 2023-08-27T11:11:45.547 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c9a905700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9415d7c0 con 0x7f8c940caa00 2023-08-27T11:11:45.547 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c937fe700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8c940af200 0x7f8c940acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:59232/0 (socket says 172.21.15.135:59232) 2023-08-27T11:11:45.547 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c937fe700 1 -- 172.21.15.135:0/1032475569 learned_addr learned my addr 172.21.15.135:0/1032475569 (peer_addr_for_me v1:172.21.15.135:0/0) 2023-08-27T11:11:45.547 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c98ea2700 1 --2- 172.21.15.135:0/1032475569 >> v2:172.21.15.135:3300/0 conn(0x7f8c940caa00 0x7f8c940cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.548 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c92ffd700 1 -- 172.21.15.135:0/1032475569 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217239667 0 0) 0x7f8c940ceec0 con 0x7f8c940af200 2023-08-27T11:11:45.548 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c92ffd700 1 -- 172.21.15.135:0/1032475569 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c7c0035b0 con 0x7f8c940af200 2023-08-27T11:11:45.548 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c98ea2700 1 -- 172.21.15.135:0/1032475569 >> v1:172.21.15.135:6789/0 conn(0x7f8c940af200 legacy=0x7f8c940acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.548 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c92ffd700 1 -- 172.21.15.135:0/1032475569 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277847443 0 0) 0x7f8c940ce340 con 0x7f8c940ca620 2023-08-27T11:11:45.549 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c98ea2700 1 -- 172.21.15.135:0/1032475569 >> v1:172.21.15.45:6789/0 conn(0x7f8c940ca620 legacy=0x7f8c940ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.549 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c98ea2700 1 -- 172.21.15.135:0/1032475569 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940cc570 con 0x7f8c940caa00 2023-08-27T11:11:45.549 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.543+0000 7f8c98ea2700 1 --2- 172.21.15.135:0/1032475569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940caa00 0x7f8c940cf350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009d90 tx=0x7f8c8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bada7045ca9b1317 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:45.549 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c92ffd700 1 -- 172.21.15.135:0/1032475569 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00e070 con 0x7f8c940caa00 2023-08-27T11:11:45.550 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c92ffd700 1 -- 172.21.15.135:0/1032475569 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00bb20 con 0x7f8c940caa00 2023-08-27T11:11:45.550 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c92ffd700 1 -- 172.21.15.135:0/1032475569 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00be90 con 0x7f8c940caa00 2023-08-27T11:11:45.550 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c9a905700 1 -- 172.21.15.135:0/1032475569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940caa00 msgr2=0x7f8c940cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.550 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c9a905700 1 --2- 172.21.15.135:0/1032475569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940caa00 0x7f8c940cf350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009d90 tx=0x7f8c8c00b330 comp rx=0 tx=0).stop 2023-08-27T11:11:45.550 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c9a905700 1 -- 172.21.15.135:0/1032475569 shutdown_connections 2023-08-27T11:11:45.551 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c9a905700 1 --2- 172.21.15.135:0/1032475569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940caa00 0x7f8c940cf350 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.551 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c9a905700 1 -- 172.21.15.135:0/1032475569 >> 172.21.15.135:0/1032475569 conn(0x7f8c940cc7e0 msgr2=0x7f8c940ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:45.551 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c9a905700 1 -- 172.21.15.135:0/1032475569 shutdown_connections 2023-08-27T11:11:45.551 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.544+0000 7f8c9a905700 1 -- 172.21.15.135:0/1032475569 wait complete. 2023-08-27T11:11:45.554 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.546+0000 7f8c9a905700 1 Processor -- start 2023-08-27T11:11:45.554 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.546+0000 7f8c9a905700 1 -- start start 2023-08-27T11:11:45.555 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c9a905700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c940af200 0x7f8c940c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.555 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c9a905700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940ca620 0x7f8c940b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.555 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c9a905700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c940caa00 0x7f8c94148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.556 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c9a905700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c83c0 con 0x7f8c940af200 2023-08-27T11:11:45.556 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c9a905700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940b3fb0 con 0x7f8c940ca620 2023-08-27T11:11:45.556 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c9a905700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c94148940 con 0x7f8c940caa00 2023-08-27T11:11:45.556 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940ca620 0x7f8c940b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.557 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940ca620 0x7f8c940b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:47994/0 (socket says 172.21.15.135:47994) 2023-08-27T11:11:45.557 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c937fe700 1 -- 172.21.15.135:0/991138050 learned_addr learned my addr 172.21.15.135:0/991138050 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-27T11:11:45.557 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c98ea2700 1 --2- 172.21.15.135:0/991138050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c940caa00 0x7f8c94148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.557 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c93fff700 1 --2- 172.21.15.135:0/991138050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c940af200 0x7f8c940c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.558 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c937fe700 1 -- 172.21.15.135:0/991138050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c940caa00 msgr2=0x7f8c94148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.558 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c937fe700 1 --2- 172.21.15.135:0/991138050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c940caa00 0x7f8c94148400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.558 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c937fe700 1 -- 172.21.15.135:0/991138050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c940af200 msgr2=0x7f8c940c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.558 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c937fe700 1 --2- 172.21.15.135:0/991138050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c940af200 0x7f8c940c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.558 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c937fe700 1 -- 172.21.15.135:0/991138050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c009a70 con 0x7f8c940ca620 2023-08-27T11:11:45.559 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.547+0000 7f8c937fe700 1 --2- 172.21.15.135:0/991138050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940ca620 0x7f8c940b3a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c88001eb0 tx=0x7f8c8800c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:45.559 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.548+0000 7f8c90ff9700 1 -- 172.21.15.135:0/991138050 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88005d70 con 0x7f8c940ca620 2023-08-27T11:11:45.559 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.548+0000 7f8c90ff9700 1 -- 172.21.15.135:0/991138050 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c880070d0 con 0x7f8c940ca620 2023-08-27T11:11:45.559 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.548+0000 7f8c9a905700 1 -- 172.21.15.135:0/991138050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c940c85b0 con 0x7f8c940ca620 2023-08-27T11:11:45.560 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.548+0000 7f8c90ff9700 1 -- 172.21.15.135:0/991138050 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88007480 con 0x7f8c940ca620 2023-08-27T11:11:45.560 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.548+0000 7f8c9a905700 1 -- 172.21.15.135:0/991138050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c940c8b70 con 0x7f8c940ca620 2023-08-27T11:11:45.560 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.548+0000 7f8c90ff9700 1 -- 172.21.15.135:0/991138050 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8c8801c440 con 0x7f8c940ca620 2023-08-27T11:11:45.560 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.548+0000 7f8c90ff9700 1 -- 172.21.15.135:0/991138050 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f8c88012a60 con 0x7f8c940ca620 2023-08-27T11:11:45.560 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.549+0000 7f8c9a905700 1 -- 172.21.15.135:0/991138050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c940676b0 con 0x7f8c940ca620 2023-08-27T11:11:45.561 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.551+0000 7f8c90ff9700 1 -- 172.21.15.135:0/991138050 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8c940676b0 con 0x7f8c940ca620 2023-08-27T11:11:45.635 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.634+0000 7f8c9a905700 1 -- 172.21.15.135:0/991138050 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d1119d04-f9a6-40e1-a79b-bab5eb998f2b", "id": 7} v 0) v1 -- 0x7f8c940b16b0 con 0x7f8c940ca620 2023-08-27T11:11:45.643 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.642+0000 7f8c90ff9700 1 -- 172.21.15.135:0/991138050 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1119d04-f9a6-40e1-a79b-bab5eb998f2b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8c88017070 con 0x7f8c940ca620 2023-08-27T11:11:45.643 INFO:teuthology.orchestra.run.smithi135.stdout:7 2023-08-27T11:11:45.644 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.643+0000 7f8c9a905700 1 -- 172.21.15.135:0/991138050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940ca620 msgr2=0x7f8c940b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.645 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.643+0000 7f8c9a905700 1 --2- 172.21.15.135:0/991138050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940ca620 0x7f8c940b3a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c88001eb0 tx=0x7f8c8800c7b0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.645 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.643+0000 7f8c9a905700 1 -- 172.21.15.135:0/991138050 shutdown_connections 2023-08-27T11:11:45.645 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.643+0000 7f8c9a905700 1 --2- 172.21.15.135:0/991138050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c940af200 0x7f8c940c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.645 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.643+0000 7f8c9a905700 1 --2- 172.21.15.135:0/991138050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c940caa00 0x7f8c94148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.646 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.643+0000 7f8c9a905700 1 --2- 172.21.15.135:0/991138050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c940ca620 0x7f8c940b3a70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.646 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.643+0000 7f8c9a905700 1 -- 172.21.15.135:0/991138050 >> 172.21.15.135:0/991138050 conn(0x7f8c940cc7e0 msgr2=0x7f8c940ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:45.648 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.643+0000 7f8c9a905700 1 -- 172.21.15.135:0/991138050 shutdown_connections 2023-08-27T11:11:45.648 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-27T11:11:45.643+0000 7f8c9a905700 1 -- 172.21.15.135:0/991138050 wait complete. 2023-08-27T11:11:45.656 INFO:tasks.ceph.osd.0:Restarting daemon 2023-08-27T11:11:45.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-08-27T11:11:45.659 INFO:tasks.ceph.osd.0:Started 2023-08-27T11:11:45.659 INFO:tasks.ceph.osd.1:Restarting daemon 2023-08-27T11:11:45.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-08-27T11:11:45.661 INFO:tasks.ceph.osd.1:Started 2023-08-27T11:11:45.662 INFO:tasks.ceph.osd.2:Restarting daemon 2023-08-27T11:11:45.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-08-27T11:11:45.664 INFO:tasks.ceph.osd.2:Started 2023-08-27T11:11:45.665 INFO:tasks.ceph.osd.3:Restarting daemon 2023-08-27T11:11:45.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-08-27T11:11:45.667 INFO:tasks.ceph.osd.3:Started 2023-08-27T11:11:45.667 INFO:tasks.ceph.osd.4:Restarting daemon 2023-08-27T11:11:45.668 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-08-27T11:11:45.698 INFO:tasks.ceph.osd.4:Started 2023-08-27T11:11:45.698 INFO:tasks.ceph.osd.5:Restarting daemon 2023-08-27T11:11:45.699 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-08-27T11:11:45.701 INFO:tasks.ceph.osd.5:Started 2023-08-27T11:11:45.701 INFO:tasks.ceph.osd.6:Restarting daemon 2023-08-27T11:11:45.702 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-08-27T11:11:45.703 INFO:tasks.ceph.osd.6:Started 2023-08-27T11:11:45.704 INFO:tasks.ceph.osd.7:Restarting daemon 2023-08-27T11:11:45.704 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-08-27T11:11:45.706 INFO:tasks.ceph.osd.7:Started 2023-08-27T11:11:45.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:11:45.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.844+0000 7f99d1b66700 1 Processor -- start 2023-08-27T11:11:45.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.844+0000 7f99d1b66700 1 -- start start 2023-08-27T11:11:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99d1b66700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f99cc15dbc0 0x7f99cc15a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99d1b66700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99cc15a750 0x7f99cc0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99d1b66700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc0caef0 con 0x7f99cc070e40 2023-08-27T11:11:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99d1b66700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc0cac50 con 0x7f99cc15a750 2023-08-27T11:11:45.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99d1b66700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc0ce000 con 0x7f99cc15dbc0 2023-08-27T11:11:45.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99cbfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99cc15a750 0x7f99cc0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99cb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f99cc070e40 0x7f99cc071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37638/0 (socket says 172.21.15.45:37638) 2023-08-27T11:11:45.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99cb7fe700 1 -- 172.21.15.45:0/91448539 learned_addr learned my addr 172.21.15.45:0/91448539 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:11:45.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99caffd700 1 --2- 172.21.15.45:0/91448539 >> v2:172.21.15.135:3300/0 conn(0x7f99cc15dbc0 0x7f99cc15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99cbfff700 1 -- 172.21.15.45:0/91448539 >> v2:172.21.15.135:3300/0 conn(0x7f99cc15dbc0 msgr2=0x7f99cc15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:45.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99cbfff700 1 --2- 172.21.15.45:0/91448539 >> v2:172.21.15.135:3300/0 conn(0x7f99cc15dbc0 0x7f99cc15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99cbfff700 1 -- 172.21.15.45:0/91448539 >> v1:172.21.15.45:6789/0 conn(0x7f99cc070e40 legacy=0x7f99cc071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99ca7fc700 1 -- 172.21.15.45:0/91448539 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847892975 0 0) 0x7f99cc0caef0 con 0x7f99cc070e40 2023-08-27T11:11:45.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99cbfff700 1 -- 172.21.15.45:0/91448539 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc15b5c0 con 0x7f99cc15a750 2023-08-27T11:11:45.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.845+0000 7f99caffd700 1 --2- 172.21.15.45:0/91448539 >> v2:172.21.15.135:3300/0 conn(0x7f99cc15dbc0 0x7f99cc15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:11:45.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99cbfff700 1 --2- 172.21.15.45:0/91448539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15a750 0x7f99cc0cb380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009d90 tx=0x7f99c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cd6dccf693665d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:45.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99ca7fc700 1 -- 172.21.15.45:0/91448539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c400e070 con 0x7f99cc15a750 2023-08-27T11:11:45.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99ca7fc700 1 -- 172.21.15.45:0/91448539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c400bb90 con 0x7f99cc15a750 2023-08-27T11:11:45.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99ca7fc700 1 -- 172.21.15.45:0/91448539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4012430 con 0x7f99cc15a750 2023-08-27T11:11:45.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99d1b66700 1 -- 172.21.15.45:0/91448539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15a750 msgr2=0x7f99cc0cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99d1b66700 1 --2- 172.21.15.45:0/91448539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15a750 0x7f99cc0cb380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009d90 tx=0x7f99c400b330 comp rx=0 tx=0).stop 2023-08-27T11:11:45.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99d1b66700 1 -- 172.21.15.45:0/91448539 shutdown_connections 2023-08-27T11:11:45.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99d1b66700 1 --2- 172.21.15.45:0/91448539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15a750 0x7f99cc0cb380 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99d1b66700 1 --2- 172.21.15.45:0/91448539 >> v2:172.21.15.135:3300/0 conn(0x7f99cc15dbc0 0x7f99cc15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99d1b66700 1 -- 172.21.15.45:0/91448539 >> 172.21.15.45:0/91448539 conn(0x7f99cc0cfd60 msgr2=0x7f99cc0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:45.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99d1b66700 1 -- 172.21.15.45:0/91448539 shutdown_connections 2023-08-27T11:11:45.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.846+0000 7f99d1b66700 1 -- 172.21.15.45:0/91448539 wait complete. 2023-08-27T11:11:45.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.848+0000 7f99d1b66700 1 Processor -- start 2023-08-27T11:11:45.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.848+0000 7f99d1b66700 1 -- start start 2023-08-27T11:11:45.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99d1b66700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99cc070e40 0x7f99cc0c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99d1b66700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99cc15a750 0x7f99cc1f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99d1b66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15dbc0 0x7f99cc0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:45.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99d1b66700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0c82b0 con 0x7f99cc15dbc0 2023-08-27T11:11:45.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99d1b66700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc151e70 con 0x7f99cc070e40 2023-08-27T11:11:45.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99d1b66700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc152150 con 0x7f99cc15a750 2023-08-27T11:11:45.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15dbc0 0x7f99cc0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15dbc0 0x7f99cc0c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50872/0 (socket says 172.21.15.45:50872) 2023-08-27T11:11:45.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cbfff700 1 -- 172.21.15.45:0/2492662959 learned_addr learned my addr 172.21.15.45:0/2492662959 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:11:45.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cb7fe700 1 --2- 172.21.15.45:0/2492662959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99cc070e40 0x7f99cc0c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99caffd700 1 --2- 172.21.15.45:0/2492662959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99cc15a750 0x7f99cc1f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:45.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cbfff700 1 -- 172.21.15.45:0/2492662959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99cc15a750 msgr2=0x7f99cc1f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cbfff700 1 --2- 172.21.15.45:0/2492662959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99cc15a750 0x7f99cc1f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cbfff700 1 -- 172.21.15.45:0/2492662959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99cc070e40 msgr2=0x7f99cc0c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cbfff700 1 --2- 172.21.15.45:0/2492662959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99cc070e40 0x7f99cc0c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cbfff700 1 -- 172.21.15.45:0/2492662959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4009a70 con 0x7f99cc15dbc0 2023-08-27T11:11:45.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99cbfff700 1 --2- 172.21.15.45:0/2492662959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15dbc0 0x7f99cc0c86d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f99c4014040 tx=0x7f99c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:45.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99b3fff700 1 -- 172.21.15.45:0/2492662959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4012430 con 0x7f99cc15dbc0 2023-08-27T11:11:45.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99b3fff700 1 -- 172.21.15.45:0/2492662959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c4020070 con 0x7f99cc15dbc0 2023-08-27T11:11:45.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99d1b66700 1 -- 172.21.15.45:0/2492662959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc0c7ed0 con 0x7f99cc15dbc0 2023-08-27T11:11:45.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99b3fff700 1 -- 172.21.15.45:0/2492662959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4004280 con 0x7f99cc15dbc0 2023-08-27T11:11:45.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.849+0000 7f99d1b66700 1 -- 172.21.15.45:0/2492662959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc0c9fc0 con 0x7f99cc15dbc0 2023-08-27T11:11:45.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.850+0000 7f99b3fff700 1 -- 172.21.15.45:0/2492662959 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f99c40046e0 con 0x7f99cc15dbc0 2023-08-27T11:11:45.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.850+0000 7f99b3fff700 1 -- 172.21.15.45:0/2492662959 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f99c4022070 con 0x7f99cc15dbc0 2023-08-27T11:11:45.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.850+0000 7f99d1b66700 1 -- 172.21.15.45:0/2492662959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc072180 con 0x7f99cc15dbc0 2023-08-27T11:11:45.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.854+0000 7f99b3fff700 1 -- 172.21.15.45:0/2492662959 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f99c4046b00 con 0x7f99cc15dbc0 2023-08-27T11:11:45.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.926+0000 7f99d1b66700 1 -- 172.21.15.45:0/2492662959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99cc0c7d20 con 0x7f99cc15dbc0 2023-08-27T11:11:45.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.927+0000 7f99b3fff700 1 -- 172.21.15.45:0/2492662959 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f99c4035030 con 0x7f99cc15dbc0 2023-08-27T11:11:45.929 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:11:45.929 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:11:45.636261+0000","last_up_change":"0.000000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:11:45.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.927+0000 7f99d1b66700 1 -- 172.21.15.45:0/2492662959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15dbc0 msgr2=0x7f99cc0c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:45.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.927+0000 7f99d1b66700 1 --2- 172.21.15.45:0/2492662959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15dbc0 0x7f99cc0c86d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f99c4014040 tx=0x7f99c4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.928+0000 7f99d1b66700 1 -- 172.21.15.45:0/2492662959 shutdown_connections 2023-08-27T11:11:45.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.928+0000 7f99d1b66700 1 --2- 172.21.15.45:0/2492662959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99cc15dbc0 0x7f99cc0c86d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.928+0000 7f99d1b66700 1 --2- 172.21.15.45:0/2492662959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99cc070e40 0x7f99cc0c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.928+0000 7f99d1b66700 1 --2- 172.21.15.45:0/2492662959 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99cc15a750 0x7f99cc1f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:45.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.928+0000 7f99d1b66700 1 -- 172.21.15.45:0/2492662959 >> 172.21.15.45:0/2492662959 conn(0x7f99cc0cfd60 msgr2=0x7f99cc15cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:45.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.928+0000 7f99d1b66700 1 -- 172.21.15.45:0/2492662959 shutdown_connections 2023-08-27T11:11:45.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:11:45.928+0000 7f99d1b66700 1 -- 172.21.15.45:0/2492662959 wait complete. 2023-08-27T11:11:45.942 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-08-27T11:11:45.943 INFO:tasks.ceph:Waiting for OSDs to come up 2023-08-27T11:11:46.247 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:46.245+0000 7f74d6019200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:11:46.264 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:46.263+0000 7f22b76ca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:11:46.275 INFO:tasks.ceph.osd.0.smithi045.stderr:2023-08-27T11:11:46.273+0000 7f775ccbe640 -1 Falling back to public interface 2023-08-27T11:11:46.283 INFO:tasks.ceph.osd.1.smithi045.stderr:2023-08-27T11:11:46.281+0000 7f173350f640 -1 Falling back to public interface 2023-08-27T11:11:46.293 INFO:tasks.ceph.osd.3.smithi045.stderr:2023-08-27T11:11:46.291+0000 7f258c219640 -1 Falling back to public interface 2023-08-27T11:11:46.300 INFO:tasks.ceph.osd.2.smithi045.stderr:2023-08-27T11:11:46.298+0000 7f4cd2130640 -1 Falling back to public interface 2023-08-27T11:11:46.301 INFO:tasks.ceph.osd.4.smithi135.stderr:2023-08-27T11:11:46.300+0000 7fb482b94640 -1 Falling back to public interface 2023-08-27T11:11:46.316 INFO:tasks.ceph.osd.7.smithi135.stderr:2023-08-27T11:11:46.315+0000 7f4059f5e640 -1 Falling back to public interface 2023-08-27T11:11:46.317 INFO:tasks.ceph.osd.6.smithi135.stderr:2023-08-27T11:11:46.316+0000 7f15dc85c640 -1 Falling back to public interface 2023-08-27T11:11:46.320 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:46.319+0000 7f74d6019200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:11:46.327 INFO:tasks.ceph.osd.5.smithi135.stderr:2023-08-27T11:11:46.326+0000 7f0c0b052640 -1 Falling back to public interface 2023-08-27T11:11:46.344 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:46.343+0000 7f22b76ca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:11:46.486 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:46.485+0000 7f22b76ca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:11:46.723 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:46.722+0000 7f22b76ca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:11:46.749 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-27T11:11:46.869 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.867+0000 7f680fab4700 1 Processor -- start 2023-08-27T11:11:46.869 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.867+0000 7f680fab4700 1 -- start start 2023-08-27T11:11:46.871 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680fab4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 0x7f680806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:46.872 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680fab4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f680806d3a0 0x7f680815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:46.872 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680fab4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68080cc5c0 con 0x7f6808158cb0 2023-08-27T11:11:46.872 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680fab4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680806b230 con 0x7f680806d3a0 2023-08-27T11:11:46.873 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680fab4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680815b780 con 0x7f680806ac60 2023-08-27T11:11:46.873 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680e051700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f680806d3a0 0x7f680815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:46.874 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680d850700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6808158cb0 0x7f68080ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37650/0 (socket says 172.21.15.45:37650) 2023-08-27T11:11:46.874 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680d850700 1 -- 172.21.15.45:0/1528674178 learned_addr learned my addr 172.21.15.45:0/1528674178 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:11:46.874 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680d04f700 1 --2- 172.21.15.45:0/1528674178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 0x7f680806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:46.875 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680e051700 1 -- 172.21.15.45:0/1528674178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 msgr2=0x7f680806ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:46.875 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680e051700 1 --2- 172.21.15.45:0/1528674178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 0x7f680806ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:46.875 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680e051700 1 -- 172.21.15.45:0/1528674178 >> v1:172.21.15.45:6789/0 conn(0x7f6808158cb0 legacy=0x7f68080ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:46.875 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.869+0000 7f680e051700 1 -- 172.21.15.45:0/1528674178 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68080cbd80 con 0x7f680806d3a0 2023-08-27T11:11:46.875 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.868+0000 7f680c84e700 1 -- 172.21.15.45:0/1528674178 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068872768 0 0) 0x7f68080cc5c0 con 0x7f6808158cb0 2023-08-27T11:11:46.876 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.869+0000 7f680e051700 1 --2- 172.21.15.45:0/1528674178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 0x7f680815be70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6800009d90 tx=0x7f680000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a568d7b176b0e4ab server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:46.876 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.869+0000 7f680c84e700 1 -- 172.21.15.45:0/1528674178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000e070 con 0x7f680806d3a0 2023-08-27T11:11:46.876 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.869+0000 7f680c84e700 1 -- 172.21.15.45:0/1528674178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680000bb20 con 0x7f680806d3a0 2023-08-27T11:11:46.876 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.869+0000 7f680c84e700 1 -- 172.21.15.45:0/1528674178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800012430 con 0x7f680806d3a0 2023-08-27T11:11:46.877 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.870+0000 7f680fab4700 1 -- 172.21.15.45:0/1528674178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 msgr2=0x7f680815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:46.877 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.870+0000 7f680fab4700 1 --2- 172.21.15.45:0/1528674178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 0x7f680815be70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6800009d90 tx=0x7f680000b330 comp rx=0 tx=0).stop 2023-08-27T11:11:46.877 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.870+0000 7f680fab4700 1 -- 172.21.15.45:0/1528674178 shutdown_connections 2023-08-27T11:11:46.877 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.870+0000 7f680fab4700 1 --2- 172.21.15.45:0/1528674178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 0x7f680815be70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:46.877 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.870+0000 7f680fab4700 1 --2- 172.21.15.45:0/1528674178 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 0x7f680806ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:46.878 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.870+0000 7f680fab4700 1 -- 172.21.15.45:0/1528674178 >> 172.21.15.45:0/1528674178 conn(0x7f68080cfd60 msgr2=0x7f68080d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:46.878 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.870+0000 7f680fab4700 1 -- 172.21.15.45:0/1528674178 shutdown_connections 2023-08-27T11:11:46.878 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.870+0000 7f680fab4700 1 -- 172.21.15.45:0/1528674178 wait complete. 2023-08-27T11:11:46.881 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.871+0000 7f680fab4700 1 Processor -- start 2023-08-27T11:11:46.881 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.871+0000 7f680fab4700 1 -- start start 2023-08-27T11:11:46.881 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680fab4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 0x7f68080c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:46.881 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680fab4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 0x7f680814f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:46.882 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680fab4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808158cb0 0x7f68080c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:46.882 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680fab4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68080c8bf0 con 0x7f680806d3a0 2023-08-27T11:11:46.882 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680fab4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680814f8c0 con 0x7f6808158cb0 2023-08-27T11:11:46.882 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680fab4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68080c7fe0 con 0x7f680806ac60 2023-08-27T11:11:46.883 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d04f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 0x7f680814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:46.883 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d04f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 0x7f680814f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50882/0 (socket says 172.21.15.45:50882) 2023-08-27T11:11:46.883 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d04f700 1 -- 172.21.15.45:0/1094530165 learned_addr learned my addr 172.21.15.45:0/1094530165 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:11:46.883 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d850700 1 --2- 172.21.15.45:0/1094530165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 0x7f68080c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:46.883 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680e051700 1 --2- 172.21.15.45:0/1094530165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808158cb0 0x7f68080c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:46.884 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d04f700 1 -- 172.21.15.45:0/1094530165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 msgr2=0x7f68080c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:46.884 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d04f700 1 --2- 172.21.15.45:0/1094530165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 0x7f68080c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:46.884 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d04f700 1 -- 172.21.15.45:0/1094530165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808158cb0 msgr2=0x7f68080c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:46.884 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d04f700 1 --2- 172.21.15.45:0/1094530165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808158cb0 0x7f68080c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:46.884 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d04f700 1 -- 172.21.15.45:0/1094530165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800009a70 con 0x7f680806d3a0 2023-08-27T11:11:46.885 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f680d04f700 1 --2- 172.21.15.45:0/1094530165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 0x7f680814f380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f67f800e9b0 tx=0x7f67f800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:46.885 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.872+0000 7f67fe7fc700 1 -- 172.21.15.45:0/1094530165 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8009710 con 0x7f680806d3a0 2023-08-27T11:11:46.885 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.873+0000 7f67fe7fc700 1 -- 172.21.15.45:0/1094530165 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f800ee50 con 0x7f680806d3a0 2023-08-27T11:11:46.885 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.873+0000 7f680fab4700 1 -- 172.21.15.45:0/1094530165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68080c82c0 con 0x7f680806d3a0 2023-08-27T11:11:46.886 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.873+0000 7f67fe7fc700 1 -- 172.21.15.45:0/1094530165 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800c9f0 con 0x7f680806d3a0 2023-08-27T11:11:46.886 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.873+0000 7f680fab4700 1 -- 172.21.15.45:0/1094530165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68080c8880 con 0x7f680806d3a0 2023-08-27T11:11:46.886 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.873+0000 7f67fe7fc700 1 -- 172.21.15.45:0/1094530165 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f67f8005a20 con 0x7f680806d3a0 2023-08-27T11:11:46.886 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.873+0000 7f67fe7fc700 1 -- 172.21.15.45:0/1094530165 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f67f8006700 con 0x7f680806d3a0 2023-08-27T11:11:46.886 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.874+0000 7f680fab4700 1 -- 172.21.15.45:0/1094530165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68080c8da0 con 0x7f680806d3a0 2023-08-27T11:11:46.887 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.876+0000 7f67fe7fc700 1 -- 172.21.15.45:0/1094530165 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f68080c8da0 con 0x7f680806d3a0 2023-08-27T11:11:46.939 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:46.937+0000 7f22b76ca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:11:46.950 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.947+0000 7f680fab4700 1 -- 172.21.15.45:0/1094530165 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6808072780 con 0x7f680806d3a0 2023-08-27T11:11:46.950 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.948+0000 7f67fe7fc700 1 -- 172.21.15.45:0/1094530165 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f67f8021060 con 0x7f680806d3a0 2023-08-27T11:11:46.951 INFO:teuthology.misc.health.smithi045.stdout: 2023-08-27T11:11:46.951 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":10,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:11:45.636261+0000","last_up_change":"0.000000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:11:46.952 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.948+0000 7f680fab4700 1 -- 172.21.15.45:0/1094530165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 msgr2=0x7f680814f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:46.952 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.948+0000 7f680fab4700 1 --2- 172.21.15.45:0/1094530165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 0x7f680814f380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f67f800e9b0 tx=0x7f67f800ecc0 comp rx=0 tx=0).stop 2023-08-27T11:11:46.952 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.949+0000 7f680fab4700 1 -- 172.21.15.45:0/1094530165 shutdown_connections 2023-08-27T11:11:46.953 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.949+0000 7f680fab4700 1 --2- 172.21.15.45:0/1094530165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6808158cb0 0x7f68080c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:46.953 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.949+0000 7f680fab4700 1 --2- 172.21.15.45:0/1094530165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f680806ac60 0x7f68080c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:46.953 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.949+0000 7f680fab4700 1 --2- 172.21.15.45:0/1094530165 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680806d3a0 0x7f680814f380 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:46.953 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.949+0000 7f680fab4700 1 -- 172.21.15.45:0/1094530165 >> 172.21.15.45:0/1094530165 conn(0x7f68080cfd60 msgr2=0x7f680815de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:46.954 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.949+0000 7f680fab4700 1 -- 172.21.15.45:0/1094530165 shutdown_connections 2023-08-27T11:11:46.954 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:46.949+0000 7f680fab4700 1 -- 172.21.15.45:0/1094530165 wait complete. 2023-08-27T11:11:46.962 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-08-27T11:11:47.047 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:47.045+0000 7f74d6019200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:11:47.070 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:47.069+0000 7f22b76ca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:11:47.128 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:47.126+0000 7f74d6019200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:11:47.275 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:47.273+0000 7f74d6019200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:11:47.294 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:47.293+0000 7f22b76ca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:11:47.452 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:47.451+0000 7f22b76ca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:11:47.520 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:47.518+0000 7f74d6019200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:11:47.567 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:47.566+0000 7f22b76ca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:11:47.739 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:47.738+0000 7f74d6019200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:11:47.878 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:47.876+0000 7f74d6019200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:11:47.893 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:47.892+0000 7f22b76ca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:11:47.995 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:47.994+0000 7f22b76ca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:11:48.105 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:48.104+0000 7f74d6019200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:11:48.181 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:48.180+0000 7f22b76ca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:11:48.267 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:48.265+0000 7f74d6019200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:11:48.387 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:48.385+0000 7f74d6019200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:11:48.720 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:48.719+0000 7f74d6019200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:11:48.778 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:48.777+0000 7f22b76ca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:11:48.834 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:48.832+0000 7f74d6019200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:11:49.039 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:49.038+0000 7f74d6019200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:11:49.716 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:11:49.714+0000 7f74d6019200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:11:49.809 INFO:tasks.ceph.osd.2.smithi045.stderr:2023-08-27T11:11:49.807+0000 7f4cd2130640 -1 osd.2 0 log_to_monitors true 2023-08-27T11:11:49.810 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:49.809+0000 7f226f6db700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-08-27T11:11:49.810 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:11:49.809+0000 7f226f6db700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-08-27T11:11:49.812 INFO:tasks.ceph.osd.4.smithi135.stderr:2023-08-27T11:11:49.811+0000 7fb482b94640 -1 osd.4 0 log_to_monitors true 2023-08-27T11:11:49.819 INFO:tasks.ceph.osd.0.smithi045.stderr:2023-08-27T11:11:49.817+0000 7f775ccbe640 -1 osd.0 0 log_to_monitors true 2023-08-27T11:11:49.829 INFO:tasks.ceph.osd.3.smithi045.stderr:2023-08-27T11:11:49.827+0000 7f258c219640 -1 osd.3 0 log_to_monitors true 2023-08-27T11:11:49.831 INFO:tasks.ceph.osd.1.smithi045.stderr:2023-08-27T11:11:49.829+0000 7f173350f640 -1 osd.1 0 log_to_monitors true 2023-08-27T11:11:49.861 INFO:tasks.ceph.osd.5.smithi135.stderr:2023-08-27T11:11:49.859+0000 7f0c0b052640 -1 osd.5 0 log_to_monitors true 2023-08-27T11:11:49.876 INFO:tasks.ceph.osd.6.smithi135.stderr:2023-08-27T11:11:49.875+0000 7f15dc85c640 -1 osd.6 0 log_to_monitors true 2023-08-27T11:11:49.877 INFO:tasks.ceph.osd.7.smithi135.stderr:2023-08-27T11:11:49.877+0000 7f4059f5e640 -1 osd.7 0 log_to_monitors true 2023-08-27T11:11:53.186 INFO:tasks.ceph.osd.1.smithi045.stderr:2023-08-27T11:11:53.184+0000 7f17278c7700 -1 osd.1 0 waiting for initial osdmap 2023-08-27T11:11:53.200 INFO:tasks.ceph.osd.1.smithi045.stderr:2023-08-27T11:11:53.199+0000 7f171f4a1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:11:53.206 INFO:tasks.ceph.osd.0.smithi045.stderr:2023-08-27T11:11:53.204+0000 7f7751076700 -1 osd.0 0 waiting for initial osdmap 2023-08-27T11:11:53.211 INFO:tasks.ceph.osd.2.smithi045.stderr:2023-08-27T11:11:53.209+0000 7f4cc64e8700 -1 osd.2 0 waiting for initial osdmap 2023-08-27T11:11:53.220 INFO:tasks.ceph.osd.3.smithi045.stderr:2023-08-27T11:11:53.219+0000 7f25805d1700 -1 osd.3 0 waiting for initial osdmap 2023-08-27T11:11:53.224 INFO:tasks.ceph.osd.2.smithi045.stderr:2023-08-27T11:11:53.221+0000 7f4cbe0c2700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:11:53.224 INFO:tasks.ceph.osd.0.smithi045.stderr:2023-08-27T11:11:53.221+0000 7f7748c50700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:11:53.232 INFO:tasks.ceph.osd.3.smithi045.stderr:2023-08-27T11:11:53.232+0000 7f25781ab700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:11:53.769 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-27T11:11:53.818 INFO:tasks.ceph.osd.5.smithi135.stderr:2023-08-27T11:11:53.816+0000 7f0bff40a700 -1 osd.5 0 waiting for initial osdmap 2023-08-27T11:11:53.840 INFO:tasks.ceph.osd.5.smithi135.stderr:2023-08-27T11:11:53.837+0000 7f0bf6fe4700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:11:53.845 INFO:tasks.ceph.osd.7.smithi135.stderr:2023-08-27T11:11:53.844+0000 7f404e316700 -1 osd.7 0 waiting for initial osdmap 2023-08-27T11:11:53.845 INFO:tasks.ceph.osd.6.smithi135.stderr:2023-08-27T11:11:53.844+0000 7f15d0c14700 -1 osd.6 0 waiting for initial osdmap 2023-08-27T11:11:53.871 INFO:tasks.ceph.osd.7.smithi135.stderr:2023-08-27T11:11:53.870+0000 7f4045ef0700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:11:53.871 INFO:tasks.ceph.osd.6.smithi135.stderr:2023-08-27T11:11:53.870+0000 7f15c87ee700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:11:53.887 INFO:tasks.ceph.osd.4.smithi135.stderr:2023-08-27T11:11:53.887+0000 7fb476f4c700 -1 osd.4 0 waiting for initial osdmap 2023-08-27T11:11:53.901 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f20458e8700 1 Processor -- start 2023-08-27T11:11:53.901 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f20458e8700 1 -- start start 2023-08-27T11:11:53.902 INFO:tasks.ceph.osd.4.smithi135.stderr:2023-08-27T11:11:53.900+0000 7fb46eb26700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-27T11:11:53.904 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f20458e8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f204015b070 0x7f20400cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:53.904 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f20458e8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 0x7f20400ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:53.904 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f20458e8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204015dad0 con 0x7f20400cdf60 2023-08-27T11:11:53.905 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f20458e8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204015d8e0 con 0x7f204015b070 2023-08-27T11:11:53.905 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f20458e8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204015ddb0 con 0x7f204006e980 2023-08-27T11:11:53.905 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f203effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f204015b070 0x7f20400cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:53.905 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f203effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f204015b070 0x7f20400cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50940/0 (socket says 172.21.15.45:50940) 2023-08-27T11:11:53.906 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f203effd700 1 -- 172.21.15.45:0/516905998 learned_addr learned my addr 172.21.15.45:0/516905998 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:11:53.906 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f203f7fe700 1 --2- 172.21.15.45:0/516905998 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 0x7f20400ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:53.906 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f203effd700 1 -- 172.21.15.45:0/516905998 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 msgr2=0x7f20400ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:11:53.906 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f203effd700 1 --2- 172.21.15.45:0/516905998 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 0x7f20400ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:53.906 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f203effd700 1 -- 172.21.15.45:0/516905998 >> v1:172.21.15.135:6789/0 conn(0x7f20400cdf60 legacy=0x7f204006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:53.907 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.900+0000 7f203effd700 1 -- 172.21.15.45:0/516905998 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204015e090 con 0x7f204015b070 2023-08-27T11:11:53.907 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f203effd700 1 --2- 172.21.15.45:0/516905998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 0x7f20400cda20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20340098f0 tx=0x7f2034009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92f9e9305ec6e59 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:53.907 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f203dffb700 1 -- 172.21.15.45:0/516905998 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034011830 con 0x7f204015b070 2023-08-27T11:11:53.907 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f203dffb700 1 -- 172.21.15.45:0/516905998 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2034011990 con 0x7f204015b070 2023-08-27T11:11:53.907 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f203dffb700 1 -- 172.21.15.45:0/516905998 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034011d20 con 0x7f204015b070 2023-08-27T11:11:53.908 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f20458e8700 1 -- 172.21.15.45:0/516905998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 msgr2=0x7f20400cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:53.908 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f20458e8700 1 --2- 172.21.15.45:0/516905998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 0x7f20400cda20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20340098f0 tx=0x7f2034009c00 comp rx=0 tx=0).stop 2023-08-27T11:11:53.908 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f20458e8700 1 -- 172.21.15.45:0/516905998 shutdown_connections 2023-08-27T11:11:53.908 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f20458e8700 1 --2- 172.21.15.45:0/516905998 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 0x7f20400ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:53.909 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f20458e8700 1 --2- 172.21.15.45:0/516905998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 0x7f20400cda20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:53.909 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f20458e8700 1 -- 172.21.15.45:0/516905998 >> 172.21.15.45:0/516905998 conn(0x7f20400cfe50 msgr2=0x7f20400d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:53.909 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f20458e8700 1 -- 172.21.15.45:0/516905998 shutdown_connections 2023-08-27T11:11:53.909 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.901+0000 7f20458e8700 1 -- 172.21.15.45:0/516905998 wait complete. 2023-08-27T11:11:53.909 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.903+0000 7f20458e8700 1 Processor -- start 2023-08-27T11:11:53.912 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.903+0000 7f20458e8700 1 -- start start 2023-08-27T11:11:53.912 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f20458e8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 0x7f20400b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:53.913 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f20458e8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20400cdf60 0x7f204014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:53.913 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f20458e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 0x7f20400b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:53.913 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f20458e8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400b3940 con 0x7f204015b070 2023-08-27T11:11:53.913 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f20458e8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2040147dd0 con 0x7f20400cdf60 2023-08-27T11:11:53.913 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f20458e8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204014b050 con 0x7f204006e980 2023-08-27T11:11:53.914 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 0x7f20400b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:53.914 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 0x7f20400b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50948/0 (socket says 172.21.15.45:50948) 2023-08-27T11:11:53.914 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203f7fe700 1 -- 172.21.15.45:0/875593252 learned_addr learned my addr 172.21.15.45:0/875593252 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:11:53.914 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203effd700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 0x7f20400b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:53.914 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203e7fc700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20400cdf60 0x7f204014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:53.915 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203f7fe700 1 -- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 msgr2=0x7f20400b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:53.915 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203f7fe700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 0x7f20400b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:53.915 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203f7fe700 1 -- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20400cdf60 msgr2=0x7f204014a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:53.915 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203f7fe700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20400cdf60 0x7f204014a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:53.915 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203f7fe700 1 -- 172.21.15.45:0/875593252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2034009550 con 0x7f204015b070 2023-08-27T11:11:53.916 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f203f7fe700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 0x7f20400b3400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2038009e30 tx=0x7f203800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:53.916 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f2027fff700 1 -- 172.21.15.45:0/875593252 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038009ca0 con 0x7f204015b070 2023-08-27T11:11:53.916 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f2027fff700 1 -- 172.21.15.45:0/875593252 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203800f040 con 0x7f204015b070 2023-08-27T11:11:53.916 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204014a7b0 con 0x7f204015b070 2023-08-27T11:11:53.916 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f2027fff700 1 -- 172.21.15.45:0/875593252 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038014480 con 0x7f204015b070 2023-08-27T11:11:53.916 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204014ad70 con 0x7f204015b070 2023-08-27T11:11:53.917 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.905+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20400cf8d0 con 0x7f204015b070 2023-08-27T11:11:53.917 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.906+0000 7f2027fff700 1 -- 172.21.15.45:0/875593252 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f20380145e0 con 0x7f204015b070 2023-08-27T11:11:53.917 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.906+0000 7f2027fff700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f202c06a050 0x7f202c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:11:53.917 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.907+0000 7f2027fff700 1 -- 172.21.15.45:0/875593252 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f2038088130 con 0x7f204015b070 2023-08-27T11:11:53.917 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.907+0000 7f203effd700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f202c06a050 0x7f202c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:11:53.918 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.907+0000 7f203effd700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f202c06a050 0x7f202c06c510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2034000c00 tx=0x7f2034008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:11:53.918 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:53.912+0000 7f2027fff700 1 -- 172.21.15.45:0/875593252 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2038054750 con 0x7f204015b070 2023-08-27T11:11:54.114 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.112+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2040072810 con 0x7f204015b070 2023-08-27T11:11:54.115 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.113+0000 7f2027fff700 1 -- 172.21.15.45:0/875593252 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2038018020 con 0x7f204015b070 2023-08-27T11:11:54.115 INFO:teuthology.misc.health.smithi045.stdout: 2023-08-27T11:11:54.115 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":12,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:11:51.804051+0000","last_up_change":"0.000000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:11:54.117 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f202c06a050 msgr2=0x7f202c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:54.118 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f202c06a050 0x7f202c06c510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2034000c00 tx=0x7f2034008030 comp rx=0 tx=0).stop 2023-08-27T11:11:54.118 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 msgr2=0x7f20400b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:11:54.118 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 0x7f20400b3400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2038009e30 tx=0x7f203800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:11:54.118 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 shutdown_connections 2023-08-27T11:11:54.118 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f204015b070 0x7f20400b3400 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:54.119 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f202c06a050 0x7f202c06c510 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:54.119 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f204006e980 0x7f20400b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:54.119 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 --2- 172.21.15.45:0/875593252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20400cdf60 0x7f204014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:11:54.119 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 >> 172.21.15.45:0/875593252 conn(0x7f20400cfe50 msgr2=0x7f2040159070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:11:54.119 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 shutdown_connections 2023-08-27T11:11:54.120 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:11:54.116+0000 7f20458e8700 1 -- 172.21.15.45:0/875593252 wait complete. 2023-08-27T11:11:54.128 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-08-27T11:12:00.935 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:01.063 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.063+0000 7f0010d42700 1 Processor -- start 2023-08-27T11:12:01.064 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.063+0000 7f0010d42700 1 -- start start 2023-08-27T11:12:01.066 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.063+0000 7f0010d42700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f000c15a240 0x7f000c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.066 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f0010d42700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0ceba0 0x7f000c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.066 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f0010d42700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c06d690 con 0x7f000c0cbe70 2023-08-27T11:12:01.067 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f0010d42700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0cc440 con 0x7f000c15a240 2023-08-27T11:12:01.067 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f0010d42700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c06d040 con 0x7f000c0ceba0 2023-08-27T11:12:01.067 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000a59c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f000c15a240 0x7f000c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.067 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000ad9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0ceba0 0x7f000c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.067 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000a59c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f000c15a240 0x7f000c0cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:39984/0 (socket says 172.21.15.45:39984) 2023-08-27T11:12:01.068 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f0009d9b700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f000c0cbe70 0x7f000c0ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:33568/0 (socket says 172.21.15.45:33568) 2023-08-27T11:12:01.068 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000a59c700 1 -- 172.21.15.45:0/4076827680 learned_addr learned my addr 172.21.15.45:0/4076827680 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:01.068 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000959a700 1 -- 172.21.15.45:0/4076827680 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261204711 0 0) 0x7f000c06d690 con 0x7f000c0cbe70 2023-08-27T11:12:01.068 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000959a700 1 -- 172.21.15.45:0/4076827680 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff80035b0 con 0x7f000c0cbe70 2023-08-27T11:12:01.068 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000ad9d700 1 -- 172.21.15.45:0/4076827680 >> v2:172.21.15.135:3300/0 conn(0x7f000c15a240 msgr2=0x7f000c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:01.069 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000ad9d700 1 --2- 172.21.15.45:0/4076827680 >> v2:172.21.15.135:3300/0 conn(0x7f000c15a240 0x7f000c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.069 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000ad9d700 1 -- 172.21.15.45:0/4076827680 >> v1:172.21.15.135:6789/0 conn(0x7f000c0cbe70 legacy=0x7f000c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.069 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000ad9d700 1 -- 172.21.15.45:0/4076827680 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c06d320 con 0x7f000c0ceba0 2023-08-27T11:12:01.069 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000a59c700 1 --2- 172.21.15.45:0/4076827680 >> v2:172.21.15.135:3300/0 conn(0x7f000c15a240 0x7f000c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:01.069 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.064+0000 7f000ad9d700 1 --2- 172.21.15.45:0/4076827680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0ceba0 0x7f000c06db20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0004009d90 tx=0x7f000400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=288ca39a305fc568 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:01.070 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f000959a700 1 -- 172.21.15.45:0/4076827680 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400e070 con 0x7f000c0ceba0 2023-08-27T11:12:01.070 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f000959a700 1 -- 172.21.15.45:0/4076827680 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000400bb20 con 0x7f000c0ceba0 2023-08-27T11:12:01.070 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f000959a700 1 -- 172.21.15.45:0/4076827680 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004012430 con 0x7f000c0ceba0 2023-08-27T11:12:01.070 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f0010d42700 1 -- 172.21.15.45:0/4076827680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0ceba0 msgr2=0x7f000c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.070 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f0010d42700 1 --2- 172.21.15.45:0/4076827680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0ceba0 0x7f000c06db20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0004009d90 tx=0x7f000400b330 comp rx=0 tx=0).stop 2023-08-27T11:12:01.070 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f0010d42700 1 -- 172.21.15.45:0/4076827680 shutdown_connections 2023-08-27T11:12:01.071 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f0010d42700 1 --2- 172.21.15.45:0/4076827680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0ceba0 0x7f000c06db20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.071 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f0010d42700 1 --2- 172.21.15.45:0/4076827680 >> v2:172.21.15.135:3300/0 conn(0x7f000c15a240 0x7f000c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.071 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f0010d42700 1 -- 172.21.15.45:0/4076827680 >> 172.21.15.45:0/4076827680 conn(0x7f000c0cfe50 msgr2=0x7f000c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:01.071 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f0010d42700 1 -- 172.21.15.45:0/4076827680 shutdown_connections 2023-08-27T11:12:01.071 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.065+0000 7f0010d42700 1 -- 172.21.15.45:0/4076827680 wait complete. 2023-08-27T11:12:01.074 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.067+0000 7f0010d42700 1 Processor -- start 2023-08-27T11:12:01.074 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.067+0000 7f0010d42700 1 -- start start 2023-08-27T11:12:01.074 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.067+0000 7f0010d42700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0cbe70 0x7f000c1f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.075 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.067+0000 7f0010d42700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f000c0ceba0 0x7f000c1f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.075 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.067+0000 7f0010d42700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000c15a240 0x7f000c1f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.075 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.067+0000 7f0010d42700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c1f36d0 con 0x7f000c15a240 2023-08-27T11:12:01.075 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.067+0000 7f0010d42700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c1f2d00 con 0x7f000c0ceba0 2023-08-27T11:12:01.075 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.067+0000 7f0010d42700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c1f26c0 con 0x7f000c0cbe70 2023-08-27T11:12:01.076 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000c15a240 0x7f000c1f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.076 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000c15a240 0x7f000c1f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51138/0 (socket says 172.21.15.45:51138) 2023-08-27T11:12:01.076 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000ad9d700 1 -- 172.21.15.45:0/742921943 learned_addr learned my addr 172.21.15.45:0/742921943 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:01.076 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000a59c700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0cbe70 0x7f000c1f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.076 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f0009d9b700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f000c0ceba0 0x7f000c1f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.077 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000ad9d700 1 -- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0cbe70 msgr2=0x7f000c1f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.077 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000ad9d700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0cbe70 0x7f000c1f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.077 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000ad9d700 1 -- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f000c0ceba0 msgr2=0x7f000c1f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.077 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000ad9d700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f000c0ceba0 0x7f000c1f2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.077 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000ad9d700 1 -- 172.21.15.45:0/742921943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc009550 con 0x7f000c15a240 2023-08-27T11:12:01.077 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f000ad9d700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000c15a240 0x7f000c1f5cc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0004014040 tx=0x7f0004004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:01.078 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f0002ffd700 1 -- 172.21.15.45:0/742921943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00040129f0 con 0x7f000c15a240 2023-08-27T11:12:01.078 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f0002ffd700 1 -- 172.21.15.45:0/742921943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0004020070 con 0x7f000c15a240 2023-08-27T11:12:01.078 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0004009a70 con 0x7f000c15a240 2023-08-27T11:12:01.078 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f0002ffd700 1 -- 172.21.15.45:0/742921943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004012db0 con 0x7f000c15a240 2023-08-27T11:12:01.078 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.068+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c1f6a20 con 0x7f000c15a240 2023-08-27T11:12:01.079 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.069+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c072810 con 0x7f000c15a240 2023-08-27T11:12:01.079 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.070+0000 7f0002ffd700 1 -- 172.21.15.45:0/742921943 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0004004040 con 0x7f000c15a240 2023-08-27T11:12:01.079 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.070+0000 7f0002ffd700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7effec06a110 0x7effec06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.079 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.070+0000 7f0002ffd700 1 -- 172.21.15.45:0/742921943 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0004022070 con 0x7f000c15a240 2023-08-27T11:12:01.079 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.071+0000 7f000a59c700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7effec06a110 0x7effec06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.080 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.071+0000 7f000a59c700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7effec06a110 0x7effec06c5d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efffc006d30 tx=0x7efffc0092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:01.080 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.075+0000 7f0002ffd700 1 -- 172.21.15.45:0/742921943 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0004055980 con 0x7f000c15a240 2023-08-27T11:12:01.278 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.277+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f000c1f2e40 con 0x7f000c15a240 2023-08-27T11:12:01.278 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.278+0000 7f0002ffd700 1 -- 172.21.15.45:0/742921943 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12862 (secure 0 0 0) 0x7f0004058fa0 con 0x7f000c15a240 2023-08-27T11:12:01.279 INFO:teuthology.misc.health.smithi045.stdout: 2023-08-27T11:12:01.279 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":16,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:11:57.217023+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:01.281 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7effec06a110 msgr2=0x7effec06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.281 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7effec06a110 0x7effec06c5d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efffc006d30 tx=0x7efffc0092c0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.281 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000c15a240 msgr2=0x7f000c1f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.282 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000c15a240 0x7f000c1f5cc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0004014040 tx=0x7f0004004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.282 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 shutdown_connections 2023-08-27T11:12:01.282 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000c15a240 0x7f000c1f5cc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.282 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7effec06a110 0x7effec06c5d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.282 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f000c0cbe70 0x7f000c1f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.283 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 --2- 172.21.15.45:0/742921943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f000c0ceba0 0x7f000c1f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.283 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 >> 172.21.15.45:0/742921943 conn(0x7f000c0cfe50 msgr2=0x7f000c0b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:01.283 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 shutdown_connections 2023-08-27T11:12:01.283 INFO:teuthology.misc.health.smithi045.stderr:2023-08-27T11:12:01.280+0000 7f0010d42700 1 -- 172.21.15.45:0/742921943 wait complete. 2023-08-27T11:12:01.292 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-08-27T11:12:01.292 INFO:tasks.ceph:Creating RBD pool 2023-08-27T11:12:01.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd pool create rbd 8 2023-08-27T11:12:01.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.416+0000 7fb204d8f700 1 Processor -- start 2023-08-27T11:12:01.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.416+0000 7fb204d8f700 1 -- start start 2023-08-27T11:12:01.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.417+0000 7fb204d8f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2001599e0 0x7fb2000700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.417+0000 7fb204d8f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2000cd350 con 0x7fb2000705f0 2023-08-27T11:12:01.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.417+0000 7fb204d8f700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20006bfd0 con 0x7fb2000ca210 2023-08-27T11:12:01.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.417+0000 7fb204d8f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20006bdc0 con 0x7fb2001599e0 2023-08-27T11:12:01.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.417+0000 7fb1fdd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb2000705f0 0x7fb2000c9b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37918/0 (socket says 172.21.15.45:37918) 2023-08-27T11:12:01.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.417+0000 7fb1fdd9b700 1 -- 172.21.15.45:0/1182737362 learned_addr learned my addr 172.21.15.45:0/1182737362 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:01.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fe59c700 1 --2- 172.21.15.45:0/1182737362 >> v2:172.21.15.135:3300/0 conn(0x7fb2001599e0 0x7fb2000700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fd59a700 1 -- 172.21.15.45:0/1182737362 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957345184 0 0) 0x7fb2000cd350 con 0x7fb2000705f0 2023-08-27T11:12:01.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fd59a700 1 -- 172.21.15.45:0/1182737362 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e80035b0 con 0x7fb2000705f0 2023-08-27T11:12:01.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fd59a700 1 -- 172.21.15.45:0/1182737362 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556012962 0 0) 0x7fb20006bfd0 con 0x7fb2000ca210 2023-08-27T11:12:01.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fd59a700 1 -- 172.21.15.45:0/1182737362 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2000cd350 con 0x7fb2000ca210 2023-08-27T11:12:01.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fe59c700 1 -- 172.21.15.45:0/1182737362 >> v1:172.21.15.135:6789/0 conn(0x7fb2000ca210 legacy=0x7fb2000cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fe59c700 1 -- 172.21.15.45:0/1182737362 >> v1:172.21.15.45:6789/0 conn(0x7fb2000705f0 legacy=0x7fb2000c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fe59c700 1 -- 172.21.15.45:0/1182737362 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20006c2b0 con 0x7fb2001599e0 2023-08-27T11:12:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fe59c700 1 --2- 172.21.15.45:0/1182737362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2001599e0 0x7fb2000700b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400a9e0 tx=0x7fb1f400acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c075609bb28166c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fd59a700 1 -- 172.21.15.45:0/1182737362 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f400cb70 con 0x7fb2001599e0 2023-08-27T11:12:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fd59a700 1 -- 172.21.15.45:0/1182737362 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f400ccd0 con 0x7fb2001599e0 2023-08-27T11:12:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.418+0000 7fb1fd59a700 1 -- 172.21.15.45:0/1182737362 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4018620 con 0x7fb2001599e0 2023-08-27T11:12:01.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.419+0000 7fb204d8f700 1 -- 172.21.15.45:0/1182737362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2001599e0 msgr2=0x7fb2000700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.419+0000 7fb204d8f700 1 --2- 172.21.15.45:0/1182737362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2001599e0 0x7fb2000700b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400a9e0 tx=0x7fb1f400acf0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.419+0000 7fb204d8f700 1 -- 172.21.15.45:0/1182737362 shutdown_connections 2023-08-27T11:12:01.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.419+0000 7fb204d8f700 1 --2- 172.21.15.45:0/1182737362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2001599e0 0x7fb2000700b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.419+0000 7fb204d8f700 1 -- 172.21.15.45:0/1182737362 >> 172.21.15.45:0/1182737362 conn(0x7fb2000b07c0 msgr2=0x7fb2000b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:01.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.419+0000 7fb204d8f700 1 -- 172.21.15.45:0/1182737362 shutdown_connections 2023-08-27T11:12:01.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.419+0000 7fb204d8f700 1 -- 172.21.15.45:0/1182737362 wait complete. 2023-08-27T11:12:01.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.420+0000 7fb204d8f700 1 Processor -- start 2023-08-27T11:12:01.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.420+0000 7fb204d8f700 1 -- start start 2023-08-27T11:12:01.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb204d8f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2000705f0 0x7fb2000c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb204d8f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2000ca210 0x7fb20015c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb204d8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2001599e0 0x7fb2000c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb204d8f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000b4570 con 0x7fb2001599e0 2023-08-27T11:12:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb204d8f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000b3fb0 con 0x7fb2000705f0 2023-08-27T11:12:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb204d8f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000b4290 con 0x7fb2000ca210 2023-08-27T11:12:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb1fed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2001599e0 0x7fb2000c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb1fed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2001599e0 0x7fb2000c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51152/0 (socket says 172.21.15.45:51152) 2023-08-27T11:12:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb1fed9d700 1 -- 172.21.15.45:0/1949140220 learned_addr learned my addr 172.21.15.45:0/1949140220 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb1fe59c700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2000705f0 0x7fb2000c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.421+0000 7fb1fdd9b700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2000ca210 0x7fb20015c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb1fed9d700 1 -- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2000ca210 msgr2=0x7fb20015c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb1fed9d700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2000ca210 0x7fb20015c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb1fed9d700 1 -- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2000705f0 msgr2=0x7fb2000c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb1fed9d700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2000705f0 0x7fb2000c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb1fed9d700 1 -- 172.21.15.45:0/1949140220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f400a640 con 0x7fb2001599e0 2023-08-27T11:12:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb1fed9d700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2001599e0 0x7fb2000c7f10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800cc50 tx=0x7fb1f800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb1eeffd700 1 -- 172.21.15.45:0/1949140220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8003c00 con 0x7fb2001599e0 2023-08-27T11:12:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb1eeffd700 1 -- 172.21.15.45:0/1949140220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f8003d60 con 0x7fb2001599e0 2023-08-27T11:12:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb200159ef0 con 0x7fb2001599e0 2023-08-27T11:12:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb1eeffd700 1 -- 172.21.15.45:0/1949140220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800aab0 con 0x7fb2001599e0 2023-08-27T11:12:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.422+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20015cc20 con 0x7fb2001599e0 2023-08-27T11:12:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.423+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20015a410 con 0x7fb2001599e0 2023-08-27T11:12:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.423+0000 7fb1eeffd700 1 -- 172.21.15.45:0/1949140220 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb1f800ac10 con 0x7fb2001599e0 2023-08-27T11:12:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.424+0000 7fb1eeffd700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1e006a0e0 0x7fb1e006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.424+0000 7fb1eeffd700 1 -- 172.21.15.45:0/1949140220 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb1f8088180 con 0x7fb2001599e0 2023-08-27T11:12:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.424+0000 7fb1fe59c700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1e006a0e0 0x7fb1e006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.424+0000 7fb1fe59c700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1e006a0e0 0x7fb1e006c5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400a1c0 tx=0x7fb1f401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.428+0000 7fb1eeffd700 1 -- 172.21.15.45:0/1949140220 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb20015a410 con 0x7fb2001599e0 2023-08-27T11:12:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.628+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fb20015a410 con 0x7fb2001599e0 2023-08-27T11:12:01.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.857+0000 7fb1eeffd700 1 -- 172.21.15.45:0/1949140220 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fb20015a410 con 0x7fb2001599e0 2023-08-27T11:12:01.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1e006a0e0 msgr2=0x7fb1e006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1e006a0e0 0x7fb1e006c5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400a1c0 tx=0x7fb1f401b040 comp rx=0 tx=0).stop 2023-08-27T11:12:01.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2001599e0 msgr2=0x7fb2000c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:01.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2001599e0 0x7fb2000c7f10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800cc50 tx=0x7fb1f800cf60 comp rx=0 tx=0).stop 2023-08-27T11:12:01.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 shutdown_connections 2023-08-27T11:12:01.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2001599e0 0x7fb2000c7f10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1e006a0e0 0x7fb1e006c5a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2000705f0 0x7fb2000c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 --2- 172.21.15.45:0/1949140220 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2000ca210 0x7fb20015c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 >> 172.21.15.45:0/1949140220 conn(0x7fb2000b07c0 msgr2=0x7fb200154600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 shutdown_connections 2023-08-27T11:12:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:01.860+0000 7fb204d8f700 1 -- 172.21.15.45:0/1949140220 wait complete. 2023-08-27T11:12:01.864 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'rbd' created 2023-08-27T11:12:01.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-08-27T11:12:02.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.001+0000 7f80a09ab700 1 Processor -- start 2023-08-27T11:12:02.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.001+0000 7f80a09ab700 1 -- start start 2023-08-27T11:12:02.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f80a09ab700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f809c0b65d0 0x7f809c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:02.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f80a09ab700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809c15ec50 con 0x7f809c0d0520 2023-08-27T11:12:02.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f80a09ab700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809c15eea0 con 0x7f809c0cb5c0 2023-08-27T11:12:02.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f80a09ab700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809c15f180 con 0x7f809c0b65d0 2023-08-27T11:12:02.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f809a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f809c0b65d0 0x7f809c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:02.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f809ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f809c0d0520 0x7f809c0cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37936/0 (socket says 172.21.15.45:37936) 2023-08-27T11:12:02.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f809ad9d700 1 -- 172.21.15.45:0/2872537480 learned_addr learned my addr 172.21.15.45:0/2872537480 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f809a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f809c0b65d0 0x7f809c0ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51156/0 (socket says 172.21.15.45:51156) 2023-08-27T11:12:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f809959a700 1 -- 172.21.15.45:0/2872537480 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686973669 0 0) 0x7f809c15ec50 con 0x7f809c0d0520 2023-08-27T11:12:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f809959a700 1 -- 172.21.15.45:0/2872537480 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80840035b0 con 0x7f809c0d0520 2023-08-27T11:12:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f809a59c700 1 -- 172.21.15.45:0/2872537480 >> v1:172.21.15.135:6789/0 conn(0x7f809c0cb5c0 legacy=0x7f809c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:02.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f809a59c700 1 -- 172.21.15.45:0/2872537480 >> v1:172.21.15.45:6789/0 conn(0x7f809c0d0520 legacy=0x7f809c0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.002+0000 7f809a59c700 1 -- 172.21.15.45:0/2872537480 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c15f460 con 0x7f809c0b65d0 2023-08-27T11:12:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f809a59c700 1 --2- 172.21.15.45:0/2872537480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 0x7f809c0ccdb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8090009870 tx=0x7f8090009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb4150410d02572 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f809959a700 1 -- 172.21.15.45:0/2872537480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090011830 con 0x7f809c0b65d0 2023-08-27T11:12:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f809959a700 1 -- 172.21.15.45:0/2872537480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8090011990 con 0x7f809c0b65d0 2023-08-27T11:12:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f809959a700 1 -- 172.21.15.45:0/2872537480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090011d20 con 0x7f809c0b65d0 2023-08-27T11:12:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f80a09ab700 1 -- 172.21.15.45:0/2872537480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 msgr2=0x7f809c0ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f80a09ab700 1 --2- 172.21.15.45:0/2872537480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 0x7f809c0ccdb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8090009870 tx=0x7f8090009b80 comp rx=0 tx=0).stop 2023-08-27T11:12:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f80a09ab700 1 -- 172.21.15.45:0/2872537480 shutdown_connections 2023-08-27T11:12:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f80a09ab700 1 --2- 172.21.15.45:0/2872537480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 0x7f809c0ccdb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f80a09ab700 1 -- 172.21.15.45:0/2872537480 >> 172.21.15.45:0/2872537480 conn(0x7f809c06f960 msgr2=0x7f809c06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:02.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f80a09ab700 1 -- 172.21.15.45:0/2872537480 shutdown_connections 2023-08-27T11:12:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.003+0000 7f80a09ab700 1 -- 172.21.15.45:0/2872537480 wait complete. 2023-08-27T11:12:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.005+0000 7f80a09ab700 1 Processor -- start 2023-08-27T11:12:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.005+0000 7f80a09ab700 1 -- start start 2023-08-27T11:12:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.005+0000 7f80a09ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 0x7f809c153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f80a09ab700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f809c0cb5c0 0x7f809c155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f80a09ab700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f809c0d0520 0x7f809c1564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f80a09ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c0c8a40 con 0x7f809c0b65d0 2023-08-27T11:12:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f80a09ab700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c0cf8b0 con 0x7f809c0cb5c0 2023-08-27T11:12:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f80a09ab700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c0c80e0 con 0x7f809c0d0520 2023-08-27T11:12:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 0x7f809c153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 0x7f809c153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51162/0 (socket says 172.21.15.45:51162) 2023-08-27T11:12:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809a59c700 1 -- 172.21.15.45:0/3348092123 learned_addr learned my addr 172.21.15.45:0/3348092123 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f8099d9b700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f809c0cb5c0 0x7f809c155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809ad9d700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f809c0d0520 0x7f809c1564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809a59c700 1 -- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f809c0d0520 msgr2=0x7f809c1564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809a59c700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f809c0d0520 0x7f809c1564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809a59c700 1 -- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f809c0cb5c0 msgr2=0x7f809c155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:02.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809a59c700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f809c0cb5c0 0x7f809c155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:02.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809a59c700 1 -- 172.21.15.45:0/3348092123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80900094a0 con 0x7f809c0b65d0 2023-08-27T11:12:02.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f809a59c700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 0x7f809c153280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80900090b0 tx=0x7f8090004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:02.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f808affd700 1 -- 172.21.15.45:0/3348092123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80900036a0 con 0x7f809c0b65d0 2023-08-27T11:12:02.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f808affd700 1 -- 172.21.15.45:0/3348092123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809001f070 con 0x7f809c0b65d0 2023-08-27T11:12:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c0c8380 con 0x7f809c0b65d0 2023-08-27T11:12:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f808affd700 1 -- 172.21.15.45:0/3348092123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090003a00 con 0x7f809c0b65d0 2023-08-27T11:12:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.006+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c1537c0 con 0x7f809c0b65d0 2023-08-27T11:12:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.007+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c06adb0 con 0x7f809c0b65d0 2023-08-27T11:12:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.008+0000 7f808affd700 1 -- 172.21.15.45:0/3348092123 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8090003b60 con 0x7f809c0b65d0 2023-08-27T11:12:02.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.009+0000 7f808affd700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f807c06a0e0 0x7f807c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:02.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.009+0000 7f808affd700 1 -- 172.21.15.45:0/3348092123 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f8090021070 con 0x7f809c0b65d0 2023-08-27T11:12:02.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.009+0000 7f8099d9b700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f807c06a0e0 0x7f807c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:02.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.009+0000 7f8099d9b700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f807c06a0e0 0x7f807c06c5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f809c0d03f0 tx=0x7f808c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.013+0000 7f808affd700 1 -- 172.21.15.45:0/3348092123 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8090055970 con 0x7f809c0b65d0 2023-08-27T11:12:02.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.215+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f809c06f4a0 con 0x7f809c0b65d0 2023-08-27T11:12:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.867+0000 7f808affd700 1 -- 172.21.15.45:0/3348092123 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f80900150c0 con 0x7f809c0b65d0 2023-08-27T11:12:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f807c06a0e0 msgr2=0x7f807c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f807c06a0e0 0x7f807c06c5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f809c0d03f0 tx=0x7f808c009040 comp rx=0 tx=0).stop 2023-08-27T11:12:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 msgr2=0x7f809c153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 0x7f809c153280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80900090b0 tx=0x7f8090004750 comp rx=0 tx=0).stop 2023-08-27T11:12:02.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 shutdown_connections 2023-08-27T11:12:02.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809c0b65d0 0x7f809c153280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:02.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f807c06a0e0 0x7f807c06c5a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:02.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f809c0d0520 0x7f809c1564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:02.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 --2- 172.21.15.45:0/3348092123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f809c0cb5c0 0x7f809c155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 >> 172.21.15.45:0/3348092123 conn(0x7f809c06f960 msgr2=0x7f809c15de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 shutdown_connections 2023-08-27T11:12:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:02.871+0000 7f80a09ab700 1 -- 172.21.15.45:0/3348092123 wait complete. 2023-08-27T11:12:02.874 INFO:teuthology.orchestra.run.smithi045.stderr:enabled application 'rbd' on pool 'rbd' 2023-08-27T11:12:02.887 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-08-27T11:12:02.888 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:12:02.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:12:02.890 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:12:02.891 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:12:02.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:12:02.893 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:12:02.894 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:12:02.894 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:12:02.897 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:12:02.897 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:12:02.897 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:12:02.899 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:12:02.900 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-08-27T11:12:02.900 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:02.980 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-08-27T11:12:02.980 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-08-27T11:12:02.981 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2023-08-27T11:12:02.981 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2023-08-27T11:12:03.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.036+0000 7f84768db700 1 Processor -- start 2023-08-27T11:12:03.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.036+0000 7f84768db700 1 -- start start 2023-08-27T11:12:03.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f84768db700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f84700ca620 0x7f84700cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f84768db700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84700ce340 con 0x7f84700af200 2023-08-27T11:12:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f84768db700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84700ceec0 con 0x7f84700caa00 2023-08-27T11:12:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f84768db700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847015d7c0 con 0x7f84700ca620 2023-08-27T11:12:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f846f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f84700af200 0x7f84700ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37972/0 (socket says 172.21.15.45:37972) 2023-08-27T11:12:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f846f7fe700 1 -- 172.21.15.45:0/203768090 learned_addr learned my addr 172.21.15.45:0/203768090 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f846ffff700 1 --2- 172.21.15.45:0/203768090 >> v2:172.21.15.135:3300/0 conn(0x7f84700ca620 0x7f84700cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f846effd700 1 -- 172.21.15.45:0/203768090 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929409688 0 0) 0x7f84700ce340 con 0x7f84700af200 2023-08-27T11:12:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f846effd700 1 -- 172.21.15.45:0/203768090 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84580035b0 con 0x7f84700af200 2023-08-27T11:12:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f846ffff700 1 -- 172.21.15.45:0/203768090 >> v1:172.21.15.135:6789/0 conn(0x7f84700caa00 legacy=0x7f84700acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f846effd700 1 -- 172.21.15.45:0/203768090 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093680282 0 0) 0x7f84700ceec0 con 0x7f84700caa00 2023-08-27T11:12:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f846ffff700 1 -- 172.21.15.45:0/203768090 >> v1:172.21.15.45:6789/0 conn(0x7f84700af200 legacy=0x7f84700ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.037+0000 7f846ffff700 1 -- 172.21.15.45:0/203768090 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84700cc570 con 0x7f84700ca620 2023-08-27T11:12:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f846ffff700 1 --2- 172.21.15.45:0/203768090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84700ca620 0x7f84700cf350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8460009840 tx=0x7f8460009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d82f5b9b39947bb1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f846effd700 1 -- 172.21.15.45:0/203768090 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464001670 con 0x7f84700ca620 2023-08-27T11:12:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f846effd700 1 -- 172.21.15.45:0/203768090 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8460004030 con 0x7f84700ca620 2023-08-27T11:12:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f846effd700 1 -- 172.21.15.45:0/203768090 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84600043c0 con 0x7f84700ca620 2023-08-27T11:12:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f84768db700 1 -- 172.21.15.45:0/203768090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84700ca620 msgr2=0x7f84700cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f84768db700 1 --2- 172.21.15.45:0/203768090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84700ca620 0x7f84700cf350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8460009840 tx=0x7f8460009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f84768db700 1 -- 172.21.15.45:0/203768090 shutdown_connections 2023-08-27T11:12:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f84768db700 1 --2- 172.21.15.45:0/203768090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84700ca620 0x7f84700cf350 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f84768db700 1 -- 172.21.15.45:0/203768090 >> 172.21.15.45:0/203768090 conn(0x7f84700cc7e0 msgr2=0x7f84700ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f84768db700 1 -- 172.21.15.45:0/203768090 shutdown_connections 2023-08-27T11:12:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.038+0000 7f84768db700 1 -- 172.21.15.45:0/203768090 wait complete. 2023-08-27T11:12:03.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.040+0000 7f84768db700 1 Processor -- start 2023-08-27T11:12:03.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.040+0000 7f84768db700 1 -- start start 2023-08-27T11:12:03.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.040+0000 7f84768db700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84700af200 0x7f84700b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.040+0000 7f84768db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84700ca620 0x7f8470155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.040+0000 7f84768db700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84700caa00 0x7f84700c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.040+0000 7f84768db700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700c8b70 con 0x7f84700ca620 2023-08-27T11:12:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.040+0000 7f84768db700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8470155d50 con 0x7f84700af200 2023-08-27T11:12:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.040+0000 7f84768db700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700c5040 con 0x7f84700caa00 2023-08-27T11:12:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84700ca620 0x7f8470155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84700ca620 0x7f8470155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51202/0 (socket says 172.21.15.45:51202) 2023-08-27T11:12:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846f7fe700 1 -- 172.21.15.45:0/3166781050 learned_addr learned my addr 172.21.15.45:0/3166781050 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846ffff700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84700af200 0x7f84700b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f8474e78700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84700caa00 0x7f84700c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846f7fe700 1 -- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84700caa00 msgr2=0x7f84700c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846f7fe700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84700caa00 0x7f84700c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846f7fe700 1 -- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84700af200 msgr2=0x7f84700b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846f7fe700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84700af200 0x7f84700b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846f7fe700 1 -- 172.21.15.45:0/3166781050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84600094a0 con 0x7f84700ca620 2023-08-27T11:12:03.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846f7fe700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84700ca620 0x7f8470155810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f846400c780 tx=0x7f846400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846cff9700 1 -- 172.21.15.45:0/3166781050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464005d70 con 0x7f84700ca620 2023-08-27T11:12:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846cff9700 1 -- 172.21.15.45:0/3166781050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84640070d0 con 0x7f84700ca620 2023-08-27T11:12:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84700c8520 con 0x7f84700ca620 2023-08-27T11:12:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f846cff9700 1 -- 172.21.15.45:0/3166781050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464007460 con 0x7f84700ca620 2023-08-27T11:12:03.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.041+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84700c5320 con 0x7f84700ca620 2023-08-27T11:12:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.042+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84700c8d20 con 0x7f84700ca620 2023-08-27T11:12:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.043+0000 7f846cff9700 1 -- 172.21.15.45:0/3166781050 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f84640075c0 con 0x7f84700ca620 2023-08-27T11:12:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.043+0000 7f846cff9700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f845c06a0e0 0x7f845c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.043+0000 7f846cff9700 1 -- 172.21.15.45:0/3166781050 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8464087690 con 0x7f84700ca620 2023-08-27T11:12:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.043+0000 7f846ffff700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f845c06a0e0 0x7f845c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.044+0000 7f846ffff700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f845c06a0e0 0x7f845c06c5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8460009b90 tx=0x7f8460008000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:03.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.048+0000 7f846cff9700 1 -- 172.21.15.45:0/3166781050 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f84700c8d20 con 0x7f84700ca620 2023-08-27T11:12:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.254+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84700c8cb0 con 0x7f84700ca620 2023-08-27T11:12:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.255+0000 7f846cff9700 1 -- 172.21.15.45:0/3166781050 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14661 (secure 0 0 0) 0x7f8464053210 con 0x7f84700ca620 2023-08-27T11:12:03.256 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:03.256 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":18,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:02.860904+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f845c06a0e0 msgr2=0x7f845c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f845c06a0e0 0x7f845c06c5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8460009b90 tx=0x7f8460008000 comp rx=0 tx=0).stop 2023-08-27T11:12:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84700ca620 msgr2=0x7f8470155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84700ca620 0x7f8470155810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f846400c780 tx=0x7f846400ca90 comp rx=0 tx=0).stop 2023-08-27T11:12:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 shutdown_connections 2023-08-27T11:12:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f845c06a0e0 0x7f845c06c5a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f84700caa00 0x7f84700c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84700af200 0x7f84700b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 --2- 172.21.15.45:0/3166781050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84700ca620 0x7f8470155810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 >> 172.21.15.45:0/3166781050 conn(0x7f84700cc7e0 msgr2=0x7f84700cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:03.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 shutdown_connections 2023-08-27T11:12:03.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.257+0000 7f84768db700 1 -- 172.21.15.45:0/3166781050 wait complete. 2023-08-27T11:12:03.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:12:03.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:12:03.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:12:03.320 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:12:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.400+0000 7f0592e51700 1 Processor -- start 2023-08-27T11:12:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.400+0000 7f0592e51700 1 -- start start 2023-08-27T11:12:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f0592e51700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f058c15a220 0x7f058c0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f0592e51700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c0b3220 con 0x7f058c0aea00 2023-08-27T11:12:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f0592e51700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c0b2c60 con 0x7f058c072080 2023-08-27T11:12:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f0592e51700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058c0b2f40 con 0x7f058c15a220 2023-08-27T11:12:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f058bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f058c0aea00 0x7f058c071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37984/0 (socket says 172.21.15.45:37984) 2023-08-27T11:12:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f058bfff700 1 -- 172.21.15.45:0/462866788 learned_addr learned my addr 172.21.15.45:0/462866788 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f0590bed700 1 --2- 172.21.15.45:0/462866788 >> v2:172.21.15.135:3300/0 conn(0x7f058c15a220 0x7f058c0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f058b7fe700 1 -- 172.21.15.45:0/462866788 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252069870 0 0) 0x7f058c0b3220 con 0x7f058c0aea00 2023-08-27T11:12:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f058b7fe700 1 -- 172.21.15.45:0/462866788 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f057c0035b0 con 0x7f058c0aea00 2023-08-27T11:12:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f0590bed700 1 -- 172.21.15.45:0/462866788 >> v1:172.21.15.135:6789/0 conn(0x7f058c072080 legacy=0x7f058c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f0590bed700 1 -- 172.21.15.45:0/462866788 >> v1:172.21.15.45:6789/0 conn(0x7f058c0aea00 legacy=0x7f058c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f058b7fe700 1 -- 172.21.15.45:0/462866788 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678607327 0 0) 0x7f058c0b2c60 con 0x7f058c072080 2023-08-27T11:12:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.401+0000 7f0590bed700 1 -- 172.21.15.45:0/462866788 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c0cec70 con 0x7f058c15a220 2023-08-27T11:12:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f0590bed700 1 --2- 172.21.15.45:0/462866788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f058c15a220 0x7f058c0cc780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0580009840 tx=0x7f0580009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d8434f7cf5680d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f058b7fe700 1 -- 172.21.15.45:0/462866788 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580011830 con 0x7f058c15a220 2023-08-27T11:12:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f058b7fe700 1 -- 172.21.15.45:0/462866788 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0580011990 con 0x7f058c15a220 2023-08-27T11:12:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f058b7fe700 1 -- 172.21.15.45:0/462866788 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580011d20 con 0x7f058c15a220 2023-08-27T11:12:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f0592e51700 1 -- 172.21.15.45:0/462866788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f058c15a220 msgr2=0x7f058c0cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f0592e51700 1 --2- 172.21.15.45:0/462866788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f058c15a220 0x7f058c0cc780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0580009840 tx=0x7f0580009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f0592e51700 1 -- 172.21.15.45:0/462866788 shutdown_connections 2023-08-27T11:12:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f0592e51700 1 --2- 172.21.15.45:0/462866788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f058c15a220 0x7f058c0cc780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f0592e51700 1 -- 172.21.15.45:0/462866788 >> 172.21.15.45:0/462866788 conn(0x7f058c0b4650 msgr2=0x7f058c0b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f0592e51700 1 -- 172.21.15.45:0/462866788 shutdown_connections 2023-08-27T11:12:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.402+0000 7f0592e51700 1 -- 172.21.15.45:0/462866788 wait complete. 2023-08-27T11:12:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.404+0000 7f0592e51700 1 Processor -- start 2023-08-27T11:12:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.404+0000 7f0592e51700 1 -- start start 2023-08-27T11:12:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0592e51700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c072080 0x7f058c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0592e51700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f058c0aea00 0x7f058c1f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0592e51700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f058c15a220 0x7f058c0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0592e51700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c1f6430 con 0x7f058c072080 2023-08-27T11:12:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0592e51700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0c3750 con 0x7f058c15a220 2023-08-27T11:12:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0592e51700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0cfc70 con 0x7f058c0aea00 2023-08-27T11:12:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0590bed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c072080 0x7f058c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0590bed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c072080 0x7f058c0b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51214/0 (socket says 172.21.15.45:51214) 2023-08-27T11:12:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0590bed700 1 -- 172.21.15.45:0/2461102828 learned_addr learned my addr 172.21.15.45:0/2461102828 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f05913ee700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f058c15a220 0x7f058c0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f058bfff700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f058c0aea00 0x7f058c1f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0590bed700 1 -- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f058c0aea00 msgr2=0x7f058c1f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0590bed700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f058c0aea00 0x7f058c1f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0590bed700 1 -- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f058c15a220 msgr2=0x7f058c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0590bed700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f058c15a220 0x7f058c0ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0590bed700 1 -- 172.21.15.45:0/2461102828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800094a0 con 0x7f058c072080 2023-08-27T11:12:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0590bed700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c072080 0x7f058c0b1af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05800090b0 tx=0x7f0580004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f05897fa700 1 -- 172.21.15.45:0/2461102828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058001a7e0 con 0x7f058c072080 2023-08-27T11:12:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f05897fa700 1 -- 172.21.15.45:0/2461102828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058001a940 con 0x7f058c072080 2023-08-27T11:12:03.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058c0c3470 con 0x7f058c072080 2023-08-27T11:12:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f05897fa700 1 -- 172.21.15.45:0/2461102828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058001acf0 con 0x7f058c072080 2023-08-27T11:12:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.405+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c0cf900 con 0x7f058c072080 2023-08-27T11:12:03.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.406+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c0ccf10 con 0x7f058c072080 2023-08-27T11:12:03.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.407+0000 7f05897fa700 1 -- 172.21.15.45:0/2461102828 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0580022070 con 0x7f058c072080 2023-08-27T11:12:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.407+0000 7f05897fa700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f057006a0e0 0x7f057006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.407+0000 7f05897fa700 1 -- 172.21.15.45:0/2461102828 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0580088f60 con 0x7f058c072080 2023-08-27T11:12:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.407+0000 7f058bfff700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f057006a0e0 0x7f057006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.408+0000 7f058bfff700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f057006a0e0 0x7f057006c5a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f058c158c90 tx=0x7f0578009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:03.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.412+0000 7f05897fa700 1 -- 172.21.15.45:0/2461102828 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f05800549b0 con 0x7f058c072080 2023-08-27T11:12:03.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.609+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f058c0b2940 con 0x7f058c072080 2023-08-27T11:12:03.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.609+0000 7f05897fa700 1 -- 172.21.15.45:0/2461102828 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0580057fd0 con 0x7f058c072080 2023-08-27T11:12:03.611 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:12:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.611+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f057006a0e0 msgr2=0x7f057006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.611+0000 7f0592e51700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f057006a0e0 0x7f057006c5a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f058c158c90 tx=0x7f0578009040 comp rx=0 tx=0).stop 2023-08-27T11:12:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c072080 msgr2=0x7f058c0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c072080 0x7f058c0b1af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05800090b0 tx=0x7f0580004750 comp rx=0 tx=0).stop 2023-08-27T11:12:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 shutdown_connections 2023-08-27T11:12:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f058c072080 0x7f058c0b1af0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f057006a0e0 0x7f057006c5a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f058c15a220 0x7f058c0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 --2- 172.21.15.45:0/2461102828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f058c0aea00 0x7f058c1f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 >> 172.21.15.45:0/2461102828 conn(0x7f058c0b4650 msgr2=0x7f058c06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 shutdown_connections 2023-08-27T11:12:03.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.612+0000 7f0592e51700 1 -- 172.21.15.45:0/2461102828 wait complete. 2023-08-27T11:12:03.624 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:12:03.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.791+0000 7f6435b00700 1 Processor -- start 2023-08-27T11:12:03.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.792+0000 7f6435b00700 1 -- start start 2023-08-27T11:12:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.792+0000 7f6435b00700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643015a700 0x7f643015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.792+0000 7f6435b00700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f643006f620 0x7f643015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.792+0000 7f6435b00700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64300cb060 con 0x7f643015aae0 2023-08-27T11:12:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.792+0000 7f6435b00700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643006e320 con 0x7f643006f620 2023-08-27T11:12:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.792+0000 7f6435b00700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643015d7d0 con 0x7f643015a700 2023-08-27T11:12:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.792+0000 7f642ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f643006f620 0x7f643015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f643006f620 0x7f643015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51226/0 (socket says 172.21.15.45:51226) 2023-08-27T11:12:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642ffff700 1 -- 172.21.15.45:0/4115899117 learned_addr learned my addr 172.21.15.45:0/4115899117 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642f7fe700 1 --2- 172.21.15.45:0/4115899117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643015a700 0x7f643015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642ffff700 1 -- 172.21.15.45:0/4115899117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643015a700 msgr2=0x7f643015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642ffff700 1 --2- 172.21.15.45:0/4115899117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643015a700 0x7f643015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642ffff700 1 -- 172.21.15.45:0/4115899117 >> v1:172.21.15.135:6789/0 conn(0x7f643015aae0 legacy=0x7f643006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642ffff700 1 -- 172.21.15.45:0/4115899117 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300ca450 con 0x7f643006f620 2023-08-27T11:12:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642ffff700 1 --2- 172.21.15.45:0/4115899117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643006f620 0x7f643015af60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6428009e50 tx=0x7f642800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552c880a687e1080 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642e7fc700 1 -- 172.21.15.45:0/4115899117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642800e070 con 0x7f643006f620 2023-08-27T11:12:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642e7fc700 1 -- 172.21.15.45:0/4115899117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642800b910 con 0x7f643006f620 2023-08-27T11:12:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.793+0000 7f642e7fc700 1 -- 172.21.15.45:0/4115899117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642800bcc0 con 0x7f643006f620 2023-08-27T11:12:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.794+0000 7f6435b00700 1 -- 172.21.15.45:0/4115899117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643006f620 msgr2=0x7f643015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.794+0000 7f6435b00700 1 --2- 172.21.15.45:0/4115899117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643006f620 0x7f643015af60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6428009e50 tx=0x7f642800b330 comp rx=0 tx=0).stop 2023-08-27T11:12:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.794+0000 7f6435b00700 1 -- 172.21.15.45:0/4115899117 shutdown_connections 2023-08-27T11:12:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.794+0000 7f6435b00700 1 --2- 172.21.15.45:0/4115899117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643015a700 0x7f643015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.794+0000 7f6435b00700 1 --2- 172.21.15.45:0/4115899117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643006f620 0x7f643015af60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.794+0000 7f6435b00700 1 -- 172.21.15.45:0/4115899117 >> 172.21.15.45:0/4115899117 conn(0x7f64300cc7e0 msgr2=0x7f64300ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:03.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.794+0000 7f6435b00700 1 -- 172.21.15.45:0/4115899117 shutdown_connections 2023-08-27T11:12:03.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.794+0000 7f6435b00700 1 -- 172.21.15.45:0/4115899117 wait complete. 2023-08-27T11:12:03.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.795+0000 7f6435b00700 1 Processor -- start 2023-08-27T11:12:03.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.796+0000 7f6435b00700 1 -- start start 2023-08-27T11:12:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.796+0000 7f6435b00700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643006f620 0x7f64300c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.796+0000 7f6435b00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643015a700 0x7f64301533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f6435b00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643015aae0 0x7f6430157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f6435b00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6430153900 con 0x7f643015aae0 2023-08-27T11:12:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f6435b00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64300c8030 con 0x7f643015a700 2023-08-27T11:12:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f6435b00700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64300c8310 con 0x7f643006f620 2023-08-27T11:12:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643015aae0 0x7f6430157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643015aae0 0x7f6430157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51232/0 (socket says 172.21.15.45:51232) 2023-08-27T11:12:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642ffff700 1 -- 172.21.15.45:0/4057468126 learned_addr learned my addr 172.21.15.45:0/4057468126 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642f7fe700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643006f620 0x7f64300c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642effd700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643015a700 0x7f64301533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642ffff700 1 -- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643006f620 msgr2=0x7f64300c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642ffff700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643006f620 0x7f64300c35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642ffff700 1 -- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643015a700 msgr2=0x7f64301533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:03.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642ffff700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643015a700 0x7f64301533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:03.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642ffff700 1 -- 172.21.15.45:0/4057468126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428009b30 con 0x7f643015aae0 2023-08-27T11:12:03.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642effd700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643015a700 0x7f64301533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:03.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f642ffff700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643015aae0 0x7f6430157570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6428012000 tx=0x7f6428004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:03.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f6427fff700 1 -- 172.21.15.45:0/4057468126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428004960 con 0x7f643015aae0 2023-08-27T11:12:03.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f6427fff700 1 -- 172.21.15.45:0/4057468126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64280099b0 con 0x7f643015aae0 2023-08-27T11:12:03.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64300c85f0 con 0x7f643015aae0 2023-08-27T11:12:03.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f6427fff700 1 -- 172.21.15.45:0/4057468126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642801b670 con 0x7f643015aae0 2023-08-27T11:12:03.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.797+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64300c8bb0 con 0x7f643015aae0 2023-08-27T11:12:03.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.798+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64300726d0 con 0x7f643015aae0 2023-08-27T11:12:03.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.799+0000 7f6427fff700 1 -- 172.21.15.45:0/4057468126 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f642801b7d0 con 0x7f643015aae0 2023-08-27T11:12:03.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.799+0000 7f6427fff700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f641006a0e0 0x7f641006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:03.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.799+0000 7f6427fff700 1 -- 172.21.15.45:0/4057468126 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6428089b20 con 0x7f643015aae0 2023-08-27T11:12:03.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.799+0000 7f642f7fe700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f641006a0e0 0x7f641006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:03.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.800+0000 7f642f7fe700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f641006a0e0 0x7f641006c5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6420009ad0 tx=0x7f6420008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:03.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:03.804+0000 7f6427fff700 1 -- 172.21.15.45:0/4057468126 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f64280555f0 con 0x7f643015aae0 2023-08-27T11:12:04.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.005+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f64300734a0 con 0x7f643015aae0 2023-08-27T11:12:04.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.005+0000 7f6427fff700 1 -- 172.21.15.45:0/4057468126 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f642801f020 con 0x7f643015aae0 2023-08-27T11:12:04.007 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:12:04.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f641006a0e0 msgr2=0x7f641006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f641006a0e0 0x7f641006c5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6420009ad0 tx=0x7f6420008040 comp rx=0 tx=0).stop 2023-08-27T11:12:04.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643015aae0 msgr2=0x7f6430157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643015aae0 0x7f6430157570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6428012000 tx=0x7f6428004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 shutdown_connections 2023-08-27T11:12:04.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f643015aae0 0x7f6430157570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f641006a0e0 0x7f641006c5a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f643006f620 0x7f64300c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 --2- 172.21.15.45:0/4057468126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643015a700 0x7f64301533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 >> 172.21.15.45:0/4057468126 conn(0x7f64300cc7e0 msgr2=0x7f6430070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:04.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.007+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 shutdown_connections 2023-08-27T11:12:04.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.008+0000 7f6435b00700 1 -- 172.21.15.45:0/4057468126 wait complete. 2023-08-27T11:12:04.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:12:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.145+0000 7fafc02cd700 1 Processor -- start 2023-08-27T11:12:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.145+0000 7fafc02cd700 1 -- start start 2023-08-27T11:12:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafc02cd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb80cbe70 0x7fafb80cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafc02cd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb806d690 con 0x7fafb815a240 2023-08-27T11:12:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafc02cd700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb80cc440 con 0x7fafb80ceba0 2023-08-27T11:12:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafc02cd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb806d040 con 0x7fafb80cbe70 2023-08-27T11:12:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbe069700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fafb815a240 0x7fafb80ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38016/0 (socket says 172.21.15.45:38016) 2023-08-27T11:12:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbe069700 1 -- 172.21.15.45:0/2315272201 learned_addr learned my addr 172.21.15.45:0/2315272201 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbe86a700 1 --2- 172.21.15.45:0/2315272201 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb80cbe70 0x7fafb80cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbd067700 1 -- 172.21.15.45:0/2315272201 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180913504 0 0) 0x7fafb806d690 con 0x7fafb815a240 2023-08-27T11:12:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbd067700 1 -- 172.21.15.45:0/2315272201 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa40035b0 con 0x7fafb815a240 2023-08-27T11:12:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbd067700 1 -- 172.21.15.45:0/2315272201 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454904222 0 0) 0x7fafb80cc440 con 0x7fafb80ceba0 2023-08-27T11:12:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbd067700 1 -- 172.21.15.45:0/2315272201 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb806d690 con 0x7fafb80ceba0 2023-08-27T11:12:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbe86a700 1 -- 172.21.15.45:0/2315272201 >> v1:172.21.15.135:6789/0 conn(0x7fafb80ceba0 legacy=0x7fafb806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbe86a700 1 -- 172.21.15.45:0/2315272201 >> v1:172.21.15.45:6789/0 conn(0x7fafb815a240 legacy=0x7fafb80ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbe86a700 1 -- 172.21.15.45:0/2315272201 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb806d320 con 0x7fafb80cbe70 2023-08-27T11:12:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.146+0000 7fafbe86a700 1 --2- 172.21.15.45:0/2315272201 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb80cbe70 0x7fafb80cae80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009dc0 tx=0x7fafb000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96013d427eb72222 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafbd067700 1 -- 172.21.15.45:0/2315272201 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb000e070 con 0x7fafb80cbe70 2023-08-27T11:12:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafbd067700 1 -- 172.21.15.45:0/2315272201 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb000bb20 con 0x7fafb80cbe70 2023-08-27T11:12:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafbd067700 1 -- 172.21.15.45:0/2315272201 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb000be90 con 0x7fafb80cbe70 2023-08-27T11:12:04.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafc02cd700 1 -- 172.21.15.45:0/2315272201 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb80cbe70 msgr2=0x7fafb80cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafc02cd700 1 --2- 172.21.15.45:0/2315272201 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb80cbe70 0x7fafb80cae80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009dc0 tx=0x7fafb000b330 comp rx=0 tx=0).stop 2023-08-27T11:12:04.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafc02cd700 1 -- 172.21.15.45:0/2315272201 shutdown_connections 2023-08-27T11:12:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafc02cd700 1 --2- 172.21.15.45:0/2315272201 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb80cbe70 0x7fafb80cae80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafc02cd700 1 -- 172.21.15.45:0/2315272201 >> 172.21.15.45:0/2315272201 conn(0x7fafb80cfe50 msgr2=0x7fafb80d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafc02cd700 1 -- 172.21.15.45:0/2315272201 shutdown_connections 2023-08-27T11:12:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.147+0000 7fafc02cd700 1 -- 172.21.15.45:0/2315272201 wait complete. 2023-08-27T11:12:04.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.149+0000 7fafc02cd700 1 Processor -- start 2023-08-27T11:12:04.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.149+0000 7fafc02cd700 1 -- start start 2023-08-27T11:12:04.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafc02cd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb80cbe70 0x7fafb80c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafc02cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb80ceba0 0x7fafb80c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafc02cd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb815a240 0x7fafb815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafc02cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb8156120 con 0x7fafb80ceba0 2023-08-27T11:12:04.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafc02cd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb8155910 con 0x7fafb80cbe70 2023-08-27T11:12:04.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafc02cd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80c85d0 con 0x7fafb815a240 2023-08-27T11:12:04.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbd868700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb80ceba0 0x7fafb80c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbd868700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb80ceba0 0x7fafb80c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51248/0 (socket says 172.21.15.45:51248) 2023-08-27T11:12:04.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbd868700 1 -- 172.21.15.45:0/3924170372 learned_addr learned my addr 172.21.15.45:0/3924170372 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:04.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbe86a700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb815a240 0x7fafb815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbe069700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb80cbe70 0x7fafb80c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbd868700 1 -- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb815a240 msgr2=0x7fafb815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbd868700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb815a240 0x7fafb815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbd868700 1 -- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb80cbe70 msgr2=0x7fafb80c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbd868700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb80cbe70 0x7fafb80c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbd868700 1 -- 172.21.15.45:0/3924170372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0009a70 con 0x7fafb80ceba0 2023-08-27T11:12:04.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbe86a700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb815a240 0x7fafb815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:04.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.150+0000 7fafbd868700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb80ceba0 0x7fafb80c6460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c490 tx=0x7fafa800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:04.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.151+0000 7fafaeffd700 1 -- 172.21.15.45:0/3924170372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa80052d0 con 0x7fafb80ceba0 2023-08-27T11:12:04.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.151+0000 7fafaeffd700 1 -- 172.21.15.45:0/3924170372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa8005450 con 0x7fafb80ceba0 2023-08-27T11:12:04.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.151+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb8155b60 con 0x7fafb80ceba0 2023-08-27T11:12:04.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.151+0000 7fafaeffd700 1 -- 172.21.15.45:0/3924170372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa80038c0 con 0x7fafb80ceba0 2023-08-27T11:12:04.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.151+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb80c70c0 con 0x7fafb80ceba0 2023-08-27T11:12:04.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.151+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb8072810 con 0x7fafb80ceba0 2023-08-27T11:12:04.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.152+0000 7fafaeffd700 1 -- 172.21.15.45:0/3924170372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fafa8003a20 con 0x7fafb80ceba0 2023-08-27T11:12:04.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.152+0000 7fafaeffd700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7faf9c06a0e0 0x7faf9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.152+0000 7fafaeffd700 1 -- 172.21.15.45:0/3924170372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fafa8087670 con 0x7fafb80ceba0 2023-08-27T11:12:04.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.153+0000 7fafbe069700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7faf9c06a0e0 0x7faf9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.153+0000 7fafbe069700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7faf9c06a0e0 0x7faf9c06c5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fafb4007fd0 tx=0x7fafb4007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:04.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.157+0000 7fafaeffd700 1 -- 172.21.15.45:0/3924170372 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fafa80530b0 con 0x7fafb80ceba0 2023-08-27T11:12:04.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.404+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafb80734a0 con 0x7fafb80ceba0 2023-08-27T11:12:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.405+0000 7fafaeffd700 1 -- 172.21.15.45:0/3924170372 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7fafa80566d0 con 0x7fafb80ceba0 2023-08-27T11:12:04.406 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:04.406 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-27T11:12:04.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7faf9c06a0e0 msgr2=0x7faf9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7faf9c06a0e0 0x7faf9c06c5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fafb4007fd0 tx=0x7fafb4007dc0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb80ceba0 msgr2=0x7fafb80c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb80ceba0 0x7fafb80c6460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c490 tx=0x7fafa800c7a0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 shutdown_connections 2023-08-27T11:12:04.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7faf9c06a0e0 0x7faf9c06c5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fafb815a240 0x7fafb815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb80cbe70 0x7fafb80c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 --2- 172.21.15.45:0/3924170372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafb80ceba0 0x7fafb80c6460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 >> 172.21.15.45:0/3924170372 conn(0x7fafb80cfe50 msgr2=0x7fafb8156e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 shutdown_connections 2023-08-27T11:12:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.407+0000 7fafc02cd700 1 -- 172.21.15.45:0/3924170372 wait complete. 2023-08-27T11:12:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2023-08-27T11:12:04.420 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.544+0000 7f8777d0f700 1 Processor -- start 2023-08-27T11:12:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.544+0000 7f8777d0f700 1 -- start start 2023-08-27T11:12:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f8777d0f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f87700ceba0 0x7f87700cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f8777d0f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f87700cbe70 0x7f877006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f8777d0f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877006d690 con 0x7f877015a240 2023-08-27T11:12:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f8777d0f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87700cc440 con 0x7f87700cbe70 2023-08-27T11:12:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f8777d0f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877006d040 con 0x7f87700ceba0 2023-08-27T11:12:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f87762ac700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f87700cbe70 0x7f877006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f8775aab700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f877015a240 0x7f87700ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38030/0 (socket says 172.21.15.45:38030) 2023-08-27T11:12:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f8775aab700 1 -- 172.21.15.45:0/1147957319 learned_addr learned my addr 172.21.15.45:0/1147957319 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f87752aa700 1 --2- 172.21.15.45:0/1147957319 >> v2:172.21.15.135:3300/0 conn(0x7f87700ceba0 0x7f87700cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f87762ac700 1 -- 172.21.15.45:0/1147957319 >> v2:172.21.15.135:3300/0 conn(0x7f87700ceba0 msgr2=0x7f87700cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f87762ac700 1 --2- 172.21.15.45:0/1147957319 >> v2:172.21.15.135:3300/0 conn(0x7f87700ceba0 0x7f87700cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f87762ac700 1 -- 172.21.15.45:0/1147957319 >> v1:172.21.15.45:6789/0 conn(0x7f877015a240 legacy=0x7f87700ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f87762ac700 1 -- 172.21.15.45:0/1147957319 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877006d320 con 0x7f87700cbe70 2023-08-27T11:12:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f8774aa9700 1 -- 172.21.15.45:0/1147957319 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080543950 0 0) 0x7f877006d690 con 0x7f877015a240 2023-08-27T11:12:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.545+0000 7f87762ac700 1 --2- 172.21.15.45:0/1147957319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87700cbe70 0x7f877006db20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8770074910 tx=0x7f876800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef0a54216212e1d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8774aa9700 1 -- 172.21.15.45:0/1147957319 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876800e070 con 0x7f87700cbe70 2023-08-27T11:12:04.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8774aa9700 1 -- 172.21.15.45:0/1147957319 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876800bbd0 con 0x7f87700cbe70 2023-08-27T11:12:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8774aa9700 1 -- 172.21.15.45:0/1147957319 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768012430 con 0x7f87700cbe70 2023-08-27T11:12:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8777d0f700 1 -- 172.21.15.45:0/1147957319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87700cbe70 msgr2=0x7f877006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8777d0f700 1 --2- 172.21.15.45:0/1147957319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87700cbe70 0x7f877006db20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8770074910 tx=0x7f876800b330 comp rx=0 tx=0).stop 2023-08-27T11:12:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8777d0f700 1 -- 172.21.15.45:0/1147957319 shutdown_connections 2023-08-27T11:12:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8777d0f700 1 --2- 172.21.15.45:0/1147957319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87700cbe70 0x7f877006db20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8777d0f700 1 --2- 172.21.15.45:0/1147957319 >> v2:172.21.15.135:3300/0 conn(0x7f87700ceba0 0x7f87700cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8777d0f700 1 -- 172.21.15.45:0/1147957319 >> 172.21.15.45:0/1147957319 conn(0x7f87700cfe50 msgr2=0x7f87700d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8777d0f700 1 -- 172.21.15.45:0/1147957319 shutdown_connections 2023-08-27T11:12:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.546+0000 7f8777d0f700 1 -- 172.21.15.45:0/1147957319 wait complete. 2023-08-27T11:12:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.548+0000 7f8777d0f700 1 Processor -- start 2023-08-27T11:12:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.548+0000 7f8777d0f700 1 -- start start 2023-08-27T11:12:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f8777d0f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87700cbe70 0x7f8770152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f8777d0f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87700ceba0 0x7f87700c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f8777d0f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877015a240 0x7f877006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f8777d0f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87701503e0 con 0x7f877015a240 2023-08-27T11:12:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f8777d0f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87700c3fd0 con 0x7f87700cbe70 2023-08-27T11:12:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f8777d0f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87700c42b0 con 0x7f87700ceba0 2023-08-27T11:12:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87762ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877015a240 0x7f877006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87762ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877015a240 0x7f877006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51262/0 (socket says 172.21.15.45:51262) 2023-08-27T11:12:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87762ac700 1 -- 172.21.15.45:0/715301722 learned_addr learned my addr 172.21.15.45:0/715301722 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f8775aab700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87700cbe70 0x7f8770152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87752aa700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87700ceba0 0x7f87700c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87762ac700 1 -- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87700ceba0 msgr2=0x7f87700c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87762ac700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87700ceba0 0x7f87700c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87762ac700 1 -- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87700cbe70 msgr2=0x7f8770152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87762ac700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87700cbe70 0x7f8770152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87762ac700 1 -- 172.21.15.45:0/715301722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8768009a70 con 0x7f877015a240 2023-08-27T11:12:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87752aa700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87700ceba0 0x7f87700c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.549+0000 7f87762ac700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877015a240 0x7f877006ac90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f877006c730 tx=0x7f8768004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.550+0000 7f87667fc700 1 -- 172.21.15.45:0/715301722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768012430 con 0x7f877015a240 2023-08-27T11:12:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.550+0000 7f87667fc700 1 -- 172.21.15.45:0/715301722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8768020070 con 0x7f877015a240 2023-08-27T11:12:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.550+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87700c4590 con 0x7f877015a240 2023-08-27T11:12:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.550+0000 7f87667fc700 1 -- 172.21.15.45:0/715301722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768004280 con 0x7f877015a240 2023-08-27T11:12:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.550+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877014c640 con 0x7f877015a240 2023-08-27T11:12:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.551+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8770150520 con 0x7f877015a240 2023-08-27T11:12:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.551+0000 7f87667fc700 1 -- 172.21.15.45:0/715301722 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f87680043e0 con 0x7f877015a240 2023-08-27T11:12:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.551+0000 7f87667fc700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f875406a0e0 0x7f875406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.551+0000 7f87667fc700 1 -- 172.21.15.45:0/715301722 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8768022070 con 0x7f877015a240 2023-08-27T11:12:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.552+0000 7f8775aab700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f875406a0e0 0x7f875406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.552+0000 7f8775aab700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f875406a0e0 0x7f875406c5a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8760000f60 tx=0x7f8760007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.556+0000 7f87667fc700 1 -- 172.21.15.45:0/715301722 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8768055ba0 con 0x7f877015a240 2023-08-27T11:12:04.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.758+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87700b2770 con 0x7f877015a240 2023-08-27T11:12:04.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.759+0000 7f87667fc700 1 -- 172.21.15.45:0/715301722 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14648 (secure 0 0 0) 0x7f87680591c0 con 0x7f877015a240 2023-08-27T11:12:04.760 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:04.760 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:03.865561+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f875406a0e0 msgr2=0x7f875406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f875406a0e0 0x7f875406c5a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8760000f60 tx=0x7f8760007e70 comp rx=0 tx=0).stop 2023-08-27T11:12:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877015a240 msgr2=0x7f877006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877015a240 0x7f877006ac90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f877006c730 tx=0x7f8768004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 shutdown_connections 2023-08-27T11:12:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877015a240 0x7f877006ac90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f875406a0e0 0x7f875406c5a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87700cbe70 0x7f8770152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 --2- 172.21.15.45:0/715301722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87700ceba0 0x7f87700c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 >> 172.21.15.45:0/715301722 conn(0x7f87700cfe50 msgr2=0x7f87700cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:04.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 shutdown_connections 2023-08-27T11:12:04.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.761+0000 7f8777d0f700 1 -- 172.21.15.45:0/715301722 wait complete. 2023-08-27T11:12:04.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:12:04.774 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:12:04.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.904+0000 7fc4870be700 1 Processor -- start 2023-08-27T11:12:04.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.904+0000 7fc4870be700 1 -- start start 2023-08-27T11:12:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.904+0000 7fc4870be700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4800d0520 0x7fc4800ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.904+0000 7fc4870be700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800cb5c0 0x7fc48015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.904+0000 7fc4870be700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48015ec50 con 0x7fc4800b65d0 2023-08-27T11:12:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.904+0000 7fc4870be700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48015c620 con 0x7fc4800d0520 2023-08-27T11:12:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.904+0000 7fc4870be700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48015eea0 con 0x7fc4800cb5c0 2023-08-27T11:12:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4800d0520 0x7fc4800ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4800d0520 0x7fc4800ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51268/0 (socket says 172.21.15.45:51268) 2023-08-27T11:12:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47ffff700 1 -- 172.21.15.45:0/1886493876 learned_addr learned my addr 172.21.15.45:0/1886493876 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc48565b700 1 --2- 172.21.15.45:0/1886493876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800cb5c0 0x7fc48015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47ffff700 1 -- 172.21.15.45:0/1886493876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800cb5c0 msgr2=0x7fc48015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47ffff700 1 --2- 172.21.15.45:0/1886493876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800cb5c0 0x7fc48015c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47ffff700 1 -- 172.21.15.45:0/1886493876 >> v1:172.21.15.135:6789/0 conn(0x7fc4800b65d0 legacy=0x7fc4800cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47ffff700 1 -- 172.21.15.45:0/1886493876 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48015f180 con 0x7fc4800d0520 2023-08-27T11:12:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47ffff700 1 --2- 172.21.15.45:0/1886493876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800d0520 0x7fc4800ccdb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0098f0 tx=0x7fc46c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25096b7e86ecf59 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47f7fe700 1 -- 172.21.15.45:0/1886493876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c004230 con 0x7fc4800d0520 2023-08-27T11:12:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47f7fe700 1 -- 172.21.15.45:0/1886493876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c004390 con 0x7fc4800d0520 2023-08-27T11:12:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.905+0000 7fc47f7fe700 1 -- 172.21.15.45:0/1886493876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0036a0 con 0x7fc4800d0520 2023-08-27T11:12:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.906+0000 7fc4870be700 1 -- 172.21.15.45:0/1886493876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800d0520 msgr2=0x7fc4800ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.906+0000 7fc4870be700 1 --2- 172.21.15.45:0/1886493876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800d0520 0x7fc4800ccdb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0098f0 tx=0x7fc46c009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.906+0000 7fc4870be700 1 -- 172.21.15.45:0/1886493876 shutdown_connections 2023-08-27T11:12:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.906+0000 7fc4870be700 1 --2- 172.21.15.45:0/1886493876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800cb5c0 0x7fc48015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.906+0000 7fc4870be700 1 --2- 172.21.15.45:0/1886493876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800d0520 0x7fc4800ccdb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.906+0000 7fc4870be700 1 -- 172.21.15.45:0/1886493876 >> 172.21.15.45:0/1886493876 conn(0x7fc48006f960 msgr2=0x7fc48006fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:04.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.906+0000 7fc4870be700 1 -- 172.21.15.45:0/1886493876 shutdown_connections 2023-08-27T11:12:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.906+0000 7fc4870be700 1 -- 172.21.15.45:0/1886493876 wait complete. 2023-08-27T11:12:04.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.907+0000 7fc4870be700 1 Processor -- start 2023-08-27T11:12:04.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.907+0000 7fc4870be700 1 -- start start 2023-08-27T11:12:04.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc4870be700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4800b65d0 0x7fc4801533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc4870be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800cb5c0 0x7fc480155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc4870be700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800d0520 0x7fc480159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc4870be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800c47b0 con 0x7fc4800cb5c0 2023-08-27T11:12:04.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc4870be700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc480156750 con 0x7fc4800b65d0 2023-08-27T11:12:04.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc4870be700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc480156a30 con 0x7fc4800d0520 2023-08-27T11:12:04.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc47ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800cb5c0 0x7fc480155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc47ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800cb5c0 0x7fc480155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51272/0 (socket says 172.21.15.45:51272) 2023-08-27T11:12:04.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc47ffff700 1 -- 172.21.15.45:0/1417032607 learned_addr learned my addr 172.21.15.45:0/1417032607 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:04.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc48565b700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800d0520 0x7fc480159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc484e5a700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4800b65d0 0x7fc4801533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc47ffff700 1 -- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800d0520 msgr2=0x7fc480159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc47ffff700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800d0520 0x7fc480159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc47ffff700 1 -- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4800b65d0 msgr2=0x7fc4801533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc47ffff700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4800b65d0 0x7fc4801533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.908+0000 7fc47ffff700 1 -- 172.21.15.45:0/1417032607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c009550 con 0x7fc4800cb5c0 2023-08-27T11:12:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.909+0000 7fc47ffff700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800cb5c0 0x7fc480155c90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009130 tx=0x7fc46c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.909+0000 7fc47d7fa700 1 -- 172.21.15.45:0/1417032607 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c00b660 con 0x7fc4800cb5c0 2023-08-27T11:12:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.909+0000 7fc47d7fa700 1 -- 172.21.15.45:0/1417032607 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c00b7c0 con 0x7fc4800cb5c0 2023-08-27T11:12:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.909+0000 7fc47d7fa700 1 -- 172.21.15.45:0/1417032607 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c00bb70 con 0x7fc4800cb5c0 2023-08-27T11:12:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.909+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc480156d10 con 0x7fc4800cb5c0 2023-08-27T11:12:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.909+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4800c7660 con 0x7fc4800cb5c0 2023-08-27T11:12:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.910+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4800c48f0 con 0x7fc4800cb5c0 2023-08-27T11:12:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.910+0000 7fc47d7fa700 1 -- 172.21.15.45:0/1417032607 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc46c00bcd0 con 0x7fc4800cb5c0 2023-08-27T11:12:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.911+0000 7fc47d7fa700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc46406a0e0 0x7fc46406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.911+0000 7fc47d7fa700 1 -- 172.21.15.45:0/1417032607 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc46c089a40 con 0x7fc4800cb5c0 2023-08-27T11:12:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.911+0000 7fc484e5a700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc46406a0e0 0x7fc46406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.911+0000 7fc484e5a700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc46406a0e0 0x7fc46406c5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc474007fd0 tx=0x7fc474007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:04.915+0000 7fc47d7fa700 1 -- 172.21.15.45:0/1417032607 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc4800c48f0 con 0x7fc4800cb5c0 2023-08-27T11:12:05.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.115+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4800c48f0 con 0x7fc4800cb5c0 2023-08-27T11:12:05.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.116+0000 7fc47d7fa700 1 -- 172.21.15.45:0/1417032607 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc46c029020 con 0x7fc4800cb5c0 2023-08-27T11:12:05.117 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:12:05.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.117+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc46406a0e0 msgr2=0x7fc46406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.117+0000 7fc4870be700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc46406a0e0 0x7fc46406c5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc474007fd0 tx=0x7fc474007ec0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800cb5c0 msgr2=0x7fc480155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800cb5c0 0x7fc480155c90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009130 tx=0x7fc46c004750 comp rx=0 tx=0).stop 2023-08-27T11:12:05.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 shutdown_connections 2023-08-27T11:12:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc46406a0e0 0x7fc46406c5a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4800d0520 0x7fc480159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4800b65d0 0x7fc4801533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 --2- 172.21.15.45:0/1417032607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4800cb5c0 0x7fc480155c90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 >> 172.21.15.45:0/1417032607 conn(0x7fc48006f960 msgr2=0x7fc48015d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 shutdown_connections 2023-08-27T11:12:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.118+0000 7fc4870be700 1 -- 172.21.15.45:0/1417032607 wait complete. 2023-08-27T11:12:05.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:12:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.258+0000 7fa57bb3d700 1 Processor -- start 2023-08-27T11:12:05.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.258+0000 7fa57bb3d700 1 -- start start 2023-08-27T11:12:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57bb3d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740b02c0 0x7fa5740cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57bb3d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5740cdf10 0x7fa57415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57bb3d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740d04f0 con 0x7fa57406c260 2023-08-27T11:12:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57bb3d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5740b06a0 con 0x7fa5740cdf10 2023-08-27T11:12:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57bb3d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740cf620 con 0x7fa5740b02c0 2023-08-27T11:12:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57a0da700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5740cdf10 0x7fa57415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57a0da700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5740cdf10 0x7fa57415ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51284/0 (socket says 172.21.15.45:51284) 2023-08-27T11:12:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57a0da700 1 -- 172.21.15.45:0/4207996168 learned_addr learned my addr 172.21.15.45:0/4207996168 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa5798d9700 1 --2- 172.21.15.45:0/4207996168 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740b02c0 0x7fa5740cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57a0da700 1 -- 172.21.15.45:0/4207996168 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740b02c0 msgr2=0x7fa5740cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57a0da700 1 --2- 172.21.15.45:0/4207996168 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740b02c0 0x7fa5740cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57a0da700 1 -- 172.21.15.45:0/4207996168 >> v1:172.21.15.135:6789/0 conn(0x7fa57406c260 legacy=0x7fa5740cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57a0da700 1 -- 172.21.15.45:0/4207996168 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740cf8e0 con 0x7fa5740cdf10 2023-08-27T11:12:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa57a0da700 1 --2- 172.21.15.45:0/4207996168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740cdf10 0x7fa57415ca80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009e50 tx=0x7fa56c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4273d4427e5843 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa5788d7700 1 -- 172.21.15.45:0/4207996168 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00e070 con 0x7fa5740cdf10 2023-08-27T11:12:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa5788d7700 1 -- 172.21.15.45:0/4207996168 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c00b9c0 con 0x7fa5740cdf10 2023-08-27T11:12:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.259+0000 7fa5788d7700 1 -- 172.21.15.45:0/4207996168 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00bd70 con 0x7fa5740cdf10 2023-08-27T11:12:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.260+0000 7fa57bb3d700 1 -- 172.21.15.45:0/4207996168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740cdf10 msgr2=0x7fa57415ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.260+0000 7fa57bb3d700 1 --2- 172.21.15.45:0/4207996168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740cdf10 0x7fa57415ca80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009e50 tx=0x7fa56c00b330 comp rx=0 tx=0).stop 2023-08-27T11:12:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.260+0000 7fa57bb3d700 1 -- 172.21.15.45:0/4207996168 shutdown_connections 2023-08-27T11:12:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.260+0000 7fa57bb3d700 1 --2- 172.21.15.45:0/4207996168 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740b02c0 0x7fa5740cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.260+0000 7fa57bb3d700 1 --2- 172.21.15.45:0/4207996168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740cdf10 0x7fa57415ca80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.260+0000 7fa57bb3d700 1 -- 172.21.15.45:0/4207996168 >> 172.21.15.45:0/4207996168 conn(0x7fa57409a1a0 msgr2=0x7fa5740d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:05.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.260+0000 7fa57bb3d700 1 -- 172.21.15.45:0/4207996168 shutdown_connections 2023-08-27T11:12:05.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.260+0000 7fa57bb3d700 1 -- 172.21.15.45:0/4207996168 wait complete. 2023-08-27T11:12:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.261+0000 7fa57bb3d700 1 Processor -- start 2023-08-27T11:12:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.261+0000 7fa57bb3d700 1 -- start start 2023-08-27T11:12:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa57bb3d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa57406c260 0x7fa57409e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa57bb3d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740b02c0 0x7fa57415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa57bb3d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740cdf10 0x7fa57406eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa57bb3d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741f6600 con 0x7fa5740b02c0 2023-08-27T11:12:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa57bb3d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741f5d10 con 0x7fa57406c260 2023-08-27T11:12:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa57bb3d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57415bb60 con 0x7fa5740cdf10 2023-08-27T11:12:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5790d8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740b02c0 0x7fa57415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5790d8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740b02c0 0x7fa57415a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51288/0 (socket says 172.21.15.45:51288) 2023-08-27T11:12:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5790d8700 1 -- 172.21.15.45:0/1682495967 learned_addr learned my addr 172.21.15.45:0/1682495967 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5798d9700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa57406c260 0x7fa57409e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa57a0da700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740cdf10 0x7fa57406eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5790d8700 1 -- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740cdf10 msgr2=0x7fa57406eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5790d8700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740cdf10 0x7fa57406eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5790d8700 1 -- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa57406c260 msgr2=0x7fa57409e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5790d8700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa57406c260 0x7fa57409e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5790d8700 1 -- 172.21.15.45:0/1682495967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c009b30 con 0x7fa5740b02c0 2023-08-27T11:12:05.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.262+0000 7fa5790d8700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740b02c0 0x7fa57415a3b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa56400c4e0 tx=0x7fa56400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:05.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.263+0000 7fa56a7fc700 1 -- 172.21.15.45:0/1682495967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640052d0 con 0x7fa5740b02c0 2023-08-27T11:12:05.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.263+0000 7fa56a7fc700 1 -- 172.21.15.45:0/1682495967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564005430 con 0x7fa5740b02c0 2023-08-27T11:12:05.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.263+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5741f5fb0 con 0x7fa5740b02c0 2023-08-27T11:12:05.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.263+0000 7fa56a7fc700 1 -- 172.21.15.45:0/1682495967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640038c0 con 0x7fa5740b02c0 2023-08-27T11:12:05.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.263+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57415a8f0 con 0x7fa5740b02c0 2023-08-27T11:12:05.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.264+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5740734a0 con 0x7fa5740b02c0 2023-08-27T11:12:05.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.264+0000 7fa56a7fc700 1 -- 172.21.15.45:0/1682495967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa564003a20 con 0x7fa5740b02c0 2023-08-27T11:12:05.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.264+0000 7fa56a7fc700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa55806a0e0 0x7fa55806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.265+0000 7fa56a7fc700 1 -- 172.21.15.45:0/1682495967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa564087580 con 0x7fa5740b02c0 2023-08-27T11:12:05.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.265+0000 7fa5798d9700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa55806a0e0 0x7fa55806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.265+0000 7fa5798d9700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa55806a0e0 0x7fa55806c5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa57409e190 tx=0x7fa570008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:05.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.269+0000 7fa56a7fc700 1 -- 172.21.15.45:0/1682495967 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa564052fc0 con 0x7fa5740b02c0 2023-08-27T11:12:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.470+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa57406c930 con 0x7fa5740b02c0 2023-08-27T11:12:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.471+0000 7fa56a7fc700 1 -- 172.21.15.45:0/1682495967 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa5640565e0 con 0x7fa5740b02c0 2023-08-27T11:12:05.472 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:12:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa55806a0e0 msgr2=0x7fa55806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa55806a0e0 0x7fa55806c5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa57409e190 tx=0x7fa570008040 comp rx=0 tx=0).stop 2023-08-27T11:12:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740b02c0 msgr2=0x7fa57415a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740b02c0 0x7fa57415a3b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa56400c4e0 tx=0x7fa56400c7f0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 shutdown_connections 2023-08-27T11:12:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa55806a0e0 0x7fa55806c5a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5740cdf10 0x7fa57406eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa57406c260 0x7fa57409e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 --2- 172.21.15.45:0/1682495967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5740b02c0 0x7fa57415a3b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 >> 172.21.15.45:0/1682495967 conn(0x7fa57409a1a0 msgr2=0x7fa5740b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 shutdown_connections 2023-08-27T11:12:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.473+0000 7fa57bb3d700 1 -- 172.21.15.45:0/1682495967 wait complete. 2023-08-27T11:12:05.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:12:05.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.609+0000 7f44d1f06700 1 Processor -- start 2023-08-27T11:12:05.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.609+0000 7f44d1f06700 1 -- start start 2023-08-27T11:12:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.610+0000 7f44d1f06700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f44cc15d2c0 0x7f44cc0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.610+0000 7f44d1f06700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0b8c20 0x7f44cc0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.610+0000 7f44d1f06700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc15d6a0 con 0x7f44cc0ce2e0 2023-08-27T11:12:05.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.610+0000 7f44d1f06700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44cc0ce770 con 0x7f44cc15d2c0 2023-08-27T11:12:05.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.610+0000 7f44d1f06700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0d2c90 con 0x7f44cc0b8c20 2023-08-27T11:12:05.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.610+0000 7f44cbfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f44cc0ce2e0 0x7f44cc06c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38074/0 (socket says 172.21.15.45:38074) 2023-08-27T11:12:05.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.610+0000 7f44cbfff700 1 -- 172.21.15.45:0/230299336 learned_addr learned my addr 172.21.15.45:0/230299336 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.610+0000 7f44cb7fe700 1 --2- 172.21.15.45:0/230299336 >> v2:172.21.15.135:3300/0 conn(0x7f44cc15d2c0 0x7f44cc0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.610+0000 7f44caffd700 1 --2- 172.21.15.45:0/230299336 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0b8c20 0x7f44cc0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44ca7fc700 1 -- 172.21.15.45:0/230299336 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270074892 0 0) 0x7f44c4001110 con 0x7f44cc0ce2e0 2023-08-27T11:12:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44ca7fc700 1 -- 172.21.15.45:0/230299336 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b40035b0 con 0x7f44cc0ce2e0 2023-08-27T11:12:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44caffd700 1 -- 172.21.15.45:0/230299336 >> v2:172.21.15.135:3300/0 conn(0x7f44cc15d2c0 msgr2=0x7f44cc0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44caffd700 1 --2- 172.21.15.45:0/230299336 >> v2:172.21.15.135:3300/0 conn(0x7f44cc15d2c0 0x7f44cc0d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44caffd700 1 -- 172.21.15.45:0/230299336 >> v1:172.21.15.45:6789/0 conn(0x7f44cc0ce2e0 legacy=0x7f44cc06c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44ca7fc700 1 -- 172.21.15.45:0/230299336 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782477549 0 0) 0x7f44b40035b0 con 0x7f44cc0ce2e0 2023-08-27T11:12:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44caffd700 1 -- 172.21.15.45:0/230299336 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0d1bf0 con 0x7f44cc0b8c20 2023-08-27T11:12:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44caffd700 1 --2- 172.21.15.45:0/230299336 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0b8c20 0x7f44cc0cdda0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009d90 tx=0x7f44c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de69f9db0580beef server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44ca7fc700 1 -- 172.21.15.45:0/230299336 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c000e070 con 0x7f44cc0b8c20 2023-08-27T11:12:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44ca7fc700 1 -- 172.21.15.45:0/230299336 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c0003a20 con 0x7f44cc0b8c20 2023-08-27T11:12:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.611+0000 7f44ca7fc700 1 -- 172.21.15.45:0/230299336 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4001800 con 0x7f44cc0b8c20 2023-08-27T11:12:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.612+0000 7f44d1f06700 1 -- 172.21.15.45:0/230299336 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0b8c20 msgr2=0x7f44cc0cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.612+0000 7f44d1f06700 1 --2- 172.21.15.45:0/230299336 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0b8c20 0x7f44cc0cdda0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009d90 tx=0x7f44c000b330 comp rx=0 tx=0).stop 2023-08-27T11:12:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.612+0000 7f44d1f06700 1 -- 172.21.15.45:0/230299336 shutdown_connections 2023-08-27T11:12:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.612+0000 7f44d1f06700 1 --2- 172.21.15.45:0/230299336 >> v2:172.21.15.135:3300/0 conn(0x7f44cc15d2c0 0x7f44cc0d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.612+0000 7f44d1f06700 1 --2- 172.21.15.45:0/230299336 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0b8c20 0x7f44cc0cdda0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.612+0000 7f44d1f06700 1 -- 172.21.15.45:0/230299336 >> 172.21.15.45:0/230299336 conn(0x7f44cc0d2fd0 msgr2=0x7f44cc0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.612+0000 7f44d1f06700 1 -- 172.21.15.45:0/230299336 shutdown_connections 2023-08-27T11:12:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.612+0000 7f44d1f06700 1 -- 172.21.15.45:0/230299336 wait complete. 2023-08-27T11:12:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.613+0000 7f44d1f06700 1 Processor -- start 2023-08-27T11:12:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.613+0000 7f44d1f06700 1 -- start start 2023-08-27T11:12:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44d1f06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc0b8c20 0x7f44cc14b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44d1f06700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0ce2e0 0x7f44cc0cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44d1f06700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44cc15d2c0 0x7f44cc15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44d1f06700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0c85e0 con 0x7f44cc0b8c20 2023-08-27T11:12:05.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44d1f06700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0c79f0 con 0x7f44cc15d2c0 2023-08-27T11:12:05.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44d1f06700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc14bcc0 con 0x7f44cc0ce2e0 2023-08-27T11:12:05.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc0b8c20 0x7f44cc14b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc0b8c20 0x7f44cc14b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51300/0 (socket says 172.21.15.45:51300) 2023-08-27T11:12:05.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cb7fe700 1 -- 172.21.15.45:0/2532171666 learned_addr learned my addr 172.21.15.45:0/2532171666 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:05.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44caffd700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0ce2e0 0x7f44cc0cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cbfff700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44cc15d2c0 0x7f44cc15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cb7fe700 1 -- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0ce2e0 msgr2=0x7f44cc0cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cb7fe700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0ce2e0 0x7f44cc0cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cb7fe700 1 -- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44cc15d2c0 msgr2=0x7f44cc15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cb7fe700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44cc15d2c0 0x7f44cc15bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cb7fe700 1 -- 172.21.15.45:0/2532171666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c0009a70 con 0x7f44cc0b8c20 2023-08-27T11:12:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.614+0000 7f44cb7fe700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc0b8c20 0x7f44cc14b780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e8a0 tx=0x7f44bc00ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.615+0000 7f44b3fff700 1 -- 172.21.15.45:0/2532171666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc009590 con 0x7f44cc0b8c20 2023-08-27T11:12:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.615+0000 7f44b3fff700 1 -- 172.21.15.45:0/2532171666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc013680 con 0x7f44cc0b8c20 2023-08-27T11:12:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.615+0000 7f44b3fff700 1 -- 172.21.15.45:0/2532171666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0139f0 con 0x7f44cc0b8c20 2023-08-27T11:12:05.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.615+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc0c7710 con 0x7f44cc0b8c20 2023-08-27T11:12:05.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.615+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc0cb5a0 con 0x7f44cc0b8c20 2023-08-27T11:12:05.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.615+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc04e620 con 0x7f44cc0b8c20 2023-08-27T11:12:05.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.616+0000 7f44b3fff700 1 -- 172.21.15.45:0/2532171666 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f44bc013b50 con 0x7f44cc0b8c20 2023-08-27T11:12:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.616+0000 7f44b3fff700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f44b806a0e0 0x7f44b806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.616+0000 7f44b3fff700 1 -- 172.21.15.45:0/2532171666 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f44bc088020 con 0x7f44cc0b8c20 2023-08-27T11:12:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.617+0000 7f44caffd700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f44b806a0e0 0x7f44b806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.617+0000 7f44caffd700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f44b806a0e0 0x7f44b806c5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44c000b370 tx=0x7f44c0011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.621+0000 7f44b3fff700 1 -- 172.21.15.45:0/2532171666 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f44bc0574f0 con 0x7f44cc0b8c20 2023-08-27T11:12:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.825+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f44cc1598d0 con 0x7f44cc0b8c20 2023-08-27T11:12:05.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.826+0000 7f44b3fff700 1 -- 172.21.15.45:0/2532171666 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f44bc021090 con 0x7f44cc0b8c20 2023-08-27T11:12:05.827 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:05.827 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:12:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:12:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-08-27T11:12:05.828 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-08-27T11:12:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:12:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:12:05.829 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-08-27T11:12:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-08-27T11:12:05.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:12:05.830 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:12:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f44b806a0e0 msgr2=0x7f44b806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f44b806a0e0 0x7f44b806c5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44c000b370 tx=0x7f44c0011000 comp rx=0 tx=0).stop 2023-08-27T11:12:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc0b8c20 msgr2=0x7f44cc14b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc0b8c20 0x7f44cc14b780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00e8a0 tx=0x7f44bc00ebb0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 shutdown_connections 2023-08-27T11:12:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44cc0b8c20 0x7f44cc14b780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f44b806a0e0 0x7f44b806c5a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44cc15d2c0 0x7f44cc15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 --2- 172.21.15.45:0/2532171666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f44cc0ce2e0 0x7f44cc0cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 >> 172.21.15.45:0/2532171666 conn(0x7f44cc0d2fd0 msgr2=0x7f44cc0b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 shutdown_connections 2023-08-27T11:12:05.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.828+0000 7f44d1f06700 1 -- 172.21.15.45:0/2532171666 wait complete. 2023-08-27T11:12:05.841 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:12:05.841 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:12:05.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.969+0000 7f988335e700 1 Processor -- start 2023-08-27T11:12:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.970+0000 7f988335e700 1 -- start start 2023-08-27T11:12:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.970+0000 7f988335e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c157be0 0x7f987c0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.970+0000 7f988335e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f987c0b17c0 0x7f987c0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.970+0000 7f988335e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987c15b460 con 0x7f987c06ced0 2023-08-27T11:12:05.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.970+0000 7f988335e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f987c157fc0 con 0x7f987c0b17c0 2023-08-27T11:12:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.970+0000 7f988335e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987c0b2280 con 0x7f987c157be0 2023-08-27T11:12:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.970+0000 7f98818fb700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f987c06ced0 0x7f987c0cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38088/0 (socket says 172.21.15.45:38088) 2023-08-27T11:12:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.970+0000 7f98818fb700 1 -- 172.21.15.45:0/3016756817 learned_addr learned my addr 172.21.15.45:0/3016756817 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259474479 0 0) 0x7f987c15b460 con 0x7f987c06ced0 2023-08-27T11:12:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98680035b0 con 0x7f987c06ced0 2023-08-27T11:12:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98808f9700 1 --2- 172.21.15.45:0/3016756817 >> v2:172.21.15.135:3300/0 conn(0x7f987c0b17c0 0x7f987c0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98810fa700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c157be0 0x7f987c0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102608098 0 0) 0x7f98680035b0 con 0x7f987c06ced0 2023-08-27T11:12:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c157be0 msgr2=0x7f987c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c157be0 0x7f987c0716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 >> v2:172.21.15.135:3300/0 conn(0x7f987c0b17c0 msgr2=0x7f987c0b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 --2- 172.21.15.45:0/3016756817 >> v2:172.21.15.135:3300/0 conn(0x7f987c0b17c0 0x7f987c0b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c0b2560 con 0x7f987c06ced0 2023-08-27T11:12:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f98740019f0 con 0x7f987c06ced0 2023-08-27T11:12:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98808f9700 1 --2- 172.21.15.45:0/3016756817 >> v2:172.21.15.135:3300/0 conn(0x7f987c0b17c0 0x7f987c0b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 >> v1:172.21.15.45:6789/0 conn(0x7f987c06ced0 legacy=0x7f987c0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9868003a00 0x7f9868003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9868004850 0x7f9868004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868005210 0x7f986800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c157fc0 con 0x7f9868005210 2023-08-27T11:12:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c0b2280 con 0x7f9868004850 2023-08-27T11:12:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986800c050 con 0x7f9868003a00 2023-08-27T11:12:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98810fa700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c157be0 0x7f987c0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98818fb700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868005210 0x7f986800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98808f9700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9868004850 0x7f9868004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98810fa700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9868003a00 0x7f9868003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98818fb700 1 -- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9868003a00 msgr2=0x7f9868003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98818fb700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9868003a00 0x7f9868003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98818fb700 1 -- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9868004850 msgr2=0x7f9868004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98818fb700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9868004850 0x7f9868004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98818fb700 1 -- 172.21.15.45:0/3016756817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680035b0 con 0x7f9868005210 2023-08-27T11:12:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.971+0000 7f98808f9700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9868004850 0x7f9868004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:05.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f98818fb700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868005210 0x7f986800ba10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f987400a890 tx=0x7f9868005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b391364336d7d22 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874012600 con 0x7f9868005210 2023-08-27T11:12:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9874012780 con 0x7f9868005210 2023-08-27T11:12:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f9873fff700 1 -- 172.21.15.45:0/3016756817 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874012b30 con 0x7f9868005210 2023-08-27T11:12:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f988335e700 1 -- 172.21.15.45:0/3016756817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868005210 msgr2=0x7f986800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f988335e700 1 --2- 172.21.15.45:0/3016756817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9868005210 0x7f986800ba10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f987400a890 tx=0x7f9868005880 comp rx=0 tx=0).stop 2023-08-27T11:12:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f98818fb700 1 -- 172.21.15.45:0/3016756817 reap_dead start 2023-08-27T11:12:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f988335e700 1 -- 172.21.15.45:0/3016756817 shutdown_connections 2023-08-27T11:12:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f988335e700 1 -- 172.21.15.45:0/3016756817 >> 172.21.15.45:0/3016756817 conn(0x7f987c0cc7c0 msgr2=0x7f987c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f988335e700 1 -- 172.21.15.45:0/3016756817 shutdown_connections 2023-08-27T11:12:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.972+0000 7f988335e700 1 -- 172.21.15.45:0/3016756817 wait complete. 2023-08-27T11:12:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.974+0000 7f988335e700 1 Processor -- start 2023-08-27T11:12:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.974+0000 7f988335e700 1 -- start start 2023-08-27T11:12:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f988335e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c1efea0 0x7f987c159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f988335e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1f16e0 0x7f987c1f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f988335e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c0b0000 0x7f987c0b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f988335e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c0b09d0 con 0x7f987c1efea0 2023-08-27T11:12:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f988335e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c0c41d0 con 0x7f987c1f16e0 2023-08-27T11:12:05.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f988335e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987c1f0ad0 con 0x7f987c0b0000 2023-08-27T11:12:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98810fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c1efea0 0x7f987c159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98810fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c1efea0 0x7f987c159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51320/0 (socket says 172.21.15.45:51320) 2023-08-27T11:12:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98810fa700 1 -- 172.21.15.45:0/3015304638 learned_addr learned my addr 172.21.15.45:0/3015304638 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98808f9700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1f16e0 0x7f987c1f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98818fb700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c0b0000 0x7f987c0b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98810fa700 1 -- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c0b0000 msgr2=0x7f987c0b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98810fa700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c0b0000 0x7f987c0b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98810fa700 1 -- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1f16e0 msgr2=0x7f987c1f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98810fa700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1f16e0 0x7f987c1f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98810fa700 1 -- 172.21.15.45:0/3015304638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c1f0db0 con 0x7f987c1efea0 2023-08-27T11:12:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98818fb700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c0b0000 0x7f987c0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.975+0000 7f98808f9700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1f16e0 0x7f987c1f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.976+0000 7f98810fa700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c1efea0 0x7f987c159b60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f986c00f4c0 tx=0x7f986c00f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.976+0000 7f9871ffb700 1 -- 172.21.15.45:0/3015304638 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c00c070 con 0x7f987c1efea0 2023-08-27T11:12:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.976+0000 7f9871ffb700 1 -- 172.21.15.45:0/3015304638 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c014890 con 0x7f987c1efea0 2023-08-27T11:12:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.976+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987c1f1090 con 0x7f987c1efea0 2023-08-27T11:12:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.976+0000 7f9871ffb700 1 -- 172.21.15.45:0/3015304638 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c014c40 con 0x7f987c1efea0 2023-08-27T11:12:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.976+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c1f07c0 con 0x7f987c1efea0 2023-08-27T11:12:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.977+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987c072690 con 0x7f987c1efea0 2023-08-27T11:12:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.977+0000 7f9871ffb700 1 -- 172.21.15.45:0/3015304638 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f986c01d400 con 0x7f987c1efea0 2023-08-27T11:12:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.978+0000 7f9871ffb700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f986006a0e0 0x7f986006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.978+0000 7f9871ffb700 1 -- 172.21.15.45:0/3015304638 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f986c088420 con 0x7f987c1efea0 2023-08-27T11:12:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.978+0000 7f98808f9700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f986006a0e0 0x7f986006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.979+0000 7f98808f9700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f986006a0e0 0x7f986006c5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9878009970 tx=0x7f9878008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:05.982+0000 7f9871ffb700 1 -- 172.21.15.45:0/3015304638 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f986c053ef0 con 0x7f987c1efea0 2023-08-27T11:12:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.185+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f987c0734a0 con 0x7f987c1efea0 2023-08-27T11:12:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.882+0000 7f9871ffb700 1 -- 172.21.15.45:0/3015304638 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f986c057510 con 0x7f987c1efea0 2023-08-27T11:12:06.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.886+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f986006a0e0 msgr2=0x7f986006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:06.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.886+0000 7f988335e700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f986006a0e0 0x7f986006c5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9878009970 tx=0x7f9878008040 comp rx=0 tx=0).stop 2023-08-27T11:12:06.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.886+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c1efea0 msgr2=0x7f987c159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.886+0000 7f988335e700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c1efea0 0x7f987c159b60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f986c00f4c0 tx=0x7f986c00f880 comp rx=0 tx=0).stop 2023-08-27T11:12:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.887+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 shutdown_connections 2023-08-27T11:12:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.887+0000 7f988335e700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f987c1efea0 0x7f987c159b60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.887+0000 7f988335e700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f986006a0e0 0x7f986006c5a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.887+0000 7f988335e700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f987c0b0000 0x7f987c0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.887+0000 7f988335e700 1 --2- 172.21.15.45:0/3015304638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1f16e0 0x7f987c1f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.887+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 >> 172.21.15.45:0/3015304638 conn(0x7f987c0cc7c0 msgr2=0x7f987c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.887+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 shutdown_connections 2023-08-27T11:12:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:06.887+0000 7f988335e700 1 -- 172.21.15.45:0/3015304638 wait complete. 2023-08-27T11:12:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-08-27T11:12:06.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-08-27T11:12:07.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.045+0000 7f7ae6cf3700 1 Processor -- start 2023-08-27T11:12:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.045+0000 7f7ae6cf3700 1 -- start start 2023-08-27T11:12:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7ae6cf3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae0160d00 0x7f7ae006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7ae6cf3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae006b160 con 0x7f7ae006a7c0 2023-08-27T11:12:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7ae6cf3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae00cdf90 con 0x7f7ae015d340 2023-08-27T11:12:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7ae6cf3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae0072010 con 0x7f7ae0160d00 2023-08-27T11:12:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7ae5290700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7ae006a7c0 0x7f7ae00b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38108/0 (socket says 172.21.15.45:38108) 2023-08-27T11:12:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7ae5290700 1 -- 172.21.15.45:0/861399101 learned_addr learned my addr 172.21.15.45:0/861399101 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adffff700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae0160d00 0x7f7ae006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:07.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019394014 0 0) 0x7f7ae006b160 con 0x7f7ae006a7c0 2023-08-27T11:12:07.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad00035b0 con 0x7f7ae006a7c0 2023-08-27T11:12:07.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3886362175 0 0) 0x7f7ad00035b0 con 0x7f7ae006a7c0 2023-08-27T11:12:07.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae0160d00 msgr2=0x7f7ae006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:07.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae0160d00 0x7f7ae006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:07.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 >> v1:172.21.15.135:6789/0 conn(0x7f7ae015d340 legacy=0x7f7ae00cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:07.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00af590 con 0x7f7ae006a7c0 2023-08-27T11:12:07.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f7ad80019f0 con 0x7f7ae006a7c0 2023-08-27T11:12:07.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 >> v1:172.21.15.45:6789/0 conn(0x7f7ae006a7c0 legacy=0x7f7ae00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:07.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.046+0000 7f7adf7fe700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ad0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:07.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7adf7fe700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ad00043e0 0x7f7ad00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7adf7fe700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae0160d00 0x7f7ad00051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae0072010 con 0x7f7ad0003a30 2023-08-27T11:12:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad000bac0 con 0x7f7ad00043e0 2023-08-27T11:12:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad000bc80 con 0x7f7ae0160d00 2023-08-27T11:12:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7ae4a8f700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ad0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7ae5290700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae0160d00 0x7f7ad00051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7adffff700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ad00043e0 0x7f7ad00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7ae4a8f700 1 -- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae0160d00 msgr2=0x7f7ad00051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7ae4a8f700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae0160d00 0x7f7ad00051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7ae4a8f700 1 -- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ad00043e0 msgr2=0x7f7ad00048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7ae4a8f700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ad00043e0 0x7f7ad00048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7ae4a8f700 1 -- 172.21.15.45:0/861399101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad00035b0 con 0x7f7ad0003a30 2023-08-27T11:12:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7ae4a8f700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ad0003e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40018d0 tx=0x7f7ad0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d4d8bd986ffa66 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400c690 con 0x7f7ad0003a30 2023-08-27T11:12:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad400c810 con 0x7f7ad0003a30 2023-08-27T11:12:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.047+0000 7f7adf7fe700 1 -- 172.21.15.45:0/861399101 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400cbc0 con 0x7f7ad0003a30 2023-08-27T11:12:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.048+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/861399101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 msgr2=0x7f7ad0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.048+0000 7f7ae6cf3700 1 --2- 172.21.15.45:0/861399101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ad0003e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40018d0 tx=0x7f7ad0005880 comp rx=0 tx=0).stop 2023-08-27T11:12:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.048+0000 7f7ae5290700 1 -- 172.21.15.45:0/861399101 reap_dead start 2023-08-27T11:12:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.048+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/861399101 shutdown_connections 2023-08-27T11:12:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.048+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/861399101 >> 172.21.15.45:0/861399101 conn(0x7f7ae00cc7c0 msgr2=0x7f7ae00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.048+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/861399101 shutdown_connections 2023-08-27T11:12:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.048+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/861399101 wait complete. 2023-08-27T11:12:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.049+0000 7f7ae6cf3700 1 Processor -- start 2023-08-27T11:12:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.049+0000 7f7ae6cf3700 1 -- start start 2023-08-27T11:12:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae6cf3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ae0158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae6cf3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae01f27a0 0x7f7ae015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae6cf3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae00ca440 0x7f7ae015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae6cf3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae0151b30 con 0x7f7ad0003a30 2023-08-27T11:12:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae6cf3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00c75b0 con 0x7f7ae00ca440 2023-08-27T11:12:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae6cf3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae01597d0 con 0x7f7ae01f27a0 2023-08-27T11:12:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae4a8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ae0158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae4a8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ae0158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51340/0 (socket says 172.21.15.45:51340) 2023-08-27T11:12:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae4a8f700 1 -- 172.21.15.45:0/1489416806 learned_addr learned my addr 172.21.15.45:0/1489416806 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7adffff700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae01f27a0 0x7f7ae015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae5290700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae00ca440 0x7f7ae015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae4a8f700 1 -- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae01f27a0 msgr2=0x7f7ae015d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae4a8f700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae01f27a0 0x7f7ae015d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae4a8f700 1 -- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae00ca440 msgr2=0x7f7ae015dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae4a8f700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae00ca440 0x7f7ae015dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae4a8f700 1 -- 172.21.15.45:0/1489416806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0159ab0 con 0x7f7ad0003a30 2023-08-27T11:12:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.050+0000 7f7ae4a8f700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ae0158e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400c0e0 tx=0x7f7ad4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.051+0000 7f7add7fa700 1 -- 172.21.15.45:0/1489416806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4001330 con 0x7f7ad0003a30 2023-08-27T11:12:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.051+0000 7f7add7fa700 1 -- 172.21.15.45:0/1489416806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad4021070 con 0x7f7ad0003a30 2023-08-27T11:12:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.051+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae0159d90 con 0x7f7ad0003a30 2023-08-27T11:12:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.051+0000 7f7add7fa700 1 -- 172.21.15.45:0/1489416806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400cbc0 con 0x7f7ad0003a30 2023-08-27T11:12:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.051+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae0159390 con 0x7f7ad0003a30 2023-08-27T11:12:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.052+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae0159fa0 con 0x7f7ad0003a30 2023-08-27T11:12:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.052+0000 7f7add7fa700 1 -- 172.21.15.45:0/1489416806 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ad4023030 con 0x7f7ad0003a30 2023-08-27T11:12:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.053+0000 7f7add7fa700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ac406a0e0 0x7f7ac406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.053+0000 7f7add7fa700 1 -- 172.21.15.45:0/1489416806 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f7ad40893a0 con 0x7f7ad0003a30 2023-08-27T11:12:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.053+0000 7f7adffff700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ac406a0e0 0x7f7ac406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.053+0000 7f7adffff700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ac406a0e0 0x7f7ac406c5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ae015e380 tx=0x7f7acc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.057+0000 7f7add7fa700 1 -- 172.21.15.45:0/1489416806 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7ad4054d00 con 0x7f7ad0003a30 2023-08-27T11:12:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.257+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f7ae0159520 con 0x7f7ad0003a30 2023-08-27T11:12:07.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.884+0000 7f7add7fa700 1 -- 172.21.15.45:0/1489416806 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) v1 ==== 171+0+0 (secure 0 0 0) 0x7f7ad4058320 con 0x7f7ad0003a30 2023-08-27T11:12:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.892+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ac406a0e0 msgr2=0x7f7ac406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.893+0000 7f7ae6cf3700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ac406a0e0 0x7f7ac406c5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ae015e380 tx=0x7f7acc009210 comp rx=0 tx=0).stop 2023-08-27T11:12:07.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.893+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 msgr2=0x7f7ae0158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:07.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.893+0000 7f7ae6cf3700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ae0158e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400c0e0 tx=0x7f7ad4001230 comp rx=0 tx=0).stop 2023-08-27T11:12:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.895+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 shutdown_connections 2023-08-27T11:12:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.895+0000 7f7ae6cf3700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ad0003a30 0x7f7ae0158e50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.895+0000 7f7ae6cf3700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ac406a0e0 0x7f7ac406c5a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.895+0000 7f7ae6cf3700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ae00ca440 0x7f7ae015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.895+0000 7f7ae6cf3700 1 --2- 172.21.15.45:0/1489416806 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ae01f27a0 0x7f7ae015d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.895+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 >> 172.21.15.45:0/1489416806 conn(0x7f7ae00cc7c0 msgr2=0x7f7ae015b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.896+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 shutdown_connections 2023-08-27T11:12:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:07.897+0000 7f7ae6cf3700 1 -- 172.21.15.45:0/1489416806 wait complete. 2023-08-27T11:12:07.903 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-08-27T11:12:07.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:12:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.052+0000 7f0ce12f1700 1 Processor -- start 2023-08-27T11:12:08.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.052+0000 7f0ce12f1700 1 -- start start 2023-08-27T11:12:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.053+0000 7f0ce12f1700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0cdc0cb5d0 0x7f0cdc0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.053+0000 7f0ce12f1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc0b2e40 0x7f0cdc06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.053+0000 7f0ce12f1700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc15f8f0 con 0x7f0cdc15b090 2023-08-27T11:12:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.053+0000 7f0ce12f1700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cdc0b32c0 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.053+0000 7f0ce12f1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc0d0c20 con 0x7f0cdc0b2e40 2023-08-27T11:12:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.053+0000 7f0cdb7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc0b2e40 0x7f0cdc06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.053+0000 7f0cda7fc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0cdc0cb5d0 0x7f0cdc0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.053+0000 7f0cdb7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc0b2e40 0x7f0cdc06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:49262/0 (socket says 172.21.15.45:49262) 2023-08-27T11:12:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.053+0000 7f0cdb7fe700 1 -- 172.21.15.45:0/1406400017 learned_addr learned my addr 172.21.15.45:0/1406400017 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0cdb7fe700 1 -- 172.21.15.45:0/1406400017 >> v2:172.21.15.135:3300/0 conn(0x7f0cdc0cb5d0 msgr2=0x7f0cdc0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:08.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0cdb7fe700 1 --2- 172.21.15.45:0/1406400017 >> v2:172.21.15.135:3300/0 conn(0x7f0cdc0cb5d0 0x7f0cdc0b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0cdb7fe700 1 -- 172.21.15.45:0/1406400017 >> v1:172.21.15.135:6789/0 conn(0x7f0cdc15b090 legacy=0x7f0cdc0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:08.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0cdb7fe700 1 -- 172.21.15.45:0/1406400017 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc15ece0 con 0x7f0cdc0b2e40 2023-08-27T11:12:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0cda7fc700 1 --2- 172.21.15.45:0/1406400017 >> v2:172.21.15.135:3300/0 conn(0x7f0cdc0cb5d0 0x7f0cdc0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0cdb7fe700 1 --2- 172.21.15.45:0/1406400017 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc0b2e40 0x7f0cdc06dc90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009e50 tx=0x7f0cd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b65b90c0918d2163 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0cd9ffb700 1 -- 172.21.15.45:0/1406400017 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400e070 con 0x7f0cdc0b2e40 2023-08-27T11:12:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0cd9ffb700 1 -- 172.21.15.45:0/1406400017 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd400b910 con 0x7f0cdc0b2e40 2023-08-27T11:12:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0cd9ffb700 1 -- 172.21.15.45:0/1406400017 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400bcc0 con 0x7f0cdc0b2e40 2023-08-27T11:12:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1406400017 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc0b2e40 msgr2=0x7f0cdc06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0ce12f1700 1 --2- 172.21.15.45:0/1406400017 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc0b2e40 0x7f0cdc06dc90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009e50 tx=0x7f0cd400b330 comp rx=0 tx=0).stop 2023-08-27T11:12:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1406400017 shutdown_connections 2023-08-27T11:12:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0ce12f1700 1 --2- 172.21.15.45:0/1406400017 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc0b2e40 0x7f0cdc06dc90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0ce12f1700 1 --2- 172.21.15.45:0/1406400017 >> v2:172.21.15.135:3300/0 conn(0x7f0cdc0cb5d0 0x7f0cdc0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.054+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1406400017 >> 172.21.15.45:0/1406400017 conn(0x7f0cdc0cd900 msgr2=0x7f0cdc0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.055+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1406400017 shutdown_connections 2023-08-27T11:12:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.055+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1406400017 wait complete. 2023-08-27T11:12:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.056+0000 7f0ce12f1700 1 Processor -- start 2023-08-27T11:12:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.056+0000 7f0ce12f1700 1 -- start start 2023-08-27T11:12:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0ce12f1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc0b2e40 0x7f0cdc14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0ce12f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0cb5d0 0x7f0cdc15b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0ce12f1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc15b090 0x7f0cdc0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0ce12f1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc15c3f0 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0ce12f1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc15be30 con 0x7f0cdc0b2e40 2023-08-27T11:12:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0ce12f1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc15c110 con 0x7f0cdc15b090 2023-08-27T11:12:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cda7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0cb5d0 0x7f0cdc15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cda7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0cb5d0 0x7f0cdc15b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51356/0 (socket says 172.21.15.45:51356) 2023-08-27T11:12:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cda7fc700 1 -- 172.21.15.45:0/1532378288 learned_addr learned my addr 172.21.15.45:0/1532378288 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cdaffd700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc0b2e40 0x7f0cdc14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cdb7fe700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc15b090 0x7f0cdc0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cda7fc700 1 -- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc15b090 msgr2=0x7f0cdc0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cda7fc700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc15b090 0x7f0cdc0ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cda7fc700 1 -- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc0b2e40 msgr2=0x7f0cdc14e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cda7fc700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc0b2e40 0x7f0cdc14e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cda7fc700 1 -- 172.21.15.45:0/1532378288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc0094a0 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cda7fc700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0cb5d0 0x7f0cdc15b470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00ebe0 tx=0x7f0ccc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cc37fe700 1 -- 172.21.15.45:0/1532378288 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc009800 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.057+0000 7f0cc37fe700 1 -- 172.21.15.45:0/1532378288 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc00c910 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.058+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd4009b30 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.058+0000 7f0cc37fe700 1 -- 172.21.15.45:0/1532378288 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00cc80 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.058+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc15d0d0 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.058+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cdc15cc50 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.059+0000 7f0cc37fe700 1 -- 172.21.15.45:0/1532378288 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0ccc013470 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.059+0000 7f0cc37fe700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0cc806a0e0 0x7f0cc806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.060+0000 7f0cc37fe700 1 -- 172.21.15.45:0/1532378288 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0ccc088690 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.060+0000 7f0cdaffd700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0cc806a0e0 0x7f0cc806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.060+0000 7f0cdaffd700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0cc806a0e0 0x7f0cc806c5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0007fd0 tx=0x7f0cd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.064+0000 7f0cc37fe700 1 -- 172.21.15.45:0/1532378288 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0cdc15cc50 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.308+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0cdc15cc50 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.913+0000 7f0cc37fe700 1 -- 172.21.15.45:0/1532378288 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0cdc15cc50 con 0x7f0cdc0cb5d0 2023-08-27T11:12:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.920+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0cc806a0e0 msgr2=0x7f0cc806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.920+0000 7f0ce12f1700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0cc806a0e0 0x7f0cc806c5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0007fd0 tx=0x7f0cd0009040 comp rx=0 tx=0).stop 2023-08-27T11:12:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.920+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0cb5d0 msgr2=0x7f0cdc15b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:08.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.920+0000 7f0ce12f1700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0cb5d0 0x7f0cdc15b470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00ebe0 tx=0x7f0ccc00eef0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.929+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 shutdown_connections 2023-08-27T11:12:08.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.929+0000 7f0ce12f1700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0cc806a0e0 0x7f0cc806c5a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.929+0000 7f0ce12f1700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0cdc15b090 0x7f0cdc0ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.929+0000 7f0ce12f1700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cdc0b2e40 0x7f0cdc14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.929+0000 7f0ce12f1700 1 --2- 172.21.15.45:0/1532378288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cdc0cb5d0 0x7f0cdc15b470 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.929+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 >> 172.21.15.45:0/1532378288 conn(0x7f0cdc0cd900 msgr2=0x7f0cdc0cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.929+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 shutdown_connections 2023-08-27T11:12:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:08.929+0000 7f0ce12f1700 1 -- 172.21.15.45:0/1532378288 wait complete. 2023-08-27T11:12:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 3 and data pool 4 2023-08-27T11:12:08.945 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-08-27T11:12:08.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-08-27T11:12:09.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.082+0000 7f481601a700 1 Processor -- start 2023-08-27T11:12:09.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.082+0000 7f481601a700 1 -- start start 2023-08-27T11:12:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f481601a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4810157be0 0x7f48100716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f481601a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48100b17c0 0x7f48100b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f481601a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481015b460 con 0x7f481006ced0 2023-08-27T11:12:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f481601a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4810157fc0 con 0x7f48100b17c0 2023-08-27T11:12:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f481601a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48100b2280 con 0x7f4810157be0 2023-08-27T11:12:09.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f480effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48100b17c0 0x7f48100b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:09.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f480effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48100b17c0 0x7f48100b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51364/0 (socket says 172.21.15.45:51364) 2023-08-27T11:12:09.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f480effd700 1 -- 172.21.15.45:0/1183097688 learned_addr learned my addr 172.21.15.45:0/1183097688 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:09.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f480f7fe700 1 --2- 172.21.15.45:0/1183097688 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4810157be0 0x7f48100716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:09.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f480effd700 1 -- 172.21.15.45:0/1183097688 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4810157be0 msgr2=0x7f48100716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:09.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f480effd700 1 --2- 172.21.15.45:0/1183097688 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4810157be0 0x7f48100716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f480effd700 1 -- 172.21.15.45:0/1183097688 >> v1:172.21.15.135:6789/0 conn(0x7f481006ced0 legacy=0x7f48100cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:09.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f480effd700 1 -- 172.21.15.45:0/1183097688 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100b2560 con 0x7f48100b17c0 2023-08-27T11:12:09.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.083+0000 7f480effd700 1 --2- 172.21.15.45:0/1183097688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48100b17c0 0x7f48100b1c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51ac7f04e11481a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f480e7fc700 1 -- 172.21.15.45:0/1183097688 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000b8a0 con 0x7f48100b17c0 2023-08-27T11:12:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f480e7fc700 1 -- 172.21.15.45:0/1183097688 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480000ba00 con 0x7f48100b17c0 2023-08-27T11:12:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f480e7fc700 1 -- 172.21.15.45:0/1183097688 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000bd90 con 0x7f48100b17c0 2023-08-27T11:12:09.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f481601a700 1 -- 172.21.15.45:0/1183097688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48100b17c0 msgr2=0x7f48100b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f481601a700 1 --2- 172.21.15.45:0/1183097688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48100b17c0 0x7f48100b1c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f481601a700 1 -- 172.21.15.45:0/1183097688 shutdown_connections 2023-08-27T11:12:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f481601a700 1 --2- 172.21.15.45:0/1183097688 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4810157be0 0x7f48100716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f481601a700 1 --2- 172.21.15.45:0/1183097688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48100b17c0 0x7f48100b1c40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f481601a700 1 -- 172.21.15.45:0/1183097688 >> 172.21.15.45:0/1183097688 conn(0x7f48100cc7c0 msgr2=0x7f48100ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f481601a700 1 -- 172.21.15.45:0/1183097688 shutdown_connections 2023-08-27T11:12:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.084+0000 7f481601a700 1 -- 172.21.15.45:0/1183097688 wait complete. 2023-08-27T11:12:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.086+0000 7f481601a700 1 Processor -- start 2023-08-27T11:12:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.086+0000 7f481601a700 1 -- start start 2023-08-27T11:12:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f481601a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481006ced0 0x7f48101f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:09.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f481601a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48100b17c0 0x7f48101f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f481601a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4810157be0 0x7f481015c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f481601a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c3e80 con 0x7f481006ced0 2023-08-27T11:12:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f481601a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c37e0 con 0x7f4810157be0 2023-08-27T11:12:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f481601a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c87c0 con 0x7f48100b17c0 2023-08-27T11:12:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481006ced0 0x7f48101f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481006ced0 0x7f48101f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51368/0 (socket says 172.21.15.45:51368) 2023-08-27T11:12:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480f7fe700 1 -- 172.21.15.45:0/2519826549 learned_addr learned my addr 172.21.15.45:0/2519826549 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480ffff700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4810157be0 0x7f481015c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480effd700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48100b17c0 0x7f48101f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480f7fe700 1 -- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48100b17c0 msgr2=0x7f48101f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480f7fe700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48100b17c0 0x7f48101f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480f7fe700 1 -- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4810157be0 msgr2=0x7f481015c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480f7fe700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4810157be0 0x7f481015c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480f7fe700 1 -- 172.21.15.45:0/2519826549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4800009550 con 0x7f481006ced0 2023-08-27T11:12:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480effd700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48100b17c0 0x7f48101f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480ffff700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4810157be0 0x7f481015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.087+0000 7f480f7fe700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481006ced0 0x7f48101f2ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47f800e810 tx=0x7f47f800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.088+0000 7f4807fff700 1 -- 172.21.15.45:0/2519826549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8013610 con 0x7f481006ced0 2023-08-27T11:12:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.088+0000 7f4807fff700 1 -- 172.21.15.45:0/2519826549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f8013770 con 0x7f481006ced0 2023-08-27T11:12:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.088+0000 7f4807fff700 1 -- 172.21.15.45:0/2519826549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8013b20 con 0x7f481006ced0 2023-08-27T11:12:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.088+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48101f3a20 con 0x7f481006ced0 2023-08-27T11:12:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.088+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48100c3aa0 con 0x7f481006ced0 2023-08-27T11:12:09.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.089+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810072690 con 0x7f481006ced0 2023-08-27T11:12:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.089+0000 7f4807fff700 1 -- 172.21.15.45:0/2519826549 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f47f8013c80 con 0x7f481006ced0 2023-08-27T11:12:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.090+0000 7f4807fff700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f47f006a110 0x7f47f006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.090+0000 7f4807fff700 1 -- 172.21.15.45:0/2519826549 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47f8088200 con 0x7f481006ced0 2023-08-27T11:12:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.090+0000 7f480effd700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f47f006a110 0x7f47f006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.090+0000 7f480effd700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f47f006a110 0x7f47f006c5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4800009c40 tx=0x7f4800008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.094+0000 7f4807fff700 1 -- 172.21.15.45:0/2519826549 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f47f8057300 con 0x7f481006ced0 2023-08-27T11:12:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.304+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f48100734a0 con 0x7f481006ced0 2023-08-27T11:12:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.917+0000 7f4807fff700 1 -- 172.21.15.45:0/2519826549 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) v1 ==== 157+0+0 (secure 0 0 0) 0x7f47f8056e90 con 0x7f481006ced0 2023-08-27T11:12:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f47f006a110 msgr2=0x7f47f006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f47f006a110 0x7f47f006c5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4800009c40 tx=0x7f4800008030 comp rx=0 tx=0).stop 2023-08-27T11:12:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481006ced0 msgr2=0x7f48101f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481006ced0 0x7f48101f2ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47f800e810 tx=0x7f47f800ebd0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 shutdown_connections 2023-08-27T11:12:09.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481006ced0 0x7f48101f2ed0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f47f006a110 0x7f47f006c5d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4810157be0 0x7f481015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 --2- 172.21.15.45:0/2519826549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f48100b17c0 0x7f48101f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:09.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 >> 172.21.15.45:0/2519826549 conn(0x7f48100cc7c0 msgr2=0x7f48100ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:09.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 shutdown_connections 2023-08-27T11:12:09.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:09.920+0000 7f481601a700 1 -- 172.21.15.45:0/2519826549 wait complete. 2023-08-27T11:12:09.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:12:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.078+0000 7f081e681700 1 Processor -- start 2023-08-27T11:12:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.078+0000 7f081e681700 1 -- start start 2023-08-27T11:12:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f081e681700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f081815d390 0x7f08180ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f081e681700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08180cd8f0 0x7f081806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f081e681700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f081806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f081e681700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081806ac50 con 0x7f08180cd8f0 2023-08-27T11:12:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f081e681700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0818071460 con 0x7f081815d390 2023-08-27T11:12:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f081e681700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08180cfad0 con 0x7f081806a7c0 2023-08-27T11:12:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f08177fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08180cd8f0 0x7f081806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f08177fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08180cd8f0 0x7f081806a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51376/0 (socket says 172.21.15.45:51376) 2023-08-27T11:12:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f08177fe700 1 -- 172.21.15.45:0/2800296823 learned_addr learned my addr 172.21.15.45:0/2800296823 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f0817fff700 1 --2- 172.21.15.45:0/2800296823 >> v2:172.21.15.135:3300/0 conn(0x7f081815d390 0x7f08180ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f081cc1e700 1 --2- 172.21.15.45:0/2800296823 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f081806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f08177fe700 1 -- 172.21.15.45:0/2800296823 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 msgr2=0x7f081806d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.079+0000 7f08177fe700 1 --2- 172.21.15.45:0/2800296823 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f081806d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f08177fe700 1 -- 172.21.15.45:0/2800296823 >> v2:172.21.15.135:3300/0 conn(0x7f081815d390 msgr2=0x7f08180ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f08177fe700 1 --2- 172.21.15.45:0/2800296823 >> v2:172.21.15.135:3300/0 conn(0x7f081815d390 0x7f08180ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f08177fe700 1 -- 172.21.15.45:0/2800296823 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818071180 con 0x7f08180cd8f0 2023-08-27T11:12:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f0817fff700 1 --2- 172.21.15.45:0/2800296823 >> v2:172.21.15.135:3300/0 conn(0x7f081815d390 0x7f08180ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081cc1e700 1 --2- 172.21.15.45:0/2800296823 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f081806d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f08177fe700 1 --2- 172.21.15.45:0/2800296823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 0x7f081806a280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f081806c3e0 tx=0x7f080c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c71b3b0e7e5e3a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f0816ffd700 1 -- 172.21.15.45:0/2800296823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c00b870 con 0x7f08180cd8f0 2023-08-27T11:12:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f0816ffd700 1 -- 172.21.15.45:0/2800296823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c00b9d0 con 0x7f08180cd8f0 2023-08-27T11:12:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f0816ffd700 1 -- 172.21.15.45:0/2800296823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c00bd60 con 0x7f08180cd8f0 2023-08-27T11:12:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081e681700 1 -- 172.21.15.45:0/2800296823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 msgr2=0x7f081806a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081e681700 1 --2- 172.21.15.45:0/2800296823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 0x7f081806a280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f081806c3e0 tx=0x7f080c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081e681700 1 -- 172.21.15.45:0/2800296823 shutdown_connections 2023-08-27T11:12:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081e681700 1 --2- 172.21.15.45:0/2800296823 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f081806d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081e681700 1 --2- 172.21.15.45:0/2800296823 >> v2:172.21.15.135:3300/0 conn(0x7f081815d390 0x7f08180ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081e681700 1 --2- 172.21.15.45:0/2800296823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 0x7f081806a280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081e681700 1 -- 172.21.15.45:0/2800296823 >> 172.21.15.45:0/2800296823 conn(0x7f08180cc7a0 msgr2=0x7f08180ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:10.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081e681700 1 -- 172.21.15.45:0/2800296823 shutdown_connections 2023-08-27T11:12:10.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.080+0000 7f081e681700 1 -- 172.21.15.45:0/2800296823 wait complete. 2023-08-27T11:12:10.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.082+0000 7f081e681700 1 Processor -- start 2023-08-27T11:12:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.082+0000 7f081e681700 1 -- start start 2023-08-27T11:12:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081e681700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f0818160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081e681700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 0x7f0818150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081e681700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f081815d390 0x7f08180b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081e681700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08180b2340 con 0x7f08180cd8f0 2023-08-27T11:12:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081e681700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08180b1ae0 con 0x7f081815d390 2023-08-27T11:12:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081e681700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08180b1d80 con 0x7f081806a7c0 2023-08-27T11:12:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f08177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 0x7f0818150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f08177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 0x7f0818150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51382/0 (socket says 172.21.15.45:51382) 2023-08-27T11:12:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f08177fe700 1 -- 172.21.15.45:0/3497369773 learned_addr learned my addr 172.21.15.45:0/3497369773 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f0817fff700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f0818160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081cc1e700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f081815d390 0x7f08180b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f08177fe700 1 -- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 msgr2=0x7f0818160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f08177fe700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f0818160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f08177fe700 1 -- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f081815d390 msgr2=0x7f08180b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f08177fe700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f081815d390 0x7f08180b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f08177fe700 1 -- 172.21.15.45:0/3497369773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c009550 con 0x7f08180cd8f0 2023-08-27T11:12:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f0817fff700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f0818160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081cc1e700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f081815d390 0x7f08180b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f08177fe700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 0x7f0818150350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f080c000c00 tx=0x7f080c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f0814ff9700 1 -- 172.21.15.45:0/3497369773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c004540 con 0x7f08180cd8f0 2023-08-27T11:12:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f0814ff9700 1 -- 172.21.15.45:0/3497369773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c0036a0 con 0x7f08180cd8f0 2023-08-27T11:12:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08180b2060 con 0x7f08180cd8f0 2023-08-27T11:12:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f0814ff9700 1 -- 172.21.15.45:0/3497369773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c003a50 con 0x7f08180cd8f0 2023-08-27T11:12:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.083+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08180afb00 con 0x7f08180cd8f0 2023-08-27T11:12:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.084+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08180676b0 con 0x7f08180cd8f0 2023-08-27T11:12:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.085+0000 7f0814ff9700 1 -- 172.21.15.45:0/3497369773 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f080c021070 con 0x7f08180cd8f0 2023-08-27T11:12:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.085+0000 7f0814ff9700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f080406a0e0 0x7f080406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.085+0000 7f0814ff9700 1 -- 172.21.15.45:0/3497369773 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f080c089d20 con 0x7f08180cd8f0 2023-08-27T11:12:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.085+0000 7f0817fff700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f080406a0e0 0x7f080406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.086+0000 7f0817fff700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f080406a0e0 0x7f080406c5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f080800c960 tx=0x7f0808008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.090+0000 7f0814ff9700 1 -- 172.21.15.45:0/3497369773 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f080c0553c0 con 0x7f08180cd8f0 2023-08-27T11:12:10.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:10.291+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f08180726b0 con 0x7f08180cd8f0 2023-08-27T11:12:11.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.925+0000 7f0814ff9700 1 -- 172.21.15.45:0/3497369773 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) v1 ==== 228+0+0 (secure 0 0 0) 0x7f080c029100 con 0x7f08180cd8f0 2023-08-27T11:12:11.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.936+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f080406a0e0 msgr2=0x7f080406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:11.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.936+0000 7f081e681700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f080406a0e0 0x7f080406c5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f080800c960 tx=0x7f0808008040 comp rx=0 tx=0).stop 2023-08-27T11:12:11.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.936+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 msgr2=0x7f0818150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:11.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.936+0000 7f081e681700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 0x7f0818150350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f080c000c00 tx=0x7f080c004750 comp rx=0 tx=0).stop 2023-08-27T11:12:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.937+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 shutdown_connections 2023-08-27T11:12:11.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.937+0000 7f081e681700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f080406a0e0 0x7f080406c5a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:11.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.937+0000 7f081e681700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f081815d390 0x7f08180b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:11.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.937+0000 7f081e681700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f081806a7c0 0x7f0818160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:11.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.937+0000 7f081e681700 1 --2- 172.21.15.45:0/3497369773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08180cd8f0 0x7f0818150350 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:11.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.937+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 >> 172.21.15.45:0/3497369773 conn(0x7f08180cc7a0 msgr2=0x7f08180ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:11.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.938+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 shutdown_connections 2023-08-27T11:12:11.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:11.939+0000 7f081e681700 1 -- 172.21.15.45:0/3497369773 wait complete. 2023-08-27T11:12:11.941 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' created 2023-08-27T11:12:11.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-08-27T11:12:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.089+0000 7fc94c71e700 1 Processor -- start 2023-08-27T11:12:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.089+0000 7fc94c71e700 1 -- start start 2023-08-27T11:12:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc94c71e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc9440b17c0 0x7fc9440b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc94c71e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94415b460 con 0x7fc944157be0 2023-08-27T11:12:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc94c71e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9440b2280 con 0x7fc94406ced0 2023-08-27T11:12:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc94c71e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc944157fc0 con 0x7fc9440b17c0 2023-08-27T11:12:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc94a4ba700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc944157be0 0x7fc9440716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38164/0 (socket says 172.21.15.45:38164) 2023-08-27T11:12:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc94a4ba700 1 -- 172.21.15.45:0/1493548556 learned_addr learned my addr 172.21.15.45:0/1493548556 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc949cb9700 1 --2- 172.21.15.45:0/1493548556 >> v2:172.21.15.135:3300/0 conn(0x7fc9440b17c0 0x7fc9440b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576443947 0 0) 0x7fc94415b460 con 0x7fc944157be0 2023-08-27T11:12:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9300035b0 con 0x7fc944157be0 2023-08-27T11:12:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 574777103 0 0) 0x7fc9300035b0 con 0x7fc944157be0 2023-08-27T11:12:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 >> v2:172.21.15.135:3300/0 conn(0x7fc9440b17c0 msgr2=0x7fc9440b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 --2- 172.21.15.45:0/1493548556 >> v2:172.21.15.135:3300/0 conn(0x7fc9440b17c0 0x7fc9440b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 >> v1:172.21.15.135:6789/0 conn(0x7fc94406ced0 legacy=0x7fc9440cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9440b2560 con 0x7fc944157be0 2023-08-27T11:12:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fc940001670 con 0x7fc944157be0 2023-08-27T11:12:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 >> v1:172.21.15.45:6789/0 conn(0x7fc944157be0 legacy=0x7fc9440716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc930003a00 0x7fc930003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc930004850 0x7fc930004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.090+0000 7fc9494b8700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc930005210 0x7fc93000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944157fc0 con 0x7fc930003a00 2023-08-27T11:12:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93000c0b0 con 0x7fc930004850 2023-08-27T11:12:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93000c270 con 0x7fc930005210 2023-08-27T11:12:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc94a4ba700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc930003a00 0x7fc930003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:12.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc949cb9700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc930004850 0x7fc930004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:12.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc94acbb700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc930005210 0x7fc93000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:12.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc94a4ba700 1 -- 172.21.15.45:0/1493548556 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc930005210 msgr2=0x7fc93000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:12.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc94a4ba700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc930005210 0x7fc93000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc94a4ba700 1 -- 172.21.15.45:0/1493548556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc930004850 msgr2=0x7fc930004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc94a4ba700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc930004850 0x7fc930004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc94a4ba700 1 -- 172.21.15.45:0/1493548556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9300035b0 con 0x7fc930003a00 2023-08-27T11:12:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc94acbb700 1 -- 172.21.15.45:0/1493548556 reap_dead start 2023-08-27T11:12:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc94a4ba700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc930003a00 0x7fc930003e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c2a0 tx=0x7fc930005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dc0fa67b9f0c93 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940007c10 con 0x7fc930003a00 2023-08-27T11:12:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94000cde0 con 0x7fc930003a00 2023-08-27T11:12:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.091+0000 7fc9494b8700 1 -- 172.21.15.45:0/1493548556 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940005ba0 con 0x7fc930003a00 2023-08-27T11:12:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.092+0000 7fc94c71e700 1 -- 172.21.15.45:0/1493548556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc930003a00 msgr2=0x7fc930003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.092+0000 7fc94c71e700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc930003a00 0x7fc930003e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc94000c2a0 tx=0x7fc930005880 comp rx=0 tx=0).stop 2023-08-27T11:12:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.092+0000 7fc94c71e700 1 -- 172.21.15.45:0/1493548556 shutdown_connections 2023-08-27T11:12:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.092+0000 7fc94c71e700 1 --2- 172.21.15.45:0/1493548556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc930003a00 0x7fc930003e20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.092+0000 7fc94c71e700 1 -- 172.21.15.45:0/1493548556 >> 172.21.15.45:0/1493548556 conn(0x7fc9440cc7c0 msgr2=0x7fc9440ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.092+0000 7fc94c71e700 1 -- 172.21.15.45:0/1493548556 shutdown_connections 2023-08-27T11:12:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.092+0000 7fc94c71e700 1 -- 172.21.15.45:0/1493548556 wait complete. 2023-08-27T11:12:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.093+0000 7fc94c71e700 1 Processor -- start 2023-08-27T11:12:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.093+0000 7fc94c71e700 1 -- start start 2023-08-27T11:12:12.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94c71e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc930003a00 0x7fc9440c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:12.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94c71e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9440b35a0 0x7fc9441ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94c71e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9441f2140 0x7fc9440c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94c71e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440b3db0 con 0x7fc9441f2140 2023-08-27T11:12:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94c71e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440b4090 con 0x7fc9440b35a0 2023-08-27T11:12:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94c71e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440b4370 con 0x7fc930003a00 2023-08-27T11:12:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94acbb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9441f2140 0x7fc9440c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94acbb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9441f2140 0x7fc9440c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51404/0 (socket says 172.21.15.45:51404) 2023-08-27T11:12:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94acbb700 1 -- 172.21.15.45:0/3299021925 learned_addr learned my addr 172.21.15.45:0/3299021925 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94a4ba700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc930003a00 0x7fc9440c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc949cb9700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9440b35a0 0x7fc9441ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94acbb700 1 -- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc930003a00 msgr2=0x7fc9440c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94acbb700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc930003a00 0x7fc9440c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94acbb700 1 -- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9440b35a0 msgr2=0x7fc9441ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94acbb700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9440b35a0 0x7fc9441ef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94acbb700 1 -- 172.21.15.45:0/3299021925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9440b4650 con 0x7fc9441f2140 2023-08-27T11:12:12.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.094+0000 7fc94acbb700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9441f2140 0x7fc9440c9590 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc944067080 tx=0x7fc93c00d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.095+0000 7fc93affd700 1 -- 172.21.15.45:0/3299021925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c003c10 con 0x7fc9441f2140 2023-08-27T11:12:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.095+0000 7fc93affd700 1 -- 172.21.15.45:0/3299021925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c003d90 con 0x7fc9441f2140 2023-08-27T11:12:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.095+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9440b4930 con 0x7fc9441f2140 2023-08-27T11:12:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.095+0000 7fc93affd700 1 -- 172.21.15.45:0/3299021925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00da10 con 0x7fc9441f2140 2023-08-27T11:12:12.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.095+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9440b3a20 con 0x7fc9441f2140 2023-08-27T11:12:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.095+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9440b4b20 con 0x7fc9441f2140 2023-08-27T11:12:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.096+0000 7fc93affd700 1 -- 172.21.15.45:0/3299021925 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc93c00db70 con 0x7fc9441f2140 2023-08-27T11:12:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.096+0000 7fc93affd700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc92806a0e0 0x7fc92806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.096+0000 7fc93affd700 1 -- 172.21.15.45:0/3299021925 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6781+0+0 (secure 0 0 0) 0x7fc93c0889d0 con 0x7fc9441f2140 2023-08-27T11:12:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.097+0000 7fc94a4ba700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc92806a0e0 0x7fc92806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.097+0000 7fc94a4ba700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc92806a0e0 0x7fc92806c5a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc940000c00 tx=0x7fc940007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.101+0000 7fc93affd700 1 -- 172.21.15.45:0/3299021925 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc93c053df0 con 0x7fc9441f2140 2023-08-27T11:12:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.300+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fc94415a790 con 0x7fc9441f2140 2023-08-27T11:12:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.931+0000 7fc93affd700 1 -- 172.21.15.45:0/3299021925 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc93c057410 con 0x7fc9441f2140 2023-08-27T11:12:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.936+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc92806a0e0 msgr2=0x7fc92806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.936+0000 7fc94c71e700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc92806a0e0 0x7fc92806c5a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc940000c00 tx=0x7fc940007000 comp rx=0 tx=0).stop 2023-08-27T11:12:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.936+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9441f2140 msgr2=0x7fc9440c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.936+0000 7fc94c71e700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9441f2140 0x7fc9440c9590 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc944067080 tx=0x7fc93c00d150 comp rx=0 tx=0).stop 2023-08-27T11:12:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.937+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 shutdown_connections 2023-08-27T11:12:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.937+0000 7fc94c71e700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9441f2140 0x7fc9440c9590 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.937+0000 7fc94c71e700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc92806a0e0 0x7fc92806c5a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.937+0000 7fc94c71e700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc930003a00 0x7fc9440c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.937+0000 7fc94c71e700 1 --2- 172.21.15.45:0/3299021925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9440b35a0 0x7fc9441ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.937+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 >> 172.21.15.45:0/3299021925 conn(0x7fc9440cc7c0 msgr2=0x7fc9440ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.937+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 shutdown_connections 2023-08-27T11:12:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:12.938+0000 7fc94c71e700 1 -- 172.21.15.45:0/3299021925 wait complete. 2023-08-27T11:12:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:set pool 5 allow_ec_overwrites to true 2023-08-27T11:12:12.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-08-27T11:12:12.992 INFO:tasks.ceph.mds.d.smithi135.stderr:2023-08-27T11:12:12.991+0000 7fdc14a59700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:12:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.101+0000 7fcf93b40700 1 Processor -- start 2023-08-27T11:12:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.101+0000 7fcf93b40700 1 -- start start 2023-08-27T11:12:13.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.102+0000 7fcf93b40700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c15d340 0x7fcf8c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:13.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.102+0000 7fcf93b40700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8c06b160 con 0x7fcf8c06a990 2023-08-27T11:12:13.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.102+0000 7fcf93b40700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8c0af590 con 0x7fcf8c0cd880 2023-08-27T11:12:13.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.102+0000 7fcf93b40700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c072010 con 0x7fcf8c15d340 2023-08-27T11:12:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.102+0000 7fcf920dd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcf8c06a990 0x7fcf8c0b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38188/0 (socket says 172.21.15.45:38188) 2023-08-27T11:12:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.102+0000 7fcf920dd700 1 -- 172.21.15.45:0/3940329115 learned_addr learned my addr 172.21.15.45:0/3940329115 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637912257 0 0) 0x7fcf8c06b160 con 0x7fcf8c06a990 2023-08-27T11:12:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf918dc700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c15d340 0x7fcf8c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf780035b0 con 0x7fcf8c06a990 2023-08-27T11:12:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2010145583 0 0) 0x7fcf780035b0 con 0x7fcf8c06a990 2023-08-27T11:12:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c15d340 msgr2=0x7fcf8c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c15d340 0x7fcf8c160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 >> v1:172.21.15.135:6789/0 conn(0x7fcf8c0cd880 legacy=0x7fcf8c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:13.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0af2b0 con 0x7fcf8c06a990 2023-08-27T11:12:13.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fcf840019f0 con 0x7fcf8c06a990 2023-08-27T11:12:13.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 >> v1:172.21.15.45:6789/0 conn(0x7fcf8c06a990 legacy=0x7fcf8c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:13.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf78003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c15d340 0x7fcf78004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf78004d90 0x7fcf78005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c072010 con 0x7fcf78003a30 2023-08-27T11:12:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7800bb40 con 0x7fcf78004d90 2023-08-27T11:12:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7800bd00 con 0x7fcf8c15d340 2023-08-27T11:12:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf918dc700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf78003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf920dd700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf78004d90 0x7fcf78005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf910db700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c15d340 0x7fcf78004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf918dc700 1 -- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c15d340 msgr2=0x7fcf78004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:13.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf918dc700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c15d340 0x7fcf78004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf918dc700 1 -- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf78004d90 msgr2=0x7fcf78005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:13.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf918dc700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf78004d90 0x7fcf78005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf918dc700 1 -- 172.21.15.45:0/3940329115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf780035b0 con 0x7fcf78003a30 2023-08-27T11:12:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.103+0000 7fcf918dc700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf78003e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009b60 tx=0x7fcf78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624cb4de2abb8f3f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00e6c0 con 0x7fcf78003a30 2023-08-27T11:12:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c00e820 con 0x7fcf78003a30 2023-08-27T11:12:13.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf908da700 1 -- 172.21.15.45:0/3940329115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00eb90 con 0x7fcf78003a30 2023-08-27T11:12:13.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf93b40700 1 -- 172.21.15.45:0/3940329115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 msgr2=0x7fcf78003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:13.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf93b40700 1 --2- 172.21.15.45:0/3940329115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf78003e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009b60 tx=0x7fcf78005880 comp rx=0 tx=0).stop 2023-08-27T11:12:13.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf920dd700 1 -- 172.21.15.45:0/3940329115 reap_dead start 2023-08-27T11:12:13.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf93b40700 1 -- 172.21.15.45:0/3940329115 shutdown_connections 2023-08-27T11:12:13.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf93b40700 1 -- 172.21.15.45:0/3940329115 >> 172.21.15.45:0/3940329115 conn(0x7fcf8c0cc7c0 msgr2=0x7fcf8c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:13.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf93b40700 1 -- 172.21.15.45:0/3940329115 shutdown_connections 2023-08-27T11:12:13.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.104+0000 7fcf93b40700 1 -- 172.21.15.45:0/3940329115 wait complete. 2023-08-27T11:12:13.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.105+0000 7fcf93b40700 1 Processor -- start 2023-08-27T11:12:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.105+0000 7fcf93b40700 1 -- start start 2023-08-27T11:12:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf93b40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf8c1f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf93b40700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c0b55f0 0x7fcf8c14b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf93b40700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf8c14b9b0 0x7fcf8c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf93b40700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0b51f0 con 0x7fcf78003a30 2023-08-27T11:12:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf93b40700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c1f68c0 con 0x7fcf8c14b9b0 2023-08-27T11:12:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf93b40700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c1f6ba0 con 0x7fcf8c0b55f0 2023-08-27T11:12:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf918dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf8c1f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf918dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf8c1f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51420/0 (socket says 172.21.15.45:51420) 2023-08-27T11:12:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf918dc700 1 -- 172.21.15.45:0/3160288145 learned_addr learned my addr 172.21.15.45:0/3160288145 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf910db700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c0b55f0 0x7fcf8c14b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.106+0000 7fcf920dd700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf8c14b9b0 0x7fcf8c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf918dc700 1 -- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c0b55f0 msgr2=0x7fcf8c14b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf918dc700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c0b55f0 0x7fcf8c14b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf918dc700 1 -- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf8c14b9b0 msgr2=0x7fcf8c0b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf918dc700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf8c14b9b0 0x7fcf8c0b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf918dc700 1 -- 172.21.15.45:0/3160288145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c1f6e80 con 0x7fcf78003a30 2023-08-27T11:12:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf920dd700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf8c14b9b0 0x7fcf8c0b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf918dc700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf8c1f62e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0090b0 tx=0x7fcf7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf827fc700 1 -- 172.21.15.45:0/3160288145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c021070 con 0x7fcf78003a30 2023-08-27T11:12:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf827fc700 1 -- 172.21.15.45:0/3160288145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c009ed0 con 0x7fcf78003a30 2023-08-27T11:12:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8c1f7160 con 0x7fcf78003a30 2023-08-27T11:12:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf827fc700 1 -- 172.21.15.45:0/3160288145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00e6c0 con 0x7fcf78003a30 2023-08-27T11:12:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.107+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8c0b5a70 con 0x7fcf78003a30 2023-08-27T11:12:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.108+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8c14de50 con 0x7fcf78003a30 2023-08-27T11:12:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.108+0000 7fcf827fc700 1 -- 172.21.15.45:0/3160288145 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcf7c025070 con 0x7fcf78003a30 2023-08-27T11:12:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.109+0000 7fcf827fc700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf7006a0e0 0x7fcf7006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:13.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.109+0000 7fcf827fc700 1 -- 172.21.15.45:0/3160288145 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7fcf7c0898e0 con 0x7fcf78003a30 2023-08-27T11:12:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.109+0000 7fcf910db700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf7006a0e0 0x7fcf7006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.109+0000 7fcf910db700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf7006a0e0 0x7fcf7006c5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0cf4f0 tx=0x7fcf88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:13.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.113+0000 7fcf827fc700 1 -- 172.21.15.45:0/3160288145 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcf7c054e10 con 0x7fcf78003a30 2023-08-27T11:12:13.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.359+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fcf8c072690 con 0x7fcf78003a30 2023-08-27T11:12:13.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.932+0000 7fcf827fc700 1 -- 172.21.15.45:0/3160288145 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fcf7c058430 con 0x7fcf78003a30 2023-08-27T11:12:13.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf7006a0e0 msgr2=0x7fcf7006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:13.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf7006a0e0 0x7fcf7006c5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0cf4f0 tx=0x7fcf88009040 comp rx=0 tx=0).stop 2023-08-27T11:12:13.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 msgr2=0x7fcf8c1f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:13.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf8c1f62e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0090b0 tx=0x7fcf7c0047c0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 shutdown_connections 2023-08-27T11:12:13.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003a30 0x7fcf8c1f62e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf7006a0e0 0x7fcf7006c5a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf8c14b9b0 0x7fcf8c0b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 --2- 172.21.15.45:0/3160288145 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf8c0b55f0 0x7fcf8c14b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:13.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.935+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 >> 172.21.15.45:0/3160288145 conn(0x7fcf8c0cc7c0 msgr2=0x7fcf8c0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:13.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.936+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 shutdown_connections 2023-08-27T11:12:13.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:13.936+0000 7fcf93b40700 1 -- 172.21.15.45:0/3160288145 wait complete. 2023-08-27T11:12:13.939 INFO:teuthology.orchestra.run.smithi045.stderr:added data pool 5 to fsmap 2023-08-27T11:12:13.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:12:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.082+0000 7fa54cb46700 1 Processor -- start 2023-08-27T11:12:14.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.082+0000 7fa54cb46700 1 -- start start 2023-08-27T11:12:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54cb46700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa54806e980 0x7fa5480ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54cb46700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54815dad0 con 0x7fa54806cec0 2023-08-27T11:12:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54cb46700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54815ddb0 con 0x7fa54815b070 2023-08-27T11:12:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54cb46700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54815d8e0 con 0x7fa54806e980 2023-08-27T11:12:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa545d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa54806cec0 0x7fa54806e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38206/0 (socket says 172.21.15.45:38206) 2023-08-27T11:12:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa545d9b700 1 -- 172.21.15.45:0/592216 learned_addr learned my addr 172.21.15.45:0/592216 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa546d9d700 1 --2- 172.21.15.45:0/592216 >> v2:172.21.15.135:3300/0 conn(0x7fa54806e980 0x7fa5480ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682230976 0 0) 0x7fa54815dad0 con 0x7fa54806cec0 2023-08-27T11:12:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5300035b0 con 0x7fa54806cec0 2023-08-27T11:12:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 771779484 0 0) 0x7fa5300035b0 con 0x7fa54806cec0 2023-08-27T11:12:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 >> v2:172.21.15.135:3300/0 conn(0x7fa54806e980 msgr2=0x7fa5480ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 --2- 172.21.15.45:0/592216 >> v2:172.21.15.135:3300/0 conn(0x7fa54806e980 0x7fa5480ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 >> v1:172.21.15.135:6789/0 conn(0x7fa54815b070 legacy=0x7fa5480cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54815e090 con 0x7fa54806cec0 2023-08-27T11:12:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fa538001670 con 0x7fa54806cec0 2023-08-27T11:12:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 >> v1:172.21.15.45:6789/0 conn(0x7fa54806cec0 legacy=0x7fa54806e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa530003a00 0x7fa530003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa530004400 0x7fa5300048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5300050d0 0x7fa53000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53000c000 con 0x7fa530004400 2023-08-27T11:12:14.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53000c170 con 0x7fa530003a00 2023-08-27T11:12:14.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53000c330 con 0x7fa5300050d0 2023-08-27T11:12:14.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.083+0000 7fa545d9b700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa530004400 0x7fa5300048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa54659c700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa530003a00 0x7fa530003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa546d9d700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5300050d0 0x7fa53000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa545d9b700 1 -- 172.21.15.45:0/592216 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5300050d0 msgr2=0x7fa53000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa545d9b700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5300050d0 0x7fa53000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa545d9b700 1 -- 172.21.15.45:0/592216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa530003a00 msgr2=0x7fa530003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa545d9b700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa530003a00 0x7fa530003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa545d9b700 1 -- 172.21.15.45:0/592216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300035b0 con 0x7fa530004400 2023-08-27T11:12:14.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa546d9d700 1 -- 172.21.15.45:0/592216 reap_dead start 2023-08-27T11:12:14.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa545d9b700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa530004400 0x7fa5300048d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa548070e30 tx=0x7fa530005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f564871d603facf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:14.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538005b60 con 0x7fa530004400 2023-08-27T11:12:14.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538005cc0 con 0x7fa530004400 2023-08-27T11:12:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa54559a700 1 -- 172.21.15.45:0/592216 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538008300 con 0x7fa530004400 2023-08-27T11:12:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.084+0000 7fa54cb46700 1 -- 172.21.15.45:0/592216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa530004400 msgr2=0x7fa5300048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.085+0000 7fa54cb46700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa530004400 0x7fa5300048d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa548070e30 tx=0x7fa530005880 comp rx=0 tx=0).stop 2023-08-27T11:12:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.085+0000 7fa54cb46700 1 -- 172.21.15.45:0/592216 shutdown_connections 2023-08-27T11:12:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.085+0000 7fa54cb46700 1 --2- 172.21.15.45:0/592216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa530004400 0x7fa5300048d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.085+0000 7fa54cb46700 1 -- 172.21.15.45:0/592216 >> 172.21.15.45:0/592216 conn(0x7fa5480cfe50 msgr2=0x7fa5480d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.085+0000 7fa54cb46700 1 -- 172.21.15.45:0/592216 shutdown_connections 2023-08-27T11:12:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.085+0000 7fa54cb46700 1 -- 172.21.15.45:0/592216 wait complete. 2023-08-27T11:12:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.086+0000 7fa54cb46700 1 Processor -- start 2023-08-27T11:12:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.086+0000 7fa54cb46700 1 -- start start 2023-08-27T11:12:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa54cb46700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa530004400 0x7fa5480b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa54cb46700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481fac40 0x7fa5480b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa54cb46700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5480b4ef0 0x7fa54815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa54cb46700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480b5510 con 0x7fa5481fac40 2023-08-27T11:12:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa54cb46700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480b57f0 con 0x7fa530004400 2023-08-27T11:12:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa54cb46700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5481fa160 con 0x7fa5480b4ef0 2023-08-27T11:12:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa545d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481fac40 0x7fa5480b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa545d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481fac40 0x7fa5480b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51442/0 (socket says 172.21.15.45:51442) 2023-08-27T11:12:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa545d9b700 1 -- 172.21.15.45:0/2255455720 learned_addr learned my addr 172.21.15.45:0/2255455720 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa546d9d700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5480b4ef0 0x7fa54815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa54659c700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa530004400 0x7fa5480b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa545d9b700 1 -- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5480b4ef0 msgr2=0x7fa54815faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa545d9b700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5480b4ef0 0x7fa54815faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa545d9b700 1 -- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa530004400 msgr2=0x7fa5480b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa545d9b700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa530004400 0x7fa5480b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa545d9b700 1 -- 172.21.15.45:0/2255455720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5481fa440 con 0x7fa5481fac40 2023-08-27T11:12:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.087+0000 7fa545d9b700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481fac40 0x7fa5480b49b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa538004b20 tx=0x7fa538004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.088+0000 7fa536ffd700 1 -- 172.21.15.45:0/2255455720 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538005e30 con 0x7fa5481fac40 2023-08-27T11:12:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.088+0000 7fa536ffd700 1 -- 172.21.15.45:0/2255455720 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538001960 con 0x7fa5481fac40 2023-08-27T11:12:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.088+0000 7fa536ffd700 1 -- 172.21.15.45:0/2255455720 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5380080e0 con 0x7fa5481fac40 2023-08-27T11:12:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.088+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5481fa720 con 0x7fa5481fac40 2023-08-27T11:12:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.088+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa548160030 con 0x7fa5481fac40 2023-08-27T11:12:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.089+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5480b52d0 con 0x7fa5481fac40 2023-08-27T11:12:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.089+0000 7fa536ffd700 1 -- 172.21.15.45:0/2255455720 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa538008240 con 0x7fa5481fac40 2023-08-27T11:12:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.090+0000 7fa536ffd700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa52806a0e0 0x7fa52806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.090+0000 7fa536ffd700 1 -- 172.21.15.45:0/2255455720 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa538088f30 con 0x7fa5481fac40 2023-08-27T11:12:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.090+0000 7fa54659c700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa52806a0e0 0x7fa52806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.090+0000 7fa54659c700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa52806a0e0 0x7fa52806c5a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa53c000c00 tx=0x7fa53c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.094+0000 7fa536ffd700 1 -- 172.21.15.45:0/2255455720 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa5480b52d0 con 0x7fa5481fac40 2023-08-27T11:12:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.336+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5481faa90 con 0x7fa5481fac40 2023-08-27T11:12:14.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.337+0000 7fa536ffd700 1 -- 172.21.15.45:0/2255455720 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fa538025090 con 0x7fa5481fac40 2023-08-27T11:12:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:14.338 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:13.928801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-27T11:12:14.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa52806a0e0 msgr2=0x7fa52806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa52806a0e0 0x7fa52806c5a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa53c000c00 tx=0x7fa53c009040 comp rx=0 tx=0).stop 2023-08-27T11:12:14.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481fac40 msgr2=0x7fa5480b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481fac40 0x7fa5480b49b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa538004b20 tx=0x7fa538004a90 comp rx=0 tx=0).stop 2023-08-27T11:12:14.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 shutdown_connections 2023-08-27T11:12:14.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa52806a0e0 0x7fa52806c5a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5480b4ef0 0x7fa54815faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa530004400 0x7fa5480b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 --2- 172.21.15.45:0/2255455720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5481fac40 0x7fa5480b49b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 >> 172.21.15.45:0/2255455720 conn(0x7fa5480cfe50 msgr2=0x7fa5480cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:14.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 shutdown_connections 2023-08-27T11:12:14.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.339+0000 7fa54cb46700 1 -- 172.21.15.45:0/2255455720 wait complete. 2023-08-27T11:12:14.343 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2023-08-27T11:12:14.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.476+0000 7f2571672700 1 Processor -- start 2023-08-27T11:12:14.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.476+0000 7f2571672700 1 -- start start 2023-08-27T11:12:14.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.476+0000 7f2571672700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f256c0cc650 0x7f256c0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.476+0000 7f2571672700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f256c0adad0 0x7f256c0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f2571672700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256c0ccb80 con 0x7f256c15e990 2023-08-27T11:12:14.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f2571672700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c0ae460 con 0x7f256c0cc650 2023-08-27T11:12:14.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f2571672700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c0cce60 con 0x7f256c0adad0 2023-08-27T11:12:14.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f256a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f256c0cc650 0x7f256c0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f256a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f256c0cc650 0x7f256c0ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51450/0 (socket says 172.21.15.45:51450) 2023-08-27T11:12:14.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f256a7fc700 1 -- 172.21.15.45:0/1234697616 learned_addr learned my addr 172.21.15.45:0/1234697616 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:14.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f256a7fc700 1 -- 172.21.15.45:0/1234697616 >> v2:172.21.15.135:3300/0 conn(0x7f256c0adad0 msgr2=0x7f256c0adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:14.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f256a7fc700 1 --2- 172.21.15.45:0/1234697616 >> v2:172.21.15.135:3300/0 conn(0x7f256c0adad0 0x7f256c0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f256a7fc700 1 -- 172.21.15.45:0/1234697616 >> v1:172.21.15.135:6789/0 conn(0x7f256c15e990 legacy=0x7f256c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:14.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f256a7fc700 1 -- 172.21.15.45:0/1234697616 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c0cd600 con 0x7f256c0cc650 2023-08-27T11:12:14.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f256a7fc700 1 --2- 172.21.15.45:0/1234697616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c0cc650 0x7f256c0ced40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f256c0cb640 tx=0x7f2560009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ddf4bba8665e5b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:14.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f2569ffb700 1 -- 172.21.15.45:0/1234697616 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560004230 con 0x7f256c0cc650 2023-08-27T11:12:14.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f2569ffb700 1 -- 172.21.15.45:0/1234697616 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560004390 con 0x7f256c0cc650 2023-08-27T11:12:14.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.477+0000 7f2569ffb700 1 -- 172.21.15.45:0/1234697616 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25600036a0 con 0x7f256c0cc650 2023-08-27T11:12:14.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.478+0000 7f2571672700 1 -- 172.21.15.45:0/1234697616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c0cc650 msgr2=0x7f256c0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.478+0000 7f2571672700 1 --2- 172.21.15.45:0/1234697616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c0cc650 0x7f256c0ced40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f256c0cb640 tx=0x7f2560009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.478+0000 7f2571672700 1 -- 172.21.15.45:0/1234697616 shutdown_connections 2023-08-27T11:12:14.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.478+0000 7f2571672700 1 --2- 172.21.15.45:0/1234697616 >> v2:172.21.15.135:3300/0 conn(0x7f256c0adad0 0x7f256c0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.478+0000 7f2571672700 1 --2- 172.21.15.45:0/1234697616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c0cc650 0x7f256c0ced40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.478+0000 7f2571672700 1 -- 172.21.15.45:0/1234697616 >> 172.21.15.45:0/1234697616 conn(0x7f256c0d0f60 msgr2=0x7f256c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:14.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.478+0000 7f2571672700 1 -- 172.21.15.45:0/1234697616 shutdown_connections 2023-08-27T11:12:14.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.478+0000 7f2571672700 1 -- 172.21.15.45:0/1234697616 wait complete. 2023-08-27T11:12:14.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.479+0000 7f2571672700 1 Processor -- start 2023-08-27T11:12:14.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.479+0000 7f2571672700 1 -- start start 2023-08-27T11:12:14.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f2571672700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f256c0adad0 0x7f256c14f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f2571672700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f256c0cc650 0x7f256c09ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f2571672700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c15e990 0x7f256c0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f2571672700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0b4de0 con 0x7f256c15e990 2023-08-27T11:12:14.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f2571672700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0b50c0 con 0x7f256c0cc650 2023-08-27T11:12:14.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f2571672700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0b53a0 con 0x7f256c0adad0 2023-08-27T11:12:14.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c15e990 0x7f256c0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c15e990 0x7f256c0b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51458/0 (socket says 172.21.15.45:51458) 2023-08-27T11:12:14.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256b7fe700 1 -- 172.21.15.45:0/3410287725 learned_addr learned my addr 172.21.15.45:0/3410287725 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:14.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256a7fc700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f256c0cc650 0x7f256c09ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256affd700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f256c0adad0 0x7f256c14f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256b7fe700 1 -- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f256c0adad0 msgr2=0x7f256c14f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256b7fe700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f256c0adad0 0x7f256c14f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256b7fe700 1 -- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f256c0cc650 msgr2=0x7f256c09ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256b7fe700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f256c0cc650 0x7f256c09ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.480+0000 7f256b7fe700 1 -- 172.21.15.45:0/3410287725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2560009550 con 0x7f256c15e990 2023-08-27T11:12:14.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.481+0000 7f256b7fe700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c15e990 0x7f256c0b4590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f256400b970 tx=0x7f256400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:14.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.481+0000 7f25537fe700 1 -- 172.21.15.45:0/3410287725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256400d860 con 0x7f256c15e990 2023-08-27T11:12:14.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.481+0000 7f25537fe700 1 -- 172.21.15.45:0/3410287725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256400d9e0 con 0x7f256c15e990 2023-08-27T11:12:14.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.481+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c0b5680 con 0x7f256c15e990 2023-08-27T11:12:14.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.481+0000 7f25537fe700 1 -- 172.21.15.45:0/3410287725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256400dd90 con 0x7f256c15e990 2023-08-27T11:12:14.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.481+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c0b4ad0 con 0x7f256c15e990 2023-08-27T11:12:14.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.482+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c0b5ba0 con 0x7f256c15e990 2023-08-27T11:12:14.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.482+0000 7f25537fe700 1 -- 172.21.15.45:0/3410287725 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f256400e430 con 0x7f256c15e990 2023-08-27T11:12:14.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.483+0000 7f25537fe700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f255806a0e0 0x7f255806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.483+0000 7f25537fe700 1 -- 172.21.15.45:0/3410287725 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f25640889a0 con 0x7f256c15e990 2023-08-27T11:12:14.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.483+0000 7f256affd700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f255806a0e0 0x7f255806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.483+0000 7f256affd700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f255806a0e0 0x7f255806c5a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f255c007fd0 tx=0x7f255c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:14.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.487+0000 7f25537fe700 1 -- 172.21.15.45:0/3410287725 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f256c0b5ba0 con 0x7f256c15e990 2023-08-27T11:12:14.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.687+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f256c0b5ba0 con 0x7f256c15e990 2023-08-27T11:12:14.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.688+0000 7f25537fe700 1 -- 172.21.15.45:0/3410287725 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f256c0b5ba0 con 0x7f256c15e990 2023-08-27T11:12:14.689 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:14.690 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:14.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f255806a0e0 msgr2=0x7f255806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f255806a0e0 0x7f255806c5a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f255c007fd0 tx=0x7f255c009040 comp rx=0 tx=0).stop 2023-08-27T11:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c15e990 msgr2=0x7f256c0b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c15e990 0x7f256c0b4590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f256400b970 tx=0x7f256400bc80 comp rx=0 tx=0).stop 2023-08-27T11:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 shutdown_connections 2023-08-27T11:12:14.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f256c15e990 0x7f256c0b4590 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f255806a0e0 0x7f255806c5a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f256c0adad0 0x7f256c14f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 --2- 172.21.15.45:0/3410287725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f256c0cc650 0x7f256c09ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 >> 172.21.15.45:0/3410287725 conn(0x7f256c0d0f60 msgr2=0x7f256c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:14.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 shutdown_connections 2023-08-27T11:12:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.690+0000 7f2571672700 1 -- 172.21.15.45:0/3410287725 wait complete. 2023-08-27T11:12:14.704 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.829+0000 7f07722a5700 1 Processor -- start 2023-08-27T11:12:14.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.829+0000 7f07722a5700 1 -- start start 2023-08-27T11:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.830+0000 7f07722a5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f076c15a0f0 0x7f076c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.830+0000 7f07722a5700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f076c0cd300 0x7f076c15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.830+0000 7f07722a5700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076c0cfae0 con 0x7f076c15e030 2023-08-27T11:12:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.830+0000 7f07722a5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076c15e900 con 0x7f076c15a0f0 2023-08-27T11:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.830+0000 7f07722a5700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076c0cae60 con 0x7f076c0cd300 2023-08-27T11:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.830+0000 7f076bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f076c15a0f0 0x7f076c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.830+0000 7f076bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f076c15a0f0 0x7f076c0cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51462/0 (socket says 172.21.15.45:51462) 2023-08-27T11:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.830+0000 7f076bfff700 1 -- 172.21.15.45:0/1743322448 learned_addr learned my addr 172.21.15.45:0/1743322448 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.830+0000 7f076b7fe700 1 --2- 172.21.15.45:0/1743322448 >> v2:172.21.15.135:3300/0 conn(0x7f076c0cd300 0x7f076c15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f076bfff700 1 -- 172.21.15.45:0/1743322448 >> v2:172.21.15.135:3300/0 conn(0x7f076c0cd300 msgr2=0x7f076c15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f076bfff700 1 --2- 172.21.15.45:0/1743322448 >> v2:172.21.15.135:3300/0 conn(0x7f076c0cd300 0x7f076c15daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f076bfff700 1 -- 172.21.15.45:0/1743322448 >> v1:172.21.15.135:6789/0 conn(0x7f076c15e030 legacy=0x7f076c0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f076bfff700 1 -- 172.21.15.45:0/1743322448 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c0cb140 con 0x7f076c15a0f0 2023-08-27T11:12:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f076b7fe700 1 --2- 172.21.15.45:0/1743322448 >> v2:172.21.15.135:3300/0 conn(0x7f076c0cd300 0x7f076c15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f076bfff700 1 --2- 172.21.15.45:0/1743322448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15a0f0 0x7f076c0cbc20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f076c0cf9e0 tx=0x7f075c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a1f45dc2df1f20 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f076affd700 1 -- 172.21.15.45:0/1743322448 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c00dd30 con 0x7f076c15a0f0 2023-08-27T11:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f076affd700 1 -- 172.21.15.45:0/1743322448 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c00de90 con 0x7f076c15a0f0 2023-08-27T11:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f076affd700 1 -- 172.21.15.45:0/1743322448 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c010690 con 0x7f076c15a0f0 2023-08-27T11:12:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f07722a5700 1 -- 172.21.15.45:0/1743322448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15a0f0 msgr2=0x7f076c0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.831+0000 7f07722a5700 1 --2- 172.21.15.45:0/1743322448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15a0f0 0x7f076c0cbc20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f076c0cf9e0 tx=0x7f075c00be70 comp rx=0 tx=0).stop 2023-08-27T11:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.832+0000 7f07722a5700 1 -- 172.21.15.45:0/1743322448 shutdown_connections 2023-08-27T11:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.832+0000 7f07722a5700 1 --2- 172.21.15.45:0/1743322448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15a0f0 0x7f076c0cbc20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.832+0000 7f07722a5700 1 --2- 172.21.15.45:0/1743322448 >> v2:172.21.15.135:3300/0 conn(0x7f076c0cd300 0x7f076c15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.832+0000 7f07722a5700 1 -- 172.21.15.45:0/1743322448 >> 172.21.15.45:0/1743322448 conn(0x7f076c0cc7e0 msgr2=0x7f076c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.832+0000 7f07722a5700 1 -- 172.21.15.45:0/1743322448 shutdown_connections 2023-08-27T11:12:14.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.832+0000 7f07722a5700 1 -- 172.21.15.45:0/1743322448 wait complete. 2023-08-27T11:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.833+0000 7f07722a5700 1 Processor -- start 2023-08-27T11:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.833+0000 7f07722a5700 1 -- start start 2023-08-27T11:12:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f07722a5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076c0cd300 0x7f076c0c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f07722a5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f076c15a0f0 0x7f076c06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f07722a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15e030 0x7f076c06f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f07722a5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c0ca050 con 0x7f076c15e030 2023-08-27T11:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f07722a5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c0c97b0 con 0x7f076c15a0f0 2023-08-27T11:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f07722a5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c0c9a90 con 0x7f076c0cd300 2023-08-27T11:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f0770842700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15e030 0x7f076c06f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f0770842700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15e030 0x7f076c06f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51472/0 (socket says 172.21.15.45:51472) 2023-08-27T11:12:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f0770842700 1 -- 172.21.15.45:0/1073019027 learned_addr learned my addr 172.21.15.45:0/1073019027 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f076b7fe700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f076c15a0f0 0x7f076c06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f076bfff700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076c0cd300 0x7f076c0c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f0770842700 1 -- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076c0cd300 msgr2=0x7f076c0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f0770842700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076c0cd300 0x7f076c0c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f0770842700 1 -- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f076c15a0f0 msgr2=0x7f076c06ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f0770842700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f076c15a0f0 0x7f076c06ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f0770842700 1 -- 172.21.15.45:0/1073019027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c00b7f0 con 0x7f076c15e030 2023-08-27T11:12:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.834+0000 7f0770842700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15e030 0x7f076c06f200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f076400cc50 tx=0x7f076400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.835+0000 7f0768ff9700 1 -- 172.21.15.45:0/1073019027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764003c00 con 0x7f076c15e030 2023-08-27T11:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.835+0000 7f0768ff9700 1 -- 172.21.15.45:0/1073019027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0764003d60 con 0x7f076c15e030 2023-08-27T11:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.835+0000 7f0768ff9700 1 -- 172.21.15.45:0/1073019027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076400aab0 con 0x7f076c15e030 2023-08-27T11:12:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.835+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076c0c9d70 con 0x7f076c15e030 2023-08-27T11:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.835+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c0ce1c0 con 0x7f076c15e030 2023-08-27T11:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.836+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076c0ce380 con 0x7f076c15e030 2023-08-27T11:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.837+0000 7f0768ff9700 1 -- 172.21.15.45:0/1073019027 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f076400ac10 con 0x7f076c15e030 2023-08-27T11:12:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.837+0000 7f0768ff9700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f075806a0e0 0x7f075806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.837+0000 7f0768ff9700 1 -- 172.21.15.45:0/1073019027 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0764089260 con 0x7f076c15e030 2023-08-27T11:12:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.842+0000 7f076bfff700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f075806a0e0 0x7f075806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.842+0000 7f0768ff9700 1 -- 172.21.15.45:0/1073019027 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f07640546d0 con 0x7f076c15e030 2023-08-27T11:12:14.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:14.842+0000 7f076bfff700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f075806a0e0 0x7f075806c5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f075c00b6f0 tx=0x7f075c012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.045+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f076c1ef740 con 0x7f076c15e030 2023-08-27T11:12:15.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.047+0000 7f0768ff9700 1 -- 172.21.15.45:0/1073019027 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f0764057cf0 con 0x7f076c15e030 2023-08-27T11:12:15.048 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:15.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f075806a0e0 msgr2=0x7f075806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f075806a0e0 0x7f075806c5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f075c00b6f0 tx=0x7f075c012000 comp rx=0 tx=0).stop 2023-08-27T11:12:15.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15e030 msgr2=0x7f076c06f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15e030 0x7f076c06f200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f076400cc50 tx=0x7f076400cf60 comp rx=0 tx=0).stop 2023-08-27T11:12:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 shutdown_connections 2023-08-27T11:12:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076c15e030 0x7f076c06f200 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f075806a0e0 0x7f075806c5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076c0cd300 0x7f076c0c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 --2- 172.21.15.45:0/1073019027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f076c15a0f0 0x7f076c06ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 >> 172.21.15.45:0/1073019027 conn(0x7f076c0cc7e0 msgr2=0x7f076c0716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:15.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 shutdown_connections 2023-08-27T11:12:15.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.049+0000 7f07722a5700 1 -- 172.21.15.45:0/1073019027 wait complete. 2023-08-27T11:12:15.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:12:15.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.372+0000 7fed71d6c700 1 Processor -- start 2023-08-27T11:12:15.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.372+0000 7fed71d6c700 1 -- start start 2023-08-27T11:12:15.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.373+0000 7fed71d6c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fed6c15a240 0x7fed6c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.373+0000 7fed71d6c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 0x7fed6c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.373+0000 7fed71d6c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6c06d690 con 0x7fed6c0ce9d0 2023-08-27T11:12:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.373+0000 7fed71d6c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c0cee20 con 0x7fed6c15a240 2023-08-27T11:12:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.373+0000 7fed71d6c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0cc440 con 0x7fed6c0cbe70 2023-08-27T11:12:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.373+0000 7fed6b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fed6c15a240 0x7fed6c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.373+0000 7fed6b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fed6c15a240 0x7fed6c0cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51476/0 (socket says 172.21.15.45:51476) 2023-08-27T11:12:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.373+0000 7fed6b7fe700 1 -- 172.21.15.45:0/343994799 learned_addr learned my addr 172.21.15.45:0/343994799 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed6affd700 1 --2- 172.21.15.45:0/343994799 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 0x7fed6c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed6b7fe700 1 -- 172.21.15.45:0/343994799 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 msgr2=0x7fed6c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed6b7fe700 1 --2- 172.21.15.45:0/343994799 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 0x7fed6c0ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed6b7fe700 1 -- 172.21.15.45:0/343994799 >> v1:172.21.15.135:6789/0 conn(0x7fed6c0ce9d0 legacy=0x7fed6c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed6b7fe700 1 -- 172.21.15.45:0/343994799 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c06d040 con 0x7fed6c15a240 2023-08-27T11:12:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed6b7fe700 1 --2- 172.21.15.45:0/343994799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 0x7fed6c0cae80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0098f0 tx=0x7fed5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e536b9c342ef3f8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed6a7fc700 1 -- 172.21.15.45:0/343994799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c004230 con 0x7fed6c15a240 2023-08-27T11:12:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed6a7fc700 1 -- 172.21.15.45:0/343994799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c004390 con 0x7fed6c15a240 2023-08-27T11:12:15.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed6a7fc700 1 -- 172.21.15.45:0/343994799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c0036a0 con 0x7fed6c15a240 2023-08-27T11:12:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed71d6c700 1 -- 172.21.15.45:0/343994799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 msgr2=0x7fed6c0cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed71d6c700 1 --2- 172.21.15.45:0/343994799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 0x7fed6c0cae80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0098f0 tx=0x7fed5c009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed71d6c700 1 -- 172.21.15.45:0/343994799 shutdown_connections 2023-08-27T11:12:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed71d6c700 1 --2- 172.21.15.45:0/343994799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 0x7fed6c0cae80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed71d6c700 1 --2- 172.21.15.45:0/343994799 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 0x7fed6c0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.374+0000 7fed71d6c700 1 -- 172.21.15.45:0/343994799 >> 172.21.15.45:0/343994799 conn(0x7fed6c0cfe50 msgr2=0x7fed6c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.375+0000 7fed71d6c700 1 -- 172.21.15.45:0/343994799 shutdown_connections 2023-08-27T11:12:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.375+0000 7fed71d6c700 1 -- 172.21.15.45:0/343994799 wait complete. 2023-08-27T11:12:15.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.376+0000 7fed71d6c700 1 Processor -- start 2023-08-27T11:12:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.376+0000 7fed71d6c700 1 -- start start 2023-08-27T11:12:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.377+0000 7fed71d6c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 0x7fed6c0c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.377+0000 7fed71d6c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c0ce9d0 0x7fed6c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.377+0000 7fed71d6c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 0x7fed6c15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.377+0000 7fed71d6c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0c32b0 con 0x7fed6c15a240 2023-08-27T11:12:15.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.377+0000 7fed71d6c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c15d6d0 con 0x7fed6c0ce9d0 2023-08-27T11:12:15.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.377+0000 7fed71d6c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0c50c0 con 0x7fed6c0cbe70 2023-08-27T11:12:15.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.377+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 0x7fed6c15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.377+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 0x7fed6c15a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51486/0 (socket says 172.21.15.45:51486) 2023-08-27T11:12:15.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.377+0000 7fed6bfff700 1 -- 172.21.15.45:0/903231731 learned_addr learned my addr 172.21.15.45:0/903231731 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:15.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed6affd700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c0ce9d0 0x7fed6c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed6b7fe700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 0x7fed6c0c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed6bfff700 1 -- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 msgr2=0x7fed6c0c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed6bfff700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 0x7fed6c0c2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed6bfff700 1 -- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c0ce9d0 msgr2=0x7fed6c15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed6bfff700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c0ce9d0 0x7fed6c15d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed6bfff700 1 -- 172.21.15.45:0/903231731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c009550 con 0x7fed6c15a240 2023-08-27T11:12:15.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed6affd700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c0ce9d0 0x7fed6c15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:15.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed6bfff700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 0x7fed6c15a620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fed6400cc10 tx=0x7fed6400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:15.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed63fff700 1 -- 172.21.15.45:0/903231731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400ab10 con 0x7fed6c15a240 2023-08-27T11:12:15.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed63fff700 1 -- 172.21.15.45:0/903231731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6400ac70 con 0x7fed6c15a240 2023-08-27T11:12:15.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c0c53a0 con 0x7fed6c15a240 2023-08-27T11:12:15.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed63fff700 1 -- 172.21.15.45:0/903231731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64012420 con 0x7fed6c15a240 2023-08-27T11:12:15.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.378+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c0c5960 con 0x7fed6c15a240 2023-08-27T11:12:15.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.379+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6c072810 con 0x7fed6c15a240 2023-08-27T11:12:15.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.380+0000 7fed63fff700 1 -- 172.21.15.45:0/903231731 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fed64012580 con 0x7fed6c15a240 2023-08-27T11:12:15.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.380+0000 7fed63fff700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fed4c06a110 0x7fed4c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.380+0000 7fed63fff700 1 -- 172.21.15.45:0/903231731 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fed64088bf0 con 0x7fed6c15a240 2023-08-27T11:12:15.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.380+0000 7fed6b7fe700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fed4c06a110 0x7fed4c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.381+0000 7fed6b7fe700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fed4c06a110 0x7fed4c06c5d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009130 tx=0x7fed5c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:15.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.385+0000 7fed63fff700 1 -- 172.21.15.45:0/903231731 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fed64053f60 con 0x7fed6c15a240 2023-08-27T11:12:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.628+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed6c0ce7b0 con 0x7fed6c15a240 2023-08-27T11:12:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.629+0000 7fed63fff700 1 -- 172.21.15.45:0/903231731 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fed64057580 con 0x7fed6c15a240 2023-08-27T11:12:15.630 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:15.630 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:13.928801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-27T11:12:15.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fed4c06a110 msgr2=0x7fed4c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fed4c06a110 0x7fed4c06c5d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009130 tx=0x7fed5c008030 comp rx=0 tx=0).stop 2023-08-27T11:12:15.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 msgr2=0x7fed6c15a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 0x7fed6c15a620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fed6400cc10 tx=0x7fed6400cf20 comp rx=0 tx=0).stop 2023-08-27T11:12:15.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 shutdown_connections 2023-08-27T11:12:15.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed6c15a240 0x7fed6c15a620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fed4c06a110 0x7fed4c06c5d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fed6c0cbe70 0x7fed6c0c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 --2- 172.21.15.45:0/903231731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c0ce9d0 0x7fed6c15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 >> 172.21.15.45:0/903231731 conn(0x7fed6c0cfe50 msgr2=0x7fed6c071900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:15.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 shutdown_connections 2023-08-27T11:12:15.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.631+0000 7fed71d6c700 1 -- 172.21.15.45:0/903231731 wait complete. 2023-08-27T11:12:15.634 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2023-08-27T11:12:15.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:15.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.768+0000 7f3016746700 1 Processor -- start 2023-08-27T11:12:15.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.768+0000 7f3016746700 1 -- start start 2023-08-27T11:12:15.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.768+0000 7f3016746700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30100cc650 0x7f30100ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.768+0000 7f3016746700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30100ccb80 con 0x7f301015e990 2023-08-27T11:12:15.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.768+0000 7f3016746700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30100cce60 con 0x7f30100adad0 2023-08-27T11:12:15.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.768+0000 7f3016746700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30100cd600 con 0x7f30100cc650 2023-08-27T11:12:15.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30100cc650 0x7f30100ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f301015e990 0x7f30100ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38266/0 (socket says 172.21.15.45:38266) 2023-08-27T11:12:15.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300ffff700 1 -- 172.21.15.45:0/2385602126 learned_addr learned my addr 172.21.15.45:0/2385602126 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:15.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300f7fe700 1 -- 172.21.15.45:0/2385602126 >> v1:172.21.15.135:6789/0 conn(0x7f30100adad0 legacy=0x7f30100cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300f7fe700 1 -- 172.21.15.45:0/2385602126 >> v1:172.21.15.45:6789/0 conn(0x7f301015e990 legacy=0x7f30100ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300f7fe700 1 -- 172.21.15.45:0/2385602126 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30100cd8e0 con 0x7f30100cc650 2023-08-27T11:12:15.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300effd700 1 -- 172.21.15.45:0/2385602126 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288036499 0 0) 0x7f30100ccb80 con 0x7f301015e990 2023-08-27T11:12:15.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300f7fe700 1 --2- 172.21.15.45:0/2385602126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100cc650 0x7f30100ced40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009840 tx=0x7f2ff8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea65d34ddca6798 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:15.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300effd700 1 -- 172.21.15.45:0/2385602126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8011830 con 0x7f30100cc650 2023-08-27T11:12:15.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300effd700 1 -- 172.21.15.45:0/2385602126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff8011990 con 0x7f30100cc650 2023-08-27T11:12:15.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.769+0000 7f300effd700 1 -- 172.21.15.45:0/2385602126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8011d20 con 0x7f30100cc650 2023-08-27T11:12:15.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.770+0000 7f3016746700 1 -- 172.21.15.45:0/2385602126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100cc650 msgr2=0x7f30100ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.770+0000 7f3016746700 1 --2- 172.21.15.45:0/2385602126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100cc650 0x7f30100ced40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009840 tx=0x7f2ff8009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:15.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.770+0000 7f3016746700 1 -- 172.21.15.45:0/2385602126 shutdown_connections 2023-08-27T11:12:15.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.770+0000 7f3016746700 1 --2- 172.21.15.45:0/2385602126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100cc650 0x7f30100ced40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.770+0000 7f3016746700 1 -- 172.21.15.45:0/2385602126 >> 172.21.15.45:0/2385602126 conn(0x7f30100d0f60 msgr2=0x7f30100d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:15.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.770+0000 7f3016746700 1 -- 172.21.15.45:0/2385602126 shutdown_connections 2023-08-27T11:12:15.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.770+0000 7f3016746700 1 -- 172.21.15.45:0/2385602126 wait complete. 2023-08-27T11:12:15.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.771+0000 7f3016746700 1 Processor -- start 2023-08-27T11:12:15.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.772+0000 7f3016746700 1 -- start start 2023-08-27T11:12:15.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.772+0000 7f3016746700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100adad0 0x7f30101f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.772+0000 7f3016746700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30100cc650 0x7f301006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.772+0000 7f3016746700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f301015e990 0x7f301006f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.772+0000 7f3016746700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30101f0af0 con 0x7f30100adad0 2023-08-27T11:12:15.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.772+0000 7f3016746700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30100b5b50 con 0x7f301015e990 2023-08-27T11:12:15.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.772+0000 7f3016746700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30100b5e30 con 0x7f30100cc650 2023-08-27T11:12:15.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100adad0 0x7f30101f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100adad0 0x7f30101f05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51496/0 (socket says 172.21.15.45:51496) 2023-08-27T11:12:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300ffff700 1 -- 172.21.15.45:0/3150168982 learned_addr learned my addr 172.21.15.45:0/3150168982 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300f7fe700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30100cc650 0x7f301006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f3014ce3700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f301015e990 0x7f301006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300ffff700 1 -- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30100cc650 msgr2=0x7f301006f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300ffff700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30100cc650 0x7f301006f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300ffff700 1 -- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f301015e990 msgr2=0x7f301006f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300ffff700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f301015e990 0x7f301006f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300ffff700 1 -- 172.21.15.45:0/3150168982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80094a0 con 0x7f30100adad0 2023-08-27T11:12:15.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300ffff700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100adad0 0x7f30101f05b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f300000c520 tx=0x7f300000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:15.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300cff9700 1 -- 172.21.15.45:0/3150168982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000004c80 con 0x7f30100adad0 2023-08-27T11:12:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300cff9700 1 -- 172.21.15.45:0/3150168982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000004e00 con 0x7f30100adad0 2023-08-27T11:12:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30100b6110 con 0x7f30100adad0 2023-08-27T11:12:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f300cff9700 1 -- 172.21.15.45:0/3150168982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000007310 con 0x7f30100adad0 2023-08-27T11:12:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.773+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30100b66d0 con 0x7f30100adad0 2023-08-27T11:12:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.774+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30100cf320 con 0x7f30100adad0 2023-08-27T11:12:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.775+0000 7f300cff9700 1 -- 172.21.15.45:0/3150168982 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3000007470 con 0x7f30100adad0 2023-08-27T11:12:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.775+0000 7f300cff9700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2ff006a0e0 0x7f2ff006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.775+0000 7f300cff9700 1 -- 172.21.15.45:0/3150168982 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3000088590 con 0x7f30100adad0 2023-08-27T11:12:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.775+0000 7f300f7fe700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2ff006a0e0 0x7f2ff006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.776+0000 7f300f7fe700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2ff006a0e0 0x7f2ff006c5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80090b0 tx=0x7f2ff801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.780+0000 7f300cff9700 1 -- 172.21.15.45:0/3150168982 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3000053990 con 0x7f30100adad0 2023-08-27T11:12:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.983+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30100b6ee0 con 0x7f30100adad0 2023-08-27T11:12:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.984+0000 7f300cff9700 1 -- 172.21.15.45:0/3150168982 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f3000056fb0 con 0x7f30100adad0 2023-08-27T11:12:15.985 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:15.986 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2ff006a0e0 msgr2=0x7f2ff006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2ff006a0e0 0x7f2ff006c5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80090b0 tx=0x7f2ff801c040 comp rx=0 tx=0).stop 2023-08-27T11:12:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100adad0 msgr2=0x7f30101f05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100adad0 0x7f30101f05b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f300000c520 tx=0x7f300000c8e0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 shutdown_connections 2023-08-27T11:12:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30100adad0 0x7f30101f05b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2ff006a0e0 0x7f2ff006c5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f301015e990 0x7f301006f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 --2- 172.21.15.45:0/3150168982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30100cc650 0x7f301006f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.986+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 >> 172.21.15.45:0/3150168982 conn(0x7f30100d0f60 msgr2=0x7f30100b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.987+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 shutdown_connections 2023-08-27T11:12:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:15.987+0000 7f3016746700 1 -- 172.21.15.45:0/3150168982 wait complete. 2023-08-27T11:12:16.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:12:08.309320+0000', 'modified': '2023-08-27T11:12:13.928801+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6836/739639682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 739639682}, {'type': 'v1', 'addr': '172.21.15.135:6836', 'nonce': 739639682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-08-27T11:12:16.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:12:16.403 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-08-27T11:12:16.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.531+0000 7f79690b5700 1 Processor -- start 2023-08-27T11:12:16.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.531+0000 7f79690b5700 1 -- start start 2023-08-27T11:12:16.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.531+0000 7f79690b5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f796415d340 0x7f7964160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.531+0000 7f79690b5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79640cd880 0x7f796406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f79690b5700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796406b160 con 0x7f79640cdd00 2023-08-27T11:12:16.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f79690b5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7964072010 con 0x7f796415d340 2023-08-27T11:12:16.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f79690b5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79640af590 con 0x7f79640cd880 2023-08-27T11:12:16.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7962d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f796415d340 0x7f7964160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7962d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f796415d340 0x7f7964160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51504/0 (socket says 172.21.15.45:51504) 2023-08-27T11:12:16.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7962d9d700 1 -- 172.21.15.45:0/2768425534 learned_addr learned my addr 172.21.15.45:0/2768425534 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:16.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f796259c700 1 --2- 172.21.15.45:0/2768425534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79640cd880 0x7f796406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7962d9d700 1 -- 172.21.15.45:0/2768425534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79640cd880 msgr2=0x7f796406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7962d9d700 1 --2- 172.21.15.45:0/2768425534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79640cd880 0x7f796406a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7962d9d700 1 -- 172.21.15.45:0/2768425534 >> v1:172.21.15.135:6789/0 conn(0x7f79640cdd00 legacy=0x7f79640b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7962d9d700 1 -- 172.21.15.45:0/2768425534 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79640af2b0 con 0x7f796415d340 2023-08-27T11:12:16.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7962d9d700 1 --2- 172.21.15.45:0/2768425534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796415d340 0x7f7964160d00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f79540098f0 tx=0x7f7954009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ec6cbc9bff50ef server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7961d9b700 1 -- 172.21.15.45:0/2768425534 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954004230 con 0x7f796415d340 2023-08-27T11:12:16.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7961d9b700 1 -- 172.21.15.45:0/2768425534 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7954004390 con 0x7f796415d340 2023-08-27T11:12:16.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.532+0000 7f7961d9b700 1 -- 172.21.15.45:0/2768425534 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540036a0 con 0x7f796415d340 2023-08-27T11:12:16.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.533+0000 7f79690b5700 1 -- 172.21.15.45:0/2768425534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796415d340 msgr2=0x7f7964160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.533+0000 7f79690b5700 1 --2- 172.21.15.45:0/2768425534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796415d340 0x7f7964160d00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f79540098f0 tx=0x7f7954009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:16.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.533+0000 7f79690b5700 1 -- 172.21.15.45:0/2768425534 shutdown_connections 2023-08-27T11:12:16.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.533+0000 7f79690b5700 1 --2- 172.21.15.45:0/2768425534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796415d340 0x7f7964160d00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.533+0000 7f79690b5700 1 --2- 172.21.15.45:0/2768425534 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79640cd880 0x7f796406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.533+0000 7f79690b5700 1 -- 172.21.15.45:0/2768425534 >> 172.21.15.45:0/2768425534 conn(0x7f79640cc7c0 msgr2=0x7f79640ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:16.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.533+0000 7f79690b5700 1 -- 172.21.15.45:0/2768425534 shutdown_connections 2023-08-27T11:12:16.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.533+0000 7f79690b5700 1 -- 172.21.15.45:0/2768425534 wait complete. 2023-08-27T11:12:16.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.534+0000 7f79690b5700 1 Processor -- start 2023-08-27T11:12:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.534+0000 7f79690b5700 1 -- start start 2023-08-27T11:12:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.535+0000 7f79690b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640cd880 0x7f79640c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.535+0000 7f79690b5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79640cdd00 0x7f7964159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.535+0000 7f79690b5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f796415d340 0x7f796415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.535+0000 7f79690b5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79640c7660 con 0x7f79640cd880 2023-08-27T11:12:16.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.535+0000 7f79690b5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79640c7940 con 0x7f79640cdd00 2023-08-27T11:12:16.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.535+0000 7f79690b5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79640c7c20 con 0x7f796415d340 2023-08-27T11:12:16.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.535+0000 7f7962d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640cd880 0x7f79640c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.535+0000 7f7962d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640cd880 0x7f79640c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51510/0 (socket says 172.21.15.45:51510) 2023-08-27T11:12:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.535+0000 7f7962d9d700 1 -- 172.21.15.45:0/75105445 learned_addr learned my addr 172.21.15.45:0/75105445 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f7962d9d700 1 -- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f796415d340 msgr2=0x7f796415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f796359e700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f796415d340 0x7f796415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f796259c700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79640cdd00 0x7f7964159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f7962d9d700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f796415d340 0x7f796415b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f7962d9d700 1 -- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79640cdd00 msgr2=0x7f7964159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f7962d9d700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79640cdd00 0x7f7964159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f7962d9d700 1 -- 172.21.15.45:0/75105445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7954009550 con 0x7f79640cd880 2023-08-27T11:12:16.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f7962d9d700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640cd880 0x7f79640c6d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7954009130 tx=0x7f7954004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f794b7fe700 1 -- 172.21.15.45:0/75105445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954003e30 con 0x7f79640cd880 2023-08-27T11:12:16.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f794b7fe700 1 -- 172.21.15.45:0/75105445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795401f070 con 0x7f79640cd880 2023-08-27T11:12:16.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f79690b5700 1 -- 172.21.15.45:0/75105445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79640c7f00 con 0x7f79640cd880 2023-08-27T11:12:16.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f794b7fe700 1 -- 172.21.15.45:0/75105445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795400b8a0 con 0x7f79640cd880 2023-08-27T11:12:16.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.536+0000 7f79690b5700 1 -- 172.21.15.45:0/75105445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79640c72c0 con 0x7f79640cd880 2023-08-27T11:12:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.537+0000 7f794b7fe700 1 -- 172.21.15.45:0/75105445 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7954021030 con 0x7f79640cd880 2023-08-27T11:12:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.538+0000 7f794b7fe700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f795006a0e0 0x7f795006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.538+0000 7f794b7fe700 1 -- 172.21.15.45:0/75105445 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7954089f10 con 0x7f79640cd880 2023-08-27T11:12:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.538+0000 7f796259c700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f795006a0e0 0x7f795006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.538+0000 7f796259c700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f795006a0e0 0x7f795006c5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f79641e8200 tx=0x7f7958008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.540+0000 7f7975277b80 1 Processor -- start 2023-08-27T11:12:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.540+0000 7f7975277b80 1 -- start start 2023-08-27T11:12:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7975277b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc08210f40 0x55cc08211320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7975277b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0820db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7975277b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 0x55cc0820e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7975277b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0813f730 con 0x55cc08222960 2023-08-27T11:12:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7975277b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0813d050 con 0x55cc081e43d0 2023-08-27T11:12:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7975277b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc08138940 con 0x55cc08210f40 2023-08-27T11:12:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0820db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0820db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51520/0 (socket says 172.21.15.45:51520) 2023-08-27T11:12:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796259c700 1 -- 172.21.15.45:0/3428477363 learned_addr learned my addr 172.21.15.45:0/3428477363 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796359e700 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 0x55cc0820e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7962d9d700 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc08210f40 0x55cc08211320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796259c700 1 -- 172.21.15.45:0/3428477363 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc08210f40 msgr2=0x55cc08211320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796259c700 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc08210f40 0x55cc08211320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796259c700 1 -- 172.21.15.45:0/3428477363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 msgr2=0x55cc0820e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796259c700 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 0x55cc0820e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796259c700 1 -- 172.21.15.45:0/3428477363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc07fb85e0 con 0x55cc08222960 2023-08-27T11:12:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796359e700 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 0x55cc0820e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f796259c700 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0820db70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79580129b0 tx=0x7f7958010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca9fa392430e1e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7948ff9700 1 -- 172.21.15.45:0/3428477363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800b050 con 0x55cc08222960 2023-08-27T11:12:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7948ff9700 1 -- 172.21.15.45:0/3428477363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7958017070 con 0x55cc08222960 2023-08-27T11:12:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.541+0000 7f7948ff9700 1 -- 172.21.15.45:0/3428477363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958013c60 con 0x55cc08222960 2023-08-27T11:12:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.542+0000 7f7975277b80 1 -- 172.21.15.45:0/3428477363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 msgr2=0x55cc0820db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.542+0000 7f7975277b80 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0820db70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79580129b0 tx=0x7f7958010000 comp rx=0 tx=0).stop 2023-08-27T11:12:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.542+0000 7f7975277b80 1 -- 172.21.15.45:0/3428477363 shutdown_connections 2023-08-27T11:12:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.542+0000 7f7975277b80 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 0x55cc0820e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.542+0000 7f7975277b80 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc08210f40 0x55cc08211320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.542+0000 7f7975277b80 1 --2- 172.21.15.45:0/3428477363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0820db70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.542+0000 7f7975277b80 1 -- 172.21.15.45:0/3428477363 >> 172.21.15.45:0/3428477363 conn(0x55cc08144c30 msgr2=0x55cc0822bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.542+0000 7f7975277b80 1 -- 172.21.15.45:0/3428477363 shutdown_connections 2023-08-27T11:12:16.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.542+0000 7f7975277b80 1 -- 172.21.15.45:0/3428477363 wait complete. 2023-08-27T11:12:16.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.543+0000 7f7975277b80 1 -- start start 2023-08-27T11:12:16.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.543+0000 7f7975277b80 1 Processor -- start 2023-08-27T11:12:16.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7975277b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 0x55cc0822df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7975277b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 0x55cc0823db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7975277b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0823e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7975277b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc080d6b60 con 0x55cc08222960 2023-08-27T11:12:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7975277b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc080d6860 con 0x55cc08210f40 2023-08-27T11:12:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7975277b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc07fba8d0 con 0x55cc081e43d0 2023-08-27T11:12:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7975277b80 10 client.0 _ll_register_callbacks cb 0x55cc080d4630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:12:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796359e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0823e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796359e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0823e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51528/0 (socket says 172.21.15.45:51528) 2023-08-27T11:12:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796359e700 1 -- 172.21.15.45:0/4239907885 learned_addr learned my addr 172.21.15.45:0/4239907885 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7962d9d700 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 0x55cc0822df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796259c700 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 0x55cc0823db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796359e700 1 -- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 msgr2=0x55cc0822df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796359e700 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 0x55cc0822df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796359e700 1 -- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 msgr2=0x55cc0823db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796359e700 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 0x55cc0823db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796359e700 1 -- 172.21.15.45:0/4239907885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7958012630 con 0x55cc08222960 2023-08-27T11:12:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f796359e700 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0823e080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f795c00e890 tx=0x7f795c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a580d654c8cc89 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7941ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:12:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7941ffb700 1 -- 172.21.15.45:0/4239907885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c013880 con 0x55cc08222960 2023-08-27T11:12:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7941ffb700 1 -- 172.21.15.45:0/4239907885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f795c0139e0 con 0x55cc08222960 2023-08-27T11:12:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7975277b80 1 -- 172.21.15.45:0/4239907885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc081d94f0 con 0x55cc08222960 2023-08-27T11:12:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7941ffb700 1 -- 172.21.15.45:0/4239907885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f795c01cea0 con 0x55cc08222960 2023-08-27T11:12:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.544+0000 7f7975277b80 10 client.4440 fetch_fsmap learned FSMap version 7 2023-08-27T11:12:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7975277b80 1 -- 172.21.15.45:0/4239907885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55cc08150750 con 0x55cc08222960 2023-08-27T11:12:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7941ffb700 1 -- 172.21.15.45:0/4239907885 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cc081d94f0 con 0x55cc08222960 2023-08-27T11:12:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7941ffb700 1 -- 172.21.15.45:0/4239907885 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f795c013ba0 con 0x55cc08222960 2023-08-27T11:12:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7975277b80 10 client.4440 fetch_fsmap finished waiting for FSMap version 7 2023-08-27T11:12:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7975277b80 10 client.4440 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-27T11:12:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7975277b80 20 client.4440 populate_metadata read hostname 'smithi045' 2023-08-27T11:12:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7975277b80 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x55cc08210700 0x55cc082488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7975277b80 4 client.4440 mds_command: new command op to 4303 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:12:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7975277b80 1 -- 172.21.15.45:0/4239907885 --> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc08131920 con 0x55cc08210700 2023-08-27T11:12:16.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7962d9d700 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x55cc08210700 0x55cc082488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7941ffb700 10 client.4440 ms_handle_connect on v2:172.21.15.135:6834/739639682 2023-08-27T11:12:16.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.545+0000 7f7962d9d700 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x55cc08210700 0x55cc082488b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c659f8a4e04168ff server_cookie=d23744a48c1d6d2b in_seq=0 out_seq=0 2023-08-27T11:12:16.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.546+0000 7f7941ffb700 1 -- 172.21.15.45:0/4239907885 <== mds.0 v2:172.21.15.135:6834/739639682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc08131920 con 0x55cc08210700 2023-08-27T11:12:16.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.546+0000 7f7941ffb700 10 client.4440 handle_command_reply: tid=0 2023-08-27T11:12:16.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.546+0000 7f7975277b80 1 client.4440 shutdown 2023-08-27T11:12:16.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.546+0000 7f7975277b80 1 -- 172.21.15.45:0/4239907885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 msgr2=0x55cc0823e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.546+0000 7f7975277b80 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0823e080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f795c00e890 tx=0x7f795c00eba0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7975277b80 1 -- 172.21.15.45:0/4239907885 shutdown_connections 2023-08-27T11:12:16.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7975277b80 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x55cc08210700 0x55cc082488b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7975277b80 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc0823e080 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7975277b80 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 0x55cc0822df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7941ffb700 0 client.4440 ms_handle_reset on v2:172.21.15.135:6834/739639682 2023-08-27T11:12:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7975277b80 1 --2- 172.21.15.45:0/4239907885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 0x55cc0823db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7975277b80 1 -- 172.21.15.45:0/4239907885 >> 172.21.15.45:0/4239907885 conn(0x55cc08144c30 msgr2=0x55cc081e6210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7975277b80 1 -- 172.21.15.45:0/4239907885 shutdown_connections 2023-08-27T11:12:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7975277b80 1 -- 172.21.15.45:0/4239907885 wait complete. 2023-08-27T11:12:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.547+0000 7f7975277b80 20 client.4440 trim_cache size 0 max 16384 2023-08-27T11:12:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.565+0000 7f7975277b80 1 Processor -- start 2023-08-27T11:12:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.565+0000 7f7975277b80 1 -- start start 2023-08-27T11:12:16.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f7975277b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 0x55cc0822bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f7975277b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08210f40 0x55cc081e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f7975277b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc08222960 0x55cc081e6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f7975277b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0820ce00 con 0x55cc08210f40 2023-08-27T11:12:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f7975277b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc081e3b90 con 0x55cc081e43d0 2023-08-27T11:12:16.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f7975277b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc0820baf0 con 0x55cc08222960 2023-08-27T11:12:16.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f796259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08210f40 0x55cc081e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f796259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08210f40 0x55cc081e5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51534/0 (socket says 172.21.15.45:51534) 2023-08-27T11:12:16.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f796259c700 1 -- 172.21.15.45:0/929864142 learned_addr learned my addr 172.21.15.45:0/929864142 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f796259c700 1 -- 172.21.15.45:0/929864142 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc08222960 msgr2=0x55cc081e6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f796259c700 1 --2- 172.21.15.45:0/929864142 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc08222960 0x55cc081e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f796259c700 1 -- 172.21.15.45:0/929864142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 msgr2=0x55cc0822bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f796259c700 1 --2- 172.21.15.45:0/929864142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 0x55cc0822bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f796259c700 1 -- 172.21.15.45:0/929864142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc07f64d90 con 0x55cc08210f40 2023-08-27T11:12:16.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f796259c700 1 --2- 172.21.15.45:0/929864142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08210f40 0x55cc081e5f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7958008010 tx=0x7f7958012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5297cd518f7247 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f79497fa700 1 -- 172.21.15.45:0/929864142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958013b30 con 0x55cc08210f40 2023-08-27T11:12:16.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f79497fa700 1 -- 172.21.15.45:0/929864142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795800b040 con 0x55cc08210f40 2023-08-27T11:12:16.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.566+0000 7f79497fa700 1 -- 172.21.15.45:0/929864142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795801e440 con 0x55cc08210f40 2023-08-27T11:12:16.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 -- 172.21.15.45:0/929864142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08210f40 msgr2=0x55cc081e5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 --2- 172.21.15.45:0/929864142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08210f40 0x55cc081e5f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7958008010 tx=0x7f7958012000 comp rx=0 tx=0).stop 2023-08-27T11:12:16.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 -- 172.21.15.45:0/929864142 shutdown_connections 2023-08-27T11:12:16.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 --2- 172.21.15.45:0/929864142 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc08222960 0x55cc081e6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 --2- 172.21.15.45:0/929864142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc081e43d0 0x55cc0822bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 --2- 172.21.15.45:0/929864142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08210f40 0x55cc081e5f10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 -- 172.21.15.45:0/929864142 >> 172.21.15.45:0/929864142 conn(0x55cc08144c30 msgr2=0x55cc0822dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:16.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 -- 172.21.15.45:0/929864142 shutdown_connections 2023-08-27T11:12:16.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 -- 172.21.15.45:0/929864142 wait complete. 2023-08-27T11:12:16.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 -- start start 2023-08-27T11:12:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.567+0000 7f7975277b80 1 Processor -- start 2023-08-27T11:12:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f7975277b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 0x55cc08244a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f7975277b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 0x55cc081e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f7975277b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc081e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f7975277b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0822e2d0 con 0x55cc08222960 2023-08-27T11:12:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f7975277b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08222dd0 con 0x55cc08210f40 2023-08-27T11:12:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f7975277b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc081e55c0 con 0x55cc081e43d0 2023-08-27T11:12:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f7975277b80 10 client.0 _ll_register_callbacks cb 0x55cc080d4630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:12:16.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f796359e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc081e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f796359e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc081e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51542/0 (socket says 172.21.15.45:51542) 2023-08-27T11:12:16.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f796359e700 1 -- 172.21.15.45:0/2367187311 learned_addr learned my addr 172.21.15.45:0/2367187311 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:16.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f7962d9d700 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 0x55cc08244a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.568+0000 7f796259c700 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 0x55cc081e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f796359e700 1 -- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 msgr2=0x55cc08244a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f796359e700 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 0x55cc08244a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f796359e700 1 -- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 msgr2=0x55cc081e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f796359e700 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 0x55cc081e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f796359e700 1 -- 172.21.15.45:0/2367187311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc07fb85e0 con 0x55cc08222960 2023-08-27T11:12:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7962d9d700 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 0x55cc08244a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f796359e700 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc081e4d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f795c01f040 tx=0x7f795c004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b51368ad4653fa1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7941ffb700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:12:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7941ffb700 1 -- 172.21.15.45:0/2367187311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c003e10 con 0x55cc08222960 2023-08-27T11:12:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7941ffb700 1 -- 172.21.15.45:0/2367187311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f795c01c580 con 0x55cc08222960 2023-08-27T11:12:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7975277b80 1 -- 172.21.15.45:0/2367187311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7958017070 con 0x55cc08222960 2023-08-27T11:12:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7941ffb700 1 -- 172.21.15.45:0/2367187311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f795c0076e0 con 0x55cc08222960 2023-08-27T11:12:16.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7975277b80 10 client.4446 fetch_fsmap learned FSMap version 7 2023-08-27T11:12:16.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7975277b80 1 -- 172.21.15.45:0/2367187311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55cc07f4dc10 con 0x55cc08222960 2023-08-27T11:12:16.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7941ffb700 1 -- 172.21.15.45:0/2367187311 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7958017070 con 0x55cc08222960 2023-08-27T11:12:16.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7941ffb700 1 -- 172.21.15.45:0/2367187311 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f795c01c780 con 0x55cc08222960 2023-08-27T11:12:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7975277b80 10 client.4446 fetch_fsmap finished waiting for FSMap version 7 2023-08-27T11:12:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7975277b80 10 client.4446 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-27T11:12:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7975277b80 20 client.4446 populate_metadata read hostname 'smithi045' 2023-08-27T11:12:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7975277b80 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x55cc081dfe40 0x55cc08295640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7975277b80 4 client.4446 mds_command: new command op to 4303 tid=0[{"prefix": "status"}] 2023-08-27T11:12:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.569+0000 7f7975277b80 1 -- 172.21.15.45:0/2367187311 --> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cc080f2000 con 0x55cc081dfe40 2023-08-27T11:12:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7962d9d700 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x55cc081dfe40 0x55cc08295640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7941ffb700 10 client.4446 ms_handle_connect on v2:172.21.15.135:6834/739639682 2023-08-27T11:12:16.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7962d9d700 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x55cc081dfe40 0x55cc08295640 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6044a0b6f60e8d server_cookie=16fcb00ff3236ddb in_seq=0 out_seq=0 2023-08-27T11:12:16.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7941ffb700 1 -- 172.21.15.45:0/2367187311 <== mds.0 v2:172.21.15.135:6834/739639682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x55cc080f2000 con 0x55cc081dfe40 2023-08-27T11:12:16.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7941ffb700 10 client.4446 handle_command_reply: tid=0 2023-08-27T11:12:16.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 client.4446 shutdown 2023-08-27T11:12:16.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 -- 172.21.15.45:0/2367187311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 msgr2=0x55cc081e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc081e4d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f795c01f040 tx=0x7f795c004b00 comp rx=0 tx=0).stop 2023-08-27T11:12:16.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 -- 172.21.15.45:0/2367187311 shutdown_connections 2023-08-27T11:12:16.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x55cc081dfe40 0x55cc08295640 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc08222960 0x55cc081e4d40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc081e43d0 0x55cc08244a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7941ffb700 0 client.4446 ms_handle_reset on v2:172.21.15.135:6834/739639682 2023-08-27T11:12:16.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 --2- 172.21.15.45:0/2367187311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc08210f40 0x55cc081e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 -- 172.21.15.45:0/2367187311 >> 172.21.15.45:0/2367187311 conn(0x55cc08144c30 msgr2=0x55cc081e3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:16.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 -- 172.21.15.45:0/2367187311 shutdown_connections 2023-08-27T11:12:16.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 1 -- 172.21.15.45:0/2367187311 wait complete. 2023-08-27T11:12:16.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.570+0000 7f7975277b80 20 client.4446 trim_cache size 0 max 16384 2023-08-27T11:12:16.602 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:12:16.602 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "8ea1ae34-1697-4b79-b2ce-3f741cd3c828", 2023-08-27T11:12:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-08-27T11:12:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4303, 2023-08-27T11:12:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-08-27T11:12:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-08-27T11:12:16.603 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-08-27T11:12:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 7.658267135, 2023-08-27T11:12:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 7, 2023-08-27T11:12:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 27, 2023-08-27T11:12:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 26, 2023-08-27T11:12:16.604 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 13.587864033000001 2023-08-27T11:12:16.605 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:12:16.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 -- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f795006a0e0 msgr2=0x7f795006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f795006a0e0 0x7f795006c5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f79641e8200 tx=0x7f7958008040 comp rx=0 tx=0).stop 2023-08-27T11:12:16.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 -- 172.21.15.45:0/75105445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640cd880 msgr2=0x7f79640c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640cd880 0x7f79640c6d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7954009130 tx=0x7f7954004750 comp rx=0 tx=0).stop 2023-08-27T11:12:16.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 -- 172.21.15.45:0/75105445 shutdown_connections 2023-08-27T11:12:16.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640cd880 0x7f79640c6d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f795006a0e0 0x7f795006c5a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f796415d340 0x7f796415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 --2- 172.21.15.45:0/75105445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79640cdd00 0x7f7964159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 -- 172.21.15.45:0/75105445 >> 172.21.15.45:0/75105445 conn(0x7f79640cc7c0 msgr2=0x7f79640af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 -- 172.21.15.45:0/75105445 shutdown_connections 2023-08-27T11:12:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.571+0000 7f79690b5700 1 -- 172.21.15.45:0/75105445 wait complete. 2023-08-27T11:12:16.609 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-27T11:12:16.609 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-27T11:12:16.610 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:12:16.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.735+0000 7f18b57a5700 1 Processor -- start 2023-08-27T11:12:16.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.735+0000 7f18b57a5700 1 -- start start 2023-08-27T11:12:16.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18b57a5700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f18b00b3ad0 0x7f18b0168730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18b57a5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b00d1d40 con 0x7f18b00b45d0 2023-08-27T11:12:16.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18b57a5700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b00ceae0 con 0x7f18b00cf980 2023-08-27T11:12:16.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18b57a5700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00cf0a0 con 0x7f18b00b3ad0 2023-08-27T11:12:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18ae7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f18b00b45d0 0x7f18b00d1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38322/0 (socket says 172.21.15.45:38322) 2023-08-27T11:12:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18ae7fc700 1 -- 172.21.15.45:0/406353635 learned_addr learned my addr 172.21.15.45:0/406353635 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18af7fe700 1 --2- 172.21.15.45:0/406353635 >> v2:172.21.15.135:3300/0 conn(0x7f18b00b3ad0 0x7f18b0168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522553333 0 0) 0x7f18b00d1d40 con 0x7f18b00b45d0 2023-08-27T11:12:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18980035b0 con 0x7f18b00b45d0 2023-08-27T11:12:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1876544636 0 0) 0x7f18980035b0 con 0x7f18b00b45d0 2023-08-27T11:12:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 >> v2:172.21.15.135:3300/0 conn(0x7f18b00b3ad0 msgr2=0x7f18b0168730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 --2- 172.21.15.45:0/406353635 >> v2:172.21.15.135:3300/0 conn(0x7f18b00b3ad0 0x7f18b0168730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 >> v1:172.21.15.135:6789/0 conn(0x7f18b00cf980 legacy=0x7f18b00d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00cedc0 con 0x7f18b00b45d0 2023-08-27T11:12:16.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f18a0001670 con 0x7f18b00b45d0 2023-08-27T11:12:16.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 >> v1:172.21.15.45:6789/0 conn(0x7f18b00b45d0 legacy=0x7f18b00d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1898003a00 0x7f1898003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.736+0000 7f18adffb700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1898004850 0x7f1898004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18adffb700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1898005210 0x7f189800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00cf0a0 con 0x7f1898003a00 2023-08-27T11:12:16.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189800c0b0 con 0x7f1898004850 2023-08-27T11:12:16.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189800c270 con 0x7f1898005210 2023-08-27T11:12:16.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18aeffd700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1898003a00 0x7f1898003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18ae7fc700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1898004850 0x7f1898004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18af7fe700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1898005210 0x7f189800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18aeffd700 1 -- 172.21.15.45:0/406353635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1898005210 msgr2=0x7f189800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18aeffd700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1898005210 0x7f189800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18aeffd700 1 -- 172.21.15.45:0/406353635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1898004850 msgr2=0x7f1898004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18aeffd700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1898004850 0x7f1898004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18aeffd700 1 -- 172.21.15.45:0/406353635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980035b0 con 0x7f1898003a00 2023-08-27T11:12:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18af7fe700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1898005210 0x7f189800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18af7fe700 1 -- 172.21.15.45:0/406353635 reap_dead start 2023-08-27T11:12:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18aeffd700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1898003a00 0x7f1898003e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f18a40017c0 tx=0x7f1898005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b931b51df82924b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40052d0 con 0x7f1898003a00 2023-08-27T11:12:16.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a4005430 con 0x7f1898003a00 2023-08-27T11:12:16.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.737+0000 7f18adffb700 1 -- 172.21.15.45:0/406353635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40038c0 con 0x7f1898003a00 2023-08-27T11:12:16.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.738+0000 7f18b57a5700 1 -- 172.21.15.45:0/406353635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1898003a00 msgr2=0x7f1898003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.738+0000 7f18b57a5700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1898003a00 0x7f1898003e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f18a40017c0 tx=0x7f1898005880 comp rx=0 tx=0).stop 2023-08-27T11:12:16.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.738+0000 7f18b57a5700 1 -- 172.21.15.45:0/406353635 shutdown_connections 2023-08-27T11:12:16.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.738+0000 7f18b57a5700 1 --2- 172.21.15.45:0/406353635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1898003a00 0x7f1898003e20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.738+0000 7f18b57a5700 1 -- 172.21.15.45:0/406353635 >> 172.21.15.45:0/406353635 conn(0x7f18b00d3020 msgr2=0x7f18b00d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:16.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.738+0000 7f18b57a5700 1 -- 172.21.15.45:0/406353635 shutdown_connections 2023-08-27T11:12:16.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.738+0000 7f18b57a5700 1 -- 172.21.15.45:0/406353635 wait complete. 2023-08-27T11:12:16.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.739+0000 7f18b57a5700 1 Processor -- start 2023-08-27T11:12:16.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.739+0000 7f18b57a5700 1 -- start start 2023-08-27T11:12:16.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18b57a5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1898003a00 0x7f18b00c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18b57a5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18b00c8640 0x7f18b015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18b57a5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b015b3b0 0x7f18b015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18b57a5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b015bda0 con 0x7f18b015b3b0 2023-08-27T11:12:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18b57a5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00c6ea0 con 0x7f1898003a00 2023-08-27T11:12:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18b57a5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b015ac90 con 0x7f18b00c8640 2023-08-27T11:12:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b015b3b0 0x7f18b015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b015b3b0 0x7f18b015b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51562/0 (socket says 172.21.15.45:51562) 2023-08-27T11:12:16.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18ae7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18b00c8640 0x7f18b015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18af7fe700 1 -- 172.21.15.45:0/1119421645 learned_addr learned my addr 172.21.15.45:0/1119421645 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:16.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18aeffd700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1898003a00 0x7f18b00c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18af7fe700 1 -- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18b00c8640 msgr2=0x7f18b015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18af7fe700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18b00c8640 0x7f18b015a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18af7fe700 1 -- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1898003a00 msgr2=0x7f18b00c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:16.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18af7fe700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1898003a00 0x7f18b00c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:16.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.740+0000 7f18af7fe700 1 -- 172.21.15.45:0/1119421645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00c8c00 con 0x7f18b015b3b0 2023-08-27T11:12:16.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.741+0000 7f18af7fe700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b015b3b0 0x7f18b015b860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f18b00b3f60 tx=0x7f18a80101c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.741+0000 7f18977fe700 1 -- 172.21.15.45:0/1119421645 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a801b070 con 0x7f18b015b3b0 2023-08-27T11:12:16.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.741+0000 7f18977fe700 1 -- 172.21.15.45:0/1119421645 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a800f040 con 0x7f18b015b3b0 2023-08-27T11:12:16.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.741+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b00c8ee0 con 0x7f18b015b3b0 2023-08-27T11:12:16.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.741+0000 7f18977fe700 1 -- 172.21.15.45:0/1119421645 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a800e050 con 0x7f18b015b3b0 2023-08-27T11:12:16.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.741+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b00c94a0 con 0x7f18b015b3b0 2023-08-27T11:12:16.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.742+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b00cb350 con 0x7f18b015b3b0 2023-08-27T11:12:16.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.742+0000 7f18977fe700 1 -- 172.21.15.45:0/1119421645 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f18a8010a50 con 0x7f18b015b3b0 2023-08-27T11:12:16.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.743+0000 7f18977fe700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f189c06a0e0 0x7f189c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:16.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.743+0000 7f18977fe700 1 -- 172.21.15.45:0/1119421645 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f18a80889a0 con 0x7f18b015b3b0 2023-08-27T11:12:16.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.743+0000 7f18aeffd700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f189c06a0e0 0x7f189c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:16.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.743+0000 7f18aeffd700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f189c06a0e0 0x7f189c06c5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18a4000c00 tx=0x7f18a4009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:16.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.747+0000 7f18977fe700 1 -- 172.21.15.45:0/1119421645 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18a8053e30 con 0x7f18b015b3b0 2023-08-27T11:12:17.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.998+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f18b00ce6a0 con 0x7f18b015b3b0 2023-08-27T11:12:17.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:16.998+0000 7f18977fe700 1 -- 172.21.15.45:0/1119421645 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f18a8057450 con 0x7f18b015b3b0 2023-08-27T11:12:17.001 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-27T11:12:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f189c06a0e0 msgr2=0x7f189c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f189c06a0e0 0x7f189c06c5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18a4000c00 tx=0x7f18a4009030 comp rx=0 tx=0).stop 2023-08-27T11:12:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b015b3b0 msgr2=0x7f18b015b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b015b3b0 0x7f18b015b860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f18b00b3f60 tx=0x7f18a80101c0 comp rx=0 tx=0).stop 2023-08-27T11:12:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 shutdown_connections 2023-08-27T11:12:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b015b3b0 0x7f18b015b860 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f189c06a0e0 0x7f189c06c5a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1898003a00 0x7f18b00c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 --2- 172.21.15.45:0/1119421645 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18b00c8640 0x7f18b015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 >> 172.21.15.45:0/1119421645 conn(0x7f18b00d3020 msgr2=0x7f18b00b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 shutdown_connections 2023-08-27T11:12:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:17.000+0000 7f18b57a5700 1 -- 172.21.15.45:0/1119421645 wait complete. 2023-08-27T11:12:17.013 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:12:17.013 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:12:17.014 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-08-27T11:12:17.014 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-08-27T11:12:17.014 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-27T11:12:17.014 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:12:17.015 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-27T11:12:17.015 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:12:17.015 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-08-27T11:12:17.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:17.016 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-08-27T11:12:17.033 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:12:17.034 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:12:17.034 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:12:17.034 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:17.034 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:12:17.035 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:17.035 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:17.035 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:17.036 INFO:teuthology.orchestra.run.smithi045.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:12:17.036 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:17.036 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-27T11:12:17.036 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42404sec preferred_lft 42404sec 2023-08-27T11:12:17.036 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-08-27T11:12:17.037 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:17.037 INFO:teuthology.orchestra.run.smithi045.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:17.037 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:17.037 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:17.037 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:17.038 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-27T11:12:17.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:17.039 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:17.039 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:17.039 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2023-08-27T11:12:17.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2023-08-27T11:12:17.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2023-08-27T11:12:17.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-27T11:12:17.041 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:17.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:17.288 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:12:17.316 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-08-27T11:12:17.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:17.318 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-08-27T11:12:17.446 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-08-27T11:12:17.446 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:12:17.447 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-27T11:12:17.447 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-27T11:12:17.447 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:12:17.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:17.448 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:17.449 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:17.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-27T11:12:17.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-27T11:12:17.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-27T11:12:17.450 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:18.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:18.332 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:12:18.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:18.352 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2023-08-27T11:12:18.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:18.410 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:18.411 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:18.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-27T11:12:18.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-08-27T11:12:18.412 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:18.509 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-08-27T11:12:18.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:18.510 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:18.511 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:18.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-08-27T11:12:18.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-27T11:12:18.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-08-27T11:12:18.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-08-27T11:12:18.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-08-27T11:12:18.513 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:18.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:18.752 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:18.752 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:18.753 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-08-27T11:12:18.753 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2023-08-27T11:12:18.754 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:18.857 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-08-27T11:12:18.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:18.858 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-08-27T11:12:18.885 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-08-27T11:12:18.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:18.886 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-08-27T11:12:18.942 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-08-27T11:12:19.011 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-27T11:12:19.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-27T11:12:19.078 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-27T11:12:19.080 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-27T11:12:19.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:19.081 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-08-27T11:12:19.139 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-27T11:12:19.139 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-08-27T11:12:19.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2023-08-27T11:12:19.209 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-27T11:12:19.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-27T11:12:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-27T11:12:19.278 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-27T11:12:19.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:19.279 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2023-08-27T11:12:19.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:2023-08-27T11:12:19.281+0000 7f9def6c55c0 -1 init, newargv = 0x55f4f16abf90 newargc=15 2023-08-27T11:12:19.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[110820]: starting ceph client 2023-08-27T11:12:19.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[110820]: starting fuse 2023-08-27T11:12:19.299 INFO:teuthology.orchestra.run.smithi045.stdout:47 2023-08-27T11:12:19.299 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-08-27T11:12:19.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo stdin-killer -- python3 -c ' 2023-08-27T11:12:19.300 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2023-08-27T11:12:19.301 DEBUG:teuthology.orchestra.run.smithi045:> import re 2023-08-27T11:12:19.301 DEBUG:teuthology.orchestra.run.smithi045:> import os 2023-08-27T11:12:19.302 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2023-08-27T11:12:19.302 DEBUG:teuthology.orchestra.run.smithi045:> 2023-08-27T11:12:19.302 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2023-08-27T11:12:19.303 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-08-27T11:12:19.303 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2023-08-27T11:12:19.303 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-08-27T11:12:19.304 DEBUG:teuthology.orchestra.run.smithi045:> 2023-08-27T11:12:19.304 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2023-08-27T11:12:19.304 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2023-08-27T11:12:19.305 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2023-08-27T11:12:19.305 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2023-08-27T11:12:19.305 DEBUG:teuthology.orchestra.run.smithi045:> 2023-08-27T11:12:19.306 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2023-08-27T11:12:19.306 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-27T11:12:19.306 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-27T11:12:19.307 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-27T11:12:19.307 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2023-08-27T11:12:19.308 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2023-08-27T11:12:19.309 DEBUG:teuthology.orchestra.run.smithi045:> return f 2023-08-27T11:12:19.309 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-27T11:12:19.310 DEBUG:teuthology.orchestra.run.smithi045:> 2023-08-27T11:12:19.311 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.admin")) 2023-08-27T11:12:19.311 DEBUG:teuthology.orchestra.run.smithi045:> ' 2023-08-27T11:12:19.425 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.admin.110820.asok 2023-08-27T11:12:19.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-27T11:12:19.434 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.110820.asok 2023-08-27T11:12:19.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:19.435 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.110820.asok status 2023-08-27T11:12:19.594 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:12:19.594 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2023-08-27T11:12:19.595 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "1da267a3afa9ab8f9b1d164359066f8bd682ffd8", 2023-08-27T11:12:19.595 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 18.0.0-5707-g1da267a3 (1da267a3afa9ab8f9b1d164359066f8bd682ffd8) reef (dev)", 2023-08-27T11:12:19.595 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "admin", 2023-08-27T11:12:19.595 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2023-08-27T11:12:19.595 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-08-27T11:12:19.596 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "110820", 2023-08-27T11:12:19.596 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2023-08-27T11:12:19.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:12:19.596 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2023-08-27T11:12:19.596 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2023-08-27T11:12:19.596 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4461, 2023-08-27T11:12:19.597 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2023-08-27T11:12:19.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2023-08-27T11:12:19.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2023-08-27T11:12:19.597 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4461 2023-08-27T11:12:19.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:12:19.598 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-08-27T11:12:19.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-08-27T11:12:19.598 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-08-27T11:12:19.598 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 251961677 2023-08-27T11:12:19.598 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:12:19.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:12:19.599 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2023-08-27T11:12:19.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2023-08-27T11:12:19.599 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2023-08-27T11:12:19.599 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 251961677 2023-08-27T11:12:19.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:12:19.600 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.4461 192.168.0.1:0/251961677", 2023-08-27T11:12:19.600 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/251961677", 2023-08-27T11:12:19.600 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2023-08-27T11:12:19.600 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 7, 2023-08-27T11:12:19.601 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 27, 2023-08-27T11:12:19.601 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2023-08-27T11:12:19.601 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2023-08-27T11:12:19.601 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2023-08-27T11:12:19.602 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:12:19.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:19.606 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2023-08-27T11:12:19.606 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-08-27T11:12:19.624 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2023-08-27T11:12:19.624 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-08-27T11:12:19.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:19.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-08-27T11:12:19.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:19.694 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-08-27T11:12:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:# file: . 2023-08-27T11:12:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-08-27T11:12:19.794 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:19.794 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:12:19.794 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:12:19.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:19.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.976+0000 7f37dbfc1700 1 Processor -- start 2023-08-27T11:12:19.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.976+0000 7f37dbfc1700 1 -- start start 2023-08-27T11:12:19.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.977+0000 7f37dbfc1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 0x7f37d40700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:19.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.977+0000 7f37dbfc1700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f37d40ca210 0x7f37d40cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:19.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.977+0000 7f37dbfc1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d40cd350 con 0x7f37d40705f0 2023-08-27T11:12:19.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.977+0000 7f37dbfc1700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d406bdc0 con 0x7f37d40ca210 2023-08-27T11:12:19.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.977+0000 7f37dbfc1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d406bfd0 con 0x7f37d41599e0 2023-08-27T11:12:19.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.977+0000 7f37d955c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f37d40705f0 0x7f37d40c9b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38360/0 (socket says 172.21.15.45:38360) 2023-08-27T11:12:19.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.977+0000 7f37d955c700 1 -- 172.21.15.45:0/959236548 learned_addr learned my addr 172.21.15.45:0/959236548 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:19.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37da55e700 1 --2- 172.21.15.45:0/959236548 >> v2:172.21.15.135:3300/0 conn(0x7f37d40ca210 0x7f37d40cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:19.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d9d5d700 1 --2- 172.21.15.45:0/959236548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 0x7f37d40700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:19.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d8d5b700 1 -- 172.21.15.45:0/959236548 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464420400 0 0) 0x7f37d40cd350 con 0x7f37d40705f0 2023-08-27T11:12:19.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d8d5b700 1 -- 172.21.15.45:0/959236548 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c00035b0 con 0x7f37d40705f0 2023-08-27T11:12:19.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d9d5d700 1 -- 172.21.15.45:0/959236548 >> v2:172.21.15.135:3300/0 conn(0x7f37d40ca210 msgr2=0x7f37d40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:19.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d9d5d700 1 --2- 172.21.15.45:0/959236548 >> v2:172.21.15.135:3300/0 conn(0x7f37d40ca210 0x7f37d40cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:19.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d9d5d700 1 -- 172.21.15.45:0/959236548 >> v1:172.21.15.45:6789/0 conn(0x7f37d40705f0 legacy=0x7f37d40c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d9d5d700 1 -- 172.21.15.45:0/959236548 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d406c2b0 con 0x7f37d41599e0 2023-08-27T11:12:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37da55e700 1 --2- 172.21.15.45:0/959236548 >> v2:172.21.15.135:3300/0 conn(0x7f37d40ca210 0x7f37d40cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d9d5d700 1 --2- 172.21.15.45:0/959236548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 0x7f37d40700b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009840 tx=0x7f37d0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7fad458892e7b95 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d8d5b700 1 -- 172.21.15.45:0/959236548 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0011830 con 0x7f37d41599e0 2023-08-27T11:12:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d8d5b700 1 -- 172.21.15.45:0/959236548 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d0011990 con 0x7f37d41599e0 2023-08-27T11:12:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.978+0000 7f37d8d5b700 1 -- 172.21.15.45:0/959236548 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0011d20 con 0x7f37d41599e0 2023-08-27T11:12:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.979+0000 7f37dbfc1700 1 -- 172.21.15.45:0/959236548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 msgr2=0x7f37d40700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.979+0000 7f37dbfc1700 1 --2- 172.21.15.45:0/959236548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 0x7f37d40700b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009840 tx=0x7f37d0009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.979+0000 7f37dbfc1700 1 -- 172.21.15.45:0/959236548 shutdown_connections 2023-08-27T11:12:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.979+0000 7f37dbfc1700 1 --2- 172.21.15.45:0/959236548 >> v2:172.21.15.135:3300/0 conn(0x7f37d40ca210 0x7f37d40cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.979+0000 7f37dbfc1700 1 --2- 172.21.15.45:0/959236548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 0x7f37d40700b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.979+0000 7f37dbfc1700 1 -- 172.21.15.45:0/959236548 >> 172.21.15.45:0/959236548 conn(0x7f37d40b07c0 msgr2=0x7f37d40b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.979+0000 7f37dbfc1700 1 -- 172.21.15.45:0/959236548 shutdown_connections 2023-08-27T11:12:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.979+0000 7f37dbfc1700 1 -- 172.21.15.45:0/959236548 wait complete. 2023-08-27T11:12:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.980+0000 7f37dbfc1700 1 Processor -- start 2023-08-27T11:12:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.980+0000 7f37dbfc1700 1 -- start start 2023-08-27T11:12:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37dbfc1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d40705f0 0x7f37d41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37dbfc1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37d40ca210 0x7f37d40b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37dbfc1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 0x7f37d40ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37dbfc1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40ac9e0 con 0x7f37d40705f0 2023-08-27T11:12:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37dbfc1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40accc0 con 0x7f37d40ca210 2023-08-27T11:12:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37dbfc1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40acfa0 con 0x7f37d41599e0 2023-08-27T11:12:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d9d5d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d40705f0 0x7f37d41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d9d5d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d40705f0 0x7f37d41552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51588/0 (socket says 172.21.15.45:51588) 2023-08-27T11:12:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d9d5d700 1 -- 172.21.15.45:0/4246221141 learned_addr learned my addr 172.21.15.45:0/4246221141 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d955c700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37d40ca210 0x7f37d40b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37da55e700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 0x7f37d40ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d9d5d700 1 -- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 msgr2=0x7f37d40ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d9d5d700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 0x7f37d40ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d9d5d700 1 -- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37d40ca210 msgr2=0x7f37d40b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d9d5d700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37d40ca210 0x7f37d40b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d9d5d700 1 -- 172.21.15.45:0/4246221141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc009a70 con 0x7f37d40705f0 2023-08-27T11:12:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.981+0000 7f37d9d5d700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d40705f0 0x7f37d41552a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f37d00090b0 tx=0x7f37d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.982+0000 7f37ca7fc700 1 -- 172.21.15.45:0/4246221141 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0004550 con 0x7f37d40705f0 2023-08-27T11:12:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.982+0000 7f37ca7fc700 1 -- 172.21.15.45:0/4246221141 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d001e070 con 0x7f37d40705f0 2023-08-27T11:12:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.982+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d00094a0 con 0x7f37d40705f0 2023-08-27T11:12:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.982+0000 7f37ca7fc700 1 -- 172.21.15.45:0/4246221141 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d00038e0 con 0x7f37d40705f0 2023-08-27T11:12:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.982+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d40ad560 con 0x7f37d40705f0 2023-08-27T11:12:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.983+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d40c9060 con 0x7f37d40705f0 2023-08-27T11:12:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.983+0000 7f37ca7fc700 1 -- 172.21.15.45:0/4246221141 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f37d0003a40 con 0x7f37d40705f0 2023-08-27T11:12:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.984+0000 7f37ca7fc700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f37b806a0e0 0x7f37b806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.984+0000 7f37ca7fc700 1 -- 172.21.15.45:0/4246221141 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f37d00895d0 con 0x7f37d40705f0 2023-08-27T11:12:19.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.984+0000 7f37d955c700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f37b806a0e0 0x7f37b806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.984+0000 7f37d955c700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f37b806a0e0 0x7f37b806c5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37c4007fd0 tx=0x7f37c4007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:19.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:19.988+0000 7f37ca7fc700 1 -- 172.21.15.45:0/4246221141 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f37d0054a60 con 0x7f37d40705f0 2023-08-27T11:12:20.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.189+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37d40ad750 con 0x7f37d40705f0 2023-08-27T11:12:20.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.190+0000 7f37ca7fc700 1 -- 172.21.15.45:0/4246221141 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f37d0058080 con 0x7f37d40705f0 2023-08-27T11:12:20.191 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:20.191 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:20.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f37b806a0e0 msgr2=0x7f37b806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f37b806a0e0 0x7f37b806c5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37c4007fd0 tx=0x7f37c4007dc0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d40705f0 msgr2=0x7f37d41552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d40705f0 0x7f37d41552a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f37d00090b0 tx=0x7f37d0004750 comp rx=0 tx=0).stop 2023-08-27T11:12:20.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 shutdown_connections 2023-08-27T11:12:20.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d40705f0 0x7f37d41552a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f37b806a0e0 0x7f37b806c5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37d41599e0 0x7f37d40ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 --2- 172.21.15.45:0/4246221141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37d40ca210 0x7f37d40b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 >> 172.21.15.45:0/4246221141 conn(0x7f37d40b07c0 msgr2=0x7f37d4071750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 shutdown_connections 2023-08-27T11:12:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.192+0000 7f37dbfc1700 1 -- 172.21.15.45:0/4246221141 wait complete. 2023-08-27T11:12:20.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:12:20.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:12:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.332+0000 7ff93b4d3700 1 Processor -- start 2023-08-27T11:12:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.333+0000 7ff93b4d3700 1 -- start start 2023-08-27T11:12:20.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.333+0000 7ff93b4d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff934160d00 0x7ff93406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.333+0000 7ff93b4d3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff93406a7c0 0x7ff9340b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.333+0000 7ff93b4d3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93406b160 con 0x7ff93415d340 2023-08-27T11:12:20.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.333+0000 7ff93b4d3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff934072010 con 0x7ff93406a7c0 2023-08-27T11:12:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.333+0000 7ff93b4d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93406ac50 con 0x7ff934160d00 2023-08-27T11:12:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff939a70700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff93406a7c0 0x7ff9340b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff939a70700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff93406a7c0 0x7ff9340b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51600/0 (socket says 172.21.15.45:51600) 2023-08-27T11:12:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff939a70700 1 -- 172.21.15.45:0/3769505251 learned_addr learned my addr 172.21.15.45:0/3769505251 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff938a6e700 1 --2- 172.21.15.45:0/3769505251 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff934160d00 0x7ff93406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff939a70700 1 -- 172.21.15.45:0/3769505251 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff934160d00 msgr2=0x7ff93406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff939a70700 1 --2- 172.21.15.45:0/3769505251 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff934160d00 0x7ff93406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff939a70700 1 -- 172.21.15.45:0/3769505251 >> v1:172.21.15.135:6789/0 conn(0x7ff93415d340 legacy=0x7ff9340cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff939a70700 1 -- 172.21.15.45:0/3769505251 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9340cdf90 con 0x7ff93406a7c0 2023-08-27T11:12:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff939a70700 1 --2- 172.21.15.45:0/3769505251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93406a7c0 0x7ff9340b18a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009e50 tx=0x7ff92c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abe4bf59be13432 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:20.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff927fff700 1 -- 172.21.15.45:0/3769505251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c00e070 con 0x7ff93406a7c0 2023-08-27T11:12:20.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff927fff700 1 -- 172.21.15.45:0/3769505251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c003a20 con 0x7ff93406a7c0 2023-08-27T11:12:20.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.334+0000 7ff927fff700 1 -- 172.21.15.45:0/3769505251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c003dd0 con 0x7ff93406a7c0 2023-08-27T11:12:20.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.335+0000 7ff93b4d3700 1 -- 172.21.15.45:0/3769505251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93406a7c0 msgr2=0x7ff9340b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.335+0000 7ff93b4d3700 1 --2- 172.21.15.45:0/3769505251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93406a7c0 0x7ff9340b18a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009e50 tx=0x7ff92c00b330 comp rx=0 tx=0).stop 2023-08-27T11:12:20.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.335+0000 7ff93b4d3700 1 -- 172.21.15.45:0/3769505251 shutdown_connections 2023-08-27T11:12:20.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.335+0000 7ff93b4d3700 1 --2- 172.21.15.45:0/3769505251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93406a7c0 0x7ff9340b18a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.335+0000 7ff93b4d3700 1 --2- 172.21.15.45:0/3769505251 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff934160d00 0x7ff93406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.335+0000 7ff93b4d3700 1 -- 172.21.15.45:0/3769505251 >> 172.21.15.45:0/3769505251 conn(0x7ff9340cc7c0 msgr2=0x7ff9340ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:20.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.335+0000 7ff93b4d3700 1 -- 172.21.15.45:0/3769505251 shutdown_connections 2023-08-27T11:12:20.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.335+0000 7ff93b4d3700 1 -- 172.21.15.45:0/3769505251 wait complete. 2023-08-27T11:12:20.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.336+0000 7ff93b4d3700 1 Processor -- start 2023-08-27T11:12:20.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.336+0000 7ff93b4d3700 1 -- start start 2023-08-27T11:12:20.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff93b4d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff93406a7c0 0x7ff93406f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff93b4d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93415d340 0x7ff9340c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff93b4d3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff934160d00 0x7ff93414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff93b4d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341f6f40 con 0x7ff93415d340 2023-08-27T11:12:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff93b4d3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340c5280 con 0x7ff934160d00 2023-08-27T11:12:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff93b4d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340c5490 con 0x7ff93406a7c0 2023-08-27T11:12:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff938a6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93415d340 0x7ff9340c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff938a6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93415d340 0x7ff9340c4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51604/0 (socket says 172.21.15.45:51604) 2023-08-27T11:12:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff938a6e700 1 -- 172.21.15.45:0/2698452508 learned_addr learned my addr 172.21.15.45:0/2698452508 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff93926f700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff93406a7c0 0x7ff93406f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff939a70700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff934160d00 0x7ff93414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff938a6e700 1 -- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff93406a7c0 msgr2=0x7ff93406f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff938a6e700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff93406a7c0 0x7ff93406f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff938a6e700 1 -- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff934160d00 msgr2=0x7ff93414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff938a6e700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff934160d00 0x7ff93414b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff938a6e700 1 -- 172.21.15.45:0/2698452508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c009b30 con 0x7ff93415d340 2023-08-27T11:12:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.337+0000 7ff938a6e700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93415d340 0x7ff9340c4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff93000e810 tx=0x7ff93000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.338+0000 7ff925ffb700 1 -- 172.21.15.45:0/2698452508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000c070 con 0x7ff93415d340 2023-08-27T11:12:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.338+0000 7ff925ffb700 1 -- 172.21.15.45:0/2698452508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff930013610 con 0x7ff93415d340 2023-08-27T11:12:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.338+0000 7ff925ffb700 1 -- 172.21.15.45:0/2698452508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9300139c0 con 0x7ff93415d340 2023-08-27T11:12:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.338+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9340c5770 con 0x7ff93415d340 2023-08-27T11:12:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.338+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9340c5d30 con 0x7ff93415d340 2023-08-27T11:12:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.339+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff934072690 con 0x7ff93415d340 2023-08-27T11:12:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.339+0000 7ff925ffb700 1 -- 172.21.15.45:0/2698452508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff930013b20 con 0x7ff93415d340 2023-08-27T11:12:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.340+0000 7ff925ffb700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff91806a0e0 0x7ff91806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.340+0000 7ff925ffb700 1 -- 172.21.15.45:0/2698452508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff9300885f0 con 0x7ff93415d340 2023-08-27T11:12:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.340+0000 7ff93926f700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff91806a0e0 0x7ff91806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.340+0000 7ff93926f700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff91806a0e0 0x7ff91806c5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff928007fd0 tx=0x7ff928007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.344+0000 7ff925ffb700 1 -- 172.21.15.45:0/2698452508 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff930053960 con 0x7ff93415d340 2023-08-27T11:12:20.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.544+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9340aeff0 con 0x7ff93415d340 2023-08-27T11:12:20.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.544+0000 7ff925ffb700 1 -- 172.21.15.45:0/2698452508 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff930056f80 con 0x7ff93415d340 2023-08-27T11:12:20.546 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:12:20.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff91806a0e0 msgr2=0x7ff91806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff91806a0e0 0x7ff91806c5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff928007fd0 tx=0x7ff928007ec0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93415d340 msgr2=0x7ff9340c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93415d340 0x7ff9340c4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff93000e810 tx=0x7ff93000eb20 comp rx=0 tx=0).stop 2023-08-27T11:12:20.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 shutdown_connections 2023-08-27T11:12:20.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff91806a0e0 0x7ff91806c5a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff934160d00 0x7ff93414b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff93406a7c0 0x7ff93406f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 --2- 172.21.15.45:0/2698452508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff93415d340 0x7ff9340c4d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 >> 172.21.15.45:0/2698452508 conn(0x7ff9340cc7c0 msgr2=0x7ff93415d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:20.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 shutdown_connections 2023-08-27T11:12:20.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.547+0000 7ff93b4d3700 1 -- 172.21.15.45:0/2698452508 wait complete. 2023-08-27T11:12:20.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:12:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.687+0000 7f3034be0700 1 Processor -- start 2023-08-27T11:12:20.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.687+0000 7f3034be0700 1 -- start start 2023-08-27T11:12:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f3034be0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f303015d340 0x7f3030160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f3034be0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f30300cd880 0x7f303006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f3034be0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303006b160 con 0x7f30300cdd00 2023-08-27T11:12:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f3034be0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3030072010 con 0x7f303015d340 2023-08-27T11:12:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f3034be0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30300af590 con 0x7f30300cd880 2023-08-27T11:12:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f302e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f303015d340 0x7f3030160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f302e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f303015d340 0x7f3030160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51610/0 (socket says 172.21.15.45:51610) 2023-08-27T11:12:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f302e59c700 1 -- 172.21.15.45:0/259839847 learned_addr learned my addr 172.21.15.45:0/259839847 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f302dd9b700 1 --2- 172.21.15.45:0/259839847 >> v2:172.21.15.135:3300/0 conn(0x7f30300cd880 0x7f303006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f302e59c700 1 -- 172.21.15.45:0/259839847 >> v2:172.21.15.135:3300/0 conn(0x7f30300cd880 msgr2=0x7f303006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f302e59c700 1 --2- 172.21.15.45:0/259839847 >> v2:172.21.15.135:3300/0 conn(0x7f30300cd880 0x7f303006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f302e59c700 1 -- 172.21.15.45:0/259839847 >> v1:172.21.15.135:6789/0 conn(0x7f30300cdd00 legacy=0x7f30300b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f302e59c700 1 -- 172.21.15.45:0/259839847 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300af2b0 con 0x7f303015d340 2023-08-27T11:12:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.688+0000 7f302e59c700 1 --2- 172.21.15.45:0/259839847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f303015d340 0x7f3030160d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30300cd7f0 tx=0x7f3024009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f5457b0f77c69f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f302d59a700 1 -- 172.21.15.45:0/259839847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024004230 con 0x7f303015d340 2023-08-27T11:12:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f302d59a700 1 -- 172.21.15.45:0/259839847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3024004390 con 0x7f303015d340 2023-08-27T11:12:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f302d59a700 1 -- 172.21.15.45:0/259839847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240036a0 con 0x7f303015d340 2023-08-27T11:12:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f3034be0700 1 -- 172.21.15.45:0/259839847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f303015d340 msgr2=0x7f3030160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f3034be0700 1 --2- 172.21.15.45:0/259839847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f303015d340 0x7f3030160d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30300cd7f0 tx=0x7f3024009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f3034be0700 1 -- 172.21.15.45:0/259839847 shutdown_connections 2023-08-27T11:12:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f3034be0700 1 --2- 172.21.15.45:0/259839847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f303015d340 0x7f3030160d00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f3034be0700 1 --2- 172.21.15.45:0/259839847 >> v2:172.21.15.135:3300/0 conn(0x7f30300cd880 0x7f303006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f3034be0700 1 -- 172.21.15.45:0/259839847 >> 172.21.15.45:0/259839847 conn(0x7f30300cc7c0 msgr2=0x7f30300ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f3034be0700 1 -- 172.21.15.45:0/259839847 shutdown_connections 2023-08-27T11:12:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.689+0000 7f3034be0700 1 -- 172.21.15.45:0/259839847 wait complete. 2023-08-27T11:12:20.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f3034be0700 1 Processor -- start 2023-08-27T11:12:20.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f3034be0700 1 -- start start 2023-08-27T11:12:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f3034be0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30300cd880 0x7f3030154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f3034be0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30300cdd00 0x7f30300c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f3034be0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303015d340 0x7f3030147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f3034be0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30300c8630 con 0x7f30300cd880 2023-08-27T11:12:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f3034be0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30300c72c0 con 0x7f303015d340 2023-08-27T11:12:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f3034be0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30300c7590 con 0x7f30300cdd00 2023-08-27T11:12:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f302e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30300cd880 0x7f3030154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f302e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30300cd880 0x7f3030154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51616/0 (socket says 172.21.15.45:51616) 2023-08-27T11:12:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.691+0000 7f302e59c700 1 -- 172.21.15.45:0/3341351019 learned_addr learned my addr 172.21.15.45:0/3341351019 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f302dd9b700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30300cdd00 0x7f30300c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f302ed9d700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303015d340 0x7f3030147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f302e59c700 1 -- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30300cdd00 msgr2=0x7f30300c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f302e59c700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30300cdd00 0x7f30300c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f302e59c700 1 -- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303015d340 msgr2=0x7f3030147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f302e59c700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303015d340 0x7f3030147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f302e59c700 1 -- 172.21.15.45:0/3341351019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024009550 con 0x7f30300cd880 2023-08-27T11:12:20.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f302ed9d700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303015d340 0x7f3030147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:20.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f302e59c700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30300cd880 0x7f3030154fa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3024009130 tx=0x7f3024004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:20.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f301effd700 1 -- 172.21.15.45:0/3341351019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024003e20 con 0x7f30300cd880 2023-08-27T11:12:20.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30300c7870 con 0x7f30300cd880 2023-08-27T11:12:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30300c7e30 con 0x7f30300cd880 2023-08-27T11:12:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f301effd700 1 -- 172.21.15.45:0/3341351019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3024011630 con 0x7f30300cd880 2023-08-27T11:12:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.692+0000 7f301effd700 1 -- 172.21.15.45:0/3341351019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240119a0 con 0x7f30300cd880 2023-08-27T11:12:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.693+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30140052f0 con 0x7f30300cd880 2023-08-27T11:12:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.694+0000 7f301effd700 1 -- 172.21.15.45:0/3341351019 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f302401b440 con 0x7f30300cd880 2023-08-27T11:12:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.694+0000 7f301effd700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f301006a0e0 0x7f301006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.695+0000 7f301effd700 1 -- 172.21.15.45:0/3341351019 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3024089f50 con 0x7f30300cd880 2023-08-27T11:12:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.695+0000 7f302dd9b700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f301006a0e0 0x7f301006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.695+0000 7f302dd9b700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f301006a0e0 0x7f301006c5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3030155990 tx=0x7f3020008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.698+0000 7f301effd700 1 -- 172.21.15.45:0/3341351019 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3024055480 con 0x7f30300cd880 2023-08-27T11:12:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.899+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3014005b20 con 0x7f30300cd880 2023-08-27T11:12:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.900+0000 7f301effd700 1 -- 172.21.15.45:0/3341351019 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f302401f020 con 0x7f30300cd880 2023-08-27T11:12:20.901 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:12:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f301006a0e0 msgr2=0x7f301006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f301006a0e0 0x7f301006c5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3030155990 tx=0x7f3020008040 comp rx=0 tx=0).stop 2023-08-27T11:12:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30300cd880 msgr2=0x7f3030154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30300cd880 0x7f3030154fa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3024009130 tx=0x7f3024004750 comp rx=0 tx=0).stop 2023-08-27T11:12:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 shutdown_connections 2023-08-27T11:12:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30300cd880 0x7f3030154fa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f301006a0e0 0x7f301006c5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303015d340 0x7f3030147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 --2- 172.21.15.45:0/3341351019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30300cdd00 0x7f30300c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 >> 172.21.15.45:0/3341351019 conn(0x7f30300cc7c0 msgr2=0x7f303015e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 shutdown_connections 2023-08-27T11:12:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:20.902+0000 7f3034be0700 1 -- 172.21.15.45:0/3341351019 wait complete. 2023-08-27T11:12:20.915 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:12:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.046+0000 7fa5ae86b700 1 Processor -- start 2023-08-27T11:12:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.046+0000 7fa5ae86b700 1 -- start start 2023-08-27T11:12:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5ae86b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5a815d480 0x7fa5a80cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5ae86b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d860 0x7fa5a80cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5ae86b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a8071d70 con 0x7fa5a806ac90 2023-08-27T11:12:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5ae86b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a8072050 con 0x7fa5a815d480 2023-08-27T11:12:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5ae86b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a8072330 con 0x7fa5a815d860 2023-08-27T11:12:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5a7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5a815d480 0x7fa5a80cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5a7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5a815d480 0x7fa5a80cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51628/0 (socket says 172.21.15.45:51628) 2023-08-27T11:12:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5a7fff700 1 -- 172.21.15.45:0/2880104023 learned_addr learned my addr 172.21.15.45:0/2880104023 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5a77fe700 1 --2- 172.21.15.45:0/2880104023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d860 0x7fa5a80cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5a7fff700 1 -- 172.21.15.45:0/2880104023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d860 msgr2=0x7fa5a80cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5a7fff700 1 --2- 172.21.15.45:0/2880104023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d860 0x7fa5a80cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5a7fff700 1 -- 172.21.15.45:0/2880104023 >> v1:172.21.15.135:6789/0 conn(0x7fa5a806ac90 legacy=0x7fa5a80b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5a7fff700 1 -- 172.21.15.45:0/2880104023 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8072610 con 0x7fa5a815d480 2023-08-27T11:12:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.047+0000 7fa5a7fff700 1 --2- 172.21.15.45:0/2880104023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a815d480 0x7fa5a80cbce0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa598006fd0 tx=0x7fa598009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414a60426ec2e2a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5a6ffd700 1 -- 172.21.15.45:0/2880104023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598005e30 con 0x7fa5a815d480 2023-08-27T11:12:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5a6ffd700 1 -- 172.21.15.45:0/2880104023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa598004030 con 0x7fa5a815d480 2023-08-27T11:12:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5a6ffd700 1 -- 172.21.15.45:0/2880104023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5980043e0 con 0x7fa5a815d480 2023-08-27T11:12:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5ae86b700 1 -- 172.21.15.45:0/2880104023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a815d480 msgr2=0x7fa5a80cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5ae86b700 1 --2- 172.21.15.45:0/2880104023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a815d480 0x7fa5a80cbce0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa598006fd0 tx=0x7fa598009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5ae86b700 1 -- 172.21.15.45:0/2880104023 shutdown_connections 2023-08-27T11:12:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5ae86b700 1 --2- 172.21.15.45:0/2880104023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a815d480 0x7fa5a80cbce0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5ae86b700 1 --2- 172.21.15.45:0/2880104023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d860 0x7fa5a80cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5ae86b700 1 -- 172.21.15.45:0/2880104023 >> 172.21.15.45:0/2880104023 conn(0x7fa5a80b2440 msgr2=0x7fa5a80b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5ae86b700 1 -- 172.21.15.45:0/2880104023 shutdown_connections 2023-08-27T11:12:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.048+0000 7fa5ae86b700 1 -- 172.21.15.45:0/2880104023 wait complete. 2023-08-27T11:12:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.049+0000 7fa5ae86b700 1 Processor -- start 2023-08-27T11:12:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.050+0000 7fa5ae86b700 1 -- start start 2023-08-27T11:12:21.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.050+0000 7fa5ae86b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a806ac90 0x7fa5a8098980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.050+0000 7fa5ae86b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d480 0x7fa5a80cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.050+0000 7fa5ae86b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5a815d860 0x7fa5a8145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.050+0000 7fa5ae86b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a80c7650 con 0x7fa5a806ac90 2023-08-27T11:12:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.050+0000 7fa5ae86b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a80c8490 con 0x7fa5a815d860 2023-08-27T11:12:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.050+0000 7fa5ae86b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a8098ec0 con 0x7fa5a815d480 2023-08-27T11:12:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a806ac90 0x7fa5a8098980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a806ac90 0x7fa5a8098980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51634/0 (socket says 172.21.15.45:51634) 2023-08-27T11:12:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a7fff700 1 -- 172.21.15.45:0/4171417946 learned_addr learned my addr 172.21.15.45:0/4171417946 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5ace08700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5a815d860 0x7fa5a8145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a77fe700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d480 0x7fa5a80cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a7fff700 1 -- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d480 msgr2=0x7fa5a80cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a7fff700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d480 0x7fa5a80cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a7fff700 1 -- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5a815d860 msgr2=0x7fa5a8145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a7fff700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5a815d860 0x7fa5a8145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a7fff700 1 -- 172.21.15.45:0/4171417946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa598009500 con 0x7fa5a806ac90 2023-08-27T11:12:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5ace08700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5a815d860 0x7fa5a8145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a7fff700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a806ac90 0x7fa5a8098980 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c00 tx=0x7fa598004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a4ff9700 1 -- 172.21.15.45:0/4171417946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598003d90 con 0x7fa5a806ac90 2023-08-27T11:12:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a4ff9700 1 -- 172.21.15.45:0/4171417946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa598011420 con 0x7fa5a806ac90 2023-08-27T11:12:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5a4ff9700 1 -- 172.21.15.45:0/4171417946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5980117d0 con 0x7fa5a806ac90 2023-08-27T11:12:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a80c7880 con 0x7fa5a806ac90 2023-08-27T11:12:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.051+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a80c7e40 con 0x7fa5a806ac90 2023-08-27T11:12:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.052+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a809d4b0 con 0x7fa5a806ac90 2023-08-27T11:12:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.053+0000 7fa5a4ff9700 1 -- 172.21.15.45:0/4171417946 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa598011930 con 0x7fa5a806ac90 2023-08-27T11:12:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.053+0000 7fa5a4ff9700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa59406a0e0 0x7fa59406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.053+0000 7fa5a4ff9700 1 -- 172.21.15.45:0/4171417946 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa598089af0 con 0x7fa5a806ac90 2023-08-27T11:12:21.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.053+0000 7fa5a77fe700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa59406a0e0 0x7fa59406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.054+0000 7fa5a77fe700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa59406a0e0 0x7fa59406c5a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa5a81f6ce0 tx=0x7fa59c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:21.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.058+0000 7fa5a4ff9700 1 -- 172.21.15.45:0/4171417946 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa598054ef0 con 0x7fa5a806ac90 2023-08-27T11:12:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.257+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5a8073440 con 0x7fa5a806ac90 2023-08-27T11:12:21.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.258+0000 7fa5a4ff9700 1 -- 172.21.15.45:0/4171417946 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa598058510 con 0x7fa5a806ac90 2023-08-27T11:12:21.259 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa59406a0e0 msgr2=0x7fa59406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa59406a0e0 0x7fa59406c5a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa5a81f6ce0 tx=0x7fa59c008040 comp rx=0 tx=0).stop 2023-08-27T11:12:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a806ac90 msgr2=0x7fa5a8098980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a806ac90 0x7fa5a8098980 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c00 tx=0x7fa598004750 comp rx=0 tx=0).stop 2023-08-27T11:12:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 shutdown_connections 2023-08-27T11:12:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5a806ac90 0x7fa5a8098980 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa59406a0e0 0x7fa59406c5a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5a815d860 0x7fa5a8145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 --2- 172.21.15.45:0/4171417946 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5a815d480 0x7fa5a80cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 >> 172.21.15.45:0/4171417946 conn(0x7fa5a80b2440 msgr2=0x7fa5a806a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 shutdown_connections 2023-08-27T11:12:21.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.260+0000 7fa5ae86b700 1 -- 172.21.15.45:0/4171417946 wait complete. 2023-08-27T11:12:21.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:12:21.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.399+0000 7f546c6a0700 1 Processor -- start 2023-08-27T11:12:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.400+0000 7f546c6a0700 1 -- start start 2023-08-27T11:12:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.400+0000 7f546c6a0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546406a990 0x7f54640b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.400+0000 7f546c6a0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546406b160 con 0x7f546415d340 2023-08-27T11:12:21.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.400+0000 7f546c6a0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54640cdf90 con 0x7f5464160d00 2023-08-27T11:12:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546c6a0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5464072010 con 0x7f546406a990 2023-08-27T11:12:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546a43c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f546415d340 0x7f54640cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38418/0 (socket says 172.21.15.45:38418) 2023-08-27T11:12:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546a43c700 1 -- 172.21.15.45:0/2170567256 learned_addr learned my addr 172.21.15.45:0/2170567256 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:21.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546ac3d700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546406a990 0x7f54640b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043988679 0 0) 0x7f546406b160 con 0x7f546415d340 2023-08-27T11:12:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54500035b0 con 0x7f546415d340 2023-08-27T11:12:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1255961450 0 0) 0x7f54500035b0 con 0x7f546415d340 2023-08-27T11:12:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546406a990 msgr2=0x7f54640b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546406a990 0x7f54640b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 >> v1:172.21.15.135:6789/0 conn(0x7f5464160d00 legacy=0x7f546406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54640af590 con 0x7f546415d340 2023-08-27T11:12:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f5454001670 con 0x7f546415d340 2023-08-27T11:12:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 >> v1:172.21.15.45:6789/0 conn(0x7f546415d340 legacy=0x7f54640cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f5450003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004850 0x7f5450004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546406a990 0x7f5450005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5464072010 con 0x7f5450003a30 2023-08-27T11:12:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545000bb70 con 0x7f5450004850 2023-08-27T11:12:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545000bd30 con 0x7f546406a990 2023-08-27T11:12:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.401+0000 7f546a43c700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f5450003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546ac3d700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546406a990 0x7f5450005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f5469c3b700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004850 0x7f5450004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546a43c700 1 -- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546406a990 msgr2=0x7f5450005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:21.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546a43c700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546406a990 0x7f5450005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546a43c700 1 -- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004850 msgr2=0x7f5450004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:21.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546a43c700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004850 0x7f5450004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546a43c700 1 -- 172.21.15.45:0/2170567256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500035b0 con 0x7f5450003a30 2023-08-27T11:12:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f5469c3b700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004850 0x7f5450004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546a43c700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f5450003e50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f545400c240 tx=0x7f5450005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd56687bf2aad4c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454007c10 con 0x7f5450003a30 2023-08-27T11:12:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545400ce00 con 0x7f5450003a30 2023-08-27T11:12:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546943a700 1 -- 172.21.15.45:0/2170567256 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454005ba0 con 0x7f5450003a30 2023-08-27T11:12:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546c6a0700 1 -- 172.21.15.45:0/2170567256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 msgr2=0x7f5450003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.402+0000 7f546c6a0700 1 --2- 172.21.15.45:0/2170567256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f5450003e50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f545400c240 tx=0x7f5450005880 comp rx=0 tx=0).stop 2023-08-27T11:12:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.403+0000 7f546ac3d700 1 -- 172.21.15.45:0/2170567256 reap_dead start 2023-08-27T11:12:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.403+0000 7f546c6a0700 1 -- 172.21.15.45:0/2170567256 shutdown_connections 2023-08-27T11:12:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.403+0000 7f546c6a0700 1 -- 172.21.15.45:0/2170567256 >> 172.21.15.45:0/2170567256 conn(0x7f54640cc7c0 msgr2=0x7f54640ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.403+0000 7f546c6a0700 1 -- 172.21.15.45:0/2170567256 shutdown_connections 2023-08-27T11:12:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.403+0000 7f546c6a0700 1 -- 172.21.15.45:0/2170567256 wait complete. 2023-08-27T11:12:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.404+0000 7f546c6a0700 1 Processor -- start 2023-08-27T11:12:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.404+0000 7f546c6a0700 1 -- start start 2023-08-27T11:12:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.405+0000 7f546c6a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f54641481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.405+0000 7f546c6a0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5464156f20 0x7f54640cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.405+0000 7f546c6a0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54640b14a0 0x7f54640b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.405+0000 7f546c6a0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54640b1e90 con 0x7f5450003a30 2023-08-27T11:12:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.405+0000 7f546c6a0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5464157490 con 0x7f5464156f20 2023-08-27T11:12:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.405+0000 7f546c6a0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5464157770 con 0x7f54640b14a0 2023-08-27T11:12:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.405+0000 7f546a43c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f54641481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.405+0000 7f546a43c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f54641481d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51652/0 (socket says 172.21.15.45:51652) 2023-08-27T11:12:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.405+0000 7f546a43c700 1 -- 172.21.15.45:0/2801732862 learned_addr learned my addr 172.21.15.45:0/2801732862 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f5469c3b700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5464156f20 0x7f54640cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f546ac3d700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54640b14a0 0x7f54640b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f546a43c700 1 -- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54640b14a0 msgr2=0x7f54640b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f546a43c700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54640b14a0 0x7f54640b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f546a43c700 1 -- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5464156f20 msgr2=0x7f54640cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f546a43c700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5464156f20 0x7f54640cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f546a43c700 1 -- 172.21.15.45:0/2801732862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54640ac950 con 0x7f5450003a30 2023-08-27T11:12:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f546a43c700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f54641481d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f545000bb40 tx=0x7f5454001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f545affd700 1 -- 172.21.15.45:0/2801732862 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454001630 con 0x7f5450003a30 2023-08-27T11:12:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f545affd700 1 -- 172.21.15.45:0/2801732862 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54540070d0 con 0x7f5450003a30 2023-08-27T11:12:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54640acc30 con 0x7f5450003a30 2023-08-27T11:12:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f545affd700 1 -- 172.21.15.45:0/2801732862 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454007480 con 0x7f5450003a30 2023-08-27T11:12:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.406+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54640cc300 con 0x7f5450003a30 2023-08-27T11:12:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.407+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5464072690 con 0x7f5450003a30 2023-08-27T11:12:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.408+0000 7f545affd700 1 -- 172.21.15.45:0/2801732862 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5454019030 con 0x7f5450003a30 2023-08-27T11:12:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.408+0000 7f545affd700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544806a0e0 0x7f544806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.408+0000 7f545affd700 1 -- 172.21.15.45:0/2801732862 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5454089c60 con 0x7f5450003a30 2023-08-27T11:12:21.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.408+0000 7f5469c3b700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544806a0e0 0x7f544806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.409+0000 7f5469c3b700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544806a0e0 0x7f544806c5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5460000c00 tx=0x7f5460009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.412+0000 7f545affd700 1 -- 172.21.15.45:0/2801732862 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5454055070 con 0x7f5450003a30 2023-08-27T11:12:21.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.610+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54640734a0 con 0x7f5450003a30 2023-08-27T11:12:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.611+0000 7f545affd700 1 -- 172.21.15.45:0/2801732862 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5454058690 con 0x7f5450003a30 2023-08-27T11:12:21.613 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:12:21.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544806a0e0 msgr2=0x7f544806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544806a0e0 0x7f544806c5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5460000c00 tx=0x7f5460009040 comp rx=0 tx=0).stop 2023-08-27T11:12:21.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 msgr2=0x7f54641481d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f54641481d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f545000bb40 tx=0x7f5454001a00 comp rx=0 tx=0).stop 2023-08-27T11:12:21.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 shutdown_connections 2023-08-27T11:12:21.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a30 0x7f54641481d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544806a0e0 0x7f544806c5a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54640b14a0 0x7f54640b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 --2- 172.21.15.45:0/2801732862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5464156f20 0x7f54640cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 >> 172.21.15.45:0/2801732862 conn(0x7f54640cc7c0 msgr2=0x7f54640cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 shutdown_connections 2023-08-27T11:12:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.613+0000 7f546c6a0700 1 -- 172.21.15.45:0/2801732862 wait complete. 2023-08-27T11:12:21.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:12:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.752+0000 7f710cf61700 1 Processor -- start 2023-08-27T11:12:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.752+0000 7f710cf61700 1 -- start start 2023-08-27T11:12:21.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.753+0000 7f710cf61700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f710806ed90 0x7f71080cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.753+0000 7f710cf61700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080cfae0 con 0x7f710815a0f0 2023-08-27T11:12:21.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.753+0000 7f710cf61700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710815e900 con 0x7f71080cd300 2023-08-27T11:12:21.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.753+0000 7f710cf61700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080cae60 con 0x7f710806ed90 2023-08-27T11:12:21.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.753+0000 7f710659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f710815a0f0 0x7f710815daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38436/0 (socket says 172.21.15.45:38436) 2023-08-27T11:12:21.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.753+0000 7f710659c700 1 -- 172.21.15.45:0/1615643826 learned_addr learned my addr 172.21.15.45:0/1615643826 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:21.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f7106d9d700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f710806ed90 0x7f71080cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779185572 0 0) 0x7f71080cfae0 con 0x7f710815a0f0 2023-08-27T11:12:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70f00035b0 con 0x7f710815a0f0 2023-08-27T11:12:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 566290851 0 0) 0x7f70f00035b0 con 0x7f710815a0f0 2023-08-27T11:12:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f710806ed90 msgr2=0x7f71080cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f710806ed90 0x7f71080cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 >> v1:172.21.15.135:6789/0 conn(0x7f71080cd300 legacy=0x7f71080cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080cb140 con 0x7f710815a0f0 2023-08-27T11:12:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f70fc001670 con 0x7f710815a0f0 2023-08-27T11:12:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 >> v1:172.21.15.45:6789/0 conn(0x7f710815a0f0 legacy=0x7f710815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f710806ed90 0x7f70f0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70f0003f70 0x7f70f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f0004d90 0x7f70f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71080cae60 con 0x7f70f0003f70 2023-08-27T11:12:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f000ba70 con 0x7f70f0004d90 2023-08-27T11:12:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f000bc30 con 0x7f710806ed90 2023-08-27T11:12:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f7105d9b700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70f0003f70 0x7f70f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f710659c700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f710806ed90 0x7f70f0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f7106d9d700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f0004d90 0x7f70f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.754+0000 7f7105d9b700 1 -- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f710806ed90 msgr2=0x7f70f0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f7105d9b700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f710806ed90 0x7f70f0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f7105d9b700 1 -- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f0004d90 msgr2=0x7f70f0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f7105d9b700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f0004d90 0x7f70f0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f7105d9b700 1 -- 172.21.15.45:0/1615643826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00035b0 con 0x7f70f0003f70 2023-08-27T11:12:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f7105d9b700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70f0003f70 0x7f70f0004850 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f70f80018b0 tx=0x7f70f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378912c7323c406f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8001370 con 0x7f70f0003f70 2023-08-27T11:12:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f8012640 con 0x7f70f0003f70 2023-08-27T11:12:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f710559a700 1 -- 172.21.15.45:0/1615643826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80129f0 con 0x7f70f0003f70 2023-08-27T11:12:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f710cf61700 1 -- 172.21.15.45:0/1615643826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70f0003f70 msgr2=0x7f70f0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f710cf61700 1 --2- 172.21.15.45:0/1615643826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70f0003f70 0x7f70f0004850 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f70f80018b0 tx=0x7f70f0005880 comp rx=0 tx=0).stop 2023-08-27T11:12:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f7106d9d700 1 -- 172.21.15.45:0/1615643826 reap_dead start 2023-08-27T11:12:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f710cf61700 1 -- 172.21.15.45:0/1615643826 shutdown_connections 2023-08-27T11:12:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.755+0000 7f710cf61700 1 -- 172.21.15.45:0/1615643826 >> 172.21.15.45:0/1615643826 conn(0x7f71080cc7e0 msgr2=0x7f71080ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.756+0000 7f710cf61700 1 -- 172.21.15.45:0/1615643826 shutdown_connections 2023-08-27T11:12:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.756+0000 7f710cf61700 1 -- 172.21.15.45:0/1615643826 wait complete. 2023-08-27T11:12:21.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.757+0000 7f710cf61700 1 Processor -- start 2023-08-27T11:12:21.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.757+0000 7f710cf61700 1 -- start start 2023-08-27T11:12:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f710cf61700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70f0003f70 0x7f71080c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f710cf61700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71080b5490 0x7f7108150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f710cf61700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71080c8160 0x7f71080c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f710cf61700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71080c8b50 con 0x7f71080b5490 2023-08-27T11:12:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f710cf61700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71080b59e0 con 0x7f71080c8160 2023-08-27T11:12:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f710cf61700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080c4780 con 0x7f70f0003f70 2023-08-27T11:12:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7105d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71080b5490 0x7f7108150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7105d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71080b5490 0x7f7108150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51674/0 (socket says 172.21.15.45:51674) 2023-08-27T11:12:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7105d9b700 1 -- 172.21.15.45:0/399293582 learned_addr learned my addr 172.21.15.45:0/399293582 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f710659c700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70f0003f70 0x7f71080c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7106d9d700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71080c8160 0x7f71080c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7105d9b700 1 -- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70f0003f70 msgr2=0x7f71080c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7105d9b700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70f0003f70 0x7f71080c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7105d9b700 1 -- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71080c8160 msgr2=0x7f71080c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7105d9b700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71080c8160 0x7f71080c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7105d9b700 1 -- 172.21.15.45:0/399293582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080c39c0 con 0x7f71080b5490 2023-08-27T11:12:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f7105d9b700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71080b5490 0x7f7108150b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f70f000ba40 tx=0x7f70f8001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f70f6ffd700 1 -- 172.21.15.45:0/399293582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8001370 con 0x7f71080b5490 2023-08-27T11:12:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f70f6ffd700 1 -- 172.21.15.45:0/399293582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f8021070 con 0x7f71080b5490 2023-08-27T11:12:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f70f6ffd700 1 -- 172.21.15.45:0/399293582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8006aa0 con 0x7f71080b5490 2023-08-27T11:12:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.758+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71080c3ca0 con 0x7f71080b5490 2023-08-27T11:12:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.759+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71080c4260 con 0x7f71080b5490 2023-08-27T11:12:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.759+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71080c43f0 con 0x7f71080b5490 2023-08-27T11:12:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.761+0000 7f70f6ffd700 1 -- 172.21.15.45:0/399293582 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f70f8023030 con 0x7f71080b5490 2023-08-27T11:12:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.761+0000 7f70f6ffd700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f70e806a0e0 0x7f70e806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.761+0000 7f70f6ffd700 1 -- 172.21.15.45:0/399293582 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f70f8089860 con 0x7f71080b5490 2023-08-27T11:12:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.761+0000 7f710659c700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f70e806a0e0 0x7f70e806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.762+0000 7f710659c700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f70e806a0e0 0x7f70e806c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f70fc001cb0 tx=0x7f70fc000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.764+0000 7f70f6ffd700 1 -- 172.21.15.45:0/399293582 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f70f8058860 con 0x7f71080b5490 2023-08-27T11:12:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.965+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f71080726d0 con 0x7f71080b5490 2023-08-27T11:12:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.966+0000 7f70f6ffd700 1 -- 172.21.15.45:0/399293582 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f70f80583f0 con 0x7f71080b5490 2023-08-27T11:12:21.967 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f70e806a0e0 msgr2=0x7f70e806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f70e806a0e0 0x7f70e806c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f70fc001cb0 tx=0x7f70fc000ed0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71080b5490 msgr2=0x7f7108150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71080b5490 0x7f7108150b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f70f000ba40 tx=0x7f70f8001110 comp rx=0 tx=0).stop 2023-08-27T11:12:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 shutdown_connections 2023-08-27T11:12:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f70e806a0e0 0x7f70e806c5a0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f70fc001cb0 tx=0x7f70fc000ed0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71080c8160 0x7f71080c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70f0003f70 0x7f71080c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 --2- 172.21.15.45:0/399293582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71080b5490 0x7f7108150b30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:21.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 >> 172.21.15.45:0/399293582 conn(0x7f71080cc7e0 msgr2=0x7f71080af720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 shutdown_connections 2023-08-27T11:12:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:21.968+0000 7f710cf61700 1 -- 172.21.15.45:0/399293582 wait complete. 2023-08-27T11:12:21.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:12:22.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.105+0000 7fb1f5da4700 1 Processor -- start 2023-08-27T11:12:22.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.105+0000 7fb1f5da4700 1 -- start start 2023-08-27T11:12:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.106+0000 7fb1f5da4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb1f00b17c0 0x7fb1f00b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.106+0000 7fb1f5da4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f015b460 con 0x7fb1f0157be0 2023-08-27T11:12:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.106+0000 7fb1f5da4700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f00b2280 con 0x7fb1f006ced0 2023-08-27T11:12:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.106+0000 7fb1f5da4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f0157fc0 con 0x7fb1f00b17c0 2023-08-27T11:12:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.106+0000 7fb1eeffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb1f00b17c0 0x7fb1f00b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.106+0000 7fb1eeffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb1f00b17c0 0x7fb1f00b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51682/0 (socket says 172.21.15.45:51682) 2023-08-27T11:12:22.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.106+0000 7fb1eeffd700 1 -- 172.21.15.45:0/3989732560 learned_addr learned my addr 172.21.15.45:0/3989732560 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1eeffd700 1 -- 172.21.15.45:0/3989732560 >> v1:172.21.15.135:6789/0 conn(0x7fb1f006ced0 legacy=0x7fb1f00cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1eeffd700 1 -- 172.21.15.45:0/3989732560 >> v1:172.21.15.45:6789/0 conn(0x7fb1f0157be0 legacy=0x7fb1f00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1eeffd700 1 -- 172.21.15.45:0/3989732560 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00b2560 con 0x7fb1f00b17c0 2023-08-27T11:12:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1ee7fc700 1 -- 172.21.15.45:0/3989732560 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541082333 0 0) 0x7fb1f015b460 con 0x7fb1f0157be0 2023-08-27T11:12:22.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1eeffd700 1 --2- 172.21.15.45:0/3989732560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f00b17c0 0x7fb1f00b1c70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80098f0 tx=0x7fb1d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1154e70d74c705d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1ee7fc700 1 -- 172.21.15.45:0/3989732560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8004230 con 0x7fb1f00b17c0 2023-08-27T11:12:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1ee7fc700 1 -- 172.21.15.45:0/3989732560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8004390 con 0x7fb1f00b17c0 2023-08-27T11:12:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1ee7fc700 1 -- 172.21.15.45:0/3989732560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80036a0 con 0x7fb1f00b17c0 2023-08-27T11:12:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1f5da4700 1 -- 172.21.15.45:0/3989732560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f00b17c0 msgr2=0x7fb1f00b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.107+0000 7fb1f5da4700 1 --2- 172.21.15.45:0/3989732560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f00b17c0 0x7fb1f00b1c70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80098f0 tx=0x7fb1d8009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.108+0000 7fb1f5da4700 1 -- 172.21.15.45:0/3989732560 shutdown_connections 2023-08-27T11:12:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.108+0000 7fb1f5da4700 1 --2- 172.21.15.45:0/3989732560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f00b17c0 0x7fb1f00b1c70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.108+0000 7fb1f5da4700 1 -- 172.21.15.45:0/3989732560 >> 172.21.15.45:0/3989732560 conn(0x7fb1f00cc7c0 msgr2=0x7fb1f00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.108+0000 7fb1f5da4700 1 -- 172.21.15.45:0/3989732560 shutdown_connections 2023-08-27T11:12:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.108+0000 7fb1f5da4700 1 -- 172.21.15.45:0/3989732560 wait complete. 2023-08-27T11:12:22.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.109+0000 7fb1f5da4700 1 Processor -- start 2023-08-27T11:12:22.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.109+0000 7fb1f5da4700 1 -- start start 2023-08-27T11:12:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1f5da4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f006ced0 0x7fb1f014e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1f5da4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1f00b17c0 0x7fb1f0159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1f5da4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0157be0 0x7fb1f006a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1f5da4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00c6f90 con 0x7fb1f006ced0 2023-08-27T11:12:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1f5da4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f014e770 con 0x7fb1f0157be0 2023-08-27T11:12:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1f5da4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f0159690 con 0x7fb1f00b17c0 2023-08-27T11:12:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f006ced0 0x7fb1f014e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1ef7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f006ced0 0x7fb1f014e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51686/0 (socket says 172.21.15.45:51686) 2023-08-27T11:12:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1ef7fe700 1 -- 172.21.15.45:0/389934678 learned_addr learned my addr 172.21.15.45:0/389934678 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1effff700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0157be0 0x7fb1f006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1eeffd700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1f00b17c0 0x7fb1f0159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1ef7fe700 1 -- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1f00b17c0 msgr2=0x7fb1f0159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1ef7fe700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1f00b17c0 0x7fb1f0159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1ef7fe700 1 -- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0157be0 msgr2=0x7fb1f006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1ef7fe700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0157be0 0x7fb1f006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1ef7fe700 1 -- 172.21.15.45:0/389934678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d8009550 con 0x7fb1f006ced0 2023-08-27T11:12:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1effff700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0157be0 0x7fb1f006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:22.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1eeffd700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1f00b17c0 0x7fb1f0159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.110+0000 7fb1ef7fe700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f006ced0 0x7fb1f014e230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c750 tx=0x7fb1e000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.111+0000 7fb1e7fff700 1 -- 172.21.15.45:0/389934678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00054e0 con 0x7fb1f006ced0 2023-08-27T11:12:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.111+0000 7fb1e7fff700 1 -- 172.21.15.45:0/389934678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0003680 con 0x7fb1f006ced0 2023-08-27T11:12:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.111+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f0154b00 con 0x7fb1f006ced0 2023-08-27T11:12:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.111+0000 7fb1e7fff700 1 -- 172.21.15.45:0/389934678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0003a30 con 0x7fb1f006ced0 2023-08-27T11:12:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.111+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f01550c0 con 0x7fb1f006ced0 2023-08-27T11:12:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.112+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f0072690 con 0x7fb1f006ced0 2023-08-27T11:12:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.112+0000 7fb1e7fff700 1 -- 172.21.15.45:0/389934678 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb1e0003b90 con 0x7fb1f006ced0 2023-08-27T11:12:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.112+0000 7fb1e7fff700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1d006a0e0 0x7fb1d006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.113+0000 7fb1e7fff700 1 -- 172.21.15.45:0/389934678 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb1e0087df0 con 0x7fb1f006ced0 2023-08-27T11:12:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.113+0000 7fb1eeffd700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1d006a0e0 0x7fb1d006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.113+0000 7fb1eeffd700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1d006a0e0 0x7fb1d006c5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80098f0 tx=0x7fb1d8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.117+0000 7fb1e7fff700 1 -- 172.21.15.45:0/389934678 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb1e00531f0 con 0x7fb1f006ced0 2023-08-27T11:12:22.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.331+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb1f0159ef0 con 0x7fb1f006ced0 2023-08-27T11:12:22.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.332+0000 7fb1e7fff700 1 -- 172.21.15.45:0/389934678 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb1e0017080 con 0x7fb1f006ced0 2023-08-27T11:12:22.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1d006a0e0 msgr2=0x7fb1d006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1d006a0e0 0x7fb1d006c5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80098f0 tx=0x7fb1d8008030 comp rx=0 tx=0).stop 2023-08-27T11:12:22.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f006ced0 msgr2=0x7fb1f014e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f006ced0 0x7fb1f014e230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c750 tx=0x7fb1e000cb10 comp rx=0 tx=0).stop 2023-08-27T11:12:22.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 shutdown_connections 2023-08-27T11:12:22.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f006ced0 0x7fb1f014e230 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1d006a0e0 0x7fb1d006c5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0157be0 0x7fb1f006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 --2- 172.21.15.45:0/389934678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1f00b17c0 0x7fb1f0159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.333+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 >> 172.21.15.45:0/389934678 conn(0x7fb1f00cc7c0 msgr2=0x7fb1f00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.334+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 shutdown_connections 2023-08-27T11:12:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.334+0000 7fb1f5da4700 1 -- 172.21.15.45:0/389934678 wait complete. 2023-08-27T11:12:22.338 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2023-08-27T11:12:22.346 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:12:22.347 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:12:22.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.529+0000 7f357f5ac700 1 Processor -- start 2023-08-27T11:12:22.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.529+0000 7f357f5ac700 1 -- start start 2023-08-27T11:12:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f357f5ac700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f35780cbe70 0x7f35780cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f357f5ac700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f357806d690 con 0x7f35780ceba0 2023-08-27T11:12:22.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f357f5ac700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35780cc440 con 0x7f357815a240 2023-08-27T11:12:22.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f357f5ac700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357806d040 con 0x7f35780cbe70 2023-08-27T11:12:22.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f357cb47700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f35780ceba0 0x7f357806db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38472/0 (socket says 172.21.15.45:38472) 2023-08-27T11:12:22.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f357cb47700 1 -- 172.21.15.45:0/590905885 learned_addr learned my addr 172.21.15.45:0/590905885 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:22.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f357db49700 1 --2- 172.21.15.45:0/590905885 >> v2:172.21.15.135:3300/0 conn(0x7f35780cbe70 0x7f35780cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763438506 0 0) 0x7f357806d690 con 0x7f35780ceba0 2023-08-27T11:12:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35640035b0 con 0x7f35780ceba0 2023-08-27T11:12:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 96001840 0 0) 0x7f35640035b0 con 0x7f35780ceba0 2023-08-27T11:12:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 >> v2:172.21.15.135:3300/0 conn(0x7f35780cbe70 msgr2=0x7f35780cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 --2- 172.21.15.45:0/590905885 >> v2:172.21.15.135:3300/0 conn(0x7f35780cbe70 0x7f35780cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 >> v1:172.21.15.135:6789/0 conn(0x7f357815a240 legacy=0x7f35780ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357806d320 con 0x7f35780ceba0 2023-08-27T11:12:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f3568001670 con 0x7f35780ceba0 2023-08-27T11:12:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 >> v1:172.21.15.45:6789/0 conn(0x7f35780ceba0 legacy=0x7f357806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3564003a00 0x7f3564003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3564004850 0x7f3564004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f356400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357806d040 con 0x7f3564005210 2023-08-27T11:12:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356400c0b0 con 0x7f3564003a00 2023-08-27T11:12:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.530+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356400c270 con 0x7f3564004850 2023-08-27T11:12:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357db49700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f356400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357d348700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3564003a00 0x7f3564003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357cb47700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3564004850 0x7f3564004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357db49700 1 -- 172.21.15.45:0/590905885 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3564004850 msgr2=0x7f3564004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357db49700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3564004850 0x7f3564004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357db49700 1 -- 172.21.15.45:0/590905885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3564003a00 msgr2=0x7f3564003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357db49700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3564003a00 0x7f3564003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357db49700 1 -- 172.21.15.45:0/590905885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640035b0 con 0x7f3564005210 2023-08-27T11:12:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357db49700 1 -- 172.21.15.45:0/590905885 reap_dead start 2023-08-27T11:12:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f357db49700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f356400ba10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3570009fc0 tx=0x7f3564005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a17efe9cac3afc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570003a20 con 0x7f3564005210 2023-08-27T11:12:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357000f040 con 0x7f3564005210 2023-08-27T11:12:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.531+0000 7f356ffff700 1 -- 172.21.15.45:0/590905885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570003de0 con 0x7f3564005210 2023-08-27T11:12:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.532+0000 7f357f5ac700 1 -- 172.21.15.45:0/590905885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 msgr2=0x7f356400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.532+0000 7f357f5ac700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f356400ba10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3570009fc0 tx=0x7f3564005880 comp rx=0 tx=0).stop 2023-08-27T11:12:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.532+0000 7f357f5ac700 1 -- 172.21.15.45:0/590905885 shutdown_connections 2023-08-27T11:12:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.532+0000 7f357f5ac700 1 --2- 172.21.15.45:0/590905885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f356400ba10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.532+0000 7f357f5ac700 1 -- 172.21.15.45:0/590905885 >> 172.21.15.45:0/590905885 conn(0x7f35780cfe50 msgr2=0x7f35780d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.532+0000 7f357f5ac700 1 -- 172.21.15.45:0/590905885 shutdown_connections 2023-08-27T11:12:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.532+0000 7f357f5ac700 1 -- 172.21.15.45:0/590905885 wait complete. 2023-08-27T11:12:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.533+0000 7f357f5ac700 1 Processor -- start 2023-08-27T11:12:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.533+0000 7f357f5ac700 1 -- start start 2023-08-27T11:12:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357f5ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f35780aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357f5ac700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3578151620 0x7f35780ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357f5ac700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35780ad6f0 0x7f3578155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357f5ac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35780adc70 con 0x7f3564005210 2023-08-27T11:12:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357f5ac700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35780adf50 con 0x7f3578151620 2023-08-27T11:12:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357f5ac700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35780ae230 con 0x7f35780ad6f0 2023-08-27T11:12:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357d348700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f35780aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357d348700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f35780aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51706/0 (socket says 172.21.15.45:51706) 2023-08-27T11:12:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357d348700 1 -- 172.21.15.45:0/2289889805 learned_addr learned my addr 172.21.15.45:0/2289889805 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357db49700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35780ad6f0 0x7f3578155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357cb47700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3578151620 0x7f35780ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357d348700 1 -- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35780ad6f0 msgr2=0x7f3578155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357d348700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35780ad6f0 0x7f3578155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357d348700 1 -- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3578151620 msgr2=0x7f35780ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357d348700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3578151620 0x7f35780ad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357d348700 1 -- 172.21.15.45:0/2289889805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35780ae510 con 0x7f3564005210 2023-08-27T11:12:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357db49700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35780ad6f0 0x7f3578155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f357d348700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f35780aed10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f357400f820 tx=0x7f357400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.534+0000 7f356dffb700 1 -- 172.21.15.45:0/2289889805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35740052e0 con 0x7f3564005210 2023-08-27T11:12:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.535+0000 7f356dffb700 1 -- 172.21.15.45:0/2289889805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3574005460 con 0x7f3564005210 2023-08-27T11:12:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.535+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35780ae7f0 con 0x7f3564005210 2023-08-27T11:12:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.535+0000 7f356dffb700 1 -- 172.21.15.45:0/2289889805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35740038c0 con 0x7f3564005210 2023-08-27T11:12:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.535+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35780c4550 con 0x7f3564005210 2023-08-27T11:12:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.536+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3578072810 con 0x7f3564005210 2023-08-27T11:12:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.536+0000 7f356dffb700 1 -- 172.21.15.45:0/2289889805 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3574003a20 con 0x7f3564005210 2023-08-27T11:12:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.536+0000 7f356dffb700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f355c06a0e0 0x7f355c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.537+0000 7f356dffb700 1 -- 172.21.15.45:0/2289889805 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f35740886a0 con 0x7f3564005210 2023-08-27T11:12:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.537+0000 7f357cb47700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f355c06a0e0 0x7f355c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.537+0000 7f357cb47700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f355c06a0e0 0x7f355c06c5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35780cefb0 tx=0x7f3568009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:22.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.541+0000 7f356dffb700 1 -- 172.21.15.45:0/2289889805 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3574053a00 con 0x7f3564005210 2023-08-27T11:12:22.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.740+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35780734a0 con 0x7f3564005210 2023-08-27T11:12:22.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.741+0000 7f356dffb700 1 -- 172.21.15.45:0/2289889805 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f3574057020 con 0x7f3564005210 2023-08-27T11:12:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:22.742 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:22.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f355c06a0e0 msgr2=0x7f355c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f355c06a0e0 0x7f355c06c5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35780cefb0 tx=0x7f3568009040 comp rx=0 tx=0).stop 2023-08-27T11:12:22.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 msgr2=0x7f35780aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f35780aed10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f357400f820 tx=0x7f357400fbe0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 shutdown_connections 2023-08-27T11:12:22.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3564005210 0x7f35780aed10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f355c06a0e0 0x7f355c06c5a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35780ad6f0 0x7f3578155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 --2- 172.21.15.45:0/2289889805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3578151620 0x7f35780ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 >> 172.21.15.45:0/2289889805 conn(0x7f35780cfe50 msgr2=0x7f357806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:22.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 shutdown_connections 2023-08-27T11:12:22.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.743+0000 7f357f5ac700 1 -- 172.21.15.45:0/2289889805 wait complete. 2023-08-27T11:12:22.756 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:12:22.757 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:12:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.885+0000 7f148eab6700 1 Processor -- start 2023-08-27T11:12:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.885+0000 7f148eab6700 1 -- start start 2023-08-27T11:12:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.886+0000 7f148eab6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f148806b690 0x7f1488161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.886+0000 7f148eab6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1488140780 con 0x7f1488140a50 2023-08-27T11:12:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.886+0000 7f148eab6700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14881415f0 con 0x7f14880c4360 2023-08-27T11:12:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.886+0000 7f148eab6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148806a5f0 con 0x7f148806b690 2023-08-27T11:12:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.886+0000 7f1487fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f148806b690 0x7f1488161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.886+0000 7f148d053700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1488140a50 0x7f1488140ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38494/0 (socket says 172.21.15.45:38494) 2023-08-27T11:12:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.886+0000 7f1487fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f148806b690 0x7f1488161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51716/0 (socket says 172.21.15.45:51716) 2023-08-27T11:12:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.886+0000 7f148d053700 1 -- 172.21.15.45:0/2462976430 learned_addr learned my addr 172.21.15.45:0/2462976430 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.887+0000 7f1487fff700 1 -- 172.21.15.45:0/2462976430 >> v1:172.21.15.135:6789/0 conn(0x7f14880c4360 legacy=0x7f14880c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.887+0000 7f1487fff700 1 -- 172.21.15.45:0/2462976430 >> v1:172.21.15.45:6789/0 conn(0x7f1488140a50 legacy=0x7f1488140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.887+0000 7f1487fff700 1 -- 172.21.15.45:0/2462976430 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148806a8d0 con 0x7f148806b690 2023-08-27T11:12:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.887+0000 7f14877fe700 1 -- 172.21.15.45:0/2462976430 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787198282 0 0) 0x7f1488140780 con 0x7f1488140a50 2023-08-27T11:12:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.887+0000 7f1487fff700 1 --2- 172.21.15.45:0/2462976430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148806b690 0x7f1488161690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1474009840 tx=0x7f1474009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4924bc88f119ec3a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.887+0000 7f14877fe700 1 -- 172.21.15.45:0/2462976430 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474011830 con 0x7f148806b690 2023-08-27T11:12:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.887+0000 7f14877fe700 1 -- 172.21.15.45:0/2462976430 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1474011990 con 0x7f148806b690 2023-08-27T11:12:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.887+0000 7f14877fe700 1 -- 172.21.15.45:0/2462976430 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474011d20 con 0x7f148806b690 2023-08-27T11:12:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.888+0000 7f148eab6700 1 -- 172.21.15.45:0/2462976430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148806b690 msgr2=0x7f1488161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.888+0000 7f148eab6700 1 --2- 172.21.15.45:0/2462976430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148806b690 0x7f1488161690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1474009840 tx=0x7f1474009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.888+0000 7f148eab6700 1 -- 172.21.15.45:0/2462976430 shutdown_connections 2023-08-27T11:12:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.888+0000 7f148eab6700 1 --2- 172.21.15.45:0/2462976430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f148806b690 0x7f1488161690 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.888+0000 7f148eab6700 1 -- 172.21.15.45:0/2462976430 >> 172.21.15.45:0/2462976430 conn(0x7f14880cbf20 msgr2=0x7f14880cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:22.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.888+0000 7f148eab6700 1 -- 172.21.15.45:0/2462976430 shutdown_connections 2023-08-27T11:12:22.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.888+0000 7f148eab6700 1 -- 172.21.15.45:0/2462976430 wait complete. 2023-08-27T11:12:22.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.889+0000 7f148eab6700 1 Processor -- start 2023-08-27T11:12:22.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.889+0000 7f148eab6700 1 -- start start 2023-08-27T11:12:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f148eab6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f148806b690 0x7f14881453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f148eab6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14880c4360 0x7f148815e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f148eab6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1488140a50 0x7f14881f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f148eab6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880ac940 con 0x7f14880c4360 2023-08-27T11:12:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f148eab6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148815ebd0 con 0x7f148806b690 2023-08-27T11:12:22.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f148eab6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148815eeb0 con 0x7f1488140a50 2023-08-27T11:12:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f1487fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14880c4360 0x7f148815e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f1487fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14880c4360 0x7f148815e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51722/0 (socket says 172.21.15.45:51722) 2023-08-27T11:12:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f1487fff700 1 -- 172.21.15.45:0/1464813064 learned_addr learned my addr 172.21.15.45:0/1464813064 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f148c852700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f148806b690 0x7f14881453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f148d053700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1488140a50 0x7f14881f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f1487fff700 1 -- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1488140a50 msgr2=0x7f14881f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f1487fff700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1488140a50 0x7f14881f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f1487fff700 1 -- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f148806b690 msgr2=0x7f14881453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f1487fff700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f148806b690 0x7f14881453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:22.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f1487fff700 1 -- 172.21.15.45:0/1464813064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14740094a0 con 0x7f14880c4360 2023-08-27T11:12:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.890+0000 7f1487fff700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14880c4360 0x7f148815e4d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1474014040 tx=0x7f1474004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.891+0000 7f14857fa700 1 -- 172.21.15.45:0/1464813064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147401a7e0 con 0x7f14880c4360 2023-08-27T11:12:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.891+0000 7f14857fa700 1 -- 172.21.15.45:0/1464813064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147401a940 con 0x7f14880c4360 2023-08-27T11:12:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.891+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148815f190 con 0x7f14880c4360 2023-08-27T11:12:22.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.891+0000 7f14857fa700 1 -- 172.21.15.45:0/1464813064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147401acb0 con 0x7f14880c4360 2023-08-27T11:12:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.891+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14881f2700 con 0x7f14880c4360 2023-08-27T11:12:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.892+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148806dbb0 con 0x7f14880c4360 2023-08-27T11:12:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.892+0000 7f14857fa700 1 -- 172.21.15.45:0/1464813064 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1474022070 con 0x7f14880c4360 2023-08-27T11:12:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.893+0000 7f14857fa700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f146c06a0e0 0x7f146c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.893+0000 7f14857fa700 1 -- 172.21.15.45:0/1464813064 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1474089bb0 con 0x7f14880c4360 2023-08-27T11:12:22.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.893+0000 7f148c852700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f146c06a0e0 0x7f146c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:22.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.893+0000 7f148c852700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f146c06a0e0 0x7f146c06c5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f147c007fd0 tx=0x7f147c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:22.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:22.897+0000 7f14857fa700 1 -- 172.21.15.45:0/1464813064 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1474055160 con 0x7f14880c4360 2023-08-27T11:12:23.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.098+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1488160c50 con 0x7f14880c4360 2023-08-27T11:12:23.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.099+0000 7f14857fa700 1 -- 172.21.15.45:0/1464813064 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14740290a0 con 0x7f14880c4360 2023-08-27T11:12:23.101 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:12:23.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f146c06a0e0 msgr2=0x7f146c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f146c06a0e0 0x7f146c06c5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f147c007fd0 tx=0x7f147c009040 comp rx=0 tx=0).stop 2023-08-27T11:12:23.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14880c4360 msgr2=0x7f148815e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14880c4360 0x7f148815e4d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1474014040 tx=0x7f1474004750 comp rx=0 tx=0).stop 2023-08-27T11:12:23.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 shutdown_connections 2023-08-27T11:12:23.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f146c06a0e0 0x7f146c06c5a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1488140a50 0x7f14881f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f148806b690 0x7f14881453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 --2- 172.21.15.45:0/1464813064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14880c4360 0x7f148815e4d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 >> 172.21.15.45:0/1464813064 conn(0x7f14880cbf20 msgr2=0x7f14880ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 shutdown_connections 2023-08-27T11:12:23.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.101+0000 7f148eab6700 1 -- 172.21.15.45:0/1464813064 wait complete. 2023-08-27T11:12:23.114 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:12:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.240+0000 7f70a7ca0700 1 Processor -- start 2023-08-27T11:12:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.240+0000 7f70a7ca0700 1 -- start start 2023-08-27T11:12:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.241+0000 7f70a7ca0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f70a00b17c0 0x7f70a00b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.241+0000 7f70a7ca0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a006ced0 0x7f70a006d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.241+0000 7f70a7ca0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a015b460 con 0x7f70a0157be0 2023-08-27T11:12:23.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.241+0000 7f70a7ca0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a0157fc0 con 0x7f70a00b17c0 2023-08-27T11:12:23.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.241+0000 7f70a7ca0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0071dc0 con 0x7f70a006ced0 2023-08-27T11:12:23.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.241+0000 7f70a623d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a006ced0 0x7f70a006d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a523b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f70a00b17c0 0x7f70a00b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a623d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a006ced0 0x7f70a006d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:49642/0 (socket says 172.21.15.45:49642) 2023-08-27T11:12:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a523b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f70a00b17c0 0x7f70a00b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:40586/0 (socket says 172.21.15.45:40586) 2023-08-27T11:12:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a623d700 1 -- 172.21.15.45:0/1233606609 learned_addr learned my addr 172.21.15.45:0/1233606609 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a623d700 1 -- 172.21.15.45:0/1233606609 >> v2:172.21.15.135:3300/0 conn(0x7f70a00b17c0 msgr2=0x7f70a00b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a623d700 1 --2- 172.21.15.45:0/1233606609 >> v2:172.21.15.135:3300/0 conn(0x7f70a00b17c0 0x7f70a00b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a623d700 1 -- 172.21.15.45:0/1233606609 >> v1:172.21.15.135:6789/0 conn(0x7f70a0157be0 legacy=0x7f70a00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a623d700 1 -- 172.21.15.45:0/1233606609 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a00b2280 con 0x7f70a006ced0 2023-08-27T11:12:23.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a523b700 1 --2- 172.21.15.45:0/1233606609 >> v2:172.21.15.135:3300/0 conn(0x7f70a00b17c0 0x7f70a00b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:23.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a623d700 1 --2- 172.21.15.45:0/1233606609 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a006ced0 0x7f70a006d340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7098009e50 tx=0x7f709800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d35d2bc3c6575bf4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:23.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a4a3a700 1 -- 172.21.15.45:0/1233606609 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709800e070 con 0x7f70a006ced0 2023-08-27T11:12:23.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a4a3a700 1 -- 172.21.15.45:0/1233606609 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709800b910 con 0x7f70a006ced0 2023-08-27T11:12:23.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.242+0000 7f70a4a3a700 1 -- 172.21.15.45:0/1233606609 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709800bcc0 con 0x7f70a006ced0 2023-08-27T11:12:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.243+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1233606609 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a006ced0 msgr2=0x7f70a006d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.243+0000 7f70a7ca0700 1 --2- 172.21.15.45:0/1233606609 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a006ced0 0x7f70a006d340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7098009e50 tx=0x7f709800b330 comp rx=0 tx=0).stop 2023-08-27T11:12:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.243+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1233606609 shutdown_connections 2023-08-27T11:12:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.243+0000 7f70a7ca0700 1 --2- 172.21.15.45:0/1233606609 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a006ced0 0x7f70a006d340 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.243+0000 7f70a7ca0700 1 --2- 172.21.15.45:0/1233606609 >> v2:172.21.15.135:3300/0 conn(0x7f70a00b17c0 0x7f70a00b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.243+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1233606609 >> 172.21.15.45:0/1233606609 conn(0x7f70a00cc7c0 msgr2=0x7f70a00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.243+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1233606609 shutdown_connections 2023-08-27T11:12:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.243+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1233606609 wait complete. 2023-08-27T11:12:23.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.244+0000 7f70a7ca0700 1 Processor -- start 2023-08-27T11:12:23.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.244+0000 7f70a7ca0700 1 -- start start 2023-08-27T11:12:23.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a7ca0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70a006ced0 0x7f70a0166910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a7ca0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a00b17c0 0x7f70a01451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a7ca0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a0157be0 0x7f70a014a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a7ca0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a014c070 con 0x7f70a0157be0 2023-08-27T11:12:23.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a7ca0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0166e50 con 0x7f70a006ced0 2023-08-27T11:12:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a7ca0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01644a0 con 0x7f70a00b17c0 2023-08-27T11:12:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a623d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a0157be0 0x7f70a014a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a523b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a00b17c0 0x7f70a01451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a5a3c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70a006ced0 0x7f70a0166910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a623d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a0157be0 0x7f70a014a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51738/0 (socket says 172.21.15.45:51738) 2023-08-27T11:12:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a623d700 1 -- 172.21.15.45:0/1978449944 learned_addr learned my addr 172.21.15.45:0/1978449944 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a623d700 1 -- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a00b17c0 msgr2=0x7f70a01451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.245+0000 7f70a623d700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a00b17c0 0x7f70a01451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70a623d700 1 -- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70a006ced0 msgr2=0x7f70a0166910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70a623d700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70a006ced0 0x7f70a0166910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70a623d700 1 -- 172.21.15.45:0/1978449944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0094a0 con 0x7f70a0157be0 2023-08-27T11:12:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70a523b700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a00b17c0 0x7f70a01451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70a623d700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a0157be0 0x7f70a014a000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7098004270 tx=0x7f7098004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70967fc700 1 -- 172.21.15.45:0/1978449944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980048e0 con 0x7f70a0157be0 2023-08-27T11:12:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70967fc700 1 -- 172.21.15.45:0/1978449944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70980099b0 con 0x7f70a0157be0 2023-08-27T11:12:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70967fc700 1 -- 172.21.15.45:0/1978449944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709801b670 con 0x7f70a0157be0 2023-08-27T11:12:23.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7098009b30 con 0x7f70a0157be0 2023-08-27T11:12:23.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.246+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a014bd00 con 0x7f70a0157be0 2023-08-27T11:12:23.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.247+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a014c220 con 0x7f70a0157be0 2023-08-27T11:12:23.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.247+0000 7f70967fc700 1 -- 172.21.15.45:0/1978449944 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f709801b7d0 con 0x7f70a0157be0 2023-08-27T11:12:23.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.248+0000 7f70967fc700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f708406a0e0 0x7f708406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.248+0000 7f70967fc700 1 -- 172.21.15.45:0/1978449944 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f709808a0f0 con 0x7f70a0157be0 2023-08-27T11:12:23.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.248+0000 7f70a5a3c700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f708406a0e0 0x7f708406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.248+0000 7f70a5a3c700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f708406a0e0 0x7f708406c5a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7090007fd0 tx=0x7f7090009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:23.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.252+0000 7f70967fc700 1 -- 172.21.15.45:0/1978449944 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f70a014c220 con 0x7f70a0157be0 2023-08-27T11:12:23.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.449+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f70a0072690 con 0x7f70a0157be0 2023-08-27T11:12:23.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.449+0000 7f70967fc700 1 -- 172.21.15.45:0/1978449944 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f70980554e0 con 0x7f70a0157be0 2023-08-27T11:12:23.451 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:12:23.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f708406a0e0 msgr2=0x7f708406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f708406a0e0 0x7f708406c5a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7090007fd0 tx=0x7f7090009040 comp rx=0 tx=0).stop 2023-08-27T11:12:23.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a0157be0 msgr2=0x7f70a014a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a0157be0 0x7f70a014a000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7098004270 tx=0x7f7098004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 shutdown_connections 2023-08-27T11:12:23.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a0157be0 0x7f70a014a000 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f708406a0e0 0x7f708406c5a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70a006ced0 0x7f70a0166910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 --2- 172.21.15.45:0/1978449944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70a00b17c0 0x7f70a01451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.451+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 >> 172.21.15.45:0/1978449944 conn(0x7f70a00cc7c0 msgr2=0x7f70a006e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:23.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.452+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 shutdown_connections 2023-08-27T11:12:23.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.452+0000 7f70a7ca0700 1 -- 172.21.15.45:0/1978449944 wait complete. 2023-08-27T11:12:23.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:12:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.590+0000 7f9dc706b700 1 Processor -- start 2023-08-27T11:12:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.591+0000 7f9dc706b700 1 -- start start 2023-08-27T11:12:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.591+0000 7f9dc706b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9dc0157be0 0x7f9dc00716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.591+0000 7f9dc706b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc00b17c0 0x7f9dc00b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.591+0000 7f9dc706b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc015b460 con 0x7f9dc006ced0 2023-08-27T11:12:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.591+0000 7f9dc706b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc0157fc0 con 0x7f9dc0157be0 2023-08-27T11:12:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.591+0000 7f9dc706b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b2280 con 0x7f9dc00b17c0 2023-08-27T11:12:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.591+0000 7f9dc5608700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9dc006ced0 0x7f9dc00cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38522/0 (socket says 172.21.15.45:38522) 2023-08-27T11:12:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.591+0000 7f9dc5608700 1 -- 172.21.15.45:0/3458527139 learned_addr learned my addr 172.21.15.45:0/3458527139 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dc4e07700 1 --2- 172.21.15.45:0/3458527139 >> v2:172.21.15.135:3300/0 conn(0x7f9dc0157be0 0x7f9dc00716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbffff700 1 --2- 172.21.15.45:0/3458527139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc00b17c0 0x7f9dc00b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbf7fe700 1 -- 172.21.15.45:0/3458527139 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679894650 0 0) 0x7f9dc015b460 con 0x7f9dc006ced0 2023-08-27T11:12:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbf7fe700 1 -- 172.21.15.45:0/3458527139 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db00035b0 con 0x7f9dc006ced0 2023-08-27T11:12:23.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbffff700 1 -- 172.21.15.45:0/3458527139 >> v2:172.21.15.135:3300/0 conn(0x7f9dc0157be0 msgr2=0x7f9dc00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:23.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbffff700 1 --2- 172.21.15.45:0/3458527139 >> v2:172.21.15.135:3300/0 conn(0x7f9dc0157be0 0x7f9dc00716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbffff700 1 -- 172.21.15.45:0/3458527139 >> v1:172.21.15.45:6789/0 conn(0x7f9dc006ced0 legacy=0x7f9dc00cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbffff700 1 -- 172.21.15.45:0/3458527139 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00b2560 con 0x7f9dc00b17c0 2023-08-27T11:12:23.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dc4e07700 1 --2- 172.21.15.45:0/3458527139 >> v2:172.21.15.135:3300/0 conn(0x7f9dc0157be0 0x7f9dc00716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:23.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbffff700 1 --2- 172.21.15.45:0/3458527139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc00b17c0 0x7f9dc00b1c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009840 tx=0x7f9dac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f76a6686f754e49 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:23.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbf7fe700 1 -- 172.21.15.45:0/3458527139 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac011830 con 0x7f9dc00b17c0 2023-08-27T11:12:23.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbf7fe700 1 -- 172.21.15.45:0/3458527139 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac011990 con 0x7f9dc00b17c0 2023-08-27T11:12:23.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.592+0000 7f9dbf7fe700 1 -- 172.21.15.45:0/3458527139 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac011d20 con 0x7f9dc00b17c0 2023-08-27T11:12:23.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.593+0000 7f9dc706b700 1 -- 172.21.15.45:0/3458527139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc00b17c0 msgr2=0x7f9dc00b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.593+0000 7f9dc706b700 1 --2- 172.21.15.45:0/3458527139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc00b17c0 0x7f9dc00b1c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009840 tx=0x7f9dac009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.593+0000 7f9dc706b700 1 -- 172.21.15.45:0/3458527139 shutdown_connections 2023-08-27T11:12:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.593+0000 7f9dc706b700 1 --2- 172.21.15.45:0/3458527139 >> v2:172.21.15.135:3300/0 conn(0x7f9dc0157be0 0x7f9dc00716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.593+0000 7f9dc706b700 1 --2- 172.21.15.45:0/3458527139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc00b17c0 0x7f9dc00b1c40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.593+0000 7f9dc706b700 1 -- 172.21.15.45:0/3458527139 >> 172.21.15.45:0/3458527139 conn(0x7f9dc00cc7c0 msgr2=0x7f9dc00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.593+0000 7f9dc706b700 1 -- 172.21.15.45:0/3458527139 shutdown_connections 2023-08-27T11:12:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.593+0000 7f9dc706b700 1 -- 172.21.15.45:0/3458527139 wait complete. 2023-08-27T11:12:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.594+0000 7f9dc706b700 1 Processor -- start 2023-08-27T11:12:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.594+0000 7f9dc706b700 1 -- start start 2023-08-27T11:12:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dc706b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dc006ced0 0x7f9dc014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dc706b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dc00b17c0 0x7f9dc015c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dc706b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc0157be0 0x7f9dc014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dc706b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc014bc60 con 0x7f9dc00b17c0 2023-08-27T11:12:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dc706b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc014b930 con 0x7f9dc006ced0 2023-08-27T11:12:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dc706b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c9870 con 0x7f9dc0157be0 2023-08-27T11:12:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dbffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dc00b17c0 0x7f9dc015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dbffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dc00b17c0 0x7f9dc015c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51750/0 (socket says 172.21.15.45:51750) 2023-08-27T11:12:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dbffff700 1 -- 172.21.15.45:0/2350799489 learned_addr learned my addr 172.21.15.45:0/2350799489 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:23.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.595+0000 7f9dc5608700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc0157be0 0x7f9dc014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dc4e07700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dc006ced0 0x7f9dc014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dbffff700 1 -- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc0157be0 msgr2=0x7f9dc014c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dbffff700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc0157be0 0x7f9dc014c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dbffff700 1 -- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dc006ced0 msgr2=0x7f9dc014eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dbffff700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dc006ced0 0x7f9dc014eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dbffff700 1 -- 172.21.15.45:0/2350799489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40094a0 con 0x7f9dc00b17c0 2023-08-27T11:12:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dc5608700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc0157be0 0x7f9dc014c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dbffff700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dc00b17c0 0x7f9dc015c910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0090b0 tx=0x7f9dac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dbd7fa700 1 -- 172.21.15.45:0/2350799489 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0036a0 con 0x7f9dc00b17c0 2023-08-27T11:12:23.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dbd7fa700 1 -- 172.21.15.45:0/2350799489 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac01f070 con 0x7f9dc00b17c0 2023-08-27T11:12:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dac0094a0 con 0x7f9dc00b17c0 2023-08-27T11:12:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dbd7fa700 1 -- 172.21.15.45:0/2350799489 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac003a40 con 0x7f9dc00b17c0 2023-08-27T11:12:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.596+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc00c9520 con 0x7f9dc00b17c0 2023-08-27T11:12:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.597+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc014be10 con 0x7f9dc00b17c0 2023-08-27T11:12:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.597+0000 7f9dbd7fa700 1 -- 172.21.15.45:0/2350799489 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9dac003ba0 con 0x7f9dc00b17c0 2023-08-27T11:12:23.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.598+0000 7f9dbd7fa700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9da406a0e0 0x7f9da406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.598+0000 7f9dbd7fa700 1 -- 172.21.15.45:0/2350799489 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9dac021070 con 0x7f9dc00b17c0 2023-08-27T11:12:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.598+0000 7f9dc4e07700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9da406a0e0 0x7f9da406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.598+0000 7f9dc4e07700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9da406a0e0 0x7f9da406c5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9dc015d690 tx=0x7f9db4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:23.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.603+0000 7f9dbd7fa700 1 -- 172.21.15.45:0/2350799489 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9dac055510 con 0x7f9dc00b17c0 2023-08-27T11:12:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.804+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9dc0072690 con 0x7f9dc00b17c0 2023-08-27T11:12:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.805+0000 7f9dbd7fa700 1 -- 172.21.15.45:0/2350799489 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9dac015020 con 0x7f9dc00b17c0 2023-08-27T11:12:23.806 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9da406a0e0 msgr2=0x7f9da406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9da406a0e0 0x7f9da406c5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9dc015d690 tx=0x7f9db4008040 comp rx=0 tx=0).stop 2023-08-27T11:12:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dc00b17c0 msgr2=0x7f9dc015c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dc00b17c0 0x7f9dc015c910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0090b0 tx=0x7f9dac004750 comp rx=0 tx=0).stop 2023-08-27T11:12:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 shutdown_connections 2023-08-27T11:12:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9da406a0e0 0x7f9da406c5a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dc0157be0 0x7f9dc014c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dc006ced0 0x7f9dc014eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 --2- 172.21.15.45:0/2350799489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dc00b17c0 0x7f9dc015c910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 >> 172.21.15.45:0/2350799489 conn(0x7f9dc00cc7c0 msgr2=0x7f9dc014e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 shutdown_connections 2023-08-27T11:12:23.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.807+0000 7f9dc706b700 1 -- 172.21.15.45:0/2350799489 wait complete. 2023-08-27T11:12:23.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:12:23.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.948+0000 7f26c5931700 1 Processor -- start 2023-08-27T11:12:23.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.948+0000 7f26c5931700 1 -- start start 2023-08-27T11:12:23.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.949+0000 7f26c5931700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f26c006a280 0x7f26c006a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.949+0000 7f26c5931700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f26c0071400 0x7f26c0071890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.949+0000 7f26c5931700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c00b2290 con 0x7f26c006ced0 2023-08-27T11:12:23.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.949+0000 7f26c5931700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c015c5c0 con 0x7f26c0071400 2023-08-27T11:12:23.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.949+0000 7f26c5931700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c006aba0 con 0x7f26c006a280 2023-08-27T11:12:23.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.949+0000 7f26be7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f26c0071400 0x7f26c0071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.949+0000 7f26bf7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f26c006ced0 0x7f26c006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38536/0 (socket says 172.21.15.45:38536) 2023-08-27T11:12:23.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.949+0000 7f26bf7fe700 1 -- 172.21.15.45:0/3064889453 learned_addr learned my addr 172.21.15.45:0/3064889453 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:23.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.949+0000 7f26be7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f26c0071400 0x7f26c0071890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51758/0 (socket says 172.21.15.45:51758) 2023-08-27T11:12:23.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26beffd700 1 --2- 172.21.15.45:0/3064889453 >> v2:172.21.15.135:3300/0 conn(0x7f26c006a280 0x7f26c006a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26be7fc700 1 -- 172.21.15.45:0/3064889453 >> v2:172.21.15.135:3300/0 conn(0x7f26c006a280 msgr2=0x7f26c006a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:23.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26be7fc700 1 --2- 172.21.15.45:0/3064889453 >> v2:172.21.15.135:3300/0 conn(0x7f26c006a280 0x7f26c006a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26be7fc700 1 -- 172.21.15.45:0/3064889453 >> v1:172.21.15.45:6789/0 conn(0x7f26c006ced0 legacy=0x7f26c006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26be7fc700 1 -- 172.21.15.45:0/3064889453 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c00b1960 con 0x7f26c0071400 2023-08-27T11:12:23.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26bdffb700 1 -- 172.21.15.45:0/3064889453 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296061840 0 0) 0x7f26c00b2290 con 0x7f26c006ced0 2023-08-27T11:12:23.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26be7fc700 1 --2- 172.21.15.45:0/3064889453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c0071400 0x7f26c0071890 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f26c00cd830 tx=0x7f26b0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a12afc57986d10 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:23.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26bdffb700 1 -- 172.21.15.45:0/3064889453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0011830 con 0x7f26c0071400 2023-08-27T11:12:23.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26bdffb700 1 -- 172.21.15.45:0/3064889453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b0011990 con 0x7f26c0071400 2023-08-27T11:12:23.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.950+0000 7f26bdffb700 1 -- 172.21.15.45:0/3064889453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0011d20 con 0x7f26c0071400 2023-08-27T11:12:23.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.951+0000 7f26c5931700 1 -- 172.21.15.45:0/3064889453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c0071400 msgr2=0x7f26c0071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.951+0000 7f26c5931700 1 --2- 172.21.15.45:0/3064889453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c0071400 0x7f26c0071890 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f26c00cd830 tx=0x7f26b0009b20 comp rx=0 tx=0).stop 2023-08-27T11:12:23.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.951+0000 7f26c5931700 1 -- 172.21.15.45:0/3064889453 shutdown_connections 2023-08-27T11:12:23.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.951+0000 7f26c5931700 1 --2- 172.21.15.45:0/3064889453 >> v2:172.21.15.135:3300/0 conn(0x7f26c006a280 0x7f26c006a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.951+0000 7f26c5931700 1 --2- 172.21.15.45:0/3064889453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c0071400 0x7f26c0071890 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.951+0000 7f26c5931700 1 -- 172.21.15.45:0/3064889453 >> 172.21.15.45:0/3064889453 conn(0x7f26c00cc7c0 msgr2=0x7f26c00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:23.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.951+0000 7f26c5931700 1 -- 172.21.15.45:0/3064889453 shutdown_connections 2023-08-27T11:12:23.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.951+0000 7f26c5931700 1 -- 172.21.15.45:0/3064889453 wait complete. 2023-08-27T11:12:23.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.952+0000 7f26c5931700 1 Processor -- start 2023-08-27T11:12:23.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.952+0000 7f26c5931700 1 -- start start 2023-08-27T11:12:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26c5931700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c006a280 0x7f26c01effa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26c5931700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c006ced0 0x7f26c00b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26c5931700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c0071400 0x7f26c00b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26c5931700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c0155b70 con 0x7f26c006ced0 2023-08-27T11:12:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26c5931700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c0156410 con 0x7f26c006a280 2023-08-27T11:12:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26c5931700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c01f4b30 con 0x7f26c0071400 2023-08-27T11:12:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26be7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c006ced0 0x7f26c00b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26be7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c006ced0 0x7f26c00b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51764/0 (socket says 172.21.15.45:51764) 2023-08-27T11:12:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26be7fc700 1 -- 172.21.15.45:0/2684789963 learned_addr learned my addr 172.21.15.45:0/2684789963 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26beffd700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c006a280 0x7f26c01effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26bf7fe700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c0071400 0x7f26c00b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26be7fc700 1 -- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c0071400 msgr2=0x7f26c00b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26be7fc700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c0071400 0x7f26c00b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26be7fc700 1 -- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c006a280 msgr2=0x7f26c01effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26be7fc700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c006a280 0x7f26c01effa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.953+0000 7f26be7fc700 1 -- 172.21.15.45:0/2684789963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b00094a0 con 0x7f26c006ced0 2023-08-27T11:12:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.954+0000 7f26be7fc700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c006ced0 0x7f26c00b4180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f26c006cbb0 tx=0x7f26b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.954+0000 7f26a7fff700 1 -- 172.21.15.45:0/2684789963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0004590 con 0x7f26c006ced0 2023-08-27T11:12:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.954+0000 7f26a7fff700 1 -- 172.21.15.45:0/2684789963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b00036a0 con 0x7f26c006ced0 2023-08-27T11:12:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.954+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c0155dc0 con 0x7f26c006ced0 2023-08-27T11:12:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.954+0000 7f26a7fff700 1 -- 172.21.15.45:0/2684789963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0003a30 con 0x7f26c006ced0 2023-08-27T11:12:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.954+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c01f04e0 con 0x7f26c006ced0 2023-08-27T11:12:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.955+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c00b1c20 con 0x7f26c006ced0 2023-08-27T11:12:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.956+0000 7f26a7fff700 1 -- 172.21.15.45:0/2684789963 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f26b0003b90 con 0x7f26c006ced0 2023-08-27T11:12:23.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.957+0000 7f26a7fff700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f26ac06a0e0 0x7f26ac06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:23.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.957+0000 7f26a7fff700 1 -- 172.21.15.45:0/2684789963 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f26b008a1d0 con 0x7f26c006ced0 2023-08-27T11:12:23.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.957+0000 7f26beffd700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f26ac06a0e0 0x7f26ac06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:23.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.957+0000 7f26beffd700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f26ac06a0e0 0x7f26ac06c5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009b40 tx=0x7f26b4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:23.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:23.960+0000 7f26a7fff700 1 -- 172.21.15.45:0/2684789963 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f26b0055640 con 0x7f26c006ced0 2023-08-27T11:12:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.158+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26c00b1c20 con 0x7f26c006ced0 2023-08-27T11:12:24.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.158+0000 7f26a7fff700 1 -- 172.21.15.45:0/2684789963 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26c00b1c20 con 0x7f26c006ced0 2023-08-27T11:12:24.159 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:12:24.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f26ac06a0e0 msgr2=0x7f26ac06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f26ac06a0e0 0x7f26ac06c5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009b40 tx=0x7f26b4009210 comp rx=0 tx=0).stop 2023-08-27T11:12:24.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c006ced0 msgr2=0x7f26c00b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c006ced0 0x7f26c00b4180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f26c006cbb0 tx=0x7f26b0004750 comp rx=0 tx=0).stop 2023-08-27T11:12:24.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 shutdown_connections 2023-08-27T11:12:24.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f26ac06a0e0 0x7f26ac06c5a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26c0071400 0x7f26c00b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26c006a280 0x7f26c01effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 --2- 172.21.15.45:0/2684789963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26c006ced0 0x7f26c00b4180 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 >> 172.21.15.45:0/2684789963 conn(0x7f26c00cc7c0 msgr2=0x7f26c015d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:24.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.160+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 shutdown_connections 2023-08-27T11:12:24.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.161+0000 7f26c5931700 1 -- 172.21.15.45:0/2684789963 wait complete. 2023-08-27T11:12:24.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:12:24.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.301+0000 7f2388d11700 1 Processor -- start 2023-08-27T11:12:24.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.301+0000 7f2388d11700 1 -- start start 2023-08-27T11:12:24.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2388d11700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f238415d340 0x7f2384160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2388d11700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23840cd880 0x7f238406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2388d11700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238406b160 con 0x7f23840cdd00 2023-08-27T11:12:24.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2388d11700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2384072010 con 0x7f23840cd880 2023-08-27T11:12:24.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2388d11700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23840af590 con 0x7f238415d340 2023-08-27T11:12:24.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2381d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23840cd880 0x7f238406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2382d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f23840cdd00 0x7f23840b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38548/0 (socket says 172.21.15.45:38548) 2023-08-27T11:12:24.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2382d9d700 1 -- 172.21.15.45:0/2668959810 learned_addr learned my addr 172.21.15.45:0/2668959810 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:24.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2381d9b700 1 -- 172.21.15.45:0/2668959810 >> v2:172.21.15.135:3300/0 conn(0x7f238415d340 msgr2=0x7f2384160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:24.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2381d9b700 1 --2- 172.21.15.45:0/2668959810 >> v2:172.21.15.135:3300/0 conn(0x7f238415d340 0x7f2384160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2381d9b700 1 -- 172.21.15.45:0/2668959810 >> v1:172.21.15.45:6789/0 conn(0x7f23840cdd00 legacy=0x7f23840b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2381d9b700 1 -- 172.21.15.45:0/2668959810 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840af2b0 con 0x7f23840cd880 2023-08-27T11:12:24.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f238159a700 1 -- 172.21.15.45:0/2668959810 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237585299 0 0) 0x7f238406b160 con 0x7f23840cdd00 2023-08-27T11:12:24.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f2381d9b700 1 --2- 172.21.15.45:0/2668959810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cd880 0x7f238406a280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f23840732d0 tx=0x7f2374009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df871db2c144a49 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:24.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f238159a700 1 -- 172.21.15.45:0/2668959810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374011830 con 0x7f23840cd880 2023-08-27T11:12:24.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f238159a700 1 -- 172.21.15.45:0/2668959810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2374011990 con 0x7f23840cd880 2023-08-27T11:12:24.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.302+0000 7f238159a700 1 -- 172.21.15.45:0/2668959810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374011d20 con 0x7f23840cd880 2023-08-27T11:12:24.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.303+0000 7f2388d11700 1 -- 172.21.15.45:0/2668959810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cd880 msgr2=0x7f238406a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.303+0000 7f2388d11700 1 --2- 172.21.15.45:0/2668959810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cd880 0x7f238406a280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f23840732d0 tx=0x7f2374009b20 comp rx=0 tx=0).stop 2023-08-27T11:12:24.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.303+0000 7f2388d11700 1 -- 172.21.15.45:0/2668959810 shutdown_connections 2023-08-27T11:12:24.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.303+0000 7f2388d11700 1 --2- 172.21.15.45:0/2668959810 >> v2:172.21.15.135:3300/0 conn(0x7f238415d340 0x7f2384160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.303+0000 7f2388d11700 1 --2- 172.21.15.45:0/2668959810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cd880 0x7f238406a280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.303+0000 7f2388d11700 1 -- 172.21.15.45:0/2668959810 >> 172.21.15.45:0/2668959810 conn(0x7f23840cc7c0 msgr2=0x7f23840ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:24.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.303+0000 7f2388d11700 1 -- 172.21.15.45:0/2668959810 shutdown_connections 2023-08-27T11:12:24.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.303+0000 7f2388d11700 1 -- 172.21.15.45:0/2668959810 wait complete. 2023-08-27T11:12:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.304+0000 7f2388d11700 1 Processor -- start 2023-08-27T11:12:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.304+0000 7f2388d11700 1 -- start start 2023-08-27T11:12:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2388d11700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23840cd880 0x7f238415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2388d11700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cdd00 0x7f238414e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2388d11700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f238415d340 0x7f23840cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2388d11700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23840ca3c0 con 0x7f23840cdd00 2023-08-27T11:12:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2388d11700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23840ca6a0 con 0x7f23840cd880 2023-08-27T11:12:24.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2388d11700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23840ca980 con 0x7f238415d340 2023-08-27T11:12:24.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2381d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cdd00 0x7f238414e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2381d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cdd00 0x7f238414e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51778/0 (socket says 172.21.15.45:51778) 2023-08-27T11:12:24.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2381d9b700 1 -- 172.21.15.45:0/1871333177 learned_addr learned my addr 172.21.15.45:0/1871333177 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:24.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f2382d9d700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f238415d340 0x7f23840cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.305+0000 7f238259c700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23840cd880 0x7f238415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f2381d9b700 1 -- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f238415d340 msgr2=0x7f23840cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f2381d9b700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f238415d340 0x7f23840cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f2381d9b700 1 -- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23840cd880 msgr2=0x7f238415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f2381d9b700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23840cd880 0x7f238415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f2381d9b700 1 -- 172.21.15.45:0/1871333177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740094a0 con 0x7f23840cdd00 2023-08-27T11:12:24.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f238259c700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23840cd880 0x7f238415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:24.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f2382d9d700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f238415d340 0x7f23840cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:24.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f2381d9b700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cdd00 0x7f238414e070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2374014040 tx=0x7f2374004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:24.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f237affd700 1 -- 172.21.15.45:0/1871333177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740038a0 con 0x7f23840cdd00 2023-08-27T11:12:24.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f237affd700 1 -- 172.21.15.45:0/1871333177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237401e070 con 0x7f23840cdd00 2023-08-27T11:12:24.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23840cac60 con 0x7f23840cdd00 2023-08-27T11:12:24.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f237affd700 1 -- 172.21.15.45:0/1871333177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374003c60 con 0x7f23840cdd00 2023-08-27T11:12:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.306+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238415f780 con 0x7f23840cdd00 2023-08-27T11:12:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.307+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2384072690 con 0x7f23840cdd00 2023-08-27T11:12:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.307+0000 7f237affd700 1 -- 172.21.15.45:0/1871333177 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f237402d450 con 0x7f23840cdd00 2023-08-27T11:12:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.308+0000 7f237affd700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f236406a0e0 0x7f236406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.308+0000 7f237affd700 1 -- 172.21.15.45:0/1871333177 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2374089990 con 0x7f23840cdd00 2023-08-27T11:12:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.308+0000 7f238259c700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f236406a0e0 0x7f236406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.308+0000 7f238259c700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f236406a0e0 0x7f236406c5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f236c006fd0 tx=0x7f236c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.312+0000 7f237affd700 1 -- 172.21.15.45:0/1871333177 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2374054e20 con 0x7f23840cdd00 2023-08-27T11:12:24.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.513+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f23840734a0 con 0x7f23840cdd00 2023-08-27T11:12:24.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.514+0000 7f237affd700 1 -- 172.21.15.45:0/1871333177 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2374015020 con 0x7f23840cdd00 2023-08-27T11:12:24.515 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:24.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f236406a0e0 msgr2=0x7f236406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f236406a0e0 0x7f236406c5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f236c006fd0 tx=0x7f236c008040 comp rx=0 tx=0).stop 2023-08-27T11:12:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cdd00 msgr2=0x7f238414e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cdd00 0x7f238414e070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2374014040 tx=0x7f2374004750 comp rx=0 tx=0).stop 2023-08-27T11:12:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 shutdown_connections 2023-08-27T11:12:24.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f236406a0e0 0x7f236406c5a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f238415d340 0x7f23840cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23840cd880 0x7f238415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 --2- 172.21.15.45:0/1871333177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23840cdd00 0x7f238414e070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 >> 172.21.15.45:0/1871333177 conn(0x7f23840cc7c0 msgr2=0x7f23840af760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:24.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 shutdown_connections 2023-08-27T11:12:24.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.516+0000 7f2388d11700 1 -- 172.21.15.45:0/1871333177 wait complete. 2023-08-27T11:12:24.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:12:24.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.654+0000 7fb4715a4700 1 Processor -- start 2023-08-27T11:12:24.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.654+0000 7fb4715a4700 1 -- start start 2023-08-27T11:12:24.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb4715a4700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb46c0ce4b0 0x7fb46c167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb4715a4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c15d6a0 con 0x7fb46c15d2c0 2023-08-27T11:12:24.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb4715a4700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c0d2c90 con 0x7fb46c0b8c20 2023-08-27T11:12:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb4715a4700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c0d1bf0 con 0x7fb46c0ce4b0 2023-08-27T11:12:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb46affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb46c15d2c0 0x7fb46c0d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38560/0 (socket says 172.21.15.45:38560) 2023-08-27T11:12:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb46affd700 1 -- 172.21.15.45:0/3411053524 learned_addr learned my addr 172.21.15.45:0/3411053524 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb46b7fe700 1 --2- 172.21.15.45:0/3411053524 >> v2:172.21.15.135:3300/0 conn(0x7fb46c0ce4b0 0x7fb46c167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb469ffb700 1 -- 172.21.15.45:0/3411053524 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981343133 0 0) 0x7fb460001110 con 0x7fb46c15d2c0 2023-08-27T11:12:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb469ffb700 1 -- 172.21.15.45:0/3411053524 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4540035b0 con 0x7fb46c15d2c0 2023-08-27T11:12:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb469ffb700 1 -- 172.21.15.45:0/3411053524 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3810376902 0 0) 0x7fb4540035b0 con 0x7fb46c15d2c0 2023-08-27T11:12:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb46b7fe700 1 -- 172.21.15.45:0/3411053524 >> v1:172.21.15.135:6789/0 conn(0x7fb46c0b8c20 legacy=0x7fb46c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb46b7fe700 1 -- 172.21.15.45:0/3411053524 >> v1:172.21.15.45:6789/0 conn(0x7fb46c15d2c0 legacy=0x7fb46c0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.655+0000 7fb46b7fe700 1 -- 172.21.15.45:0/3411053524 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0d1ed0 con 0x7fb46c0ce4b0 2023-08-27T11:12:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb46b7fe700 1 --2- 172.21.15.45:0/3411053524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 0x7fb46c167d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb464009dc0 tx=0x7fb46400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869d99340ef63aa8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb469ffb700 1 -- 172.21.15.45:0/3411053524 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46400e070 con 0x7fb46c0ce4b0 2023-08-27T11:12:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb469ffb700 1 -- 172.21.15.45:0/3411053524 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb464003a20 con 0x7fb46c0ce4b0 2023-08-27T11:12:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb469ffb700 1 -- 172.21.15.45:0/3411053524 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460001800 con 0x7fb46c0ce4b0 2023-08-27T11:12:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb4715a4700 1 -- 172.21.15.45:0/3411053524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 msgr2=0x7fb46c167d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb4715a4700 1 --2- 172.21.15.45:0/3411053524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 0x7fb46c167d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb464009dc0 tx=0x7fb46400b330 comp rx=0 tx=0).stop 2023-08-27T11:12:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb4715a4700 1 -- 172.21.15.45:0/3411053524 shutdown_connections 2023-08-27T11:12:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb4715a4700 1 --2- 172.21.15.45:0/3411053524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 0x7fb46c167d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb4715a4700 1 -- 172.21.15.45:0/3411053524 >> 172.21.15.45:0/3411053524 conn(0x7fb46c0d2fd0 msgr2=0x7fb46c0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb4715a4700 1 -- 172.21.15.45:0/3411053524 shutdown_connections 2023-08-27T11:12:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.656+0000 7fb4715a4700 1 -- 172.21.15.45:0/3411053524 wait complete. 2023-08-27T11:12:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.658+0000 7fb4715a4700 1 Processor -- start 2023-08-27T11:12:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.658+0000 7fb4715a4700 1 -- start start 2023-08-27T11:12:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.658+0000 7fb4715a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb46c0b8c20 0x7fb46c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.658+0000 7fb4715a4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 0x7fb46c149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.658+0000 7fb4715a4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb46c15d2c0 0x7fb46c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.658+0000 7fb4715a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0b7480 con 0x7fb46c0b8c20 2023-08-27T11:12:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.658+0000 7fb4715a4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0b6800 con 0x7fb46c0ce4b0 2023-08-27T11:12:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.658+0000 7fb4715a4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0b6ae0 con 0x7fb46c15d2c0 2023-08-27T11:12:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.658+0000 7fb46affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb46c0b8c20 0x7fb46c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb46c0b8c20 0x7fb46c0b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51790/0 (socket says 172.21.15.45:51790) 2023-08-27T11:12:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46affd700 1 -- 172.21.15.45:0/2156293096 learned_addr learned my addr 172.21.15.45:0/2156293096 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46b7fe700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb46c15d2c0 0x7fb46c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46a7fc700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 0x7fb46c149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46affd700 1 -- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb46c15d2c0 msgr2=0x7fb46c0c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46affd700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb46c15d2c0 0x7fb46c0c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46affd700 1 -- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 msgr2=0x7fb46c149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46affd700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 0x7fb46c149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46affd700 1 -- 172.21.15.45:0/2156293096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb464009a70 con 0x7fb46c0b8c20 2023-08-27T11:12:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46a7fc700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 0x7fb46c149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb46affd700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb46c0b8c20 0x7fb46c0b6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c780 tx=0x7fb46000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb4537fe700 1 -- 172.21.15.45:0/2156293096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460007560 con 0x7fb46c0b8c20 2023-08-27T11:12:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb4537fe700 1 -- 172.21.15.45:0/2156293096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4600076e0 con 0x7fb46c0b8c20 2023-08-27T11:12:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c0b6dc0 con 0x7fb46c0b8c20 2023-08-27T11:12:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb4537fe700 1 -- 172.21.15.45:0/2156293096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460004ab0 con 0x7fb46c0b8c20 2023-08-27T11:12:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.659+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c0c7000 con 0x7fb46c0b8c20 2023-08-27T11:12:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.660+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c0c7220 con 0x7fb46c0b8c20 2023-08-27T11:12:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.661+0000 7fb4537fe700 1 -- 172.21.15.45:0/2156293096 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb460004c10 con 0x7fb46c0b8c20 2023-08-27T11:12:24.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.661+0000 7fb4537fe700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb45806a0e0 0x7fb45806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.661+0000 7fb4537fe700 1 -- 172.21.15.45:0/2156293096 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb460088460 con 0x7fb46c0b8c20 2023-08-27T11:12:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.661+0000 7fb46a7fc700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb45806a0e0 0x7fb45806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:24.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.662+0000 7fb46a7fc700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb45806a0e0 0x7fb45806c5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb46c153fa0 tx=0x7fb45c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:24.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.665+0000 7fb4537fe700 1 -- 172.21.15.45:0/2156293096 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb460053870 con 0x7fb46c0b8c20 2023-08-27T11:12:24.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.882+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb46c04e620 con 0x7fb46c0b8c20 2023-08-27T11:12:24.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.883+0000 7fb4537fe700 1 -- 172.21.15.45:0/2156293096 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb460056e90 con 0x7fb46c0b8c20 2023-08-27T11:12:24.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb45806a0e0 msgr2=0x7fb45806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb45806a0e0 0x7fb45806c5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb46c153fa0 tx=0x7fb45c009040 comp rx=0 tx=0).stop 2023-08-27T11:12:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb46c0b8c20 msgr2=0x7fb46c0b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb46c0b8c20 0x7fb46c0b6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c780 tx=0x7fb46000cb40 comp rx=0 tx=0).stop 2023-08-27T11:12:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 shutdown_connections 2023-08-27T11:12:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb46c0b8c20 0x7fb46c0b6100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb45806a0e0 0x7fb45806c5a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb46c15d2c0 0x7fb46c0c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 --2- 172.21.15.45:0/2156293096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb46c0ce4b0 0x7fb46c149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:24.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 >> 172.21.15.45:0/2156293096 conn(0x7fb46c0d2fd0 msgr2=0x7fb46c0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:24.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 shutdown_connections 2023-08-27T11:12:24.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:24.885+0000 7fb4715a4700 1 -- 172.21.15.45:0/2156293096 wait complete. 2023-08-27T11:12:24.890 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2023-08-27T11:12:24.898 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2023-08-27T11:12:24.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:24.898 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-08-27T11:12:24.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[110820]: fuse finished with error 0 and tester_r 0 2023-08-27T11:12:24.935 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:12:31.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:12:31.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:31.447 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-08-27T11:12:31.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:31.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.663+0000 7f384a0e5700 1 Processor -- start 2023-08-27T11:12:31.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.663+0000 7f384a0e5700 1 -- start start 2023-08-27T11:12:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f384a0e5700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f384406b130 0x7f384406b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f384a0e5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38440c9c00 0x7f38440ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f384a0e5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38440b0af0 con 0x7f38440aea90 2023-08-27T11:12:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f384a0e5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384406bb20 con 0x7f38440c9c00 2023-08-27T11:12:31.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f384a0e5700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384406a510 con 0x7f384406b130 2023-08-27T11:12:31.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f3843fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38440c9c00 0x7f38440ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:31.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f38437fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f38440aea90 0x7f38440cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38574/0 (socket says 172.21.15.45:38574) 2023-08-27T11:12:31.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f38437fe700 1 -- 172.21.15.45:0/2463543829 learned_addr learned my addr 172.21.15.45:0/2463543829 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:31.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f3842ffd700 1 --2- 172.21.15.45:0/2463543829 >> v2:172.21.15.135:3300/0 conn(0x7f384406b130 0x7f384406b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:31.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f3843fff700 1 -- 172.21.15.45:0/2463543829 >> v2:172.21.15.135:3300/0 conn(0x7f384406b130 msgr2=0x7f384406b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:31.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f3843fff700 1 --2- 172.21.15.45:0/2463543829 >> v2:172.21.15.135:3300/0 conn(0x7f384406b130 0x7f384406b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f3843fff700 1 -- 172.21.15.45:0/2463543829 >> v1:172.21.15.45:6789/0 conn(0x7f38440aea90 legacy=0x7f38440cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:31.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f38427fc700 1 -- 172.21.15.45:0/2463543829 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615418842 0 0) 0x7f38440b0af0 con 0x7f38440aea90 2023-08-27T11:12:31.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.664+0000 7f3843fff700 1 -- 172.21.15.45:0/2463543829 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384406a7b0 con 0x7f38440c9c00 2023-08-27T11:12:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f3843fff700 1 --2- 172.21.15.45:0/2463543829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440c9c00 0x7f38440ca070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f38440b34c0 tx=0x7f383c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5016516692430cd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f38427fc700 1 -- 172.21.15.45:0/2463543829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c00e070 con 0x7f38440c9c00 2023-08-27T11:12:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f38427fc700 1 -- 172.21.15.45:0/2463543829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c00bbd0 con 0x7f38440c9c00 2023-08-27T11:12:31.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f38427fc700 1 -- 172.21.15.45:0/2463543829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c012430 con 0x7f38440c9c00 2023-08-27T11:12:31.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f384a0e5700 1 -- 172.21.15.45:0/2463543829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440c9c00 msgr2=0x7f38440ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:31.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f384a0e5700 1 --2- 172.21.15.45:0/2463543829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440c9c00 0x7f38440ca070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f38440b34c0 tx=0x7f383c00b330 comp rx=0 tx=0).stop 2023-08-27T11:12:31.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f384a0e5700 1 -- 172.21.15.45:0/2463543829 shutdown_connections 2023-08-27T11:12:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f384a0e5700 1 --2- 172.21.15.45:0/2463543829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440c9c00 0x7f38440ca070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f384a0e5700 1 --2- 172.21.15.45:0/2463543829 >> v2:172.21.15.135:3300/0 conn(0x7f384406b130 0x7f384406b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.665+0000 7f384a0e5700 1 -- 172.21.15.45:0/2463543829 >> 172.21.15.45:0/2463543829 conn(0x7f38440b2470 msgr2=0x7f38440b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.666+0000 7f384a0e5700 1 -- 172.21.15.45:0/2463543829 shutdown_connections 2023-08-27T11:12:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.666+0000 7f384a0e5700 1 -- 172.21.15.45:0/2463543829 wait complete. 2023-08-27T11:12:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.667+0000 7f384a0e5700 1 Processor -- start 2023-08-27T11:12:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.667+0000 7f384a0e5700 1 -- start start 2023-08-27T11:12:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f384a0e5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f384406b130 0x7f38440c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f384a0e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440aea90 0x7f38440c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f384a0e5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38440c9c00 0x7f38440adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f384a0e5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38441f3090 con 0x7f38440aea90 2023-08-27T11:12:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f384a0e5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38440c9430 con 0x7f384406b130 2023-08-27T11:12:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f384a0e5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38440c4d20 con 0x7f38440c9c00 2023-08-27T11:12:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3842ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440aea90 0x7f38440c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:31.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3842ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440aea90 0x7f38440c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51806/0 (socket says 172.21.15.45:51806) 2023-08-27T11:12:31.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3842ffd700 1 -- 172.21.15.45:0/2404105613 learned_addr learned my addr 172.21.15.45:0/2404105613 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:31.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3843fff700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38440c9c00 0x7f38440adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f38437fe700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f384406b130 0x7f38440c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3842ffd700 1 -- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38440c9c00 msgr2=0x7f38440adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3842ffd700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38440c9c00 0x7f38440adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3842ffd700 1 -- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f384406b130 msgr2=0x7f38440c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3842ffd700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f384406b130 0x7f38440c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3842ffd700 1 -- 172.21.15.45:0/2404105613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c009a70 con 0x7f38440aea90 2023-08-27T11:12:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f38437fe700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f384406b130 0x7f38440c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3843fff700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38440c9c00 0x7f38440adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f3842ffd700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440aea90 0x7f38440c8ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f382c00e810 tx=0x7f382c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.668+0000 7f383bfff700 1 -- 172.21.15.45:0/2404105613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c013610 con 0x7f38440aea90 2023-08-27T11:12:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.669+0000 7f383bfff700 1 -- 172.21.15.45:0/2404105613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c013770 con 0x7f38440aea90 2023-08-27T11:12:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.669+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38440c5000 con 0x7f38440aea90 2023-08-27T11:12:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.669+0000 7f383bfff700 1 -- 172.21.15.45:0/2404105613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c013b20 con 0x7f38440aea90 2023-08-27T11:12:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.669+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38440c55c0 con 0x7f38440aea90 2023-08-27T11:12:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.669+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38440aee70 con 0x7f38440aea90 2023-08-27T11:12:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.671+0000 7f383bfff700 1 -- 172.21.15.45:0/2404105613 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f382c013c80 con 0x7f38440aea90 2023-08-27T11:12:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.671+0000 7f383bfff700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f382406a0e0 0x7f382406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.672+0000 7f383bfff700 1 -- 172.21.15.45:0/2404105613 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f382c0888e0 con 0x7f38440aea90 2023-08-27T11:12:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.672+0000 7f38437fe700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f382406a0e0 0x7f382406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:31.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.672+0000 7f38437fe700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f382406a0e0 0x7f382406c5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3834000f60 tx=0x7f3834007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:31.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.675+0000 7f383bfff700 1 -- 172.21.15.45:0/2404105613 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f382c053d70 con 0x7f38440aea90 2023-08-27T11:12:31.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.879+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38440b2d80 con 0x7f38440aea90 2023-08-27T11:12:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.881+0000 7f383bfff700 1 -- 172.21.15.45:0/2404105613 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f382c057390 con 0x7f38440aea90 2023-08-27T11:12:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:31.883 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f382406a0e0 msgr2=0x7f382406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f382406a0e0 0x7f382406c5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3834000f60 tx=0x7f3834007ca0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440aea90 msgr2=0x7f38440c8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:31.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440aea90 0x7f38440c8ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f382c00e810 tx=0x7f382c00ebd0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 shutdown_connections 2023-08-27T11:12:31.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f382406a0e0 0x7f382406c5a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38440c9c00 0x7f38440adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f384406b130 0x7f38440c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 --2- 172.21.15.45:0/2404105613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38440aea90 0x7f38440c8ef0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:31.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 >> 172.21.15.45:0/2404105613 conn(0x7f38440b2470 msgr2=0x7f38440cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:31.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 shutdown_connections 2023-08-27T11:12:31.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:31.883+0000 7f384a0e5700 1 -- 172.21.15.45:0/2404105613 wait complete. 2023-08-27T11:12:31.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.029+0000 7fc45edda700 1 Processor -- start 2023-08-27T11:12:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.029+0000 7fc45edda700 1 -- start start 2023-08-27T11:12:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.029+0000 7fc45edda700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc4580d0910 0x7fc45806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.029+0000 7fc45edda700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc45806e920 0x7fc45806a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.029+0000 7fc45edda700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4580cf5c0 con 0x7fc45815d2d0 2023-08-27T11:12:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.029+0000 7fc45edda700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4580cfb90 con 0x7fc45806e920 2023-08-27T11:12:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.029+0000 7fc45edda700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc458161880 con 0x7fc4580d0910 2023-08-27T11:12:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc45d377700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc45806e920 0x7fc45806a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc45d377700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc45806e920 0x7fc45806a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51816/0 (socket says 172.21.15.45:51816) 2023-08-27T11:12:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc45d377700 1 -- 172.21.15.45:0/4052545868 learned_addr learned my addr 172.21.15.45:0/4052545868 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc457fff700 1 --2- 172.21.15.45:0/4052545868 >> v2:172.21.15.135:3300/0 conn(0x7fc4580d0910 0x7fc45806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc45d377700 1 -- 172.21.15.45:0/4052545868 >> v2:172.21.15.135:3300/0 conn(0x7fc4580d0910 msgr2=0x7fc45806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:32.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc45d377700 1 --2- 172.21.15.45:0/4052545868 >> v2:172.21.15.135:3300/0 conn(0x7fc4580d0910 0x7fc45806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc45d377700 1 -- 172.21.15.45:0/4052545868 >> v1:172.21.15.135:6789/0 conn(0x7fc45815d2d0 legacy=0x7fc4580cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc45d377700 1 -- 172.21.15.45:0/4052545868 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc458161db0 con 0x7fc45806e920 2023-08-27T11:12:32.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc45d377700 1 --2- 172.21.15.45:0/4052545868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45806e920 0x7fc45806a7c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc458067640 tx=0x7fc45000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a66f56f517064b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc4577fe700 1 -- 172.21.15.45:0/4052545868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000e070 con 0x7fc45806e920 2023-08-27T11:12:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc4577fe700 1 -- 172.21.15.45:0/4052545868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45000b9c0 con 0x7fc45806e920 2023-08-27T11:12:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.030+0000 7fc4577fe700 1 -- 172.21.15.45:0/4052545868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000bd70 con 0x7fc45806e920 2023-08-27T11:12:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.031+0000 7fc45edda700 1 -- 172.21.15.45:0/4052545868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45806e920 msgr2=0x7fc45806a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.031+0000 7fc45edda700 1 --2- 172.21.15.45:0/4052545868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45806e920 0x7fc45806a7c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc458067640 tx=0x7fc45000b330 comp rx=0 tx=0).stop 2023-08-27T11:12:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.031+0000 7fc45edda700 1 -- 172.21.15.45:0/4052545868 shutdown_connections 2023-08-27T11:12:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.031+0000 7fc45edda700 1 --2- 172.21.15.45:0/4052545868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45806e920 0x7fc45806a7c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.031+0000 7fc45edda700 1 --2- 172.21.15.45:0/4052545868 >> v2:172.21.15.135:3300/0 conn(0x7fc4580d0910 0x7fc45806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.031+0000 7fc45edda700 1 -- 172.21.15.45:0/4052545868 >> 172.21.15.45:0/4052545868 conn(0x7fc4580cfdf0 msgr2=0x7fc4580d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.031+0000 7fc45edda700 1 -- 172.21.15.45:0/4052545868 shutdown_connections 2023-08-27T11:12:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.031+0000 7fc45edda700 1 -- 172.21.15.45:0/4052545868 wait complete. 2023-08-27T11:12:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.032+0000 7fc45edda700 1 Processor -- start 2023-08-27T11:12:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.032+0000 7fc45edda700 1 -- start start 2023-08-27T11:12:32.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45edda700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc45806e920 0x7fc45815fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45edda700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4580d0910 0x7fc4581603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45edda700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45815d2d0 0x7fc4580c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45edda700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45815c150 con 0x7fc45815d2d0 2023-08-27T11:12:32.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45edda700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580c5bc0 con 0x7fc45806e920 2023-08-27T11:12:32.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45edda700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580c5ea0 con 0x7fc4580d0910 2023-08-27T11:12:32.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45d377700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45815d2d0 0x7fc4580c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45d377700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45815d2d0 0x7fc4580c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51820/0 (socket says 172.21.15.45:51820) 2023-08-27T11:12:32.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45d377700 1 -- 172.21.15.45:0/4270554825 learned_addr learned my addr 172.21.15.45:0/4270554825 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:32.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45cb76700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc45806e920 0x7fc45815fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc457fff700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4580d0910 0x7fc4581603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45d377700 1 -- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4580d0910 msgr2=0x7fc4581603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45d377700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4580d0910 0x7fc4581603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45d377700 1 -- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc45806e920 msgr2=0x7fc45815fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45d377700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc45806e920 0x7fc45815fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45d377700 1 -- 172.21.15.45:0/4270554825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc450009b30 con 0x7fc45815d2d0 2023-08-27T11:12:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.033+0000 7fc45d377700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45815d2d0 0x7fc4580c4df0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc45806c2a0 tx=0x7fc450004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.034+0000 7fc4557fa700 1 -- 172.21.15.45:0/4270554825 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450004880 con 0x7fc45815d2d0 2023-08-27T11:12:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.034+0000 7fc4557fa700 1 -- 172.21.15.45:0/4270554825 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4500099b0 con 0x7fc45815d2d0 2023-08-27T11:12:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.034+0000 7fc4557fa700 1 -- 172.21.15.45:0/4270554825 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45001b670 con 0x7fc45815d2d0 2023-08-27T11:12:32.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.034+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4580c6180 con 0x7fc45815d2d0 2023-08-27T11:12:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.034+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc458148760 con 0x7fc45815d2d0 2023-08-27T11:12:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.035+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4580c67d0 con 0x7fc45815d2d0 2023-08-27T11:12:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.035+0000 7fc4557fa700 1 -- 172.21.15.45:0/4270554825 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc45001b7d0 con 0x7fc45815d2d0 2023-08-27T11:12:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.036+0000 7fc4557fa700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc43c06a0e0 0x7fc43c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.036+0000 7fc4557fa700 1 -- 172.21.15.45:0/4270554825 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc450089fe0 con 0x7fc45815d2d0 2023-08-27T11:12:32.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.036+0000 7fc45cb76700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc43c06a0e0 0x7fc43c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.036+0000 7fc45cb76700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc43c06a0e0 0x7fc43c06c5a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc44c007fd0 tx=0x7fc44c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.040+0000 7fc4557fa700 1 -- 172.21.15.45:0/4270554825 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc4580c67d0 con 0x7fc45815d2d0 2023-08-27T11:12:32.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.243+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4580c67d0 con 0x7fc45815d2d0 2023-08-27T11:12:32.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.244+0000 7fc4557fa700 1 -- 172.21.15.45:0/4270554825 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7fc4580c67d0 con 0x7fc45815d2d0 2023-08-27T11:12:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:32.246 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:32.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc43c06a0e0 msgr2=0x7fc43c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc43c06a0e0 0x7fc43c06c5a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc44c007fd0 tx=0x7fc44c009040 comp rx=0 tx=0).stop 2023-08-27T11:12:32.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45815d2d0 msgr2=0x7fc4580c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45815d2d0 0x7fc4580c4df0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc45806c2a0 tx=0x7fc450004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 shutdown_connections 2023-08-27T11:12:32.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45815d2d0 0x7fc4580c4df0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc43c06a0e0 0x7fc43c06c5a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc45806e920 0x7fc45815fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 --2- 172.21.15.45:0/4270554825 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4580d0910 0x7fc4581603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 >> 172.21.15.45:0/4270554825 conn(0x7fc4580cfdf0 msgr2=0x7fc45806cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:32.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 shutdown_connections 2023-08-27T11:12:32.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.246+0000 7fc45edda700 1 -- 172.21.15.45:0/4270554825 wait complete. 2023-08-27T11:12:32.260 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:12:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.388+0000 7fec94239700 1 Processor -- start 2023-08-27T11:12:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.389+0000 7fec94239700 1 -- start start 2023-08-27T11:12:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec94239700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c0b17c0 0x7fec8c0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec94239700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c15b460 con 0x7fec8c06ced0 2023-08-27T11:12:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec94239700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c0b2280 con 0x7fec8c157be0 2023-08-27T11:12:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec94239700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c157fc0 con 0x7fec8c0b17c0 2023-08-27T11:12:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec927d6700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fec8c06ced0 0x7fec8c0cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38606/0 (socket says 172.21.15.45:38606) 2023-08-27T11:12:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec927d6700 1 -- 172.21.15.45:0/3590134821 learned_addr learned my addr 172.21.15.45:0/3590134821 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec917d4700 1 --2- 172.21.15.45:0/3590134821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c0b17c0 0x7fec8c0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec90fd3700 1 -- 172.21.15.45:0/3590134821 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906817935 0 0) 0x7fec8c15b460 con 0x7fec8c06ced0 2023-08-27T11:12:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec90fd3700 1 -- 172.21.15.45:0/3590134821 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec780035b0 con 0x7fec8c06ced0 2023-08-27T11:12:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec90fd3700 1 -- 172.21.15.45:0/3590134821 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320495689 0 0) 0x7fec8c0b2280 con 0x7fec8c157be0 2023-08-27T11:12:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec90fd3700 1 -- 172.21.15.45:0/3590134821 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec8c15b460 con 0x7fec8c157be0 2023-08-27T11:12:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec917d4700 1 -- 172.21.15.45:0/3590134821 >> v1:172.21.15.135:6789/0 conn(0x7fec8c157be0 legacy=0x7fec8c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec917d4700 1 -- 172.21.15.45:0/3590134821 >> v1:172.21.15.45:6789/0 conn(0x7fec8c06ced0 legacy=0x7fec8c0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec90fd3700 1 -- 172.21.15.45:0/3590134821 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2729181090 0 0) 0x7fec780035b0 con 0x7fec8c06ced0 2023-08-27T11:12:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.390+0000 7fec917d4700 1 -- 172.21.15.45:0/3590134821 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c0b2560 con 0x7fec8c0b17c0 2023-08-27T11:12:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.391+0000 7fec917d4700 1 --2- 172.21.15.45:0/3590134821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c0b17c0 0x7fec8c0b1c40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec88009870 tx=0x7fec88009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=312695ccf3d0f0a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.391+0000 7fec90fd3700 1 -- 172.21.15.45:0/3590134821 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec840019f0 con 0x7fec8c0b17c0 2023-08-27T11:12:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.391+0000 7fec90fd3700 1 -- 172.21.15.45:0/3590134821 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec88004030 con 0x7fec8c0b17c0 2023-08-27T11:12:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.391+0000 7fec90fd3700 1 -- 172.21.15.45:0/3590134821 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880043c0 con 0x7fec8c0b17c0 2023-08-27T11:12:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.391+0000 7fec94239700 1 -- 172.21.15.45:0/3590134821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c0b17c0 msgr2=0x7fec8c0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.391+0000 7fec94239700 1 --2- 172.21.15.45:0/3590134821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c0b17c0 0x7fec8c0b1c40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec88009870 tx=0x7fec88009b80 comp rx=0 tx=0).stop 2023-08-27T11:12:32.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.391+0000 7fec94239700 1 -- 172.21.15.45:0/3590134821 shutdown_connections 2023-08-27T11:12:32.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.391+0000 7fec94239700 1 --2- 172.21.15.45:0/3590134821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c0b17c0 0x7fec8c0b1c40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.392+0000 7fec94239700 1 -- 172.21.15.45:0/3590134821 >> 172.21.15.45:0/3590134821 conn(0x7fec8c0cc7c0 msgr2=0x7fec8c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:32.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.392+0000 7fec94239700 1 -- 172.21.15.45:0/3590134821 shutdown_connections 2023-08-27T11:12:32.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.392+0000 7fec94239700 1 -- 172.21.15.45:0/3590134821 wait complete. 2023-08-27T11:12:32.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.393+0000 7fec94239700 1 Processor -- start 2023-08-27T11:12:32.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.393+0000 7fec94239700 1 -- start start 2023-08-27T11:12:32.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec94239700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec8c06ced0 0x7fec8c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec94239700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec8c0b17c0 0x7fec8c14cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec94239700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c157be0 0x7fec8c152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec94239700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c1536b0 con 0x7fec8c0b17c0 2023-08-27T11:12:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec94239700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c14d870 con 0x7fec8c06ced0 2023-08-27T11:12:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec94239700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0b3f50 con 0x7fec8c157be0 2023-08-27T11:12:32.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec917d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec8c0b17c0 0x7fec8c14cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec917d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec8c0b17c0 0x7fec8c14cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51834/0 (socket says 172.21.15.45:51834) 2023-08-27T11:12:32.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec917d4700 1 -- 172.21.15.45:0/1790259193 learned_addr learned my addr 172.21.15.45:0/1790259193 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:32.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec927d6700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c157be0 0x7fec8c152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec91fd5700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec8c06ced0 0x7fec8c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec917d4700 1 -- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c157be0 msgr2=0x7fec8c152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec917d4700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c157be0 0x7fec8c152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec917d4700 1 -- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec8c06ced0 msgr2=0x7fec8c0b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec917d4700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec8c06ced0 0x7fec8c0b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec917d4700 1 -- 172.21.15.45:0/1790259193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880094a0 con 0x7fec8c0b17c0 2023-08-27T11:12:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec927d6700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c157be0 0x7fec8c152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec917d4700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec8c0b17c0 0x7fec8c14cfa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fec88000c00 tx=0x7fec88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec7e7fc700 1 -- 172.21.15.45:0/1790259193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88011630 con 0x7fec8c0b17c0 2023-08-27T11:12:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec7e7fc700 1 -- 172.21.15.45:0/1790259193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8801f070 con 0x7fec8c0b17c0 2023-08-27T11:12:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8c0b41f0 con 0x7fec8c0b17c0 2023-08-27T11:12:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec7e7fc700 1 -- 172.21.15.45:0/1790259193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880119f0 con 0x7fec8c0b17c0 2023-08-27T11:12:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.394+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c0b47b0 con 0x7fec8c0b17c0 2023-08-27T11:12:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.395+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c072690 con 0x7fec8c0b17c0 2023-08-27T11:12:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.396+0000 7fec7e7fc700 1 -- 172.21.15.45:0/1790259193 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fec88011b50 con 0x7fec8c0b17c0 2023-08-27T11:12:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.396+0000 7fec7e7fc700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec7006a0e0 0x7fec7006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.396+0000 7fec7e7fc700 1 -- 172.21.15.45:0/1790259193 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fec88021070 con 0x7fec8c0b17c0 2023-08-27T11:12:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.397+0000 7fec91fd5700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec7006a0e0 0x7fec7006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.397+0000 7fec91fd5700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec7006a0e0 0x7fec7006c5a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec80007fd0 tx=0x7fec80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.401+0000 7fec7e7fc700 1 -- 172.21.15.45:0/1790259193 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fec880bc990 con 0x7fec8c0b17c0 2023-08-27T11:12:32.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.648+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fec8c0cbbb0 con 0x7fec8c0b17c0 2023-08-27T11:12:32.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.983+0000 7fec7e7fc700 1 -- 172.21.15.45:0/1790259193 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fec88055600 con 0x7fec8c0b17c0 2023-08-27T11:12:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec7006a0e0 msgr2=0x7fec7006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec7006a0e0 0x7fec7006c5a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec80007fd0 tx=0x7fec80009040 comp rx=0 tx=0).stop 2023-08-27T11:12:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec8c0b17c0 msgr2=0x7fec8c14cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec8c0b17c0 0x7fec8c14cfa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fec88000c00 tx=0x7fec88004750 comp rx=0 tx=0).stop 2023-08-27T11:12:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 shutdown_connections 2023-08-27T11:12:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec7006a0e0 0x7fec7006c5a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec8c157be0 0x7fec8c152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec8c06ced0 0x7fec8c0b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 --2- 172.21.15.45:0/1790259193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec8c0b17c0 0x7fec8c14cfa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 >> 172.21.15.45:0/1790259193 conn(0x7fec8c0cc7c0 msgr2=0x7fec8c071570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 shutdown_connections 2023-08-27T11:12:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:32.986+0000 7fec94239700 1 -- 172.21.15.45:0/1790259193 wait complete. 2023-08-27T11:12:33.000 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-08-27T11:12:33.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:12:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.129+0000 7f756ff28700 1 Processor -- start 2023-08-27T11:12:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.129+0000 7f756ff28700 1 -- start start 2023-08-27T11:12:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756ff28700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f756815d340 0x7f7568160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756ff28700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 0x7f756806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756ff28700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756806b160 con 0x7f75680cdd00 2023-08-27T11:12:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756ff28700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7568072010 con 0x7f756815d340 2023-08-27T11:12:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756ff28700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75680af590 con 0x7f75680cd880 2023-08-27T11:12:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756dcc4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f756815d340 0x7f7568160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756e4c5700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f75680cdd00 0x7f75680b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38620/0 (socket says 172.21.15.45:38620) 2023-08-27T11:12:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756e4c5700 1 -- 172.21.15.45:0/3774489960 learned_addr learned my addr 172.21.15.45:0/3774489960 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756d4c3700 1 --2- 172.21.15.45:0/3774489960 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 0x7f756806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756dcc4700 1 -- 172.21.15.45:0/3774489960 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 msgr2=0x7f756806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756dcc4700 1 --2- 172.21.15.45:0/3774489960 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 0x7f756806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756dcc4700 1 -- 172.21.15.45:0/3774489960 >> v1:172.21.15.45:6789/0 conn(0x7f75680cdd00 legacy=0x7f75680b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756dcc4700 1 -- 172.21.15.45:0/3774489960 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680af2b0 con 0x7f756815d340 2023-08-27T11:12:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.130+0000 7f756dcc4700 1 --2- 172.21.15.45:0/3774489960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 0x7f7568160d00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7558009840 tx=0x7f7558009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c8b85fac607fe7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ccc2700 1 -- 172.21.15.45:0/3774489960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558011830 con 0x7f756815d340 2023-08-27T11:12:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ccc2700 1 -- 172.21.15.45:0/3774489960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7558011990 con 0x7f756815d340 2023-08-27T11:12:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ccc2700 1 -- 172.21.15.45:0/3774489960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558011d20 con 0x7f756815d340 2023-08-27T11:12:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ff28700 1 -- 172.21.15.45:0/3774489960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 msgr2=0x7f7568160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ff28700 1 --2- 172.21.15.45:0/3774489960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 0x7f7568160d00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7558009840 tx=0x7f7558009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ff28700 1 -- 172.21.15.45:0/3774489960 shutdown_connections 2023-08-27T11:12:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ff28700 1 --2- 172.21.15.45:0/3774489960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 0x7f7568160d00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ff28700 1 --2- 172.21.15.45:0/3774489960 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 0x7f756806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ff28700 1 -- 172.21.15.45:0/3774489960 >> 172.21.15.45:0/3774489960 conn(0x7f75680cc7c0 msgr2=0x7f75680ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ff28700 1 -- 172.21.15.45:0/3774489960 shutdown_connections 2023-08-27T11:12:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.131+0000 7f756ff28700 1 -- 172.21.15.45:0/3774489960 wait complete. 2023-08-27T11:12:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.132+0000 7f756ff28700 1 Processor -- start 2023-08-27T11:12:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756ff28700 1 -- start start 2023-08-27T11:12:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756ff28700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 0x7f75680caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756ff28700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75680cdd00 0x7f7568158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:33.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756ff28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 0x7f7568157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756ff28700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75680c6e80 con 0x7f756815d340 2023-08-27T11:12:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756ff28700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7568158780 con 0x7f75680cdd00 2023-08-27T11:12:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756ff28700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75680ca110 con 0x7f75680cd880 2023-08-27T11:12:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756e4c5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 0x7f7568157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756e4c5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 0x7f7568157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51850/0 (socket says 172.21.15.45:51850) 2023-08-27T11:12:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756e4c5700 1 -- 172.21.15.45:0/2167016508 learned_addr learned my addr 172.21.15.45:0/2167016508 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756dcc4700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 0x7f75680caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.133+0000 7f756d4c3700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75680cdd00 0x7f7568158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:33.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f756e4c5700 1 -- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 msgr2=0x7f75680caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f756e4c5700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 0x7f75680caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f756e4c5700 1 -- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75680cdd00 msgr2=0x7f7568158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f756e4c5700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75680cdd00 0x7f7568158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f756e4c5700 1 -- 172.21.15.45:0/2167016508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580094a0 con 0x7f756815d340 2023-08-27T11:12:33.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f756e4c5700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 0x7f7568157710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f756000caa0 tx=0x7f756000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:33.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f755e7fc700 1 -- 172.21.15.45:0/2167016508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756000a9e0 con 0x7f756815d340 2023-08-27T11:12:33.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f755e7fc700 1 -- 172.21.15.45:0/2167016508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756000ab40 con 0x7f756815d340 2023-08-27T11:12:33.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75680ca3f0 con 0x7f756815d340 2023-08-27T11:12:33.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f755e7fc700 1 -- 172.21.15.45:0/2167016508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560012400 con 0x7f756815d340 2023-08-27T11:12:33.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.134+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75680ca9b0 con 0x7f756815d340 2023-08-27T11:12:33.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.135+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75680caba0 con 0x7f756815d340 2023-08-27T11:12:33.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.136+0000 7f755e7fc700 1 -- 172.21.15.45:0/2167016508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f756001b3b0 con 0x7f756815d340 2023-08-27T11:12:33.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.136+0000 7f755e7fc700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f754c06a0e0 0x7f754c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:33.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.136+0000 7f755e7fc700 1 -- 172.21.15.45:0/2167016508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f75600881c0 con 0x7f756815d340 2023-08-27T11:12:33.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.136+0000 7f756dcc4700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f754c06a0e0 0x7f754c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:33.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.137+0000 7f756dcc4700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f754c06a0e0 0x7f754c06c5a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f75580090b0 tx=0x7f755801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:33.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.140+0000 7f755e7fc700 1 -- 172.21.15.45:0/2167016508 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f75600537c0 con 0x7f756815d340 2023-08-27T11:12:33.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.386+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f75680734a0 con 0x7f756815d340 2023-08-27T11:12:33.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:12:33.396 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:12:33.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.988+0000 7f755e7fc700 1 -- 172.21.15.45:0/2167016508 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7560056de0 con 0x7f756815d340 2023-08-27T11:12:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f754c06a0e0 msgr2=0x7f754c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f754c06a0e0 0x7f754c06c5a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f75580090b0 tx=0x7f755801c040 comp rx=0 tx=0).stop 2023-08-27T11:12:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 msgr2=0x7f7568157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 0x7f7568157710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f756000caa0 tx=0x7f756000cdb0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 shutdown_connections 2023-08-27T11:12:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f756815d340 0x7f7568157710 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f754c06a0e0 0x7f754c06c5a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75680cd880 0x7f75680caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 --2- 172.21.15.45:0/2167016508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75680cdd00 0x7f7568158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 >> 172.21.15.45:0/2167016508 conn(0x7f75680cc7c0 msgr2=0x7f756815d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 shutdown_connections 2023-08-27T11:12:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:33.991+0000 7f756ff28700 1 -- 172.21.15.45:0/2167016508 wait complete. 2023-08-27T11:12:34.007 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:12:34.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:12:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.138+0000 7f3d9f672700 1 Processor -- start 2023-08-27T11:12:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.138+0000 7f3d9f672700 1 -- start start 2023-08-27T11:12:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9f672700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d9815a240 0x7f3d980cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9f672700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3d980cbe70 0x7f3d980ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9f672700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9806d690 con 0x7f3d980ce9d0 2023-08-27T11:12:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9f672700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d980cede0 con 0x7f3d9815a240 2023-08-27T11:12:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9f672700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d980cc440 con 0x7f3d980cbe70 2023-08-27T11:12:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9d40e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d9815a240 0x7f3d980cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9d40e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d9815a240 0x7f3d980cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51854/0 (socket says 172.21.15.45:51854) 2023-08-27T11:12:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9d40e700 1 -- 172.21.15.45:0/2709031853 learned_addr learned my addr 172.21.15.45:0/2709031853 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9cc0d700 1 --2- 172.21.15.45:0/2709031853 >> v2:172.21.15.135:3300/0 conn(0x7f3d980cbe70 0x7f3d980ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9d40e700 1 -- 172.21.15.45:0/2709031853 >> v2:172.21.15.135:3300/0 conn(0x7f3d980cbe70 msgr2=0x7f3d980ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9d40e700 1 --2- 172.21.15.45:0/2709031853 >> v2:172.21.15.135:3300/0 conn(0x7f3d980cbe70 0x7f3d980ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9d40e700 1 -- 172.21.15.45:0/2709031853 >> v1:172.21.15.135:6789/0 conn(0x7f3d980ce9d0 legacy=0x7f3d98070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9d40e700 1 -- 172.21.15.45:0/2709031853 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9806d040 con 0x7f3d9815a240 2023-08-27T11:12:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.139+0000 7f3d9cc0d700 1 --2- 172.21.15.45:0/2709031853 >> v2:172.21.15.135:3300/0 conn(0x7f3d980cbe70 0x7f3d980ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d9d40e700 1 --2- 172.21.15.45:0/2709031853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9815a240 0x7f3d980cae80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d98074910 tx=0x7f3d94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df36b5df84fec9c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d8ffff700 1 -- 172.21.15.45:0/2709031853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400b870 con 0x7f3d9815a240 2023-08-27T11:12:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d8ffff700 1 -- 172.21.15.45:0/2709031853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9400b9d0 con 0x7f3d9815a240 2023-08-27T11:12:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d8ffff700 1 -- 172.21.15.45:0/2709031853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400bd60 con 0x7f3d9815a240 2023-08-27T11:12:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d9f672700 1 -- 172.21.15.45:0/2709031853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9815a240 msgr2=0x7f3d980cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d9f672700 1 --2- 172.21.15.45:0/2709031853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9815a240 0x7f3d980cae80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d98074910 tx=0x7f3d94009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d9f672700 1 -- 172.21.15.45:0/2709031853 shutdown_connections 2023-08-27T11:12:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d9f672700 1 --2- 172.21.15.45:0/2709031853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9815a240 0x7f3d980cae80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d9f672700 1 --2- 172.21.15.45:0/2709031853 >> v2:172.21.15.135:3300/0 conn(0x7f3d980cbe70 0x7f3d980ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d9f672700 1 -- 172.21.15.45:0/2709031853 >> 172.21.15.45:0/2709031853 conn(0x7f3d980cfe50 msgr2=0x7f3d980d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d9f672700 1 -- 172.21.15.45:0/2709031853 shutdown_connections 2023-08-27T11:12:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.140+0000 7f3d9f672700 1 -- 172.21.15.45:0/2709031853 wait complete. 2023-08-27T11:12:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.142+0000 7f3d9f672700 1 Processor -- start 2023-08-27T11:12:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.142+0000 7f3d9f672700 1 -- start start 2023-08-27T11:12:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9f672700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d980cbe70 0x7f3d981555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9f672700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d980ce9d0 0x7f3d9815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9f672700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d9815a240 0x7f3d980b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9f672700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980c8520 con 0x7f3d980ce9d0 2023-08-27T11:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9f672700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980c8800 con 0x7f3d980cbe70 2023-08-27T11:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9f672700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980c8ae0 con 0x7f3d9815a240 2023-08-27T11:12:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9cc0d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d980ce9d0 0x7f3d9815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9cc0d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d980ce9d0 0x7f3d9815a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51862/0 (socket says 172.21.15.45:51862) 2023-08-27T11:12:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9cc0d700 1 -- 172.21.15.45:0/1837938755 learned_addr learned my addr 172.21.15.45:0/1837938755 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9dc0f700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d9815a240 0x7f3d980b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9d40e700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d980cbe70 0x7f3d981555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9cc0d700 1 -- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d9815a240 msgr2=0x7f3d980b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9cc0d700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d9815a240 0x7f3d980b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9cc0d700 1 -- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d980cbe70 msgr2=0x7f3d981555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9cc0d700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d980cbe70 0x7f3d981555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9cc0d700 1 -- 172.21.15.45:0/1837938755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d94009550 con 0x7f3d980ce9d0 2023-08-27T11:12:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.143+0000 7f3d9dc0f700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d9815a240 0x7f3d980b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.144+0000 7f3d9cc0d700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d980ce9d0 0x7f3d9815a620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800e810 tx=0x7f3d8800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.144+0000 7f3d8dffb700 1 -- 172.21.15.45:0/1837938755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88004230 con 0x7f3d980ce9d0 2023-08-27T11:12:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.144+0000 7f3d8dffb700 1 -- 172.21.15.45:0/1837938755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d88004390 con 0x7f3d980ce9d0 2023-08-27T11:12:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.144+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d980c8dc0 con 0x7f3d980ce9d0 2023-08-27T11:12:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.144+0000 7f3d8dffb700 1 -- 172.21.15.45:0/1837938755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d880036a0 con 0x7f3d980ce9d0 2023-08-27T11:12:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.144+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9815cb70 con 0x7f3d980ce9d0 2023-08-27T11:12:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.145+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9815cd90 con 0x7f3d980ce9d0 2023-08-27T11:12:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.145+0000 7f3d8dffb700 1 -- 172.21.15.45:0/1837938755 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3d88003800 con 0x7f3d980ce9d0 2023-08-27T11:12:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.146+0000 7f3d8dffb700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d7c06a0e0 0x7f3d7c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.146+0000 7f3d8dffb700 1 -- 172.21.15.45:0/1837938755 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3d880882a0 con 0x7f3d980ce9d0 2023-08-27T11:12:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.146+0000 7f3d9d40e700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d7c06a0e0 0x7f3d7c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.147+0000 7f3d9d40e700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d7c06a0e0 0x7f3d7c06c5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d94014040 tx=0x7f3d94008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.150+0000 7f3d8dffb700 1 -- 172.21.15.45:0/1837938755 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3d88053730 con 0x7f3d980ce9d0 2023-08-27T11:12:34.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.392+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d98072810 con 0x7f3d980ce9d0 2023-08-27T11:12:34.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.392+0000 7f3d8dffb700 1 -- 172.21.15.45:0/1837938755 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f3d88056d50 con 0x7f3d980ce9d0 2023-08-27T11:12:34.395 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:34.395 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:33.983169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:12:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d7c06a0e0 msgr2=0x7f3d7c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d7c06a0e0 0x7f3d7c06c5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d94014040 tx=0x7f3d94008030 comp rx=0 tx=0).stop 2023-08-27T11:12:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d980ce9d0 msgr2=0x7f3d9815a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d980ce9d0 0x7f3d9815a620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800e810 tx=0x7f3d8800ebd0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 shutdown_connections 2023-08-27T11:12:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d7c06a0e0 0x7f3d7c06c5a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d9815a240 0x7f3d980b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d980cbe70 0x7f3d981555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 --2- 172.21.15.45:0/1837938755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d980ce9d0 0x7f3d9815a620 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.394+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 >> 172.21.15.45:0/1837938755 conn(0x7f3d980cfe50 msgr2=0x7f3d98158040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.395+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 shutdown_connections 2023-08-27T11:12:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.395+0000 7f3d9f672700 1 -- 172.21.15.45:0/1837938755 wait complete. 2023-08-27T11:12:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-08-27T11:12:34.409 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:34.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.532+0000 7f34a0605700 1 Processor -- start 2023-08-27T11:12:34.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.532+0000 7f34a0605700 1 -- start start 2023-08-27T11:12:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.533+0000 7f34a0605700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f349806cec0 0x7f349806e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.533+0000 7f34a0605700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806e7b0 0x7f34980ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.533+0000 7f34a0605700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349815dad0 con 0x7f349815b070 2023-08-27T11:12:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.533+0000 7f34a0605700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349815d8e0 con 0x7f349806cec0 2023-08-27T11:12:34.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.533+0000 7f34a0605700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349806ec20 con 0x7f349806e7b0 2023-08-27T11:12:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.533+0000 7f349eba2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806e7b0 0x7f34980ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.533+0000 7f349dba0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f349806cec0 0x7f349806e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.533+0000 7f349eba2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806e7b0 0x7f34980ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:49782/0 (socket says 172.21.15.45:49782) 2023-08-27T11:12:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.533+0000 7f349eba2700 1 -- 172.21.15.45:0/2644065115 learned_addr learned my addr 172.21.15.45:0/2644065115 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f349eba2700 1 -- 172.21.15.45:0/2644065115 >> v2:172.21.15.135:3300/0 conn(0x7f349806cec0 msgr2=0x7f349806e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f349eba2700 1 --2- 172.21.15.45:0/2644065115 >> v2:172.21.15.135:3300/0 conn(0x7f349806cec0 0x7f349806e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f349eba2700 1 -- 172.21.15.45:0/2644065115 >> v1:172.21.15.135:6789/0 conn(0x7f349815b070 legacy=0x7f34980cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f349eba2700 1 -- 172.21.15.45:0/2644065115 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349815ddb0 con 0x7f349806e7b0 2023-08-27T11:12:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f349dba0700 1 --2- 172.21.15.45:0/2644065115 >> v2:172.21.15.135:3300/0 conn(0x7f349806cec0 0x7f349806e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f349eba2700 1 --2- 172.21.15.45:0/2644065115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806e7b0 0x7f34980ae8d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3490009e50 tx=0x7f349000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fb32960f7f853dc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f349d39f700 1 -- 172.21.15.45:0/2644065115 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000e070 con 0x7f349806e7b0 2023-08-27T11:12:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f349d39f700 1 -- 172.21.15.45:0/2644065115 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349000b910 con 0x7f349806e7b0 2023-08-27T11:12:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f349d39f700 1 -- 172.21.15.45:0/2644065115 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000bcc0 con 0x7f349806e7b0 2023-08-27T11:12:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f34a0605700 1 -- 172.21.15.45:0/2644065115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806e7b0 msgr2=0x7f34980ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.534+0000 7f34a0605700 1 --2- 172.21.15.45:0/2644065115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806e7b0 0x7f34980ae8d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3490009e50 tx=0x7f349000b330 comp rx=0 tx=0).stop 2023-08-27T11:12:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.535+0000 7f34a0605700 1 -- 172.21.15.45:0/2644065115 shutdown_connections 2023-08-27T11:12:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.535+0000 7f34a0605700 1 --2- 172.21.15.45:0/2644065115 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806e7b0 0x7f34980ae8d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.535+0000 7f34a0605700 1 --2- 172.21.15.45:0/2644065115 >> v2:172.21.15.135:3300/0 conn(0x7f349806cec0 0x7f349806e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.535+0000 7f34a0605700 1 -- 172.21.15.45:0/2644065115 >> 172.21.15.45:0/2644065115 conn(0x7f34980cfe50 msgr2=0x7f34980d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.535+0000 7f34a0605700 1 -- 172.21.15.45:0/2644065115 shutdown_connections 2023-08-27T11:12:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.535+0000 7f34a0605700 1 -- 172.21.15.45:0/2644065115 wait complete. 2023-08-27T11:12:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.536+0000 7f34a0605700 1 Processor -- start 2023-08-27T11:12:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.536+0000 7f34a0605700 1 -- start start 2023-08-27T11:12:34.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f34a0605700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806cec0 0x7f34980cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f34a0605700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f349806e7b0 0x7f34980c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f34a0605700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f349815b070 0x7f3498158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f34a0605700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349815c160 con 0x7f349815b070 2023-08-27T11:12:34.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f34a0605700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34980b3c40 con 0x7f349806e7b0 2023-08-27T11:12:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f34a0605700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34980c7e70 con 0x7f349806cec0 2023-08-27T11:12:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349eba2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f349815b070 0x7f3498158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349eba2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f349815b070 0x7f3498158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51878/0 (socket says 172.21.15.45:51878) 2023-08-27T11:12:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349eba2700 1 -- 172.21.15.45:0/1696818640 learned_addr learned my addr 172.21.15.45:0/1696818640 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349dba0700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f349806e7b0 0x7f34980c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349e3a1700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806cec0 0x7f34980cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349eba2700 1 -- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806cec0 msgr2=0x7f34980cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349eba2700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806cec0 0x7f34980cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349eba2700 1 -- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f349806e7b0 msgr2=0x7f34980c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349eba2700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f349806e7b0 0x7f34980c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349eba2700 1 -- 172.21.15.45:0/1696818640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34940094a0 con 0x7f349815b070 2023-08-27T11:12:34.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f349eba2700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f349815b070 0x7f3498158370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3490004270 tx=0x7f3490004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:34.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f348effd700 1 -- 172.21.15.45:0/1696818640 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34900099b0 con 0x7f349815b070 2023-08-27T11:12:34.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f348effd700 1 -- 172.21.15.45:0/1696818640 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34900092e0 con 0x7f349815b070 2023-08-27T11:12:34.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3490009b30 con 0x7f349815b070 2023-08-27T11:12:34.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f348effd700 1 -- 172.21.15.45:0/1696818640 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349001c660 con 0x7f349815b070 2023-08-27T11:12:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.537+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34980c8420 con 0x7f349815b070 2023-08-27T11:12:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.538+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349815c2d0 con 0x7f349815b070 2023-08-27T11:12:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.539+0000 7f348effd700 1 -- 172.21.15.45:0/1696818640 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f349001c7c0 con 0x7f349815b070 2023-08-27T11:12:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.539+0000 7f348effd700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f347c06a0e0 0x7f347c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.540+0000 7f348effd700 1 -- 172.21.15.45:0/1696818640 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f349008a600 con 0x7f349815b070 2023-08-27T11:12:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.540+0000 7f349e3a1700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f347c06a0e0 0x7f347c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.540+0000 7f349e3a1700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f347c06a0e0 0x7f347c06c5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3488000f60 tx=0x7f3488009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:34.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.544+0000 7f348effd700 1 -- 172.21.15.45:0/1696818640 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3490055a10 con 0x7f349815b070 2023-08-27T11:12:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.744+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f349815e340 con 0x7f349815b070 2023-08-27T11:12:34.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.745+0000 7f348effd700 1 -- 172.21.15.45:0/1696818640 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f3490059030 con 0x7f349815b070 2023-08-27T11:12:34.747 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:34.747 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f347c06a0e0 msgr2=0x7f347c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f347c06a0e0 0x7f347c06c5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3488000f60 tx=0x7f3488009040 comp rx=0 tx=0).stop 2023-08-27T11:12:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f349815b070 msgr2=0x7f3498158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f349815b070 0x7f3498158370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3490004270 tx=0x7f3490004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 shutdown_connections 2023-08-27T11:12:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f349815b070 0x7f3498158370 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f347c06a0e0 0x7f347c06c5a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f349806cec0 0x7f34980cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 --2- 172.21.15.45:0/1696818640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f349806e7b0 0x7f34980c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 >> 172.21.15.45:0/1696818640 conn(0x7f34980cfe50 msgr2=0x7f34980cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 shutdown_connections 2023-08-27T11:12:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.747+0000 7f34a0605700 1 -- 172.21.15.45:0/1696818640 wait complete. 2023-08-27T11:12:34.764 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:12:34.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:12:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.888+0000 7f39b1e44700 1 Processor -- start 2023-08-27T11:12:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.888+0000 7f39b1e44700 1 -- start start 2023-08-27T11:12:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39b1e44700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f39ac155660 0x7f39ac06e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39b1e44700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ac0cbf00 con 0x7f39ac06a990 2023-08-27T11:12:34.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39b1e44700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ac0cc180 con 0x7f39ac070e80 2023-08-27T11:12:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39b1e44700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac0caa60 con 0x7f39ac155660 2023-08-27T11:12:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39abfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f39ac06a990 0x7f39ac06cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38660/0 (socket says 172.21.15.45:38660) 2023-08-27T11:12:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39abfff700 1 -- 172.21.15.45:0/2792852392 learned_addr learned my addr 172.21.15.45:0/2792852392 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39ab7fe700 1 --2- 172.21.15.45:0/2792852392 >> v2:172.21.15.135:3300/0 conn(0x7f39ac155660 0x7f39ac06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2792852392 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361144725 0 0) 0x7f39ac0cbf00 con 0x7f39ac06a990 2023-08-27T11:12:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2792852392 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39940035b0 con 0x7f39ac06a990 2023-08-27T11:12:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2792852392 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1508301975 0 0) 0x7f39940035b0 con 0x7f39ac06a990 2023-08-27T11:12:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39ab7fe700 1 -- 172.21.15.45:0/2792852392 >> v1:172.21.15.135:6789/0 conn(0x7f39ac070e80 legacy=0x7f39ac06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39ab7fe700 1 -- 172.21.15.45:0/2792852392 >> v1:172.21.15.45:6789/0 conn(0x7f39ac06a990 legacy=0x7f39ac06cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.889+0000 7f39ab7fe700 1 -- 172.21.15.45:0/2792852392 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac0ca4a0 con 0x7f39ac155660 2023-08-27T11:12:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39ab7fe700 1 --2- 172.21.15.45:0/2792852392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac155660 0x7f39ac06e850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f399c009840 tx=0x7f399c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e88c970b2d5387 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2792852392 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40019f0 con 0x7f39ac155660 2023-08-27T11:12:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2792852392 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c004030 con 0x7f39ac155660 2023-08-27T11:12:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2792852392 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c0043c0 con 0x7f39ac155660 2023-08-27T11:12:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39b1e44700 1 -- 172.21.15.45:0/2792852392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac155660 msgr2=0x7f39ac06e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39b1e44700 1 --2- 172.21.15.45:0/2792852392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac155660 0x7f39ac06e850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f399c009840 tx=0x7f399c009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39b1e44700 1 -- 172.21.15.45:0/2792852392 shutdown_connections 2023-08-27T11:12:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39b1e44700 1 --2- 172.21.15.45:0/2792852392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac155660 0x7f39ac06e850 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39b1e44700 1 -- 172.21.15.45:0/2792852392 >> 172.21.15.45:0/2792852392 conn(0x7f39ac0cfe50 msgr2=0x7f39ac0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39b1e44700 1 -- 172.21.15.45:0/2792852392 shutdown_connections 2023-08-27T11:12:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.890+0000 7f39b1e44700 1 -- 172.21.15.45:0/2792852392 wait complete. 2023-08-27T11:12:34.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.892+0000 7f39b1e44700 1 Processor -- start 2023-08-27T11:12:34.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.892+0000 7f39b1e44700 1 -- start start 2023-08-27T11:12:34.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.892+0000 7f39b1e44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac06a990 0x7f39ac1f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.892+0000 7f39b1e44700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac070e80 0x7f39ac1f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.892+0000 7f39b1e44700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac155660 0x7f39ac157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.892+0000 7f39b1e44700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0a9400 con 0x7f39ac06a990 2023-08-27T11:12:34.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.892+0000 7f39b1e44700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac1f3760 con 0x7f39ac155660 2023-08-27T11:12:34.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39b1e44700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0a84a0 con 0x7f39ac070e80 2023-08-27T11:12:34.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39ab7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac06a990 0x7f39ac1f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39ab7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac06a990 0x7f39ac1f1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51886/0 (socket says 172.21.15.45:51886) 2023-08-27T11:12:34.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39ab7fe700 1 -- 172.21.15.45:0/881851461 learned_addr learned my addr 172.21.15.45:0/881851461 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:34.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39abfff700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac155660 0x7f39ac157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39aaffd700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac070e80 0x7f39ac1f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39ab7fe700 1 -- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac070e80 msgr2=0x7f39ac1f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39ab7fe700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac070e80 0x7f39ac1f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39ab7fe700 1 -- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac155660 msgr2=0x7f39ac157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:34.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39ab7fe700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac155660 0x7f39ac157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:34.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39ab7fe700 1 -- 172.21.15.45:0/881851461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0094a0 con 0x7f39ac06a990 2023-08-27T11:12:34.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39ab7fe700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac06a990 0x7f39ac1f1470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f399c00a010 tx=0x7f399c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:34.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f3993fff700 1 -- 172.21.15.45:0/881851461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c003e30 con 0x7f39ac06a990 2023-08-27T11:12:34.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f3993fff700 1 -- 172.21.15.45:0/881851461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c01f070 con 0x7f39ac06a990 2023-08-27T11:12:34.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac0a8780 con 0x7f39ac06a990 2023-08-27T11:12:34.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f3993fff700 1 -- 172.21.15.45:0/881851461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c011850 con 0x7f39ac06a990 2023-08-27T11:12:34.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.893+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac0a8d40 con 0x7f39ac06a990 2023-08-27T11:12:34.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.894+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac072810 con 0x7f39ac06a990 2023-08-27T11:12:34.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.895+0000 7f3993fff700 1 -- 172.21.15.45:0/881851461 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f399c021030 con 0x7f39ac06a990 2023-08-27T11:12:34.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.895+0000 7f3993fff700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f399806a0e0 0x7f399806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:34.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.895+0000 7f3993fff700 1 -- 172.21.15.45:0/881851461 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f399c089ca0 con 0x7f39ac06a990 2023-08-27T11:12:34.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.895+0000 7f39aaffd700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f399806a0e0 0x7f399806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:34.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.896+0000 7f39aaffd700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f399806a0e0 0x7f399806c5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0c63b0 tx=0x7f39a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:34.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:34.900+0000 7f3993fff700 1 -- 172.21.15.45:0/881851461 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f399c055130 con 0x7f39ac06a990 2023-08-27T11:12:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.097+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 --> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39ac0cc420 con 0x7f399806a0e0 2023-08-27T11:12:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.108+0000 7f3993fff700 1 -- 172.21.15.45:0/881851461 <== mgr.4100 v2:172.21.15.135:6832/4120550306 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367365 (secure 0 0 0) 0x7f39a0001490 con 0x7f399806a0e0 2023-08-27T11:12:35.114 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:35.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f399806a0e0 msgr2=0x7f399806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f399806a0e0 0x7f399806c5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0c63b0 tx=0x7f39a0009040 comp rx=0 tx=0).stop 2023-08-27T11:12:35.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac06a990 msgr2=0x7f39ac1f1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac06a990 0x7f39ac1f1470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f399c00a010 tx=0x7f399c004750 comp rx=0 tx=0).stop 2023-08-27T11:12:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 shutdown_connections 2023-08-27T11:12:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac06a990 0x7f39ac1f1470 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f399806a0e0 0x7f399806c5a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac155660 0x7f39ac157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 --2- 172.21.15.45:0/881851461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac070e80 0x7f39ac1f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 >> 172.21.15.45:0/881851461 conn(0x7f39ac0cfe50 msgr2=0x7f39ac0b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:35.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 shutdown_connections 2023-08-27T11:12:35.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.112+0000 7f39b1e44700 1 -- 172.21.15.45:0/881851461 wait complete. 2023-08-27T11:12:35.120 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:12:35.126 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-08-27T11:12:34.811125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219664,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755056,"statfs":{"total":773094113280,"available":772869177344,"internally_reserved":0,"allocated":3579904,"data_stored":2149224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003565"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932872+0000","last_change":"2023-08-27T11:12:08.913049+0000","last_active":"2023-08-27T11:12:13.932872+0000","last_peered":"2023-08-27T11:12:13.932872+0000","last_clean":"2023-08-27T11:12:13.932872+0000","last_became_active":"2023-08-27T11:12:08.912757+0000","last_became_peered":"2023-08-27T11:12:08.912757+0000","last_unstale":"2023-08-27T11:12:13.932872+0000","last_undegraded":"2023-08-27T11:12:13.932872+0000","last_fullsized":"2023-08-27T11:12:13.932872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:00.165033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967658+0000","last_change":"2023-08-27T11:12:08.918994+0000","last_active":"2023-08-27T11:12:14.967658+0000","last_peered":"2023-08-27T11:12:14.967658+0000","last_clean":"2023-08-27T11:12:14.967658+0000","last_became_active":"2023-08-27T11:12:08.918683+0000","last_became_peered":"2023-08-27T11:12:08.918683+0000","last_unstale":"2023-08-27T11:12:14.967658+0000","last_undegraded":"2023-08-27T11:12:14.967658+0000","last_fullsized":"2023-08-27T11:12:14.967658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:40.545799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934025+0000","last_change":"2023-08-27T11:12:08.911497+0000","last_active":"2023-08-27T11:12:13.934025+0000","last_peered":"2023-08-27T11:12:13.934025+0000","last_clean":"2023-08-27T11:12:13.934025+0000","last_became_active":"2023-08-27T11:12:08.911184+0000","last_became_peered":"2023-08-27T11:12:08.911184+0000","last_unstale":"2023-08-27T11:12:13.934025+0000","last_undegraded":"2023-08-27T11:12:13.934025+0000","last_fullsized":"2023-08-27T11:12:13.934025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:45.042697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934063+0000","last_change":"2023-08-27T11:12:08.914949+0000","last_active":"2023-08-27T11:12:13.934063+0000","last_peered":"2023-08-27T11:12:13.934063+0000","last_clean":"2023-08-27T11:12:13.934063+0000","last_became_active":"2023-08-27T11:12:08.914642+0000","last_became_peered":"2023-08-27T11:12:08.914642+0000","last_unstale":"2023-08-27T11:12:13.934063+0000","last_undegraded":"2023-08-27T11:12:13.934063+0000","last_fullsized":"2023-08-27T11:12:13.934063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:38.888108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726727+0000","last_change":"2023-08-27T11:12:08.929257+0000","last_active":"2023-08-27T11:12:14.726727+0000","last_peered":"2023-08-27T11:12:14.726727+0000","last_clean":"2023-08-27T11:12:14.726727+0000","last_became_active":"2023-08-27T11:12:08.928926+0000","last_became_peered":"2023-08-27T11:12:08.928926+0000","last_unstale":"2023-08-27T11:12:14.726727+0000","last_undegraded":"2023-08-27T11:12:14.726727+0000","last_fullsized":"2023-08-27T11:12:14.726727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:09.445014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934838+0000","last_change":"2023-08-27T11:12:08.916024+0000","last_active":"2023-08-27T11:12:13.934838+0000","last_peered":"2023-08-27T11:12:13.934838+0000","last_clean":"2023-08-27T11:12:13.934838+0000","last_became_active":"2023-08-27T11:12:08.913502+0000","last_became_peered":"2023-08-27T11:12:08.913502+0000","last_unstale":"2023-08-27T11:12:13.934838+0000","last_undegraded":"2023-08-27T11:12:13.934838+0000","last_fullsized":"2023-08-27T11:12:13.934838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:50:51.397518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384498+0000","last_change":"2023-08-27T11:12:08.925080+0000","last_active":"2023-08-27T11:12:14.384498+0000","last_peered":"2023-08-27T11:12:14.384498+0000","last_clean":"2023-08-27T11:12:14.384498+0000","last_became_active":"2023-08-27T11:12:08.924784+0000","last_became_peered":"2023-08-27T11:12:08.924784+0000","last_unstale":"2023-08-27T11:12:14.384498+0000","last_undegraded":"2023-08-27T11:12:14.384498+0000","last_fullsized":"2023-08-27T11:12:14.384498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967188+0000","last_change":"2023-08-27T11:12:08.917218+0000","last_active":"2023-08-27T11:12:14.967188+0000","last_peered":"2023-08-27T11:12:14.967188+0000","last_clean":"2023-08-27T11:12:14.967188+0000","last_became_active":"2023-08-27T11:12:08.916984+0000","last_became_peered":"2023-08-27T11:12:08.916984+0000","last_unstale":"2023-08-27T11:12:14.967188+0000","last_undegraded":"2023-08-27T11:12:14.967188+0000","last_fullsized":"2023-08-27T11:12:14.967188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:02.783147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726887+0000","last_change":"2023-08-27T11:12:08.918120+0000","last_active":"2023-08-27T11:12:14.726887+0000","last_peered":"2023-08-27T11:12:14.726887+0000","last_clean":"2023-08-27T11:12:14.726887+0000","last_became_active":"2023-08-27T11:12:08.917811+0000","last_became_peered":"2023-08-27T11:12:08.917811+0000","last_unstale":"2023-08-27T11:12:14.726887+0000","last_undegraded":"2023-08-27T11:12:14.726887+0000","last_fullsized":"2023-08-27T11:12:14.726887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:16.000827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967968+0000","last_change":"2023-08-27T11:12:08.912850+0000","last_active":"2023-08-27T11:12:14.967968+0000","last_peered":"2023-08-27T11:12:14.967968+0000","last_clean":"2023-08-27T11:12:14.967968+0000","last_became_active":"2023-08-27T11:12:08.912458+0000","last_became_peered":"2023-08-27T11:12:08.912458+0000","last_unstale":"2023-08-27T11:12:14.967968+0000","last_undegraded":"2023-08-27T11:12:14.967968+0000","last_fullsized":"2023-08-27T11:12:14.967968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:49.762265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933954+0000","last_change":"2023-08-27T11:12:08.922288+0000","last_active":"2023-08-27T11:12:13.933954+0000","last_peered":"2023-08-27T11:12:13.933954+0000","last_clean":"2023-08-27T11:12:13.933954+0000","last_became_active":"2023-08-27T11:12:08.921531+0000","last_became_peered":"2023-08-27T11:12:08.921531+0000","last_unstale":"2023-08-27T11:12:13.933954+0000","last_undegraded":"2023-08-27T11:12:13.933954+0000","last_fullsized":"2023-08-27T11:12:13.933954+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:20.157877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933160+0000","last_change":"2023-08-27T11:12:08.911702+0000","last_active":"2023-08-27T11:12:13.933160+0000","last_peered":"2023-08-27T11:12:13.933160+0000","last_clean":"2023-08-27T11:12:13.933160+0000","last_became_active":"2023-08-27T11:12:08.911391+0000","last_became_peered":"2023-08-27T11:12:08.911391+0000","last_unstale":"2023-08-27T11:12:13.933160+0000","last_undegraded":"2023-08-27T11:12:13.933160+0000","last_fullsized":"2023-08-27T11:12:13.933160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:53.589324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932909+0000","last_change":"2023-08-27T11:12:08.914165+0000","last_active":"2023-08-27T11:12:13.932909+0000","last_peered":"2023-08-27T11:12:13.932909+0000","last_clean":"2023-08-27T11:12:13.932909+0000","last_became_active":"2023-08-27T11:12:08.913886+0000","last_became_peered":"2023-08-27T11:12:08.913886+0000","last_unstale":"2023-08-27T11:12:13.932909+0000","last_undegraded":"2023-08-27T11:12:13.932909+0000","last_fullsized":"2023-08-27T11:12:13.932909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:12.238621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967356+0000","last_change":"2023-08-27T11:12:08.911899+0000","last_active":"2023-08-27T11:12:14.967356+0000","last_peered":"2023-08-27T11:12:14.967356+0000","last_clean":"2023-08-27T11:12:14.967356+0000","last_became_active":"2023-08-27T11:12:08.911620+0000","last_became_peered":"2023-08-27T11:12:08.911620+0000","last_unstale":"2023-08-27T11:12:14.967356+0000","last_undegraded":"2023-08-27T11:12:14.967356+0000","last_fullsized":"2023-08-27T11:12:14.967356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:07.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384118+0000","last_change":"2023-08-27T11:12:08.925181+0000","last_active":"2023-08-27T11:12:14.384118+0000","last_peered":"2023-08-27T11:12:14.384118+0000","last_clean":"2023-08-27T11:12:14.384118+0000","last_became_active":"2023-08-27T11:12:08.924909+0000","last_became_peered":"2023-08-27T11:12:08.924909+0000","last_unstale":"2023-08-27T11:12:14.384118+0000","last_undegraded":"2023-08-27T11:12:14.384118+0000","last_fullsized":"2023-08-27T11:12:14.384118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:10.885579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934515+0000","last_change":"2023-08-27T11:12:08.911662+0000","last_active":"2023-08-27T11:12:13.934515+0000","last_peered":"2023-08-27T11:12:13.934515+0000","last_clean":"2023-08-27T11:12:13.934515+0000","last_became_active":"2023-08-27T11:12:08.911345+0000","last_became_peered":"2023-08-27T11:12:08.911345+0000","last_unstale":"2023-08-27T11:12:13.934515+0000","last_undegraded":"2023-08-27T11:12:13.934515+0000","last_fullsized":"2023-08-27T11:12:13.934515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:57.932637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726927+0000","last_change":"2023-08-27T11:12:08.916813+0000","last_active":"2023-08-27T11:12:14.726927+0000","last_peered":"2023-08-27T11:12:14.726927+0000","last_clean":"2023-08-27T11:12:14.726927+0000","last_became_active":"2023-08-27T11:12:08.916478+0000","last_became_peered":"2023-08-27T11:12:08.916478+0000","last_unstale":"2023-08-27T11:12:14.726927+0000","last_undegraded":"2023-08-27T11:12:14.726927+0000","last_fullsized":"2023-08-27T11:12:14.726927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:29.674355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933703+0000","last_change":"2023-08-27T11:12:08.907326+0000","last_active":"2023-08-27T11:12:13.933703+0000","last_peered":"2023-08-27T11:12:13.933703+0000","last_clean":"2023-08-27T11:12:13.933703+0000","last_became_active":"2023-08-27T11:12:08.907047+0000","last_became_peered":"2023-08-27T11:12:08.907047+0000","last_unstale":"2023-08-27T11:12:13.933703+0000","last_undegraded":"2023-08-27T11:12:13.933703+0000","last_fullsized":"2023-08-27T11:12:13.933703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:32:17.983727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932728+0000","last_change":"2023-08-27T11:12:08.919764+0000","last_active":"2023-08-27T11:12:13.932728+0000","last_peered":"2023-08-27T11:12:13.932728+0000","last_clean":"2023-08-27T11:12:13.932728+0000","last_became_active":"2023-08-27T11:12:08.914867+0000","last_became_peered":"2023-08-27T11:12:08.914867+0000","last_unstale":"2023-08-27T11:12:13.932728+0000","last_undegraded":"2023-08-27T11:12:13.932728+0000","last_fullsized":"2023-08-27T11:12:13.932728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:05.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933919+0000","last_change":"2023-08-27T11:12:08.922111+0000","last_active":"2023-08-27T11:12:13.933919+0000","last_peered":"2023-08-27T11:12:13.933919+0000","last_clean":"2023-08-27T11:12:13.933919+0000","last_became_active":"2023-08-27T11:12:08.917591+0000","last_became_peered":"2023-08-27T11:12:08.917591+0000","last_unstale":"2023-08-27T11:12:13.933919+0000","last_undegraded":"2023-08-27T11:12:13.933919+0000","last_fullsized":"2023-08-27T11:12:13.933919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:59.881615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384562+0000","last_change":"2023-08-27T11:12:08.931238+0000","last_active":"2023-08-27T11:12:14.384562+0000","last_peered":"2023-08-27T11:12:14.384562+0000","last_clean":"2023-08-27T11:12:14.384562+0000","last_became_active":"2023-08-27T11:12:08.930972+0000","last_became_peered":"2023-08-27T11:12:08.930972+0000","last_unstale":"2023-08-27T11:12:14.384562+0000","last_undegraded":"2023-08-27T11:12:14.384562+0000","last_fullsized":"2023-08-27T11:12:14.384562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:41.570266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726461+0000","last_change":"2023-08-27T11:12:08.919835+0000","last_active":"2023-08-27T11:12:14.726461+0000","last_peered":"2023-08-27T11:12:14.726461+0000","last_clean":"2023-08-27T11:12:14.726461+0000","last_became_active":"2023-08-27T11:12:08.919439+0000","last_became_peered":"2023-08-27T11:12:08.919439+0000","last_unstale":"2023-08-27T11:12:14.726461+0000","last_undegraded":"2023-08-27T11:12:14.726461+0000","last_fullsized":"2023-08-27T11:12:14.726461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:25:20.526417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982143+0000","last_change":"2023-08-27T11:12:08.913113+0000","last_active":"2023-08-27T11:12:14.982143+0000","last_peered":"2023-08-27T11:12:14.982143+0000","last_clean":"2023-08-27T11:12:14.982143+0000","last_became_active":"2023-08-27T11:12:08.912792+0000","last_became_peered":"2023-08-27T11:12:08.912792+0000","last_unstale":"2023-08-27T11:12:14.982143+0000","last_undegraded":"2023-08-27T11:12:14.982143+0000","last_fullsized":"2023-08-27T11:12:14.982143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:53.041694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384187+0000","last_change":"2023-08-27T11:12:08.926432+0000","last_active":"2023-08-27T11:12:14.384187+0000","last_peered":"2023-08-27T11:12:14.384187+0000","last_clean":"2023-08-27T11:12:14.384187+0000","last_became_active":"2023-08-27T11:12:08.926137+0000","last_became_peered":"2023-08-27T11:12:08.926137+0000","last_unstale":"2023-08-27T11:12:14.384187+0000","last_undegraded":"2023-08-27T11:12:14.384187+0000","last_fullsized":"2023-08-27T11:12:14.384187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:07.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838528+0000","last_change":"2023-08-27T11:12:08.919071+0000","last_active":"2023-08-27T11:12:14.838528+0000","last_peered":"2023-08-27T11:12:14.838528+0000","last_clean":"2023-08-27T11:12:14.838528+0000","last_became_active":"2023-08-27T11:12:08.918772+0000","last_became_peered":"2023-08-27T11:12:08.918772+0000","last_unstale":"2023-08-27T11:12:14.838528+0000","last_undegraded":"2023-08-27T11:12:14.838528+0000","last_fullsized":"2023-08-27T11:12:14.838528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:48.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838629+0000","last_change":"2023-08-27T11:12:08.911583+0000","last_active":"2023-08-27T11:12:14.838629+0000","last_peered":"2023-08-27T11:12:14.838629+0000","last_clean":"2023-08-27T11:12:14.838629+0000","last_became_active":"2023-08-27T11:12:08.911295+0000","last_became_peered":"2023-08-27T11:12:08.911295+0000","last_unstale":"2023-08-27T11:12:14.838629+0000","last_undegraded":"2023-08-27T11:12:14.838629+0000","last_fullsized":"2023-08-27T11:12:14.838629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:22.748117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967776+0000","last_change":"2023-08-27T11:12:08.931465+0000","last_active":"2023-08-27T11:12:14.967776+0000","last_peered":"2023-08-27T11:12:14.967776+0000","last_clean":"2023-08-27T11:12:14.967776+0000","last_became_active":"2023-08-27T11:12:08.930966+0000","last_became_peered":"2023-08-27T11:12:08.930966+0000","last_unstale":"2023-08-27T11:12:14.967776+0000","last_undegraded":"2023-08-27T11:12:14.967776+0000","last_fullsized":"2023-08-27T11:12:14.967776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:41.570266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934397+0000","last_change":"2023-08-27T11:12:12.959219+0000","last_active":"2023-08-27T11:12:13.934397+0000","last_peered":"2023-08-27T11:12:13.934397+0000","last_clean":"2023-08-27T11:12:13.934397+0000","last_became_active":"2023-08-27T11:12:12.958817+0000","last_became_peered":"2023-08-27T11:12:12.958817+0000","last_unstale":"2023-08-27T11:12:13.934397+0000","last_undegraded":"2023-08-27T11:12:13.934397+0000","last_fullsized":"2023-08-27T11:12:13.934397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:44.882336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933845+0000","last_change":"2023-08-27T11:12:07.903534+0000","last_active":"2023-08-27T11:12:13.933845+0000","last_peered":"2023-08-27T11:12:13.933845+0000","last_clean":"2023-08-27T11:12:13.933845+0000","last_became_active":"2023-08-27T11:12:07.903216+0000","last_became_peered":"2023-08-27T11:12:07.903216+0000","last_unstale":"2023-08-27T11:12:13.933845+0000","last_undegraded":"2023-08-27T11:12:13.933845+0000","last_fullsized":"2023-08-27T11:12:13.933845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:35:15.612642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982446+0000","last_change":"2023-08-27T11:12:08.923471+0000","last_active":"2023-08-27T11:12:14.982446+0000","last_peered":"2023-08-27T11:12:14.982446+0000","last_clean":"2023-08-27T11:12:14.982446+0000","last_became_active":"2023-08-27T11:12:08.918459+0000","last_became_peered":"2023-08-27T11:12:08.918459+0000","last_unstale":"2023-08-27T11:12:14.982446+0000","last_undegraded":"2023-08-27T11:12:14.982446+0000","last_fullsized":"2023-08-27T11:12:14.982446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934474+0000","last_change":"2023-08-27T11:12:12.953626+0000","last_active":"2023-08-27T11:12:13.934474+0000","last_peered":"2023-08-27T11:12:13.934474+0000","last_clean":"2023-08-27T11:12:13.934474+0000","last_became_active":"2023-08-27T11:12:12.953231+0000","last_became_peered":"2023-08-27T11:12:12.953231+0000","last_unstale":"2023-08-27T11:12:13.934474+0000","last_undegraded":"2023-08-27T11:12:13.934474+0000","last_fullsized":"2023-08-27T11:12:13.934474+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:57.631242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837663+0000","last_change":"2023-08-27T11:12:07.895584+0000","last_active":"2023-08-27T11:12:14.837663+0000","last_peered":"2023-08-27T11:12:14.837663+0000","last_clean":"2023-08-27T11:12:14.837663+0000","last_became_active":"2023-08-27T11:12:07.895276+0000","last_became_peered":"2023-08-27T11:12:07.895276+0000","last_unstale":"2023-08-27T11:12:14.837663+0000","last_undegraded":"2023-08-27T11:12:14.837663+0000","last_fullsized":"2023-08-27T11:12:14.837663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:44.245501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726897+0000","last_change":"2023-08-27T11:12:08.928505+0000","last_active":"2023-08-27T11:12:14.726897+0000","last_peered":"2023-08-27T11:12:14.726897+0000","last_clean":"2023-08-27T11:12:14.726897+0000","last_became_active":"2023-08-27T11:12:08.926233+0000","last_became_peered":"2023-08-27T11:12:08.926233+0000","last_unstale":"2023-08-27T11:12:14.726897+0000","last_undegraded":"2023-08-27T11:12:14.726897+0000","last_fullsized":"2023-08-27T11:12:14.726897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:17.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727088+0000","last_change":"2023-08-27T11:12:12.962888+0000","last_active":"2023-08-27T11:12:14.727088+0000","last_peered":"2023-08-27T11:12:14.727088+0000","last_clean":"2023-08-27T11:12:14.727088+0000","last_became_active":"2023-08-27T11:12:12.962501+0000","last_became_peered":"2023-08-27T11:12:12.962501+0000","last_unstale":"2023-08-27T11:12:14.727088+0000","last_undegraded":"2023-08-27T11:12:14.727088+0000","last_fullsized":"2023-08-27T11:12:14.727088+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:06.825066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982310+0000","last_change":"2023-08-27T11:12:07.898500+0000","last_active":"2023-08-27T11:12:14.982310+0000","last_peered":"2023-08-27T11:12:14.982310+0000","last_clean":"2023-08-27T11:12:14.982310+0000","last_became_active":"2023-08-27T11:12:07.898197+0000","last_became_peered":"2023-08-27T11:12:07.898197+0000","last_unstale":"2023-08-27T11:12:14.982310+0000","last_undegraded":"2023-08-27T11:12:14.982310+0000","last_fullsized":"2023-08-27T11:12:14.982310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:39.541198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934586+0000","last_change":"2023-08-27T11:12:08.925037+0000","last_active":"2023-08-27T11:12:13.934586+0000","last_peered":"2023-08-27T11:12:13.934586+0000","last_clean":"2023-08-27T11:12:13.934586+0000","last_became_active":"2023-08-27T11:12:08.924740+0000","last_became_peered":"2023-08-27T11:12:08.924740+0000","last_unstale":"2023-08-27T11:12:13.934586+0000","last_undegraded":"2023-08-27T11:12:13.934586+0000","last_fullsized":"2023-08-27T11:12:13.934586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:21:21.628536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933622+0000","last_change":"2023-08-27T11:12:12.956101+0000","last_active":"2023-08-27T11:12:13.933622+0000","last_peered":"2023-08-27T11:12:13.933622+0000","last_clean":"2023-08-27T11:12:13.933622+0000","last_became_active":"2023-08-27T11:12:12.952220+0000","last_became_peered":"2023-08-27T11:12:12.952220+0000","last_unstale":"2023-08-27T11:12:13.933622+0000","last_undegraded":"2023-08-27T11:12:13.933622+0000","last_fullsized":"2023-08-27T11:12:13.933622+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:16.280540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727033+0000","last_change":"2023-08-27T11:12:07.902491+0000","last_active":"2023-08-27T11:12:14.727033+0000","last_peered":"2023-08-27T11:12:14.727033+0000","last_clean":"2023-08-27T11:12:14.727033+0000","last_became_active":"2023-08-27T11:12:07.902083+0000","last_became_peered":"2023-08-27T11:12:07.902083+0000","last_unstale":"2023-08-27T11:12:14.727033+0000","last_undegraded":"2023-08-27T11:12:14.727033+0000","last_fullsized":"2023-08-27T11:12:14.727033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:19.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982268+0000","last_change":"2023-08-27T11:12:08.918017+0000","last_active":"2023-08-27T11:12:14.982268+0000","last_peered":"2023-08-27T11:12:14.982268+0000","last_clean":"2023-08-27T11:12:14.982268+0000","last_became_active":"2023-08-27T11:12:08.917728+0000","last_became_peered":"2023-08-27T11:12:08.917728+0000","last_unstale":"2023-08-27T11:12:14.982268+0000","last_undegraded":"2023-08-27T11:12:14.982268+0000","last_fullsized":"2023-08-27T11:12:14.982268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:51.458617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384060+0000","last_change":"2023-08-27T11:12:12.958663+0000","last_active":"2023-08-27T11:12:14.384060+0000","last_peered":"2023-08-27T11:12:14.384060+0000","last_clean":"2023-08-27T11:12:14.384060+0000","last_became_active":"2023-08-27T11:12:12.958275+0000","last_became_peered":"2023-08-27T11:12:12.958275+0000","last_unstale":"2023-08-27T11:12:14.384060+0000","last_undegraded":"2023-08-27T11:12:14.384060+0000","last_fullsized":"2023-08-27T11:12:14.384060+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:16.222119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967840+0000","last_change":"2023-08-27T11:12:07.909123+0000","last_active":"2023-08-27T11:12:14.967840+0000","last_peered":"2023-08-27T11:12:14.967840+0000","last_clean":"2023-08-27T11:12:14.967840+0000","last_became_active":"2023-08-27T11:12:07.908788+0000","last_became_peered":"2023-08-27T11:12:07.908788+0000","last_unstale":"2023-08-27T11:12:14.967840+0000","last_undegraded":"2023-08-27T11:12:14.967840+0000","last_fullsized":"2023-08-27T11:12:14.967840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:40.858637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982172+0000","last_change":"2023-08-27T11:12:08.927810+0000","last_active":"2023-08-27T11:12:14.982172+0000","last_peered":"2023-08-27T11:12:14.982172+0000","last_clean":"2023-08-27T11:12:14.982172+0000","last_became_active":"2023-08-27T11:12:08.927231+0000","last_became_peered":"2023-08-27T11:12:08.927231+0000","last_unstale":"2023-08-27T11:12:14.982172+0000","last_undegraded":"2023-08-27T11:12:14.982172+0000","last_fullsized":"2023-08-27T11:12:14.982172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:22.748117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982828+0000","last_change":"2023-08-27T11:12:12.954999+0000","last_active":"2023-08-27T11:12:14.982828+0000","last_peered":"2023-08-27T11:12:14.982828+0000","last_clean":"2023-08-27T11:12:14.982828+0000","last_became_active":"2023-08-27T11:12:12.954572+0000","last_became_peered":"2023-08-27T11:12:12.954572+0000","last_unstale":"2023-08-27T11:12:14.982828+0000","last_undegraded":"2023-08-27T11:12:14.982828+0000","last_fullsized":"2023-08-27T11:12:14.982828+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:38.482531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384847+0000","last_change":"2023-08-27T11:12:07.908859+0000","last_active":"2023-08-27T11:12:14.384847+0000","last_peered":"2023-08-27T11:12:14.384847+0000","last_clean":"2023-08-27T11:12:14.384847+0000","last_became_active":"2023-08-27T11:12:07.908605+0000","last_became_peered":"2023-08-27T11:12:07.908605+0000","last_unstale":"2023-08-27T11:12:14.384847+0000","last_undegraded":"2023-08-27T11:12:14.384847+0000","last_fullsized":"2023-08-27T11:12:14.384847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:20.463561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838554+0000","last_change":"2023-08-27T11:12:08.925960+0000","last_active":"2023-08-27T11:12:14.838554+0000","last_peered":"2023-08-27T11:12:14.838554+0000","last_clean":"2023-08-27T11:12:14.838554+0000","last_became_active":"2023-08-27T11:12:08.925641+0000","last_became_peered":"2023-08-27T11:12:08.925641+0000","last_unstale":"2023-08-27T11:12:14.838554+0000","last_undegraded":"2023-08-27T11:12:14.838554+0000","last_fullsized":"2023-08-27T11:12:14.838554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:53.041694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837918+0000","last_change":"2023-08-27T11:12:12.949220+0000","last_active":"2023-08-27T11:12:14.837918+0000","last_peered":"2023-08-27T11:12:14.837918+0000","last_clean":"2023-08-27T11:12:14.837918+0000","last_became_active":"2023-08-27T11:12:12.948926+0000","last_became_peered":"2023-08-27T11:12:12.948926+0000","last_unstale":"2023-08-27T11:12:14.837918+0000","last_undegraded":"2023-08-27T11:12:14.837918+0000","last_fullsized":"2023-08-27T11:12:14.837918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:20.042745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'10","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:24.935040+0000","last_change":"2023-08-27T11:12:07.908014+0000","last_active":"2023-08-27T11:12:24.935040+0000","last_peered":"2023-08-27T11:12:24.935040+0000","last_clean":"2023-08-27T11:12:24.935040+0000","last_became_active":"2023-08-27T11:12:07.907722+0000","last_became_peered":"2023-08-27T11:12:07.907722+0000","last_unstale":"2023-08-27T11:12:24.935040+0000","last_undegraded":"2023-08-27T11:12:24.935040+0000","last_fullsized":"2023-08-27T11:12:24.935040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:11.313074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933433+0000","last_change":"2023-08-27T11:12:08.923838+0000","last_active":"2023-08-27T11:12:13.933433+0000","last_peered":"2023-08-27T11:12:13.933433+0000","last_clean":"2023-08-27T11:12:13.933433+0000","last_became_active":"2023-08-27T11:12:08.919224+0000","last_became_peered":"2023-08-27T11:12:08.919224+0000","last_unstale":"2023-08-27T11:12:13.933433+0000","last_undegraded":"2023-08-27T11:12:13.933433+0000","last_fullsized":"2023-08-27T11:12:13.933433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:49.585442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967969+0000","last_change":"2023-08-27T11:12:12.953888+0000","last_active":"2023-08-27T11:12:14.967969+0000","last_peered":"2023-08-27T11:12:14.967969+0000","last_clean":"2023-08-27T11:12:14.967969+0000","last_became_active":"2023-08-27T11:12:12.953499+0000","last_became_peered":"2023-08-27T11:12:12.953499+0000","last_unstale":"2023-08-27T11:12:14.967969+0000","last_undegraded":"2023-08-27T11:12:14.967969+0000","last_fullsized":"2023-08-27T11:12:14.967969+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:42.319064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384452+0000","last_change":"2023-08-27T11:12:07.907713+0000","last_active":"2023-08-27T11:12:14.384452+0000","last_peered":"2023-08-27T11:12:14.384452+0000","last_clean":"2023-08-27T11:12:14.384452+0000","last_became_active":"2023-08-27T11:12:07.907429+0000","last_became_peered":"2023-08-27T11:12:07.907429+0000","last_unstale":"2023-08-27T11:12:14.384452+0000","last_undegraded":"2023-08-27T11:12:14.384452+0000","last_fullsized":"2023-08-27T11:12:14.384452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:29.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982602+0000","last_change":"2023-08-27T11:12:08.923764+0000","last_active":"2023-08-27T11:12:14.982602+0000","last_peered":"2023-08-27T11:12:14.982602+0000","last_clean":"2023-08-27T11:12:14.982602+0000","last_became_active":"2023-08-27T11:12:08.923333+0000","last_became_peered":"2023-08-27T11:12:08.923333+0000","last_unstale":"2023-08-27T11:12:14.982602+0000","last_undegraded":"2023-08-27T11:12:14.982602+0000","last_fullsized":"2023-08-27T11:12:14.982602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:41.863238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982543+0000","last_change":"2023-08-27T11:12:12.948620+0000","last_active":"2023-08-27T11:12:14.982543+0000","last_peered":"2023-08-27T11:12:14.982543+0000","last_clean":"2023-08-27T11:12:14.982543+0000","last_became_active":"2023-08-27T11:12:12.948161+0000","last_became_peered":"2023-08-27T11:12:12.948161+0000","last_unstale":"2023-08-27T11:12:14.982543+0000","last_undegraded":"2023-08-27T11:12:14.982543+0000","last_fullsized":"2023-08-27T11:12:14.982543+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:25.510081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726415+0000","last_change":"2023-08-27T11:12:07.905444+0000","last_active":"2023-08-27T11:12:14.726415+0000","last_peered":"2023-08-27T11:12:14.726415+0000","last_clean":"2023-08-27T11:12:14.726415+0000","last_became_active":"2023-08-27T11:12:07.905102+0000","last_became_peered":"2023-08-27T11:12:07.905102+0000","last_unstale":"2023-08-27T11:12:14.726415+0000","last_undegraded":"2023-08-27T11:12:14.726415+0000","last_fullsized":"2023-08-27T11:12:14.726415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:59.453384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967752+0000","last_change":"2023-08-27T11:12:08.931630+0000","last_active":"2023-08-27T11:12:14.967752+0000","last_peered":"2023-08-27T11:12:14.967752+0000","last_clean":"2023-08-27T11:12:14.967752+0000","last_became_active":"2023-08-27T11:12:08.931239+0000","last_became_peered":"2023-08-27T11:12:08.931239+0000","last_unstale":"2023-08-27T11:12:14.967752+0000","last_undegraded":"2023-08-27T11:12:14.967752+0000","last_fullsized":"2023-08-27T11:12:14.967752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982960+0000","last_change":"2023-08-27T11:12:12.954525+0000","last_active":"2023-08-27T11:12:14.982960+0000","last_peered":"2023-08-27T11:12:14.982960+0000","last_clean":"2023-08-27T11:12:14.982960+0000","last_became_active":"2023-08-27T11:12:12.954077+0000","last_became_peered":"2023-08-27T11:12:12.954077+0000","last_unstale":"2023-08-27T11:12:14.982960+0000","last_undegraded":"2023-08-27T11:12:14.982960+0000","last_fullsized":"2023-08-27T11:12:14.982960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:07:06.449989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384221+0000","last_change":"2023-08-27T11:12:07.907504+0000","last_active":"2023-08-27T11:12:14.384221+0000","last_peered":"2023-08-27T11:12:14.384221+0000","last_clean":"2023-08-27T11:12:14.384221+0000","last_became_active":"2023-08-27T11:12:07.907206+0000","last_became_peered":"2023-08-27T11:12:07.907206+0000","last_unstale":"2023-08-27T11:12:14.384221+0000","last_undegraded":"2023-08-27T11:12:14.384221+0000","last_fullsized":"2023-08-27T11:12:14.384221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:47.521180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726958+0000","last_change":"2023-08-27T11:12:08.918640+0000","last_active":"2023-08-27T11:12:14.726958+0000","last_peered":"2023-08-27T11:12:14.726958+0000","last_clean":"2023-08-27T11:12:14.726958+0000","last_became_active":"2023-08-27T11:12:08.917846+0000","last_became_peered":"2023-08-27T11:12:08.917846+0000","last_unstale":"2023-08-27T11:12:14.726958+0000","last_undegraded":"2023-08-27T11:12:14.726958+0000","last_fullsized":"2023-08-27T11:12:14.726958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:29:50.086138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967688+0000","last_change":"2023-08-27T11:12:12.962983+0000","last_active":"2023-08-27T11:12:14.967688+0000","last_peered":"2023-08-27T11:12:14.967688+0000","last_clean":"2023-08-27T11:12:14.967688+0000","last_became_active":"2023-08-27T11:12:12.962518+0000","last_became_peered":"2023-08-27T11:12:12.962518+0000","last_unstale":"2023-08-27T11:12:14.967688+0000","last_undegraded":"2023-08-27T11:12:14.967688+0000","last_fullsized":"2023-08-27T11:12:14.967688+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:34.829806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932864+0000","last_change":"2023-08-27T11:12:07.905529+0000","last_active":"2023-08-27T11:12:13.932864+0000","last_peered":"2023-08-27T11:12:13.932864+0000","last_clean":"2023-08-27T11:12:13.932864+0000","last_became_active":"2023-08-27T11:12:07.905203+0000","last_became_peered":"2023-08-27T11:12:07.905203+0000","last_unstale":"2023-08-27T11:12:13.932864+0000","last_undegraded":"2023-08-27T11:12:13.932864+0000","last_fullsized":"2023-08-27T11:12:13.932864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:54.273519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.983012+0000","last_change":"2023-08-27T11:12:08.926915+0000","last_active":"2023-08-27T11:12:14.983012+0000","last_peered":"2023-08-27T11:12:14.983012+0000","last_clean":"2023-08-27T11:12:14.983012+0000","last_became_active":"2023-08-27T11:12:08.923355+0000","last_became_peered":"2023-08-27T11:12:08.923355+0000","last_unstale":"2023-08-27T11:12:14.983012+0000","last_undegraded":"2023-08-27T11:12:14.983012+0000","last_fullsized":"2023-08-27T11:12:14.983012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:12.215932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384503+0000","last_change":"2023-08-27T11:12:12.958000+0000","last_active":"2023-08-27T11:12:14.384503+0000","last_peered":"2023-08-27T11:12:14.384503+0000","last_clean":"2023-08-27T11:12:14.384503+0000","last_became_active":"2023-08-27T11:12:12.957611+0000","last_became_peered":"2023-08-27T11:12:12.957611+0000","last_unstale":"2023-08-27T11:12:14.384503+0000","last_undegraded":"2023-08-27T11:12:14.384503+0000","last_fullsized":"2023-08-27T11:12:14.384503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:44.587717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384852+0000","last_change":"2023-08-27T11:12:07.907385+0000","last_active":"2023-08-27T11:12:14.384852+0000","last_peered":"2023-08-27T11:12:14.384852+0000","last_clean":"2023-08-27T11:12:14.384852+0000","last_became_active":"2023-08-27T11:12:07.907116+0000","last_became_peered":"2023-08-27T11:12:07.907116+0000","last_unstale":"2023-08-27T11:12:14.384852+0000","last_undegraded":"2023-08-27T11:12:14.384852+0000","last_fullsized":"2023-08-27T11:12:14.384852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:37.272544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967084+0000","last_change":"2023-08-27T11:12:08.927597+0000","last_active":"2023-08-27T11:12:14.967084+0000","last_peered":"2023-08-27T11:12:14.967084+0000","last_clean":"2023-08-27T11:12:14.967084+0000","last_became_active":"2023-08-27T11:12:08.927306+0000","last_became_peered":"2023-08-27T11:12:08.927306+0000","last_unstale":"2023-08-27T11:12:14.967084+0000","last_undegraded":"2023-08-27T11:12:14.967084+0000","last_fullsized":"2023-08-27T11:12:14.967084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:20.782136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932501+0000","last_change":"2023-08-27T11:12:12.955644+0000","last_active":"2023-08-27T11:12:13.932501+0000","last_peered":"2023-08-27T11:12:13.932501+0000","last_clean":"2023-08-27T11:12:13.932501+0000","last_became_active":"2023-08-27T11:12:12.952322+0000","last_became_peered":"2023-08-27T11:12:12.952322+0000","last_unstale":"2023-08-27T11:12:13.932501+0000","last_undegraded":"2023-08-27T11:12:13.932501+0000","last_fullsized":"2023-08-27T11:12:13.932501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:16:14.569342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384846+0000","last_change":"2023-08-27T11:12:07.907495+0000","last_active":"2023-08-27T11:12:14.384846+0000","last_peered":"2023-08-27T11:12:14.384846+0000","last_clean":"2023-08-27T11:12:14.384846+0000","last_became_active":"2023-08-27T11:12:07.907216+0000","last_became_peered":"2023-08-27T11:12:07.907216+0000","last_unstale":"2023-08-27T11:12:14.384846+0000","last_undegraded":"2023-08-27T11:12:14.384846+0000","last_fullsized":"2023-08-27T11:12:14.384846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:40.858637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933350+0000","last_change":"2023-08-27T11:12:08.924297+0000","last_active":"2023-08-27T11:12:13.933350+0000","last_peered":"2023-08-27T11:12:13.933350+0000","last_clean":"2023-08-27T11:12:13.933350+0000","last_became_active":"2023-08-27T11:12:08.919603+0000","last_became_peered":"2023-08-27T11:12:08.919603+0000","last_unstale":"2023-08-27T11:12:13.933350+0000","last_undegraded":"2023-08-27T11:12:13.933350+0000","last_fullsized":"2023-08-27T11:12:13.933350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:20:35.025362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932779+0000","last_change":"2023-08-27T11:12:12.954662+0000","last_active":"2023-08-27T11:12:13.932779+0000","last_peered":"2023-08-27T11:12:13.932779+0000","last_clean":"2023-08-27T11:12:13.932779+0000","last_became_active":"2023-08-27T11:12:12.954250+0000","last_became_peered":"2023-08-27T11:12:12.954250+0000","last_unstale":"2023-08-27T11:12:13.932779+0000","last_undegraded":"2023-08-27T11:12:13.932779+0000","last_fullsized":"2023-08-27T11:12:13.932779+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:50.028006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384990+0000","last_change":"2023-08-27T11:12:07.907454+0000","last_active":"2023-08-27T11:12:14.384990+0000","last_peered":"2023-08-27T11:12:14.384990+0000","last_clean":"2023-08-27T11:12:14.384990+0000","last_became_active":"2023-08-27T11:12:07.907161+0000","last_became_peered":"2023-08-27T11:12:07.907161+0000","last_unstale":"2023-08-27T11:12:14.384990+0000","last_undegraded":"2023-08-27T11:12:14.384990+0000","last_fullsized":"2023-08-27T11:12:14.384990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:33.436011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933127+0000","last_change":"2023-08-27T11:12:08.920905+0000","last_active":"2023-08-27T11:12:13.933127+0000","last_peered":"2023-08-27T11:12:13.933127+0000","last_clean":"2023-08-27T11:12:13.933127+0000","last_became_active":"2023-08-27T11:12:08.920628+0000","last_became_peered":"2023-08-27T11:12:08.920628+0000","last_unstale":"2023-08-27T11:12:13.933127+0000","last_undegraded":"2023-08-27T11:12:13.933127+0000","last_fullsized":"2023-08-27T11:12:13.933127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:05.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932699+0000","last_change":"2023-08-27T11:12:12.953446+0000","last_active":"2023-08-27T11:12:13.932699+0000","last_peered":"2023-08-27T11:12:13.932699+0000","last_clean":"2023-08-27T11:12:13.932699+0000","last_became_active":"2023-08-27T11:12:12.953019+0000","last_became_peered":"2023-08-27T11:12:12.953019+0000","last_unstale":"2023-08-27T11:12:13.932699+0000","last_undegraded":"2023-08-27T11:12:13.932699+0000","last_fullsized":"2023-08-27T11:12:13.932699+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:11.191056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967365+0000","last_change":"2023-08-27T11:12:07.896745+0000","last_active":"2023-08-27T11:12:14.967365+0000","last_peered":"2023-08-27T11:12:14.967365+0000","last_clean":"2023-08-27T11:12:14.967365+0000","last_became_active":"2023-08-27T11:12:07.896471+0000","last_became_peered":"2023-08-27T11:12:07.896471+0000","last_unstale":"2023-08-27T11:12:14.967365+0000","last_undegraded":"2023-08-27T11:12:14.967365+0000","last_fullsized":"2023-08-27T11:12:14.967365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:09.470216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726539+0000","last_change":"2023-08-27T11:12:08.918280+0000","last_active":"2023-08-27T11:12:14.726539+0000","last_peered":"2023-08-27T11:12:14.726539+0000","last_clean":"2023-08-27T11:12:14.726539+0000","last_became_active":"2023-08-27T11:12:08.917786+0000","last_became_peered":"2023-08-27T11:12:08.917786+0000","last_unstale":"2023-08-27T11:12:14.726539+0000","last_undegraded":"2023-08-27T11:12:14.726539+0000","last_fullsized":"2023-08-27T11:12:14.726539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:45.655952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932983+0000","last_change":"2023-08-27T11:12:12.957044+0000","last_active":"2023-08-27T11:12:13.932983+0000","last_peered":"2023-08-27T11:12:13.932983+0000","last_clean":"2023-08-27T11:12:13.932983+0000","last_became_active":"2023-08-27T11:12:12.953571+0000","last_became_peered":"2023-08-27T11:12:12.953571+0000","last_unstale":"2023-08-27T11:12:13.932983+0000","last_undegraded":"2023-08-27T11:12:13.932983+0000","last_fullsized":"2023-08-27T11:12:13.932983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:42.930026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982654+0000","last_change":"2023-08-27T11:12:07.899209+0000","last_active":"2023-08-27T11:12:14.982654+0000","last_peered":"2023-08-27T11:12:14.982654+0000","last_clean":"2023-08-27T11:12:14.982654+0000","last_became_active":"2023-08-27T11:12:07.898806+0000","last_became_peered":"2023-08-27T11:12:07.898806+0000","last_unstale":"2023-08-27T11:12:14.982654+0000","last_undegraded":"2023-08-27T11:12:14.982654+0000","last_fullsized":"2023-08-27T11:12:14.982654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:37.272544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933574+0000","last_change":"2023-08-27T11:12:08.919614+0000","last_active":"2023-08-27T11:12:13.933574+0000","last_peered":"2023-08-27T11:12:13.933574+0000","last_clean":"2023-08-27T11:12:13.933574+0000","last_became_active":"2023-08-27T11:12:08.916338+0000","last_became_peered":"2023-08-27T11:12:08.916338+0000","last_unstale":"2023-08-27T11:12:13.933574+0000","last_undegraded":"2023-08-27T11:12:13.933574+0000","last_fullsized":"2023-08-27T11:12:13.933574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:49:01.974112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838031+0000","last_change":"2023-08-27T11:12:12.962859+0000","last_active":"2023-08-27T11:12:14.838031+0000","last_peered":"2023-08-27T11:12:14.838031+0000","last_clean":"2023-08-27T11:12:14.838031+0000","last_became_active":"2023-08-27T11:12:12.962421+0000","last_became_peered":"2023-08-27T11:12:12.962421+0000","last_unstale":"2023-08-27T11:12:14.838031+0000","last_undegraded":"2023-08-27T11:12:14.838031+0000","last_fullsized":"2023-08-27T11:12:14.838031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:21.753280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933251+0000","last_change":"2023-08-27T11:12:07.901923+0000","last_active":"2023-08-27T11:12:13.933251+0000","last_peered":"2023-08-27T11:12:13.933251+0000","last_clean":"2023-08-27T11:12:13.933251+0000","last_became_active":"2023-08-27T11:12:07.901634+0000","last_became_peered":"2023-08-27T11:12:07.901634+0000","last_unstale":"2023-08-27T11:12:13.933251+0000","last_undegraded":"2023-08-27T11:12:13.933251+0000","last_fullsized":"2023-08-27T11:12:13.933251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:44.981487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933505+0000","last_change":"2023-08-27T11:12:08.911380+0000","last_active":"2023-08-27T11:12:13.933505+0000","last_peered":"2023-08-27T11:12:13.933505+0000","last_clean":"2023-08-27T11:12:13.933505+0000","last_became_active":"2023-08-27T11:12:08.911077+0000","last_became_peered":"2023-08-27T11:12:08.911077+0000","last_unstale":"2023-08-27T11:12:13.933505+0000","last_undegraded":"2023-08-27T11:12:13.933505+0000","last_fullsized":"2023-08-27T11:12:13.933505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:59.416171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967787+0000","last_change":"2023-08-27T11:12:12.953058+0000","last_active":"2023-08-27T11:12:14.967787+0000","last_peered":"2023-08-27T11:12:14.967787+0000","last_clean":"2023-08-27T11:12:14.967787+0000","last_became_active":"2023-08-27T11:12:12.952571+0000","last_became_peered":"2023-08-27T11:12:12.952571+0000","last_unstale":"2023-08-27T11:12:14.967787+0000","last_undegraded":"2023-08-27T11:12:14.967787+0000","last_fullsized":"2023-08-27T11:12:14.967787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:25.510081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933016+0000","last_change":"2023-08-27T11:12:07.903262+0000","last_active":"2023-08-27T11:12:13.933016+0000","last_peered":"2023-08-27T11:12:13.933016+0000","last_clean":"2023-08-27T11:12:13.933016+0000","last_became_active":"2023-08-27T11:12:07.902549+0000","last_became_peered":"2023-08-27T11:12:07.902549+0000","last_unstale":"2023-08-27T11:12:13.933016+0000","last_undegraded":"2023-08-27T11:12:13.933016+0000","last_fullsized":"2023-08-27T11:12:13.933016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:25.089023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933460+0000","last_change":"2023-08-27T11:12:08.919554+0000","last_active":"2023-08-27T11:12:13.933460+0000","last_peered":"2023-08-27T11:12:13.933460+0000","last_clean":"2023-08-27T11:12:13.933460+0000","last_became_active":"2023-08-27T11:12:08.919252+0000","last_became_peered":"2023-08-27T11:12:08.919252+0000","last_unstale":"2023-08-27T11:12:13.933460+0000","last_undegraded":"2023-08-27T11:12:13.933460+0000","last_fullsized":"2023-08-27T11:12:13.933460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:14:35.778200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933262+0000","last_change":"2023-08-27T11:12:12.954063+0000","last_active":"2023-08-27T11:12:13.933262+0000","last_peered":"2023-08-27T11:12:13.933262+0000","last_clean":"2023-08-27T11:12:13.933262+0000","last_became_active":"2023-08-27T11:12:12.953652+0000","last_became_peered":"2023-08-27T11:12:12.953652+0000","last_unstale":"2023-08-27T11:12:13.933262+0000","last_undegraded":"2023-08-27T11:12:13.933262+0000","last_fullsized":"2023-08-27T11:12:13.933262+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:31.699925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982128+0000","last_change":"2023-08-27T11:12:07.902468+0000","last_active":"2023-08-27T11:12:14.982128+0000","last_peered":"2023-08-27T11:12:14.982128+0000","last_clean":"2023-08-27T11:12:14.982128+0000","last_became_active":"2023-08-27T11:12:07.902193+0000","last_became_peered":"2023-08-27T11:12:07.902193+0000","last_unstale":"2023-08-27T11:12:14.982128+0000","last_undegraded":"2023-08-27T11:12:14.982128+0000","last_fullsized":"2023-08-27T11:12:14.982128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:06.455977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838774+0000","last_change":"2023-08-27T11:12:08.921979+0000","last_active":"2023-08-27T11:12:14.838774+0000","last_peered":"2023-08-27T11:12:14.838774+0000","last_clean":"2023-08-27T11:12:14.838774+0000","last_became_active":"2023-08-27T11:12:08.921659+0000","last_became_peered":"2023-08-27T11:12:08.921659+0000","last_unstale":"2023-08-27T11:12:14.838774+0000","last_undegraded":"2023-08-27T11:12:14.838774+0000","last_fullsized":"2023-08-27T11:12:14.838774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:40.545799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726360+0000","last_change":"2023-08-27T11:12:12.959327+0000","last_active":"2023-08-27T11:12:14.726360+0000","last_peered":"2023-08-27T11:12:14.726360+0000","last_clean":"2023-08-27T11:12:14.726360+0000","last_became_active":"2023-08-27T11:12:12.957064+0000","last_became_peered":"2023-08-27T11:12:12.957064+0000","last_unstale":"2023-08-27T11:12:14.726360+0000","last_undegraded":"2023-08-27T11:12:14.726360+0000","last_fullsized":"2023-08-27T11:12:14.726360+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:20.210306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838189+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:12:14.838189+0000","last_peered":"2023-08-27T11:12:14.838189+0000","last_clean":"2023-08-27T11:12:14.838189+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:12:14.838189+0000","last_undegraded":"2023-08-27T11:12:14.838189+0000","last_fullsized":"2023-08-27T11:12:14.838189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:04.448471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967607+0000","last_change":"2023-08-27T11:12:07.902012+0000","last_active":"2023-08-27T11:12:14.967607+0000","last_peered":"2023-08-27T11:12:14.967607+0000","last_clean":"2023-08-27T11:12:14.967607+0000","last_became_active":"2023-08-27T11:12:07.901706+0000","last_became_peered":"2023-08-27T11:12:07.901706+0000","last_unstale":"2023-08-27T11:12:14.967607+0000","last_undegraded":"2023-08-27T11:12:14.967607+0000","last_fullsized":"2023-08-27T11:12:14.967607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:21.743516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838056+0000","last_change":"2023-08-27T11:12:08.916047+0000","last_active":"2023-08-27T11:12:14.838056+0000","last_peered":"2023-08-27T11:12:14.838056+0000","last_clean":"2023-08-27T11:12:14.838056+0000","last_became_active":"2023-08-27T11:12:08.915742+0000","last_became_peered":"2023-08-27T11:12:08.915742+0000","last_unstale":"2023-08-27T11:12:14.838056+0000","last_undegraded":"2023-08-27T11:12:14.838056+0000","last_fullsized":"2023-08-27T11:12:14.838056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:29.674355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726449+0000","last_change":"2023-08-27T11:12:12.952690+0000","last_active":"2023-08-27T11:12:14.726449+0000","last_peered":"2023-08-27T11:12:14.726449+0000","last_clean":"2023-08-27T11:12:14.726449+0000","last_became_active":"2023-08-27T11:12:12.952209+0000","last_became_peered":"2023-08-27T11:12:12.952209+0000","last_unstale":"2023-08-27T11:12:14.726449+0000","last_undegraded":"2023-08-27T11:12:14.726449+0000","last_fullsized":"2023-08-27T11:12:14.726449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:28.731513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967363+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:12:14.967363+0000","last_peered":"2023-08-27T11:12:14.967363+0000","last_clean":"2023-08-27T11:12:14.967363+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:12:14.967363+0000","last_undegraded":"2023-08-27T11:12:14.967363+0000","last_fullsized":"2023-08-27T11:12:14.967363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:04.859233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933791+0000","last_change":"2023-08-27T11:12:07.905363+0000","last_active":"2023-08-27T11:12:13.933791+0000","last_peered":"2023-08-27T11:12:13.933791+0000","last_clean":"2023-08-27T11:12:13.933791+0000","last_became_active":"2023-08-27T11:12:07.905002+0000","last_became_peered":"2023-08-27T11:12:07.905002+0000","last_unstale":"2023-08-27T11:12:13.933791+0000","last_undegraded":"2023-08-27T11:12:13.933791+0000","last_fullsized":"2023-08-27T11:12:13.933791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:06:55.872046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838055+0000","last_change":"2023-08-27T11:12:08.920259+0000","last_active":"2023-08-27T11:12:14.838055+0000","last_peered":"2023-08-27T11:12:14.838055+0000","last_clean":"2023-08-27T11:12:14.838055+0000","last_became_active":"2023-08-27T11:12:08.916978+0000","last_became_peered":"2023-08-27T11:12:08.916978+0000","last_unstale":"2023-08-27T11:12:14.838055+0000","last_undegraded":"2023-08-27T11:12:14.838055+0000","last_fullsized":"2023-08-27T11:12:14.838055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:29:50.086138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933007+0000","last_change":"2023-08-27T11:12:12.959735+0000","last_active":"2023-08-27T11:12:13.933007+0000","last_peered":"2023-08-27T11:12:13.933007+0000","last_clean":"2023-08-27T11:12:13.933007+0000","last_became_active":"2023-08-27T11:12:12.959275+0000","last_became_peered":"2023-08-27T11:12:12.959275+0000","last_unstale":"2023-08-27T11:12:13.933007+0000","last_undegraded":"2023-08-27T11:12:13.933007+0000","last_fullsized":"2023-08-27T11:12:13.933007+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:57.120477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932550+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:12:13.932550+0000","last_peered":"2023-08-27T11:12:13.932550+0000","last_clean":"2023-08-27T11:12:13.932550+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:12:13.932550+0000","last_undegraded":"2023-08-27T11:12:13.932550+0000","last_fullsized":"2023-08-27T11:12:13.932550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:49.251775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837761+0000","last_change":"2023-08-27T11:12:07.902482+0000","last_active":"2023-08-27T11:12:14.837761+0000","last_peered":"2023-08-27T11:12:14.837761+0000","last_clean":"2023-08-27T11:12:14.837761+0000","last_became_active":"2023-08-27T11:12:07.902184+0000","last_became_peered":"2023-08-27T11:12:07.902184+0000","last_unstale":"2023-08-27T11:12:14.837761+0000","last_undegraded":"2023-08-27T11:12:14.837761+0000","last_fullsized":"2023-08-27T11:12:14.837761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:46.293833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838666+0000","last_change":"2023-08-27T11:12:08.919949+0000","last_active":"2023-08-27T11:12:14.838666+0000","last_peered":"2023-08-27T11:12:14.838666+0000","last_clean":"2023-08-27T11:12:14.838666+0000","last_became_active":"2023-08-27T11:12:08.918796+0000","last_became_peered":"2023-08-27T11:12:08.918796+0000","last_unstale":"2023-08-27T11:12:14.838666+0000","last_undegraded":"2023-08-27T11:12:14.838666+0000","last_fullsized":"2023-08-27T11:12:14.838666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:51.458617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384661+0000","last_change":"2023-08-27T11:12:12.965405+0000","last_active":"2023-08-27T11:12:14.384661+0000","last_peered":"2023-08-27T11:12:14.384661+0000","last_clean":"2023-08-27T11:12:14.384661+0000","last_became_active":"2023-08-27T11:12:12.965085+0000","last_became_peered":"2023-08-27T11:12:12.965085+0000","last_unstale":"2023-08-27T11:12:14.384661+0000","last_undegraded":"2023-08-27T11:12:14.384661+0000","last_fullsized":"2023-08-27T11:12:14.384661+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:45.516388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967468+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:12:14.967468+0000","last_peered":"2023-08-27T11:12:14.967468+0000","last_clean":"2023-08-27T11:12:14.967468+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:12:14.967468+0000","last_undegraded":"2023-08-27T11:12:14.967468+0000","last_fullsized":"2023-08-27T11:12:14.967468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:47.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384165+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:12:14.384165+0000","last_peered":"2023-08-27T11:12:14.384165+0000","last_clean":"2023-08-27T11:12:14.384165+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:12:14.384165+0000","last_undegraded":"2023-08-27T11:12:14.384165+0000","last_fullsized":"2023-08-27T11:12:14.384165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:10.078531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982472+0000","last_change":"2023-08-27T11:12:07.903434+0000","last_active":"2023-08-27T11:12:14.982472+0000","last_peered":"2023-08-27T11:12:14.982472+0000","last_clean":"2023-08-27T11:12:14.982472+0000","last_became_active":"2023-08-27T11:12:07.903142+0000","last_became_peered":"2023-08-27T11:12:07.903142+0000","last_unstale":"2023-08-27T11:12:14.982472+0000","last_undegraded":"2023-08-27T11:12:14.982472+0000","last_fullsized":"2023-08-27T11:12:14.982472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:40.565665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934388+0000","last_change":"2023-08-27T11:12:08.915968+0000","last_active":"2023-08-27T11:12:13.934388+0000","last_peered":"2023-08-27T11:12:13.934388+0000","last_clean":"2023-08-27T11:12:13.934388+0000","last_became_active":"2023-08-27T11:12:08.915646+0000","last_became_peered":"2023-08-27T11:12:08.915646+0000","last_unstale":"2023-08-27T11:12:13.934388+0000","last_undegraded":"2023-08-27T11:12:13.934388+0000","last_fullsized":"2023-08-27T11:12:13.934388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:53.078559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967776+0000","last_change":"2023-08-27T11:12:12.953539+0000","last_active":"2023-08-27T11:12:14.967776+0000","last_peered":"2023-08-27T11:12:14.967776+0000","last_clean":"2023-08-27T11:12:14.967776+0000","last_became_active":"2023-08-27T11:12:12.953154+0000","last_became_peered":"2023-08-27T11:12:12.953154+0000","last_unstale":"2023-08-27T11:12:14.967776+0000","last_undegraded":"2023-08-27T11:12:14.967776+0000","last_fullsized":"2023-08-27T11:12:14.967776+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:45.516388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384351+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:12:14.384351+0000","last_peered":"2023-08-27T11:12:14.384351+0000","last_clean":"2023-08-27T11:12:14.384351+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:12:14.384351+0000","last_undegraded":"2023-08-27T11:12:14.384351+0000","last_fullsized":"2023-08-27T11:12:14.384351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:47.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.725986+0000","last_change":"2023-08-27T11:12:07.899116+0000","last_active":"2023-08-27T11:12:14.725986+0000","last_peered":"2023-08-27T11:12:14.725986+0000","last_clean":"2023-08-27T11:12:14.725986+0000","last_became_active":"2023-08-27T11:12:07.898805+0000","last_became_peered":"2023-08-27T11:12:07.898805+0000","last_unstale":"2023-08-27T11:12:14.725986+0000","last_undegraded":"2023-08-27T11:12:14.725986+0000","last_fullsized":"2023-08-27T11:12:14.725986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:07.168983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726764+0000","last_change":"2023-08-27T11:12:08.918851+0000","last_active":"2023-08-27T11:12:14.726764+0000","last_peered":"2023-08-27T11:12:14.726764+0000","last_clean":"2023-08-27T11:12:14.726764+0000","last_became_active":"2023-08-27T11:12:08.918248+0000","last_became_peered":"2023-08-27T11:12:08.918248+0000","last_unstale":"2023-08-27T11:12:14.726764+0000","last_undegraded":"2023-08-27T11:12:14.726764+0000","last_fullsized":"2023-08-27T11:12:14.726764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:45.250102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933527+0000","last_change":"2023-08-27T11:12:12.959470+0000","last_active":"2023-08-27T11:12:13.933527+0000","last_peered":"2023-08-27T11:12:13.933527+0000","last_clean":"2023-08-27T11:12:13.933527+0000","last_became_active":"2023-08-27T11:12:12.959082+0000","last_became_peered":"2023-08-27T11:12:12.959082+0000","last_unstale":"2023-08-27T11:12:13.933527+0000","last_undegraded":"2023-08-27T11:12:13.933527+0000","last_fullsized":"2023-08-27T11:12:13.933527+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:04.206952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384869+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:12:14.384869+0000","last_peered":"2023-08-27T11:12:14.384869+0000","last_clean":"2023-08-27T11:12:14.384869+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:12:14.384869+0000","last_undegraded":"2023-08-27T11:12:14.384869+0000","last_fullsized":"2023-08-27T11:12:14.384869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:43.735919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838283+0000","last_change":"2023-08-27T11:12:07.903460+0000","last_active":"2023-08-27T11:12:14.838283+0000","last_peered":"2023-08-27T11:12:14.838283+0000","last_clean":"2023-08-27T11:12:14.838283+0000","last_became_active":"2023-08-27T11:12:07.903110+0000","last_became_peered":"2023-08-27T11:12:07.903110+0000","last_unstale":"2023-08-27T11:12:14.838283+0000","last_undegraded":"2023-08-27T11:12:14.838283+0000","last_fullsized":"2023-08-27T11:12:14.838283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:09.880978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726625+0000","last_change":"2023-08-27T11:12:08.913520+0000","last_active":"2023-08-27T11:12:14.726625+0000","last_peered":"2023-08-27T11:12:14.726625+0000","last_clean":"2023-08-27T11:12:14.726625+0000","last_became_active":"2023-08-27T11:12:08.913192+0000","last_became_peered":"2023-08-27T11:12:08.913192+0000","last_unstale":"2023-08-27T11:12:14.726625+0000","last_undegraded":"2023-08-27T11:12:14.726625+0000","last_fullsized":"2023-08-27T11:12:14.726625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:12:06.235129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933906+0000","last_change":"2023-08-27T11:12:12.959458+0000","last_active":"2023-08-27T11:12:13.933906+0000","last_peered":"2023-08-27T11:12:13.933906+0000","last_clean":"2023-08-27T11:12:13.933906+0000","last_became_active":"2023-08-27T11:12:12.959073+0000","last_became_peered":"2023-08-27T11:12:12.959073+0000","last_unstale":"2023-08-27T11:12:13.933906+0000","last_undegraded":"2023-08-27T11:12:13.933906+0000","last_fullsized":"2023-08-27T11:12:13.933906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:37:24.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.983100+0000","last_change":"2023-08-27T11:12:07.908908+0000","last_active":"2023-08-27T11:12:14.983100+0000","last_peered":"2023-08-27T11:12:14.983100+0000","last_clean":"2023-08-27T11:12:14.983100+0000","last_became_active":"2023-08-27T11:12:07.908670+0000","last_became_peered":"2023-08-27T11:12:07.908670+0000","last_unstale":"2023-08-27T11:12:14.983100+0000","last_undegraded":"2023-08-27T11:12:14.983100+0000","last_fullsized":"2023-08-27T11:12:14.983100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:06:59.757966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838348+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:12:14.838348+0000","last_peered":"2023-08-27T11:12:14.838348+0000","last_clean":"2023-08-27T11:12:14.838348+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:12:14.838348+0000","last_undegraded":"2023-08-27T11:12:14.838348+0000","last_fullsized":"2023-08-27T11:12:14.838348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:01.434232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982129+0000","last_change":"2023-08-27T11:12:08.917056+0000","last_active":"2023-08-27T11:12:14.982129+0000","last_peered":"2023-08-27T11:12:14.982129+0000","last_clean":"2023-08-27T11:12:14.982129+0000","last_became_active":"2023-08-27T11:12:08.916853+0000","last_became_peered":"2023-08-27T11:12:08.916853+0000","last_unstale":"2023-08-27T11:12:14.982129+0000","last_undegraded":"2023-08-27T11:12:14.982129+0000","last_fullsized":"2023-08-27T11:12:14.982129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:48.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982032+0000","last_change":"2023-08-27T11:12:12.955385+0000","last_active":"2023-08-27T11:12:14.982032+0000","last_peered":"2023-08-27T11:12:14.982032+0000","last_clean":"2023-08-27T11:12:14.982032+0000","last_became_active":"2023-08-27T11:12:12.953960+0000","last_became_peered":"2023-08-27T11:12:12.953960+0000","last_unstale":"2023-08-27T11:12:14.982032+0000","last_undegraded":"2023-08-27T11:12:14.982032+0000","last_fullsized":"2023-08-27T11:12:14.982032+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:14.927498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934591+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:12:13.934591+0000","last_peered":"2023-08-27T11:12:13.934591+0000","last_clean":"2023-08-27T11:12:13.934591+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:12:13.934591+0000","last_undegraded":"2023-08-27T11:12:13.934591+0000","last_fullsized":"2023-08-27T11:12:13.934591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:05:01.448436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726808+0000","last_change":"2023-08-27T11:12:07.902830+0000","last_active":"2023-08-27T11:12:14.726808+0000","last_peered":"2023-08-27T11:12:14.726808+0000","last_clean":"2023-08-27T11:12:14.726808+0000","last_became_active":"2023-08-27T11:12:07.902452+0000","last_became_peered":"2023-08-27T11:12:07.902452+0000","last_unstale":"2023-08-27T11:12:14.726808+0000","last_undegraded":"2023-08-27T11:12:14.726808+0000","last_fullsized":"2023-08-27T11:12:14.726808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:46.293833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933163+0000","last_change":"2023-08-27T11:12:08.919355+0000","last_active":"2023-08-27T11:12:13.933163+0000","last_peered":"2023-08-27T11:12:13.933163+0000","last_clean":"2023-08-27T11:12:13.933163+0000","last_became_active":"2023-08-27T11:12:08.917203+0000","last_became_peered":"2023-08-27T11:12:08.917203+0000","last_unstale":"2023-08-27T11:12:13.933163+0000","last_undegraded":"2023-08-27T11:12:13.933163+0000","last_fullsized":"2023-08-27T11:12:13.933163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:07:00.762567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838062+0000","last_change":"2023-08-27T11:12:12.963107+0000","last_active":"2023-08-27T11:12:14.838062+0000","last_peered":"2023-08-27T11:12:14.838062+0000","last_clean":"2023-08-27T11:12:14.838062+0000","last_became_active":"2023-08-27T11:12:12.962714+0000","last_became_peered":"2023-08-27T11:12:12.962714+0000","last_unstale":"2023-08-27T11:12:14.838062+0000","last_undegraded":"2023-08-27T11:12:14.838062+0000","last_fullsized":"2023-08-27T11:12:14.838062+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:24.824055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726473+0000","last_change":"2023-08-27T11:12:07.900588+0000","last_active":"2023-08-27T11:12:14.726473+0000","last_peered":"2023-08-27T11:12:14.726473+0000","last_clean":"2023-08-27T11:12:14.726473+0000","last_became_active":"2023-08-27T11:12:07.900223+0000","last_became_peered":"2023-08-27T11:12:07.900223+0000","last_unstale":"2023-08-27T11:12:14.726473+0000","last_undegraded":"2023-08-27T11:12:14.726473+0000","last_fullsized":"2023-08-27T11:12:14.726473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:54.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933266+0000","last_change":"2023-08-27T11:12:08.921029+0000","last_active":"2023-08-27T11:12:13.933266+0000","last_peered":"2023-08-27T11:12:13.933266+0000","last_clean":"2023-08-27T11:12:13.933266+0000","last_became_active":"2023-08-27T11:12:08.920756+0000","last_became_peered":"2023-08-27T11:12:08.920756+0000","last_unstale":"2023-08-27T11:12:13.933266+0000","last_undegraded":"2023-08-27T11:12:13.933266+0000","last_fullsized":"2023-08-27T11:12:13.933266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:21:21.628536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384440+0000","last_change":"2023-08-27T11:12:12.958972+0000","last_active":"2023-08-27T11:12:14.384440+0000","last_peered":"2023-08-27T11:12:14.384440+0000","last_clean":"2023-08-27T11:12:14.384440+0000","last_became_active":"2023-08-27T11:12:12.958605+0000","last_became_peered":"2023-08-27T11:12:12.958605+0000","last_unstale":"2023-08-27T11:12:14.384440+0000","last_undegraded":"2023-08-27T11:12:14.384440+0000","last_fullsized":"2023-08-27T11:12:14.384440+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:55.500535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967443+0000","last_change":"2023-08-27T11:12:07.898515+0000","last_active":"2023-08-27T11:12:14.967443+0000","last_peered":"2023-08-27T11:12:14.967443+0000","last_clean":"2023-08-27T11:12:14.967443+0000","last_became_active":"2023-08-27T11:12:07.898243+0000","last_became_peered":"2023-08-27T11:12:07.898243+0000","last_unstale":"2023-08-27T11:12:14.967443+0000","last_undegraded":"2023-08-27T11:12:14.967443+0000","last_fullsized":"2023-08-27T11:12:14.967443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:47.521180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934374+0000","last_change":"2023-08-27T11:12:08.919278+0000","last_active":"2023-08-27T11:12:13.934374+0000","last_peered":"2023-08-27T11:12:13.934374+0000","last_clean":"2023-08-27T11:12:13.934374+0000","last_became_active":"2023-08-27T11:12:08.913663+0000","last_became_peered":"2023-08-27T11:12:08.913663+0000","last_unstale":"2023-08-27T11:12:13.934374+0000","last_undegraded":"2023-08-27T11:12:13.934374+0000","last_fullsized":"2023-08-27T11:12:13.934374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:20:17.707884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967392+0000","last_change":"2023-08-27T11:12:12.955103+0000","last_active":"2023-08-27T11:12:14.967392+0000","last_peered":"2023-08-27T11:12:14.967392+0000","last_clean":"2023-08-27T11:12:14.967392+0000","last_became_active":"2023-08-27T11:12:12.954719+0000","last_became_peered":"2023-08-27T11:12:12.954719+0000","last_unstale":"2023-08-27T11:12:14.967392+0000","last_undegraded":"2023-08-27T11:12:14.967392+0000","last_fullsized":"2023-08-27T11:12:14.967392+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:16.222119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967603+0000","last_change":"2023-08-27T11:12:07.899028+0000","last_active":"2023-08-27T11:12:14.967603+0000","last_peered":"2023-08-27T11:12:14.967603+0000","last_clean":"2023-08-27T11:12:14.967603+0000","last_became_active":"2023-08-27T11:12:07.898749+0000","last_became_peered":"2023-08-27T11:12:07.898749+0000","last_unstale":"2023-08-27T11:12:14.967603+0000","last_undegraded":"2023-08-27T11:12:14.967603+0000","last_fullsized":"2023-08-27T11:12:14.967603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:50.454016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933154+0000","last_change":"2023-08-27T11:12:08.917140+0000","last_active":"2023-08-27T11:12:13.933154+0000","last_peered":"2023-08-27T11:12:13.933154+0000","last_clean":"2023-08-27T11:12:13.933154+0000","last_became_active":"2023-08-27T11:12:08.916844+0000","last_became_peered":"2023-08-27T11:12:08.916844+0000","last_unstale":"2023-08-27T11:12:13.933154+0000","last_undegraded":"2023-08-27T11:12:13.933154+0000","last_fullsized":"2023-08-27T11:12:13.933154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:12.215932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933507+0000","last_change":"2023-08-27T11:12:12.952640+0000","last_active":"2023-08-27T11:12:13.933507+0000","last_peered":"2023-08-27T11:12:13.933507+0000","last_clean":"2023-08-27T11:12:13.933507+0000","last_became_active":"2023-08-27T11:12:12.952235+0000","last_became_peered":"2023-08-27T11:12:12.952235+0000","last_unstale":"2023-08-27T11:12:13.933507+0000","last_undegraded":"2023-08-27T11:12:13.933507+0000","last_fullsized":"2023-08-27T11:12:13.933507+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:44.222308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982940+0000","last_change":"2023-08-27T11:12:07.898854+0000","last_active":"2023-08-27T11:12:14.982940+0000","last_peered":"2023-08-27T11:12:14.982940+0000","last_clean":"2023-08-27T11:12:14.982940+0000","last_became_active":"2023-08-27T11:12:07.898427+0000","last_became_peered":"2023-08-27T11:12:07.898427+0000","last_unstale":"2023-08-27T11:12:14.982940+0000","last_undegraded":"2023-08-27T11:12:14.982940+0000","last_fullsized":"2023-08-27T11:12:14.982940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:25.089023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982399+0000","last_change":"2023-08-27T11:12:08.913067+0000","last_active":"2023-08-27T11:12:14.982399+0000","last_peered":"2023-08-27T11:12:14.982399+0000","last_clean":"2023-08-27T11:12:14.982399+0000","last_became_active":"2023-08-27T11:12:08.912345+0000","last_became_peered":"2023-08-27T11:12:08.912345+0000","last_unstale":"2023-08-27T11:12:14.982399+0000","last_undegraded":"2023-08-27T11:12:14.982399+0000","last_fullsized":"2023-08-27T11:12:14.982399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:41.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838060+0000","last_change":"2023-08-27T11:12:12.948294+0000","last_active":"2023-08-27T11:12:14.838060+0000","last_peered":"2023-08-27T11:12:14.838060+0000","last_clean":"2023-08-27T11:12:14.838060+0000","last_became_active":"2023-08-27T11:12:12.947742+0000","last_became_peered":"2023-08-27T11:12:12.947742+0000","last_unstale":"2023-08-27T11:12:14.838060+0000","last_undegraded":"2023-08-27T11:12:14.838060+0000","last_fullsized":"2023-08-27T11:12:14.838060+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:06.825066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727133+0000","last_change":"2023-08-27T11:12:07.902780+0000","last_active":"2023-08-27T11:12:14.727133+0000","last_peered":"2023-08-27T11:12:14.727133+0000","last_clean":"2023-08-27T11:12:14.727133+0000","last_became_active":"2023-08-27T11:12:07.902465+0000","last_became_peered":"2023-08-27T11:12:07.902465+0000","last_unstale":"2023-08-27T11:12:14.727133+0000","last_undegraded":"2023-08-27T11:12:14.727133+0000","last_fullsized":"2023-08-27T11:12:14.727133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:09.470216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933956+0000","last_change":"2023-08-27T11:12:08.925508+0000","last_active":"2023-08-27T11:12:13.933956+0000","last_peered":"2023-08-27T11:12:13.933956+0000","last_clean":"2023-08-27T11:12:13.933956+0000","last_became_active":"2023-08-27T11:12:08.919770+0000","last_became_peered":"2023-08-27T11:12:08.919770+0000","last_unstale":"2023-08-27T11:12:13.933956+0000","last_undegraded":"2023-08-27T11:12:13.933956+0000","last_fullsized":"2023-08-27T11:12:13.933956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:57.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932961+0000","last_change":"2023-08-27T11:12:08.914782+0000","last_active":"2023-08-27T11:12:13.932961+0000","last_peered":"2023-08-27T11:12:13.932961+0000","last_clean":"2023-08-27T11:12:13.932961+0000","last_became_active":"2023-08-27T11:12:08.914443+0000","last_became_peered":"2023-08-27T11:12:08.914443+0000","last_unstale":"2023-08-27T11:12:13.932961+0000","last_undegraded":"2023-08-27T11:12:13.932961+0000","last_fullsized":"2023-08-27T11:12:13.932961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:07.149137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982607+0000","last_change":"2023-08-27T11:12:08.923263+0000","last_active":"2023-08-27T11:12:14.982607+0000","last_peered":"2023-08-27T11:12:14.982607+0000","last_clean":"2023-08-27T11:12:14.982607+0000","last_became_active":"2023-08-27T11:12:08.922904+0000","last_became_peered":"2023-08-27T11:12:08.922904+0000","last_unstale":"2023-08-27T11:12:14.982607+0000","last_undegraded":"2023-08-27T11:12:14.982607+0000","last_fullsized":"2023-08-27T11:12:14.982607+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:49.762265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934690+0000","last_change":"2023-08-27T11:12:08.925804+0000","last_active":"2023-08-27T11:12:13.934690+0000","last_peered":"2023-08-27T11:12:13.934690+0000","last_clean":"2023-08-27T11:12:13.934690+0000","last_became_active":"2023-08-27T11:12:08.925554+0000","last_became_peered":"2023-08-27T11:12:08.925554+0000","last_unstale":"2023-08-27T11:12:13.934690+0000","last_undegraded":"2023-08-27T11:12:13.934690+0000","last_fullsized":"2023-08-27T11:12:13.934690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:15.724420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982308+0000","last_change":"2023-08-27T11:12:08.929175+0000","last_active":"2023-08-27T11:12:14.982308+0000","last_peered":"2023-08-27T11:12:14.982308+0000","last_clean":"2023-08-27T11:12:14.982308+0000","last_became_active":"2023-08-27T11:12:08.928863+0000","last_became_peered":"2023-08-27T11:12:08.928863+0000","last_unstale":"2023-08-27T11:12:14.982308+0000","last_undegraded":"2023-08-27T11:12:14.982308+0000","last_fullsized":"2023-08-27T11:12:14.982308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:30.787887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838203+0000","last_change":"2023-08-27T11:12:08.924528+0000","last_active":"2023-08-27T11:12:14.838203+0000","last_peered":"2023-08-27T11:12:14.838203+0000","last_clean":"2023-08-27T11:12:14.838203+0000","last_became_active":"2023-08-27T11:12:08.924221+0000","last_became_peered":"2023-08-27T11:12:08.924221+0000","last_unstale":"2023-08-27T11:12:14.838203+0000","last_undegraded":"2023-08-27T11:12:14.838203+0000","last_fullsized":"2023-08-27T11:12:14.838203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:12.180200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":710851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":120553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:12:35.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:12:35.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.289+0000 7fcad185b700 1 Processor -- start 2023-08-27T11:12:35.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.289+0000 7fcad185b700 1 -- start start 2023-08-27T11:12:35.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.289+0000 7fcad185b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcacc0cf980 0x7fcacc0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.289+0000 7fcad185b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc0ceae0 con 0x7fcacc0b45d0 2023-08-27T11:12:35.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.289+0000 7fcad185b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc0cedc0 con 0x7fcacc0cfd60 2023-08-27T11:12:35.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.289+0000 7fcad185b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc0cf0a0 con 0x7fcacc0cf980 2023-08-27T11:12:35.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcacaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcacc0cf980 0x7fcacc0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcaca7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcacc0b45d0 0x7fcacc0d1460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38670/0 (socket says 172.21.15.45:38670) 2023-08-27T11:12:35.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcaca7fc700 1 -- 172.21.15.45:0/3423783736 learned_addr learned my addr 172.21.15.45:0/3423783736 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:35.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcacaffd700 1 -- 172.21.15.45:0/3423783736 >> v1:172.21.15.135:6789/0 conn(0x7fcacc0cfd60 legacy=0x7fcacc0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcacaffd700 1 -- 172.21.15.45:0/3423783736 >> v1:172.21.15.45:6789/0 conn(0x7fcacc0b45d0 legacy=0x7fcacc0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcacaffd700 1 -- 172.21.15.45:0/3423783736 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc0b22c0 con 0x7fcacc0cf980 2023-08-27T11:12:35.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcac9ffb700 1 -- 172.21.15.45:0/3423783736 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141057519 0 0) 0x7fcacc0ceae0 con 0x7fcacc0b45d0 2023-08-27T11:12:35.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcacaffd700 1 --2- 172.21.15.45:0/3423783736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cf980 0x7fcacc0d0f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009840 tx=0x7fcabc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6473c2c44654497 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:35.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcac9ffb700 1 -- 172.21.15.45:0/3423783736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc011830 con 0x7fcacc0cf980 2023-08-27T11:12:35.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcac9ffb700 1 -- 172.21.15.45:0/3423783736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc011990 con 0x7fcacc0cf980 2023-08-27T11:12:35.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.290+0000 7fcac9ffb700 1 -- 172.21.15.45:0/3423783736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc011d20 con 0x7fcacc0cf980 2023-08-27T11:12:35.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.291+0000 7fcad185b700 1 -- 172.21.15.45:0/3423783736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cf980 msgr2=0x7fcacc0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.291+0000 7fcad185b700 1 --2- 172.21.15.45:0/3423783736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cf980 0x7fcacc0d0f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009840 tx=0x7fcabc009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:35.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.291+0000 7fcad185b700 1 -- 172.21.15.45:0/3423783736 shutdown_connections 2023-08-27T11:12:35.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.291+0000 7fcad185b700 1 --2- 172.21.15.45:0/3423783736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cf980 0x7fcacc0d0f20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.291+0000 7fcad185b700 1 -- 172.21.15.45:0/3423783736 >> 172.21.15.45:0/3423783736 conn(0x7fcacc0d3020 msgr2=0x7fcacc0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:35.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.291+0000 7fcad185b700 1 -- 172.21.15.45:0/3423783736 shutdown_connections 2023-08-27T11:12:35.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.291+0000 7fcad185b700 1 -- 172.21.15.45:0/3423783736 wait complete. 2023-08-27T11:12:35.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.292+0000 7fcad185b700 1 Processor -- start 2023-08-27T11:12:35.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.292+0000 7fcad185b700 1 -- start start 2023-08-27T11:12:35.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcad185b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcacc0b45d0 0x7fcacc0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcad185b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcacc0cf980 0x7fcacc0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcad185b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cfd60 0x7fcacc15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcad185b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0c8830 con 0x7fcacc0cfd60 2023-08-27T11:12:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcad185b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0c8b10 con 0x7fcacc0cf980 2023-08-27T11:12:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcad185b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0c8df0 con 0x7fcacc0b45d0 2023-08-27T11:12:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcacb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cfd60 0x7fcacc15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcacb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cfd60 0x7fcacc15a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51904/0 (socket says 172.21.15.45:51904) 2023-08-27T11:12:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcacb7fe700 1 -- 172.21.15.45:0/3601667950 learned_addr learned my addr 172.21.15.45:0/3601667950 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcacaffd700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcacc0b45d0 0x7fcacc0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcaca7fc700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcacc0cf980 0x7fcacc0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcacb7fe700 1 -- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcacc0b45d0 msgr2=0x7fcacc0c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcacb7fe700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcacc0b45d0 0x7fcacc0c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcacb7fe700 1 -- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcacc0cf980 msgr2=0x7fcacc0c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcacb7fe700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcacc0cf980 0x7fcacc0c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.293+0000 7fcacb7fe700 1 -- 172.21.15.45:0/3601667950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc0094a0 con 0x7fcacc0cfd60 2023-08-27T11:12:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.294+0000 7fcacb7fe700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cfd60 0x7fcacc15a8c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcac400cbc0 tx=0x7fcac400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.294+0000 7fcab3fff700 1 -- 172.21.15.45:0/3601667950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4003c00 con 0x7fcacc0cfd60 2023-08-27T11:12:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.294+0000 7fcab3fff700 1 -- 172.21.15.45:0/3601667950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac4003d60 con 0x7fcacc0cfd60 2023-08-27T11:12:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.294+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc0c90d0 con 0x7fcacc0cfd60 2023-08-27T11:12:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.294+0000 7fcab3fff700 1 -- 172.21.15.45:0/3601667950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400aa20 con 0x7fcacc0cfd60 2023-08-27T11:12:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.294+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc0cb9a0 con 0x7fcacc0cfd60 2023-08-27T11:12:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.295+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc0a1030 con 0x7fcacc0cfd60 2023-08-27T11:12:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.295+0000 7fcab3fff700 1 -- 172.21.15.45:0/3601667950 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcac400ab80 con 0x7fcacc0cfd60 2023-08-27T11:12:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.296+0000 7fcab3fff700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcab806a0e0 0x7fcab806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.296+0000 7fcab3fff700 1 -- 172.21.15.45:0/3601667950 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcac4088740 con 0x7fcacc0cfd60 2023-08-27T11:12:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.296+0000 7fcacaffd700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcab806a0e0 0x7fcab806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.296+0000 7fcacaffd700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcab806a0e0 0x7fcab806c5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0090b0 tx=0x7fcabc01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:35.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.300+0000 7fcab3fff700 1 -- 172.21.15.45:0/3601667950 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcac4053bf0 con 0x7fcacc0cfd60 2023-08-27T11:12:35.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.496+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 --> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcacc0b25a0 con 0x7fcab806a0e0 2023-08-27T11:12:35.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.506+0000 7fcab3fff700 1 -- 172.21.15.45:0/3601667950 <== mgr.4100 v2:172.21.15.135:6832/4120550306 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367365 (secure 0 0 0) 0x7fcabc011c80 con 0x7fcab806a0e0 2023-08-27T11:12:35.512 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:35.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcab806a0e0 msgr2=0x7fcab806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcab806a0e0 0x7fcab806c5a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0090b0 tx=0x7fcabc01c040 comp rx=0 tx=0).stop 2023-08-27T11:12:35.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cfd60 msgr2=0x7fcacc15a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cfd60 0x7fcacc15a8c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcac400cbc0 tx=0x7fcac400ced0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 shutdown_connections 2023-08-27T11:12:35.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcacc0cfd60 0x7fcacc15a8c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcab806a0e0 0x7fcab806c5a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcacc0b45d0 0x7fcacc0c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 --2- 172.21.15.45:0/3601667950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcacc0cf980 0x7fcacc0c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.509+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 >> 172.21.15.45:0/3601667950 conn(0x7fcacc0d3020 msgr2=0x7fcacc0b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:35.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.510+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 shutdown_connections 2023-08-27T11:12:35.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.510+0000 7fcad185b700 1 -- 172.21.15.45:0/3601667950 wait complete. 2023-08-27T11:12:35.518 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:12:35.524 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-08-27T11:12:34.811125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219664,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755056,"statfs":{"total":773094113280,"available":772869177344,"internally_reserved":0,"allocated":3579904,"data_stored":2149224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003565"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932872+0000","last_change":"2023-08-27T11:12:08.913049+0000","last_active":"2023-08-27T11:12:13.932872+0000","last_peered":"2023-08-27T11:12:13.932872+0000","last_clean":"2023-08-27T11:12:13.932872+0000","last_became_active":"2023-08-27T11:12:08.912757+0000","last_became_peered":"2023-08-27T11:12:08.912757+0000","last_unstale":"2023-08-27T11:12:13.932872+0000","last_undegraded":"2023-08-27T11:12:13.932872+0000","last_fullsized":"2023-08-27T11:12:13.932872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:00.165033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967658+0000","last_change":"2023-08-27T11:12:08.918994+0000","last_active":"2023-08-27T11:12:14.967658+0000","last_peered":"2023-08-27T11:12:14.967658+0000","last_clean":"2023-08-27T11:12:14.967658+0000","last_became_active":"2023-08-27T11:12:08.918683+0000","last_became_peered":"2023-08-27T11:12:08.918683+0000","last_unstale":"2023-08-27T11:12:14.967658+0000","last_undegraded":"2023-08-27T11:12:14.967658+0000","last_fullsized":"2023-08-27T11:12:14.967658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:40.545799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934025+0000","last_change":"2023-08-27T11:12:08.911497+0000","last_active":"2023-08-27T11:12:13.934025+0000","last_peered":"2023-08-27T11:12:13.934025+0000","last_clean":"2023-08-27T11:12:13.934025+0000","last_became_active":"2023-08-27T11:12:08.911184+0000","last_became_peered":"2023-08-27T11:12:08.911184+0000","last_unstale":"2023-08-27T11:12:13.934025+0000","last_undegraded":"2023-08-27T11:12:13.934025+0000","last_fullsized":"2023-08-27T11:12:13.934025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:45.042697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934063+0000","last_change":"2023-08-27T11:12:08.914949+0000","last_active":"2023-08-27T11:12:13.934063+0000","last_peered":"2023-08-27T11:12:13.934063+0000","last_clean":"2023-08-27T11:12:13.934063+0000","last_became_active":"2023-08-27T11:12:08.914642+0000","last_became_peered":"2023-08-27T11:12:08.914642+0000","last_unstale":"2023-08-27T11:12:13.934063+0000","last_undegraded":"2023-08-27T11:12:13.934063+0000","last_fullsized":"2023-08-27T11:12:13.934063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:38.888108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726727+0000","last_change":"2023-08-27T11:12:08.929257+0000","last_active":"2023-08-27T11:12:14.726727+0000","last_peered":"2023-08-27T11:12:14.726727+0000","last_clean":"2023-08-27T11:12:14.726727+0000","last_became_active":"2023-08-27T11:12:08.928926+0000","last_became_peered":"2023-08-27T11:12:08.928926+0000","last_unstale":"2023-08-27T11:12:14.726727+0000","last_undegraded":"2023-08-27T11:12:14.726727+0000","last_fullsized":"2023-08-27T11:12:14.726727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:09.445014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934838+0000","last_change":"2023-08-27T11:12:08.916024+0000","last_active":"2023-08-27T11:12:13.934838+0000","last_peered":"2023-08-27T11:12:13.934838+0000","last_clean":"2023-08-27T11:12:13.934838+0000","last_became_active":"2023-08-27T11:12:08.913502+0000","last_became_peered":"2023-08-27T11:12:08.913502+0000","last_unstale":"2023-08-27T11:12:13.934838+0000","last_undegraded":"2023-08-27T11:12:13.934838+0000","last_fullsized":"2023-08-27T11:12:13.934838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:50:51.397518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384498+0000","last_change":"2023-08-27T11:12:08.925080+0000","last_active":"2023-08-27T11:12:14.384498+0000","last_peered":"2023-08-27T11:12:14.384498+0000","last_clean":"2023-08-27T11:12:14.384498+0000","last_became_active":"2023-08-27T11:12:08.924784+0000","last_became_peered":"2023-08-27T11:12:08.924784+0000","last_unstale":"2023-08-27T11:12:14.384498+0000","last_undegraded":"2023-08-27T11:12:14.384498+0000","last_fullsized":"2023-08-27T11:12:14.384498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967188+0000","last_change":"2023-08-27T11:12:08.917218+0000","last_active":"2023-08-27T11:12:14.967188+0000","last_peered":"2023-08-27T11:12:14.967188+0000","last_clean":"2023-08-27T11:12:14.967188+0000","last_became_active":"2023-08-27T11:12:08.916984+0000","last_became_peered":"2023-08-27T11:12:08.916984+0000","last_unstale":"2023-08-27T11:12:14.967188+0000","last_undegraded":"2023-08-27T11:12:14.967188+0000","last_fullsized":"2023-08-27T11:12:14.967188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:02.783147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726887+0000","last_change":"2023-08-27T11:12:08.918120+0000","last_active":"2023-08-27T11:12:14.726887+0000","last_peered":"2023-08-27T11:12:14.726887+0000","last_clean":"2023-08-27T11:12:14.726887+0000","last_became_active":"2023-08-27T11:12:08.917811+0000","last_became_peered":"2023-08-27T11:12:08.917811+0000","last_unstale":"2023-08-27T11:12:14.726887+0000","last_undegraded":"2023-08-27T11:12:14.726887+0000","last_fullsized":"2023-08-27T11:12:14.726887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:16.000827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967968+0000","last_change":"2023-08-27T11:12:08.912850+0000","last_active":"2023-08-27T11:12:14.967968+0000","last_peered":"2023-08-27T11:12:14.967968+0000","last_clean":"2023-08-27T11:12:14.967968+0000","last_became_active":"2023-08-27T11:12:08.912458+0000","last_became_peered":"2023-08-27T11:12:08.912458+0000","last_unstale":"2023-08-27T11:12:14.967968+0000","last_undegraded":"2023-08-27T11:12:14.967968+0000","last_fullsized":"2023-08-27T11:12:14.967968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:49.762265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933954+0000","last_change":"2023-08-27T11:12:08.922288+0000","last_active":"2023-08-27T11:12:13.933954+0000","last_peered":"2023-08-27T11:12:13.933954+0000","last_clean":"2023-08-27T11:12:13.933954+0000","last_became_active":"2023-08-27T11:12:08.921531+0000","last_became_peered":"2023-08-27T11:12:08.921531+0000","last_unstale":"2023-08-27T11:12:13.933954+0000","last_undegraded":"2023-08-27T11:12:13.933954+0000","last_fullsized":"2023-08-27T11:12:13.933954+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:20.157877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933160+0000","last_change":"2023-08-27T11:12:08.911702+0000","last_active":"2023-08-27T11:12:13.933160+0000","last_peered":"2023-08-27T11:12:13.933160+0000","last_clean":"2023-08-27T11:12:13.933160+0000","last_became_active":"2023-08-27T11:12:08.911391+0000","last_became_peered":"2023-08-27T11:12:08.911391+0000","last_unstale":"2023-08-27T11:12:13.933160+0000","last_undegraded":"2023-08-27T11:12:13.933160+0000","last_fullsized":"2023-08-27T11:12:13.933160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:53.589324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932909+0000","last_change":"2023-08-27T11:12:08.914165+0000","last_active":"2023-08-27T11:12:13.932909+0000","last_peered":"2023-08-27T11:12:13.932909+0000","last_clean":"2023-08-27T11:12:13.932909+0000","last_became_active":"2023-08-27T11:12:08.913886+0000","last_became_peered":"2023-08-27T11:12:08.913886+0000","last_unstale":"2023-08-27T11:12:13.932909+0000","last_undegraded":"2023-08-27T11:12:13.932909+0000","last_fullsized":"2023-08-27T11:12:13.932909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:12.238621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967356+0000","last_change":"2023-08-27T11:12:08.911899+0000","last_active":"2023-08-27T11:12:14.967356+0000","last_peered":"2023-08-27T11:12:14.967356+0000","last_clean":"2023-08-27T11:12:14.967356+0000","last_became_active":"2023-08-27T11:12:08.911620+0000","last_became_peered":"2023-08-27T11:12:08.911620+0000","last_unstale":"2023-08-27T11:12:14.967356+0000","last_undegraded":"2023-08-27T11:12:14.967356+0000","last_fullsized":"2023-08-27T11:12:14.967356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:07.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384118+0000","last_change":"2023-08-27T11:12:08.925181+0000","last_active":"2023-08-27T11:12:14.384118+0000","last_peered":"2023-08-27T11:12:14.384118+0000","last_clean":"2023-08-27T11:12:14.384118+0000","last_became_active":"2023-08-27T11:12:08.924909+0000","last_became_peered":"2023-08-27T11:12:08.924909+0000","last_unstale":"2023-08-27T11:12:14.384118+0000","last_undegraded":"2023-08-27T11:12:14.384118+0000","last_fullsized":"2023-08-27T11:12:14.384118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:10.885579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934515+0000","last_change":"2023-08-27T11:12:08.911662+0000","last_active":"2023-08-27T11:12:13.934515+0000","last_peered":"2023-08-27T11:12:13.934515+0000","last_clean":"2023-08-27T11:12:13.934515+0000","last_became_active":"2023-08-27T11:12:08.911345+0000","last_became_peered":"2023-08-27T11:12:08.911345+0000","last_unstale":"2023-08-27T11:12:13.934515+0000","last_undegraded":"2023-08-27T11:12:13.934515+0000","last_fullsized":"2023-08-27T11:12:13.934515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:57.932637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726927+0000","last_change":"2023-08-27T11:12:08.916813+0000","last_active":"2023-08-27T11:12:14.726927+0000","last_peered":"2023-08-27T11:12:14.726927+0000","last_clean":"2023-08-27T11:12:14.726927+0000","last_became_active":"2023-08-27T11:12:08.916478+0000","last_became_peered":"2023-08-27T11:12:08.916478+0000","last_unstale":"2023-08-27T11:12:14.726927+0000","last_undegraded":"2023-08-27T11:12:14.726927+0000","last_fullsized":"2023-08-27T11:12:14.726927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:29.674355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933703+0000","last_change":"2023-08-27T11:12:08.907326+0000","last_active":"2023-08-27T11:12:13.933703+0000","last_peered":"2023-08-27T11:12:13.933703+0000","last_clean":"2023-08-27T11:12:13.933703+0000","last_became_active":"2023-08-27T11:12:08.907047+0000","last_became_peered":"2023-08-27T11:12:08.907047+0000","last_unstale":"2023-08-27T11:12:13.933703+0000","last_undegraded":"2023-08-27T11:12:13.933703+0000","last_fullsized":"2023-08-27T11:12:13.933703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:32:17.983727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932728+0000","last_change":"2023-08-27T11:12:08.919764+0000","last_active":"2023-08-27T11:12:13.932728+0000","last_peered":"2023-08-27T11:12:13.932728+0000","last_clean":"2023-08-27T11:12:13.932728+0000","last_became_active":"2023-08-27T11:12:08.914867+0000","last_became_peered":"2023-08-27T11:12:08.914867+0000","last_unstale":"2023-08-27T11:12:13.932728+0000","last_undegraded":"2023-08-27T11:12:13.932728+0000","last_fullsized":"2023-08-27T11:12:13.932728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:05.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933919+0000","last_change":"2023-08-27T11:12:08.922111+0000","last_active":"2023-08-27T11:12:13.933919+0000","last_peered":"2023-08-27T11:12:13.933919+0000","last_clean":"2023-08-27T11:12:13.933919+0000","last_became_active":"2023-08-27T11:12:08.917591+0000","last_became_peered":"2023-08-27T11:12:08.917591+0000","last_unstale":"2023-08-27T11:12:13.933919+0000","last_undegraded":"2023-08-27T11:12:13.933919+0000","last_fullsized":"2023-08-27T11:12:13.933919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:59.881615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384562+0000","last_change":"2023-08-27T11:12:08.931238+0000","last_active":"2023-08-27T11:12:14.384562+0000","last_peered":"2023-08-27T11:12:14.384562+0000","last_clean":"2023-08-27T11:12:14.384562+0000","last_became_active":"2023-08-27T11:12:08.930972+0000","last_became_peered":"2023-08-27T11:12:08.930972+0000","last_unstale":"2023-08-27T11:12:14.384562+0000","last_undegraded":"2023-08-27T11:12:14.384562+0000","last_fullsized":"2023-08-27T11:12:14.384562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:41.570266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726461+0000","last_change":"2023-08-27T11:12:08.919835+0000","last_active":"2023-08-27T11:12:14.726461+0000","last_peered":"2023-08-27T11:12:14.726461+0000","last_clean":"2023-08-27T11:12:14.726461+0000","last_became_active":"2023-08-27T11:12:08.919439+0000","last_became_peered":"2023-08-27T11:12:08.919439+0000","last_unstale":"2023-08-27T11:12:14.726461+0000","last_undegraded":"2023-08-27T11:12:14.726461+0000","last_fullsized":"2023-08-27T11:12:14.726461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:25:20.526417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982143+0000","last_change":"2023-08-27T11:12:08.913113+0000","last_active":"2023-08-27T11:12:14.982143+0000","last_peered":"2023-08-27T11:12:14.982143+0000","last_clean":"2023-08-27T11:12:14.982143+0000","last_became_active":"2023-08-27T11:12:08.912792+0000","last_became_peered":"2023-08-27T11:12:08.912792+0000","last_unstale":"2023-08-27T11:12:14.982143+0000","last_undegraded":"2023-08-27T11:12:14.982143+0000","last_fullsized":"2023-08-27T11:12:14.982143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:53.041694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384187+0000","last_change":"2023-08-27T11:12:08.926432+0000","last_active":"2023-08-27T11:12:14.384187+0000","last_peered":"2023-08-27T11:12:14.384187+0000","last_clean":"2023-08-27T11:12:14.384187+0000","last_became_active":"2023-08-27T11:12:08.926137+0000","last_became_peered":"2023-08-27T11:12:08.926137+0000","last_unstale":"2023-08-27T11:12:14.384187+0000","last_undegraded":"2023-08-27T11:12:14.384187+0000","last_fullsized":"2023-08-27T11:12:14.384187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:07.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838528+0000","last_change":"2023-08-27T11:12:08.919071+0000","last_active":"2023-08-27T11:12:14.838528+0000","last_peered":"2023-08-27T11:12:14.838528+0000","last_clean":"2023-08-27T11:12:14.838528+0000","last_became_active":"2023-08-27T11:12:08.918772+0000","last_became_peered":"2023-08-27T11:12:08.918772+0000","last_unstale":"2023-08-27T11:12:14.838528+0000","last_undegraded":"2023-08-27T11:12:14.838528+0000","last_fullsized":"2023-08-27T11:12:14.838528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:48.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838629+0000","last_change":"2023-08-27T11:12:08.911583+0000","last_active":"2023-08-27T11:12:14.838629+0000","last_peered":"2023-08-27T11:12:14.838629+0000","last_clean":"2023-08-27T11:12:14.838629+0000","last_became_active":"2023-08-27T11:12:08.911295+0000","last_became_peered":"2023-08-27T11:12:08.911295+0000","last_unstale":"2023-08-27T11:12:14.838629+0000","last_undegraded":"2023-08-27T11:12:14.838629+0000","last_fullsized":"2023-08-27T11:12:14.838629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:22.748117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967776+0000","last_change":"2023-08-27T11:12:08.931465+0000","last_active":"2023-08-27T11:12:14.967776+0000","last_peered":"2023-08-27T11:12:14.967776+0000","last_clean":"2023-08-27T11:12:14.967776+0000","last_became_active":"2023-08-27T11:12:08.930966+0000","last_became_peered":"2023-08-27T11:12:08.930966+0000","last_unstale":"2023-08-27T11:12:14.967776+0000","last_undegraded":"2023-08-27T11:12:14.967776+0000","last_fullsized":"2023-08-27T11:12:14.967776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:41.570266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934397+0000","last_change":"2023-08-27T11:12:12.959219+0000","last_active":"2023-08-27T11:12:13.934397+0000","last_peered":"2023-08-27T11:12:13.934397+0000","last_clean":"2023-08-27T11:12:13.934397+0000","last_became_active":"2023-08-27T11:12:12.958817+0000","last_became_peered":"2023-08-27T11:12:12.958817+0000","last_unstale":"2023-08-27T11:12:13.934397+0000","last_undegraded":"2023-08-27T11:12:13.934397+0000","last_fullsized":"2023-08-27T11:12:13.934397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:44.882336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933845+0000","last_change":"2023-08-27T11:12:07.903534+0000","last_active":"2023-08-27T11:12:13.933845+0000","last_peered":"2023-08-27T11:12:13.933845+0000","last_clean":"2023-08-27T11:12:13.933845+0000","last_became_active":"2023-08-27T11:12:07.903216+0000","last_became_peered":"2023-08-27T11:12:07.903216+0000","last_unstale":"2023-08-27T11:12:13.933845+0000","last_undegraded":"2023-08-27T11:12:13.933845+0000","last_fullsized":"2023-08-27T11:12:13.933845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:35:15.612642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982446+0000","last_change":"2023-08-27T11:12:08.923471+0000","last_active":"2023-08-27T11:12:14.982446+0000","last_peered":"2023-08-27T11:12:14.982446+0000","last_clean":"2023-08-27T11:12:14.982446+0000","last_became_active":"2023-08-27T11:12:08.918459+0000","last_became_peered":"2023-08-27T11:12:08.918459+0000","last_unstale":"2023-08-27T11:12:14.982446+0000","last_undegraded":"2023-08-27T11:12:14.982446+0000","last_fullsized":"2023-08-27T11:12:14.982446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934474+0000","last_change":"2023-08-27T11:12:12.953626+0000","last_active":"2023-08-27T11:12:13.934474+0000","last_peered":"2023-08-27T11:12:13.934474+0000","last_clean":"2023-08-27T11:12:13.934474+0000","last_became_active":"2023-08-27T11:12:12.953231+0000","last_became_peered":"2023-08-27T11:12:12.953231+0000","last_unstale":"2023-08-27T11:12:13.934474+0000","last_undegraded":"2023-08-27T11:12:13.934474+0000","last_fullsized":"2023-08-27T11:12:13.934474+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:57.631242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837663+0000","last_change":"2023-08-27T11:12:07.895584+0000","last_active":"2023-08-27T11:12:14.837663+0000","last_peered":"2023-08-27T11:12:14.837663+0000","last_clean":"2023-08-27T11:12:14.837663+0000","last_became_active":"2023-08-27T11:12:07.895276+0000","last_became_peered":"2023-08-27T11:12:07.895276+0000","last_unstale":"2023-08-27T11:12:14.837663+0000","last_undegraded":"2023-08-27T11:12:14.837663+0000","last_fullsized":"2023-08-27T11:12:14.837663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:44.245501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726897+0000","last_change":"2023-08-27T11:12:08.928505+0000","last_active":"2023-08-27T11:12:14.726897+0000","last_peered":"2023-08-27T11:12:14.726897+0000","last_clean":"2023-08-27T11:12:14.726897+0000","last_became_active":"2023-08-27T11:12:08.926233+0000","last_became_peered":"2023-08-27T11:12:08.926233+0000","last_unstale":"2023-08-27T11:12:14.726897+0000","last_undegraded":"2023-08-27T11:12:14.726897+0000","last_fullsized":"2023-08-27T11:12:14.726897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:17.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727088+0000","last_change":"2023-08-27T11:12:12.962888+0000","last_active":"2023-08-27T11:12:14.727088+0000","last_peered":"2023-08-27T11:12:14.727088+0000","last_clean":"2023-08-27T11:12:14.727088+0000","last_became_active":"2023-08-27T11:12:12.962501+0000","last_became_peered":"2023-08-27T11:12:12.962501+0000","last_unstale":"2023-08-27T11:12:14.727088+0000","last_undegraded":"2023-08-27T11:12:14.727088+0000","last_fullsized":"2023-08-27T11:12:14.727088+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:06.825066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982310+0000","last_change":"2023-08-27T11:12:07.898500+0000","last_active":"2023-08-27T11:12:14.982310+0000","last_peered":"2023-08-27T11:12:14.982310+0000","last_clean":"2023-08-27T11:12:14.982310+0000","last_became_active":"2023-08-27T11:12:07.898197+0000","last_became_peered":"2023-08-27T11:12:07.898197+0000","last_unstale":"2023-08-27T11:12:14.982310+0000","last_undegraded":"2023-08-27T11:12:14.982310+0000","last_fullsized":"2023-08-27T11:12:14.982310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:39.541198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934586+0000","last_change":"2023-08-27T11:12:08.925037+0000","last_active":"2023-08-27T11:12:13.934586+0000","last_peered":"2023-08-27T11:12:13.934586+0000","last_clean":"2023-08-27T11:12:13.934586+0000","last_became_active":"2023-08-27T11:12:08.924740+0000","last_became_peered":"2023-08-27T11:12:08.924740+0000","last_unstale":"2023-08-27T11:12:13.934586+0000","last_undegraded":"2023-08-27T11:12:13.934586+0000","last_fullsized":"2023-08-27T11:12:13.934586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:21:21.628536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933622+0000","last_change":"2023-08-27T11:12:12.956101+0000","last_active":"2023-08-27T11:12:13.933622+0000","last_peered":"2023-08-27T11:12:13.933622+0000","last_clean":"2023-08-27T11:12:13.933622+0000","last_became_active":"2023-08-27T11:12:12.952220+0000","last_became_peered":"2023-08-27T11:12:12.952220+0000","last_unstale":"2023-08-27T11:12:13.933622+0000","last_undegraded":"2023-08-27T11:12:13.933622+0000","last_fullsized":"2023-08-27T11:12:13.933622+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:16.280540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727033+0000","last_change":"2023-08-27T11:12:07.902491+0000","last_active":"2023-08-27T11:12:14.727033+0000","last_peered":"2023-08-27T11:12:14.727033+0000","last_clean":"2023-08-27T11:12:14.727033+0000","last_became_active":"2023-08-27T11:12:07.902083+0000","last_became_peered":"2023-08-27T11:12:07.902083+0000","last_unstale":"2023-08-27T11:12:14.727033+0000","last_undegraded":"2023-08-27T11:12:14.727033+0000","last_fullsized":"2023-08-27T11:12:14.727033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:19.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982268+0000","last_change":"2023-08-27T11:12:08.918017+0000","last_active":"2023-08-27T11:12:14.982268+0000","last_peered":"2023-08-27T11:12:14.982268+0000","last_clean":"2023-08-27T11:12:14.982268+0000","last_became_active":"2023-08-27T11:12:08.917728+0000","last_became_peered":"2023-08-27T11:12:08.917728+0000","last_unstale":"2023-08-27T11:12:14.982268+0000","last_undegraded":"2023-08-27T11:12:14.982268+0000","last_fullsized":"2023-08-27T11:12:14.982268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:51.458617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384060+0000","last_change":"2023-08-27T11:12:12.958663+0000","last_active":"2023-08-27T11:12:14.384060+0000","last_peered":"2023-08-27T11:12:14.384060+0000","last_clean":"2023-08-27T11:12:14.384060+0000","last_became_active":"2023-08-27T11:12:12.958275+0000","last_became_peered":"2023-08-27T11:12:12.958275+0000","last_unstale":"2023-08-27T11:12:14.384060+0000","last_undegraded":"2023-08-27T11:12:14.384060+0000","last_fullsized":"2023-08-27T11:12:14.384060+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:16.222119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967840+0000","last_change":"2023-08-27T11:12:07.909123+0000","last_active":"2023-08-27T11:12:14.967840+0000","last_peered":"2023-08-27T11:12:14.967840+0000","last_clean":"2023-08-27T11:12:14.967840+0000","last_became_active":"2023-08-27T11:12:07.908788+0000","last_became_peered":"2023-08-27T11:12:07.908788+0000","last_unstale":"2023-08-27T11:12:14.967840+0000","last_undegraded":"2023-08-27T11:12:14.967840+0000","last_fullsized":"2023-08-27T11:12:14.967840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:40.858637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982172+0000","last_change":"2023-08-27T11:12:08.927810+0000","last_active":"2023-08-27T11:12:14.982172+0000","last_peered":"2023-08-27T11:12:14.982172+0000","last_clean":"2023-08-27T11:12:14.982172+0000","last_became_active":"2023-08-27T11:12:08.927231+0000","last_became_peered":"2023-08-27T11:12:08.927231+0000","last_unstale":"2023-08-27T11:12:14.982172+0000","last_undegraded":"2023-08-27T11:12:14.982172+0000","last_fullsized":"2023-08-27T11:12:14.982172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:22.748117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982828+0000","last_change":"2023-08-27T11:12:12.954999+0000","last_active":"2023-08-27T11:12:14.982828+0000","last_peered":"2023-08-27T11:12:14.982828+0000","last_clean":"2023-08-27T11:12:14.982828+0000","last_became_active":"2023-08-27T11:12:12.954572+0000","last_became_peered":"2023-08-27T11:12:12.954572+0000","last_unstale":"2023-08-27T11:12:14.982828+0000","last_undegraded":"2023-08-27T11:12:14.982828+0000","last_fullsized":"2023-08-27T11:12:14.982828+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:38.482531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384847+0000","last_change":"2023-08-27T11:12:07.908859+0000","last_active":"2023-08-27T11:12:14.384847+0000","last_peered":"2023-08-27T11:12:14.384847+0000","last_clean":"2023-08-27T11:12:14.384847+0000","last_became_active":"2023-08-27T11:12:07.908605+0000","last_became_peered":"2023-08-27T11:12:07.908605+0000","last_unstale":"2023-08-27T11:12:14.384847+0000","last_undegraded":"2023-08-27T11:12:14.384847+0000","last_fullsized":"2023-08-27T11:12:14.384847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:20.463561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838554+0000","last_change":"2023-08-27T11:12:08.925960+0000","last_active":"2023-08-27T11:12:14.838554+0000","last_peered":"2023-08-27T11:12:14.838554+0000","last_clean":"2023-08-27T11:12:14.838554+0000","last_became_active":"2023-08-27T11:12:08.925641+0000","last_became_peered":"2023-08-27T11:12:08.925641+0000","last_unstale":"2023-08-27T11:12:14.838554+0000","last_undegraded":"2023-08-27T11:12:14.838554+0000","last_fullsized":"2023-08-27T11:12:14.838554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:53.041694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837918+0000","last_change":"2023-08-27T11:12:12.949220+0000","last_active":"2023-08-27T11:12:14.837918+0000","last_peered":"2023-08-27T11:12:14.837918+0000","last_clean":"2023-08-27T11:12:14.837918+0000","last_became_active":"2023-08-27T11:12:12.948926+0000","last_became_peered":"2023-08-27T11:12:12.948926+0000","last_unstale":"2023-08-27T11:12:14.837918+0000","last_undegraded":"2023-08-27T11:12:14.837918+0000","last_fullsized":"2023-08-27T11:12:14.837918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:20.042745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'10","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:24.935040+0000","last_change":"2023-08-27T11:12:07.908014+0000","last_active":"2023-08-27T11:12:24.935040+0000","last_peered":"2023-08-27T11:12:24.935040+0000","last_clean":"2023-08-27T11:12:24.935040+0000","last_became_active":"2023-08-27T11:12:07.907722+0000","last_became_peered":"2023-08-27T11:12:07.907722+0000","last_unstale":"2023-08-27T11:12:24.935040+0000","last_undegraded":"2023-08-27T11:12:24.935040+0000","last_fullsized":"2023-08-27T11:12:24.935040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:11.313074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933433+0000","last_change":"2023-08-27T11:12:08.923838+0000","last_active":"2023-08-27T11:12:13.933433+0000","last_peered":"2023-08-27T11:12:13.933433+0000","last_clean":"2023-08-27T11:12:13.933433+0000","last_became_active":"2023-08-27T11:12:08.919224+0000","last_became_peered":"2023-08-27T11:12:08.919224+0000","last_unstale":"2023-08-27T11:12:13.933433+0000","last_undegraded":"2023-08-27T11:12:13.933433+0000","last_fullsized":"2023-08-27T11:12:13.933433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:49.585442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967969+0000","last_change":"2023-08-27T11:12:12.953888+0000","last_active":"2023-08-27T11:12:14.967969+0000","last_peered":"2023-08-27T11:12:14.967969+0000","last_clean":"2023-08-27T11:12:14.967969+0000","last_became_active":"2023-08-27T11:12:12.953499+0000","last_became_peered":"2023-08-27T11:12:12.953499+0000","last_unstale":"2023-08-27T11:12:14.967969+0000","last_undegraded":"2023-08-27T11:12:14.967969+0000","last_fullsized":"2023-08-27T11:12:14.967969+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:42.319064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384452+0000","last_change":"2023-08-27T11:12:07.907713+0000","last_active":"2023-08-27T11:12:14.384452+0000","last_peered":"2023-08-27T11:12:14.384452+0000","last_clean":"2023-08-27T11:12:14.384452+0000","last_became_active":"2023-08-27T11:12:07.907429+0000","last_became_peered":"2023-08-27T11:12:07.907429+0000","last_unstale":"2023-08-27T11:12:14.384452+0000","last_undegraded":"2023-08-27T11:12:14.384452+0000","last_fullsized":"2023-08-27T11:12:14.384452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:29.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982602+0000","last_change":"2023-08-27T11:12:08.923764+0000","last_active":"2023-08-27T11:12:14.982602+0000","last_peered":"2023-08-27T11:12:14.982602+0000","last_clean":"2023-08-27T11:12:14.982602+0000","last_became_active":"2023-08-27T11:12:08.923333+0000","last_became_peered":"2023-08-27T11:12:08.923333+0000","last_unstale":"2023-08-27T11:12:14.982602+0000","last_undegraded":"2023-08-27T11:12:14.982602+0000","last_fullsized":"2023-08-27T11:12:14.982602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:41.863238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982543+0000","last_change":"2023-08-27T11:12:12.948620+0000","last_active":"2023-08-27T11:12:14.982543+0000","last_peered":"2023-08-27T11:12:14.982543+0000","last_clean":"2023-08-27T11:12:14.982543+0000","last_became_active":"2023-08-27T11:12:12.948161+0000","last_became_peered":"2023-08-27T11:12:12.948161+0000","last_unstale":"2023-08-27T11:12:14.982543+0000","last_undegraded":"2023-08-27T11:12:14.982543+0000","last_fullsized":"2023-08-27T11:12:14.982543+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:25.510081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726415+0000","last_change":"2023-08-27T11:12:07.905444+0000","last_active":"2023-08-27T11:12:14.726415+0000","last_peered":"2023-08-27T11:12:14.726415+0000","last_clean":"2023-08-27T11:12:14.726415+0000","last_became_active":"2023-08-27T11:12:07.905102+0000","last_became_peered":"2023-08-27T11:12:07.905102+0000","last_unstale":"2023-08-27T11:12:14.726415+0000","last_undegraded":"2023-08-27T11:12:14.726415+0000","last_fullsized":"2023-08-27T11:12:14.726415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:59.453384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967752+0000","last_change":"2023-08-27T11:12:08.931630+0000","last_active":"2023-08-27T11:12:14.967752+0000","last_peered":"2023-08-27T11:12:14.967752+0000","last_clean":"2023-08-27T11:12:14.967752+0000","last_became_active":"2023-08-27T11:12:08.931239+0000","last_became_peered":"2023-08-27T11:12:08.931239+0000","last_unstale":"2023-08-27T11:12:14.967752+0000","last_undegraded":"2023-08-27T11:12:14.967752+0000","last_fullsized":"2023-08-27T11:12:14.967752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982960+0000","last_change":"2023-08-27T11:12:12.954525+0000","last_active":"2023-08-27T11:12:14.982960+0000","last_peered":"2023-08-27T11:12:14.982960+0000","last_clean":"2023-08-27T11:12:14.982960+0000","last_became_active":"2023-08-27T11:12:12.954077+0000","last_became_peered":"2023-08-27T11:12:12.954077+0000","last_unstale":"2023-08-27T11:12:14.982960+0000","last_undegraded":"2023-08-27T11:12:14.982960+0000","last_fullsized":"2023-08-27T11:12:14.982960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:07:06.449989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384221+0000","last_change":"2023-08-27T11:12:07.907504+0000","last_active":"2023-08-27T11:12:14.384221+0000","last_peered":"2023-08-27T11:12:14.384221+0000","last_clean":"2023-08-27T11:12:14.384221+0000","last_became_active":"2023-08-27T11:12:07.907206+0000","last_became_peered":"2023-08-27T11:12:07.907206+0000","last_unstale":"2023-08-27T11:12:14.384221+0000","last_undegraded":"2023-08-27T11:12:14.384221+0000","last_fullsized":"2023-08-27T11:12:14.384221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:47.521180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726958+0000","last_change":"2023-08-27T11:12:08.918640+0000","last_active":"2023-08-27T11:12:14.726958+0000","last_peered":"2023-08-27T11:12:14.726958+0000","last_clean":"2023-08-27T11:12:14.726958+0000","last_became_active":"2023-08-27T11:12:08.917846+0000","last_became_peered":"2023-08-27T11:12:08.917846+0000","last_unstale":"2023-08-27T11:12:14.726958+0000","last_undegraded":"2023-08-27T11:12:14.726958+0000","last_fullsized":"2023-08-27T11:12:14.726958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:29:50.086138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967688+0000","last_change":"2023-08-27T11:12:12.962983+0000","last_active":"2023-08-27T11:12:14.967688+0000","last_peered":"2023-08-27T11:12:14.967688+0000","last_clean":"2023-08-27T11:12:14.967688+0000","last_became_active":"2023-08-27T11:12:12.962518+0000","last_became_peered":"2023-08-27T11:12:12.962518+0000","last_unstale":"2023-08-27T11:12:14.967688+0000","last_undegraded":"2023-08-27T11:12:14.967688+0000","last_fullsized":"2023-08-27T11:12:14.967688+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:34.829806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932864+0000","last_change":"2023-08-27T11:12:07.905529+0000","last_active":"2023-08-27T11:12:13.932864+0000","last_peered":"2023-08-27T11:12:13.932864+0000","last_clean":"2023-08-27T11:12:13.932864+0000","last_became_active":"2023-08-27T11:12:07.905203+0000","last_became_peered":"2023-08-27T11:12:07.905203+0000","last_unstale":"2023-08-27T11:12:13.932864+0000","last_undegraded":"2023-08-27T11:12:13.932864+0000","last_fullsized":"2023-08-27T11:12:13.932864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:54.273519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.983012+0000","last_change":"2023-08-27T11:12:08.926915+0000","last_active":"2023-08-27T11:12:14.983012+0000","last_peered":"2023-08-27T11:12:14.983012+0000","last_clean":"2023-08-27T11:12:14.983012+0000","last_became_active":"2023-08-27T11:12:08.923355+0000","last_became_peered":"2023-08-27T11:12:08.923355+0000","last_unstale":"2023-08-27T11:12:14.983012+0000","last_undegraded":"2023-08-27T11:12:14.983012+0000","last_fullsized":"2023-08-27T11:12:14.983012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:12.215932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384503+0000","last_change":"2023-08-27T11:12:12.958000+0000","last_active":"2023-08-27T11:12:14.384503+0000","last_peered":"2023-08-27T11:12:14.384503+0000","last_clean":"2023-08-27T11:12:14.384503+0000","last_became_active":"2023-08-27T11:12:12.957611+0000","last_became_peered":"2023-08-27T11:12:12.957611+0000","last_unstale":"2023-08-27T11:12:14.384503+0000","last_undegraded":"2023-08-27T11:12:14.384503+0000","last_fullsized":"2023-08-27T11:12:14.384503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:44.587717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384852+0000","last_change":"2023-08-27T11:12:07.907385+0000","last_active":"2023-08-27T11:12:14.384852+0000","last_peered":"2023-08-27T11:12:14.384852+0000","last_clean":"2023-08-27T11:12:14.384852+0000","last_became_active":"2023-08-27T11:12:07.907116+0000","last_became_peered":"2023-08-27T11:12:07.907116+0000","last_unstale":"2023-08-27T11:12:14.384852+0000","last_undegraded":"2023-08-27T11:12:14.384852+0000","last_fullsized":"2023-08-27T11:12:14.384852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:37.272544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967084+0000","last_change":"2023-08-27T11:12:08.927597+0000","last_active":"2023-08-27T11:12:14.967084+0000","last_peered":"2023-08-27T11:12:14.967084+0000","last_clean":"2023-08-27T11:12:14.967084+0000","last_became_active":"2023-08-27T11:12:08.927306+0000","last_became_peered":"2023-08-27T11:12:08.927306+0000","last_unstale":"2023-08-27T11:12:14.967084+0000","last_undegraded":"2023-08-27T11:12:14.967084+0000","last_fullsized":"2023-08-27T11:12:14.967084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:20.782136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932501+0000","last_change":"2023-08-27T11:12:12.955644+0000","last_active":"2023-08-27T11:12:13.932501+0000","last_peered":"2023-08-27T11:12:13.932501+0000","last_clean":"2023-08-27T11:12:13.932501+0000","last_became_active":"2023-08-27T11:12:12.952322+0000","last_became_peered":"2023-08-27T11:12:12.952322+0000","last_unstale":"2023-08-27T11:12:13.932501+0000","last_undegraded":"2023-08-27T11:12:13.932501+0000","last_fullsized":"2023-08-27T11:12:13.932501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:16:14.569342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384846+0000","last_change":"2023-08-27T11:12:07.907495+0000","last_active":"2023-08-27T11:12:14.384846+0000","last_peered":"2023-08-27T11:12:14.384846+0000","last_clean":"2023-08-27T11:12:14.384846+0000","last_became_active":"2023-08-27T11:12:07.907216+0000","last_became_peered":"2023-08-27T11:12:07.907216+0000","last_unstale":"2023-08-27T11:12:14.384846+0000","last_undegraded":"2023-08-27T11:12:14.384846+0000","last_fullsized":"2023-08-27T11:12:14.384846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:40.858637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933350+0000","last_change":"2023-08-27T11:12:08.924297+0000","last_active":"2023-08-27T11:12:13.933350+0000","last_peered":"2023-08-27T11:12:13.933350+0000","last_clean":"2023-08-27T11:12:13.933350+0000","last_became_active":"2023-08-27T11:12:08.919603+0000","last_became_peered":"2023-08-27T11:12:08.919603+0000","last_unstale":"2023-08-27T11:12:13.933350+0000","last_undegraded":"2023-08-27T11:12:13.933350+0000","last_fullsized":"2023-08-27T11:12:13.933350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:20:35.025362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932779+0000","last_change":"2023-08-27T11:12:12.954662+0000","last_active":"2023-08-27T11:12:13.932779+0000","last_peered":"2023-08-27T11:12:13.932779+0000","last_clean":"2023-08-27T11:12:13.932779+0000","last_became_active":"2023-08-27T11:12:12.954250+0000","last_became_peered":"2023-08-27T11:12:12.954250+0000","last_unstale":"2023-08-27T11:12:13.932779+0000","last_undegraded":"2023-08-27T11:12:13.932779+0000","last_fullsized":"2023-08-27T11:12:13.932779+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:50.028006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384990+0000","last_change":"2023-08-27T11:12:07.907454+0000","last_active":"2023-08-27T11:12:14.384990+0000","last_peered":"2023-08-27T11:12:14.384990+0000","last_clean":"2023-08-27T11:12:14.384990+0000","last_became_active":"2023-08-27T11:12:07.907161+0000","last_became_peered":"2023-08-27T11:12:07.907161+0000","last_unstale":"2023-08-27T11:12:14.384990+0000","last_undegraded":"2023-08-27T11:12:14.384990+0000","last_fullsized":"2023-08-27T11:12:14.384990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:33.436011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933127+0000","last_change":"2023-08-27T11:12:08.920905+0000","last_active":"2023-08-27T11:12:13.933127+0000","last_peered":"2023-08-27T11:12:13.933127+0000","last_clean":"2023-08-27T11:12:13.933127+0000","last_became_active":"2023-08-27T11:12:08.920628+0000","last_became_peered":"2023-08-27T11:12:08.920628+0000","last_unstale":"2023-08-27T11:12:13.933127+0000","last_undegraded":"2023-08-27T11:12:13.933127+0000","last_fullsized":"2023-08-27T11:12:13.933127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:05.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932699+0000","last_change":"2023-08-27T11:12:12.953446+0000","last_active":"2023-08-27T11:12:13.932699+0000","last_peered":"2023-08-27T11:12:13.932699+0000","last_clean":"2023-08-27T11:12:13.932699+0000","last_became_active":"2023-08-27T11:12:12.953019+0000","last_became_peered":"2023-08-27T11:12:12.953019+0000","last_unstale":"2023-08-27T11:12:13.932699+0000","last_undegraded":"2023-08-27T11:12:13.932699+0000","last_fullsized":"2023-08-27T11:12:13.932699+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:11.191056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967365+0000","last_change":"2023-08-27T11:12:07.896745+0000","last_active":"2023-08-27T11:12:14.967365+0000","last_peered":"2023-08-27T11:12:14.967365+0000","last_clean":"2023-08-27T11:12:14.967365+0000","last_became_active":"2023-08-27T11:12:07.896471+0000","last_became_peered":"2023-08-27T11:12:07.896471+0000","last_unstale":"2023-08-27T11:12:14.967365+0000","last_undegraded":"2023-08-27T11:12:14.967365+0000","last_fullsized":"2023-08-27T11:12:14.967365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:09.470216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726539+0000","last_change":"2023-08-27T11:12:08.918280+0000","last_active":"2023-08-27T11:12:14.726539+0000","last_peered":"2023-08-27T11:12:14.726539+0000","last_clean":"2023-08-27T11:12:14.726539+0000","last_became_active":"2023-08-27T11:12:08.917786+0000","last_became_peered":"2023-08-27T11:12:08.917786+0000","last_unstale":"2023-08-27T11:12:14.726539+0000","last_undegraded":"2023-08-27T11:12:14.726539+0000","last_fullsized":"2023-08-27T11:12:14.726539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:45.655952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932983+0000","last_change":"2023-08-27T11:12:12.957044+0000","last_active":"2023-08-27T11:12:13.932983+0000","last_peered":"2023-08-27T11:12:13.932983+0000","last_clean":"2023-08-27T11:12:13.932983+0000","last_became_active":"2023-08-27T11:12:12.953571+0000","last_became_peered":"2023-08-27T11:12:12.953571+0000","last_unstale":"2023-08-27T11:12:13.932983+0000","last_undegraded":"2023-08-27T11:12:13.932983+0000","last_fullsized":"2023-08-27T11:12:13.932983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:42.930026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982654+0000","last_change":"2023-08-27T11:12:07.899209+0000","last_active":"2023-08-27T11:12:14.982654+0000","last_peered":"2023-08-27T11:12:14.982654+0000","last_clean":"2023-08-27T11:12:14.982654+0000","last_became_active":"2023-08-27T11:12:07.898806+0000","last_became_peered":"2023-08-27T11:12:07.898806+0000","last_unstale":"2023-08-27T11:12:14.982654+0000","last_undegraded":"2023-08-27T11:12:14.982654+0000","last_fullsized":"2023-08-27T11:12:14.982654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:37.272544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933574+0000","last_change":"2023-08-27T11:12:08.919614+0000","last_active":"2023-08-27T11:12:13.933574+0000","last_peered":"2023-08-27T11:12:13.933574+0000","last_clean":"2023-08-27T11:12:13.933574+0000","last_became_active":"2023-08-27T11:12:08.916338+0000","last_became_peered":"2023-08-27T11:12:08.916338+0000","last_unstale":"2023-08-27T11:12:13.933574+0000","last_undegraded":"2023-08-27T11:12:13.933574+0000","last_fullsized":"2023-08-27T11:12:13.933574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:49:01.974112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838031+0000","last_change":"2023-08-27T11:12:12.962859+0000","last_active":"2023-08-27T11:12:14.838031+0000","last_peered":"2023-08-27T11:12:14.838031+0000","last_clean":"2023-08-27T11:12:14.838031+0000","last_became_active":"2023-08-27T11:12:12.962421+0000","last_became_peered":"2023-08-27T11:12:12.962421+0000","last_unstale":"2023-08-27T11:12:14.838031+0000","last_undegraded":"2023-08-27T11:12:14.838031+0000","last_fullsized":"2023-08-27T11:12:14.838031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:21.753280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933251+0000","last_change":"2023-08-27T11:12:07.901923+0000","last_active":"2023-08-27T11:12:13.933251+0000","last_peered":"2023-08-27T11:12:13.933251+0000","last_clean":"2023-08-27T11:12:13.933251+0000","last_became_active":"2023-08-27T11:12:07.901634+0000","last_became_peered":"2023-08-27T11:12:07.901634+0000","last_unstale":"2023-08-27T11:12:13.933251+0000","last_undegraded":"2023-08-27T11:12:13.933251+0000","last_fullsized":"2023-08-27T11:12:13.933251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:44.981487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933505+0000","last_change":"2023-08-27T11:12:08.911380+0000","last_active":"2023-08-27T11:12:13.933505+0000","last_peered":"2023-08-27T11:12:13.933505+0000","last_clean":"2023-08-27T11:12:13.933505+0000","last_became_active":"2023-08-27T11:12:08.911077+0000","last_became_peered":"2023-08-27T11:12:08.911077+0000","last_unstale":"2023-08-27T11:12:13.933505+0000","last_undegraded":"2023-08-27T11:12:13.933505+0000","last_fullsized":"2023-08-27T11:12:13.933505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:59.416171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967787+0000","last_change":"2023-08-27T11:12:12.953058+0000","last_active":"2023-08-27T11:12:14.967787+0000","last_peered":"2023-08-27T11:12:14.967787+0000","last_clean":"2023-08-27T11:12:14.967787+0000","last_became_active":"2023-08-27T11:12:12.952571+0000","last_became_peered":"2023-08-27T11:12:12.952571+0000","last_unstale":"2023-08-27T11:12:14.967787+0000","last_undegraded":"2023-08-27T11:12:14.967787+0000","last_fullsized":"2023-08-27T11:12:14.967787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:25.510081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933016+0000","last_change":"2023-08-27T11:12:07.903262+0000","last_active":"2023-08-27T11:12:13.933016+0000","last_peered":"2023-08-27T11:12:13.933016+0000","last_clean":"2023-08-27T11:12:13.933016+0000","last_became_active":"2023-08-27T11:12:07.902549+0000","last_became_peered":"2023-08-27T11:12:07.902549+0000","last_unstale":"2023-08-27T11:12:13.933016+0000","last_undegraded":"2023-08-27T11:12:13.933016+0000","last_fullsized":"2023-08-27T11:12:13.933016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:25.089023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933460+0000","last_change":"2023-08-27T11:12:08.919554+0000","last_active":"2023-08-27T11:12:13.933460+0000","last_peered":"2023-08-27T11:12:13.933460+0000","last_clean":"2023-08-27T11:12:13.933460+0000","last_became_active":"2023-08-27T11:12:08.919252+0000","last_became_peered":"2023-08-27T11:12:08.919252+0000","last_unstale":"2023-08-27T11:12:13.933460+0000","last_undegraded":"2023-08-27T11:12:13.933460+0000","last_fullsized":"2023-08-27T11:12:13.933460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:14:35.778200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933262+0000","last_change":"2023-08-27T11:12:12.954063+0000","last_active":"2023-08-27T11:12:13.933262+0000","last_peered":"2023-08-27T11:12:13.933262+0000","last_clean":"2023-08-27T11:12:13.933262+0000","last_became_active":"2023-08-27T11:12:12.953652+0000","last_became_peered":"2023-08-27T11:12:12.953652+0000","last_unstale":"2023-08-27T11:12:13.933262+0000","last_undegraded":"2023-08-27T11:12:13.933262+0000","last_fullsized":"2023-08-27T11:12:13.933262+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:31.699925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982128+0000","last_change":"2023-08-27T11:12:07.902468+0000","last_active":"2023-08-27T11:12:14.982128+0000","last_peered":"2023-08-27T11:12:14.982128+0000","last_clean":"2023-08-27T11:12:14.982128+0000","last_became_active":"2023-08-27T11:12:07.902193+0000","last_became_peered":"2023-08-27T11:12:07.902193+0000","last_unstale":"2023-08-27T11:12:14.982128+0000","last_undegraded":"2023-08-27T11:12:14.982128+0000","last_fullsized":"2023-08-27T11:12:14.982128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:06.455977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838774+0000","last_change":"2023-08-27T11:12:08.921979+0000","last_active":"2023-08-27T11:12:14.838774+0000","last_peered":"2023-08-27T11:12:14.838774+0000","last_clean":"2023-08-27T11:12:14.838774+0000","last_became_active":"2023-08-27T11:12:08.921659+0000","last_became_peered":"2023-08-27T11:12:08.921659+0000","last_unstale":"2023-08-27T11:12:14.838774+0000","last_undegraded":"2023-08-27T11:12:14.838774+0000","last_fullsized":"2023-08-27T11:12:14.838774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:40.545799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726360+0000","last_change":"2023-08-27T11:12:12.959327+0000","last_active":"2023-08-27T11:12:14.726360+0000","last_peered":"2023-08-27T11:12:14.726360+0000","last_clean":"2023-08-27T11:12:14.726360+0000","last_became_active":"2023-08-27T11:12:12.957064+0000","last_became_peered":"2023-08-27T11:12:12.957064+0000","last_unstale":"2023-08-27T11:12:14.726360+0000","last_undegraded":"2023-08-27T11:12:14.726360+0000","last_fullsized":"2023-08-27T11:12:14.726360+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:20.210306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838189+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:12:14.838189+0000","last_peered":"2023-08-27T11:12:14.838189+0000","last_clean":"2023-08-27T11:12:14.838189+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:12:14.838189+0000","last_undegraded":"2023-08-27T11:12:14.838189+0000","last_fullsized":"2023-08-27T11:12:14.838189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:04.448471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967607+0000","last_change":"2023-08-27T11:12:07.902012+0000","last_active":"2023-08-27T11:12:14.967607+0000","last_peered":"2023-08-27T11:12:14.967607+0000","last_clean":"2023-08-27T11:12:14.967607+0000","last_became_active":"2023-08-27T11:12:07.901706+0000","last_became_peered":"2023-08-27T11:12:07.901706+0000","last_unstale":"2023-08-27T11:12:14.967607+0000","last_undegraded":"2023-08-27T11:12:14.967607+0000","last_fullsized":"2023-08-27T11:12:14.967607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:21.743516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838056+0000","last_change":"2023-08-27T11:12:08.916047+0000","last_active":"2023-08-27T11:12:14.838056+0000","last_peered":"2023-08-27T11:12:14.838056+0000","last_clean":"2023-08-27T11:12:14.838056+0000","last_became_active":"2023-08-27T11:12:08.915742+0000","last_became_peered":"2023-08-27T11:12:08.915742+0000","last_unstale":"2023-08-27T11:12:14.838056+0000","last_undegraded":"2023-08-27T11:12:14.838056+0000","last_fullsized":"2023-08-27T11:12:14.838056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:29.674355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726449+0000","last_change":"2023-08-27T11:12:12.952690+0000","last_active":"2023-08-27T11:12:14.726449+0000","last_peered":"2023-08-27T11:12:14.726449+0000","last_clean":"2023-08-27T11:12:14.726449+0000","last_became_active":"2023-08-27T11:12:12.952209+0000","last_became_peered":"2023-08-27T11:12:12.952209+0000","last_unstale":"2023-08-27T11:12:14.726449+0000","last_undegraded":"2023-08-27T11:12:14.726449+0000","last_fullsized":"2023-08-27T11:12:14.726449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:28.731513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967363+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:12:14.967363+0000","last_peered":"2023-08-27T11:12:14.967363+0000","last_clean":"2023-08-27T11:12:14.967363+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:12:14.967363+0000","last_undegraded":"2023-08-27T11:12:14.967363+0000","last_fullsized":"2023-08-27T11:12:14.967363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:04.859233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933791+0000","last_change":"2023-08-27T11:12:07.905363+0000","last_active":"2023-08-27T11:12:13.933791+0000","last_peered":"2023-08-27T11:12:13.933791+0000","last_clean":"2023-08-27T11:12:13.933791+0000","last_became_active":"2023-08-27T11:12:07.905002+0000","last_became_peered":"2023-08-27T11:12:07.905002+0000","last_unstale":"2023-08-27T11:12:13.933791+0000","last_undegraded":"2023-08-27T11:12:13.933791+0000","last_fullsized":"2023-08-27T11:12:13.933791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:06:55.872046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838055+0000","last_change":"2023-08-27T11:12:08.920259+0000","last_active":"2023-08-27T11:12:14.838055+0000","last_peered":"2023-08-27T11:12:14.838055+0000","last_clean":"2023-08-27T11:12:14.838055+0000","last_became_active":"2023-08-27T11:12:08.916978+0000","last_became_peered":"2023-08-27T11:12:08.916978+0000","last_unstale":"2023-08-27T11:12:14.838055+0000","last_undegraded":"2023-08-27T11:12:14.838055+0000","last_fullsized":"2023-08-27T11:12:14.838055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:29:50.086138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933007+0000","last_change":"2023-08-27T11:12:12.959735+0000","last_active":"2023-08-27T11:12:13.933007+0000","last_peered":"2023-08-27T11:12:13.933007+0000","last_clean":"2023-08-27T11:12:13.933007+0000","last_became_active":"2023-08-27T11:12:12.959275+0000","last_became_peered":"2023-08-27T11:12:12.959275+0000","last_unstale":"2023-08-27T11:12:13.933007+0000","last_undegraded":"2023-08-27T11:12:13.933007+0000","last_fullsized":"2023-08-27T11:12:13.933007+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:57.120477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932550+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:12:13.932550+0000","last_peered":"2023-08-27T11:12:13.932550+0000","last_clean":"2023-08-27T11:12:13.932550+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:12:13.932550+0000","last_undegraded":"2023-08-27T11:12:13.932550+0000","last_fullsized":"2023-08-27T11:12:13.932550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:49.251775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837761+0000","last_change":"2023-08-27T11:12:07.902482+0000","last_active":"2023-08-27T11:12:14.837761+0000","last_peered":"2023-08-27T11:12:14.837761+0000","last_clean":"2023-08-27T11:12:14.837761+0000","last_became_active":"2023-08-27T11:12:07.902184+0000","last_became_peered":"2023-08-27T11:12:07.902184+0000","last_unstale":"2023-08-27T11:12:14.837761+0000","last_undegraded":"2023-08-27T11:12:14.837761+0000","last_fullsized":"2023-08-27T11:12:14.837761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:46.293833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838666+0000","last_change":"2023-08-27T11:12:08.919949+0000","last_active":"2023-08-27T11:12:14.838666+0000","last_peered":"2023-08-27T11:12:14.838666+0000","last_clean":"2023-08-27T11:12:14.838666+0000","last_became_active":"2023-08-27T11:12:08.918796+0000","last_became_peered":"2023-08-27T11:12:08.918796+0000","last_unstale":"2023-08-27T11:12:14.838666+0000","last_undegraded":"2023-08-27T11:12:14.838666+0000","last_fullsized":"2023-08-27T11:12:14.838666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:51.458617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384661+0000","last_change":"2023-08-27T11:12:12.965405+0000","last_active":"2023-08-27T11:12:14.384661+0000","last_peered":"2023-08-27T11:12:14.384661+0000","last_clean":"2023-08-27T11:12:14.384661+0000","last_became_active":"2023-08-27T11:12:12.965085+0000","last_became_peered":"2023-08-27T11:12:12.965085+0000","last_unstale":"2023-08-27T11:12:14.384661+0000","last_undegraded":"2023-08-27T11:12:14.384661+0000","last_fullsized":"2023-08-27T11:12:14.384661+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:45.516388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967468+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:12:14.967468+0000","last_peered":"2023-08-27T11:12:14.967468+0000","last_clean":"2023-08-27T11:12:14.967468+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:12:14.967468+0000","last_undegraded":"2023-08-27T11:12:14.967468+0000","last_fullsized":"2023-08-27T11:12:14.967468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:47.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384165+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:12:14.384165+0000","last_peered":"2023-08-27T11:12:14.384165+0000","last_clean":"2023-08-27T11:12:14.384165+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:12:14.384165+0000","last_undegraded":"2023-08-27T11:12:14.384165+0000","last_fullsized":"2023-08-27T11:12:14.384165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:10.078531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982472+0000","last_change":"2023-08-27T11:12:07.903434+0000","last_active":"2023-08-27T11:12:14.982472+0000","last_peered":"2023-08-27T11:12:14.982472+0000","last_clean":"2023-08-27T11:12:14.982472+0000","last_became_active":"2023-08-27T11:12:07.903142+0000","last_became_peered":"2023-08-27T11:12:07.903142+0000","last_unstale":"2023-08-27T11:12:14.982472+0000","last_undegraded":"2023-08-27T11:12:14.982472+0000","last_fullsized":"2023-08-27T11:12:14.982472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:40.565665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934388+0000","last_change":"2023-08-27T11:12:08.915968+0000","last_active":"2023-08-27T11:12:13.934388+0000","last_peered":"2023-08-27T11:12:13.934388+0000","last_clean":"2023-08-27T11:12:13.934388+0000","last_became_active":"2023-08-27T11:12:08.915646+0000","last_became_peered":"2023-08-27T11:12:08.915646+0000","last_unstale":"2023-08-27T11:12:13.934388+0000","last_undegraded":"2023-08-27T11:12:13.934388+0000","last_fullsized":"2023-08-27T11:12:13.934388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:53.078559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967776+0000","last_change":"2023-08-27T11:12:12.953539+0000","last_active":"2023-08-27T11:12:14.967776+0000","last_peered":"2023-08-27T11:12:14.967776+0000","last_clean":"2023-08-27T11:12:14.967776+0000","last_became_active":"2023-08-27T11:12:12.953154+0000","last_became_peered":"2023-08-27T11:12:12.953154+0000","last_unstale":"2023-08-27T11:12:14.967776+0000","last_undegraded":"2023-08-27T11:12:14.967776+0000","last_fullsized":"2023-08-27T11:12:14.967776+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:45.516388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384351+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:12:14.384351+0000","last_peered":"2023-08-27T11:12:14.384351+0000","last_clean":"2023-08-27T11:12:14.384351+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:12:14.384351+0000","last_undegraded":"2023-08-27T11:12:14.384351+0000","last_fullsized":"2023-08-27T11:12:14.384351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:47.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.725986+0000","last_change":"2023-08-27T11:12:07.899116+0000","last_active":"2023-08-27T11:12:14.725986+0000","last_peered":"2023-08-27T11:12:14.725986+0000","last_clean":"2023-08-27T11:12:14.725986+0000","last_became_active":"2023-08-27T11:12:07.898805+0000","last_became_peered":"2023-08-27T11:12:07.898805+0000","last_unstale":"2023-08-27T11:12:14.725986+0000","last_undegraded":"2023-08-27T11:12:14.725986+0000","last_fullsized":"2023-08-27T11:12:14.725986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:07.168983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726764+0000","last_change":"2023-08-27T11:12:08.918851+0000","last_active":"2023-08-27T11:12:14.726764+0000","last_peered":"2023-08-27T11:12:14.726764+0000","last_clean":"2023-08-27T11:12:14.726764+0000","last_became_active":"2023-08-27T11:12:08.918248+0000","last_became_peered":"2023-08-27T11:12:08.918248+0000","last_unstale":"2023-08-27T11:12:14.726764+0000","last_undegraded":"2023-08-27T11:12:14.726764+0000","last_fullsized":"2023-08-27T11:12:14.726764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:45.250102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933527+0000","last_change":"2023-08-27T11:12:12.959470+0000","last_active":"2023-08-27T11:12:13.933527+0000","last_peered":"2023-08-27T11:12:13.933527+0000","last_clean":"2023-08-27T11:12:13.933527+0000","last_became_active":"2023-08-27T11:12:12.959082+0000","last_became_peered":"2023-08-27T11:12:12.959082+0000","last_unstale":"2023-08-27T11:12:13.933527+0000","last_undegraded":"2023-08-27T11:12:13.933527+0000","last_fullsized":"2023-08-27T11:12:13.933527+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:04.206952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384869+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:12:14.384869+0000","last_peered":"2023-08-27T11:12:14.384869+0000","last_clean":"2023-08-27T11:12:14.384869+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:12:14.384869+0000","last_undegraded":"2023-08-27T11:12:14.384869+0000","last_fullsized":"2023-08-27T11:12:14.384869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:43.735919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838283+0000","last_change":"2023-08-27T11:12:07.903460+0000","last_active":"2023-08-27T11:12:14.838283+0000","last_peered":"2023-08-27T11:12:14.838283+0000","last_clean":"2023-08-27T11:12:14.838283+0000","last_became_active":"2023-08-27T11:12:07.903110+0000","last_became_peered":"2023-08-27T11:12:07.903110+0000","last_unstale":"2023-08-27T11:12:14.838283+0000","last_undegraded":"2023-08-27T11:12:14.838283+0000","last_fullsized":"2023-08-27T11:12:14.838283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:09.880978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726625+0000","last_change":"2023-08-27T11:12:08.913520+0000","last_active":"2023-08-27T11:12:14.726625+0000","last_peered":"2023-08-27T11:12:14.726625+0000","last_clean":"2023-08-27T11:12:14.726625+0000","last_became_active":"2023-08-27T11:12:08.913192+0000","last_became_peered":"2023-08-27T11:12:08.913192+0000","last_unstale":"2023-08-27T11:12:14.726625+0000","last_undegraded":"2023-08-27T11:12:14.726625+0000","last_fullsized":"2023-08-27T11:12:14.726625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:12:06.235129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933906+0000","last_change":"2023-08-27T11:12:12.959458+0000","last_active":"2023-08-27T11:12:13.933906+0000","last_peered":"2023-08-27T11:12:13.933906+0000","last_clean":"2023-08-27T11:12:13.933906+0000","last_became_active":"2023-08-27T11:12:12.959073+0000","last_became_peered":"2023-08-27T11:12:12.959073+0000","last_unstale":"2023-08-27T11:12:13.933906+0000","last_undegraded":"2023-08-27T11:12:13.933906+0000","last_fullsized":"2023-08-27T11:12:13.933906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:37:24.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.983100+0000","last_change":"2023-08-27T11:12:07.908908+0000","last_active":"2023-08-27T11:12:14.983100+0000","last_peered":"2023-08-27T11:12:14.983100+0000","last_clean":"2023-08-27T11:12:14.983100+0000","last_became_active":"2023-08-27T11:12:07.908670+0000","last_became_peered":"2023-08-27T11:12:07.908670+0000","last_unstale":"2023-08-27T11:12:14.983100+0000","last_undegraded":"2023-08-27T11:12:14.983100+0000","last_fullsized":"2023-08-27T11:12:14.983100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:06:59.757966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838348+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:12:14.838348+0000","last_peered":"2023-08-27T11:12:14.838348+0000","last_clean":"2023-08-27T11:12:14.838348+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:12:14.838348+0000","last_undegraded":"2023-08-27T11:12:14.838348+0000","last_fullsized":"2023-08-27T11:12:14.838348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:01.434232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982129+0000","last_change":"2023-08-27T11:12:08.917056+0000","last_active":"2023-08-27T11:12:14.982129+0000","last_peered":"2023-08-27T11:12:14.982129+0000","last_clean":"2023-08-27T11:12:14.982129+0000","last_became_active":"2023-08-27T11:12:08.916853+0000","last_became_peered":"2023-08-27T11:12:08.916853+0000","last_unstale":"2023-08-27T11:12:14.982129+0000","last_undegraded":"2023-08-27T11:12:14.982129+0000","last_fullsized":"2023-08-27T11:12:14.982129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:48.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982032+0000","last_change":"2023-08-27T11:12:12.955385+0000","last_active":"2023-08-27T11:12:14.982032+0000","last_peered":"2023-08-27T11:12:14.982032+0000","last_clean":"2023-08-27T11:12:14.982032+0000","last_became_active":"2023-08-27T11:12:12.953960+0000","last_became_peered":"2023-08-27T11:12:12.953960+0000","last_unstale":"2023-08-27T11:12:14.982032+0000","last_undegraded":"2023-08-27T11:12:14.982032+0000","last_fullsized":"2023-08-27T11:12:14.982032+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:14.927498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934591+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:12:13.934591+0000","last_peered":"2023-08-27T11:12:13.934591+0000","last_clean":"2023-08-27T11:12:13.934591+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:12:13.934591+0000","last_undegraded":"2023-08-27T11:12:13.934591+0000","last_fullsized":"2023-08-27T11:12:13.934591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:05:01.448436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726808+0000","last_change":"2023-08-27T11:12:07.902830+0000","last_active":"2023-08-27T11:12:14.726808+0000","last_peered":"2023-08-27T11:12:14.726808+0000","last_clean":"2023-08-27T11:12:14.726808+0000","last_became_active":"2023-08-27T11:12:07.902452+0000","last_became_peered":"2023-08-27T11:12:07.902452+0000","last_unstale":"2023-08-27T11:12:14.726808+0000","last_undegraded":"2023-08-27T11:12:14.726808+0000","last_fullsized":"2023-08-27T11:12:14.726808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:46.293833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933163+0000","last_change":"2023-08-27T11:12:08.919355+0000","last_active":"2023-08-27T11:12:13.933163+0000","last_peered":"2023-08-27T11:12:13.933163+0000","last_clean":"2023-08-27T11:12:13.933163+0000","last_became_active":"2023-08-27T11:12:08.917203+0000","last_became_peered":"2023-08-27T11:12:08.917203+0000","last_unstale":"2023-08-27T11:12:13.933163+0000","last_undegraded":"2023-08-27T11:12:13.933163+0000","last_fullsized":"2023-08-27T11:12:13.933163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:07:00.762567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838062+0000","last_change":"2023-08-27T11:12:12.963107+0000","last_active":"2023-08-27T11:12:14.838062+0000","last_peered":"2023-08-27T11:12:14.838062+0000","last_clean":"2023-08-27T11:12:14.838062+0000","last_became_active":"2023-08-27T11:12:12.962714+0000","last_became_peered":"2023-08-27T11:12:12.962714+0000","last_unstale":"2023-08-27T11:12:14.838062+0000","last_undegraded":"2023-08-27T11:12:14.838062+0000","last_fullsized":"2023-08-27T11:12:14.838062+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:24.824055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726473+0000","last_change":"2023-08-27T11:12:07.900588+0000","last_active":"2023-08-27T11:12:14.726473+0000","last_peered":"2023-08-27T11:12:14.726473+0000","last_clean":"2023-08-27T11:12:14.726473+0000","last_became_active":"2023-08-27T11:12:07.900223+0000","last_became_peered":"2023-08-27T11:12:07.900223+0000","last_unstale":"2023-08-27T11:12:14.726473+0000","last_undegraded":"2023-08-27T11:12:14.726473+0000","last_fullsized":"2023-08-27T11:12:14.726473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:54.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933266+0000","last_change":"2023-08-27T11:12:08.921029+0000","last_active":"2023-08-27T11:12:13.933266+0000","last_peered":"2023-08-27T11:12:13.933266+0000","last_clean":"2023-08-27T11:12:13.933266+0000","last_became_active":"2023-08-27T11:12:08.920756+0000","last_became_peered":"2023-08-27T11:12:08.920756+0000","last_unstale":"2023-08-27T11:12:13.933266+0000","last_undegraded":"2023-08-27T11:12:13.933266+0000","last_fullsized":"2023-08-27T11:12:13.933266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:21:21.628536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384440+0000","last_change":"2023-08-27T11:12:12.958972+0000","last_active":"2023-08-27T11:12:14.384440+0000","last_peered":"2023-08-27T11:12:14.384440+0000","last_clean":"2023-08-27T11:12:14.384440+0000","last_became_active":"2023-08-27T11:12:12.958605+0000","last_became_peered":"2023-08-27T11:12:12.958605+0000","last_unstale":"2023-08-27T11:12:14.384440+0000","last_undegraded":"2023-08-27T11:12:14.384440+0000","last_fullsized":"2023-08-27T11:12:14.384440+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:55.500535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967443+0000","last_change":"2023-08-27T11:12:07.898515+0000","last_active":"2023-08-27T11:12:14.967443+0000","last_peered":"2023-08-27T11:12:14.967443+0000","last_clean":"2023-08-27T11:12:14.967443+0000","last_became_active":"2023-08-27T11:12:07.898243+0000","last_became_peered":"2023-08-27T11:12:07.898243+0000","last_unstale":"2023-08-27T11:12:14.967443+0000","last_undegraded":"2023-08-27T11:12:14.967443+0000","last_fullsized":"2023-08-27T11:12:14.967443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:47.521180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934374+0000","last_change":"2023-08-27T11:12:08.919278+0000","last_active":"2023-08-27T11:12:13.934374+0000","last_peered":"2023-08-27T11:12:13.934374+0000","last_clean":"2023-08-27T11:12:13.934374+0000","last_became_active":"2023-08-27T11:12:08.913663+0000","last_became_peered":"2023-08-27T11:12:08.913663+0000","last_unstale":"2023-08-27T11:12:13.934374+0000","last_undegraded":"2023-08-27T11:12:13.934374+0000","last_fullsized":"2023-08-27T11:12:13.934374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:20:17.707884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967392+0000","last_change":"2023-08-27T11:12:12.955103+0000","last_active":"2023-08-27T11:12:14.967392+0000","last_peered":"2023-08-27T11:12:14.967392+0000","last_clean":"2023-08-27T11:12:14.967392+0000","last_became_active":"2023-08-27T11:12:12.954719+0000","last_became_peered":"2023-08-27T11:12:12.954719+0000","last_unstale":"2023-08-27T11:12:14.967392+0000","last_undegraded":"2023-08-27T11:12:14.967392+0000","last_fullsized":"2023-08-27T11:12:14.967392+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:16.222119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967603+0000","last_change":"2023-08-27T11:12:07.899028+0000","last_active":"2023-08-27T11:12:14.967603+0000","last_peered":"2023-08-27T11:12:14.967603+0000","last_clean":"2023-08-27T11:12:14.967603+0000","last_became_active":"2023-08-27T11:12:07.898749+0000","last_became_peered":"2023-08-27T11:12:07.898749+0000","last_unstale":"2023-08-27T11:12:14.967603+0000","last_undegraded":"2023-08-27T11:12:14.967603+0000","last_fullsized":"2023-08-27T11:12:14.967603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:50.454016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933154+0000","last_change":"2023-08-27T11:12:08.917140+0000","last_active":"2023-08-27T11:12:13.933154+0000","last_peered":"2023-08-27T11:12:13.933154+0000","last_clean":"2023-08-27T11:12:13.933154+0000","last_became_active":"2023-08-27T11:12:08.916844+0000","last_became_peered":"2023-08-27T11:12:08.916844+0000","last_unstale":"2023-08-27T11:12:13.933154+0000","last_undegraded":"2023-08-27T11:12:13.933154+0000","last_fullsized":"2023-08-27T11:12:13.933154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:12.215932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933507+0000","last_change":"2023-08-27T11:12:12.952640+0000","last_active":"2023-08-27T11:12:13.933507+0000","last_peered":"2023-08-27T11:12:13.933507+0000","last_clean":"2023-08-27T11:12:13.933507+0000","last_became_active":"2023-08-27T11:12:12.952235+0000","last_became_peered":"2023-08-27T11:12:12.952235+0000","last_unstale":"2023-08-27T11:12:13.933507+0000","last_undegraded":"2023-08-27T11:12:13.933507+0000","last_fullsized":"2023-08-27T11:12:13.933507+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:44.222308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982940+0000","last_change":"2023-08-27T11:12:07.898854+0000","last_active":"2023-08-27T11:12:14.982940+0000","last_peered":"2023-08-27T11:12:14.982940+0000","last_clean":"2023-08-27T11:12:14.982940+0000","last_became_active":"2023-08-27T11:12:07.898427+0000","last_became_peered":"2023-08-27T11:12:07.898427+0000","last_unstale":"2023-08-27T11:12:14.982940+0000","last_undegraded":"2023-08-27T11:12:14.982940+0000","last_fullsized":"2023-08-27T11:12:14.982940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:25.089023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982399+0000","last_change":"2023-08-27T11:12:08.913067+0000","last_active":"2023-08-27T11:12:14.982399+0000","last_peered":"2023-08-27T11:12:14.982399+0000","last_clean":"2023-08-27T11:12:14.982399+0000","last_became_active":"2023-08-27T11:12:08.912345+0000","last_became_peered":"2023-08-27T11:12:08.912345+0000","last_unstale":"2023-08-27T11:12:14.982399+0000","last_undegraded":"2023-08-27T11:12:14.982399+0000","last_fullsized":"2023-08-27T11:12:14.982399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:41.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838060+0000","last_change":"2023-08-27T11:12:12.948294+0000","last_active":"2023-08-27T11:12:14.838060+0000","last_peered":"2023-08-27T11:12:14.838060+0000","last_clean":"2023-08-27T11:12:14.838060+0000","last_became_active":"2023-08-27T11:12:12.947742+0000","last_became_peered":"2023-08-27T11:12:12.947742+0000","last_unstale":"2023-08-27T11:12:14.838060+0000","last_undegraded":"2023-08-27T11:12:14.838060+0000","last_fullsized":"2023-08-27T11:12:14.838060+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:06.825066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727133+0000","last_change":"2023-08-27T11:12:07.902780+0000","last_active":"2023-08-27T11:12:14.727133+0000","last_peered":"2023-08-27T11:12:14.727133+0000","last_clean":"2023-08-27T11:12:14.727133+0000","last_became_active":"2023-08-27T11:12:07.902465+0000","last_became_peered":"2023-08-27T11:12:07.902465+0000","last_unstale":"2023-08-27T11:12:14.727133+0000","last_undegraded":"2023-08-27T11:12:14.727133+0000","last_fullsized":"2023-08-27T11:12:14.727133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:09.470216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933956+0000","last_change":"2023-08-27T11:12:08.925508+0000","last_active":"2023-08-27T11:12:13.933956+0000","last_peered":"2023-08-27T11:12:13.933956+0000","last_clean":"2023-08-27T11:12:13.933956+0000","last_became_active":"2023-08-27T11:12:08.919770+0000","last_became_peered":"2023-08-27T11:12:08.919770+0000","last_unstale":"2023-08-27T11:12:13.933956+0000","last_undegraded":"2023-08-27T11:12:13.933956+0000","last_fullsized":"2023-08-27T11:12:13.933956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:57.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932961+0000","last_change":"2023-08-27T11:12:08.914782+0000","last_active":"2023-08-27T11:12:13.932961+0000","last_peered":"2023-08-27T11:12:13.932961+0000","last_clean":"2023-08-27T11:12:13.932961+0000","last_became_active":"2023-08-27T11:12:08.914443+0000","last_became_peered":"2023-08-27T11:12:08.914443+0000","last_unstale":"2023-08-27T11:12:13.932961+0000","last_undegraded":"2023-08-27T11:12:13.932961+0000","last_fullsized":"2023-08-27T11:12:13.932961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:07.149137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982607+0000","last_change":"2023-08-27T11:12:08.923263+0000","last_active":"2023-08-27T11:12:14.982607+0000","last_peered":"2023-08-27T11:12:14.982607+0000","last_clean":"2023-08-27T11:12:14.982607+0000","last_became_active":"2023-08-27T11:12:08.922904+0000","last_became_peered":"2023-08-27T11:12:08.922904+0000","last_unstale":"2023-08-27T11:12:14.982607+0000","last_undegraded":"2023-08-27T11:12:14.982607+0000","last_fullsized":"2023-08-27T11:12:14.982607+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:49.762265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934690+0000","last_change":"2023-08-27T11:12:08.925804+0000","last_active":"2023-08-27T11:12:13.934690+0000","last_peered":"2023-08-27T11:12:13.934690+0000","last_clean":"2023-08-27T11:12:13.934690+0000","last_became_active":"2023-08-27T11:12:08.925554+0000","last_became_peered":"2023-08-27T11:12:08.925554+0000","last_unstale":"2023-08-27T11:12:13.934690+0000","last_undegraded":"2023-08-27T11:12:13.934690+0000","last_fullsized":"2023-08-27T11:12:13.934690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:15.724420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982308+0000","last_change":"2023-08-27T11:12:08.929175+0000","last_active":"2023-08-27T11:12:14.982308+0000","last_peered":"2023-08-27T11:12:14.982308+0000","last_clean":"2023-08-27T11:12:14.982308+0000","last_became_active":"2023-08-27T11:12:08.928863+0000","last_became_peered":"2023-08-27T11:12:08.928863+0000","last_unstale":"2023-08-27T11:12:14.982308+0000","last_undegraded":"2023-08-27T11:12:14.982308+0000","last_fullsized":"2023-08-27T11:12:14.982308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:30.787887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838203+0000","last_change":"2023-08-27T11:12:08.924528+0000","last_active":"2023-08-27T11:12:14.838203+0000","last_peered":"2023-08-27T11:12:14.838203+0000","last_clean":"2023-08-27T11:12:14.838203+0000","last_became_active":"2023-08-27T11:12:08.924221+0000","last_became_peered":"2023-08-27T11:12:08.924221+0000","last_unstale":"2023-08-27T11:12:14.838203+0000","last_undegraded":"2023-08-27T11:12:14.838203+0000","last_fullsized":"2023-08-27T11:12:14.838203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:12.180200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":710851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":120553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:12:35.531 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:12:35.531 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-27T11:12:35.532 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-08-27T11:12:35.532 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-08-27T11:12:35.533 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-08-27T11:12:35.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.687+0000 7f277f0d7700 1 Processor -- start 2023-08-27T11:12:35.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.687+0000 7f277f0d7700 1 -- start start 2023-08-27T11:12:35.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277f0d7700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f27780ca620 0x7f27780cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277f0d7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780caa00 0x7f27780acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277f0d7700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27780ce340 con 0x7f27780af200 2023-08-27T11:12:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277f0d7700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27780ceec0 con 0x7f27780ca620 2023-08-27T11:12:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277f0d7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277815d7c0 con 0x7f27780caa00 2023-08-27T11:12:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277ce73700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f27780ca620 0x7f27780cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277d674700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780caa00 0x7f27780acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277ce73700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f27780ca620 0x7f27780cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:40764/0 (socket says 172.21.15.45:40764) 2023-08-27T11:12:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277d674700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780caa00 0x7f27780acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:49822/0 (socket says 172.21.15.45:49822) 2023-08-27T11:12:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277ce73700 1 -- 172.21.15.45:0/232856419 learned_addr learned my addr 172.21.15.45:0/232856419 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277d674700 1 -- 172.21.15.45:0/232856419 >> v2:172.21.15.135:3300/0 conn(0x7f27780ca620 msgr2=0x7f27780cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:35.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277d674700 1 --2- 172.21.15.45:0/232856419 >> v2:172.21.15.135:3300/0 conn(0x7f27780ca620 0x7f27780cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277d674700 1 -- 172.21.15.45:0/232856419 >> v1:172.21.15.135:6789/0 conn(0x7f27780af200 legacy=0x7f27780ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.688+0000 7f277d674700 1 -- 172.21.15.45:0/232856419 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27780cc570 con 0x7f27780caa00 2023-08-27T11:12:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f277d674700 1 --2- 172.21.15.45:0/232856419 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780caa00 0x7f27780acfc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2770009d90 tx=0x7f277000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b4bf0299ff2dc4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f27777fe700 1 -- 172.21.15.45:0/232856419 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000e070 con 0x7f27780caa00 2023-08-27T11:12:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f27777fe700 1 -- 172.21.15.45:0/232856419 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277000bb20 con 0x7f27780caa00 2023-08-27T11:12:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f27777fe700 1 -- 172.21.15.45:0/232856419 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770012430 con 0x7f27780caa00 2023-08-27T11:12:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f277f0d7700 1 -- 172.21.15.45:0/232856419 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780caa00 msgr2=0x7f27780acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f277f0d7700 1 --2- 172.21.15.45:0/232856419 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780caa00 0x7f27780acfc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2770009d90 tx=0x7f277000b330 comp rx=0 tx=0).stop 2023-08-27T11:12:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f277f0d7700 1 -- 172.21.15.45:0/232856419 shutdown_connections 2023-08-27T11:12:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f277f0d7700 1 --2- 172.21.15.45:0/232856419 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780caa00 0x7f27780acfc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f277f0d7700 1 --2- 172.21.15.45:0/232856419 >> v2:172.21.15.135:3300/0 conn(0x7f27780ca620 0x7f27780cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f277f0d7700 1 -- 172.21.15.45:0/232856419 >> 172.21.15.45:0/232856419 conn(0x7f27780cc7e0 msgr2=0x7f27780ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:35.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f277f0d7700 1 -- 172.21.15.45:0/232856419 shutdown_connections 2023-08-27T11:12:35.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.689+0000 7f277f0d7700 1 -- 172.21.15.45:0/232856419 wait complete. 2023-08-27T11:12:35.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.691+0000 7f277f0d7700 1 Processor -- start 2023-08-27T11:12:35.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.691+0000 7f277f0d7700 1 -- start start 2023-08-27T11:12:35.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277f0d7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780af200 0x7f277815c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277f0d7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27780ca620 0x7f27780b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277f0d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27780caa00 0x7f27780b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277f0d7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27780c6690 con 0x7f27780caa00 2023-08-27T11:12:35.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277f0d7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781f2790 con 0x7f27780ca620 2023-08-27T11:12:35.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277f0d7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277815cba0 con 0x7f27780af200 2023-08-27T11:12:35.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277d674700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27780caa00 0x7f27780b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277d674700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27780caa00 0x7f27780b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51918/0 (socket says 172.21.15.45:51918) 2023-08-27T11:12:35.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277d674700 1 -- 172.21.15.45:0/1606470960 learned_addr learned my addr 172.21.15.45:0/1606470960 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:35.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277ce73700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780af200 0x7f277815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f2777fff700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27780ca620 0x7f27780b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277d674700 1 -- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780af200 msgr2=0x7f277815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277d674700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780af200 0x7f277815c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277d674700 1 -- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27780ca620 msgr2=0x7f27780b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277d674700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27780ca620 0x7f27780b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277d674700 1 -- 172.21.15.45:0/1606470960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770009a70 con 0x7f27780caa00 2023-08-27T11:12:35.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f277d674700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27780caa00 0x7f27780b5180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2770014040 tx=0x7f2770004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:35.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.692+0000 7f27757fa700 1 -- 172.21.15.45:0/1606470960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770003ea0 con 0x7f27780caa00 2023-08-27T11:12:35.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.693+0000 7f27757fa700 1 -- 172.21.15.45:0/1606470960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2770020070 con 0x7f27780caa00 2023-08-27T11:12:35.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.693+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27780c84c0 con 0x7f27780caa00 2023-08-27T11:12:35.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.693+0000 7f27757fa700 1 -- 172.21.15.45:0/1606470960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27700276c0 con 0x7f27780caa00 2023-08-27T11:12:35.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.693+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27780c8a80 con 0x7f27780caa00 2023-08-27T11:12:35.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.693+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27781f2920 con 0x7f27780caa00 2023-08-27T11:12:35.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.694+0000 7f27757fa700 1 -- 172.21.15.45:0/1606470960 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2770012430 con 0x7f27780caa00 2023-08-27T11:12:35.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.695+0000 7f27757fa700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f275c06a0e0 0x7f275c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:35.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.695+0000 7f27757fa700 1 -- 172.21.15.45:0/1606470960 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2770022070 con 0x7f27780caa00 2023-08-27T11:12:35.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.695+0000 7f277ce73700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f275c06a0e0 0x7f275c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:35.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.695+0000 7f277ce73700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f275c06a0e0 0x7f275c06c5a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f276c009b60 tx=0x7f276c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:35.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.698+0000 7f27757fa700 1 -- 172.21.15.45:0/1606470960 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2770056030 con 0x7f27780caa00 2023-08-27T11:12:35.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.960+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f27780cbce0 con 0x7f27780caa00 2023-08-27T11:12:35.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.963+0000 7f27757fa700 1 -- 172.21.15.45:0/1606470960 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169499 (secure 0 0 0) 0x7f2770059650 con 0x7f27780caa00 2023-08-27T11:12:35.967 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:35.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f275c06a0e0 msgr2=0x7f275c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f275c06a0e0 0x7f275c06c5a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f276c009b60 tx=0x7f276c008040 comp rx=0 tx=0).stop 2023-08-27T11:12:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27780caa00 msgr2=0x7f27780b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27780caa00 0x7f27780b5180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2770014040 tx=0x7f2770004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 shutdown_connections 2023-08-27T11:12:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27780caa00 0x7f27780b5180 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f275c06a0e0 0x7f275c06c5a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27780af200 0x7f277815c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 --2- 172.21.15.45:0/1606470960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27780ca620 0x7f27780b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 >> 172.21.15.45:0/1606470960 conn(0x7f27780cc7e0 msgr2=0x7f27780cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 shutdown_connections 2023-08-27T11:12:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:35.966+0000 7f277f0d7700 1 -- 172.21.15.45:0/1606470960 wait complete. 2023-08-27T11:12:35.980 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":4120550306},{"type":"v1","addr":"172.21.15.135:6833","nonce":4120550306}]},"active_addr":"172.21.15.135:6833/4120550306","active_change":"2023-08-27T11:11:48.792519+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":2587284130}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":2898860171}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":1692978501}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":1110669507}]}]} 2023-08-27T11:12:35.984 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-08-27T11:12:35.984 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-08-27T11:12:35.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.108+0000 7f75980c3700 1 Processor -- start 2023-08-27T11:12:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.108+0000 7f75980c3700 1 -- start start 2023-08-27T11:12:36.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f75980c3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f759015a240 0x7f75900cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f75980c3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 0x7f759006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f75980c3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759006d690 con 0x7f75900cbe70 2023-08-27T11:12:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f75980c3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75900cc440 con 0x7f759015a240 2023-08-27T11:12:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f75980c3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759006d040 con 0x7f75900ceba0 2023-08-27T11:12:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f7595e5f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f759015a240 0x7f75900cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f7595e5f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f759015a240 0x7f75900cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51922/0 (socket says 172.21.15.45:51922) 2023-08-27T11:12:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f7595e5f700 1 -- 172.21.15.45:0/3482468957 learned_addr learned my addr 172.21.15.45:0/3482468957 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f7596660700 1 --2- 172.21.15.45:0/3482468957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 0x7f759006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f7595e5f700 1 -- 172.21.15.45:0/3482468957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 msgr2=0x7f759006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f7595e5f700 1 --2- 172.21.15.45:0/3482468957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 0x7f759006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f7595e5f700 1 -- 172.21.15.45:0/3482468957 >> v1:172.21.15.135:6789/0 conn(0x7f75900cbe70 legacy=0x7f75900ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f7595e5f700 1 -- 172.21.15.45:0/3482468957 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759006d320 con 0x7f759015a240 2023-08-27T11:12:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.109+0000 7f7595e5f700 1 --2- 172.21.15.45:0/3482468957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 0x7f75900cae80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098f0 tx=0x7f758c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a736c8d75eb8f40 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f7594e5d700 1 -- 172.21.15.45:0/3482468957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c00b8a0 con 0x7f759015a240 2023-08-27T11:12:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f7594e5d700 1 -- 172.21.15.45:0/3482468957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c00ba00 con 0x7f759015a240 2023-08-27T11:12:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f7594e5d700 1 -- 172.21.15.45:0/3482468957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c00bd90 con 0x7f759015a240 2023-08-27T11:12:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f75980c3700 1 -- 172.21.15.45:0/3482468957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 msgr2=0x7f75900cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f75980c3700 1 --2- 172.21.15.45:0/3482468957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 0x7f75900cae80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098f0 tx=0x7f758c009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:36.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f75980c3700 1 -- 172.21.15.45:0/3482468957 shutdown_connections 2023-08-27T11:12:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f75980c3700 1 --2- 172.21.15.45:0/3482468957 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 0x7f759006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f75980c3700 1 --2- 172.21.15.45:0/3482468957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 0x7f75900cae80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f75980c3700 1 -- 172.21.15.45:0/3482468957 >> 172.21.15.45:0/3482468957 conn(0x7f75900cfe50 msgr2=0x7f75900d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f75980c3700 1 -- 172.21.15.45:0/3482468957 shutdown_connections 2023-08-27T11:12:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.110+0000 7f75980c3700 1 -- 172.21.15.45:0/3482468957 wait complete. 2023-08-27T11:12:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.112+0000 7f75980c3700 1 Processor -- start 2023-08-27T11:12:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.112+0000 7f75980c3700 1 -- start start 2023-08-27T11:12:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f75980c3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75900cbe70 0x7f75900c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f75980c3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 0x7f7590157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f75980c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 0x7f75901efc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f75980c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75900c4390 con 0x7f759015a240 2023-08-27T11:12:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f75980c3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75900aefa0 con 0x7f75900cbe70 2023-08-27T11:12:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f75980c3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75900a8510 con 0x7f75900ceba0 2023-08-27T11:12:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7596660700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 0x7f75901efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7596660700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 0x7f75901efc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51932/0 (socket says 172.21.15.45:51932) 2023-08-27T11:12:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7596660700 1 -- 172.21.15.45:0/150866152 learned_addr learned my addr 172.21.15.45:0/150866152 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7595e5f700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75900cbe70 0x7f75900c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f759565e700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 0x7f7590157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7596660700 1 -- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 msgr2=0x7f7590157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7596660700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 0x7f7590157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7596660700 1 -- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75900cbe70 msgr2=0x7f75900c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7596660700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75900cbe70 0x7f75900c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7596660700 1 -- 172.21.15.45:0/150866152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c009550 con 0x7f759015a240 2023-08-27T11:12:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f759565e700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 0x7f7590157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.113+0000 7f7596660700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 0x7f75901efc50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7588007de0 tx=0x7f758800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.114+0000 7f75867fc700 1 -- 172.21.15.45:0/150866152 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758800ede0 con 0x7f759015a240 2023-08-27T11:12:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.114+0000 7f75867fc700 1 -- 172.21.15.45:0/150866152 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588014420 con 0x7f759015a240 2023-08-27T11:12:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.114+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75900c45a0 con 0x7f759015a240 2023-08-27T11:12:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.114+0000 7f75867fc700 1 -- 172.21.15.45:0/150866152 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880147d0 con 0x7f759015a240 2023-08-27T11:12:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.114+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75900c4b60 con 0x7f759015a240 2023-08-27T11:12:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.115+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7590072810 con 0x7f759015a240 2023-08-27T11:12:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.115+0000 7f75867fc700 1 -- 172.21.15.45:0/150866152 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7588014930 con 0x7f759015a240 2023-08-27T11:12:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.115+0000 7f75867fc700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f757406a0e0 0x7f757406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.115+0000 7f75867fc700 1 -- 172.21.15.45:0/150866152 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f75880890c0 con 0x7f759015a240 2023-08-27T11:12:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.116+0000 7f7595e5f700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f757406a0e0 0x7f757406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.116+0000 7f7595e5f700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f757406a0e0 0x7f757406c5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f758c014040 tx=0x7f758c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.120+0000 7f75867fc700 1 -- 172.21.15.45:0/150866152 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7588054440 con 0x7f759015a240 2023-08-27T11:12:36.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.321+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75900c2cb0 con 0x7f759015a240 2023-08-27T11:12:36.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.322+0000 7f75867fc700 1 -- 172.21.15.45:0/150866152 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f7588057a60 con 0x7f759015a240 2023-08-27T11:12:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:36.325 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:36.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f757406a0e0 msgr2=0x7f757406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f757406a0e0 0x7f757406c5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f758c014040 tx=0x7f758c008030 comp rx=0 tx=0).stop 2023-08-27T11:12:36.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 msgr2=0x7f75901efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 0x7f75901efc50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7588007de0 tx=0x7f758800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 shutdown_connections 2023-08-27T11:12:36.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f759015a240 0x7f75901efc50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f757406a0e0 0x7f757406c5a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75900cbe70 0x7f75900c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 --2- 172.21.15.45:0/150866152 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75900ceba0 0x7f7590157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 >> 172.21.15.45:0/150866152 conn(0x7f75900cfe50 msgr2=0x7f75900c8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 shutdown_connections 2023-08-27T11:12:36.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.324+0000 7f75980c3700 1 -- 172.21.15.45:0/150866152 wait complete. 2023-08-27T11:12:36.339 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-08-27T11:12:36.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:36.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.462+0000 7fd412590700 1 Processor -- start 2023-08-27T11:12:36.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.462+0000 7fd412590700 1 -- start start 2023-08-27T11:12:36.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd412590700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd40c0cbe70 0x7fd40c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd412590700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c06d690 con 0x7fd40c0ceba0 2023-08-27T11:12:36.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd412590700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c0cc440 con 0x7fd40c15a240 2023-08-27T11:12:36.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd412590700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c06d040 con 0x7fd40c0cbe70 2023-08-27T11:12:36.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd40c0ceba0 0x7fd40c06db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38712/0 (socket says 172.21.15.45:38712) 2023-08-27T11:12:36.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40b7fe700 1 -- 172.21.15.45:0/3024678309 learned_addr learned my addr 172.21.15.45:0/3024678309 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:36.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd410b2d700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd40c0cbe70 0x7fd40c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523966899 0 0) 0x7fd40c06d690 con 0x7fd40c0ceba0 2023-08-27T11:12:36.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f40035b0 con 0x7fd40c0ceba0 2023-08-27T11:12:36.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2947788862 0 0) 0x7fd3f40035b0 con 0x7fd40c0ceba0 2023-08-27T11:12:36.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd40c0cbe70 msgr2=0x7fd40c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40affd700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd40c0cbe70 0x7fd40c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 >> v1:172.21.15.135:6789/0 conn(0x7fd40c15a240 legacy=0x7fd40c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c06d320 con 0x7fd40c0ceba0 2023-08-27T11:12:36.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.463+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fd3fc001670 con 0x7fd40c0ceba0 2023-08-27T11:12:36.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 >> v1:172.21.15.45:6789/0 conn(0x7fd40c0ceba0 legacy=0x7fd40c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f4003a30 0x7fd3f4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3f40043e0 0x7fd3f40048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd40c0cbe70 0x7fd3f4005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c06d040 con 0x7fd3f4003a30 2023-08-27T11:12:36.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f400ba10 con 0x7fd3f40043e0 2023-08-27T11:12:36.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f400bbd0 con 0x7fd40c0cbe70 2023-08-27T11:12:36.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40bfff700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f4003a30 0x7fd3f4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd410b2d700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd40c0cbe70 0x7fd3f4005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40b7fe700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3f40043e0 0x7fd3f40048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40bfff700 1 -- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd40c0cbe70 msgr2=0x7fd3f4005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40bfff700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd40c0cbe70 0x7fd3f4005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40bfff700 1 -- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3f40043e0 msgr2=0x7fd3f40048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40bfff700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3f40043e0 0x7fd3f40048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40bfff700 1 -- 172.21.15.45:0/3024678309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f40035b0 con 0x7fd3f4003a30 2023-08-27T11:12:36.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40bfff700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f4003a30 0x7fd3f4003e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd4000018b0 tx=0x7fd3f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fd02e393a10eb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400001370 con 0x7fd3f4003a30 2023-08-27T11:12:36.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40000c6b0 con 0x7fd3f4003a30 2023-08-27T11:12:36.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.464+0000 7fd40affd700 1 -- 172.21.15.45:0/3024678309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40000ca60 con 0x7fd3f4003a30 2023-08-27T11:12:36.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.465+0000 7fd412590700 1 -- 172.21.15.45:0/3024678309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f4003a30 msgr2=0x7fd3f4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.465+0000 7fd412590700 1 --2- 172.21.15.45:0/3024678309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f4003a30 0x7fd3f4003e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd4000018b0 tx=0x7fd3f4005880 comp rx=0 tx=0).stop 2023-08-27T11:12:36.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.465+0000 7fd410b2d700 1 -- 172.21.15.45:0/3024678309 reap_dead start 2023-08-27T11:12:36.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.465+0000 7fd412590700 1 -- 172.21.15.45:0/3024678309 shutdown_connections 2023-08-27T11:12:36.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.465+0000 7fd412590700 1 -- 172.21.15.45:0/3024678309 >> 172.21.15.45:0/3024678309 conn(0x7fd40c0cfe50 msgr2=0x7fd40c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.465+0000 7fd412590700 1 -- 172.21.15.45:0/3024678309 shutdown_connections 2023-08-27T11:12:36.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.465+0000 7fd412590700 1 -- 172.21.15.45:0/3024678309 wait complete. 2023-08-27T11:12:36.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.466+0000 7fd412590700 1 Processor -- start 2023-08-27T11:12:36.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.466+0000 7fd412590700 1 -- start start 2023-08-27T11:12:36.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd412590700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3f4003a30 0x7fd40c154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd412590700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c153f50 0x7fd40c15c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd412590700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15aa50 0x7fd40c15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd412590700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c1543d0 con 0x7fd40c15aa50 2023-08-27T11:12:36.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd412590700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c15b420 con 0x7fd40c153f50 2023-08-27T11:12:36.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd412590700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c15c670 con 0x7fd3f4003a30 2023-08-27T11:12:36.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd410b2d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15aa50 0x7fd40c15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd410b2d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15aa50 0x7fd40c15aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51952/0 (socket says 172.21.15.45:51952) 2023-08-27T11:12:36.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd410b2d700 1 -- 172.21.15.45:0/2416126659 learned_addr learned my addr 172.21.15.45:0/2416126659 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd40bfff700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3f4003a30 0x7fd40c154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd40b7fe700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c153f50 0x7fd40c15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd410b2d700 1 -- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3f4003a30 msgr2=0x7fd40c154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd410b2d700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3f4003a30 0x7fd40c154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd410b2d700 1 -- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c153f50 msgr2=0x7fd40c15c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd410b2d700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c153f50 0x7fd40c15c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.467+0000 7fd410b2d700 1 -- 172.21.15.45:0/2416126659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c154b30 con 0x7fd40c15aa50 2023-08-27T11:12:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.468+0000 7fd410b2d700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15aa50 0x7fd40c15aee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd40c15db60 tx=0x7fd404011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.468+0000 7fd408ff9700 1 -- 172.21.15.45:0/2416126659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400e040 con 0x7fd40c15aa50 2023-08-27T11:12:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.468+0000 7fd408ff9700 1 -- 172.21.15.45:0/2416126659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40400f040 con 0x7fd40c15aa50 2023-08-27T11:12:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.468+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c154e10 con 0x7fd40c15aa50 2023-08-27T11:12:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.468+0000 7fd408ff9700 1 -- 172.21.15.45:0/2416126659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400fe40 con 0x7fd40c15aa50 2023-08-27T11:12:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.468+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c1553d0 con 0x7fd40c15aa50 2023-08-27T11:12:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.469+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c072810 con 0x7fd40c15aa50 2023-08-27T11:12:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.469+0000 7fd408ff9700 1 -- 172.21.15.45:0/2416126659 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd404015430 con 0x7fd40c15aa50 2023-08-27T11:12:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.470+0000 7fd408ff9700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd3f806a0e0 0x7fd3f806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.470+0000 7fd408ff9700 1 -- 172.21.15.45:0/2416126659 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd404088ed0 con 0x7fd40c15aa50 2023-08-27T11:12:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.470+0000 7fd40bfff700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd3f806a0e0 0x7fd3f806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.470+0000 7fd40bfff700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd3f806a0e0 0x7fd3f806c5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd400014040 tx=0x7fd400005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.474+0000 7fd408ff9700 1 -- 172.21.15.45:0/2416126659 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd404057df0 con 0x7fd40c15aa50 2023-08-27T11:12:36.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.673+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd40c0734a0 con 0x7fd40c15aa50 2023-08-27T11:12:36.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.674+0000 7fd408ff9700 1 -- 172.21.15.45:0/2416126659 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7fd404057980 con 0x7fd40c15aa50 2023-08-27T11:12:36.677 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:36.677 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:36.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd3f806a0e0 msgr2=0x7fd3f806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd3f806a0e0 0x7fd3f806c5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd400014040 tx=0x7fd400005040 comp rx=0 tx=0).stop 2023-08-27T11:12:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15aa50 msgr2=0x7fd40c15aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15aa50 0x7fd40c15aee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd40c15db60 tx=0x7fd404011f60 comp rx=0 tx=0).stop 2023-08-27T11:12:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 shutdown_connections 2023-08-27T11:12:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15aa50 0x7fd40c15aee0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd3f806a0e0 0x7fd3f806c5a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd3f4003a30 0x7fd40c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 --2- 172.21.15.45:0/2416126659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd40c153f50 0x7fd40c15c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.676+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 >> 172.21.15.45:0/2416126659 conn(0x7fd40c0cfe50 msgr2=0x7fd40c156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.677+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 shutdown_connections 2023-08-27T11:12:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.677+0000 7fd412590700 1 -- 172.21.15.45:0/2416126659 wait complete. 2023-08-27T11:12:36.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-08-27T11:12:36.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-08-27T11:12:36.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-08-27T11:12:36.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-08-27T11:12:36.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-08-27T11:12:36.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-08-27T11:12:36.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-08-27T11:12:36.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-08-27T11:12:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.912+0000 7f457bb86700 1 Processor -- start 2023-08-27T11:12:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.912+0000 7f457bb86700 1 -- start start 2023-08-27T11:12:36.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.912+0000 7f08eac27700 1 Processor -- start 2023-08-27T11:12:36.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.912+0000 7fc844930700 1 Processor -- start 2023-08-27T11:12:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7f457bb86700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f45740ae080 0x7f457406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7f457bb86700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f457415e2b0 0x7f457406dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7f457bb86700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457415e690 con 0x7f4574161da0 2023-08-27T11:12:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7f457bb86700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45740ca3d0 con 0x7f457415e2b0 2023-08-27T11:12:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7f457bb86700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45741610b0 con 0x7f45740ae080 2023-08-27T11:12:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7f457a123700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f457415e2b0 0x7f457406dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7f457a123700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f457415e2b0 0x7f457406dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51958/0 (socket says 172.21.15.45:51958) 2023-08-27T11:12:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7f457a123700 1 -- 172.21.15.45:0/4212370537 learned_addr learned my addr 172.21.15.45:0/4212370537 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f4579922700 1 --2- 172.21.15.45:0/4212370537 >> v2:172.21.15.135:3300/0 conn(0x7f45740ae080 0x7f457406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7f08eac27700 1 -- start start 2023-08-27T11:12:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f08eac27700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08e4164b70 0x7f08e4165ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f08eac27700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08e40cf3e0 0x7f08e406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f08eac27700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e4164f50 con 0x7f08e40cf860 2023-08-27T11:12:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f08eac27700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e4165850 con 0x7f08e40cf3e0 2023-08-27T11:12:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f08eac27700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40cd880 con 0x7f08e4164b70 2023-08-27T11:12:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f457a123700 1 -- 172.21.15.45:0/4212370537 >> v2:172.21.15.135:3300/0 conn(0x7f45740ae080 msgr2=0x7f457406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f457a123700 1 --2- 172.21.15.45:0/4212370537 >> v2:172.21.15.135:3300/0 conn(0x7f45740ae080 0x7f457406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f457a123700 1 -- 172.21.15.45:0/4212370537 >> v1:172.21.15.135:6789/0 conn(0x7f4574161da0 legacy=0x7f457415dba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f457a123700 1 -- 172.21.15.45:0/4212370537 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457415e8d0 con 0x7f457415e2b0 2023-08-27T11:12:36.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.913+0000 7fc844930700 1 -- start start 2023-08-27T11:12:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7fc844930700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 0x7fc84006c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7fc844930700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8400c9940 con 0x7fc8400c9490 2023-08-27T11:12:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7fc844930700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8400c9b70 con 0x7fc840159590 2023-08-27T11:12:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7fc844930700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400ca130 con 0x7fc840070430 2023-08-27T11:12:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7fc83e59c700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc840159590 0x7fc8400cd720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:34402/0 (socket says 172.21.15.45:34402) 2023-08-27T11:12:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7fc83e59c700 1 -- 172.21.15.45:0/4057032994 learned_addr learned my addr 172.21.15.45:0/4057032994 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f457a123700 1 --2- 172.21.15.45:0/4212370537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f457415e2b0 0x7f457406dca0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f457406cef0 tx=0x7f456c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5d648f7b798ff server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7fc83ed9d700 1 --2- 172.21.15.45:0/4057032994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 0x7fc84006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f08e91c4700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f08e40cf860 0x7f08e4070d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:34406/0 (socket says 172.21.15.45:34406) 2023-08-27T11:12:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f08e91c4700 1 -- 172.21.15.45:0/1934966462 learned_addr learned my addr 172.21.15.45:0/1934966462 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.914+0000 7f08e89c3700 1 --2- 172.21.15.45:0/1934966462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08e4164b70 0x7f08e4165ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7f08e3fff700 1 --2- 172.21.15.45:0/1934966462 >> v2:172.21.15.45:3300/0 conn(0x7f08e40cf3e0 0x7f08e406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7f4578920700 1 -- 172.21.15.45:0/4212370537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c00e070 con 0x7f457415e2b0 2023-08-27T11:12:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7f4578920700 1 -- 172.21.15.45:0/4212370537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c003a20 con 0x7f457415e2b0 2023-08-27T11:12:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7f4578920700 1 -- 172.21.15.45:0/4212370537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c003dd0 con 0x7f457415e2b0 2023-08-27T11:12:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7fc83ed9d700 1 -- 172.21.15.45:0/4057032994 >> v1:172.21.15.135:6789/0 conn(0x7fc840159590 legacy=0x7fc8400cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7fc83ed9d700 1 -- 172.21.15.45:0/4057032994 >> v1:172.21.15.45:6789/0 conn(0x7fc8400c9490 legacy=0x7fc84006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7fc83ed9d700 1 -- 172.21.15.45:0/4057032994 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400c9e50 con 0x7fc840070430 2023-08-27T11:12:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7f08e3fff700 1 -- 172.21.15.45:0/1934966462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08e4164b70 msgr2=0x7f08e4165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7f08e3fff700 1 --2- 172.21.15.45:0/1934966462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08e4164b70 0x7f08e4165ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7f08e3fff700 1 -- 172.21.15.45:0/1934966462 >> v1:172.21.15.135:6789/0 conn(0x7f08e40cf860 legacy=0x7f08e4070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7f08e3fff700 1 -- 172.21.15.45:0/1934966462 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e40cdb20 con 0x7f08e40cf3e0 2023-08-27T11:12:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7f08e3fff700 1 --2- 172.21.15.45:0/1934966462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf3e0 0x7f08e406a280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009840 tx=0x7f08d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4918ad2ac1e9ec1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.915+0000 7fc83ed9d700 1 --2- 172.21.15.45:0/4057032994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 0x7fc84006c860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc83800aed0 tx=0x7fc83800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec3e8ad8cb24b08b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7f08e37fe700 1 -- 172.21.15.45:0/1934966462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8004230 con 0x7f08e40cf3e0 2023-08-27T11:12:36.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7f08e37fe700 1 -- 172.21.15.45:0/1934966462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d8004390 con 0x7f08e40cf3e0 2023-08-27T11:12:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7f08e37fe700 1 -- 172.21.15.45:0/1934966462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d80036a0 con 0x7f08e40cf3e0 2023-08-27T11:12:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7f457bb86700 1 -- 172.21.15.45:0/4212370537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f457415e2b0 msgr2=0x7f457406dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7f457bb86700 1 --2- 172.21.15.45:0/4212370537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f457415e2b0 0x7f457406dca0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f457406cef0 tx=0x7f456c00b330 comp rx=0 tx=0).stop 2023-08-27T11:12:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7fc83d59a700 1 -- 172.21.15.45:0/4057032994 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83800f070 con 0x7fc840070430 2023-08-27T11:12:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7fc83d59a700 1 -- 172.21.15.45:0/4057032994 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83800a2e0 con 0x7fc840070430 2023-08-27T11:12:36.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7fc83d59a700 1 -- 172.21.15.45:0/4057032994 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838003c40 con 0x7fc840070430 2023-08-27T11:12:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7f457bb86700 1 -- 172.21.15.45:0/4212370537 shutdown_connections 2023-08-27T11:12:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7f457bb86700 1 --2- 172.21.15.45:0/4212370537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f457415e2b0 0x7f457406dca0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7f457bb86700 1 --2- 172.21.15.45:0/4212370537 >> v2:172.21.15.135:3300/0 conn(0x7f45740ae080 0x7f457406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.916+0000 7f457bb86700 1 -- 172.21.15.45:0/4212370537 >> 172.21.15.45:0/4212370537 conn(0x7f45740cec30 msgr2=0x7f45740cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7fc844930700 1 -- 172.21.15.45:0/4057032994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 msgr2=0x7fc84006c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7fc844930700 1 --2- 172.21.15.45:0/4057032994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 0x7fc84006c860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc83800aed0 tx=0x7fc83800c330 comp rx=0 tx=0).stop 2023-08-27T11:12:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f08eac27700 1 -- 172.21.15.45:0/1934966462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf3e0 msgr2=0x7f08e406a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f08eac27700 1 --2- 172.21.15.45:0/1934966462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf3e0 0x7f08e406a280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009840 tx=0x7f08d8009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:36.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f457bb86700 1 -- 172.21.15.45:0/4212370537 shutdown_connections 2023-08-27T11:12:36.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f08eac27700 1 -- 172.21.15.45:0/1934966462 shutdown_connections 2023-08-27T11:12:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f08eac27700 1 --2- 172.21.15.45:0/1934966462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08e4164b70 0x7f08e4165ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f08eac27700 1 --2- 172.21.15.45:0/1934966462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf3e0 0x7f08e406a280 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f08eac27700 1 -- 172.21.15.45:0/1934966462 >> 172.21.15.45:0/1934966462 conn(0x7f08e40cc7c0 msgr2=0x7f08e40ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f457bb86700 1 -- 172.21.15.45:0/4212370537 wait complete. 2023-08-27T11:12:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f08eac27700 1 -- 172.21.15.45:0/1934966462 shutdown_connections 2023-08-27T11:12:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7f457bb86700 1 Processor -- start 2023-08-27T11:12:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7fc844930700 1 -- 172.21.15.45:0/4057032994 shutdown_connections 2023-08-27T11:12:36.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7fc844930700 1 --2- 172.21.15.45:0/4057032994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 0x7fc84006c860 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7fc844930700 1 -- 172.21.15.45:0/4057032994 >> 172.21.15.45:0/4057032994 conn(0x7fc8400cec30 msgr2=0x7fc8400cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.917+0000 7f08eac27700 1 -- 172.21.15.45:0/1934966462 wait complete. 2023-08-27T11:12:36.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7f08eac27700 1 Processor -- start 2023-08-27T11:12:36.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7fc844930700 1 -- 172.21.15.45:0/4057032994 shutdown_connections 2023-08-27T11:12:36.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7f457bb86700 1 -- start start 2023-08-27T11:12:36.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7f457bb86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740ae080 0x7f4574154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7f457bb86700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f457415e2b0 0x7f45740c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7f457bb86700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574161da0 0x7f45740af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f457bb86700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45741547d0 con 0x7f45740ae080 2023-08-27T11:12:36.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f457bb86700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45740c3fc0 con 0x7f4574161da0 2023-08-27T11:12:36.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f457bb86700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45740c42a0 con 0x7f457415e2b0 2023-08-27T11:12:36.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579922700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740ae080 0x7f4574154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579922700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740ae080 0x7f4574154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51974/0 (socket says 172.21.15.45:51974) 2023-08-27T11:12:36.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579922700 1 -- 172.21.15.45:0/3848996887 learned_addr learned my addr 172.21.15.45:0/3848996887 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f457a123700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574161da0 0x7f45740af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579121700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f457415e2b0 0x7f45740c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579922700 1 -- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f457415e2b0 msgr2=0x7f45740c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579922700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f457415e2b0 0x7f45740c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579922700 1 -- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574161da0 msgr2=0x7f45740af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579922700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574161da0 0x7f45740af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579922700 1 -- 172.21.15.45:0/3848996887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c009b30 con 0x7f45740ae080 2023-08-27T11:12:36.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f457a123700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574161da0 0x7f45740af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:36.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f4579922700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740ae080 0x7f4574154290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f456400e810 tx=0x7f456400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f456a7fc700 1 -- 172.21.15.45:0/3848996887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564013610 con 0x7f45740ae080 2023-08-27T11:12:36.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.919+0000 7f08eac27700 1 -- start start 2023-08-27T11:12:36.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f08eac27700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e40cf3e0 0x7f08e40c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f08eac27700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf860 0x7f08e40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f08eac27700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08e4164b70 0x7f08e40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f08eac27700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40af7a0 con 0x7f08e40cf860 2023-08-27T11:12:36.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f08eac27700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40b4920 con 0x7f08e40cf3e0 2023-08-27T11:12:36.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.920+0000 7f08eac27700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40b4c00 con 0x7f08e4164b70 2023-08-27T11:12:36.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf860 0x7f08e40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45740c4580 con 0x7f45740ae080 2023-08-27T11:12:36.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45740c4b40 con 0x7f45740ae080 2023-08-27T11:12:36.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e89c3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e40cf3e0 0x7f08e40c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf860 0x7f08e40b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51982/0 (socket says 172.21.15.45:51982) 2023-08-27T11:12:36.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e3fff700 1 -- 172.21.15.45:0/2958414300 learned_addr learned my addr 172.21.15.45:0/2958414300 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e91c4700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08e4164b70 0x7f08e40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e3fff700 1 -- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08e4164b70 msgr2=0x7f08e40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e3fff700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08e4164b70 0x7f08e40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e3fff700 1 -- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e40cf3e0 msgr2=0x7f08e40c14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e3fff700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08e40cf3e0 0x7f08e40c14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e3fff700 1 -- 172.21.15.45:0/2958414300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d80094a0 con 0x7f08e40cf860 2023-08-27T11:12:36.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f456a7fc700 1 -- 172.21.15.45:0/3848996887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4564013770 con 0x7f45740ae080 2023-08-27T11:12:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f456a7fc700 1 -- 172.21.15.45:0/3848996887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564013b20 con 0x7f45740ae080 2023-08-27T11:12:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.922+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f455c000fc0 con 0x7f45740ae080 2023-08-27T11:12:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e3fff700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf860 0x7f08e40b3f40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f08d8004000 tx=0x7f08d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e17fa700 1 -- 172.21.15.45:0/2958414300 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d80045f0 con 0x7f08e40cf860 2023-08-27T11:12:36.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e17fa700 1 -- 172.21.15.45:0/2958414300 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d8020070 con 0x7f08e40cf860 2023-08-27T11:12:36.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.921+0000 7f08e17fa700 1 -- 172.21.15.45:0/2958414300 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8011630 con 0x7f08e40cf860 2023-08-27T11:12:36.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.922+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e40b4ee0 con 0x7f08e40cf860 2023-08-27T11:12:36.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.922+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e40c19e0 con 0x7f08e40cf860 2023-08-27T11:12:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.922+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08e41654e0 con 0x7f08e40cf860 2023-08-27T11:12:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.923+0000 7f934cd3e700 1 Processor -- start 2023-08-27T11:12:36.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.923+0000 7f456a7fc700 1 -- 172.21.15.45:0/3848996887 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4564013c80 con 0x7f45740ae080 2023-08-27T11:12:36.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.924+0000 7f456a7fc700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f455806a0b0 0x7f455806c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.924+0000 7f08e17fa700 1 -- 172.21.15.45:0/2958414300 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f08d8022030 con 0x7f08e40cf860 2023-08-27T11:12:36.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.924+0000 7f456a7fc700 1 -- 172.21.15.45:0/3848996887 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4564088a70 con 0x7f45740ae080 2023-08-27T11:12:36.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.924+0000 7f4579121700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f455806a0b0 0x7f455806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.924+0000 7f456a7fc700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.45:6808/2953012420,v1:172.21.15.45:6809/2953012420] conn(0x7f4558073d10 0x7f4558076150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.924+0000 7f456a7fc700 1 -- 172.21.15.45:0/3848996887 --> [v2:172.21.15.45:6808/2953012420,v1:172.21.15.45:6809/2953012420] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4558076820 con 0x7f4558073d10 2023-08-27T11:12:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.924+0000 7f456a7fc700 1 -- 172.21.15.45:0/3848996887 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4564088ec0 con 0x7f45740ae080 2023-08-27T11:12:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.924+0000 7f457a123700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.45:6808/2953012420,v1:172.21.15.45:6809/2953012420] conn(0x7f4558073d10 0x7f4558076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.924+0000 7f08e17fa700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f08c806a0e0 0x7f08c806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f08e17fa700 1 -- 172.21.15.45:0/2958414300 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f08d808a5d0 con 0x7f08e40cf860 2023-08-27T11:12:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f4579121700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f455806a0b0 0x7f455806c570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f45740b00a0 tx=0x7f45700090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f457a123700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.45:6808/2953012420,v1:172.21.15.45:6809/2953012420] conn(0x7f4558073d10 0x7f4558076150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f08e17fa700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.45:6824/1621076447,v1:172.21.15.45:6825/1621076447] conn(0x7f08c8073dc0 0x7f08c8076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f08e17fa700 1 -- 172.21.15.45:0/2958414300 --> [v2:172.21.15.45:6824/1621076447,v1:172.21.15.45:6825/1621076447] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08c80768d0 con 0x7f08c8073dc0 2023-08-27T11:12:36.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f08e17fa700 1 -- 172.21.15.45:0/2958414300 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08d808a980 con 0x7f08e40cf860 2023-08-27T11:12:36.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f08e89c3700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f08c806a0e0 0x7f08c806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7fc844930700 1 -- 172.21.15.45:0/4057032994 wait complete. 2023-08-27T11:12:36.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.923+0000 7f934cd3e700 1 -- start start 2023-08-27T11:12:36.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f934cd3e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93480c32c0 0x7f93480ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f934cd3e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cab00 0x7f93480ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f934cd3e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934806d670 con 0x7f93480cef10 2023-08-27T11:12:36.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f934cd3e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93480caf80 con 0x7f93480c32c0 2023-08-27T11:12:36.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f934cd3e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93480c36a0 con 0x7f93480cab00 2023-08-27T11:12:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f934659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93480c32c0 0x7f93480ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f934659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f93480c32c0 0x7f93480ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51992/0 (socket says 172.21.15.45:51992) 2023-08-27T11:12:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f934659c700 1 -- 172.21.15.45:0/3480552705 learned_addr learned my addr 172.21.15.45:0/3480552705 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f9345d9b700 1 --2- 172.21.15.45:0/3480552705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cab00 0x7f93480ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f5ecea20700 1 Processor -- start 2023-08-27T11:12:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f9345d9b700 1 -- 172.21.15.45:0/3480552705 >> v2:172.21.15.45:3300/0 conn(0x7f93480c32c0 msgr2=0x7f93480ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f9345d9b700 1 --2- 172.21.15.45:0/3480552705 >> v2:172.21.15.45:3300/0 conn(0x7f93480c32c0 0x7f93480ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f9345d9b700 1 -- 172.21.15.45:0/3480552705 >> v1:172.21.15.135:6789/0 conn(0x7f93480cef10 legacy=0x7f93480cd930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f9345d9b700 1 -- 172.21.15.45:0/3480552705 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93480cf750 con 0x7f93480cab00 2023-08-27T11:12:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.925+0000 7f934659c700 1 --2- 172.21.15.45:0/3480552705 >> v2:172.21.15.45:3300/0 conn(0x7f93480c32c0 0x7f93480ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.926+0000 7f5ecea20700 1 -- start start 2023-08-27T11:12:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.926+0000 7f5ecea20700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ec80aeab0 0x7f5ec8071990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.926+0000 7f5ecea20700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5ec8159ee0 0x7f5ec80cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.926+0000 7f5ecea20700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec80cc160 con 0x7f5ec80ccb30 2023-08-27T11:12:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.926+0000 7f5ecea20700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec8159c40 con 0x7f5ec80aeab0 2023-08-27T11:12:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.926+0000 7f5ecea20700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec8072300 con 0x7f5ec8159ee0 2023-08-27T11:12:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ec80aeab0 0x7f5ec8071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ec80aeab0 0x7f5ec8071990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51998/0 (socket says 172.21.15.45:51998) 2023-08-27T11:12:36.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec7fff700 1 -- 172.21.15.45:0/2714994361 learned_addr learned my addr 172.21.15.45:0/2714994361 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f08e89c3700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f08c806a0e0 0x7f08c806c5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009ad0 tx=0x7f08d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec7fff700 1 -- 172.21.15.45:0/2714994361 >> v2:172.21.15.135:3300/0 conn(0x7f5ec8159ee0 msgr2=0x7f5ec80cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec7fff700 1 --2- 172.21.15.45:0/2714994361 >> v2:172.21.15.135:3300/0 conn(0x7f5ec8159ee0 0x7f5ec80cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec7fff700 1 -- 172.21.15.45:0/2714994361 >> v1:172.21.15.135:6789/0 conn(0x7f5ec80ccb30 legacy=0x7f5ec80c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec7fff700 1 -- 172.21.15.45:0/2714994361 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80725e0 con 0x7f5ec80aeab0 2023-08-27T11:12:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec6ffd700 1 -- 172.21.15.45:0/2714994361 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500490687 0 0) 0x7f5ec80cc160 con 0x7f5ec80ccb30 2023-08-27T11:12:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec7fff700 1 --2- 172.21.15.45:0/2714994361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80aeab0 0x7f5ec8071990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80cbb10 tx=0x7f5ebc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f3fcba8adbd812 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec6ffd700 1 -- 172.21.15.45:0/2714994361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc005c80 con 0x7f5ec80aeab0 2023-08-27T11:12:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec6ffd700 1 -- 172.21.15.45:0/2714994361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc005de0 con 0x7f5ec80aeab0 2023-08-27T11:12:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7f5ec6ffd700 1 -- 172.21.15.45:0/2714994361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00dc30 con 0x7f5ec80aeab0 2023-08-27T11:12:36.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7f9345d9b700 1 --2- 172.21.15.45:0/3480552705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cab00 0x7f93480ce9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9338009840 tx=0x7f9338009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0c4b3d3c0facae server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7f06c771c700 1 Processor -- start 2023-08-27T11:12:36.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7f08e91c4700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.45:6824/1621076447,v1:172.21.15.45:6825/1621076447] conn(0x7f08c8073dc0 0x7f08c8076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7f934559a700 1 -- 172.21.15.45:0/3480552705 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338004230 con 0x7f93480cab00 2023-08-27T11:12:36.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.927+0000 7fc844930700 1 Processor -- start 2023-08-27T11:12:36.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7f456a7fc700 1 -- 172.21.15.45:0/3848996887 <== osd.1 v2:172.21.15.45:6808/2953012420 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f4558076820 con 0x7f4558073d10 2023-08-27T11:12:36.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7f08e91c4700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.45:6824/1621076447,v1:172.21.15.45:6825/1621076447] conn(0x7f08c8073dc0 0x7f08c8076200 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7f5ecea20700 1 -- 172.21.15.45:0/2714994361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80aeab0 msgr2=0x7f5ec8071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7f5ecea20700 1 --2- 172.21.15.45:0/2714994361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80aeab0 0x7f5ec8071990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80cbb10 tx=0x7f5ebc00be70 comp rx=0 tx=0).stop 2023-08-27T11:12:36.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7f5ecea20700 1 -- 172.21.15.45:0/2714994361 shutdown_connections 2023-08-27T11:12:36.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7f5ecea20700 1 --2- 172.21.15.45:0/2714994361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80aeab0 0x7f5ec8071990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7f5ecea20700 1 --2- 172.21.15.45:0/2714994361 >> v2:172.21.15.135:3300/0 conn(0x7f5ec8159ee0 0x7f5ec80cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7f5ecea20700 1 -- 172.21.15.45:0/2714994361 >> 172.21.15.45:0/2714994361 conn(0x7f5ec80b2420 msgr2=0x7f5ec80b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7f08e17fa700 1 -- 172.21.15.45:0/2958414300 <== osd.2 v2:172.21.15.45:6824/1621076447 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f08c80768d0 con 0x7f08c8073dc0 2023-08-27T11:12:36.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7f934559a700 1 -- 172.21.15.45:0/3480552705 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338004390 con 0x7f93480cab00 2023-08-27T11:12:36.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7f934559a700 1 -- 172.21.15.45:0/3480552705 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93380036a0 con 0x7f93480cab00 2023-08-27T11:12:36.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7fc844930700 1 -- start start 2023-08-27T11:12:36.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7fc844930700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 0x7fc8400c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7fc844930700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8400c9490 0x7fc8401584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7fc844930700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc840159590 0x7fc84015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7fc844930700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400add60 con 0x7fc8400c9490 2023-08-27T11:12:36.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7fc844930700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8401589f0 con 0x7fc840159590 2023-08-27T11:12:36.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7fc844930700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400ad3c0 con 0x7fc840070430 2023-08-27T11:12:36.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8400c9490 0x7fc8401584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8400c9490 0x7fc8401584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52014/0 (socket says 172.21.15.45:52014) 2023-08-27T11:12:36.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83dd9b700 1 -- 172.21.15.45:0/2214478888 learned_addr learned my addr 172.21.15.45:0/2214478888 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83ed9d700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc840159590 0x7fc84015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83e59c700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 0x7fc8400c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.928+0000 7f06c771c700 1 -- start start 2023-08-27T11:12:36.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7f06c771c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c00c9990 0x7f06c0162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7f06c771c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c006e600 con 0x7f06c0161940 2023-08-27T11:12:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7f06c771c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c01625d0 con 0x7f06c006e170 2023-08-27T11:12:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.929+0000 7f06c771c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0161d20 con 0x7f06c00c9990 2023-08-27T11:12:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7f06c54b8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06c0161940 0x7f06c00c9280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38782/0 (socket says 172.21.15.45:38782) 2023-08-27T11:12:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7f06c54b8700 1 -- 172.21.15.45:0/4188907706 learned_addr learned my addr 172.21.15.45:0/4188907706 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83ed9d700 1 -- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 msgr2=0x7fc8400c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83ed9d700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc840070430 0x7fc8400c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83ed9d700 1 -- 172.21.15.45:0/2214478888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8400c9490 msgr2=0x7fc8401584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83ed9d700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8400c9490 0x7fc8401584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.930+0000 7fc83ed9d700 1 -- 172.21.15.45:0/2214478888 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83800ab80 con 0x7fc840159590 2023-08-27T11:12:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7fec2d234700 1 Processor -- start 2023-08-27T11:12:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7f8e473fc700 1 Processor -- start 2023-08-27T11:12:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7fec2d234700 1 -- start start 2023-08-27T11:12:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7f934cd3e700 1 -- 172.21.15.45:0/3480552705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cab00 msgr2=0x7f93480ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7f934cd3e700 1 --2- 172.21.15.45:0/3480552705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cab00 0x7f93480ce9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9338009840 tx=0x7f9338009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7f5ecea20700 1 -- 172.21.15.45:0/2714994361 shutdown_connections 2023-08-27T11:12:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7f5ecea20700 1 -- 172.21.15.45:0/2714994361 wait complete. 2023-08-27T11:12:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7f8e473fc700 1 -- start start 2023-08-27T11:12:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7f8e473fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e40157c80 0x7f8e4006e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7f8e473fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e4015a600 0x7f8e4015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7f8e473fc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e400cc0b0 con 0x7f8e4015afa0 2023-08-27T11:12:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7f8e473fc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e400ca3b0 con 0x7f8e4015a600 2023-08-27T11:12:36.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7f8e473fc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4006ecd0 con 0x7f8e40157c80 2023-08-27T11:12:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7f8e44997700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e4015a600 0x7f8e4015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7f8e44997700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e4015a600 0x7f8e4015aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52024/0 (socket says 172.21.15.45:52024) 2023-08-27T11:12:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7fc83ed9d700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc840159590 0x7fc84015c150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc838014000 tx=0x7fc838004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7fc82effd700 1 -- 172.21.15.45:0/2214478888 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838004240 con 0x7fc840159590 2023-08-27T11:12:36.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7fc82effd700 1 -- 172.21.15.45:0/2214478888 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8380043a0 con 0x7fc840159590 2023-08-27T11:12:36.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7fc82effd700 1 -- 172.21.15.45:0/2214478888 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838004750 con 0x7fc840159590 2023-08-27T11:12:36.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8400ad6a0 con 0x7fc840159590 2023-08-27T11:12:36.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84015c690 con 0x7fc840159590 2023-08-27T11:12:36.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fec2d234700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fec280cf440 0x7fec28165600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fec2d234700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec28071630 0x7fec28071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fec2d234700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec280b9070 con 0x7fec28165220 2023-08-27T11:12:36.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fec2d234700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec280b66c0 con 0x7fec28071630 2023-08-27T11:12:36.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fec2d234700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec280b85c0 con 0x7fec280cf440 2023-08-27T11:12:36.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fec2659c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fec280cf440 0x7fec28165600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fec2659c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fec280cf440 0x7fec28165600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:40874/0 (socket says 172.21.15.45:40874) 2023-08-27T11:12:36.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fec2659c700 1 -- 172.21.15.45:0/3207855583 learned_addr learned my addr 172.21.15.45:0/3207855583 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fec2759e700 1 --2- 172.21.15.45:0/3207855583 >> v2:172.21.15.45:3300/0 conn(0x7fec28071630 0x7fec28071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7f8e44997700 1 -- 172.21.15.45:0/3199737422 learned_addr learned my addr 172.21.15.45:0/3199737422 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:36.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.932+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8400ccf20 con 0x7fc840159590 2023-08-27T11:12:36.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.931+0000 7f06c4cb7700 1 --2- 172.21.15.45:0/4188907706 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c00c9990 0x7f06c0162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:36.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f06c4cb7700 1 -- 172.21.15.45:0/4188907706 >> v1:172.21.15.135:6789/0 conn(0x7f06c006e170 legacy=0x7f06c00cca10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:36.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f06c4cb7700 1 -- 172.21.15.45:0/4188907706 >> v1:172.21.15.45:6789/0 conn(0x7f06c0161940 legacy=0x7f06c00c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f06c4cb7700 1 -- 172.21.15.45:0/4188907706 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00c2140 con 0x7f06c00c9990 2023-08-27T11:12:36.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7fec2659c700 1 -- 172.21.15.45:0/3207855583 >> v2:172.21.15.45:3300/0 conn(0x7fec28071630 msgr2=0x7fec28071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7fec2659c700 1 --2- 172.21.15.45:0/3207855583 >> v2:172.21.15.45:3300/0 conn(0x7fec28071630 0x7fec28071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7fec2659c700 1 -- 172.21.15.45:0/3207855583 >> v1:172.21.15.45:6789/0 conn(0x7fec28165220 legacy=0x7fec280b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7fec2659c700 1 -- 172.21.15.45:0/3207855583 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec28165b40 con 0x7fec280cf440 2023-08-27T11:12:36.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f8e44997700 1 -- 172.21.15.45:0/3199737422 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e40157c80 msgr2=0x7f8e4006e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:36.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f8e44997700 1 --2- 172.21.15.45:0/3199737422 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e40157c80 0x7f8e4006e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f8e44997700 1 -- 172.21.15.45:0/3199737422 >> v1:172.21.15.45:6789/0 conn(0x7f8e4015afa0 legacy=0x7f8e4015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f8e44997700 1 -- 172.21.15.45:0/3199737422 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4015b430 con 0x7f8e4015a600 2023-08-27T11:12:36.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f934cd3e700 1 -- 172.21.15.45:0/3480552705 shutdown_connections 2023-08-27T11:12:36.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f934cd3e700 1 --2- 172.21.15.45:0/3480552705 >> v2:172.21.15.45:3300/0 conn(0x7f93480c32c0 0x7f93480ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f934cd3e700 1 --2- 172.21.15.45:0/3480552705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cab00 0x7f93480ce9d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f934cd3e700 1 -- 172.21.15.45:0/3480552705 >> 172.21.15.45:0/3480552705 conn(0x7f93480cc7e0 msgr2=0x7f93480ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7fec2659c700 1 --2- 172.21.15.45:0/3207855583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec280cf440 0x7fec28165600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec280b2ac0 tx=0x7fec18009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e9dd4d4fda5cf8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f06c4cb7700 1 --2- 172.21.15.45:0/4188907706 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c00c9990 0x7f06c0162a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009dc0 tx=0x7f06b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b6fc78ec18f5f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7f8e44997700 1 --2- 172.21.15.45:0/3199737422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e4015a600 0x7f8e4015aa60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00bb90 tx=0x7f8e3c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaaa416e74e9406 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:36.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7fec25d9b700 1 -- 172.21.15.45:0/3207855583 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18004230 con 0x7fec280cf440 2023-08-27T11:12:36.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7fec25d9b700 1 -- 172.21.15.45:0/3207855583 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec18004390 con 0x7fec280cf440 2023-08-27T11:12:36.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.933+0000 7fec25d9b700 1 -- 172.21.15.45:0/3207855583 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec180036a0 con 0x7fec280cf440 2023-08-27T11:12:36.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.934+0000 7f8e37fff700 1 -- 172.21.15.45:0/3199737422 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00dd50 con 0x7f8e4015a600 2023-08-27T11:12:36.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.934+0000 7f8e37fff700 1 -- 172.21.15.45:0/3199737422 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c010470 con 0x7f8e4015a600 2023-08-27T11:12:36.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.934+0000 7f8e37fff700 1 -- 172.21.15.45:0/3199737422 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c010800 con 0x7f8e4015a600 2023-08-27T11:12:36.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.934+0000 7f5ecea20700 1 Processor -- start 2023-08-27T11:12:36.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.934+0000 7f5ecea20700 1 -- start start 2023-08-27T11:12:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.934+0000 7f06b7fff700 1 -- 172.21.15.45:0/4188907706 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800e070 con 0x7f06c00c9990 2023-08-27T11:12:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f06b7fff700 1 -- 172.21.15.45:0/4188907706 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b800bad0 con 0x7f06c00c9990 2023-08-27T11:12:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f06b7fff700 1 -- 172.21.15.45:0/4188907706 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800be60 con 0x7f06c00c9990 2023-08-27T11:12:36.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f8e473fc700 1 -- 172.21.15.45:0/3199737422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e4015a600 msgr2=0x7f8e4015aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f8e473fc700 1 --2- 172.21.15.45:0/3199737422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e4015a600 0x7f8e4015aa60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00bb90 tx=0x7f8e3c00bea0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f8e473fc700 1 -- 172.21.15.45:0/3199737422 shutdown_connections 2023-08-27T11:12:36.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f8e473fc700 1 --2- 172.21.15.45:0/3199737422 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e40157c80 0x7f8e4006e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f8e473fc700 1 --2- 172.21.15.45:0/3199737422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e4015a600 0x7f8e4015aa60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f8e473fc700 1 -- 172.21.15.45:0/3199737422 >> 172.21.15.45:0/3199737422 conn(0x7f8e400cc7e0 msgr2=0x7f8e400ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.934+0000 7fc82effd700 1 -- 172.21.15.45:0/2214478888 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc83801e470 con 0x7fc840159590 2023-08-27T11:12:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fc82effd700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc82006a0b0 0x7fc82006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.934+0000 7fec2d234700 1 -- 172.21.15.45:0/3207855583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec280cf440 msgr2=0x7fec28165600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.934+0000 7fec2d234700 1 --2- 172.21.15.45:0/3207855583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec280cf440 0x7fec28165600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec280b2ac0 tx=0x7fec18009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fec2d234700 1 -- 172.21.15.45:0/3207855583 shutdown_connections 2023-08-27T11:12:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fec2d234700 1 --2- 172.21.15.45:0/3207855583 >> v2:172.21.15.45:3300/0 conn(0x7fec28071630 0x7fec28071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fec2d234700 1 --2- 172.21.15.45:0/3207855583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec280cf440 0x7fec28165600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fec2d234700 1 -- 172.21.15.45:0/3207855583 >> 172.21.15.45:0/3207855583 conn(0x7fec280d3090 msgr2=0x7fec280d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fec2d234700 1 -- 172.21.15.45:0/3207855583 shutdown_connections 2023-08-27T11:12:37.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f5ecea20700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ec80aeab0 0x7f5ec8157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f5ecea20700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ec8159ee0 0x7f5ec80c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f5ecea20700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80c60d0 0x7f5ec80c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f5ecea20700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80c36a0 con 0x7f5ec80c60d0 2023-08-27T11:12:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f5ecea20700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80c3940 con 0x7f5ec80aeab0 2023-08-27T11:12:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f5ecea20700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80c3c20 con 0x7f5ec8159ee0 2023-08-27T11:12:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fec2d234700 1 -- 172.21.15.45:0/3207855583 wait complete. 2023-08-27T11:12:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f8e473fc700 1 -- 172.21.15.45:0/3199737422 shutdown_connections 2023-08-27T11:12:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec77fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ec8159ee0 0x7f5ec80c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7f8e473fc700 1 -- 172.21.15.45:0/3199737422 wait complete. 2023-08-27T11:12:37.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fc82effd700 1 -- 172.21.15.45:0/2214478888 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc83808a4e0 con 0x7fc840159590 2023-08-27T11:12:37.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fc82effd700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6824/3277146230,v1:172.21.15.135:6825/3277146230] conn(0x7fc820073d10 0x7fc820076150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fc82effd700 1 -- 172.21.15.45:0/2214478888 --> [v2:172.21.15.135:6824/3277146230,v1:172.21.15.135:6825/3277146230] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc820076820 con 0x7fc820073d10 2023-08-27T11:12:37.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.935+0000 7fc82effd700 1 -- 172.21.15.45:0/2214478888 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc83808a930 con 0x7fc840159590 2023-08-27T11:12:37.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec77fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ec8159ee0 0x7f5ec80c2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:49940/0 (socket says 172.21.15.45:49940) 2023-08-27T11:12:37.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec77fe700 1 -- 172.21.15.45:0/789138877 learned_addr learned my addr 172.21.15.45:0/789138877 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec7fff700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ec80aeab0 0x7f5ec8157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f06c771c700 1 -- 172.21.15.45:0/4188907706 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c00c9990 msgr2=0x7f06c0162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f06c771c700 1 --2- 172.21.15.45:0/4188907706 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c00c9990 0x7f06c0162a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009dc0 tx=0x7f06b800b330 comp rx=0 tx=0).stop 2023-08-27T11:12:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5eccfbd700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80c60d0 0x7f5ec80c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec77fe700 1 -- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ec80aeab0 msgr2=0x7f5ec8157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec77fe700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ec80aeab0 0x7f5ec8157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec77fe700 1 -- 172.21.15.45:0/789138877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80c60d0 msgr2=0x7f5ec80c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec77fe700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80c60d0 0x7f5ec80c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec77fe700 1 -- 172.21.15.45:0/789138877 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc00b7a0 con 0x7f5ec8159ee0 2023-08-27T11:12:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec7fff700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ec80aeab0 0x7f5ec8157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5eccfbd700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80c60d0 0x7f5ec80c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f5ec77fe700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ec8159ee0 0x7f5ec80c2bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000b9c0 tx=0x7f5ec000bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f06c771c700 1 -- 172.21.15.45:0/4188907706 shutdown_connections 2023-08-27T11:12:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f06c771c700 1 --2- 172.21.15.45:0/4188907706 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c00c9990 0x7f06c0162a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.936+0000 7f06c771c700 1 -- 172.21.15.45:0/4188907706 >> 172.21.15.45:0/4188907706 conn(0x7f06c00cec30 msgr2=0x7f06c00cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7fc83dd9b700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6824/3277146230,v1:172.21.15.135:6825/3277146230] conn(0x7fc820073d10 0x7fc820076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7f06c771c700 1 -- 172.21.15.45:0/4188907706 shutdown_connections 2023-08-27T11:12:37.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7fec2d234700 1 Processor -- start 2023-08-27T11:12:37.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7f8e473fc700 1 Processor -- start 2023-08-27T11:12:37.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7f934cd3e700 1 -- 172.21.15.45:0/3480552705 shutdown_connections 2023-08-27T11:12:37.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7f5ec4ff9700 1 -- 172.21.15.45:0/789138877 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000ca30 con 0x7f5ec8159ee0 2023-08-27T11:12:37.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7f5ec4ff9700 1 -- 172.21.15.45:0/789138877 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec000cbb0 con 0x7f5ec8159ee0 2023-08-27T11:12:37.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7f5ec4ff9700 1 -- 172.21.15.45:0/789138877 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0014440 con 0x7f5ec8159ee0 2023-08-27T11:12:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec80ce2d0 con 0x7f5ec8159ee0 2023-08-27T11:12:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.938+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec8157840 con 0x7f5ec8159ee0 2023-08-27T11:12:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7f06c771c700 1 -- 172.21.15.45:0/4188907706 wait complete. 2023-08-27T11:12:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fc83e59c700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc82006a0b0 0x7fc82006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.937+0000 7fec2d234700 1 -- start start 2023-08-27T11:12:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7f06c771c700 1 Processor -- start 2023-08-27T11:12:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.938+0000 7fec2d234700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec28071630 0x7fec2814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.938+0000 7fec2d234700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec280cf440 0x7fec28161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2d234700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec28165220 0x7fec280d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2d234700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec28163100 con 0x7fec28165220 2023-08-27T11:12:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2d234700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec281624f0 con 0x7fec28071630 2023-08-27T11:12:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2d234700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec281627d0 con 0x7fec280cf440 2023-08-27T11:12:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7f06c771c700 1 -- start start 2023-08-27T11:12:37.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec28165220 0x7fec280d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec28165220 0x7fec280d16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52044/0 (socket says 172.21.15.45:52044) 2023-08-27T11:12:37.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2759e700 1 -- 172.21.15.45:0/2798189622 learned_addr learned my addr 172.21.15.45:0/2798189622 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2759e700 1 -- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec280cf440 msgr2=0x7fec28161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2759e700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec280cf440 0x7fec28161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2759e700 1 -- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec28071630 msgr2=0x7fec2814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2759e700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec28071630 0x7fec2814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2759e700 1 -- 172.21.15.45:0/2798189622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec18009550 con 0x7fec28165220 2023-08-27T11:12:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.938+0000 7f934cd3e700 1 -- 172.21.15.45:0/3480552705 wait complete. 2023-08-27T11:12:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7f934cd3e700 1 Processor -- start 2023-08-27T11:12:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.939+0000 7fec2759e700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec28165220 0x7fec280d16d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00e950 tx=0x7fec1c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7fec0f7fe700 1 -- 172.21.15.45:0/2798189622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c009730 con 0x7fec28165220 2023-08-27T11:12:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7fec0f7fe700 1 -- 172.21.15.45:0/2798189622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c00c850 con 0x7fec28165220 2023-08-27T11:12:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7fec0f7fe700 1 -- 172.21.15.45:0/2798189622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00cbe0 con 0x7fec28165220 2023-08-27T11:12:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c771c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c006e170 0x7f06c00717f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c771c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06c00c9990 0x7f06c0148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c771c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c0161940 0x7f06c00cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c771c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0148950 con 0x7f06c006e170 2023-08-27T11:12:37.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c771c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00c2b20 con 0x7f06c00c9990 2023-08-27T11:12:37.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c771c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00ae620 con 0x7f06c0161940 2023-08-27T11:12:37.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c4cb7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06c00c9990 0x7f06c0148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c4cb7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06c00c9990 0x7f06c0148410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:40904/0 (socket says 172.21.15.45:40904) 2023-08-27T11:12:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c4cb7700 1 -- 172.21.15.45:0/1653827964 learned_addr learned my addr 172.21.15.45:0/1653827964 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ec80c3ab0 con 0x7f5ec8159ee0 2023-08-27T11:12:37.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f06c5cb9700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c0161940 0x7f06c00cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f06c5cb9700 1 -- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06c00c9990 msgr2=0x7f06c0148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f06c5cb9700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06c00c9990 0x7f06c0148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f06c5cb9700 1 -- 172.21.15.45:0/1653827964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c006e170 msgr2=0x7f06c00717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f06c5cb9700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c006e170 0x7f06c00717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f06c5cb9700 1 -- 172.21.15.45:0/1653827964 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b8009a70 con 0x7f06c0161940 2023-08-27T11:12:37.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f934cd3e700 1 -- start start 2023-08-27T11:12:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.940+0000 7f934cd3e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93480c32c0 0x7f93480c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f934cd3e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93480cab00 0x7f9348153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f934cd3e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cef10 0x7f934815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f934cd3e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934814a3e0 con 0x7f93480c32c0 2023-08-27T11:12:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f934cd3e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934815e370 con 0x7f93480cab00 2023-08-27T11:12:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f934cd3e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934813dae0 con 0x7f93480cef10 2023-08-27T11:12:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f9345d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93480cab00 0x7f9348153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f9346d9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cef10 0x7f934815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f9345d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93480cab00 0x7f9348153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:40912/0 (socket says 172.21.15.45:40912) 2023-08-27T11:12:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f9345d9b700 1 -- 172.21.15.45:0/2171225309 learned_addr learned my addr 172.21.15.45:0/2171225309 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f06c5cb9700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c0161940 0x7f06c00cd880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b3c0 tx=0x7f06b000b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f06b5ffb700 1 -- 172.21.15.45:0/1653827964 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0011830 con 0x7f06c0161940 2023-08-27T11:12:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c00ae900 con 0x7f06c0161940 2023-08-27T11:12:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.942+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c00aeec0 con 0x7f06c0161940 2023-08-27T11:12:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.942+0000 7f06b5ffb700 1 -- 172.21.15.45:0/1653827964 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b00119b0 con 0x7f06c0161940 2023-08-27T11:12:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.942+0000 7f06b5ffb700 1 -- 172.21.15.45:0/1653827964 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0011d60 con 0x7f06c0161940 2023-08-27T11:12:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.942+0000 7f5ec4ff9700 1 -- 172.21.15.45:0/789138877 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5ec001d3b0 con 0x7f5ec8159ee0 2023-08-27T11:12:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec28162ab0 con 0x7fec28165220 2023-08-27T11:12:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec28162150 con 0x7fec28165220 2023-08-27T11:12:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f9345d9b700 1 -- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cef10 msgr2=0x7f934815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f9345d9b700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f93480cef10 0x7f934815e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f9345d9b700 1 -- 172.21.15.45:0/2171225309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93480c32c0 msgr2=0x7f93480c1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f9345d9b700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93480c32c0 0x7f93480c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f9345d9b700 1 -- 172.21.15.45:0/2171225309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340009b30 con 0x7f93480cab00 2023-08-27T11:12:37.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.942+0000 7f9345d9b700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93480cab00 0x7f9348153b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9338009b90 tx=0x7f9338004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f5ec4ff9700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5eb406a0e0 0x7f5eb406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f5ec4ff9700 1 -- 172.21.15.45:0/789138877 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5ec0088670 con 0x7f5ec8159ee0 2023-08-27T11:12:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.941+0000 7f8e473fc700 1 -- start start 2023-08-27T11:12:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e473fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e40157c80 0x7f8e401f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e473fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e4015a600 0x7f8e400b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e473fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e4015afa0 0x7f8e400b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e473fc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400c4a10 con 0x7f8e4015afa0 2023-08-27T11:12:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e473fc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400c3d90 con 0x7f8e40157c80 2023-08-27T11:12:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e473fc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400c4070 con 0x7f8e4015a600 2023-08-27T11:12:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e44997700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e4015a600 0x7f8e400b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e44997700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e4015a600 0x7f8e400b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:49970/0 (socket says 172.21.15.45:49970) 2023-08-27T11:12:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e44997700 1 -- 172.21.15.45:0/3007717805 learned_addr learned my addr 172.21.15.45:0/3007717805 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f5ec4ff9700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.45:6800/1741844211,v1:172.21.15.45:6801/1741844211] conn(0x7f5eb4073d40 0x7f5eb4076180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f5ec4ff9700 1 -- 172.21.15.45:0/789138877 --> [v2:172.21.15.45:6800/1741844211,v1:172.21.15.45:6801/1741844211] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5eb4076850 con 0x7f5eb4073d40 2023-08-27T11:12:37.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f5ec4ff9700 1 -- 172.21.15.45:0/789138877 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ec008abf0 con 0x7f5ec8159ee0 2023-08-27T11:12:37.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f06a0000fc0 con 0x7f06c0161940 2023-08-27T11:12:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f9336ffd700 1 -- 172.21.15.45:0/2171225309 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338003d60 con 0x7f93480cab00 2023-08-27T11:12:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f9336ffd700 1 -- 172.21.15.45:0/2171225309 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338011630 con 0x7f93480cab00 2023-08-27T11:12:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f9336ffd700 1 -- 172.21.15.45:0/2171225309 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93380119c0 con 0x7f93480cab00 2023-08-27T11:12:37.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f5ec7fff700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5eb406a0e0 0x7f5eb406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.944+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93380094a0 con 0x7f93480cab00 2023-08-27T11:12:37.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.944+0000 7f5ec7fff700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5eb406a0e0 0x7f5eb406c5a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00beb0 tx=0x7f5ebc00bf90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e45198700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e40157c80 0x7f8e401f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e44997700 1 -- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e40157c80 msgr2=0x7f8e401f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e44997700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e40157c80 0x7f8e401f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e45999700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e4015afa0 0x7f8e400b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.943+0000 7f8e44997700 1 -- 172.21.15.45:0/3007717805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e4015afa0 msgr2=0x7f8e400b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.944+0000 7f8e44997700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e4015afa0 0x7f8e400b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.944+0000 7f8e44997700 1 -- 172.21.15.45:0/3007717805 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c00b7f0 con 0x7f8e4015a600 2023-08-27T11:12:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.944+0000 7f8e45999700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e4015afa0 0x7f8e400b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.944+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934814f9e0 con 0x7f93480cab00 2023-08-27T11:12:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.944+0000 7fec0f7fe700 1 -- 172.21.15.45:0/2798189622 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fec1c00cd40 con 0x7fec28165220 2023-08-27T11:12:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f934806d020 con 0x7f93480cab00 2023-08-27T11:12:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7fec0f7fe700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec1406a0e0 0x7fec1406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7fec0f7fe700 1 -- 172.21.15.45:0/2798189622 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fec1c088990 con 0x7fec28165220 2023-08-27T11:12:37.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.944+0000 7f8e44997700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e4015a600 0x7f8e400b00d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e40074300 tx=0x7f8e3c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7f5eccfbd700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.45:6800/1741844211,v1:172.21.15.45:6801/1741844211] conn(0x7f5eb4073d40 0x7f5eb4076180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7fec26d9d700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec1406a0e0 0x7fec1406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7fc83dd9b700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6824/3277146230,v1:172.21.15.135:6825/3277146230] conn(0x7fc820073d10 0x7fc820076150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7fc83e59c700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc82006a0b0 0x7fc82006c570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc830007fd0 tx=0x7fc830000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.946+0000 7fec26d9d700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec1406a0e0 0x7fec1406c5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec2000ab90 tx=0x7fec20009230 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.944+0000 7f06b5ffb700 1 -- 172.21.15.45:0/1653827964 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f06b001a400 con 0x7f06c0161940 2023-08-27T11:12:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7f06b5ffb700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f06a406a0e0 0x7f06a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7f06b5ffb700 1 -- 172.21.15.45:0/1653827964 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f06b0088810 con 0x7f06c0161940 2023-08-27T11:12:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7f06c54b8700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f06a406a0e0 0x7f06a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7f06b5ffb700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.45:6816/1641214045,v1:172.21.15.45:6817/1641214045] conn(0x7f06a4073dc0 0x7f06a4076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7f06b5ffb700 1 -- 172.21.15.45:0/1653827964 --> [v2:172.21.15.45:6816/1641214045,v1:172.21.15.45:6817/1641214045] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f06a40768d0 con 0x7f06a4073dc0 2023-08-27T11:12:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.945+0000 7f06b5ffb700 1 -- 172.21.15.45:0/1653827964 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b008ae30 con 0x7f06c0161940 2023-08-27T11:12:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.946+0000 7f06c54b8700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f06a406a0e0 0x7f06a406c5a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f06bc000f60 tx=0x7f06bc018a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.946+0000 7f06c4cb7700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.45:6816/1641214045,v1:172.21.15.45:6817/1641214045] conn(0x7f06a4073dc0 0x7f06a4076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.946+0000 7f9336ffd700 1 -- 172.21.15.45:0/2171225309 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9338018040 con 0x7f93480cab00 2023-08-27T11:12:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.946+0000 7f5eccfbd700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.45:6800/1741844211,v1:172.21.15.45:6801/1741844211] conn(0x7f5eb4073d40 0x7f5eb4076180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.946+0000 7f06c4cb7700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.45:6816/1641214045,v1:172.21.15.45:6817/1641214045] conn(0x7f06a4073dc0 0x7f06a4076200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.946+0000 7f9336ffd700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f932806a1b0 0x7f932806c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.946+0000 7f9336ffd700 1 -- 172.21.15.45:0/2171225309 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f93380894b0 con 0x7f93480cab00 2023-08-27T11:12:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7fec2d234700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6808/3357661340,v1:172.21.15.135:6809/3357661340] conn(0x7fec281ecff0 0x7fec28163490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 --> [v2:172.21.15.135:6808/3357661340,v1:172.21.15.135:6809/3357661340] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec280b87d0 con 0x7fec281ecff0 2023-08-27T11:12:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7f8e35ffb700 1 -- 172.21.15.45:0/3007717805 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c003bf0 con 0x7f8e4015a600 2023-08-27T11:12:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7f8e35ffb700 1 -- 172.21.15.45:0/3007717805 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c003d70 con 0x7f8e4015a600 2023-08-27T11:12:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7f8e35ffb700 1 -- 172.21.15.45:0/3007717805 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c010820 con 0x7f8e4015a600 2023-08-27T11:12:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e400c4350 con 0x7f8e4015a600 2023-08-27T11:12:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e40154aa0 con 0x7f8e4015a600 2023-08-27T11:12:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7fc82effd700 1 -- 172.21.15.45:0/2214478888 <== osd.5 v2:172.21.15.135:6824/3277146230 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc820076820 con 0x7fc820073d10 2023-08-27T11:12:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7f9336ffd700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6814/2334385273,v1:172.21.15.135:6816/2334385273] conn(0x7f9328073e90 0x7f93280762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7f9336ffd700 1 -- 172.21.15.45:0/2171225309 --> [v2:172.21.15.135:6814/2334385273,v1:172.21.15.135:6816/2334385273] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93280769a0 con 0x7f9328073e90 2023-08-27T11:12:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7f9336ffd700 1 -- 172.21.15.45:0/2171225309 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9338089860 con 0x7f93480cab00 2023-08-27T11:12:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7f934659c700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f932806a1b0 0x7f932806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.948+0000 7f9346d9d700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6814/2334385273,v1:172.21.15.135:6816/2334385273] conn(0x7f9328073e90 0x7f93280762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.948+0000 7f934659c700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f932806a1b0 0x7f932806c670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f934000e330 tx=0x7f934000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.947+0000 7fec2659c700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6808/3357661340,v1:172.21.15.135:6809/3357661340] conn(0x7fec281ecff0 0x7fec28163490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.948+0000 7f06b5ffb700 1 -- 172.21.15.45:0/1653827964 <== osd.3 v2:172.21.15.45:6816/1641214045 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f06a40768d0 con 0x7f06a4073dc0 2023-08-27T11:12:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.948+0000 7f9346d9d700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6814/2334385273,v1:172.21.15.135:6816/2334385273] conn(0x7f9328073e90 0x7f93280762d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.949+0000 7f9336ffd700 1 -- 172.21.15.45:0/2171225309 <== osd.6 v2:172.21.15.135:6814/2334385273 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f93280769a0 con 0x7f9328073e90 2023-08-27T11:12:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.950+0000 7f8e35ffb700 1 -- 172.21.15.45:0/3007717805 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8e3c019400 con 0x7f8e4015a600 2023-08-27T11:12:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.951+0000 7f8e35ffb700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e2406a1b0 0x7f8e2406c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.951+0000 7f8e35ffb700 1 -- 172.21.15.45:0/3007717805 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8e3c089c60 con 0x7f8e4015a600 2023-08-27T11:12:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.951+0000 7f8e45198700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e2406a1b0 0x7f8e2406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.952+0000 7f8e45198700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e2406a1b0 0x7f8e2406c670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e4015a2c0 tx=0x7f8e30009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.960+0000 7f8e473fc700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6800/1785499536,v1:172.21.15.135:6801/1785499536] conn(0x7f8e200015e0 0x7f8e20003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.960+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 --> [v2:172.21.15.135:6800/1785499536,v1:172.21.15.135:6801/1785499536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e20006bd0 con 0x7f8e200015e0 2023-08-27T11:12:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.960+0000 7f8e45999700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6800/1785499536,v1:172.21.15.135:6801/1785499536] conn(0x7f8e200015e0 0x7f8e20003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.960+0000 7fec2659c700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6808/3357661340,v1:172.21.15.135:6809/3357661340] conn(0x7fec281ecff0 0x7fec28163490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.961+0000 7f8e45999700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6800/1785499536,v1:172.21.15.135:6801/1785499536] conn(0x7f8e200015e0 0x7f8e20003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.962+0000 7f5ec4ff9700 1 -- 172.21.15.45:0/789138877 <== osd.0 v2:172.21.15.45:6800/1741844211 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f5eb4076850 con 0x7f5eb4073d40 2023-08-27T11:12:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.962+0000 7f8e35ffb700 1 -- 172.21.15.45:0/3007717805 <== osd.4 v2:172.21.15.135:6800/1785499536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f8e20006bd0 con 0x7f8e200015e0 2023-08-27T11:12:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.961+0000 7fec0f7fe700 1 -- 172.21.15.45:0/2798189622 <== osd.7 v2:172.21.15.135:6808/3357661340 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fec18004500 con 0x7fec281ecff0 2023-08-27T11:12:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.964+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 --> [v2:172.21.15.45:6808/2953012420,v1:172.21.15.45:6809/2953012420] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f455c002d40 con 0x7f4558073d10 2023-08-27T11:12:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.964+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 --> [v2:172.21.15.45:6824/1621076447,v1:172.21.15.45:6825/1621076447] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08e4165200 con 0x7f08c8073dc0 2023-08-27T11:12:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.965+0000 7f08e17fa700 1 -- 172.21.15.45:0/2958414300 <== osd.2 v2:172.21.15.45:6824/1621076447 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08dc0020f0 con 0x7f08c8073dc0 2023-08-27T11:12:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.965+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 >> [v2:172.21.15.45:6824/1621076447,v1:172.21.15.45:6825/1621076447] conn(0x7f08c8073dc0 msgr2=0x7f08c8076200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.965+0000 7f08eac27700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.45:6824/1621076447,v1:172.21.15.45:6825/1621076447] conn(0x7f08c8073dc0 0x7f08c8076200 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.965+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f08c806a0e0 msgr2=0x7f08c806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.965+0000 7f08eac27700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f08c806a0e0 0x7f08c806c5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009ad0 tx=0x7f08d0008040 comp rx=0 tx=0).stop 2023-08-27T11:12:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.965+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf860 msgr2=0x7f08e40b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.965+0000 7f08eac27700 1 --2- 172.21.15.45:0/2958414300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08e40cf860 0x7f08e40b3f40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f08d8004000 tx=0x7f08d8004750 comp rx=0 tx=0).stop 2023-08-27T11:12:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.965+0000 7f08e91c4700 1 -- 172.21.15.45:0/2958414300 reap_dead start 2023-08-27T11:12:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.966+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 shutdown_connections 2023-08-27T11:12:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.966+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 >> 172.21.15.45:0/2958414300 conn(0x7f08e40cc7c0 msgr2=0x7f08e40c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.966+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 shutdown_connections 2023-08-27T11:12:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.966+0000 7f08eac27700 1 -- 172.21.15.45:0/2958414300 wait complete. 2023-08-27T11:12:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.976+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 --> [v2:172.21.15.135:6814/2334385273,v1:172.21.15.135:6816/2334385273] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f93480cfa30 con 0x7f9328073e90 2023-08-27T11:12:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.977+0000 7f9336ffd700 1 -- 172.21.15.45:0/2171225309 <== osd.6 v2:172.21.15.135:6814/2334385273 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f933c009530 con 0x7f9328073e90 2023-08-27T11:12:37.072 INFO:teuthology.orchestra.run.smithi045.stdout:55834574860 2023-08-27T11:12:37.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-27T11:12:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.982+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 --> [v2:172.21.15.135:6824/3277146230,v1:172.21.15.135:6825/3277146230] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc84006cda0 con 0x7fc820073d10 2023-08-27T11:12:37.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 --> [v2:172.21.15.45:6816/1641214045,v1:172.21.15.45:6817/1641214045] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f06a0002d40 con 0x7f06a4073dc0 2023-08-27T11:12:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6814/2334385273,v1:172.21.15.135:6816/2334385273] conn(0x7f9328073e90 msgr2=0x7f93280762d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6814/2334385273,v1:172.21.15.135:6816/2334385273] conn(0x7f9328073e90 0x7f93280762d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f932806a1b0 msgr2=0x7f932806c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f932806a1b0 0x7f932806c670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f934000e330 tx=0x7f934000b040 comp rx=0 tx=0).stop 2023-08-27T11:12:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93480cab00 msgr2=0x7f9348153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 --2- 172.21.15.45:0/2171225309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93480cab00 0x7f9348153b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9338009b90 tx=0x7f9338004750 comp rx=0 tx=0).stop 2023-08-27T11:12:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f9346d9d700 1 -- 172.21.15.45:0/2171225309 reap_dead start 2023-08-27T11:12:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 shutdown_connections 2023-08-27T11:12:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 >> 172.21.15.45:0/2171225309 conn(0x7f93480cc7e0 msgr2=0x7f934806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 shutdown_connections 2023-08-27T11:12:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f934cd3e700 1 -- 172.21.15.45:0/2171225309 wait complete. 2023-08-27T11:12:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7f06b5ffb700 1 -- 172.21.15.45:0/1653827964 <== osd.3 v2:172.21.15.45:6816/1641214045 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f06a0002d40 con 0x7f06a4073dc0 2023-08-27T11:12:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.983+0000 7fc82effd700 1 -- 172.21.15.45:0/2214478888 <== osd.5 v2:172.21.15.135:6824/3277146230 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc8340015e0 con 0x7fc820073d10 2023-08-27T11:12:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.984+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 >> [v2:172.21.15.45:6816/1641214045,v1:172.21.15.45:6817/1641214045] conn(0x7f06a4073dc0 msgr2=0x7f06a4076200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.984+0000 7f06c771c700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.45:6816/1641214045,v1:172.21.15.45:6817/1641214045] conn(0x7f06a4073dc0 0x7f06a4076200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.984+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f06a406a0e0 msgr2=0x7f06a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.984+0000 7f06c771c700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f06a406a0e0 0x7f06a406c5a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f06bc000f60 tx=0x7f06bc018a80 comp rx=0 tx=0).stop 2023-08-27T11:12:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.984+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c0161940 msgr2=0x7f06c00cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.984+0000 7f06c771c700 1 --2- 172.21.15.45:0/1653827964 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06c0161940 0x7f06c00cd880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b3c0 tx=0x7f06b000b780 comp rx=0 tx=0).stop 2023-08-27T11:12:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.984+0000 7f06c5cb9700 1 -- 172.21.15.45:0/1653827964 reap_dead start 2023-08-27T11:12:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.984+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 shutdown_connections 2023-08-27T11:12:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.984+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 >> 172.21.15.45:0/1653827964 conn(0x7f06c00cec30 msgr2=0x7f06c0159770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.980+0000 7f456a7fc700 1 -- 172.21.15.45:0/3848996887 <== osd.1 v2:172.21.15.45:6808/2953012420 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f455c002d40 con 0x7f4558073d10 2023-08-27T11:12:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.987+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 >> [v2:172.21.15.45:6808/2953012420,v1:172.21.15.45:6809/2953012420] conn(0x7f4558073d10 msgr2=0x7f4558076150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.987+0000 7f457bb86700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.45:6808/2953012420,v1:172.21.15.45:6809/2953012420] conn(0x7f4558073d10 0x7f4558076150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.987+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f455806a0b0 msgr2=0x7f455806c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.987+0000 7f457bb86700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f455806a0b0 0x7f455806c570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f45740b00a0 tx=0x7f45700090e0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.987+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740ae080 msgr2=0x7f4574154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.987+0000 7f457bb86700 1 --2- 172.21.15.45:0/3848996887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740ae080 0x7f4574154290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f456400e810 tx=0x7f456400eb20 comp rx=0 tx=0).stop 2023-08-27T11:12:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.987+0000 7f457a123700 1 -- 172.21.15.45:0/3848996887 reap_dead start 2023-08-27T11:12:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.988+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 shutdown_connections 2023-08-27T11:12:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.988+0000 7f06c771c700 1 -- 172.21.15.45:0/1653827964 wait complete. 2023-08-27T11:12:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.988+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 --> [v2:172.21.15.45:6800/1741844211,v1:172.21.15.45:6801/1741844211] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ec80cbdf0 con 0x7f5eb4073d40 2023-08-27T11:12:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 shutdown_connections 2023-08-27T11:12:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 >> 172.21.15.45:0/3848996887 conn(0x7f45740cec30 msgr2=0x7f457406d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.988+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6824/3277146230,v1:172.21.15.135:6825/3277146230] conn(0x7fc820073d10 msgr2=0x7fc820076150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc844930700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6824/3277146230,v1:172.21.15.135:6825/3277146230] conn(0x7fc820073d10 0x7fc820076150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc82006a0b0 msgr2=0x7fc82006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc844930700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc82006a0b0 0x7fc82006c570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc830007fd0 tx=0x7fc830000bc0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc840159590 msgr2=0x7fc84015c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc844930700 1 --2- 172.21.15.45:0/2214478888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc840159590 0x7fc84015c150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc838014000 tx=0x7fc838004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 shutdown_connections 2023-08-27T11:12:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc83ed9d700 1 -- 172.21.15.45:0/2214478888 reap_dead start 2023-08-27T11:12:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 shutdown_connections 2023-08-27T11:12:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 >> 172.21.15.45:0/2214478888 conn(0x7fc8400cec30 msgr2=0x7fc8400c54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 shutdown_connections 2023-08-27T11:12:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7fc844930700 1 -- 172.21.15.45:0/2214478888 wait complete. 2023-08-27T11:12:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7f457bb86700 1 -- 172.21.15.45:0/3848996887 wait complete. 2023-08-27T11:12:37.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.989+0000 7f5ec4ff9700 1 -- 172.21.15.45:0/789138877 <== osd.0 v2:172.21.15.45:6800/1741844211 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5eb80013d0 con 0x7f5eb4073d40 2023-08-27T11:12:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.990+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 >> [v2:172.21.15.45:6800/1741844211,v1:172.21.15.45:6801/1741844211] conn(0x7f5eb4073d40 msgr2=0x7f5eb4076180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.990+0000 7f5ecea20700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.45:6800/1741844211,v1:172.21.15.45:6801/1741844211] conn(0x7f5eb4073d40 0x7f5eb4076180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.990+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5eb406a0e0 msgr2=0x7f5eb406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.990+0000 7f5ecea20700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5eb406a0e0 0x7f5eb406c5a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00beb0 tx=0x7f5ebc00bf90 comp rx=0 tx=0).stop 2023-08-27T11:12:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.990+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ec8159ee0 msgr2=0x7f5ec80c2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.990+0000 7f5ecea20700 1 --2- 172.21.15.45:0/789138877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ec8159ee0 0x7f5ec80c2bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000b9c0 tx=0x7f5ec000bd80 comp rx=0 tx=0).stop 2023-08-27T11:12:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.990+0000 7f5eccfbd700 1 -- 172.21.15.45:0/789138877 reap_dead start 2023-08-27T11:12:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.990+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 shutdown_connections 2023-08-27T11:12:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.990+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 >> 172.21.15.45:0/789138877 conn(0x7f5ec80b2420 msgr2=0x7f5ec8159310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.991+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 shutdown_connections 2023-08-27T11:12:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.994+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 --> [v2:172.21.15.135:6800/1785499536,v1:172.21.15.135:6801/1785499536] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e20005cb0 con 0x7f8e200015e0 2023-08-27T11:12:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.995+0000 7f8e35ffb700 1 -- 172.21.15.45:0/3007717805 <== osd.4 v2:172.21.15.135:6800/1785499536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8e20005cb0 con 0x7f8e200015e0 2023-08-27T11:12:37.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.996+0000 7f5ecea20700 1 -- 172.21.15.45:0/789138877 wait complete. 2023-08-27T11:12:37.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.996+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 --> [v2:172.21.15.135:6808/3357661340,v1:172.21.15.135:6809/3357661340] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec280b8d90 con 0x7fec281ecff0 2023-08-27T11:12:37.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.996+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6800/1785499536,v1:172.21.15.135:6801/1785499536] conn(0x7f8e200015e0 msgr2=0x7f8e20003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.996+0000 7f8e473fc700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6800/1785499536,v1:172.21.15.135:6801/1785499536] conn(0x7f8e200015e0 0x7f8e20003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.996+0000 7fec0f7fe700 1 -- 172.21.15.45:0/2798189622 <== osd.7 v2:172.21.15.135:6808/3357661340 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fec18004130 con 0x7fec281ecff0 2023-08-27T11:12:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6808/3357661340,v1:172.21.15.135:6809/3357661340] conn(0x7fec281ecff0 msgr2=0x7fec28163490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7fec2d234700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6808/3357661340,v1:172.21.15.135:6809/3357661340] conn(0x7fec281ecff0 0x7fec28163490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e2406a1b0 msgr2=0x7f8e2406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7f8e473fc700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e2406a1b0 0x7f8e2406c670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e4015a2c0 tx=0x7f8e30009110 comp rx=0 tx=0).stop 2023-08-27T11:12:37.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e4015a600 msgr2=0x7f8e400b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7f8e473fc700 1 --2- 172.21.15.45:0/3007717805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e4015a600 0x7f8e400b00d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e40074300 tx=0x7f8e3c004750 comp rx=0 tx=0).stop 2023-08-27T11:12:37.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7f8e45999700 1 -- 172.21.15.45:0/3007717805 reap_dead start 2023-08-27T11:12:37.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 shutdown_connections 2023-08-27T11:12:37.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 >> 172.21.15.45:0/3007717805 conn(0x7f8e400cc7e0 msgr2=0x7f8e4006ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 shutdown_connections 2023-08-27T11:12:37.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:36.997+0000 7f8e473fc700 1 -- 172.21.15.45:0/3007717805 wait complete. 2023-08-27T11:12:37.099 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:37.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-27T11:12:37.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.001+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec1406a0e0 msgr2=0x7fec1406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.001+0000 7fec2d234700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fec1406a0e0 0x7fec1406c5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec2000ab90 tx=0x7fec20009230 comp rx=0 tx=0).stop 2023-08-27T11:12:37.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.001+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec28165220 msgr2=0x7fec280d16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.001+0000 7fec2d234700 1 --2- 172.21.15.45:0/2798189622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec28165220 0x7fec280d16d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00e950 tx=0x7fec1c00ed10 comp rx=0 tx=0).stop 2023-08-27T11:12:37.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.001+0000 7fec2759e700 1 -- 172.21.15.45:0/2798189622 reap_dead start 2023-08-27T11:12:37.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.002+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 shutdown_connections 2023-08-27T11:12:37.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.002+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 >> 172.21.15.45:0/2798189622 conn(0x7fec280d3090 msgr2=0x7fec280d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.002+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 shutdown_connections 2023-08-27T11:12:37.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.002+0000 7fec2d234700 1 -- 172.21.15.45:0/2798189622 wait complete. 2023-08-27T11:12:37.103 INFO:teuthology.orchestra.run.smithi045.stdout:55834574858 2023-08-27T11:12:37.104 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-27T11:12:37.105 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:37.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-27T11:12:37.106 INFO:teuthology.orchestra.run.smithi045.stdout:55834574861 2023-08-27T11:12:37.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-27T11:12:37.107 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:37.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-27T11:12:37.108 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:37.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-27T11:12:37.110 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:37.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-27T11:12:37.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.345+0000 7fa676bfe700 1 Processor -- start 2023-08-27T11:12:37.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.345+0000 7fa676bfe700 1 -- start start 2023-08-27T11:12:37.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.346+0000 7fa676bfe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa67006de10 0x7fa6700cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.346+0000 7fa676bfe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67015db30 0x7fa6700cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.346+0000 7fa676bfe700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700cddb0 con 0x7fa67015ca10 2023-08-27T11:12:37.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.346+0000 7fa676bfe700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6700c9cd0 con 0x7fa67006de10 2023-08-27T11:12:37.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.346+0000 7fa676bfe700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700ccd90 con 0x7fa67015db30 2023-08-27T11:12:37.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.347+0000 7fa67519b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67015db30 0x7fa6700cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.347+0000 7fa67519b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67015db30 0x7fa6700cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:49996/0 (socket says 172.21.15.45:49996) 2023-08-27T11:12:37.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.347+0000 7fa67519b700 1 -- 172.21.15.45:0/3788303648 learned_addr learned my addr 172.21.15.45:0/3788303648 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.347+0000 7fa66f7fe700 1 -- 172.21.15.45:0/3788303648 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675018689 0 0) 0x7fa6700cddb0 con 0x7fa67015ca10 2023-08-27T11:12:37.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.347+0000 7fa66f7fe700 1 -- 172.21.15.45:0/3788303648 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6580035b0 con 0x7fa67015ca10 2023-08-27T11:12:37.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.348+0000 7fa67499a700 1 --2- 172.21.15.45:0/3788303648 >> v2:172.21.15.135:3300/0 conn(0x7fa67006de10 0x7fa6700cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.348+0000 7fa67519b700 1 -- 172.21.15.45:0/3788303648 >> v2:172.21.15.135:3300/0 conn(0x7fa67006de10 msgr2=0x7fa6700cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:37.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.348+0000 7fa67519b700 1 --2- 172.21.15.45:0/3788303648 >> v2:172.21.15.135:3300/0 conn(0x7fa67006de10 0x7fa6700cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.348+0000 7fa67519b700 1 -- 172.21.15.45:0/3788303648 >> v1:172.21.15.135:6789/0 conn(0x7fa67015ca10 legacy=0x7fa6700c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.348+0000 7fa67519b700 1 -- 172.21.15.45:0/3788303648 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700cd070 con 0x7fa67015db30 2023-08-27T11:12:37.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.348+0000 7fa67499a700 1 --2- 172.21.15.45:0/3788303648 >> v2:172.21.15.135:3300/0 conn(0x7fa67006de10 0x7fa6700cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:37.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.348+0000 7fa67519b700 1 --2- 172.21.15.45:0/3788303648 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67015db30 0x7fa6700cb570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa668009e50 tx=0x7fa66800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31ccc65e2eec0ef server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.349+0000 7fa66f7fe700 1 -- 172.21.15.45:0/3788303648 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800e070 con 0x7fa67015db30 2023-08-27T11:12:37.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.349+0000 7fa66f7fe700 1 -- 172.21.15.45:0/3788303648 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66800b910 con 0x7fa67015db30 2023-08-27T11:12:37.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.349+0000 7fa66f7fe700 1 -- 172.21.15.45:0/3788303648 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800bcc0 con 0x7fa67015db30 2023-08-27T11:12:37.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.349+0000 7fa676bfe700 1 -- 172.21.15.45:0/3788303648 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67015db30 msgr2=0x7fa6700cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.349+0000 7fa676bfe700 1 --2- 172.21.15.45:0/3788303648 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67015db30 0x7fa6700cb570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa668009e50 tx=0x7fa66800b330 comp rx=0 tx=0).stop 2023-08-27T11:12:37.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.354+0000 7fa676bfe700 1 -- 172.21.15.45:0/3788303648 shutdown_connections 2023-08-27T11:12:37.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.354+0000 7fa676bfe700 1 --2- 172.21.15.45:0/3788303648 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67015db30 0x7fa6700cb570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.354+0000 7fa676bfe700 1 --2- 172.21.15.45:0/3788303648 >> v2:172.21.15.135:3300/0 conn(0x7fa67006de10 0x7fa6700cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.354+0000 7fa676bfe700 1 -- 172.21.15.45:0/3788303648 >> 172.21.15.45:0/3788303648 conn(0x7fa6700b07c0 msgr2=0x7fa6700b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.355+0000 7fa676bfe700 1 -- 172.21.15.45:0/3788303648 shutdown_connections 2023-08-27T11:12:37.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.355+0000 7fa676bfe700 1 -- 172.21.15.45:0/3788303648 wait complete. 2023-08-27T11:12:37.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.357+0000 7fa676bfe700 1 Processor -- start 2023-08-27T11:12:37.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.357+0000 7fa676bfe700 1 -- start start 2023-08-27T11:12:37.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.359+0000 7fa676bfe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67006de10 0x7fa6700b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.359+0000 7fa676bfe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67015ca10 0x7fa6700be930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.359+0000 7fa676bfe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa67015db30 0x7fa6700bc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.359+0000 7fa676bfe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700b4390 con 0x7fa67015db30 2023-08-27T11:12:37.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.359+0000 7fa676bfe700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700b3710 con 0x7fa67015ca10 2023-08-27T11:12:37.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.359+0000 7fa676bfe700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700b39f0 con 0x7fa67006de10 2023-08-27T11:12:37.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.360+0000 7fa66ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67015ca10 0x7fa6700be930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.360+0000 7fa66ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67015ca10 0x7fa6700be930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:40948/0 (socket says 172.21.15.45:40948) 2023-08-27T11:12:37.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.360+0000 7fa66ffff700 1 -- 172.21.15.45:0/3215295105 learned_addr learned my addr 172.21.15.45:0/3215295105 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.360+0000 7fa67519b700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa67015db30 0x7fa6700bc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.360+0000 7fa66ffff700 1 -- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67006de10 msgr2=0x7fa6700b1840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:12:37.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.360+0000 7fa66ffff700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67006de10 0x7fa6700b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.360+0000 7fa66ffff700 1 -- 172.21.15.45:0/3215295105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa67015db30 msgr2=0x7fa6700bc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.360+0000 7fa66ffff700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa67015db30 0x7fa6700bc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.360+0000 7fa66ffff700 1 -- 172.21.15.45:0/3215295105 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668009b30 con 0x7fa67015ca10 2023-08-27T11:12:37.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.361+0000 7fa67519b700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa67015db30 0x7fa6700bc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:37.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.361+0000 7fa66ffff700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67015ca10 0x7fa6700be930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa66000c500 tx=0x7fa66000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.361+0000 7fa66d7fa700 1 -- 172.21.15.45:0/3215295105 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6600052d0 con 0x7fa67015ca10 2023-08-27T11:12:37.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.361+0000 7fa66d7fa700 1 -- 172.21.15.45:0/3215295105 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa660005430 con 0x7fa67015ca10 2023-08-27T11:12:37.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.361+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6700b3cd0 con 0x7fa67015ca10 2023-08-27T11:12:37.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.361+0000 7fa66d7fa700 1 -- 172.21.15.45:0/3215295105 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6600038c0 con 0x7fa67015ca10 2023-08-27T11:12:37.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.361+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6700b3410 con 0x7fa67015ca10 2023-08-27T11:12:37.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.362+0000 7ff3d002b700 1 Processor -- start 2023-08-27T11:12:37.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.362+0000 7f9b6265f700 1 Processor -- start 2023-08-27T11:12:37.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.362+0000 7ff3d002b700 1 -- start start 2023-08-27T11:12:37.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.362+0000 7f9b6265f700 1 -- start start 2023-08-27T11:12:37.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7f8e5b384700 1 Processor -- start 2023-08-27T11:12:37.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7ff3d002b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff3c80cbba0 0x7ff3c8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7ff3d002b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3c8071280 0x7ff3c806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7ff3d002b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c80cc9a0 con 0x7ff3c815a1f0 2023-08-27T11:12:37.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7ff3d002b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c8072080 con 0x7ff3c80cbba0 2023-08-27T11:12:37.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7ff3d002b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c8071ac0 con 0x7ff3c8071280 2023-08-27T11:12:37.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7ff3cd5c6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff3c80cbba0 0x7ff3c8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7ff3cddc7700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff3c815a1f0 0x7ff3c80cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38874/0 (socket says 172.21.15.45:38874) 2023-08-27T11:12:37.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7ff3cddc7700 1 -- 172.21.15.45:0/4092679876 learned_addr learned my addr 172.21.15.45:0/4092679876 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:37.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7f9b6265f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b62c0 0x7f9b5c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7f9b6265f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5c0cdf40 con 0x7f9b5c160890 2023-08-27T11:12:37.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7f9b6265f700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5c0d0fd0 con 0x7f9b5c0b5220 2023-08-27T11:12:37.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7f9b6265f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0ce190 con 0x7f9b5c0b62c0 2023-08-27T11:12:37.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7f9b5b7fe700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9b5c0b5220 0x7f9b5c0d1460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:34534/0 (socket says 172.21.15.45:34534) 2023-08-27T11:12:37.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7f9b5b7fe700 1 -- 172.21.15.45:0/140126893 learned_addr learned my addr 172.21.15.45:0/140126893 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:37.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7f9b60bfc700 1 --2- 172.21.15.45:0/140126893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b62c0 0x7f9b5c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7f25a536c700 1 Processor -- start 2023-08-27T11:12:37.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7f25a536c700 1 -- start start 2023-08-27T11:12:37.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7f25a536c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25a015c070 0x7f25a00cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7f25a536c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f25a006d8b0 0x7f25a006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7f25a536c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a015c450 con 0x7f25a00c9280 2023-08-27T11:12:37.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7f25a536c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a0162e70 con 0x7f25a015c070 2023-08-27T11:12:37.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7f25a536c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a0162260 con 0x7f25a006d8b0 2023-08-27T11:12:37.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7ff3cd5c6700 1 -- 172.21.15.45:0/4092679876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3c8071280 msgr2=0x7ff3c806bdc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:12:37.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7ff3cd5c6700 1 --2- 172.21.15.45:0/4092679876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3c8071280 0x7ff3c806bdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.364+0000 7ff3cd5c6700 1 -- 172.21.15.45:0/4092679876 >> v1:172.21.15.45:6789/0 conn(0x7ff3c815a1f0 legacy=0x7ff3c80cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7ff3cd5c6700 1 -- 172.21.15.45:0/4092679876 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c8071da0 con 0x7ff3c80cbba0 2023-08-27T11:12:37.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f259f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f25a00c9280 0x7f25a006c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38878/0 (socket says 172.21.15.45:38878) 2023-08-27T11:12:37.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f259f7fe700 1 -- 172.21.15.45:0/1164190065 learned_addr learned my addr 172.21.15.45:0/1164190065 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:37.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f21e5966700 1 Processor -- start 2023-08-27T11:12:37.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.363+0000 7f8e5b384700 1 -- start start 2023-08-27T11:12:37.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f8e5b384700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8e541619d0 0x7f8e540c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f8e5b384700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e540c9990 0x7f8e540cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f8e5b384700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5406be60 con 0x7f8e54166140 2023-08-27T11:12:37.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f8e5b384700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5406c140 con 0x7f8e540c9990 2023-08-27T11:12:37.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f8e5b384700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5406c420 con 0x7f8e541619d0 2023-08-27T11:12:37.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f8e59120700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8e541619d0 0x7f8e540c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f8e59120700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8e541619d0 0x7f8e540c1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:40968/0 (socket says 172.21.15.45:40968) 2023-08-27T11:12:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f8e59120700 1 -- 172.21.15.45:0/130776891 learned_addr learned my addr 172.21.15.45:0/130776891 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f8e59921700 1 --2- 172.21.15.45:0/130776891 >> v2:172.21.15.45:3300/0 conn(0x7f8e540c9990 0x7f8e540cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7ff3cd5c6700 1 --2- 172.21.15.45:0/4092679876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c80cbba0 0x7ff3c8070d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009870 tx=0x7ff3b8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9080065dc84650f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f259e7fc700 1 --2- 172.21.15.45:0/1164190065 >> v2:172.21.15.135:3300/0 conn(0x7f25a006d8b0 0x7f25a006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f259effd700 1 --2- 172.21.15.45:0/1164190065 >> v2:172.21.15.45:3300/0 conn(0x7f25a015c070 0x7f25a00cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f9b60bfc700 1 -- 172.21.15.45:0/140126893 >> v1:172.21.15.135:6789/0 conn(0x7f9b5c0b5220 legacy=0x7f9b5c0d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f9b60bfc700 1 -- 172.21.15.45:0/140126893 >> v1:172.21.15.45:6789/0 conn(0x7f9b5c160890 legacy=0x7f9b5c0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f9b60bfc700 1 -- 172.21.15.45:0/140126893 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0ce470 con 0x7f9b5c0b62c0 2023-08-27T11:12:37.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f8e59120700 1 -- 172.21.15.45:0/130776891 >> v2:172.21.15.45:3300/0 conn(0x7f8e540c9990 msgr2=0x7f8e540cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:37.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f8e59120700 1 --2- 172.21.15.45:0/130776891 >> v2:172.21.15.45:3300/0 conn(0x7f8e540c9990 0x7f8e540cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f8e59120700 1 -- 172.21.15.45:0/130776891 >> v1:172.21.15.135:6789/0 conn(0x7f8e54166140 legacy=0x7f8e540c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f8e59120700 1 -- 172.21.15.45:0/130776891 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5406c700 con 0x7f8e541619d0 2023-08-27T11:12:37.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f8e59921700 1 --2- 172.21.15.45:0/130776891 >> v2:172.21.15.45:3300/0 conn(0x7f8e540c9990 0x7f8e540cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:37.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f259effd700 1 -- 172.21.15.45:0/1164190065 >> v2:172.21.15.135:3300/0 conn(0x7f25a006d8b0 msgr2=0x7f25a006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:37.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f259effd700 1 --2- 172.21.15.45:0/1164190065 >> v2:172.21.15.135:3300/0 conn(0x7f25a006d8b0 0x7f25a006dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f259effd700 1 -- 172.21.15.45:0/1164190065 >> v1:172.21.15.45:6789/0 conn(0x7f25a00c9280 legacy=0x7f25a006c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f259effd700 1 -- 172.21.15.45:0/1164190065 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0162540 con 0x7f25a015c070 2023-08-27T11:12:37.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f9b60bfc700 1 --2- 172.21.15.45:0/140126893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b62c0 0x7f9b5c0cd8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009870 tx=0x7f9b50009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65e3ad82f5e80c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f8e59120700 1 --2- 172.21.15.45:0/130776891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e541619d0 0x7f8e540c1c40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e540c2730 tx=0x7f8e4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5944edae138e2076 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f259effd700 1 --2- 172.21.15.45:0/1164190065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25a015c070 0x7f25a00cca20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f25a00c26f0 tx=0x7f2590009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa816c2ca402555 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7ff3ccdc5700 1 -- 172.21.15.45:0/4092679876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8004230 con 0x7ff3c80cbba0 2023-08-27T11:12:37.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7ff3ccdc5700 1 -- 172.21.15.45:0/4092679876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b8004390 con 0x7ff3c80cbba0 2023-08-27T11:12:37.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7ff3ccdc5700 1 -- 172.21.15.45:0/4092679876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80036a0 con 0x7ff3c80cbba0 2023-08-27T11:12:37.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.365+0000 7f21e5966700 1 -- start start 2023-08-27T11:12:37.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f9b5affd700 1 -- 172.21.15.45:0/140126893 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50004230 con 0x7f9b5c0b62c0 2023-08-27T11:12:37.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f9b5affd700 1 -- 172.21.15.45:0/140126893 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b50004390 con 0x7f9b5c0b62c0 2023-08-27T11:12:37.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f9b5affd700 1 -- 172.21.15.45:0/140126893 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b500036a0 con 0x7f9b5c0b62c0 2023-08-27T11:12:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f8e4bfff700 1 -- 172.21.15.45:0/130776891 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00e070 con 0x7f8e541619d0 2023-08-27T11:12:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f8e4bfff700 1 -- 172.21.15.45:0/130776891 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c003a20 con 0x7f8e541619d0 2023-08-27T11:12:37.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f8e4bfff700 1 -- 172.21.15.45:0/130776891 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c003dd0 con 0x7f8e541619d0 2023-08-27T11:12:37.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f21e5966700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f21e00cf720 0x7f21e0159590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f21e5966700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00c9490 0x7f21e006c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f21e5966700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e00c9910 con 0x7f21e00cd720 2023-08-27T11:12:37.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f21e5966700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00ca130 con 0x7f21e00cf720 2023-08-27T11:12:37.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.366+0000 7f21e5966700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e006cda0 con 0x7f21e00c9490 2023-08-27T11:12:37.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f21de7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00c9490 0x7f21e006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f21de7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00c9490 0x7f21e006c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50026/0 (socket says 172.21.15.45:50026) 2023-08-27T11:12:37.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f21de7fc700 1 -- 172.21.15.45:0/3099885666 learned_addr learned my addr 172.21.15.45:0/3099885666 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f259dffb700 1 -- 172.21.15.45:0/1164190065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590004230 con 0x7f25a015c070 2023-08-27T11:12:37.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f259dffb700 1 -- 172.21.15.45:0/1164190065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590004390 con 0x7f25a015c070 2023-08-27T11:12:37.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f259dffb700 1 -- 172.21.15.45:0/1164190065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900036a0 con 0x7f25a015c070 2023-08-27T11:12:37.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67004e620 con 0x7fa67015ca10 2023-08-27T11:12:37.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7ff3d002b700 1 -- 172.21.15.45:0/4092679876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c80cbba0 msgr2=0x7ff3c8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7ff3d002b700 1 --2- 172.21.15.45:0/4092679876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c80cbba0 0x7ff3c8070d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009870 tx=0x7ff3b8009b80 comp rx=0 tx=0).stop 2023-08-27T11:12:37.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7ff3d002b700 1 -- 172.21.15.45:0/4092679876 shutdown_connections 2023-08-27T11:12:37.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7ff3d002b700 1 --2- 172.21.15.45:0/4092679876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3c8071280 0x7ff3c806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7ff3d002b700 1 --2- 172.21.15.45:0/4092679876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c80cbba0 0x7ff3c8070d40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7ff3d002b700 1 -- 172.21.15.45:0/4092679876 >> 172.21.15.45:0/4092679876 conn(0x7ff3c80b07c0 msgr2=0x7ff3c80b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f21de7fc700 1 -- 172.21.15.45:0/3099885666 >> v2:172.21.15.135:3300/0 conn(0x7f21e00cf720 msgr2=0x7f21e0159590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:37.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f21de7fc700 1 --2- 172.21.15.45:0/3099885666 >> v2:172.21.15.135:3300/0 conn(0x7f21e00cf720 0x7f21e0159590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f21de7fc700 1 -- 172.21.15.45:0/3099885666 >> v1:172.21.15.45:6789/0 conn(0x7f21e00cd720 legacy=0x7f21e006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:37.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f21de7fc700 1 -- 172.21.15.45:0/3099885666 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e00c9b70 con 0x7f21e00c9490 2023-08-27T11:12:37.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f9b6265f700 1 -- 172.21.15.45:0/140126893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b62c0 msgr2=0x7f9b5c0cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f9b6265f700 1 --2- 172.21.15.45:0/140126893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b62c0 0x7f9b5c0cd8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009870 tx=0x7f9b50009b80 comp rx=0 tx=0).stop 2023-08-27T11:12:37.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f8e5b384700 1 -- 172.21.15.45:0/130776891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e541619d0 msgr2=0x7f8e540c1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7f8e5b384700 1 --2- 172.21.15.45:0/130776891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e541619d0 0x7f8e540c1c40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e540c2730 tx=0x7f8e4c00b330 comp rx=0 tx=0).stop 2023-08-27T11:12:37.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7ff3d002b700 1 -- 172.21.15.45:0/4092679876 shutdown_connections 2023-08-27T11:12:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f8e5b384700 1 -- 172.21.15.45:0/130776891 shutdown_connections 2023-08-27T11:12:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f8e5b384700 1 --2- 172.21.15.45:0/130776891 >> v2:172.21.15.45:3300/0 conn(0x7f8e540c9990 0x7f8e540cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f8e5b384700 1 --2- 172.21.15.45:0/130776891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e541619d0 0x7f8e540c1c40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f8e5b384700 1 -- 172.21.15.45:0/130776891 >> 172.21.15.45:0/130776891 conn(0x7f8e540b07c0 msgr2=0x7f8e540b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7ff3d002b700 1 -- 172.21.15.45:0/4092679876 wait complete. 2023-08-27T11:12:37.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f25a536c700 1 -- 172.21.15.45:0/1164190065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25a015c070 msgr2=0x7f25a00cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f25a536c700 1 --2- 172.21.15.45:0/1164190065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25a015c070 0x7f25a00cca20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f25a00c26f0 tx=0x7f2590009b20 comp rx=0 tx=0).stop 2023-08-27T11:12:37.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.367+0000 7fa66d7fa700 1 -- 172.21.15.45:0/3215295105 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa660003a20 con 0x7fa67015ca10 2023-08-27T11:12:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f21de7fc700 1 --2- 172.21.15.45:0/3099885666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00c9490 0x7f21e006c860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009e50 tx=0x7f21d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bafe4ecd2fd8bce7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f25a536c700 1 -- 172.21.15.45:0/1164190065 shutdown_connections 2023-08-27T11:12:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f25a536c700 1 --2- 172.21.15.45:0/1164190065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25a015c070 0x7f25a00cca20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f25a536c700 1 --2- 172.21.15.45:0/1164190065 >> v2:172.21.15.135:3300/0 conn(0x7f25a006d8b0 0x7f25a006dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f8e5b384700 1 -- 172.21.15.45:0/130776891 shutdown_connections 2023-08-27T11:12:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f25a536c700 1 -- 172.21.15.45:0/1164190065 >> 172.21.15.45:0/1164190065 conn(0x7f25a00cec30 msgr2=0x7f25a00cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f25a536c700 1 -- 172.21.15.45:0/1164190065 shutdown_connections 2023-08-27T11:12:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f8e5b384700 1 -- 172.21.15.45:0/130776891 wait complete. 2023-08-27T11:12:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f21ddffb700 1 -- 172.21.15.45:0/3099885666 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d800e070 con 0x7f21e00c9490 2023-08-27T11:12:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f21ddffb700 1 -- 172.21.15.45:0/3099885666 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8003a20 con 0x7f21e00c9490 2023-08-27T11:12:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f21ddffb700 1 -- 172.21.15.45:0/3099885666 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8003dd0 con 0x7f21e00c9490 2023-08-27T11:12:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f21e5966700 1 -- 172.21.15.45:0/3099885666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00c9490 msgr2=0x7f21e006c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f21e5966700 1 --2- 172.21.15.45:0/3099885666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00c9490 0x7f21e006c860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009e50 tx=0x7f21d800b330 comp rx=0 tx=0).stop 2023-08-27T11:12:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f9b6265f700 1 -- 172.21.15.45:0/140126893 shutdown_connections 2023-08-27T11:12:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f9b6265f700 1 --2- 172.21.15.45:0/140126893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b62c0 0x7f9b5c0cd8f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f9b6265f700 1 -- 172.21.15.45:0/140126893 >> 172.21.15.45:0/140126893 conn(0x7f9b5c0cfb10 msgr2=0x7f9b5c0cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f9b6265f700 1 -- 172.21.15.45:0/140126893 shutdown_connections 2023-08-27T11:12:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7fa66d7fa700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa65c06a1b0 0x7fa65c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7fa66d7fa700 1 -- 172.21.15.45:0/3215295105 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa660087dd0 con 0x7fa67015ca10 2023-08-27T11:12:37.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.368+0000 7f25a536c700 1 -- 172.21.15.45:0/1164190065 wait complete. 2023-08-27T11:12:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f9b6265f700 1 -- 172.21.15.45:0/140126893 wait complete. 2023-08-27T11:12:37.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f21e5966700 1 -- 172.21.15.45:0/3099885666 shutdown_connections 2023-08-27T11:12:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f21e5966700 1 --2- 172.21.15.45:0/3099885666 >> v2:172.21.15.135:3300/0 conn(0x7f21e00cf720 0x7f21e0159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f21e5966700 1 --2- 172.21.15.45:0/3099885666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00c9490 0x7f21e006c860 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f21e5966700 1 -- 172.21.15.45:0/3099885666 >> 172.21.15.45:0/3099885666 conn(0x7f21e00cec30 msgr2=0x7f21e00cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.370+0000 7f21e5966700 1 -- 172.21.15.45:0/3099885666 shutdown_connections 2023-08-27T11:12:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.370+0000 7ff3d002b700 1 Processor -- start 2023-08-27T11:12:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.370+0000 7f8e5b384700 1 Processor -- start 2023-08-27T11:12:37.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.370+0000 7f25a536c700 1 Processor -- start 2023-08-27T11:12:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.370+0000 7ff3d002b700 1 -- start start 2023-08-27T11:12:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7ff3d002b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3c8071280 0x7ff3c814c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7ff3d002b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c80cbba0 0x7ff3c80c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7ff3d002b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c815a1f0 0x7ff3c80c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7ff3d002b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c814efd0 con 0x7ff3c815a1f0 2023-08-27T11:12:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7ff3d002b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c814cdc0 con 0x7ff3c80cbba0 2023-08-27T11:12:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7ff3d002b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80c2e50 con 0x7ff3c8071280 2023-08-27T11:12:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.370+0000 7f8e5b384700 1 -- start start 2023-08-27T11:12:37.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f9b6265f700 1 Processor -- start 2023-08-27T11:12:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f8e5b384700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e540c9990 0x7f8e54150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f8e5b384700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e541619d0 0x7f8e540c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f8e5b384700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e540c1220 0x7f8e5414f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f8e5b384700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e540c5370 con 0x7f8e540c9990 2023-08-27T11:12:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f8e5b384700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5414ec30 con 0x7f8e541619d0 2023-08-27T11:12:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f8e5b384700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e540c4db0 con 0x7f8e540c1220 2023-08-27T11:12:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f8e59921700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e540c1220 0x7f8e5414f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.370+0000 7f21e5966700 1 -- 172.21.15.45:0/3099885666 wait complete. 2023-08-27T11:12:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f21e5966700 1 Processor -- start 2023-08-27T11:12:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f8e59921700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e540c1220 0x7f8e5414f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50038/0 (socket says 172.21.15.45:50038) 2023-08-27T11:12:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f8e59921700 1 -- 172.21.15.45:0/2895901236 learned_addr learned my addr 172.21.15.45:0/2895901236 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f8e5891f700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e541619d0 0x7f8e540c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f8e59120700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e540c9990 0x7f8e54150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.370+0000 7f25a536c700 1 -- start start 2023-08-27T11:12:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f25a536c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25a006d8b0 0x7f25a00b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f25a536c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25a00c9280 0x7f25a01f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f25a536c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25a015c070 0x7f25a01f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f25a536c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00b4540 con 0x7f25a015c070 2023-08-27T11:12:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f25a536c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00c88f0 con 0x7f25a00c9280 2023-08-27T11:12:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f25a536c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00c8bd0 con 0x7f25a006d8b0 2023-08-27T11:12:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7ff3ce5c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c815a1f0 0x7ff3c80c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7ff3cd5c6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c80cbba0 0x7ff3c80c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7ff3ce5c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c815a1f0 0x7ff3c80c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52126/0 (socket says 172.21.15.45:52126) 2023-08-27T11:12:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7ff3ce5c8700 1 -- 172.21.15.45:0/122130365 learned_addr learned my addr 172.21.15.45:0/122130365 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f8e59921700 1 -- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e541619d0 msgr2=0x7f8e540c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f8e59921700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e541619d0 0x7f8e540c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f8e59921700 1 -- 172.21.15.45:0/2895901236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e540c9990 msgr2=0x7f8e54150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f8e59921700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e540c9990 0x7f8e54150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f8e59921700 1 -- 172.21.15.45:0/2895901236 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e500094a0 con 0x7f8e540c1220 2023-08-27T11:12:37.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f259e7fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25a00c9280 0x7f25a01f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f259e7fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25a00c9280 0x7f25a01f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:40992/0 (socket says 172.21.15.45:40992) 2023-08-27T11:12:37.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f259e7fc700 1 -- 172.21.15.45:0/1774899206 learned_addr learned my addr 172.21.15.45:0/1774899206 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f259effd700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25a006d8b0 0x7f25a00b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7ff3cd5c6700 1 -- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3c8071280 msgr2=0x7ff3c814c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7ff3cd5c6700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3c8071280 0x7ff3c814c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7ff3cd5c6700 1 -- 172.21.15.45:0/122130365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c815a1f0 msgr2=0x7ff3c80c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7ff3cd5c6700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c815a1f0 0x7ff3c80c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7ff3cd5c6700 1 -- 172.21.15.45:0/122130365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b80094a0 con 0x7ff3c80cbba0 2023-08-27T11:12:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f259e7fc700 1 -- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25a006d8b0 msgr2=0x7f25a00b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f259e7fc700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25a006d8b0 0x7f25a00b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f259e7fc700 1 -- 172.21.15.45:0/1774899206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25a015c070 msgr2=0x7f25a01f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f259e7fc700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25a015c070 0x7f25a01f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f259e7fc700 1 -- 172.21.15.45:0/1774899206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25900094a0 con 0x7f25a00c9280 2023-08-27T11:12:37.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f8e59921700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e540c1220 0x7f8e5414f680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000eb90 tx=0x7f8e5000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f259effd700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25a006d8b0 0x7f25a00b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:37.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f259e7fc700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25a00c9280 0x7f25a01f7430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f259400e9b0 tx=0x7f259400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.372+0000 7f21e5966700 1 -- start start 2023-08-27T11:12:37.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21e5966700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e00c9490 0x7f21e015c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21e5966700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00cd720 0x7f21e00c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21e5966700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e00cf720 0x7f21e00b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21e5966700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e02011d0 con 0x7f21e00c9490 2023-08-27T11:12:37.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21e5966700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c8c30 con 0x7f21e00cf720 2023-08-27T11:12:37.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21e5966700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c8950 con 0x7f21e00cd720 2023-08-27T11:12:37.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e00c9490 0x7f21e015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e00c9490 0x7f21e015c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52146/0 (socket says 172.21.15.45:52146) 2023-08-27T11:12:37.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21deffd700 1 -- 172.21.15.45:0/939166226 learned_addr learned my addr 172.21.15.45:0/939166226 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f9b6265f700 1 -- start start 2023-08-27T11:12:37.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f8e49ffb700 1 -- 172.21.15.45:0/2895901236 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e500097a0 con 0x7f8e540c1220 2023-08-27T11:12:37.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21de7fc700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00cd720 0x7f21e00c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c009b30 con 0x7f8e540c1220 2023-08-27T11:12:37.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f8e49ffb700 1 -- 172.21.15.45:0/2895901236 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e50004020 con 0x7f8e540c1220 2023-08-27T11:12:37.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f8e49ffb700 1 -- 172.21.15.45:0/2895901236 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e500043b0 con 0x7f8e540c1220 2023-08-27T11:12:37.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5414ee80 con 0x7f8e540c1220 2023-08-27T11:12:37.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f21df7fe700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e00cf720 0x7f21e00b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7fa67499a700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa65c06a1b0 0x7fa65c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7fa66d7fa700 1 -- 172.21.15.45:0/3215295105 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa66008b030 con 0x7fa67015ca10 2023-08-27T11:12:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f25877fe700 1 -- 172.21.15.45:0/1774899206 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594009710 con 0x7f25a00c9280 2023-08-27T11:12:37.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f25877fe700 1 -- 172.21.15.45:0/1774899206 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259400c850 con 0x7f25a00c9280 2023-08-27T11:12:37.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f25877fe700 1 -- 172.21.15.45:0/1774899206 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400cbc0 con 0x7f25a00c9280 2023-08-27T11:12:37.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.369+0000 7f5ae43fa700 1 Processor -- start 2023-08-27T11:12:37.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.371+0000 7f5ae43fa700 1 -- start start 2023-08-27T11:12:37.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f5ae43fa700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5adc08a9a0 0x7f5adc0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f5ae43fa700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc06e170 0x7f5adc0c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f5ae43fa700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5adc0c9bf0 con 0x7f5adc0c97c0 2023-08-27T11:12:37.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f5ae43fa700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc08ad80 con 0x7f5adc08a9a0 2023-08-27T11:12:37.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f5ae43fa700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0ccff0 con 0x7f5adc06e170 2023-08-27T11:12:37.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f5ae2196700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5adc08a9a0 0x7f5adc0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f5ae2196700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5adc08a9a0 0x7f5adc0cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52142/0 (socket says 172.21.15.45:52142) 2023-08-27T11:12:37.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f5ae2196700 1 -- 172.21.15.45:0/4255906647 learned_addr learned my addr 172.21.15.45:0/4255906647 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.373+0000 7f5ae1995700 1 --2- 172.21.15.45:0/4255906647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc06e170 0x7f5adc0c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f5ae1194700 1 -- 172.21.15.45:0/4255906647 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764451740 0 0) 0x7f5adc0c9bf0 con 0x7f5adc0c97c0 2023-08-27T11:12:37.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f5ae1194700 1 -- 172.21.15.45:0/4255906647 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ac80035b0 con 0x7f5adc0c97c0 2023-08-27T11:12:37.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5404e620 con 0x7f8e540c1220 2023-08-27T11:12:37.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7ff3cd5c6700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c80cbba0 0x7ff3c80c2570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8015000 tx=0x7ff3b8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7ff3be7fc700 1 -- 172.21.15.45:0/122130365 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8011830 con 0x7ff3c80cbba0 2023-08-27T11:12:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7ff3be7fc700 1 -- 172.21.15.45:0/122130365 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b801e070 con 0x7ff3c80cbba0 2023-08-27T11:12:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7ff3be7fc700 1 -- 172.21.15.45:0/122130365 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8011bf0 con 0x7ff3c80cbba0 2023-08-27T11:12:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c80c3130 con 0x7ff3c80cbba0 2023-08-27T11:12:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.375+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c80c36f0 con 0x7ff3c80cbba0 2023-08-27T11:12:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f9b6265f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b5220 0x7f9b5c1f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f9b6265f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c0b62c0 0x7f9b5c1fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f9b6265f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b5c160890 0x7f9b5c200510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f9b6265f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c1f6fa0 con 0x7f9b5c0b62c0 2023-08-27T11:12:37.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f9b6265f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c200b00 con 0x7f9b5c160890 2023-08-27T11:12:37.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f9b6265f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c200de0 con 0x7f9b5c0b5220 2023-08-27T11:12:37.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f9b60bfc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b5c160890 0x7f9b5c200510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f9b60bfc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b5c160890 0x7f9b5c200510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41012/0 (socket says 172.21.15.45:41012) 2023-08-27T11:12:37.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f9b60bfc700 1 -- 172.21.15.45:0/1288107970 learned_addr learned my addr 172.21.15.45:0/1288107970 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7fa67499a700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa65c06a1b0 0x7fa65c06c670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa664006fd0 tx=0x7fa664008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a00c8eb0 con 0x7f25a00c9280 2023-08-27T11:12:37.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.374+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a00c3af0 con 0x7f25a00c9280 2023-08-27T11:12:37.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.375+0000 7f5ae1995700 1 -- 172.21.15.45:0/4255906647 >> v2:172.21.15.45:3300/0 conn(0x7f5adc08a9a0 msgr2=0x7f5adc0cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:37.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.375+0000 7f5ae1995700 1 --2- 172.21.15.45:0/4255906647 >> v2:172.21.15.45:3300/0 conn(0x7f5adc08a9a0 0x7f5adc0cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.375+0000 7f5ae1995700 1 -- 172.21.15.45:0/4255906647 >> v1:172.21.15.135:6789/0 conn(0x7f5adc0c97c0 legacy=0x7f5adc169780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.375+0000 7f5ae1995700 1 -- 172.21.15.45:0/4255906647 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc0cd2d0 con 0x7f5adc06e170 2023-08-27T11:12:37.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c815ecc0 con 0x7ff3c80cbba0 2023-08-27T11:12:37.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f9b5bfff700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b5220 0x7f9b5c1f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f8e49ffb700 1 -- 172.21.15.45:0/2895901236 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8e5000c850 con 0x7f8e540c1220 2023-08-27T11:12:37.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f9b60bfc700 1 -- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b5220 msgr2=0x7f9b5c1f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f9b60bfc700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b5220 0x7f9b5c1f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f9b60bfc700 1 -- 172.21.15.45:0/1288107970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c0b62c0 msgr2=0x7f9b5c1fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f9b60bfc700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c0b62c0 0x7f9b5c1fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f9b60bfc700 1 -- 172.21.15.45:0/1288107970 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b500094a0 con 0x7f9b5c160890 2023-08-27T11:12:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f8e49ffb700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e3806a130 0x7f8e3806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.377+0000 7f8e49ffb700 1 -- 172.21.15.45:0/2895901236 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8e500883e0 con 0x7f8e540c1220 2023-08-27T11:12:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.377+0000 7f8e59120700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e3806a130 0x7f8e3806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.377+0000 7ff3be7fc700 1 -- 172.21.15.45:0/122130365 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff3b8011d50 con 0x7ff3c80cbba0 2023-08-27T11:12:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.377+0000 7f8e59120700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e3806a130 0x7f8e3806c5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000f80 tx=0x7f8e4c009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.377+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a0072070 con 0x7f25a00c9280 2023-08-27T11:12:37.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.377+0000 7f25877fe700 1 -- 172.21.15.45:0/1774899206 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f259400cd20 con 0x7f25a00c9280 2023-08-27T11:12:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.378+0000 7f5ae1995700 1 --2- 172.21.15.45:0/4255906647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc06e170 0x7f5adc0c9280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400cc50 tx=0x7f5ad400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0de84d2fe4d209b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f21de7fc700 1 -- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e00cf720 msgr2=0x7f21e00b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f21de7fc700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e00cf720 0x7f21e00b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f21de7fc700 1 -- 172.21.15.45:0/939166226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e00c9490 msgr2=0x7f21e015c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f21de7fc700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e00c9490 0x7f21e015c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.378+0000 7f9b60bfc700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b5c160890 0x7f9b5c200510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b50004000 tx=0x7f9b50004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.376+0000 7f21de7fc700 1 -- 172.21.15.45:0/939166226 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8009b30 con 0x7f21e00cd720 2023-08-27T11:12:37.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.377+0000 7f21de7fc700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00cd720 0x7f21e00c4740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f21e015d000 tx=0x7f21d8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.378+0000 7ff3be7fc700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff3ac06a130 0x7ff3ac06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.378+0000 7ff3be7fc700 1 -- 172.21.15.45:0/122130365 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff3b8089870 con 0x7ff3c80cbba0 2023-08-27T11:12:37.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.378+0000 7ff3cddc7700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff3ac06a130 0x7ff3ac06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.378+0000 7f25877fe700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f258c06a130 0x7f258c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.378+0000 7f25877fe700 1 -- 172.21.15.45:0/1774899206 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2594088890 con 0x7f25a00c9280 2023-08-27T11:12:37.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.378+0000 7f259effd700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f258c06a130 0x7f258c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.378+0000 7ff3cddc7700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff3ac06a130 0x7ff3ac06c5f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000f60 tx=0x7ff3c4007a70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.379+0000 7f259effd700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f258c06a130 0x7f258c06c5f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2590000c00 tx=0x7f2590008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.382+0000 7f21c7fff700 1 -- 172.21.15.45:0/939166226 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8004440 con 0x7f21e00cd720 2023-08-27T11:12:37.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.382+0000 7f5ae1194700 1 -- 172.21.15.45:0/4255906647 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4011070 con 0x7f5adc06e170 2023-08-27T11:12:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.382+0000 7f21c7fff700 1 -- 172.21.15.45:0/939166226 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d80045a0 con 0x7f21e00cd720 2023-08-27T11:12:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.382+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e0200b10 con 0x7f21e00cd720 2023-08-27T11:12:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.382+0000 7f5ae1194700 1 -- 172.21.15.45:0/4255906647 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad400c2e0 con 0x7f5adc06e170 2023-08-27T11:12:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.382+0000 7f5ae1194700 1 -- 172.21.15.45:0/4255906647 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4007cb0 con 0x7f5adc06e170 2023-08-27T11:12:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.382+0000 7f8e49ffb700 1 -- 172.21.15.45:0/2895901236 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8e500538c0 con 0x7f8e540c1220 2023-08-27T11:12:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.382+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e00c4f90 con 0x7f21e00cd720 2023-08-27T11:12:37.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.383+0000 7ff3be7fc700 1 -- 172.21.15.45:0/122130365 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff3b8054de0 con 0x7ff3c80cbba0 2023-08-27T11:12:37.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.383+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4255906647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc06e170 msgr2=0x7f5adc0c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.383+0000 7f5ae43fa700 1 --2- 172.21.15.45:0/4255906647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc06e170 0x7f5adc0c9280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400cc50 tx=0x7f5ad400cf60 comp rx=0 tx=0).stop 2023-08-27T11:12:37.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.383+0000 7f21c7fff700 1 -- 172.21.15.45:0/939166226 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8004950 con 0x7f21e00cd720 2023-08-27T11:12:37.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.383+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4255906647 shutdown_connections 2023-08-27T11:12:37.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.383+0000 7f5ae43fa700 1 --2- 172.21.15.45:0/4255906647 >> v2:172.21.15.45:3300/0 conn(0x7f5adc08a9a0 0x7f5adc0cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.383+0000 7f5ae43fa700 1 --2- 172.21.15.45:0/4255906647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc06e170 0x7f5adc0c9280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.383+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4255906647 >> 172.21.15.45:0/4255906647 conn(0x7f5adc0cec30 msgr2=0x7f5adc0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.383+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4255906647 shutdown_connections 2023-08-27T11:12:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.384+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4255906647 wait complete. 2023-08-27T11:12:37.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.384+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c00052f0 con 0x7f21e00cd720 2023-08-27T11:12:37.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.384+0000 7f21c7fff700 1 -- 172.21.15.45:0/939166226 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f21d8012430 con 0x7f21e00cd720 2023-08-27T11:12:37.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.385+0000 7f21c7fff700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f21cc06a0e0 0x7f21cc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.385+0000 7f21c7fff700 1 -- 172.21.15.45:0/939166226 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f21d8089ee0 con 0x7f21e00cd720 2023-08-27T11:12:37.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.385+0000 7f21deffd700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f21cc06a0e0 0x7f21cc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.385+0000 7f5ae43fa700 1 Processor -- start 2023-08-27T11:12:37.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.385+0000 7f21deffd700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f21cc06a0e0 0x7f21cc06c5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009930 tx=0x7f21d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.385+0000 7f5ae43fa700 1 -- start start 2023-08-27T11:12:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae43fa700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5adc06e170 0x7f5adc0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae43fa700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc08a9a0 0x7f5adc16d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae43fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5adc0c97c0 0x7f5adc0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae43fa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0b2000 con 0x7f5adc0c97c0 2023-08-27T11:12:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae43fa700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0b22e0 con 0x7f5adc06e170 2023-08-27T11:12:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae43fa700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0b25c0 con 0x7f5adc08a9a0 2023-08-27T11:12:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae2196700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5adc06e170 0x7f5adc0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae2997700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5adc0c97c0 0x7f5adc0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae2196700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5adc06e170 0x7f5adc0b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41024/0 (socket says 172.21.15.45:41024) 2023-08-27T11:12:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae2196700 1 -- 172.21.15.45:0/4137685603 learned_addr learned my addr 172.21.15.45:0/4137685603 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.386+0000 7f5ae1995700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc08a9a0 0x7f5adc16d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f5ae2196700 1 -- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc08a9a0 msgr2=0x7f5adc16d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f5ae2196700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc08a9a0 0x7f5adc16d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f5ae2196700 1 -- 172.21.15.45:0/4137685603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5adc0c97c0 msgr2=0x7f5adc0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f5ae2196700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5adc0c97c0 0x7f5adc0b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f5ae2196700 1 -- 172.21.15.45:0/4137685603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad400c930 con 0x7f5adc06e170 2023-08-27T11:12:37.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f5ae2196700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5adc06e170 0x7f5adc0b30a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8006b20 tx=0x7f5ad800da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f9b58ff9700 1 -- 172.21.15.45:0/1288107970 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50010590 con 0x7f9b5c160890 2023-08-27T11:12:37.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f9b58ff9700 1 -- 172.21.15.45:0/1288107970 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b50010710 con 0x7f9b5c160890 2023-08-27T11:12:37.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f9b58ff9700 1 -- 172.21.15.45:0/1288107970 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50010ac0 con 0x7f9b5c160890 2023-08-27T11:12:37.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c06fa80 con 0x7f9b5c160890 2023-08-27T11:12:37.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c06f770 con 0x7f9b5c160890 2023-08-27T11:12:37.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f5ad2ffd700 1 -- 172.21.15.45:0/4137685603 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800f040 con 0x7f5adc06e170 2023-08-27T11:12:37.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.388+0000 7f5ad2ffd700 1 -- 172.21.15.45:0/4137685603 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad8012770 con 0x7f5adc06e170 2023-08-27T11:12:37.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.388+0000 7f5ad2ffd700 1 -- 172.21.15.45:0/4137685603 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8012ae0 con 0x7f5adc06e170 2023-08-27T11:12:37.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.388+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc0b28a0 con 0x7f5adc06e170 2023-08-27T11:12:37.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.388+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc16d5e0 con 0x7f5adc06e170 2023-08-27T11:12:37.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.389+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5adc0b35e0 con 0x7f5adc06e170 2023-08-27T11:12:37.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.390+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c06ffa0 con 0x7f9b5c160890 2023-08-27T11:12:37.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.390+0000 7f9b58ff9700 1 -- 172.21.15.45:0/1288107970 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9b50010c20 con 0x7f9b5c160890 2023-08-27T11:12:37.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.390+0000 7f5ad2ffd700 1 -- 172.21.15.45:0/4137685603 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5ad8012c40 con 0x7f5adc06e170 2023-08-27T11:12:37.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.390+0000 7f5ad2ffd700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5ac006a1b0 0x7f5ac006c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.390+0000 7f5ad2ffd700 1 -- 172.21.15.45:0/4137685603 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5ad8088590 con 0x7f5adc06e170 2023-08-27T11:12:37.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.391+0000 7f5ae1995700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5ac006a1b0 0x7f5ac006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.391+0000 7f9b58ff9700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9b4806a0e0 0x7f9b4806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.391+0000 7f9b58ff9700 1 -- 172.21.15.45:0/1288107970 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9b5008a100 con 0x7f9b5c160890 2023-08-27T11:12:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.391+0000 7f5ae1995700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5ac006a1b0 0x7f5ac006c670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400cfa0 tx=0x7f5ad4012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.392+0000 7f21c7fff700 1 -- 172.21.15.45:0/939166226 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f21d8058e40 con 0x7f21e00cd720 2023-08-27T11:12:37.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.387+0000 7f25877fe700 1 -- 172.21.15.45:0/1774899206 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2594057800 con 0x7f25a00c9280 2023-08-27T11:12:37.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.393+0000 7f9b5bfff700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9b4806a0e0 0x7f9b4806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.393+0000 7f9b5bfff700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9b4806a0e0 0x7f9b4806c5a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0d1830 tx=0x7f9b54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.395+0000 7f5ad2ffd700 1 -- 172.21.15.45:0/4137685603 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5ad8053a00 con 0x7f5adc06e170 2023-08-27T11:12:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.399+0000 7fbbe3dee700 1 Processor -- start 2023-08-27T11:12:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.399+0000 7fbbe3dee700 1 -- start start 2023-08-27T11:12:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe3dee700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbbdc070f10 0x7fbbdc0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe3dee700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbbdc0ce420 0x7fbbdc15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe3dee700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbdc0ce8b0 con 0x7fbbdc0712f0 2023-08-27T11:12:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe3dee700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc0cb170 con 0x7fbbdc070f10 2023-08-27T11:12:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe3dee700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc0cab80 con 0x7fbbdc0ce420 2023-08-27T11:12:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe238b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbbdc0ce420 0x7fbbdc15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe238b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbbdc0ce420 0x7fbbdc15ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41036/0 (socket says 172.21.15.45:41036) 2023-08-27T11:12:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe238b700 1 -- 172.21.15.45:0/892839891 learned_addr learned my addr 172.21.15.45:0/892839891 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe238b700 1 -- 172.21.15.45:0/892839891 >> v2:172.21.15.45:3300/0 conn(0x7fbbdc070f10 msgr2=0x7fbbdc0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe238b700 1 --2- 172.21.15.45:0/892839891 >> v2:172.21.15.45:3300/0 conn(0x7fbbdc070f10 0x7fbbdc0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe238b700 1 -- 172.21.15.45:0/892839891 >> v1:172.21.15.45:6789/0 conn(0x7fbbdc0712f0 legacy=0x7fbbdc0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.400+0000 7fbbe238b700 1 -- 172.21.15.45:0/892839891 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc15de10 con 0x7fbbdc0ce420 2023-08-27T11:12:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.401+0000 7fbbe238b700 1 --2- 172.21.15.45:0/892839891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 0x7fbbdc15ad80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc15eb20 tx=0x7fbbd400d430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e4f2c6106993a40 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.406+0000 7f9b58ff9700 1 -- 172.21.15.45:0/1288107970 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9b5c06ffa0 con 0x7f9b5c160890 2023-08-27T11:12:37.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.406+0000 7fbbe0b88700 1 -- 172.21.15.45:0/892839891 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4010070 con 0x7fbbdc0ce420 2023-08-27T11:12:37.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.406+0000 7fbbe0b88700 1 -- 172.21.15.45:0/892839891 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd400dd50 con 0x7fbbdc0ce420 2023-08-27T11:12:37.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.406+0000 7fbbe0b88700 1 -- 172.21.15.45:0/892839891 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400b310 con 0x7fbbdc0ce420 2023-08-27T11:12:37.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.407+0000 7fbbe3dee700 1 -- 172.21.15.45:0/892839891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 msgr2=0x7fbbdc15ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.407+0000 7fbbe3dee700 1 --2- 172.21.15.45:0/892839891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 0x7fbbdc15ad80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc15eb20 tx=0x7fbbd400d430 comp rx=0 tx=0).stop 2023-08-27T11:12:37.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.408+0000 7fbbe3dee700 1 -- 172.21.15.45:0/892839891 shutdown_connections 2023-08-27T11:12:37.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.408+0000 7fbbe3dee700 1 --2- 172.21.15.45:0/892839891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 0x7fbbdc15ad80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.408+0000 7fbbe3dee700 1 --2- 172.21.15.45:0/892839891 >> v2:172.21.15.45:3300/0 conn(0x7fbbdc070f10 0x7fbbdc0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.408+0000 7fbbe3dee700 1 -- 172.21.15.45:0/892839891 >> 172.21.15.45:0/892839891 conn(0x7fbbdc0cc750 msgr2=0x7fbbdc0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.409+0000 7fbbe3dee700 1 -- 172.21.15.45:0/892839891 shutdown_connections 2023-08-27T11:12:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.409+0000 7fbbe3dee700 1 -- 172.21.15.45:0/892839891 wait complete. 2023-08-27T11:12:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.411+0000 7fbbe3dee700 1 Processor -- start 2023-08-27T11:12:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.412+0000 7fbbe3dee700 1 -- start start 2023-08-27T11:12:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe3dee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbdc070f10 0x7fbbdc14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe3dee700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbdc0712f0 0x7fbbdc151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe3dee700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 0x7fbbdc202750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe3dee700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc153030 con 0x7fbbdc070f10 2023-08-27T11:12:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe3dee700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc152180 con 0x7fbbdc0ce420 2023-08-27T11:12:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe3dee700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc202c90 con 0x7fbbdc0712f0 2023-08-27T11:12:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe1389700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbdc0712f0 0x7fbbdc151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe1389700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbdc0712f0 0x7fbbdc151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50094/0 (socket says 172.21.15.45:50094) 2023-08-27T11:12:37.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe1389700 1 -- 172.21.15.45:0/2664635036 learned_addr learned my addr 172.21.15.45:0/2664635036 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:37.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.413+0000 7fbbe238b700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 0x7fbbdc202750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.414+0000 7fbbe238b700 1 -- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbdc0712f0 msgr2=0x7fbbdc151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.414+0000 7fbbe238b700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbdc0712f0 0x7fbbdc151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.414+0000 7fbbe238b700 1 -- 172.21.15.45:0/2664635036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbdc070f10 msgr2=0x7fbbdc14fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.414+0000 7fbbe238b700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbdc070f10 0x7fbbdc14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.414+0000 7fbbe238b700 1 -- 172.21.15.45:0/2664635036 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd400bd40 con 0x7fbbdc0ce420 2023-08-27T11:12:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.414+0000 7fbbe238b700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 0x7fbbdc202750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4016000 tx=0x7fbbd4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.415+0000 7fbbd27fc700 1 -- 172.21.15.45:0/2664635036 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4009a10 con 0x7fbbdc0ce420 2023-08-27T11:12:37.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.415+0000 7fbbd27fc700 1 -- 172.21.15.45:0/2664635036 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4009b70 con 0x7fbbdc0ce420 2023-08-27T11:12:37.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.415+0000 7fbbd27fc700 1 -- 172.21.15.45:0/2664635036 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4004030 con 0x7fbbdc0ce420 2023-08-27T11:12:37.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.415+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbdc152420 con 0x7fbbdc0ce420 2023-08-27T11:12:37.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.415+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbdc1529e0 con 0x7fbbdc0ce420 2023-08-27T11:12:37.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.418+0000 7fbbd27fc700 1 -- 172.21.15.45:0/2664635036 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbbd4004190 con 0x7fbbdc0ce420 2023-08-27T11:12:37.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.419+0000 7fbbd27fc700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbc806a330 0x7fbbc806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:37.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.419+0000 7fbbd27fc700 1 -- 172.21.15.45:0/2664635036 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbbd408a030 con 0x7fbbdc0ce420 2023-08-27T11:12:37.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.421+0000 7fbbe1b8a700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbc806a330 0x7fbbc806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:37.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.421+0000 7fbbe1b8a700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbc806a330 0x7fbbc806c7f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0097b0 tx=0x7fbbcc006d20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:37.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.431+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc40052f0 con 0x7fbbdc0ce420 2023-08-27T11:12:37.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.437+0000 7fbbd27fc700 1 -- 172.21.15.45:0/2664635036 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbbd4058f30 con 0x7fbbdc0ce420 2023-08-27T11:12:37.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.794+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa670071fe0 con 0x7fa67015ca10 2023-08-27T11:12:37.802 INFO:teuthology.orchestra.run.smithi045.stdout:55834574860 2023-08-27T11:12:37.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.798+0000 7fa66d7fa700 1 -- 172.21.15.45:0/3215295105 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6600531b0 con 0x7fa67015ca10 2023-08-27T11:12:37.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.801+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa65c06a1b0 msgr2=0x7fa65c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.801+0000 7fa676bfe700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa65c06a1b0 0x7fa65c06c670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa664006fd0 tx=0x7fa664008040 comp rx=0 tx=0).stop 2023-08-27T11:12:37.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.801+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67015ca10 msgr2=0x7fa6700be930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.801+0000 7fa676bfe700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67015ca10 0x7fa6700be930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa66000c500 tx=0x7fa66000c8c0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.801+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8e540ca4a0 con 0x7f8e540c1220 2023-08-27T11:12:37.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff3c814f140 con 0x7ff3c80cbba0 2023-08-27T11:12:37.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 shutdown_connections 2023-08-27T11:12:37.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7fa676bfe700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa67015db30 0x7fa6700bc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7fa676bfe700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa65c06a1b0 0x7fa65c06c670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7fa676bfe700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa67006de10 0x7fa6700b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7fa676bfe700 1 --2- 172.21.15.45:0/3215295105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa67015ca10 0x7fa6700be930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 >> 172.21.15.45:0/3215295105 conn(0x7fa6700b07c0 msgr2=0x7fa67006fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 shutdown_connections 2023-08-27T11:12:37.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7fa676bfe700 1 -- 172.21.15.45:0/3215295105 wait complete. 2023-08-27T11:12:37.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.802+0000 7f8e49ffb700 1 -- 172.21.15.45:0/2895901236 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e50056ee0 con 0x7f8e540c1220 2023-08-27T11:12:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:37.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.805+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e3806a130 msgr2=0x7f8e3806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.805+0000 7f8e5b384700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e3806a130 0x7f8e3806c5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000f80 tx=0x7f8e4c009000 comp rx=0 tx=0).stop 2023-08-27T11:12:37.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.805+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e540c1220 msgr2=0x7f8e5414f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.805+0000 7f8e5b384700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e540c1220 0x7f8e5414f680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000eb90 tx=0x7f8e5000eea0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.805+0000 7ff3be7fc700 1 -- 172.21.15.45:0/122130365 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3b8020090 con 0x7ff3c80cbba0 2023-08-27T11:12:37.810 INFO:teuthology.orchestra.run.smithi045.stdout:55834574858 2023-08-27T11:12:37.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f25a00c3d10 con 0x7f25a00c9280 2023-08-27T11:12:37.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9b5c06ffa0 con 0x7f9b5c160890 2023-08-27T11:12:37.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff3ac06a130 msgr2=0x7ff3ac06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff3ac06a130 0x7ff3ac06c5f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000f60 tx=0x7ff3c4007a70 comp rx=0 tx=0).stop 2023-08-27T11:12:37.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c80cbba0 msgr2=0x7ff3c80c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c80cbba0 0x7ff3c80c2570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8015000 tx=0x7ff3b8004750 comp rx=0 tx=0).stop 2023-08-27T11:12:37.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 shutdown_connections 2023-08-27T11:12:37.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c815a1f0 0x7ff3c80c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff3ac06a130 0x7ff3ac06c5f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff3c8071280 0x7ff3c814c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 --2- 172.21.15.45:0/122130365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3c80cbba0 0x7ff3c80c2570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 >> 172.21.15.45:0/122130365 conn(0x7ff3c80b07c0 msgr2=0x7ff3c80af840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 shutdown_connections 2023-08-27T11:12:37.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7ff3d002b700 1 -- 172.21.15.45:0/122130365 wait complete. 2023-08-27T11:12:37.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.808+0000 7f25877fe700 1 -- 172.21.15.45:0/1774899206 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2594057390 con 0x7f25a00c9280 2023-08-27T11:12:37.814 INFO:teuthology.orchestra.run.smithi045.stdout:55834574858 2023-08-27T11:12:37.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.809+0000 7f9b58ff9700 1 -- 172.21.15.45:0/1288107970 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9b5c06ffa0 con 0x7f9b5c160890 2023-08-27T11:12:37.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.809+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5adc0b2420 con 0x7f5adc06e170 2023-08-27T11:12:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.810+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 shutdown_connections 2023-08-27T11:12:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.810+0000 7f8e5b384700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e540c9990 0x7f8e54150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.810+0000 7f8e5b384700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e3806a130 0x7f8e3806c5f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.810+0000 7f8e5b384700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e540c1220 0x7f8e5414f680 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.810+0000 7f8e5b384700 1 --2- 172.21.15.45:0/2895901236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e541619d0 0x7f8e540c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.810+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 >> 172.21.15.45:0/2895901236 conn(0x7f8e540b07c0 msgr2=0x7f8e540aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.817 INFO:teuthology.orchestra.run.smithi045.stdout:55834574858 2023-08-27T11:12:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.810+0000 7f5ad2ffd700 1 -- 172.21.15.45:0/4137685603 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5adc0b2420 con 0x7f5adc06e170 2023-08-27T11:12:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.810+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 shutdown_connections 2023-08-27T11:12:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.810+0000 7f8e5b384700 1 -- 172.21.15.45:0/2895901236 wait complete. 2023-08-27T11:12:37.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.811+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f21c0005870 con 0x7f21e00cd720 2023-08-27T11:12:37.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.811+0000 7f21c7fff700 1 -- 172.21.15.45:0/939166226 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f21d8017020 con 0x7f21e00cd720 2023-08-27T11:12:37.819 INFO:teuthology.orchestra.run.smithi045.stdout:55834574857 2023-08-27T11:12:37.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.812+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f258c06a130 msgr2=0x7f258c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.812+0000 7f25a536c700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f258c06a130 0x7f258c06c5f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2590000c00 tx=0x7f2590008030 comp rx=0 tx=0).stop 2023-08-27T11:12:37.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.812+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25a00c9280 msgr2=0x7f25a01f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.812+0000 7f25a536c700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25a00c9280 0x7f25a01f7430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f259400e9b0 tx=0x7f259400ed70 comp rx=0 tx=0).stop 2023-08-27T11:12:37.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f21cc06a0e0 msgr2=0x7f21cc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f21cc06a0e0 0x7f21cc06c5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009930 tx=0x7f21d4008040 comp rx=0 tx=0).stop 2023-08-27T11:12:37.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00cd720 msgr2=0x7f21e00c4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00cd720 0x7f21e00c4740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f21e015d000 tx=0x7f21d8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 shutdown_connections 2023-08-27T11:12:37.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e00c9490 0x7f21e015c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f21cc06a0e0 0x7f21cc06c5a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e00cf720 0x7f21e00b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 --2- 172.21.15.45:0/939166226 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e00cd720 0x7f21e00c4740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.815+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 >> 172.21.15.45:0/939166226 conn(0x7f21e00cec30 msgr2=0x7f21e00afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.816+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 shutdown_connections 2023-08-27T11:12:37.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.816+0000 7f21e5966700 1 -- 172.21.15.45:0/939166226 wait complete. 2023-08-27T11:12:37.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.813+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 shutdown_connections 2023-08-27T11:12:37.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.813+0000 7f25a536c700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25a015c070 0x7f25a01f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.813+0000 7f25a536c700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f258c06a130 0x7f258c06c5f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.813+0000 7f25a536c700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25a006d8b0 0x7f25a00b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.813+0000 7f25a536c700 1 --2- 172.21.15.45:0/1774899206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25a00c9280 0x7f25a01f7430 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.813+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 >> 172.21.15.45:0/1774899206 conn(0x7f25a00cec30 msgr2=0x7f25a0159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.825 INFO:teuthology.orchestra.run.smithi045.stdout:55834574858 2023-08-27T11:12:37.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.816+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5ac006a1b0 msgr2=0x7f5ac006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.816+0000 7f5ae43fa700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5ac006a1b0 0x7f5ac006c670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400cfa0 tx=0x7f5ad4012000 comp rx=0 tx=0).stop 2023-08-27T11:12:37.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.816+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5adc06e170 msgr2=0x7f5adc0b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.816+0000 7f5ae43fa700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5adc06e170 0x7f5adc0b30a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8006b20 tx=0x7f5ad800da80 comp rx=0 tx=0).stop 2023-08-27T11:12:37.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.818+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 shutdown_connections 2023-08-27T11:12:37.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.818+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9b4806a0e0 msgr2=0x7f9b4806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.818+0000 7f9b6265f700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9b4806a0e0 0x7f9b4806c5a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0d1830 tx=0x7f9b54009040 comp rx=0 tx=0).stop 2023-08-27T11:12:37.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.818+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b5c160890 msgr2=0x7f9b5c200510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.818+0000 7f9b6265f700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b5c160890 0x7f9b5c200510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b50004000 tx=0x7f9b50004750 comp rx=0 tx=0).stop 2023-08-27T11:12:37.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.818+0000 7f25a536c700 1 -- 172.21.15.45:0/1774899206 wait complete. 2023-08-27T11:12:37.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 shutdown_connections 2023-08-27T11:12:37.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f5ae43fa700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5adc0c97c0 0x7f5adc0b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f5ae43fa700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5ac006a1b0 0x7f5ac006c670 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f5ae43fa700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5adc06e170 0x7f5adc0b30a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f5ae43fa700 1 --2- 172.21.15.45:0/4137685603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5adc08a9a0 0x7f5adc16d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 >> 172.21.15.45:0/4137685603 conn(0x7f5adc0cec30 msgr2=0x7f5adc1f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 shutdown_connections 2023-08-27T11:12:37.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f9b6265f700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9b4806a0e0 0x7f9b4806c5a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f9b6265f700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b5c160890 0x7f9b5c200510 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f9b6265f700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9b5c0b5220 0x7f9b5c1f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f9b6265f700 1 --2- 172.21.15.45:0/1288107970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c0b62c0 0x7f9b5c1fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.823+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 >> 172.21.15.45:0/1288107970 conn(0x7f9b5c0cfb10 msgr2=0x7f9b5c161070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.825+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 shutdown_connections 2023-08-27T11:12:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.825+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 shutdown_connections 2023-08-27T11:12:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.825+0000 7f9b6265f700 1 -- 172.21.15.45:0/1288107970 wait complete. 2023-08-27T11:12:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.825+0000 7f5ae43fa700 1 -- 172.21.15.45:0/4137685603 wait complete. 2023-08-27T11:12:37.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2023-08-27T11:12:37.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574860 for osd.3 2023-08-27T11:12:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.838+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbbc40055a0 con 0x7fbbdc0ce420 2023-08-27T11:12:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.839+0000 7fbbd27fc700 1 -- 172.21.15.45:0/2664635036 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbbd4022020 con 0x7fbbdc0ce420 2023-08-27T11:12:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:55834574858 2023-08-27T11:12:37.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2023-08-27T11:12:37.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.2 2023-08-27T11:12:37.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbc806a330 msgr2=0x7fbbc806c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbc806a330 0x7fbbc806c7f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0097b0 tx=0x7fbbcc006d20 comp rx=0 tx=0).stop 2023-08-27T11:12:37.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 msgr2=0x7fbbdc202750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:37.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 0x7fbbdc202750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4016000 tx=0x7fbbd4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2023-08-27T11:12:37.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 shutdown_connections 2023-08-27T11:12:37.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbdc070f10 0x7fbbdc14fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbc806a330 0x7fbbc806c7f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbdc0ce420 0x7fbbdc202750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 --2- 172.21.15.45:0/2664635036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbdc0712f0 0x7fbbdc151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:37.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 >> 172.21.15.45:0/2664635036 conn(0x7fbbdc0cc750 msgr2=0x7fbbdc159290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:37.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 shutdown_connections 2023-08-27T11:12:37.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:37.841+0000 7fbbe3dee700 1 -- 172.21.15.45:0/2664635036 wait complete. 2023-08-27T11:12:37.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2023-08-27T11:12:37.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2023-08-27T11:12:37.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2023-08-27T11:12:38.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-27T11:12:38.838 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-27T11:12:38.845 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-27T11:12:38.846 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-27T11:12:38.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-27T11:12:38.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-27T11:12:38.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-27T11:12:38.858 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-27T11:12:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.049+0000 7ff467959700 1 Processor -- start 2023-08-27T11:12:39.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.050+0000 7ff467959700 1 -- start start 2023-08-27T11:12:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.051+0000 7ff467959700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff460159590 0x7ff4600c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.051+0000 7ff467959700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff46006fd20 0x7ff460070150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.051+0000 7ff467959700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff460070690 con 0x7ff46006c860 2023-08-27T11:12:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.051+0000 7ff467959700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600ca130 con 0x7ff460159590 2023-08-27T11:12:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.051+0000 7ff467959700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46006cd10 con 0x7ff46006fd20 2023-08-27T11:12:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.052+0000 7ff465ef6700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff46006fd20 0x7ff460070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.052+0000 7ff465ef6700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff46006fd20 0x7ff460070150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41052/0 (socket says 172.21.15.45:41052) 2023-08-27T11:12:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.052+0000 7ff465ef6700 1 -- 172.21.15.45:0/2396958694 learned_addr learned my addr 172.21.15.45:0/2396958694 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.052+0000 7ff4656f5700 1 --2- 172.21.15.45:0/2396958694 >> v2:172.21.15.45:3300/0 conn(0x7ff460159590 0x7ff4600c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.052+0000 7ff465ef6700 1 -- 172.21.15.45:0/2396958694 >> v2:172.21.15.45:3300/0 conn(0x7ff460159590 msgr2=0x7ff4600c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.052+0000 7ff465ef6700 1 --2- 172.21.15.45:0/2396958694 >> v2:172.21.15.45:3300/0 conn(0x7ff460159590 0x7ff4600c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.052+0000 7ff465ef6700 1 -- 172.21.15.45:0/2396958694 >> v1:172.21.15.135:6789/0 conn(0x7ff46006c860 legacy=0x7ff4600cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.052+0000 7ff465ef6700 1 -- 172.21.15.45:0/2396958694 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600c9b70 con 0x7ff46006fd20 2023-08-27T11:12:39.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.053+0000 7fcd2e286700 1 Processor -- start 2023-08-27T11:12:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.053+0000 7ff465ef6700 1 --2- 172.21.15.45:0/2396958694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 0x7ff460070150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff4600ca2c0 tx=0x7ff45000d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da4a1fdc07be39fe server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.053+0000 7fcd2e286700 1 -- start start 2023-08-27T11:12:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.054+0000 7fcd2e286700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcd280c9490 0x7fcd2806c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.054+0000 7fcd2e286700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd2806fd20 0x7fcd28070150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.054+0000 7fcd2e286700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd280c9940 con 0x7fcd28159590 2023-08-27T11:12:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.054+0000 7fcd2e286700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd280ca130 con 0x7fcd2806fd20 2023-08-27T11:12:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.054+0000 7fcd2e286700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2806cda0 con 0x7fcd280c9490 2023-08-27T11:12:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.053+0000 7ff457fff700 1 -- 172.21.15.45:0/2396958694 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450010070 con 0x7ff46006fd20 2023-08-27T11:12:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.053+0000 7ff457fff700 1 -- 172.21.15.45:0/2396958694 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45000da40 con 0x7ff46006fd20 2023-08-27T11:12:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.053+0000 7ff457fff700 1 -- 172.21.15.45:0/2396958694 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45000ddb0 con 0x7ff46006fd20 2023-08-27T11:12:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.054+0000 7ff467959700 1 -- 172.21.15.45:0/2396958694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 msgr2=0x7ff460070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.054+0000 7ff467959700 1 --2- 172.21.15.45:0/2396958694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 0x7ff460070150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff4600ca2c0 tx=0x7ff45000d330 comp rx=0 tx=0).stop 2023-08-27T11:12:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.055+0000 7ff467959700 1 -- 172.21.15.45:0/2396958694 shutdown_connections 2023-08-27T11:12:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.055+0000 7ff467959700 1 --2- 172.21.15.45:0/2396958694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 0x7ff460070150 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.055+0000 7ff467959700 1 --2- 172.21.15.45:0/2396958694 >> v2:172.21.15.45:3300/0 conn(0x7ff460159590 0x7ff4600c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.055+0000 7ff467959700 1 -- 172.21.15.45:0/2396958694 >> 172.21.15.45:0/2396958694 conn(0x7ff4600cec30 msgr2=0x7ff4600cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.055+0000 7ff467959700 1 -- 172.21.15.45:0/2396958694 shutdown_connections 2023-08-27T11:12:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.055+0000 7fcd2c823700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd2806fd20 0x7fcd28070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.055+0000 7fcd2c823700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd2806fd20 0x7fcd28070150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52200/0 (socket says 172.21.15.45:52200) 2023-08-27T11:12:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.055+0000 7fcd2c823700 1 -- 172.21.15.45:0/3535344406 learned_addr learned my addr 172.21.15.45:0/3535344406 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.056+0000 7fcd2c823700 1 -- 172.21.15.45:0/3535344406 >> v2:172.21.15.135:3300/0 conn(0x7fcd280c9490 msgr2=0x7fcd2806c860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.056+0000 7fcd2c823700 1 --2- 172.21.15.45:0/3535344406 >> v2:172.21.15.135:3300/0 conn(0x7fcd280c9490 0x7fcd2806c860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.056+0000 7fcd2c823700 1 -- 172.21.15.45:0/3535344406 >> v1:172.21.15.135:6789/0 conn(0x7fcd28159590 legacy=0x7fcd280cd720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.056+0000 7fcd2c823700 1 -- 172.21.15.45:0/3535344406 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd28070690 con 0x7fcd2806fd20 2023-08-27T11:12:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.056+0000 7fcd2c823700 1 --2- 172.21.15.45:0/3535344406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 0x7fcd28070150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd28071d10 tx=0x7fcd2000d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fcc8acfdc64b28 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.055+0000 7ff467959700 1 -- 172.21.15.45:0/2396958694 wait complete. 2023-08-27T11:12:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.057+0000 7ff467959700 1 Processor -- start 2023-08-27T11:12:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.057+0000 7ff467959700 1 -- start start 2023-08-27T11:12:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.058+0000 7ff467959700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff46006c860 0x7ff460154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.058+0000 7ff467959700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 0x7ff460157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.058+0000 7ff467959700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff460159590 0x7ff4601575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.058+0000 7ff467959700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4601e8f50 con 0x7ff46006c860 2023-08-27T11:12:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.058+0000 7ff467959700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460203c60 con 0x7ff46006fd20 2023-08-27T11:12:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.058+0000 7ff467959700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff460203f40 con 0x7ff460159590 2023-08-27T11:12:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.058+0000 7ff4656f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff46006c860 0x7ff460154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff4656f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff46006c860 0x7ff460154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52204/0 (socket says 172.21.15.45:52204) 2023-08-27T11:12:39.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff4656f5700 1 -- 172.21.15.45:0/3440166837 learned_addr learned my addr 172.21.15.45:0/3440166837 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff464ef4700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 0x7ff460157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff465ef6700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff460159590 0x7ff4601575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff464ef4700 1 -- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff460159590 msgr2=0x7ff4601575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff464ef4700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff460159590 0x7ff4601575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff464ef4700 1 -- 172.21.15.45:0/3440166837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff46006c860 msgr2=0x7ff460154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff464ef4700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff46006c860 0x7ff460154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff464ef4700 1 -- 172.21.15.45:0/3440166837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4500085d0 con 0x7ff46006fd20 2023-08-27T11:12:39.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff4656f5700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff46006c860 0x7ff460154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.059+0000 7ff465ef6700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff460159590 0x7ff4601575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.060+0000 7ff464ef4700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 0x7ff460157070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff45800cb90 tx=0x7ff45800cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.060+0000 7ff455ffb700 1 -- 172.21.15.45:0/3440166837 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800aac0 con 0x7ff46006fd20 2023-08-27T11:12:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.060+0000 7ff455ffb700 1 -- 172.21.15.45:0/3440166837 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800ac20 con 0x7ff46006fd20 2023-08-27T11:12:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.060+0000 7ff455ffb700 1 -- 172.21.15.45:0/3440166837 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458012420 con 0x7ff46006fd20 2023-08-27T11:12:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.060+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4601e6130 con 0x7ff46006fd20 2023-08-27T11:12:39.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.060+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4601e66f0 con 0x7ff46006fd20 2023-08-27T11:12:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.061+0000 7fcd26ffd700 1 -- 172.21.15.45:0/3535344406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20010070 con 0x7fcd2806fd20 2023-08-27T11:12:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.061+0000 7fcd26ffd700 1 -- 172.21.15.45:0/3535344406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2000dc90 con 0x7fcd2806fd20 2023-08-27T11:12:39.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.061+0000 7fcd26ffd700 1 -- 172.21.15.45:0/3535344406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20007b30 con 0x7fcd2806fd20 2023-08-27T11:12:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.063+0000 7ff455ffb700 1 -- 172.21.15.45:0/3440166837 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff458012580 con 0x7ff46006fd20 2023-08-27T11:12:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.064+0000 7ff455ffb700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff44c06a260 0x7ff44c06c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.064+0000 7ff455ffb700 1 -- 172.21.15.45:0/3440166837 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff458088f50 con 0x7ff46006fd20 2023-08-27T11:12:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.064+0000 7ff4656f5700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff44c06a260 0x7ff44c06c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.063+0000 7fcd2e286700 1 -- 172.21.15.45:0/3535344406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 msgr2=0x7fcd28070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.063+0000 7fcd2e286700 1 --2- 172.21.15.45:0/3535344406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 0x7fcd28070150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcd28071d10 tx=0x7fcd2000d680 comp rx=0 tx=0).stop 2023-08-27T11:12:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.063+0000 7fcd2e286700 1 -- 172.21.15.45:0/3535344406 shutdown_connections 2023-08-27T11:12:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.063+0000 7fcd2e286700 1 --2- 172.21.15.45:0/3535344406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 0x7fcd28070150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.063+0000 7fcd2e286700 1 --2- 172.21.15.45:0/3535344406 >> v2:172.21.15.135:3300/0 conn(0x7fcd280c9490 0x7fcd2806c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.063+0000 7fcd2e286700 1 -- 172.21.15.45:0/3535344406 >> 172.21.15.45:0/3535344406 conn(0x7fcd280cec30 msgr2=0x7fcd280cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.065+0000 7ff4656f5700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff44c06a260 0x7ff44c06c720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff45c007b10 tx=0x7ff45c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.068+0000 7fcd2e286700 1 -- 172.21.15.45:0/3535344406 shutdown_connections 2023-08-27T11:12:39.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.069+0000 7fcd2e286700 1 -- 172.21.15.45:0/3535344406 wait complete. 2023-08-27T11:12:39.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.070+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4601e6910 con 0x7ff46006fd20 2023-08-27T11:12:39.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.071+0000 7fcd2e286700 1 Processor -- start 2023-08-27T11:12:39.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.072+0000 7fcd2e286700 1 -- start start 2023-08-27T11:12:39.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.073+0000 7fcd2e286700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 0x7fcd281f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.073+0000 7fcd2e286700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280c9490 0x7fcd28153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.073+0000 7fcd2e286700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd28159590 0x7fcd2815c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.073+0000 7fcd2e286700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd280a9c20 con 0x7fcd2806fd20 2023-08-27T11:12:39.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.073+0000 7fcd2e286700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd28159970 con 0x7fcd280c9490 2023-08-27T11:12:39.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.073+0000 7fcd2e286700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd281f7750 con 0x7fcd28159590 2023-08-27T11:12:39.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.073+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 0x7fcd281f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.073+0000 7fcd27fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 0x7fcd281f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52212/0 (socket says 172.21.15.45:52212) 2023-08-27T11:12:39.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.073+0000 7fcd27fff700 1 -- 172.21.15.45:0/760461375 learned_addr learned my addr 172.21.15.45:0/760461375 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd27fff700 1 -- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd28159590 msgr2=0x7fcd2815c430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-27T11:12:39.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd27fff700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd28159590 0x7fcd2815c430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd27fff700 1 -- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280c9490 msgr2=0x7fcd28153a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:12:39.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd27fff700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280c9490 0x7fcd28153a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd27fff700 1 -- 172.21.15.45:0/760461375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2000d050 con 0x7fcd2806fd20 2023-08-27T11:12:39.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd27fff700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 0x7fcd281f6f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00b470 tx=0x7fcd1c00b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd24ff9700 1 -- 172.21.15.45:0/760461375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c011830 con 0x7fcd2806fd20 2023-08-27T11:12:39.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd24ff9700 1 -- 172.21.15.45:0/760461375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c0119b0 con 0x7fcd2806fd20 2023-08-27T11:12:39.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd24ff9700 1 -- 172.21.15.45:0/760461375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c011d60 con 0x7fcd2806fd20 2023-08-27T11:12:39.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd281f7a30 con 0x7fcd2806fd20 2023-08-27T11:12:39.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.074+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd28159b80 con 0x7fcd2806fd20 2023-08-27T11:12:39.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.076+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd28159da0 con 0x7fcd2806fd20 2023-08-27T11:12:39.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.087+0000 7ff455ffb700 1 -- 172.21.15.45:0/3440166837 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff458016020 con 0x7ff46006fd20 2023-08-27T11:12:39.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.078+0000 7fcd24ff9700 1 -- 172.21.15.45:0/760461375 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcd1c01a400 con 0x7fcd2806fd20 2023-08-27T11:12:39.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.078+0000 7fcd24ff9700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcd1406a0e0 0x7fcd1406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.078+0000 7fcd24ff9700 1 -- 172.21.15.45:0/760461375 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcd1c088910 con 0x7fcd2806fd20 2023-08-27T11:12:39.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.082+0000 7fcd277fe700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcd1406a0e0 0x7fcd1406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.082+0000 7fcd277fe700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcd1406a0e0 0x7fcd1406c5a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcd28153790 tx=0x7fcd18006c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.088+0000 7fcd24ff9700 1 -- 172.21.15.45:0/760461375 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcd1c053d10 con 0x7fcd2806fd20 2023-08-27T11:12:39.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.109+0000 7f1ba0b30700 1 Processor -- start 2023-08-27T11:12:39.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.109+0000 7f1ba0b30700 1 -- start start 2023-08-27T11:12:39.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.110+0000 7f1ba0b30700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c165ce0 0x7f1b9c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.110+0000 7f1ba0b30700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c164f50 con 0x7f1b9c06a7c0 2023-08-27T11:12:39.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.110+0000 7f1ba0b30700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c165850 con 0x7f1b9c164b70 2023-08-27T11:12:39.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.110+0000 7f1ba0b30700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c06ac50 con 0x7f1b9c165ce0 2023-08-27T11:12:39.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b9ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1b9c06a7c0 0x7f1b9c070d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38998/0 (socket says 172.21.15.45:38998) 2023-08-27T11:12:39.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b9ad9d700 1 -- 172.21.15.45:0/734596518 learned_addr learned my addr 172.21.15.45:0/734596518 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:39.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b99d9b700 1 --2- 172.21.15.45:0/734596518 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c165ce0 0x7f1b9c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b9959a700 1 -- 172.21.15.45:0/734596518 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902734753 0 0) 0x7f1b9c165850 con 0x7f1b9c164b70 2023-08-27T11:12:39.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b9959a700 1 -- 172.21.15.45:0/734596518 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b840035b0 con 0x7f1b9c164b70 2023-08-27T11:12:39.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b9959a700 1 -- 172.21.15.45:0/734596518 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276255387 0 0) 0x7f1b90001110 con 0x7f1b9c06a7c0 2023-08-27T11:12:39.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b9959a700 1 -- 172.21.15.45:0/734596518 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b9c165850 con 0x7f1b9c06a7c0 2023-08-27T11:12:39.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b99d9b700 1 -- 172.21.15.45:0/734596518 >> v1:172.21.15.135:6789/0 conn(0x7f1b9c164b70 legacy=0x7f1b9c0cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b99d9b700 1 -- 172.21.15.45:0/734596518 >> v1:172.21.15.45:6789/0 conn(0x7f1b9c06a7c0 legacy=0x7f1b9c070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b99d9b700 1 -- 172.21.15.45:0/734596518 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c0cfaf0 con 0x7f1b9c165ce0 2023-08-27T11:12:39.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.111+0000 7f1b99d9b700 1 --2- 172.21.15.45:0/734596518 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c165ce0 0x7f1b9c06a280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009dc0 tx=0x7f1b9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cffb81ad5212aa7b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.112+0000 7f7361555700 1 Processor -- start 2023-08-27T11:12:39.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f663ffbc700 1 Processor -- start 2023-08-27T11:12:39.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.112+0000 7f7361555700 1 -- start start 2023-08-27T11:12:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f7361555700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f735c159590 0x7f735c0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f7361555700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c06c860 0x7f735c0cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f7361555700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c0c9b70 con 0x7f735c0cdc60 2023-08-27T11:12:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f7361555700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c0ca130 con 0x7f735c159590 2023-08-27T11:12:39.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f7361555700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0ce090 con 0x7f735c06c860 2023-08-27T11:12:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f735affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f735c159590 0x7f735c0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f735affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f735c159590 0x7f735c0c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52224/0 (socket says 172.21.15.45:52224) 2023-08-27T11:12:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f735affd700 1 -- 172.21.15.45:0/1391266926 learned_addr learned my addr 172.21.15.45:0/1391266926 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f663ffbc700 1 -- start start 2023-08-27T11:12:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.114+0000 7f735affd700 1 -- 172.21.15.45:0/1391266926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c06c860 msgr2=0x7f735c0cd720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.114+0000 7f735affd700 1 --2- 172.21.15.45:0/1391266926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c06c860 0x7f735c0cd720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.114+0000 7f735affd700 1 -- 172.21.15.45:0/1391266926 >> v1:172.21.15.45:6789/0 conn(0x7f735c0cdc60 legacy=0x7f735c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:39.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.114+0000 7f735affd700 1 -- 172.21.15.45:0/1391266926 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c0c9e50 con 0x7f735c159590 2023-08-27T11:12:39.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.114+0000 7f735affd700 1 --2- 172.21.15.45:0/1391266926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735c159590 0x7f735c0c9490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f735400d370 tx=0x7f735400d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934ec9b866021b7a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f1b9959a700 1 -- 172.21.15.45:0/734596518 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400e070 con 0x7f1b9c165ce0 2023-08-27T11:12:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f1b9959a700 1 -- 172.21.15.45:0/734596518 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400bb20 con 0x7f1b9c165ce0 2023-08-27T11:12:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f1b9959a700 1 -- 172.21.15.45:0/734596518 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94012430 con 0x7f1b9c165ce0 2023-08-27T11:12:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f7d9fa45700 1 Processor -- start 2023-08-27T11:12:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.113+0000 7f7d9fa45700 1 -- start start 2023-08-27T11:12:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f7d9fa45700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d9815c070 0x7f7d980cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f7d9fa45700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7d9806d8b0 0x7f7d9806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f7d9fa45700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f7d9fa45700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d980c9c30 con 0x7f7d9815c070 2023-08-27T11:12:39.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f7d9fa45700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9815c450 con 0x7f7d9806d8b0 2023-08-27T11:12:39.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f7d9fa45700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98162e70 con 0x7f7d980c9280 2023-08-27T11:12:39.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f663ffbc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66380cc3f0 0x7f66380ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f663ffbc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6638071d70 con 0x7f663815d860 2023-08-27T11:12:39.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f663ffbc700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6638072050 con 0x7f663815d480 2023-08-27T11:12:39.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f663ffbc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6638072330 con 0x7f66380cc3f0 2023-08-27T11:12:39.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f663d557700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f663815d860 0x7f66380cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39010/0 (socket says 172.21.15.45:39010) 2023-08-27T11:12:39.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f663d557700 1 -- 172.21.15.45:0/3702969598 learned_addr learned my addr 172.21.15.45:0/3702969598 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:39.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f7d9dfe2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f7d9dfe2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50142/0 (socket says 172.21.15.45:50142) 2023-08-27T11:12:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.115+0000 7f7d9dfe2700 1 -- 172.21.15.45:0/3366821321 learned_addr learned my addr 172.21.15.45:0/3366821321 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f7d9dfe2700 1 -- 172.21.15.45:0/3366821321 >> v2:172.21.15.135:3300/0 conn(0x7f7d9806d8b0 msgr2=0x7f7d9806dd10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f7d9dfe2700 1 --2- 172.21.15.45:0/3366821321 >> v2:172.21.15.135:3300/0 conn(0x7f7d9806d8b0 0x7f7d9806dd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f7d9dfe2700 1 -- 172.21.15.45:0/3366821321 >> v2:172.21.15.45:3300/0 conn(0x7f7d9815c070 msgr2=0x7f7d980cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:39.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f7d9dfe2700 1 --2- 172.21.15.45:0/3366821321 >> v2:172.21.15.45:3300/0 conn(0x7f7d9815c070 0x7f7d980cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f7d9dfe2700 1 -- 172.21.15.45:0/3366821321 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d98162260 con 0x7f7d980c9280 2023-08-27T11:12:39.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646650307 0 0) 0x7f6638071d70 con 0x7f663815d860 2023-08-27T11:12:39.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f1ba0b30700 1 -- 172.21.15.45:0/734596518 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c165ce0 msgr2=0x7f1b9c06a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f1ba0b30700 1 --2- 172.21.15.45:0/734596518 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c165ce0 0x7f1b9c06a280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009dc0 tx=0x7f1b9400b330 comp rx=0 tx=0).stop 2023-08-27T11:12:39.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f1ba0b30700 1 -- 172.21.15.45:0/734596518 shutdown_connections 2023-08-27T11:12:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f1ba0b30700 1 --2- 172.21.15.45:0/734596518 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c165ce0 0x7f1b9c06a280 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f1ba0b30700 1 -- 172.21.15.45:0/734596518 >> 172.21.15.45:0/734596518 conn(0x7f1b9c0cc7c0 msgr2=0x7f1b9c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66240035b0 con 0x7f663815d860 2023-08-27T11:12:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681618787 0 0) 0x7f6638072050 con 0x7f663815d480 2023-08-27T11:12:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6638071d70 con 0x7f663815d480 2023-08-27T11:12:39.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2114756569 0 0) 0x7f66240035b0 con 0x7f663815d860 2023-08-27T11:12:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 >> v2:172.21.15.135:3300/0 conn(0x7f66380cc3f0 msgr2=0x7f66380ce440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 --2- 172.21.15.45:0/3702969598 >> v2:172.21.15.135:3300/0 conn(0x7f66380cc3f0 0x7f66380ce440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 >> v1:172.21.15.135:6789/0 conn(0x7f663815d480 legacy=0x7f66380cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6638072610 con 0x7f663815d860 2023-08-27T11:12:39.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f6634001670 con 0x7f663815d860 2023-08-27T11:12:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f7d9dfe2700 1 --2- 172.21.15.45:0/3366821321 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c96f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d980c26f0 tx=0x7f7d9000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e4c9ffbf81be9a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 >> v1:172.21.15.45:6789/0 conn(0x7f663815d860 legacy=0x7f66380cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6624004bf0 0x7f6624004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66240037f0 0x7f6624003c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66240042d0 0x7f662400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6638072330 con 0x7f66240037f0 2023-08-27T11:12:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662400c230 con 0x7f6624004bf0 2023-08-27T11:12:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.116+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662400c3f0 con 0x7f66240042d0 2023-08-27T11:12:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663dd58700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6624004bf0 0x7f6624004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663d557700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66240037f0 0x7f6624003c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663dd58700 1 -- 172.21.15.45:0/3702969598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66240042d0 msgr2=0x7f662400ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:12:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663dd58700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66240042d0 0x7f662400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663dd58700 1 -- 172.21.15.45:0/3702969598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66240037f0 msgr2=0x7f6624003c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663dd58700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66240037f0 0x7f6624003c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663dd58700 1 -- 172.21.15.45:0/3702969598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6638072610 con 0x7f6624004bf0 2023-08-27T11:12:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663e559700 1 -- 172.21.15.45:0/3702969598 reap_dead start 2023-08-27T11:12:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663dd58700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6624004bf0 0x7f6624004ff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6628001870 tx=0x7f6624005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82d75635035e5c13 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f1ba0b30700 1 -- 172.21.15.45:0/734596518 shutdown_connections 2023-08-27T11:12:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f7359ffb700 1 -- 172.21.15.45:0/1391266926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354010070 con 0x7f735c159590 2023-08-27T11:12:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f7359ffb700 1 -- 172.21.15.45:0/1391266926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735400be90 con 0x7f735c159590 2023-08-27T11:12:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f7359ffb700 1 -- 172.21.15.45:0/1391266926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354003a20 con 0x7f735c159590 2023-08-27T11:12:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628001360 con 0x7f6624004bf0 2023-08-27T11:12:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66280052d0 con 0x7f6624004bf0 2023-08-27T11:12:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f663cd56700 1 -- 172.21.15.45:0/3702969598 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628006860 con 0x7f6624004bf0 2023-08-27T11:12:39.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7361555700 1 -- 172.21.15.45:0/1391266926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735c159590 msgr2=0x7f735c0c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7361555700 1 --2- 172.21.15.45:0/1391266926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735c159590 0x7f735c0c9490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f735400d370 tx=0x7f735400d680 comp rx=0 tx=0).stop 2023-08-27T11:12:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7361555700 1 -- 172.21.15.45:0/1391266926 shutdown_connections 2023-08-27T11:12:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7361555700 1 --2- 172.21.15.45:0/1391266926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735c159590 0x7f735c0c9490 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7361555700 1 --2- 172.21.15.45:0/1391266926 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c06c860 0x7f735c0cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7361555700 1 -- 172.21.15.45:0/1391266926 >> 172.21.15.45:0/1391266926 conn(0x7f735c0cec30 msgr2=0x7f735c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f663ffbc700 1 -- 172.21.15.45:0/3702969598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6624004bf0 msgr2=0x7f6624004ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f663ffbc700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6624004bf0 0x7f6624004ff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6628001870 tx=0x7f6624005880 comp rx=0 tx=0).stop 2023-08-27T11:12:39.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f663ffbc700 1 -- 172.21.15.45:0/3702969598 shutdown_connections 2023-08-27T11:12:39.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f663ffbc700 1 --2- 172.21.15.45:0/3702969598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6624004bf0 0x7f6624004ff0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f663ffbc700 1 -- 172.21.15.45:0/3702969598 >> 172.21.15.45:0/3702969598 conn(0x7f66380b2440 msgr2=0x7f66380b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.119+0000 7f663ffbc700 1 -- 172.21.15.45:0/3702969598 shutdown_connections 2023-08-27T11:12:39.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.119+0000 7f663ffbc700 1 -- 172.21.15.45:0/3702969598 wait complete. 2023-08-27T11:12:39.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f7d8ffff700 1 -- 172.21.15.45:0/3366821321 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000e070 con 0x7f7d980c9280 2023-08-27T11:12:39.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f7d8ffff700 1 -- 172.21.15.45:0/3366821321 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9000b980 con 0x7f7d980c9280 2023-08-27T11:12:39.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.117+0000 7f7d8ffff700 1 -- 172.21.15.45:0/3366821321 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000bd30 con 0x7f7d980c9280 2023-08-27T11:12:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3366821321 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 msgr2=0x7f7d980c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3366821321 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c96f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d980c26f0 tx=0x7f7d9000b330 comp rx=0 tx=0).stop 2023-08-27T11:12:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3366821321 shutdown_connections 2023-08-27T11:12:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3366821321 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c96f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3366821321 >> v2:172.21.15.45:3300/0 conn(0x7f7d9815c070 0x7f7d980cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3366821321 >> v2:172.21.15.135:3300/0 conn(0x7f7d9806d8b0 0x7f7d9806dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3366821321 >> 172.21.15.45:0/3366821321 conn(0x7f7d980cec30 msgr2=0x7f7d980cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3366821321 shutdown_connections 2023-08-27T11:12:39.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.118+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3366821321 wait complete. 2023-08-27T11:12:39.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.120+0000 7f7d9fa45700 1 Processor -- start 2023-08-27T11:12:39.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.120+0000 7f7d9fa45700 1 -- start start 2023-08-27T11:12:39.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f663ffbc700 1 Processor -- start 2023-08-27T11:12:39.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f663ffbc700 1 -- start start 2023-08-27T11:12:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.119+0000 7f1ba0b30700 1 -- 172.21.15.45:0/734596518 wait complete. 2023-08-27T11:12:39.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f1ba0b30700 1 Processor -- start 2023-08-27T11:12:39.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f7d9fa45700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d9806d8b0 0x7f7d98070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f7d9fa45700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f7d9fa45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d9815c070 0x7f7d980c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f7d9fa45700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98071460 con 0x7f7d9815c070 2023-08-27T11:12:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f7d9fa45700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980c3770 con 0x7f7d9806d8b0 2023-08-27T11:12:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f7d9fa45700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980c4510 con 0x7f7d980c9280 2023-08-27T11:12:39.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f1ba0b30700 1 -- start start 2023-08-27T11:12:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f663ffbc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6624004bf0 0x7f66380c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f663ffbc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66380ad3f0 0x7f66381f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.121+0000 7f7d9dfe2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d9815c070 0x7f7d980c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f7d9dfe2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d9815c070 0x7f7d980c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52250/0 (socket says 172.21.15.45:52250) 2023-08-27T11:12:39.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f7d9dfe2700 1 -- 172.21.15.45:0/3732487939 learned_addr learned my addr 172.21.15.45:0/3732487939 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663ffbc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66380ad870 0x7f66381f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f7d9cfe0700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663ffbc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66380c7ab0 con 0x7f6624004bf0 2023-08-27T11:12:39.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663ffbc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66381526c0 con 0x7f66380ad3f0 2023-08-27T11:12:39.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663ffbc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66381f24e0 con 0x7f66380ad870 2023-08-27T11:12:39.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f7d9d7e1700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d9806d8b0 0x7f7d98070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663d557700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66380ad3f0 0x7f66381f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663e559700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66380ad870 0x7f66381f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663d557700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66380ad3f0 0x7f66381f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41108/0 (socket says 172.21.15.45:41108) 2023-08-27T11:12:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663d557700 1 -- 172.21.15.45:0/4050079888 learned_addr learned my addr 172.21.15.45:0/4050079888 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663dd58700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6624004bf0 0x7f66380c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f7d9cfe0700 1 -- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d9806d8b0 msgr2=0x7f7d98070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f7d9cfe0700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d9806d8b0 0x7f7d98070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f7d9cfe0700 1 -- 172.21.15.45:0/3732487939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d9815c070 msgr2=0x7f7d980c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f7d9cfe0700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d9815c070 0x7f7d980c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f7d9cfe0700 1 -- 172.21.15.45:0/3732487939 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90009b30 con 0x7f7d980c9280 2023-08-27T11:12:39.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1ba0b30700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b9c06a7c0 0x7f1b9c158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1ba0b30700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c164b70 0x7f1b9c1f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1ba0b30700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b9c165ce0 0x7f1b9c1f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1ba0b30700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0b34a0 con 0x7f1b9c165ce0 2023-08-27T11:12:39.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1ba0b30700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0b2b90 con 0x7f1b9c06a7c0 2023-08-27T11:12:39.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1ba0b30700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c1f6010 con 0x7f1b9c164b70 2023-08-27T11:12:39.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1b9ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b9c165ce0 0x7f1b9c1f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1b9ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b9c165ce0 0x7f1b9c1f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52260/0 (socket says 172.21.15.45:52260) 2023-08-27T11:12:39.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1b9ad9d700 1 -- 172.21.15.45:0/714381256 learned_addr learned my addr 172.21.15.45:0/714381256 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663d557700 1 -- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66380ad870 msgr2=0x7f66381f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663d557700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66380ad870 0x7f66381f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663d557700 1 -- 172.21.15.45:0/4050079888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6624004bf0 msgr2=0x7f66380c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663d557700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6624004bf0 0x7f66380c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663d557700 1 -- 172.21.15.45:0/4050079888 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66380af3c0 con 0x7f66380ad3f0 2023-08-27T11:12:39.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f663dd58700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6624004bf0 0x7f66380c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:39.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1b9a59c700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b9c06a7c0 0x7f1b9c158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.122+0000 7f1b99d9b700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c164b70 0x7f1b9c1f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f663d557700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66380ad3f0 0x7f66381f1a60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6634007400 tx=0x7f6634007710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f7d9cfe0700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c3230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b410 tx=0x7f7d94009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f662e7fc700 1 -- 172.21.15.45:0/4050079888 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663400f8d0 con 0x7f66380ad3f0 2023-08-27T11:12:39.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f460f210700 1 Processor -- start 2023-08-27T11:12:39.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f662e7fc700 1 -- 172.21.15.45:0/4050079888 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663400fa50 con 0x7f66380ad3f0 2023-08-27T11:12:39.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f662e7fc700 1 -- 172.21.15.45:0/4050079888 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663400fe00 con 0x7f66380ad3f0 2023-08-27T11:12:39.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66380af6a0 con 0x7f66380ad3f0 2023-08-27T11:12:39.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66380afc60 con 0x7f66380ad3f0 2023-08-27T11:12:39.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f1b9ad9d700 1 -- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c164b70 msgr2=0x7f1b9c1f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f1b9ad9d700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c164b70 0x7f1b9c1f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f1b9ad9d700 1 -- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b9c06a7c0 msgr2=0x7f1b9c158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f1b9ad9d700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b9c06a7c0 0x7f1b9c158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f1b9ad9d700 1 -- 172.21.15.45:0/714381256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b94009a70 con 0x7f1b9c165ce0 2023-08-27T11:12:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f7361555700 1 -- 172.21.15.45:0/1391266926 shutdown_connections 2023-08-27T11:12:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f7361555700 1 -- 172.21.15.45:0/1391266926 wait complete. 2023-08-27T11:12:39.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f1b9ad9d700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b9c165ce0 0x7f1b9c1f5ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c4d0 tx=0x7f1b9000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.123+0000 7f1b8affd700 1 -- 172.21.15.45:0/714381256 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90005d70 con 0x7f1b9c165ce0 2023-08-27T11:12:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.124+0000 7f1b8affd700 1 -- 172.21.15.45:0/714381256 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b900070d0 con 0x7f1b9c165ce0 2023-08-27T11:12:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.124+0000 7f1b8affd700 1 -- 172.21.15.45:0/714381256 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90007460 con 0x7f1b9c165ce0 2023-08-27T11:12:39.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.124+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c1f62c0 con 0x7f1b9c165ce0 2023-08-27T11:12:39.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.124+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66380af830 con 0x7f66380ad3f0 2023-08-27T11:12:39.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.124+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c0b30c0 con 0x7f1b9c165ce0 2023-08-27T11:12:39.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.125+0000 7ffb65dd7700 1 Processor -- start 2023-08-27T11:12:39.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.125+0000 7ffb65dd7700 1 -- start start 2023-08-27T11:12:39.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.125+0000 7f7361555700 1 Processor -- start 2023-08-27T11:12:39.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.125+0000 7f7d8dffb700 1 -- 172.21.15.45:0/3732487939 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94011620 con 0x7f7d980c9280 2023-08-27T11:12:39.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7f7d8dffb700 1 -- 172.21.15.45:0/3732487939 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d940117a0 con 0x7f7d980c9280 2023-08-27T11:12:39.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7f7d8dffb700 1 -- 172.21.15.45:0/3732487939 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94011b50 con 0x7f7d980c9280 2023-08-27T11:12:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7ffb65dd7700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffb600cf440 0x7ffb60165600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7ffb65dd7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb60071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7ffb65dd7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb600b9070 con 0x7ffb60165220 2023-08-27T11:12:39.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7ffb65dd7700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb600b66c0 con 0x7ffb600cf440 2023-08-27T11:12:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7ffb65dd7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600b85c0 con 0x7ffb60071630 2023-08-27T11:12:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7ffb5f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffb60165220 0x7ffb600b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39040/0 (socket says 172.21.15.45:39040) 2023-08-27T11:12:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7ffb5f7fe700 1 -- 172.21.15.45:0/2379384774 learned_addr learned my addr 172.21.15.45:0/2379384774 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c0cd860 con 0x7f1b9c165ce0 2023-08-27T11:12:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7f1b8affd700 1 -- 172.21.15.45:0/714381256 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1b900075c0 con 0x7f1b9c165ce0 2023-08-27T11:12:39.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7ffb5ffff700 1 --2- 172.21.15.45:0/2379384774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb60071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7ffb5ffff700 1 -- 172.21.15.45:0/2379384774 >> v2:172.21.15.135:3300/0 conn(0x7ffb600cf440 msgr2=0x7ffb60165600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-27T11:12:39.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7ffb5ffff700 1 --2- 172.21.15.45:0/2379384774 >> v2:172.21.15.135:3300/0 conn(0x7ffb600cf440 0x7ffb60165600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7ffb5ffff700 1 -- 172.21.15.45:0/2379384774 >> v1:172.21.15.45:6789/0 conn(0x7ffb60165220 legacy=0x7ffb600b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7ffb5ffff700 1 -- 172.21.15.45:0/2379384774 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60165b40 con 0x7ffb60071630 2023-08-27T11:12:39.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d98071650 con 0x7f7d980c9280 2023-08-27T11:12:39.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d980c3ec0 con 0x7f7d980c9280 2023-08-27T11:12:39.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f1b8affd700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1b7c06a0e0 0x7f1b7c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f1b8affd700 1 -- 172.21.15.45:0/714381256 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1b90087c60 con 0x7f1b9c165ce0 2023-08-27T11:12:39.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7ffb5ffff700 1 --2- 172.21.15.45:0/2379384774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb60071ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009840 tx=0x7ffb50009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b5f9c0796c693 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.128+0000 7ffb5e7fc700 1 -- 172.21.15.45:0/2379384774 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50011830 con 0x7ffb60071630 2023-08-27T11:12:39.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.128+0000 7ffb5e7fc700 1 -- 172.21.15.45:0/2379384774 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb50011990 con 0x7ffb60071630 2023-08-27T11:12:39.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.128+0000 7ffb5e7fc700 1 -- 172.21.15.45:0/2379384774 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50011d20 con 0x7ffb60071630 2023-08-27T11:12:39.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.126+0000 7f7361555700 1 -- start start 2023-08-27T11:12:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f7361555700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f735c06c860 0x7f735c1f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f7361555700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735c0cdc60 0x7f735c1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f7361555700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c159590 0x7f735c0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f7361555700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c1f8160 con 0x7f735c0cdc60 2023-08-27T11:12:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f7361555700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0c6030 con 0x7f735c06c860 2023-08-27T11:12:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f7361555700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0c6310 con 0x7f735c159590 2023-08-27T11:12:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735b7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c159590 0x7f735c0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735b7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c159590 0x7f735c0c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50184/0 (socket says 172.21.15.45:50184) 2023-08-27T11:12:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735b7fe700 1 -- 172.21.15.45:0/169813461 learned_addr learned my addr 172.21.15.45:0/169813461 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735affd700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f735c06c860 0x7f735c1f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735b7fe700 1 -- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f735c06c860 msgr2=0x7f735c1f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735b7fe700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f735c06c860 0x7f735c1f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735b7fe700 1 -- 172.21.15.45:0/169813461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735c0cdc60 msgr2=0x7f735c1f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735b7fe700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735c0cdc60 0x7f735c1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735b7fe700 1 -- 172.21.15.45:0/169813461 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735400d050 con 0x7f735c159590 2023-08-27T11:12:39.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f735affd700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f735c06c860 0x7f735c1f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:39.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.128+0000 7f735b7fe700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c159590 0x7f735c0c79e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f735000c490 tx=0x7f7350001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7ffb65dd7700 1 -- 172.21.15.45:0/2379384774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 msgr2=0x7ffb60071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7ffb65dd7700 1 --2- 172.21.15.45:0/2379384774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb60071ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009840 tx=0x7ffb50009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7ffb65dd7700 1 -- 172.21.15.45:0/2379384774 shutdown_connections 2023-08-27T11:12:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7ffb65dd7700 1 --2- 172.21.15.45:0/2379384774 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb60071ac0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7ffb65dd7700 1 --2- 172.21.15.45:0/2379384774 >> v2:172.21.15.135:3300/0 conn(0x7ffb600cf440 0x7ffb60165600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7ffb65dd7700 1 -- 172.21.15.45:0/2379384774 >> 172.21.15.45:0/2379384774 conn(0x7ffb600d3090 msgr2=0x7ffb600d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7f7d8dffb700 1 -- 172.21.15.45:0/3732487939 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7d94011cb0 con 0x7f7d980c9280 2023-08-27T11:12:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9804e620 con 0x7f7d980c9280 2023-08-27T11:12:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7f7d8dffb700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7d7c06a0e0 0x7f7d7c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f7d8dffb700 1 -- 172.21.15.45:0/3732487939 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7d940886e0 con 0x7f7d980c9280 2023-08-27T11:12:39.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f7d9d7e1700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7d7c06a0e0 0x7f7d7c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f662e7fc700 1 -- 172.21.15.45:0/4050079888 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f663401d460 con 0x7f66380ad3f0 2023-08-27T11:12:39.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.129+0000 7f460f210700 1 -- start start 2023-08-27T11:12:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7ffb65dd7700 1 -- 172.21.15.45:0/2379384774 shutdown_connections 2023-08-27T11:12:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f7d9d7e1700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7d7c06a0e0 0x7f7d7c06c5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009a10 tx=0x7f7d88009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f460f210700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46080c9490 0x7f460806c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f460f210700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46080cf720 0x7f460806d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f460f210700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080c9940 con 0x7f4608159590 2023-08-27T11:12:39.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f460f210700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46080ca130 con 0x7f46080c9490 2023-08-27T11:12:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f460f210700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460806cda0 con 0x7f46080cf720 2023-08-27T11:12:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f4607fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46080c9490 0x7f460806c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f4607fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46080c9490 0x7f460806c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52282/0 (socket says 172.21.15.45:52282) 2023-08-27T11:12:39.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7f4607fff700 1 -- 172.21.15.45:0/1031605727 learned_addr learned my addr 172.21.15.45:0/1031605727 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.131+0000 7f662e7fc700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f661c06a130 0x7f661c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.131+0000 7f662e7fc700 1 -- 172.21.15.45:0/4050079888 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f66340894e0 con 0x7f66380ad3f0 2023-08-27T11:12:39.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.131+0000 7f663dd58700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f661c06a130 0x7f661c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.131+0000 7f4607fff700 1 -- 172.21.15.45:0/1031605727 >> v2:172.21.15.135:3300/0 conn(0x7f46080cf720 msgr2=0x7f460806d760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-27T11:12:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.131+0000 7f4607fff700 1 --2- 172.21.15.45:0/1031605727 >> v2:172.21.15.135:3300/0 conn(0x7f46080cf720 0x7f460806d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.131+0000 7f4607fff700 1 -- 172.21.15.45:0/1031605727 >> v1:172.21.15.45:6789/0 conn(0x7f4608159590 legacy=0x7f46080cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.131+0000 7f4607fff700 1 -- 172.21.15.45:0/1031605727 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080c9b70 con 0x7f46080c9490 2023-08-27T11:12:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.131+0000 7f4607fff700 1 --2- 172.21.15.45:0/1031605727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080c9490 0x7f460806c860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f46080ca2c0 tx=0x7f460000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a011bcb55b4b04 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.127+0000 7f1b9a59c700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1b7c06a0e0 0x7f1b7c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.132+0000 7f73437fe700 1 -- 172.21.15.45:0/169813461 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73500054e0 con 0x7f735c159590 2023-08-27T11:12:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.132+0000 7f73437fe700 1 -- 172.21.15.45:0/169813461 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7350003680 con 0x7f735c159590 2023-08-27T11:12:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.132+0000 7f73437fe700 1 -- 172.21.15.45:0/169813461 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350003a30 con 0x7f735c159590 2023-08-27T11:12:39.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.132+0000 7f1b9a59c700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1b7c06a0e0 0x7f1b7c06c5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c001010 tx=0x7f1b8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.132+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c0c65f0 con 0x7f735c159590 2023-08-27T11:12:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.132+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c0c6bb0 con 0x7f735c159590 2023-08-27T11:12:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.130+0000 7ffb65dd7700 1 -- 172.21.15.45:0/2379384774 wait complete. 2023-08-27T11:12:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.132+0000 7ffb65dd7700 1 Processor -- start 2023-08-27T11:12:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.132+0000 7ffb65dd7700 1 -- start start 2023-08-27T11:12:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.133+0000 7f662e7fc700 1 -- 172.21.15.45:0/4050079888 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6634054940 con 0x7f66380ad3f0 2023-08-27T11:12:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c04e620 con 0x7f735c159590 2023-08-27T11:12:39.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7f663dd58700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f661c06a130 0x7f661c06c5f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6628014040 tx=0x7f6628011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb65dd7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb600b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb65dd7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb60165220 0x7ffb60160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb65dd7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb601fa6b0 0x7ffb601fab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb65dd7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60161d40 con 0x7ffb601fa6b0 2023-08-27T11:12:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb65dd7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601fb0a0 con 0x7ffb60165220 2023-08-27T11:12:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb65dd7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60160780 con 0x7ffb60071630 2023-08-27T11:12:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5f7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb600b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5effd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb60165220 0x7ffb60160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5f7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb600b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50196/0 (socket says 172.21.15.45:50196) 2023-08-27T11:12:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5effd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb60165220 0x7ffb60160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41144/0 (socket says 172.21.15.45:41144) 2023-08-27T11:12:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5f7fe700 1 -- 172.21.15.45:0/3258613928 learned_addr learned my addr 172.21.15.45:0/3258613928 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5ffff700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb601fa6b0 0x7ffb601fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5ffff700 1 -- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 msgr2=0x7ffb600b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5ffff700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb600b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5ffff700 1 -- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb60165220 msgr2=0x7ffb60160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.134+0000 7ffb5ffff700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb60165220 0x7ffb60160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.135+0000 7ffb5ffff700 1 -- 172.21.15.45:0/3258613928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500094a0 con 0x7ffb601fa6b0 2023-08-27T11:12:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.135+0000 7ffb5effd700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb60165220 0x7ffb60160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.135+0000 7f7d8dffb700 1 -- 172.21.15.45:0/3732487939 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7d94053a40 con 0x7f7d980c9280 2023-08-27T11:12:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.135+0000 7f46077fe700 1 -- 172.21.15.45:0/1031605727 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460000e070 con 0x7f46080c9490 2023-08-27T11:12:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.135+0000 7f46077fe700 1 -- 172.21.15.45:0/1031605727 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4600003a20 con 0x7f46080c9490 2023-08-27T11:12:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.135+0000 7f46077fe700 1 -- 172.21.15.45:0/1031605727 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600003dd0 con 0x7f46080c9490 2023-08-27T11:12:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.135+0000 7f1b8affd700 1 -- 172.21.15.45:0/714381256 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1b9c0cd860 con 0x7f1b9c165ce0 2023-08-27T11:12:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.136+0000 7f460f210700 1 -- 172.21.15.45:0/1031605727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080c9490 msgr2=0x7f460806c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.136+0000 7f460f210700 1 --2- 172.21.15.45:0/1031605727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080c9490 0x7f460806c860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f46080ca2c0 tx=0x7f460000b330 comp rx=0 tx=0).stop 2023-08-27T11:12:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.135+0000 7ffb5ffff700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb601fa6b0 0x7ffb601fab60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb6006bf70 tx=0x7ffb50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.136+0000 7f460f210700 1 -- 172.21.15.45:0/1031605727 shutdown_connections 2023-08-27T11:12:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.136+0000 7f460f210700 1 --2- 172.21.15.45:0/1031605727 >> v2:172.21.15.135:3300/0 conn(0x7f46080cf720 0x7f460806d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.136+0000 7f460f210700 1 --2- 172.21.15.45:0/1031605727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080c9490 0x7f460806c860 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.137+0000 7f460f210700 1 -- 172.21.15.45:0/1031605727 >> 172.21.15.45:0/1031605727 conn(0x7f46080cec30 msgr2=0x7f46080cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.137+0000 7f460f210700 1 -- 172.21.15.45:0/1031605727 shutdown_connections 2023-08-27T11:12:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.136+0000 7f73437fe700 1 -- 172.21.15.45:0/169813461 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7350003b90 con 0x7f735c159590 2023-08-27T11:12:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.137+0000 7f73437fe700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f734806a130 0x7f734806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.137+0000 7f73437fe700 1 -- 172.21.15.45:0/169813461 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7350088110 con 0x7f735c159590 2023-08-27T11:12:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.137+0000 7f735affd700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f734806a130 0x7f734806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.138+0000 7f735affd700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f734806a130 0x7f734806c5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7354000f80 tx=0x7f7354012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.137+0000 7f460f210700 1 -- 172.21.15.45:0/1031605727 wait complete. 2023-08-27T11:12:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.136+0000 7ffb47fff700 1 -- 172.21.15.45:0/3258613928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500038a0 con 0x7ffb601fa6b0 2023-08-27T11:12:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.137+0000 7ffb47fff700 1 -- 172.21.15.45:0/3258613928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5001f070 con 0x7ffb601fa6b0 2023-08-27T11:12:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.137+0000 7ffb47fff700 1 -- 172.21.15.45:0/3258613928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50003c60 con 0x7ffb601fa6b0 2023-08-27T11:12:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.139+0000 7f460f210700 1 Processor -- start 2023-08-27T11:12:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.140+0000 7f73437fe700 1 -- 172.21.15.45:0/169813461 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f73500535f0 con 0x7f735c159590 2023-08-27T11:12:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.139+0000 7f460f210700 1 -- start start 2023-08-27T11:12:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.140+0000 7f460f210700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46080c9490 0x7f46080c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.140+0000 7f460f210700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080cf720 0x7f460815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.140+0000 7f460f210700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4608159590 0x7f46080cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.140+0000 7f460f210700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460815c8e0 con 0x7f46080cf720 2023-08-27T11:12:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.140+0000 7f460f210700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4608165e20 con 0x7f46080c9490 2023-08-27T11:12:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.140+0000 7f460f210700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46080c8070 con 0x7f4608159590 2023-08-27T11:12:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f460cfac700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46080c9490 0x7f46080c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f460cfac700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46080c9490 0x7f46080c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41150/0 (socket says 172.21.15.45:41150) 2023-08-27T11:12:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f460cfac700 1 -- 172.21.15.45:0/942400081 learned_addr learned my addr 172.21.15.45:0/942400081 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f4607fff700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080cf720 0x7f460815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f460d7ad700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4608159590 0x7f46080cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f4607fff700 1 -- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4608159590 msgr2=0x7f46080cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f4607fff700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4608159590 0x7f46080cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f4607fff700 1 -- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46080c9490 msgr2=0x7f46080c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f4607fff700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46080c9490 0x7f46080c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f4607fff700 1 -- 172.21.15.45:0/942400081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4600009b30 con 0x7f46080cf720 2023-08-27T11:12:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.141+0000 7f460cfac700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46080c9490 0x7f46080c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:39.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.142+0000 7f4607fff700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080cf720 0x7f460815c3a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4600014000 tx=0x7f4600004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.143+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb600703a0 con 0x7ffb601fa6b0 2023-08-27T11:12:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.143+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb60070960 con 0x7ffb601fa6b0 2023-08-27T11:12:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.143+0000 7f46057fa700 1 -- 172.21.15.45:0/942400081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600004440 con 0x7f46080cf720 2023-08-27T11:12:39.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.143+0000 7f46057fa700 1 -- 172.21.15.45:0/942400081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4600004600 con 0x7f46080cf720 2023-08-27T11:12:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.143+0000 7f46057fa700 1 -- 172.21.15.45:0/942400081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600012430 con 0x7f46080cf720 2023-08-27T11:12:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.143+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46080c8350 con 0x7f46080cf720 2023-08-27T11:12:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.143+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46080cbd50 con 0x7f46080cf720 2023-08-27T11:12:39.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.145+0000 7ffb47fff700 1 -- 172.21.15.45:0/3258613928 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffb50026480 con 0x7ffb601fa6b0 2023-08-27T11:12:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.146+0000 7ffb47fff700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb4c06a1b0 0x7ffb4c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.146+0000 7ffb47fff700 1 -- 172.21.15.45:0/3258613928 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffb50021070 con 0x7ffb601fa6b0 2023-08-27T11:12:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.146+0000 7f46057fa700 1 -- 172.21.15.45:0/942400081 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4600012590 con 0x7f46080cf720 2023-08-27T11:12:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.147+0000 7ffb5f7fe700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb4c06a1b0 0x7ffb4c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.147+0000 7f46057fa700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f45f406a130 0x7f45f406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.147+0000 7f46057fa700 1 -- 172.21.15.45:0/942400081 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4600089fe0 con 0x7f46080cf720 2023-08-27T11:12:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.147+0000 7f460cfac700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f45f406a130 0x7f45f406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.147+0000 7ffb5f7fe700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb4c06a1b0 0x7ffb4c06c670 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005910 tx=0x7ffb54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.148+0000 7f460cfac700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f45f406a130 0x7f45f406c5f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f45f8006fd0 tx=0x7f45f8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.151+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb60161e80 con 0x7ffb601fa6b0 2023-08-27T11:12:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.162+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e80052f0 con 0x7f46080cf720 2023-08-27T11:12:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.162+0000 7ffb47fff700 1 -- 172.21.15.45:0/3258613928 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffb60161e80 con 0x7ffb601fa6b0 2023-08-27T11:12:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.172+0000 7f46057fa700 1 -- 172.21.15.45:0/942400081 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f46000555b0 con 0x7f46080cf720 2023-08-27T11:12:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.496+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fcd2804e620 con 0x7fcd2806fd20 2023-08-27T11:12:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.496+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff46004e620 con 0x7ff46006fd20 2023-08-27T11:12:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.497+0000 7fcd24ff9700 1 -- 172.21.15.45:0/760461375 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcd1c057330 con 0x7fcd2806fd20 2023-08-27T11:12:39.500 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.498+0000 7ff455ffb700 1 -- 172.21.15.45:0/3440166837 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff458054340 con 0x7ff46006fd20 2023-08-27T11:12:39.501 INFO:teuthology.orchestra.run.smithi045.stdout:55834574861 2023-08-27T11:12:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcd1406a0e0 msgr2=0x7fcd1406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7fcd2e286700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcd1406a0e0 0x7fcd1406c5a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcd28153790 tx=0x7fcd18006c60 comp rx=0 tx=0).stop 2023-08-27T11:12:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 msgr2=0x7fcd281f6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7fcd2e286700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 0x7fcd281f6f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00b470 tx=0x7fcd1c00b830 comp rx=0 tx=0).stop 2023-08-27T11:12:39.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff44c06a260 msgr2=0x7ff44c06c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff44c06a260 0x7ff44c06c720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff45c007b10 tx=0x7ff45c008040 comp rx=0 tx=0).stop 2023-08-27T11:12:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 msgr2=0x7ff460157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 0x7ff460157070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff45800cb90 tx=0x7ff45800cea0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 shutdown_connections 2023-08-27T11:12:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff46006c860 0x7ff460154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff44c06a260 0x7ff44c06c720 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff460159590 0x7ff4601575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 --2- 172.21.15.45:0/3440166837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff46006fd20 0x7ff460157070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 >> 172.21.15.45:0/3440166837 conn(0x7ff4600cec30 msgr2=0x7ff460071580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.506+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 shutdown_connections 2023-08-27T11:12:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.507+0000 7ff467959700 1 -- 172.21.15.45:0/3440166837 wait complete. 2023-08-27T11:12:39.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.508+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 shutdown_connections 2023-08-27T11:12:39.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.508+0000 7fcd2e286700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd2806fd20 0x7fcd281f6f20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.508+0000 7fcd2e286700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcd1406a0e0 0x7fcd1406c5a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.508+0000 7fcd2e286700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcd28159590 0x7fcd2815c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.508+0000 7fcd2e286700 1 --2- 172.21.15.45:0/760461375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd280c9490 0x7fcd28153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.508+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 >> 172.21.15.45:0/760461375 conn(0x7fcd280cec30 msgr2=0x7fcd28071540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.508+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 shutdown_connections 2023-08-27T11:12:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.508+0000 7fcd2e286700 1 -- 172.21.15.45:0/760461375 wait complete. 2023-08-27T11:12:39.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2023-08-27T11:12:39.531 DEBUG:teuthology.parallel:result is None 2023-08-27T11:12:39.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574861 for osd.3 2023-08-27T11:12:39.536 DEBUG:teuthology.parallel:result is None 2023-08-27T11:12:39.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.542+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f735c0c6450 con 0x7f735c159590 2023-08-27T11:12:39.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.543+0000 7f73437fe700 1 -- 172.21.15.45:0/169813461 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7350017020 con 0x7f735c159590 2023-08-27T11:12:39.545 INFO:teuthology.orchestra.run.smithi045.stdout:55834574858 2023-08-27T11:12:39.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.545+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f734806a130 msgr2=0x7f734806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.545+0000 7f7361555700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f734806a130 0x7f734806c5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7354000f80 tx=0x7f7354012000 comp rx=0 tx=0).stop 2023-08-27T11:12:39.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.545+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c159590 msgr2=0x7f735c0c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.545+0000 7f7361555700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c159590 0x7f735c0c79e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f735000c490 tx=0x7f7350001e70 comp rx=0 tx=0).stop 2023-08-27T11:12:39.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 shutdown_connections 2023-08-27T11:12:39.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7361555700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f734806a130 0x7f734806c5f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7361555700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f735c159590 0x7f735c0c79e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7361555700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f735c06c860 0x7f735c1f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7361555700 1 --2- 172.21.15.45:0/169813461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f735c0cdc60 0x7f735c1f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 >> 172.21.15.45:0/169813461 conn(0x7f735c0cec30 msgr2=0x7f735c071370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 shutdown_connections 2023-08-27T11:12:39.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7361555700 1 -- 172.21.15.45:0/169813461 wait complete. 2023-08-27T11:12:39.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7d9816c6b0 con 0x7f7d980c9280 2023-08-27T11:12:39.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.546+0000 7f7d8dffb700 1 -- 172.21.15.45:0/3732487939 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d94057060 con 0x7f7d980c9280 2023-08-27T11:12:39.551 INFO:teuthology.orchestra.run.smithi045.stdout:55834574860 2023-08-27T11:12:39.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.550+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f66380734a0 con 0x7f66380ad3f0 2023-08-27T11:12:39.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.550+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7d7c06a0e0 msgr2=0x7f7d7c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.550+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7d7c06a0e0 0x7f7d7c06c5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009a10 tx=0x7f7d88009190 comp rx=0 tx=0).stop 2023-08-27T11:12:39.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.550+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 msgr2=0x7f7d980c3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.550+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c3230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b410 tx=0x7f7d94009f90 comp rx=0 tx=0).stop 2023-08-27T11:12:39.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.550+0000 7f662e7fc700 1 -- 172.21.15.45:0/4050079888 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6634057f60 con 0x7f66380ad3f0 2023-08-27T11:12:39.554 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:39.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 shutdown_connections 2023-08-27T11:12:39.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d9815c070 0x7f7d980c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7d7c06a0e0 0x7f7d7c06c5a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d9806d8b0 0x7f7d98070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7f7d9fa45700 1 --2- 172.21.15.45:0/3732487939 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d980c9280 0x7f7d980c3230 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 >> 172.21.15.45:0/3732487939 conn(0x7f7d980cec30 msgr2=0x7f7d9814e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ffb60161e80 con 0x7ffb601fa6b0 2023-08-27T11:12:39.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 shutdown_connections 2023-08-27T11:12:39.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7f7d9fa45700 1 -- 172.21.15.45:0/3732487939 wait complete. 2023-08-27T11:12:39.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.553+0000 7ffb47fff700 1 -- 172.21.15.45:0/3258613928 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb60161e80 con 0x7ffb601fa6b0 2023-08-27T11:12:39.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f661c06a130 msgr2=0x7f661c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f661c06a130 0x7f661c06c5f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6628014040 tx=0x7f6628011000 comp rx=0 tx=0).stop 2023-08-27T11:12:39.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66380ad3f0 msgr2=0x7f66381f1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66380ad3f0 0x7f66381f1a60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6634007400 tx=0x7f6634007710 comp rx=0 tx=0).stop 2023-08-27T11:12:39.559 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:39.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 shutdown_connections 2023-08-27T11:12:39.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6624004bf0 0x7f66380c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f661c06a130 0x7f661c06c5f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66380ad870 0x7f66381f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 --2- 172.21.15.45:0/4050079888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66380ad3f0 0x7f66381f1a60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 >> 172.21.15.45:0/4050079888 conn(0x7f66380b2440 msgr2=0x7f663806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 shutdown_connections 2023-08-27T11:12:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.554+0000 7f663ffbc700 1 -- 172.21.15.45:0/4050079888 wait complete. 2023-08-27T11:12:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.557+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb4c06a1b0 msgr2=0x7ffb4c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.557+0000 7ffb65dd7700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb4c06a1b0 0x7ffb4c06c670 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005910 tx=0x7ffb54009040 comp rx=0 tx=0).stop 2023-08-27T11:12:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.557+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb601fa6b0 msgr2=0x7ffb601fab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.557+0000 7ffb65dd7700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb601fa6b0 0x7ffb601fab60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb6006bf70 tx=0x7ffb50004750 comp rx=0 tx=0).stop 2023-08-27T11:12:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.558+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1b9c0b35e0 con 0x7f1b9c165ce0 2023-08-27T11:12:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.559+0000 7f1b8affd700 1 -- 172.21.15.45:0/714381256 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b9c0b35e0 con 0x7f1b9c165ce0 2023-08-27T11:12:39.564 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.560+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 shutdown_connections 2023-08-27T11:12:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.560+0000 7ffb65dd7700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb601fa6b0 0x7ffb601fab60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.560+0000 7ffb65dd7700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb4c06a1b0 0x7ffb4c06c670 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.560+0000 7ffb65dd7700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb60071630 0x7ffb600b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.560+0000 7ffb65dd7700 1 --2- 172.21.15.45:0/3258613928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb60165220 0x7ffb60160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.560+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 >> 172.21.15.45:0/3258613928 conn(0x7ffb600d3090 msgr2=0x7ffb60160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.560+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 shutdown_connections 2023-08-27T11:12:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.560+0000 7ffb65dd7700 1 -- 172.21.15.45:0/3258613928 wait complete. 2023-08-27T11:12:39.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2023-08-27T11:12:39.567 DEBUG:teuthology.parallel:result is None 2023-08-27T11:12:39.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1b7c06a0e0 msgr2=0x7f1b7c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1b7c06a0e0 0x7f1b7c06c5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c001010 tx=0x7f1b8c009040 comp rx=0 tx=0).stop 2023-08-27T11:12:39.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b9c165ce0 msgr2=0x7f1b9c1f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b9c165ce0 0x7f1b9c1f5ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c4d0 tx=0x7f1b9000c7e0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 shutdown_connections 2023-08-27T11:12:39.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b9c165ce0 0x7f1b9c1f5ad0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1b7c06a0e0 0x7f1b7c06c5a0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c001010 tx=0x7f1b8c009040 comp rx=0 tx=0).stop 2023-08-27T11:12:39.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b9c06a7c0 0x7f1b9c158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 --2- 172.21.15.45:0/714381256 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1b9c164b70 0x7f1b9c1f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.562+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 >> 172.21.15.45:0/714381256 conn(0x7f1b9c0cc7c0 msgr2=0x7f1b9c06e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.564+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 shutdown_connections 2023-08-27T11:12:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.564+0000 7f1ba0b30700 1 -- 172.21.15.45:0/714381256 wait complete. 2023-08-27T11:12:39.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2023-08-27T11:12:39.575 DEBUG:teuthology.parallel:result is None 2023-08-27T11:12:39.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.2 2023-08-27T11:12:39.576 DEBUG:teuthology.parallel:result is None 2023-08-27T11:12:39.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2023-08-27T11:12:39.579 DEBUG:teuthology.parallel:result is None 2023-08-27T11:12:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.583+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f45e8005870 con 0x7f46080cf720 2023-08-27T11:12:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.584+0000 7f46057fa700 1 -- 172.21.15.45:0/942400081 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4600017070 con 0x7f46080cf720 2023-08-27T11:12:39.586 INFO:teuthology.orchestra.run.smithi045.stdout:55834574859 2023-08-27T11:12:39.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2023-08-27T11:12:39.587 DEBUG:teuthology.parallel:result is None 2023-08-27T11:12:39.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f45f406a130 msgr2=0x7f45f406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f45f406a130 0x7f45f406c5f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f45f8006fd0 tx=0x7f45f8009190 comp rx=0 tx=0).stop 2023-08-27T11:12:39.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080cf720 msgr2=0x7f460815c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080cf720 0x7f460815c3a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4600014000 tx=0x7f4600004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 shutdown_connections 2023-08-27T11:12:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f45f406a130 0x7f45f406c5f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4608159590 0x7f46080cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46080c9490 0x7f46080c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 --2- 172.21.15.45:0/942400081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46080cf720 0x7f460815c3a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 >> 172.21.15.45:0/942400081 conn(0x7f46080cec30 msgr2=0x7f46080affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 shutdown_connections 2023-08-27T11:12:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.586+0000 7f460f210700 1 -- 172.21.15.45:0/942400081 wait complete. 2023-08-27T11:12:39.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2023-08-27T11:12:39.600 DEBUG:teuthology.parallel:result is None 2023-08-27T11:12:39.600 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-27T11:12:39.601 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:12:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.762+0000 7fbd4c69d700 1 Processor -- start 2023-08-27T11:12:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.762+0000 7fbd4c69d700 1 -- start start 2023-08-27T11:12:39.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd4c69d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4415b070 0x7fbd440cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd4c69d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd440cdf60 0x7fbd4406e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd4c69d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4415dad0 con 0x7fbd4406cec0 2023-08-27T11:12:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd4c69d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd4415d8e0 con 0x7fbd440cdf60 2023-08-27T11:12:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd4c69d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4415ddb0 con 0x7fbd4415b070 2023-08-27T11:12:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd49c38700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd440cdf60 0x7fbd4406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd4ac3a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbd4406cec0 0x7fbd440ae8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39084/0 (socket says 172.21.15.45:39084) 2023-08-27T11:12:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd4ac3a700 1 -- 172.21.15.45:0/3810352186 learned_addr learned my addr 172.21.15.45:0/3810352186 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd4a439700 1 --2- 172.21.15.45:0/3810352186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4415b070 0x7fbd440cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd49c38700 1 -- 172.21.15.45:0/3810352186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4415b070 msgr2=0x7fbd440cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd49c38700 1 --2- 172.21.15.45:0/3810352186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4415b070 0x7fbd440cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd49c38700 1 -- 172.21.15.45:0/3810352186 >> v1:172.21.15.45:6789/0 conn(0x7fbd4406cec0 legacy=0x7fbd440ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd49c38700 1 -- 172.21.15.45:0/3810352186 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4415e090 con 0x7fbd440cdf60 2023-08-27T11:12:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.763+0000 7fbd49437700 1 -- 172.21.15.45:0/3810352186 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152103020 0 0) 0x7fbd4415dad0 con 0x7fbd4406cec0 2023-08-27T11:12:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd49c38700 1 --2- 172.21.15.45:0/3810352186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 0x7fbd4406e270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009840 tx=0x7fbd40009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8544729e0c724fea server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd49437700 1 -- 172.21.15.45:0/3810352186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40011830 con 0x7fbd440cdf60 2023-08-27T11:12:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd49437700 1 -- 172.21.15.45:0/3810352186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd40011990 con 0x7fbd440cdf60 2023-08-27T11:12:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd49437700 1 -- 172.21.15.45:0/3810352186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40011d20 con 0x7fbd440cdf60 2023-08-27T11:12:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd4c69d700 1 -- 172.21.15.45:0/3810352186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 msgr2=0x7fbd4406e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd4c69d700 1 --2- 172.21.15.45:0/3810352186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 0x7fbd4406e270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009840 tx=0x7fbd40009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd4c69d700 1 -- 172.21.15.45:0/3810352186 shutdown_connections 2023-08-27T11:12:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd4c69d700 1 --2- 172.21.15.45:0/3810352186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4415b070 0x7fbd440cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd4c69d700 1 --2- 172.21.15.45:0/3810352186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 0x7fbd4406e270 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd4c69d700 1 -- 172.21.15.45:0/3810352186 >> 172.21.15.45:0/3810352186 conn(0x7fbd440cfe50 msgr2=0x7fbd440d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd4c69d700 1 -- 172.21.15.45:0/3810352186 shutdown_connections 2023-08-27T11:12:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.764+0000 7fbd4c69d700 1 -- 172.21.15.45:0/3810352186 wait complete. 2023-08-27T11:12:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.766+0000 7fbd4c69d700 1 Processor -- start 2023-08-27T11:12:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.766+0000 7fbd4c69d700 1 -- start start 2023-08-27T11:12:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4c69d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4406cec0 0x7fbd441f0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4c69d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 0x7fbd440c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4c69d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4415b070 0x7fbd440c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4c69d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd441f0650 con 0x7fbd440cdf60 2023-08-27T11:12:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4c69d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd441e8a50 con 0x7fbd4415b070 2023-08-27T11:12:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4c69d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd441e8d30 con 0x7fbd4406cec0 2023-08-27T11:12:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd49c38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 0x7fbd440c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd49c38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 0x7fbd440c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52312/0 (socket says 172.21.15.45:52312) 2023-08-27T11:12:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd49c38700 1 -- 172.21.15.45:0/1910920983 learned_addr learned my addr 172.21.15.45:0/1910920983 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4a439700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4406cec0 0x7fbd441f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4ac3a700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4415b070 0x7fbd440c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd49c38700 1 -- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4406cec0 msgr2=0x7fbd441f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd49c38700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4406cec0 0x7fbd441f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd49c38700 1 -- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4415b070 msgr2=0x7fbd440c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd49c38700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4415b070 0x7fbd440c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd49c38700 1 -- 172.21.15.45:0/1910920983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd400094a0 con 0x7fbd440cdf60 2023-08-27T11:12:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4a439700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4406cec0 0x7fbd441f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.767+0000 7fbd4ac3a700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4415b070 0x7fbd440c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.768+0000 7fbd49c38700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 0x7fbd440c6000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009fd0 tx=0x7fbd40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.768+0000 7fbd3affd700 1 -- 172.21.15.45:0/1910920983 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd400038a0 con 0x7fbd440cdf60 2023-08-27T11:12:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.768+0000 7fbd3affd700 1 -- 172.21.15.45:0/1910920983 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4001f070 con 0x7fbd440cdf60 2023-08-27T11:12:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.768+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd441e9010 con 0x7fbd440cdf60 2023-08-27T11:12:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.768+0000 7fbd3affd700 1 -- 172.21.15.45:0/1910920983 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40003c40 con 0x7fbd440cdf60 2023-08-27T11:12:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.768+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd441e95d0 con 0x7fbd440cdf60 2023-08-27T11:12:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.769+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd440cf8d0 con 0x7fbd440cdf60 2023-08-27T11:12:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.769+0000 7fbd3affd700 1 -- 172.21.15.45:0/1910920983 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbd40026480 con 0x7fbd440cdf60 2023-08-27T11:12:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.769+0000 7fbd3affd700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd2806a0e0 0x7fbd2806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.770+0000 7fbd3affd700 1 -- 172.21.15.45:0/1910920983 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbd40021070 con 0x7fbd440cdf60 2023-08-27T11:12:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.770+0000 7fbd4a439700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd2806a0e0 0x7fbd2806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.770+0000 7fbd4a439700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd2806a0e0 0x7fbd2806c5a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009ac0 tx=0x7fbd34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.774+0000 7fbd3affd700 1 -- 172.21.15.45:0/1910920983 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbd400bc990 con 0x7fbd440cdf60 2023-08-27T11:12:39.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.972+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 --> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd4415e650 con 0x7fbd2806a0e0 2023-08-27T11:12:39.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.982+0000 7fbd3affd700 1 -- 172.21.15.45:0/1910920983 <== mgr.4100 v2:172.21.15.135:6832/4120550306 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367365 (secure 0 0 0) 0x7fbd34004030 con 0x7fbd2806a0e0 2023-08-27T11:12:39.988 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:39.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd2806a0e0 msgr2=0x7fbd2806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd2806a0e0 0x7fbd2806c5a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009ac0 tx=0x7fbd34008040 comp rx=0 tx=0).stop 2023-08-27T11:12:39.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 msgr2=0x7fbd440c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:39.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 0x7fbd440c6000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009fd0 tx=0x7fbd40004750 comp rx=0 tx=0).stop 2023-08-27T11:12:39.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 shutdown_connections 2023-08-27T11:12:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd2806a0e0 0x7fbd2806c5a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd4415b070 0x7fbd440c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbd4406cec0 0x7fbd441f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 --2- 172.21.15.45:0/1910920983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd440cdf60 0x7fbd440c6000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 >> 172.21.15.45:0/1910920983 conn(0x7fbd440cfe50 msgr2=0x7fbd4415baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:39.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 shutdown_connections 2023-08-27T11:12:39.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:39.985+0000 7fbd4c69d700 1 -- 172.21.15.45:0/1910920983 wait complete. 2023-08-27T11:12:39.993 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:12:39.999 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-08-27T11:12:38.812510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219664,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755056,"statfs":{"total":773094113280,"available":772869177344,"internally_reserved":0,"allocated":3579904,"data_stored":2149224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003845"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932872+0000","last_change":"2023-08-27T11:12:08.913049+0000","last_active":"2023-08-27T11:12:13.932872+0000","last_peered":"2023-08-27T11:12:13.932872+0000","last_clean":"2023-08-27T11:12:13.932872+0000","last_became_active":"2023-08-27T11:12:08.912757+0000","last_became_peered":"2023-08-27T11:12:08.912757+0000","last_unstale":"2023-08-27T11:12:13.932872+0000","last_undegraded":"2023-08-27T11:12:13.932872+0000","last_fullsized":"2023-08-27T11:12:13.932872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:00.165033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967658+0000","last_change":"2023-08-27T11:12:08.918994+0000","last_active":"2023-08-27T11:12:14.967658+0000","last_peered":"2023-08-27T11:12:14.967658+0000","last_clean":"2023-08-27T11:12:14.967658+0000","last_became_active":"2023-08-27T11:12:08.918683+0000","last_became_peered":"2023-08-27T11:12:08.918683+0000","last_unstale":"2023-08-27T11:12:14.967658+0000","last_undegraded":"2023-08-27T11:12:14.967658+0000","last_fullsized":"2023-08-27T11:12:14.967658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:40.545799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934025+0000","last_change":"2023-08-27T11:12:08.911497+0000","last_active":"2023-08-27T11:12:13.934025+0000","last_peered":"2023-08-27T11:12:13.934025+0000","last_clean":"2023-08-27T11:12:13.934025+0000","last_became_active":"2023-08-27T11:12:08.911184+0000","last_became_peered":"2023-08-27T11:12:08.911184+0000","last_unstale":"2023-08-27T11:12:13.934025+0000","last_undegraded":"2023-08-27T11:12:13.934025+0000","last_fullsized":"2023-08-27T11:12:13.934025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:45.042697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934063+0000","last_change":"2023-08-27T11:12:08.914949+0000","last_active":"2023-08-27T11:12:13.934063+0000","last_peered":"2023-08-27T11:12:13.934063+0000","last_clean":"2023-08-27T11:12:13.934063+0000","last_became_active":"2023-08-27T11:12:08.914642+0000","last_became_peered":"2023-08-27T11:12:08.914642+0000","last_unstale":"2023-08-27T11:12:13.934063+0000","last_undegraded":"2023-08-27T11:12:13.934063+0000","last_fullsized":"2023-08-27T11:12:13.934063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:38.888108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726727+0000","last_change":"2023-08-27T11:12:08.929257+0000","last_active":"2023-08-27T11:12:14.726727+0000","last_peered":"2023-08-27T11:12:14.726727+0000","last_clean":"2023-08-27T11:12:14.726727+0000","last_became_active":"2023-08-27T11:12:08.928926+0000","last_became_peered":"2023-08-27T11:12:08.928926+0000","last_unstale":"2023-08-27T11:12:14.726727+0000","last_undegraded":"2023-08-27T11:12:14.726727+0000","last_fullsized":"2023-08-27T11:12:14.726727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:09.445014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934838+0000","last_change":"2023-08-27T11:12:08.916024+0000","last_active":"2023-08-27T11:12:13.934838+0000","last_peered":"2023-08-27T11:12:13.934838+0000","last_clean":"2023-08-27T11:12:13.934838+0000","last_became_active":"2023-08-27T11:12:08.913502+0000","last_became_peered":"2023-08-27T11:12:08.913502+0000","last_unstale":"2023-08-27T11:12:13.934838+0000","last_undegraded":"2023-08-27T11:12:13.934838+0000","last_fullsized":"2023-08-27T11:12:13.934838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:50:51.397518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384498+0000","last_change":"2023-08-27T11:12:08.925080+0000","last_active":"2023-08-27T11:12:14.384498+0000","last_peered":"2023-08-27T11:12:14.384498+0000","last_clean":"2023-08-27T11:12:14.384498+0000","last_became_active":"2023-08-27T11:12:08.924784+0000","last_became_peered":"2023-08-27T11:12:08.924784+0000","last_unstale":"2023-08-27T11:12:14.384498+0000","last_undegraded":"2023-08-27T11:12:14.384498+0000","last_fullsized":"2023-08-27T11:12:14.384498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967188+0000","last_change":"2023-08-27T11:12:08.917218+0000","last_active":"2023-08-27T11:12:14.967188+0000","last_peered":"2023-08-27T11:12:14.967188+0000","last_clean":"2023-08-27T11:12:14.967188+0000","last_became_active":"2023-08-27T11:12:08.916984+0000","last_became_peered":"2023-08-27T11:12:08.916984+0000","last_unstale":"2023-08-27T11:12:14.967188+0000","last_undegraded":"2023-08-27T11:12:14.967188+0000","last_fullsized":"2023-08-27T11:12:14.967188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:02.783147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726887+0000","last_change":"2023-08-27T11:12:08.918120+0000","last_active":"2023-08-27T11:12:14.726887+0000","last_peered":"2023-08-27T11:12:14.726887+0000","last_clean":"2023-08-27T11:12:14.726887+0000","last_became_active":"2023-08-27T11:12:08.917811+0000","last_became_peered":"2023-08-27T11:12:08.917811+0000","last_unstale":"2023-08-27T11:12:14.726887+0000","last_undegraded":"2023-08-27T11:12:14.726887+0000","last_fullsized":"2023-08-27T11:12:14.726887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:16.000827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967968+0000","last_change":"2023-08-27T11:12:08.912850+0000","last_active":"2023-08-27T11:12:14.967968+0000","last_peered":"2023-08-27T11:12:14.967968+0000","last_clean":"2023-08-27T11:12:14.967968+0000","last_became_active":"2023-08-27T11:12:08.912458+0000","last_became_peered":"2023-08-27T11:12:08.912458+0000","last_unstale":"2023-08-27T11:12:14.967968+0000","last_undegraded":"2023-08-27T11:12:14.967968+0000","last_fullsized":"2023-08-27T11:12:14.967968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:49.762265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933954+0000","last_change":"2023-08-27T11:12:08.922288+0000","last_active":"2023-08-27T11:12:13.933954+0000","last_peered":"2023-08-27T11:12:13.933954+0000","last_clean":"2023-08-27T11:12:13.933954+0000","last_became_active":"2023-08-27T11:12:08.921531+0000","last_became_peered":"2023-08-27T11:12:08.921531+0000","last_unstale":"2023-08-27T11:12:13.933954+0000","last_undegraded":"2023-08-27T11:12:13.933954+0000","last_fullsized":"2023-08-27T11:12:13.933954+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:20.157877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933160+0000","last_change":"2023-08-27T11:12:08.911702+0000","last_active":"2023-08-27T11:12:13.933160+0000","last_peered":"2023-08-27T11:12:13.933160+0000","last_clean":"2023-08-27T11:12:13.933160+0000","last_became_active":"2023-08-27T11:12:08.911391+0000","last_became_peered":"2023-08-27T11:12:08.911391+0000","last_unstale":"2023-08-27T11:12:13.933160+0000","last_undegraded":"2023-08-27T11:12:13.933160+0000","last_fullsized":"2023-08-27T11:12:13.933160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:53.589324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932909+0000","last_change":"2023-08-27T11:12:08.914165+0000","last_active":"2023-08-27T11:12:13.932909+0000","last_peered":"2023-08-27T11:12:13.932909+0000","last_clean":"2023-08-27T11:12:13.932909+0000","last_became_active":"2023-08-27T11:12:08.913886+0000","last_became_peered":"2023-08-27T11:12:08.913886+0000","last_unstale":"2023-08-27T11:12:13.932909+0000","last_undegraded":"2023-08-27T11:12:13.932909+0000","last_fullsized":"2023-08-27T11:12:13.932909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:12.238621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967356+0000","last_change":"2023-08-27T11:12:08.911899+0000","last_active":"2023-08-27T11:12:14.967356+0000","last_peered":"2023-08-27T11:12:14.967356+0000","last_clean":"2023-08-27T11:12:14.967356+0000","last_became_active":"2023-08-27T11:12:08.911620+0000","last_became_peered":"2023-08-27T11:12:08.911620+0000","last_unstale":"2023-08-27T11:12:14.967356+0000","last_undegraded":"2023-08-27T11:12:14.967356+0000","last_fullsized":"2023-08-27T11:12:14.967356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:07.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384118+0000","last_change":"2023-08-27T11:12:08.925181+0000","last_active":"2023-08-27T11:12:14.384118+0000","last_peered":"2023-08-27T11:12:14.384118+0000","last_clean":"2023-08-27T11:12:14.384118+0000","last_became_active":"2023-08-27T11:12:08.924909+0000","last_became_peered":"2023-08-27T11:12:08.924909+0000","last_unstale":"2023-08-27T11:12:14.384118+0000","last_undegraded":"2023-08-27T11:12:14.384118+0000","last_fullsized":"2023-08-27T11:12:14.384118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:10.885579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934515+0000","last_change":"2023-08-27T11:12:08.911662+0000","last_active":"2023-08-27T11:12:13.934515+0000","last_peered":"2023-08-27T11:12:13.934515+0000","last_clean":"2023-08-27T11:12:13.934515+0000","last_became_active":"2023-08-27T11:12:08.911345+0000","last_became_peered":"2023-08-27T11:12:08.911345+0000","last_unstale":"2023-08-27T11:12:13.934515+0000","last_undegraded":"2023-08-27T11:12:13.934515+0000","last_fullsized":"2023-08-27T11:12:13.934515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:57.932637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726927+0000","last_change":"2023-08-27T11:12:08.916813+0000","last_active":"2023-08-27T11:12:14.726927+0000","last_peered":"2023-08-27T11:12:14.726927+0000","last_clean":"2023-08-27T11:12:14.726927+0000","last_became_active":"2023-08-27T11:12:08.916478+0000","last_became_peered":"2023-08-27T11:12:08.916478+0000","last_unstale":"2023-08-27T11:12:14.726927+0000","last_undegraded":"2023-08-27T11:12:14.726927+0000","last_fullsized":"2023-08-27T11:12:14.726927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:29.674355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933703+0000","last_change":"2023-08-27T11:12:08.907326+0000","last_active":"2023-08-27T11:12:13.933703+0000","last_peered":"2023-08-27T11:12:13.933703+0000","last_clean":"2023-08-27T11:12:13.933703+0000","last_became_active":"2023-08-27T11:12:08.907047+0000","last_became_peered":"2023-08-27T11:12:08.907047+0000","last_unstale":"2023-08-27T11:12:13.933703+0000","last_undegraded":"2023-08-27T11:12:13.933703+0000","last_fullsized":"2023-08-27T11:12:13.933703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:32:17.983727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932728+0000","last_change":"2023-08-27T11:12:08.919764+0000","last_active":"2023-08-27T11:12:13.932728+0000","last_peered":"2023-08-27T11:12:13.932728+0000","last_clean":"2023-08-27T11:12:13.932728+0000","last_became_active":"2023-08-27T11:12:08.914867+0000","last_became_peered":"2023-08-27T11:12:08.914867+0000","last_unstale":"2023-08-27T11:12:13.932728+0000","last_undegraded":"2023-08-27T11:12:13.932728+0000","last_fullsized":"2023-08-27T11:12:13.932728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:05.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933919+0000","last_change":"2023-08-27T11:12:08.922111+0000","last_active":"2023-08-27T11:12:13.933919+0000","last_peered":"2023-08-27T11:12:13.933919+0000","last_clean":"2023-08-27T11:12:13.933919+0000","last_became_active":"2023-08-27T11:12:08.917591+0000","last_became_peered":"2023-08-27T11:12:08.917591+0000","last_unstale":"2023-08-27T11:12:13.933919+0000","last_undegraded":"2023-08-27T11:12:13.933919+0000","last_fullsized":"2023-08-27T11:12:13.933919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:59.881615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384562+0000","last_change":"2023-08-27T11:12:08.931238+0000","last_active":"2023-08-27T11:12:14.384562+0000","last_peered":"2023-08-27T11:12:14.384562+0000","last_clean":"2023-08-27T11:12:14.384562+0000","last_became_active":"2023-08-27T11:12:08.930972+0000","last_became_peered":"2023-08-27T11:12:08.930972+0000","last_unstale":"2023-08-27T11:12:14.384562+0000","last_undegraded":"2023-08-27T11:12:14.384562+0000","last_fullsized":"2023-08-27T11:12:14.384562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:41.570266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726461+0000","last_change":"2023-08-27T11:12:08.919835+0000","last_active":"2023-08-27T11:12:14.726461+0000","last_peered":"2023-08-27T11:12:14.726461+0000","last_clean":"2023-08-27T11:12:14.726461+0000","last_became_active":"2023-08-27T11:12:08.919439+0000","last_became_peered":"2023-08-27T11:12:08.919439+0000","last_unstale":"2023-08-27T11:12:14.726461+0000","last_undegraded":"2023-08-27T11:12:14.726461+0000","last_fullsized":"2023-08-27T11:12:14.726461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:25:20.526417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982143+0000","last_change":"2023-08-27T11:12:08.913113+0000","last_active":"2023-08-27T11:12:14.982143+0000","last_peered":"2023-08-27T11:12:14.982143+0000","last_clean":"2023-08-27T11:12:14.982143+0000","last_became_active":"2023-08-27T11:12:08.912792+0000","last_became_peered":"2023-08-27T11:12:08.912792+0000","last_unstale":"2023-08-27T11:12:14.982143+0000","last_undegraded":"2023-08-27T11:12:14.982143+0000","last_fullsized":"2023-08-27T11:12:14.982143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:53.041694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384187+0000","last_change":"2023-08-27T11:12:08.926432+0000","last_active":"2023-08-27T11:12:14.384187+0000","last_peered":"2023-08-27T11:12:14.384187+0000","last_clean":"2023-08-27T11:12:14.384187+0000","last_became_active":"2023-08-27T11:12:08.926137+0000","last_became_peered":"2023-08-27T11:12:08.926137+0000","last_unstale":"2023-08-27T11:12:14.384187+0000","last_undegraded":"2023-08-27T11:12:14.384187+0000","last_fullsized":"2023-08-27T11:12:14.384187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:07.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838528+0000","last_change":"2023-08-27T11:12:08.919071+0000","last_active":"2023-08-27T11:12:14.838528+0000","last_peered":"2023-08-27T11:12:14.838528+0000","last_clean":"2023-08-27T11:12:14.838528+0000","last_became_active":"2023-08-27T11:12:08.918772+0000","last_became_peered":"2023-08-27T11:12:08.918772+0000","last_unstale":"2023-08-27T11:12:14.838528+0000","last_undegraded":"2023-08-27T11:12:14.838528+0000","last_fullsized":"2023-08-27T11:12:14.838528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:48.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838629+0000","last_change":"2023-08-27T11:12:08.911583+0000","last_active":"2023-08-27T11:12:14.838629+0000","last_peered":"2023-08-27T11:12:14.838629+0000","last_clean":"2023-08-27T11:12:14.838629+0000","last_became_active":"2023-08-27T11:12:08.911295+0000","last_became_peered":"2023-08-27T11:12:08.911295+0000","last_unstale":"2023-08-27T11:12:14.838629+0000","last_undegraded":"2023-08-27T11:12:14.838629+0000","last_fullsized":"2023-08-27T11:12:14.838629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:22.748117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967776+0000","last_change":"2023-08-27T11:12:08.931465+0000","last_active":"2023-08-27T11:12:14.967776+0000","last_peered":"2023-08-27T11:12:14.967776+0000","last_clean":"2023-08-27T11:12:14.967776+0000","last_became_active":"2023-08-27T11:12:08.930966+0000","last_became_peered":"2023-08-27T11:12:08.930966+0000","last_unstale":"2023-08-27T11:12:14.967776+0000","last_undegraded":"2023-08-27T11:12:14.967776+0000","last_fullsized":"2023-08-27T11:12:14.967776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:41.570266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934397+0000","last_change":"2023-08-27T11:12:12.959219+0000","last_active":"2023-08-27T11:12:13.934397+0000","last_peered":"2023-08-27T11:12:13.934397+0000","last_clean":"2023-08-27T11:12:13.934397+0000","last_became_active":"2023-08-27T11:12:12.958817+0000","last_became_peered":"2023-08-27T11:12:12.958817+0000","last_unstale":"2023-08-27T11:12:13.934397+0000","last_undegraded":"2023-08-27T11:12:13.934397+0000","last_fullsized":"2023-08-27T11:12:13.934397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:44.882336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933845+0000","last_change":"2023-08-27T11:12:07.903534+0000","last_active":"2023-08-27T11:12:13.933845+0000","last_peered":"2023-08-27T11:12:13.933845+0000","last_clean":"2023-08-27T11:12:13.933845+0000","last_became_active":"2023-08-27T11:12:07.903216+0000","last_became_peered":"2023-08-27T11:12:07.903216+0000","last_unstale":"2023-08-27T11:12:13.933845+0000","last_undegraded":"2023-08-27T11:12:13.933845+0000","last_fullsized":"2023-08-27T11:12:13.933845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:35:15.612642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982446+0000","last_change":"2023-08-27T11:12:08.923471+0000","last_active":"2023-08-27T11:12:14.982446+0000","last_peered":"2023-08-27T11:12:14.982446+0000","last_clean":"2023-08-27T11:12:14.982446+0000","last_became_active":"2023-08-27T11:12:08.918459+0000","last_became_peered":"2023-08-27T11:12:08.918459+0000","last_unstale":"2023-08-27T11:12:14.982446+0000","last_undegraded":"2023-08-27T11:12:14.982446+0000","last_fullsized":"2023-08-27T11:12:14.982446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934474+0000","last_change":"2023-08-27T11:12:12.953626+0000","last_active":"2023-08-27T11:12:13.934474+0000","last_peered":"2023-08-27T11:12:13.934474+0000","last_clean":"2023-08-27T11:12:13.934474+0000","last_became_active":"2023-08-27T11:12:12.953231+0000","last_became_peered":"2023-08-27T11:12:12.953231+0000","last_unstale":"2023-08-27T11:12:13.934474+0000","last_undegraded":"2023-08-27T11:12:13.934474+0000","last_fullsized":"2023-08-27T11:12:13.934474+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:57.631242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837663+0000","last_change":"2023-08-27T11:12:07.895584+0000","last_active":"2023-08-27T11:12:14.837663+0000","last_peered":"2023-08-27T11:12:14.837663+0000","last_clean":"2023-08-27T11:12:14.837663+0000","last_became_active":"2023-08-27T11:12:07.895276+0000","last_became_peered":"2023-08-27T11:12:07.895276+0000","last_unstale":"2023-08-27T11:12:14.837663+0000","last_undegraded":"2023-08-27T11:12:14.837663+0000","last_fullsized":"2023-08-27T11:12:14.837663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:44.245501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726897+0000","last_change":"2023-08-27T11:12:08.928505+0000","last_active":"2023-08-27T11:12:14.726897+0000","last_peered":"2023-08-27T11:12:14.726897+0000","last_clean":"2023-08-27T11:12:14.726897+0000","last_became_active":"2023-08-27T11:12:08.926233+0000","last_became_peered":"2023-08-27T11:12:08.926233+0000","last_unstale":"2023-08-27T11:12:14.726897+0000","last_undegraded":"2023-08-27T11:12:14.726897+0000","last_fullsized":"2023-08-27T11:12:14.726897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:17.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727088+0000","last_change":"2023-08-27T11:12:12.962888+0000","last_active":"2023-08-27T11:12:14.727088+0000","last_peered":"2023-08-27T11:12:14.727088+0000","last_clean":"2023-08-27T11:12:14.727088+0000","last_became_active":"2023-08-27T11:12:12.962501+0000","last_became_peered":"2023-08-27T11:12:12.962501+0000","last_unstale":"2023-08-27T11:12:14.727088+0000","last_undegraded":"2023-08-27T11:12:14.727088+0000","last_fullsized":"2023-08-27T11:12:14.727088+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:06.825066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982310+0000","last_change":"2023-08-27T11:12:07.898500+0000","last_active":"2023-08-27T11:12:14.982310+0000","last_peered":"2023-08-27T11:12:14.982310+0000","last_clean":"2023-08-27T11:12:14.982310+0000","last_became_active":"2023-08-27T11:12:07.898197+0000","last_became_peered":"2023-08-27T11:12:07.898197+0000","last_unstale":"2023-08-27T11:12:14.982310+0000","last_undegraded":"2023-08-27T11:12:14.982310+0000","last_fullsized":"2023-08-27T11:12:14.982310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:39.541198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934586+0000","last_change":"2023-08-27T11:12:08.925037+0000","last_active":"2023-08-27T11:12:13.934586+0000","last_peered":"2023-08-27T11:12:13.934586+0000","last_clean":"2023-08-27T11:12:13.934586+0000","last_became_active":"2023-08-27T11:12:08.924740+0000","last_became_peered":"2023-08-27T11:12:08.924740+0000","last_unstale":"2023-08-27T11:12:13.934586+0000","last_undegraded":"2023-08-27T11:12:13.934586+0000","last_fullsized":"2023-08-27T11:12:13.934586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:21:21.628536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933622+0000","last_change":"2023-08-27T11:12:12.956101+0000","last_active":"2023-08-27T11:12:13.933622+0000","last_peered":"2023-08-27T11:12:13.933622+0000","last_clean":"2023-08-27T11:12:13.933622+0000","last_became_active":"2023-08-27T11:12:12.952220+0000","last_became_peered":"2023-08-27T11:12:12.952220+0000","last_unstale":"2023-08-27T11:12:13.933622+0000","last_undegraded":"2023-08-27T11:12:13.933622+0000","last_fullsized":"2023-08-27T11:12:13.933622+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:16.280540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727033+0000","last_change":"2023-08-27T11:12:07.902491+0000","last_active":"2023-08-27T11:12:14.727033+0000","last_peered":"2023-08-27T11:12:14.727033+0000","last_clean":"2023-08-27T11:12:14.727033+0000","last_became_active":"2023-08-27T11:12:07.902083+0000","last_became_peered":"2023-08-27T11:12:07.902083+0000","last_unstale":"2023-08-27T11:12:14.727033+0000","last_undegraded":"2023-08-27T11:12:14.727033+0000","last_fullsized":"2023-08-27T11:12:14.727033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:19.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982268+0000","last_change":"2023-08-27T11:12:08.918017+0000","last_active":"2023-08-27T11:12:14.982268+0000","last_peered":"2023-08-27T11:12:14.982268+0000","last_clean":"2023-08-27T11:12:14.982268+0000","last_became_active":"2023-08-27T11:12:08.917728+0000","last_became_peered":"2023-08-27T11:12:08.917728+0000","last_unstale":"2023-08-27T11:12:14.982268+0000","last_undegraded":"2023-08-27T11:12:14.982268+0000","last_fullsized":"2023-08-27T11:12:14.982268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:51.458617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384060+0000","last_change":"2023-08-27T11:12:12.958663+0000","last_active":"2023-08-27T11:12:14.384060+0000","last_peered":"2023-08-27T11:12:14.384060+0000","last_clean":"2023-08-27T11:12:14.384060+0000","last_became_active":"2023-08-27T11:12:12.958275+0000","last_became_peered":"2023-08-27T11:12:12.958275+0000","last_unstale":"2023-08-27T11:12:14.384060+0000","last_undegraded":"2023-08-27T11:12:14.384060+0000","last_fullsized":"2023-08-27T11:12:14.384060+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:16.222119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967840+0000","last_change":"2023-08-27T11:12:07.909123+0000","last_active":"2023-08-27T11:12:14.967840+0000","last_peered":"2023-08-27T11:12:14.967840+0000","last_clean":"2023-08-27T11:12:14.967840+0000","last_became_active":"2023-08-27T11:12:07.908788+0000","last_became_peered":"2023-08-27T11:12:07.908788+0000","last_unstale":"2023-08-27T11:12:14.967840+0000","last_undegraded":"2023-08-27T11:12:14.967840+0000","last_fullsized":"2023-08-27T11:12:14.967840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:40.858637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982172+0000","last_change":"2023-08-27T11:12:08.927810+0000","last_active":"2023-08-27T11:12:14.982172+0000","last_peered":"2023-08-27T11:12:14.982172+0000","last_clean":"2023-08-27T11:12:14.982172+0000","last_became_active":"2023-08-27T11:12:08.927231+0000","last_became_peered":"2023-08-27T11:12:08.927231+0000","last_unstale":"2023-08-27T11:12:14.982172+0000","last_undegraded":"2023-08-27T11:12:14.982172+0000","last_fullsized":"2023-08-27T11:12:14.982172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:22.748117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982828+0000","last_change":"2023-08-27T11:12:12.954999+0000","last_active":"2023-08-27T11:12:14.982828+0000","last_peered":"2023-08-27T11:12:14.982828+0000","last_clean":"2023-08-27T11:12:14.982828+0000","last_became_active":"2023-08-27T11:12:12.954572+0000","last_became_peered":"2023-08-27T11:12:12.954572+0000","last_unstale":"2023-08-27T11:12:14.982828+0000","last_undegraded":"2023-08-27T11:12:14.982828+0000","last_fullsized":"2023-08-27T11:12:14.982828+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:38.482531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384847+0000","last_change":"2023-08-27T11:12:07.908859+0000","last_active":"2023-08-27T11:12:14.384847+0000","last_peered":"2023-08-27T11:12:14.384847+0000","last_clean":"2023-08-27T11:12:14.384847+0000","last_became_active":"2023-08-27T11:12:07.908605+0000","last_became_peered":"2023-08-27T11:12:07.908605+0000","last_unstale":"2023-08-27T11:12:14.384847+0000","last_undegraded":"2023-08-27T11:12:14.384847+0000","last_fullsized":"2023-08-27T11:12:14.384847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:20.463561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838554+0000","last_change":"2023-08-27T11:12:08.925960+0000","last_active":"2023-08-27T11:12:14.838554+0000","last_peered":"2023-08-27T11:12:14.838554+0000","last_clean":"2023-08-27T11:12:14.838554+0000","last_became_active":"2023-08-27T11:12:08.925641+0000","last_became_peered":"2023-08-27T11:12:08.925641+0000","last_unstale":"2023-08-27T11:12:14.838554+0000","last_undegraded":"2023-08-27T11:12:14.838554+0000","last_fullsized":"2023-08-27T11:12:14.838554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:53.041694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837918+0000","last_change":"2023-08-27T11:12:12.949220+0000","last_active":"2023-08-27T11:12:14.837918+0000","last_peered":"2023-08-27T11:12:14.837918+0000","last_clean":"2023-08-27T11:12:14.837918+0000","last_became_active":"2023-08-27T11:12:12.948926+0000","last_became_peered":"2023-08-27T11:12:12.948926+0000","last_unstale":"2023-08-27T11:12:14.837918+0000","last_undegraded":"2023-08-27T11:12:14.837918+0000","last_fullsized":"2023-08-27T11:12:14.837918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:20.042745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'10","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:24.935040+0000","last_change":"2023-08-27T11:12:07.908014+0000","last_active":"2023-08-27T11:12:24.935040+0000","last_peered":"2023-08-27T11:12:24.935040+0000","last_clean":"2023-08-27T11:12:24.935040+0000","last_became_active":"2023-08-27T11:12:07.907722+0000","last_became_peered":"2023-08-27T11:12:07.907722+0000","last_unstale":"2023-08-27T11:12:24.935040+0000","last_undegraded":"2023-08-27T11:12:24.935040+0000","last_fullsized":"2023-08-27T11:12:24.935040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:11.313074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933433+0000","last_change":"2023-08-27T11:12:08.923838+0000","last_active":"2023-08-27T11:12:13.933433+0000","last_peered":"2023-08-27T11:12:13.933433+0000","last_clean":"2023-08-27T11:12:13.933433+0000","last_became_active":"2023-08-27T11:12:08.919224+0000","last_became_peered":"2023-08-27T11:12:08.919224+0000","last_unstale":"2023-08-27T11:12:13.933433+0000","last_undegraded":"2023-08-27T11:12:13.933433+0000","last_fullsized":"2023-08-27T11:12:13.933433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:49.585442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967969+0000","last_change":"2023-08-27T11:12:12.953888+0000","last_active":"2023-08-27T11:12:14.967969+0000","last_peered":"2023-08-27T11:12:14.967969+0000","last_clean":"2023-08-27T11:12:14.967969+0000","last_became_active":"2023-08-27T11:12:12.953499+0000","last_became_peered":"2023-08-27T11:12:12.953499+0000","last_unstale":"2023-08-27T11:12:14.967969+0000","last_undegraded":"2023-08-27T11:12:14.967969+0000","last_fullsized":"2023-08-27T11:12:14.967969+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:42.319064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384452+0000","last_change":"2023-08-27T11:12:07.907713+0000","last_active":"2023-08-27T11:12:14.384452+0000","last_peered":"2023-08-27T11:12:14.384452+0000","last_clean":"2023-08-27T11:12:14.384452+0000","last_became_active":"2023-08-27T11:12:07.907429+0000","last_became_peered":"2023-08-27T11:12:07.907429+0000","last_unstale":"2023-08-27T11:12:14.384452+0000","last_undegraded":"2023-08-27T11:12:14.384452+0000","last_fullsized":"2023-08-27T11:12:14.384452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:29.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982602+0000","last_change":"2023-08-27T11:12:08.923764+0000","last_active":"2023-08-27T11:12:14.982602+0000","last_peered":"2023-08-27T11:12:14.982602+0000","last_clean":"2023-08-27T11:12:14.982602+0000","last_became_active":"2023-08-27T11:12:08.923333+0000","last_became_peered":"2023-08-27T11:12:08.923333+0000","last_unstale":"2023-08-27T11:12:14.982602+0000","last_undegraded":"2023-08-27T11:12:14.982602+0000","last_fullsized":"2023-08-27T11:12:14.982602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:41.863238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982543+0000","last_change":"2023-08-27T11:12:12.948620+0000","last_active":"2023-08-27T11:12:14.982543+0000","last_peered":"2023-08-27T11:12:14.982543+0000","last_clean":"2023-08-27T11:12:14.982543+0000","last_became_active":"2023-08-27T11:12:12.948161+0000","last_became_peered":"2023-08-27T11:12:12.948161+0000","last_unstale":"2023-08-27T11:12:14.982543+0000","last_undegraded":"2023-08-27T11:12:14.982543+0000","last_fullsized":"2023-08-27T11:12:14.982543+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:25.510081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726415+0000","last_change":"2023-08-27T11:12:07.905444+0000","last_active":"2023-08-27T11:12:14.726415+0000","last_peered":"2023-08-27T11:12:14.726415+0000","last_clean":"2023-08-27T11:12:14.726415+0000","last_became_active":"2023-08-27T11:12:07.905102+0000","last_became_peered":"2023-08-27T11:12:07.905102+0000","last_unstale":"2023-08-27T11:12:14.726415+0000","last_undegraded":"2023-08-27T11:12:14.726415+0000","last_fullsized":"2023-08-27T11:12:14.726415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:59.453384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967752+0000","last_change":"2023-08-27T11:12:08.931630+0000","last_active":"2023-08-27T11:12:14.967752+0000","last_peered":"2023-08-27T11:12:14.967752+0000","last_clean":"2023-08-27T11:12:14.967752+0000","last_became_active":"2023-08-27T11:12:08.931239+0000","last_became_peered":"2023-08-27T11:12:08.931239+0000","last_unstale":"2023-08-27T11:12:14.967752+0000","last_undegraded":"2023-08-27T11:12:14.967752+0000","last_fullsized":"2023-08-27T11:12:14.967752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982960+0000","last_change":"2023-08-27T11:12:12.954525+0000","last_active":"2023-08-27T11:12:14.982960+0000","last_peered":"2023-08-27T11:12:14.982960+0000","last_clean":"2023-08-27T11:12:14.982960+0000","last_became_active":"2023-08-27T11:12:12.954077+0000","last_became_peered":"2023-08-27T11:12:12.954077+0000","last_unstale":"2023-08-27T11:12:14.982960+0000","last_undegraded":"2023-08-27T11:12:14.982960+0000","last_fullsized":"2023-08-27T11:12:14.982960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:07:06.449989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384221+0000","last_change":"2023-08-27T11:12:07.907504+0000","last_active":"2023-08-27T11:12:14.384221+0000","last_peered":"2023-08-27T11:12:14.384221+0000","last_clean":"2023-08-27T11:12:14.384221+0000","last_became_active":"2023-08-27T11:12:07.907206+0000","last_became_peered":"2023-08-27T11:12:07.907206+0000","last_unstale":"2023-08-27T11:12:14.384221+0000","last_undegraded":"2023-08-27T11:12:14.384221+0000","last_fullsized":"2023-08-27T11:12:14.384221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:47.521180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726958+0000","last_change":"2023-08-27T11:12:08.918640+0000","last_active":"2023-08-27T11:12:14.726958+0000","last_peered":"2023-08-27T11:12:14.726958+0000","last_clean":"2023-08-27T11:12:14.726958+0000","last_became_active":"2023-08-27T11:12:08.917846+0000","last_became_peered":"2023-08-27T11:12:08.917846+0000","last_unstale":"2023-08-27T11:12:14.726958+0000","last_undegraded":"2023-08-27T11:12:14.726958+0000","last_fullsized":"2023-08-27T11:12:14.726958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:29:50.086138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967688+0000","last_change":"2023-08-27T11:12:12.962983+0000","last_active":"2023-08-27T11:12:14.967688+0000","last_peered":"2023-08-27T11:12:14.967688+0000","last_clean":"2023-08-27T11:12:14.967688+0000","last_became_active":"2023-08-27T11:12:12.962518+0000","last_became_peered":"2023-08-27T11:12:12.962518+0000","last_unstale":"2023-08-27T11:12:14.967688+0000","last_undegraded":"2023-08-27T11:12:14.967688+0000","last_fullsized":"2023-08-27T11:12:14.967688+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:34.829806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932864+0000","last_change":"2023-08-27T11:12:07.905529+0000","last_active":"2023-08-27T11:12:13.932864+0000","last_peered":"2023-08-27T11:12:13.932864+0000","last_clean":"2023-08-27T11:12:13.932864+0000","last_became_active":"2023-08-27T11:12:07.905203+0000","last_became_peered":"2023-08-27T11:12:07.905203+0000","last_unstale":"2023-08-27T11:12:13.932864+0000","last_undegraded":"2023-08-27T11:12:13.932864+0000","last_fullsized":"2023-08-27T11:12:13.932864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:54.273519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.983012+0000","last_change":"2023-08-27T11:12:08.926915+0000","last_active":"2023-08-27T11:12:14.983012+0000","last_peered":"2023-08-27T11:12:14.983012+0000","last_clean":"2023-08-27T11:12:14.983012+0000","last_became_active":"2023-08-27T11:12:08.923355+0000","last_became_peered":"2023-08-27T11:12:08.923355+0000","last_unstale":"2023-08-27T11:12:14.983012+0000","last_undegraded":"2023-08-27T11:12:14.983012+0000","last_fullsized":"2023-08-27T11:12:14.983012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:12.215932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384503+0000","last_change":"2023-08-27T11:12:12.958000+0000","last_active":"2023-08-27T11:12:14.384503+0000","last_peered":"2023-08-27T11:12:14.384503+0000","last_clean":"2023-08-27T11:12:14.384503+0000","last_became_active":"2023-08-27T11:12:12.957611+0000","last_became_peered":"2023-08-27T11:12:12.957611+0000","last_unstale":"2023-08-27T11:12:14.384503+0000","last_undegraded":"2023-08-27T11:12:14.384503+0000","last_fullsized":"2023-08-27T11:12:14.384503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:44.587717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384852+0000","last_change":"2023-08-27T11:12:07.907385+0000","last_active":"2023-08-27T11:12:14.384852+0000","last_peered":"2023-08-27T11:12:14.384852+0000","last_clean":"2023-08-27T11:12:14.384852+0000","last_became_active":"2023-08-27T11:12:07.907116+0000","last_became_peered":"2023-08-27T11:12:07.907116+0000","last_unstale":"2023-08-27T11:12:14.384852+0000","last_undegraded":"2023-08-27T11:12:14.384852+0000","last_fullsized":"2023-08-27T11:12:14.384852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:37.272544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967084+0000","last_change":"2023-08-27T11:12:08.927597+0000","last_active":"2023-08-27T11:12:14.967084+0000","last_peered":"2023-08-27T11:12:14.967084+0000","last_clean":"2023-08-27T11:12:14.967084+0000","last_became_active":"2023-08-27T11:12:08.927306+0000","last_became_peered":"2023-08-27T11:12:08.927306+0000","last_unstale":"2023-08-27T11:12:14.967084+0000","last_undegraded":"2023-08-27T11:12:14.967084+0000","last_fullsized":"2023-08-27T11:12:14.967084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:20.782136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932501+0000","last_change":"2023-08-27T11:12:12.955644+0000","last_active":"2023-08-27T11:12:13.932501+0000","last_peered":"2023-08-27T11:12:13.932501+0000","last_clean":"2023-08-27T11:12:13.932501+0000","last_became_active":"2023-08-27T11:12:12.952322+0000","last_became_peered":"2023-08-27T11:12:12.952322+0000","last_unstale":"2023-08-27T11:12:13.932501+0000","last_undegraded":"2023-08-27T11:12:13.932501+0000","last_fullsized":"2023-08-27T11:12:13.932501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:16:14.569342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384846+0000","last_change":"2023-08-27T11:12:07.907495+0000","last_active":"2023-08-27T11:12:14.384846+0000","last_peered":"2023-08-27T11:12:14.384846+0000","last_clean":"2023-08-27T11:12:14.384846+0000","last_became_active":"2023-08-27T11:12:07.907216+0000","last_became_peered":"2023-08-27T11:12:07.907216+0000","last_unstale":"2023-08-27T11:12:14.384846+0000","last_undegraded":"2023-08-27T11:12:14.384846+0000","last_fullsized":"2023-08-27T11:12:14.384846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:40.858637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933350+0000","last_change":"2023-08-27T11:12:08.924297+0000","last_active":"2023-08-27T11:12:13.933350+0000","last_peered":"2023-08-27T11:12:13.933350+0000","last_clean":"2023-08-27T11:12:13.933350+0000","last_became_active":"2023-08-27T11:12:08.919603+0000","last_became_peered":"2023-08-27T11:12:08.919603+0000","last_unstale":"2023-08-27T11:12:13.933350+0000","last_undegraded":"2023-08-27T11:12:13.933350+0000","last_fullsized":"2023-08-27T11:12:13.933350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:20:35.025362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932779+0000","last_change":"2023-08-27T11:12:12.954662+0000","last_active":"2023-08-27T11:12:13.932779+0000","last_peered":"2023-08-27T11:12:13.932779+0000","last_clean":"2023-08-27T11:12:13.932779+0000","last_became_active":"2023-08-27T11:12:12.954250+0000","last_became_peered":"2023-08-27T11:12:12.954250+0000","last_unstale":"2023-08-27T11:12:13.932779+0000","last_undegraded":"2023-08-27T11:12:13.932779+0000","last_fullsized":"2023-08-27T11:12:13.932779+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:50.028006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384990+0000","last_change":"2023-08-27T11:12:07.907454+0000","last_active":"2023-08-27T11:12:14.384990+0000","last_peered":"2023-08-27T11:12:14.384990+0000","last_clean":"2023-08-27T11:12:14.384990+0000","last_became_active":"2023-08-27T11:12:07.907161+0000","last_became_peered":"2023-08-27T11:12:07.907161+0000","last_unstale":"2023-08-27T11:12:14.384990+0000","last_undegraded":"2023-08-27T11:12:14.384990+0000","last_fullsized":"2023-08-27T11:12:14.384990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:33.436011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933127+0000","last_change":"2023-08-27T11:12:08.920905+0000","last_active":"2023-08-27T11:12:13.933127+0000","last_peered":"2023-08-27T11:12:13.933127+0000","last_clean":"2023-08-27T11:12:13.933127+0000","last_became_active":"2023-08-27T11:12:08.920628+0000","last_became_peered":"2023-08-27T11:12:08.920628+0000","last_unstale":"2023-08-27T11:12:13.933127+0000","last_undegraded":"2023-08-27T11:12:13.933127+0000","last_fullsized":"2023-08-27T11:12:13.933127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:05.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932699+0000","last_change":"2023-08-27T11:12:12.953446+0000","last_active":"2023-08-27T11:12:13.932699+0000","last_peered":"2023-08-27T11:12:13.932699+0000","last_clean":"2023-08-27T11:12:13.932699+0000","last_became_active":"2023-08-27T11:12:12.953019+0000","last_became_peered":"2023-08-27T11:12:12.953019+0000","last_unstale":"2023-08-27T11:12:13.932699+0000","last_undegraded":"2023-08-27T11:12:13.932699+0000","last_fullsized":"2023-08-27T11:12:13.932699+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:11.191056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967365+0000","last_change":"2023-08-27T11:12:07.896745+0000","last_active":"2023-08-27T11:12:14.967365+0000","last_peered":"2023-08-27T11:12:14.967365+0000","last_clean":"2023-08-27T11:12:14.967365+0000","last_became_active":"2023-08-27T11:12:07.896471+0000","last_became_peered":"2023-08-27T11:12:07.896471+0000","last_unstale":"2023-08-27T11:12:14.967365+0000","last_undegraded":"2023-08-27T11:12:14.967365+0000","last_fullsized":"2023-08-27T11:12:14.967365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:09.470216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726539+0000","last_change":"2023-08-27T11:12:08.918280+0000","last_active":"2023-08-27T11:12:14.726539+0000","last_peered":"2023-08-27T11:12:14.726539+0000","last_clean":"2023-08-27T11:12:14.726539+0000","last_became_active":"2023-08-27T11:12:08.917786+0000","last_became_peered":"2023-08-27T11:12:08.917786+0000","last_unstale":"2023-08-27T11:12:14.726539+0000","last_undegraded":"2023-08-27T11:12:14.726539+0000","last_fullsized":"2023-08-27T11:12:14.726539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:45.655952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932983+0000","last_change":"2023-08-27T11:12:12.957044+0000","last_active":"2023-08-27T11:12:13.932983+0000","last_peered":"2023-08-27T11:12:13.932983+0000","last_clean":"2023-08-27T11:12:13.932983+0000","last_became_active":"2023-08-27T11:12:12.953571+0000","last_became_peered":"2023-08-27T11:12:12.953571+0000","last_unstale":"2023-08-27T11:12:13.932983+0000","last_undegraded":"2023-08-27T11:12:13.932983+0000","last_fullsized":"2023-08-27T11:12:13.932983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:42.930026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982654+0000","last_change":"2023-08-27T11:12:07.899209+0000","last_active":"2023-08-27T11:12:14.982654+0000","last_peered":"2023-08-27T11:12:14.982654+0000","last_clean":"2023-08-27T11:12:14.982654+0000","last_became_active":"2023-08-27T11:12:07.898806+0000","last_became_peered":"2023-08-27T11:12:07.898806+0000","last_unstale":"2023-08-27T11:12:14.982654+0000","last_undegraded":"2023-08-27T11:12:14.982654+0000","last_fullsized":"2023-08-27T11:12:14.982654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:37.272544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933574+0000","last_change":"2023-08-27T11:12:08.919614+0000","last_active":"2023-08-27T11:12:13.933574+0000","last_peered":"2023-08-27T11:12:13.933574+0000","last_clean":"2023-08-27T11:12:13.933574+0000","last_became_active":"2023-08-27T11:12:08.916338+0000","last_became_peered":"2023-08-27T11:12:08.916338+0000","last_unstale":"2023-08-27T11:12:13.933574+0000","last_undegraded":"2023-08-27T11:12:13.933574+0000","last_fullsized":"2023-08-27T11:12:13.933574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:49:01.974112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838031+0000","last_change":"2023-08-27T11:12:12.962859+0000","last_active":"2023-08-27T11:12:14.838031+0000","last_peered":"2023-08-27T11:12:14.838031+0000","last_clean":"2023-08-27T11:12:14.838031+0000","last_became_active":"2023-08-27T11:12:12.962421+0000","last_became_peered":"2023-08-27T11:12:12.962421+0000","last_unstale":"2023-08-27T11:12:14.838031+0000","last_undegraded":"2023-08-27T11:12:14.838031+0000","last_fullsized":"2023-08-27T11:12:14.838031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:21.753280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933251+0000","last_change":"2023-08-27T11:12:07.901923+0000","last_active":"2023-08-27T11:12:13.933251+0000","last_peered":"2023-08-27T11:12:13.933251+0000","last_clean":"2023-08-27T11:12:13.933251+0000","last_became_active":"2023-08-27T11:12:07.901634+0000","last_became_peered":"2023-08-27T11:12:07.901634+0000","last_unstale":"2023-08-27T11:12:13.933251+0000","last_undegraded":"2023-08-27T11:12:13.933251+0000","last_fullsized":"2023-08-27T11:12:13.933251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:44.981487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933505+0000","last_change":"2023-08-27T11:12:08.911380+0000","last_active":"2023-08-27T11:12:13.933505+0000","last_peered":"2023-08-27T11:12:13.933505+0000","last_clean":"2023-08-27T11:12:13.933505+0000","last_became_active":"2023-08-27T11:12:08.911077+0000","last_became_peered":"2023-08-27T11:12:08.911077+0000","last_unstale":"2023-08-27T11:12:13.933505+0000","last_undegraded":"2023-08-27T11:12:13.933505+0000","last_fullsized":"2023-08-27T11:12:13.933505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:59.416171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967787+0000","last_change":"2023-08-27T11:12:12.953058+0000","last_active":"2023-08-27T11:12:14.967787+0000","last_peered":"2023-08-27T11:12:14.967787+0000","last_clean":"2023-08-27T11:12:14.967787+0000","last_became_active":"2023-08-27T11:12:12.952571+0000","last_became_peered":"2023-08-27T11:12:12.952571+0000","last_unstale":"2023-08-27T11:12:14.967787+0000","last_undegraded":"2023-08-27T11:12:14.967787+0000","last_fullsized":"2023-08-27T11:12:14.967787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:25.510081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933016+0000","last_change":"2023-08-27T11:12:07.903262+0000","last_active":"2023-08-27T11:12:13.933016+0000","last_peered":"2023-08-27T11:12:13.933016+0000","last_clean":"2023-08-27T11:12:13.933016+0000","last_became_active":"2023-08-27T11:12:07.902549+0000","last_became_peered":"2023-08-27T11:12:07.902549+0000","last_unstale":"2023-08-27T11:12:13.933016+0000","last_undegraded":"2023-08-27T11:12:13.933016+0000","last_fullsized":"2023-08-27T11:12:13.933016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:25.089023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933460+0000","last_change":"2023-08-27T11:12:08.919554+0000","last_active":"2023-08-27T11:12:13.933460+0000","last_peered":"2023-08-27T11:12:13.933460+0000","last_clean":"2023-08-27T11:12:13.933460+0000","last_became_active":"2023-08-27T11:12:08.919252+0000","last_became_peered":"2023-08-27T11:12:08.919252+0000","last_unstale":"2023-08-27T11:12:13.933460+0000","last_undegraded":"2023-08-27T11:12:13.933460+0000","last_fullsized":"2023-08-27T11:12:13.933460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:14:35.778200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933262+0000","last_change":"2023-08-27T11:12:12.954063+0000","last_active":"2023-08-27T11:12:13.933262+0000","last_peered":"2023-08-27T11:12:13.933262+0000","last_clean":"2023-08-27T11:12:13.933262+0000","last_became_active":"2023-08-27T11:12:12.953652+0000","last_became_peered":"2023-08-27T11:12:12.953652+0000","last_unstale":"2023-08-27T11:12:13.933262+0000","last_undegraded":"2023-08-27T11:12:13.933262+0000","last_fullsized":"2023-08-27T11:12:13.933262+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:31.699925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982128+0000","last_change":"2023-08-27T11:12:07.902468+0000","last_active":"2023-08-27T11:12:14.982128+0000","last_peered":"2023-08-27T11:12:14.982128+0000","last_clean":"2023-08-27T11:12:14.982128+0000","last_became_active":"2023-08-27T11:12:07.902193+0000","last_became_peered":"2023-08-27T11:12:07.902193+0000","last_unstale":"2023-08-27T11:12:14.982128+0000","last_undegraded":"2023-08-27T11:12:14.982128+0000","last_fullsized":"2023-08-27T11:12:14.982128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:06.455977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838774+0000","last_change":"2023-08-27T11:12:08.921979+0000","last_active":"2023-08-27T11:12:14.838774+0000","last_peered":"2023-08-27T11:12:14.838774+0000","last_clean":"2023-08-27T11:12:14.838774+0000","last_became_active":"2023-08-27T11:12:08.921659+0000","last_became_peered":"2023-08-27T11:12:08.921659+0000","last_unstale":"2023-08-27T11:12:14.838774+0000","last_undegraded":"2023-08-27T11:12:14.838774+0000","last_fullsized":"2023-08-27T11:12:14.838774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:40.545799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726360+0000","last_change":"2023-08-27T11:12:12.959327+0000","last_active":"2023-08-27T11:12:14.726360+0000","last_peered":"2023-08-27T11:12:14.726360+0000","last_clean":"2023-08-27T11:12:14.726360+0000","last_became_active":"2023-08-27T11:12:12.957064+0000","last_became_peered":"2023-08-27T11:12:12.957064+0000","last_unstale":"2023-08-27T11:12:14.726360+0000","last_undegraded":"2023-08-27T11:12:14.726360+0000","last_fullsized":"2023-08-27T11:12:14.726360+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:20.210306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838189+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:12:14.838189+0000","last_peered":"2023-08-27T11:12:14.838189+0000","last_clean":"2023-08-27T11:12:14.838189+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:12:14.838189+0000","last_undegraded":"2023-08-27T11:12:14.838189+0000","last_fullsized":"2023-08-27T11:12:14.838189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:04.448471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967607+0000","last_change":"2023-08-27T11:12:07.902012+0000","last_active":"2023-08-27T11:12:14.967607+0000","last_peered":"2023-08-27T11:12:14.967607+0000","last_clean":"2023-08-27T11:12:14.967607+0000","last_became_active":"2023-08-27T11:12:07.901706+0000","last_became_peered":"2023-08-27T11:12:07.901706+0000","last_unstale":"2023-08-27T11:12:14.967607+0000","last_undegraded":"2023-08-27T11:12:14.967607+0000","last_fullsized":"2023-08-27T11:12:14.967607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:21.743516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838056+0000","last_change":"2023-08-27T11:12:08.916047+0000","last_active":"2023-08-27T11:12:14.838056+0000","last_peered":"2023-08-27T11:12:14.838056+0000","last_clean":"2023-08-27T11:12:14.838056+0000","last_became_active":"2023-08-27T11:12:08.915742+0000","last_became_peered":"2023-08-27T11:12:08.915742+0000","last_unstale":"2023-08-27T11:12:14.838056+0000","last_undegraded":"2023-08-27T11:12:14.838056+0000","last_fullsized":"2023-08-27T11:12:14.838056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:29.674355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726449+0000","last_change":"2023-08-27T11:12:12.952690+0000","last_active":"2023-08-27T11:12:14.726449+0000","last_peered":"2023-08-27T11:12:14.726449+0000","last_clean":"2023-08-27T11:12:14.726449+0000","last_became_active":"2023-08-27T11:12:12.952209+0000","last_became_peered":"2023-08-27T11:12:12.952209+0000","last_unstale":"2023-08-27T11:12:14.726449+0000","last_undegraded":"2023-08-27T11:12:14.726449+0000","last_fullsized":"2023-08-27T11:12:14.726449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:28.731513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967363+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:12:14.967363+0000","last_peered":"2023-08-27T11:12:14.967363+0000","last_clean":"2023-08-27T11:12:14.967363+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:12:14.967363+0000","last_undegraded":"2023-08-27T11:12:14.967363+0000","last_fullsized":"2023-08-27T11:12:14.967363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:04.859233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933791+0000","last_change":"2023-08-27T11:12:07.905363+0000","last_active":"2023-08-27T11:12:13.933791+0000","last_peered":"2023-08-27T11:12:13.933791+0000","last_clean":"2023-08-27T11:12:13.933791+0000","last_became_active":"2023-08-27T11:12:07.905002+0000","last_became_peered":"2023-08-27T11:12:07.905002+0000","last_unstale":"2023-08-27T11:12:13.933791+0000","last_undegraded":"2023-08-27T11:12:13.933791+0000","last_fullsized":"2023-08-27T11:12:13.933791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:06:55.872046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838055+0000","last_change":"2023-08-27T11:12:08.920259+0000","last_active":"2023-08-27T11:12:14.838055+0000","last_peered":"2023-08-27T11:12:14.838055+0000","last_clean":"2023-08-27T11:12:14.838055+0000","last_became_active":"2023-08-27T11:12:08.916978+0000","last_became_peered":"2023-08-27T11:12:08.916978+0000","last_unstale":"2023-08-27T11:12:14.838055+0000","last_undegraded":"2023-08-27T11:12:14.838055+0000","last_fullsized":"2023-08-27T11:12:14.838055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:29:50.086138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933007+0000","last_change":"2023-08-27T11:12:12.959735+0000","last_active":"2023-08-27T11:12:13.933007+0000","last_peered":"2023-08-27T11:12:13.933007+0000","last_clean":"2023-08-27T11:12:13.933007+0000","last_became_active":"2023-08-27T11:12:12.959275+0000","last_became_peered":"2023-08-27T11:12:12.959275+0000","last_unstale":"2023-08-27T11:12:13.933007+0000","last_undegraded":"2023-08-27T11:12:13.933007+0000","last_fullsized":"2023-08-27T11:12:13.933007+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:57.120477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932550+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:12:13.932550+0000","last_peered":"2023-08-27T11:12:13.932550+0000","last_clean":"2023-08-27T11:12:13.932550+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:12:13.932550+0000","last_undegraded":"2023-08-27T11:12:13.932550+0000","last_fullsized":"2023-08-27T11:12:13.932550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:49.251775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837761+0000","last_change":"2023-08-27T11:12:07.902482+0000","last_active":"2023-08-27T11:12:14.837761+0000","last_peered":"2023-08-27T11:12:14.837761+0000","last_clean":"2023-08-27T11:12:14.837761+0000","last_became_active":"2023-08-27T11:12:07.902184+0000","last_became_peered":"2023-08-27T11:12:07.902184+0000","last_unstale":"2023-08-27T11:12:14.837761+0000","last_undegraded":"2023-08-27T11:12:14.837761+0000","last_fullsized":"2023-08-27T11:12:14.837761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:46.293833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838666+0000","last_change":"2023-08-27T11:12:08.919949+0000","last_active":"2023-08-27T11:12:14.838666+0000","last_peered":"2023-08-27T11:12:14.838666+0000","last_clean":"2023-08-27T11:12:14.838666+0000","last_became_active":"2023-08-27T11:12:08.918796+0000","last_became_peered":"2023-08-27T11:12:08.918796+0000","last_unstale":"2023-08-27T11:12:14.838666+0000","last_undegraded":"2023-08-27T11:12:14.838666+0000","last_fullsized":"2023-08-27T11:12:14.838666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:51.458617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384661+0000","last_change":"2023-08-27T11:12:12.965405+0000","last_active":"2023-08-27T11:12:14.384661+0000","last_peered":"2023-08-27T11:12:14.384661+0000","last_clean":"2023-08-27T11:12:14.384661+0000","last_became_active":"2023-08-27T11:12:12.965085+0000","last_became_peered":"2023-08-27T11:12:12.965085+0000","last_unstale":"2023-08-27T11:12:14.384661+0000","last_undegraded":"2023-08-27T11:12:14.384661+0000","last_fullsized":"2023-08-27T11:12:14.384661+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:45.516388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967468+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:12:14.967468+0000","last_peered":"2023-08-27T11:12:14.967468+0000","last_clean":"2023-08-27T11:12:14.967468+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:12:14.967468+0000","last_undegraded":"2023-08-27T11:12:14.967468+0000","last_fullsized":"2023-08-27T11:12:14.967468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:47.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384165+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:12:14.384165+0000","last_peered":"2023-08-27T11:12:14.384165+0000","last_clean":"2023-08-27T11:12:14.384165+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:12:14.384165+0000","last_undegraded":"2023-08-27T11:12:14.384165+0000","last_fullsized":"2023-08-27T11:12:14.384165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:10.078531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982472+0000","last_change":"2023-08-27T11:12:07.903434+0000","last_active":"2023-08-27T11:12:14.982472+0000","last_peered":"2023-08-27T11:12:14.982472+0000","last_clean":"2023-08-27T11:12:14.982472+0000","last_became_active":"2023-08-27T11:12:07.903142+0000","last_became_peered":"2023-08-27T11:12:07.903142+0000","last_unstale":"2023-08-27T11:12:14.982472+0000","last_undegraded":"2023-08-27T11:12:14.982472+0000","last_fullsized":"2023-08-27T11:12:14.982472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:40.565665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934388+0000","last_change":"2023-08-27T11:12:08.915968+0000","last_active":"2023-08-27T11:12:13.934388+0000","last_peered":"2023-08-27T11:12:13.934388+0000","last_clean":"2023-08-27T11:12:13.934388+0000","last_became_active":"2023-08-27T11:12:08.915646+0000","last_became_peered":"2023-08-27T11:12:08.915646+0000","last_unstale":"2023-08-27T11:12:13.934388+0000","last_undegraded":"2023-08-27T11:12:13.934388+0000","last_fullsized":"2023-08-27T11:12:13.934388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:53.078559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967776+0000","last_change":"2023-08-27T11:12:12.953539+0000","last_active":"2023-08-27T11:12:14.967776+0000","last_peered":"2023-08-27T11:12:14.967776+0000","last_clean":"2023-08-27T11:12:14.967776+0000","last_became_active":"2023-08-27T11:12:12.953154+0000","last_became_peered":"2023-08-27T11:12:12.953154+0000","last_unstale":"2023-08-27T11:12:14.967776+0000","last_undegraded":"2023-08-27T11:12:14.967776+0000","last_fullsized":"2023-08-27T11:12:14.967776+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:45.516388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384351+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:12:14.384351+0000","last_peered":"2023-08-27T11:12:14.384351+0000","last_clean":"2023-08-27T11:12:14.384351+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:12:14.384351+0000","last_undegraded":"2023-08-27T11:12:14.384351+0000","last_fullsized":"2023-08-27T11:12:14.384351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:47.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.725986+0000","last_change":"2023-08-27T11:12:07.899116+0000","last_active":"2023-08-27T11:12:14.725986+0000","last_peered":"2023-08-27T11:12:14.725986+0000","last_clean":"2023-08-27T11:12:14.725986+0000","last_became_active":"2023-08-27T11:12:07.898805+0000","last_became_peered":"2023-08-27T11:12:07.898805+0000","last_unstale":"2023-08-27T11:12:14.725986+0000","last_undegraded":"2023-08-27T11:12:14.725986+0000","last_fullsized":"2023-08-27T11:12:14.725986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:07.168983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726764+0000","last_change":"2023-08-27T11:12:08.918851+0000","last_active":"2023-08-27T11:12:14.726764+0000","last_peered":"2023-08-27T11:12:14.726764+0000","last_clean":"2023-08-27T11:12:14.726764+0000","last_became_active":"2023-08-27T11:12:08.918248+0000","last_became_peered":"2023-08-27T11:12:08.918248+0000","last_unstale":"2023-08-27T11:12:14.726764+0000","last_undegraded":"2023-08-27T11:12:14.726764+0000","last_fullsized":"2023-08-27T11:12:14.726764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:45.250102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933527+0000","last_change":"2023-08-27T11:12:12.959470+0000","last_active":"2023-08-27T11:12:13.933527+0000","last_peered":"2023-08-27T11:12:13.933527+0000","last_clean":"2023-08-27T11:12:13.933527+0000","last_became_active":"2023-08-27T11:12:12.959082+0000","last_became_peered":"2023-08-27T11:12:12.959082+0000","last_unstale":"2023-08-27T11:12:13.933527+0000","last_undegraded":"2023-08-27T11:12:13.933527+0000","last_fullsized":"2023-08-27T11:12:13.933527+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:04.206952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384869+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:12:14.384869+0000","last_peered":"2023-08-27T11:12:14.384869+0000","last_clean":"2023-08-27T11:12:14.384869+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:12:14.384869+0000","last_undegraded":"2023-08-27T11:12:14.384869+0000","last_fullsized":"2023-08-27T11:12:14.384869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:43.735919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838283+0000","last_change":"2023-08-27T11:12:07.903460+0000","last_active":"2023-08-27T11:12:14.838283+0000","last_peered":"2023-08-27T11:12:14.838283+0000","last_clean":"2023-08-27T11:12:14.838283+0000","last_became_active":"2023-08-27T11:12:07.903110+0000","last_became_peered":"2023-08-27T11:12:07.903110+0000","last_unstale":"2023-08-27T11:12:14.838283+0000","last_undegraded":"2023-08-27T11:12:14.838283+0000","last_fullsized":"2023-08-27T11:12:14.838283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:09.880978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726625+0000","last_change":"2023-08-27T11:12:08.913520+0000","last_active":"2023-08-27T11:12:14.726625+0000","last_peered":"2023-08-27T11:12:14.726625+0000","last_clean":"2023-08-27T11:12:14.726625+0000","last_became_active":"2023-08-27T11:12:08.913192+0000","last_became_peered":"2023-08-27T11:12:08.913192+0000","last_unstale":"2023-08-27T11:12:14.726625+0000","last_undegraded":"2023-08-27T11:12:14.726625+0000","last_fullsized":"2023-08-27T11:12:14.726625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:12:06.235129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933906+0000","last_change":"2023-08-27T11:12:12.959458+0000","last_active":"2023-08-27T11:12:13.933906+0000","last_peered":"2023-08-27T11:12:13.933906+0000","last_clean":"2023-08-27T11:12:13.933906+0000","last_became_active":"2023-08-27T11:12:12.959073+0000","last_became_peered":"2023-08-27T11:12:12.959073+0000","last_unstale":"2023-08-27T11:12:13.933906+0000","last_undegraded":"2023-08-27T11:12:13.933906+0000","last_fullsized":"2023-08-27T11:12:13.933906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:37:24.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.983100+0000","last_change":"2023-08-27T11:12:07.908908+0000","last_active":"2023-08-27T11:12:14.983100+0000","last_peered":"2023-08-27T11:12:14.983100+0000","last_clean":"2023-08-27T11:12:14.983100+0000","last_became_active":"2023-08-27T11:12:07.908670+0000","last_became_peered":"2023-08-27T11:12:07.908670+0000","last_unstale":"2023-08-27T11:12:14.983100+0000","last_undegraded":"2023-08-27T11:12:14.983100+0000","last_fullsized":"2023-08-27T11:12:14.983100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:06:59.757966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838348+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:12:14.838348+0000","last_peered":"2023-08-27T11:12:14.838348+0000","last_clean":"2023-08-27T11:12:14.838348+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:12:14.838348+0000","last_undegraded":"2023-08-27T11:12:14.838348+0000","last_fullsized":"2023-08-27T11:12:14.838348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:01.434232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982129+0000","last_change":"2023-08-27T11:12:08.917056+0000","last_active":"2023-08-27T11:12:14.982129+0000","last_peered":"2023-08-27T11:12:14.982129+0000","last_clean":"2023-08-27T11:12:14.982129+0000","last_became_active":"2023-08-27T11:12:08.916853+0000","last_became_peered":"2023-08-27T11:12:08.916853+0000","last_unstale":"2023-08-27T11:12:14.982129+0000","last_undegraded":"2023-08-27T11:12:14.982129+0000","last_fullsized":"2023-08-27T11:12:14.982129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:48.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982032+0000","last_change":"2023-08-27T11:12:12.955385+0000","last_active":"2023-08-27T11:12:14.982032+0000","last_peered":"2023-08-27T11:12:14.982032+0000","last_clean":"2023-08-27T11:12:14.982032+0000","last_became_active":"2023-08-27T11:12:12.953960+0000","last_became_peered":"2023-08-27T11:12:12.953960+0000","last_unstale":"2023-08-27T11:12:14.982032+0000","last_undegraded":"2023-08-27T11:12:14.982032+0000","last_fullsized":"2023-08-27T11:12:14.982032+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:14.927498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934591+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:12:13.934591+0000","last_peered":"2023-08-27T11:12:13.934591+0000","last_clean":"2023-08-27T11:12:13.934591+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:12:13.934591+0000","last_undegraded":"2023-08-27T11:12:13.934591+0000","last_fullsized":"2023-08-27T11:12:13.934591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:05:01.448436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726808+0000","last_change":"2023-08-27T11:12:07.902830+0000","last_active":"2023-08-27T11:12:14.726808+0000","last_peered":"2023-08-27T11:12:14.726808+0000","last_clean":"2023-08-27T11:12:14.726808+0000","last_became_active":"2023-08-27T11:12:07.902452+0000","last_became_peered":"2023-08-27T11:12:07.902452+0000","last_unstale":"2023-08-27T11:12:14.726808+0000","last_undegraded":"2023-08-27T11:12:14.726808+0000","last_fullsized":"2023-08-27T11:12:14.726808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:46.293833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933163+0000","last_change":"2023-08-27T11:12:08.919355+0000","last_active":"2023-08-27T11:12:13.933163+0000","last_peered":"2023-08-27T11:12:13.933163+0000","last_clean":"2023-08-27T11:12:13.933163+0000","last_became_active":"2023-08-27T11:12:08.917203+0000","last_became_peered":"2023-08-27T11:12:08.917203+0000","last_unstale":"2023-08-27T11:12:13.933163+0000","last_undegraded":"2023-08-27T11:12:13.933163+0000","last_fullsized":"2023-08-27T11:12:13.933163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:07:00.762567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838062+0000","last_change":"2023-08-27T11:12:12.963107+0000","last_active":"2023-08-27T11:12:14.838062+0000","last_peered":"2023-08-27T11:12:14.838062+0000","last_clean":"2023-08-27T11:12:14.838062+0000","last_became_active":"2023-08-27T11:12:12.962714+0000","last_became_peered":"2023-08-27T11:12:12.962714+0000","last_unstale":"2023-08-27T11:12:14.838062+0000","last_undegraded":"2023-08-27T11:12:14.838062+0000","last_fullsized":"2023-08-27T11:12:14.838062+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:24.824055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726473+0000","last_change":"2023-08-27T11:12:07.900588+0000","last_active":"2023-08-27T11:12:14.726473+0000","last_peered":"2023-08-27T11:12:14.726473+0000","last_clean":"2023-08-27T11:12:14.726473+0000","last_became_active":"2023-08-27T11:12:07.900223+0000","last_became_peered":"2023-08-27T11:12:07.900223+0000","last_unstale":"2023-08-27T11:12:14.726473+0000","last_undegraded":"2023-08-27T11:12:14.726473+0000","last_fullsized":"2023-08-27T11:12:14.726473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:54.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933266+0000","last_change":"2023-08-27T11:12:08.921029+0000","last_active":"2023-08-27T11:12:13.933266+0000","last_peered":"2023-08-27T11:12:13.933266+0000","last_clean":"2023-08-27T11:12:13.933266+0000","last_became_active":"2023-08-27T11:12:08.920756+0000","last_became_peered":"2023-08-27T11:12:08.920756+0000","last_unstale":"2023-08-27T11:12:13.933266+0000","last_undegraded":"2023-08-27T11:12:13.933266+0000","last_fullsized":"2023-08-27T11:12:13.933266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:21:21.628536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384440+0000","last_change":"2023-08-27T11:12:12.958972+0000","last_active":"2023-08-27T11:12:14.384440+0000","last_peered":"2023-08-27T11:12:14.384440+0000","last_clean":"2023-08-27T11:12:14.384440+0000","last_became_active":"2023-08-27T11:12:12.958605+0000","last_became_peered":"2023-08-27T11:12:12.958605+0000","last_unstale":"2023-08-27T11:12:14.384440+0000","last_undegraded":"2023-08-27T11:12:14.384440+0000","last_fullsized":"2023-08-27T11:12:14.384440+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:55.500535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967443+0000","last_change":"2023-08-27T11:12:07.898515+0000","last_active":"2023-08-27T11:12:14.967443+0000","last_peered":"2023-08-27T11:12:14.967443+0000","last_clean":"2023-08-27T11:12:14.967443+0000","last_became_active":"2023-08-27T11:12:07.898243+0000","last_became_peered":"2023-08-27T11:12:07.898243+0000","last_unstale":"2023-08-27T11:12:14.967443+0000","last_undegraded":"2023-08-27T11:12:14.967443+0000","last_fullsized":"2023-08-27T11:12:14.967443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:47.521180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934374+0000","last_change":"2023-08-27T11:12:08.919278+0000","last_active":"2023-08-27T11:12:13.934374+0000","last_peered":"2023-08-27T11:12:13.934374+0000","last_clean":"2023-08-27T11:12:13.934374+0000","last_became_active":"2023-08-27T11:12:08.913663+0000","last_became_peered":"2023-08-27T11:12:08.913663+0000","last_unstale":"2023-08-27T11:12:13.934374+0000","last_undegraded":"2023-08-27T11:12:13.934374+0000","last_fullsized":"2023-08-27T11:12:13.934374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:20:17.707884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967392+0000","last_change":"2023-08-27T11:12:12.955103+0000","last_active":"2023-08-27T11:12:14.967392+0000","last_peered":"2023-08-27T11:12:14.967392+0000","last_clean":"2023-08-27T11:12:14.967392+0000","last_became_active":"2023-08-27T11:12:12.954719+0000","last_became_peered":"2023-08-27T11:12:12.954719+0000","last_unstale":"2023-08-27T11:12:14.967392+0000","last_undegraded":"2023-08-27T11:12:14.967392+0000","last_fullsized":"2023-08-27T11:12:14.967392+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:16.222119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967603+0000","last_change":"2023-08-27T11:12:07.899028+0000","last_active":"2023-08-27T11:12:14.967603+0000","last_peered":"2023-08-27T11:12:14.967603+0000","last_clean":"2023-08-27T11:12:14.967603+0000","last_became_active":"2023-08-27T11:12:07.898749+0000","last_became_peered":"2023-08-27T11:12:07.898749+0000","last_unstale":"2023-08-27T11:12:14.967603+0000","last_undegraded":"2023-08-27T11:12:14.967603+0000","last_fullsized":"2023-08-27T11:12:14.967603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:50.454016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933154+0000","last_change":"2023-08-27T11:12:08.917140+0000","last_active":"2023-08-27T11:12:13.933154+0000","last_peered":"2023-08-27T11:12:13.933154+0000","last_clean":"2023-08-27T11:12:13.933154+0000","last_became_active":"2023-08-27T11:12:08.916844+0000","last_became_peered":"2023-08-27T11:12:08.916844+0000","last_unstale":"2023-08-27T11:12:13.933154+0000","last_undegraded":"2023-08-27T11:12:13.933154+0000","last_fullsized":"2023-08-27T11:12:13.933154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:12.215932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933507+0000","last_change":"2023-08-27T11:12:12.952640+0000","last_active":"2023-08-27T11:12:13.933507+0000","last_peered":"2023-08-27T11:12:13.933507+0000","last_clean":"2023-08-27T11:12:13.933507+0000","last_became_active":"2023-08-27T11:12:12.952235+0000","last_became_peered":"2023-08-27T11:12:12.952235+0000","last_unstale":"2023-08-27T11:12:13.933507+0000","last_undegraded":"2023-08-27T11:12:13.933507+0000","last_fullsized":"2023-08-27T11:12:13.933507+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:44.222308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982940+0000","last_change":"2023-08-27T11:12:07.898854+0000","last_active":"2023-08-27T11:12:14.982940+0000","last_peered":"2023-08-27T11:12:14.982940+0000","last_clean":"2023-08-27T11:12:14.982940+0000","last_became_active":"2023-08-27T11:12:07.898427+0000","last_became_peered":"2023-08-27T11:12:07.898427+0000","last_unstale":"2023-08-27T11:12:14.982940+0000","last_undegraded":"2023-08-27T11:12:14.982940+0000","last_fullsized":"2023-08-27T11:12:14.982940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:25.089023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982399+0000","last_change":"2023-08-27T11:12:08.913067+0000","last_active":"2023-08-27T11:12:14.982399+0000","last_peered":"2023-08-27T11:12:14.982399+0000","last_clean":"2023-08-27T11:12:14.982399+0000","last_became_active":"2023-08-27T11:12:08.912345+0000","last_became_peered":"2023-08-27T11:12:08.912345+0000","last_unstale":"2023-08-27T11:12:14.982399+0000","last_undegraded":"2023-08-27T11:12:14.982399+0000","last_fullsized":"2023-08-27T11:12:14.982399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:41.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838060+0000","last_change":"2023-08-27T11:12:12.948294+0000","last_active":"2023-08-27T11:12:14.838060+0000","last_peered":"2023-08-27T11:12:14.838060+0000","last_clean":"2023-08-27T11:12:14.838060+0000","last_became_active":"2023-08-27T11:12:12.947742+0000","last_became_peered":"2023-08-27T11:12:12.947742+0000","last_unstale":"2023-08-27T11:12:14.838060+0000","last_undegraded":"2023-08-27T11:12:14.838060+0000","last_fullsized":"2023-08-27T11:12:14.838060+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:06.825066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727133+0000","last_change":"2023-08-27T11:12:07.902780+0000","last_active":"2023-08-27T11:12:14.727133+0000","last_peered":"2023-08-27T11:12:14.727133+0000","last_clean":"2023-08-27T11:12:14.727133+0000","last_became_active":"2023-08-27T11:12:07.902465+0000","last_became_peered":"2023-08-27T11:12:07.902465+0000","last_unstale":"2023-08-27T11:12:14.727133+0000","last_undegraded":"2023-08-27T11:12:14.727133+0000","last_fullsized":"2023-08-27T11:12:14.727133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:09.470216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933956+0000","last_change":"2023-08-27T11:12:08.925508+0000","last_active":"2023-08-27T11:12:13.933956+0000","last_peered":"2023-08-27T11:12:13.933956+0000","last_clean":"2023-08-27T11:12:13.933956+0000","last_became_active":"2023-08-27T11:12:08.919770+0000","last_became_peered":"2023-08-27T11:12:08.919770+0000","last_unstale":"2023-08-27T11:12:13.933956+0000","last_undegraded":"2023-08-27T11:12:13.933956+0000","last_fullsized":"2023-08-27T11:12:13.933956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:57.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932961+0000","last_change":"2023-08-27T11:12:08.914782+0000","last_active":"2023-08-27T11:12:13.932961+0000","last_peered":"2023-08-27T11:12:13.932961+0000","last_clean":"2023-08-27T11:12:13.932961+0000","last_became_active":"2023-08-27T11:12:08.914443+0000","last_became_peered":"2023-08-27T11:12:08.914443+0000","last_unstale":"2023-08-27T11:12:13.932961+0000","last_undegraded":"2023-08-27T11:12:13.932961+0000","last_fullsized":"2023-08-27T11:12:13.932961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:07.149137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982607+0000","last_change":"2023-08-27T11:12:08.923263+0000","last_active":"2023-08-27T11:12:14.982607+0000","last_peered":"2023-08-27T11:12:14.982607+0000","last_clean":"2023-08-27T11:12:14.982607+0000","last_became_active":"2023-08-27T11:12:08.922904+0000","last_became_peered":"2023-08-27T11:12:08.922904+0000","last_unstale":"2023-08-27T11:12:14.982607+0000","last_undegraded":"2023-08-27T11:12:14.982607+0000","last_fullsized":"2023-08-27T11:12:14.982607+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:49.762265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934690+0000","last_change":"2023-08-27T11:12:08.925804+0000","last_active":"2023-08-27T11:12:13.934690+0000","last_peered":"2023-08-27T11:12:13.934690+0000","last_clean":"2023-08-27T11:12:13.934690+0000","last_became_active":"2023-08-27T11:12:08.925554+0000","last_became_peered":"2023-08-27T11:12:08.925554+0000","last_unstale":"2023-08-27T11:12:13.934690+0000","last_undegraded":"2023-08-27T11:12:13.934690+0000","last_fullsized":"2023-08-27T11:12:13.934690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:15.724420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982308+0000","last_change":"2023-08-27T11:12:08.929175+0000","last_active":"2023-08-27T11:12:14.982308+0000","last_peered":"2023-08-27T11:12:14.982308+0000","last_clean":"2023-08-27T11:12:14.982308+0000","last_became_active":"2023-08-27T11:12:08.928863+0000","last_became_peered":"2023-08-27T11:12:08.928863+0000","last_unstale":"2023-08-27T11:12:14.982308+0000","last_undegraded":"2023-08-27T11:12:14.982308+0000","last_fullsized":"2023-08-27T11:12:14.982308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:30.787887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838203+0000","last_change":"2023-08-27T11:12:08.924528+0000","last_active":"2023-08-27T11:12:14.838203+0000","last_peered":"2023-08-27T11:12:14.838203+0000","last_clean":"2023-08-27T11:12:14.838203+0000","last_became_active":"2023-08-27T11:12:08.924221+0000","last_became_peered":"2023-08-27T11:12:08.924221+0000","last_unstale":"2023-08-27T11:12:14.838203+0000","last_undegraded":"2023-08-27T11:12:14.838203+0000","last_fullsized":"2023-08-27T11:12:14.838203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:12.180200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":710851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":120553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:12:40.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:12:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.162+0000 7ff5882e1700 1 Processor -- start 2023-08-27T11:12:40.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.162+0000 7ff5882e1700 1 -- start start 2023-08-27T11:12:40.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff5882e1700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff58015a0f0 0x7ff5800cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff5882e1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5800cd300 0x7ff58015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff5882e1700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5800cfae0 con 0x7ff58015e030 2023-08-27T11:12:40.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff5882e1700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58015e900 con 0x7ff58015a0f0 2023-08-27T11:12:40.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff5882e1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5800cae60 con 0x7ff5800cd300 2023-08-27T11:12:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff58607d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff58015a0f0 0x7ff5800cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff58587c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5800cd300 0x7ff58015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff58587c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5800cd300 0x7ff58015daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50230/0 (socket says 172.21.15.45:50230) 2023-08-27T11:12:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff58607d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff58015a0f0 0x7ff5800cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41174/0 (socket says 172.21.15.45:41174) 2023-08-27T11:12:40.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.163+0000 7ff58587c700 1 -- 172.21.15.45:0/2298354750 learned_addr learned my addr 172.21.15.45:0/2298354750 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58587c700 1 -- 172.21.15.45:0/2298354750 >> v2:172.21.15.135:3300/0 conn(0x7ff58015a0f0 msgr2=0x7ff5800cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58587c700 1 --2- 172.21.15.45:0/2298354750 >> v2:172.21.15.135:3300/0 conn(0x7ff58015a0f0 0x7ff5800cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58587c700 1 -- 172.21.15.45:0/2298354750 >> v1:172.21.15.135:6789/0 conn(0x7ff58015e030 legacy=0x7ff5800cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58587c700 1 -- 172.21.15.45:0/2298354750 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5800cb140 con 0x7ff5800cd300 2023-08-27T11:12:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58607d700 1 --2- 172.21.15.45:0/2298354750 >> v2:172.21.15.135:3300/0 conn(0x7ff58015a0f0 0x7ff5800cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:40.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58507b700 1 -- 172.21.15.45:0/2298354750 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421583189 0 0) 0x7ff5800cfae0 con 0x7ff58015e030 2023-08-27T11:12:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58587c700 1 --2- 172.21.15.45:0/2298354750 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5800cd300 0x7ff58015daf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5700098f0 tx=0x7ff570009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b52a70ef197501 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58507b700 1 -- 172.21.15.45:0/2298354750 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570011830 con 0x7ff5800cd300 2023-08-27T11:12:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58507b700 1 -- 172.21.15.45:0/2298354750 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff570011990 con 0x7ff5800cd300 2023-08-27T11:12:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff58507b700 1 -- 172.21.15.45:0/2298354750 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570011d20 con 0x7ff5800cd300 2023-08-27T11:12:40.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff5882e1700 1 -- 172.21.15.45:0/2298354750 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5800cd300 msgr2=0x7ff58015daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.164+0000 7ff5882e1700 1 --2- 172.21.15.45:0/2298354750 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5800cd300 0x7ff58015daf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5700098f0 tx=0x7ff570009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:40.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.165+0000 7ff5882e1700 1 -- 172.21.15.45:0/2298354750 shutdown_connections 2023-08-27T11:12:40.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.165+0000 7ff5882e1700 1 --2- 172.21.15.45:0/2298354750 >> v2:172.21.15.135:3300/0 conn(0x7ff58015a0f0 0x7ff5800cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.165+0000 7ff5882e1700 1 --2- 172.21.15.45:0/2298354750 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5800cd300 0x7ff58015daf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.165+0000 7ff5882e1700 1 -- 172.21.15.45:0/2298354750 >> 172.21.15.45:0/2298354750 conn(0x7ff5800cc7e0 msgr2=0x7ff5800ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:40.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.165+0000 7ff5882e1700 1 -- 172.21.15.45:0/2298354750 shutdown_connections 2023-08-27T11:12:40.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.165+0000 7ff5882e1700 1 -- 172.21.15.45:0/2298354750 wait complete. 2023-08-27T11:12:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.166+0000 7ff5882e1700 1 Processor -- start 2023-08-27T11:12:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.166+0000 7ff5882e1700 1 -- start start 2023-08-27T11:12:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff5882e1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800cd300 0x7ff5800b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff5882e1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58015a0f0 0x7ff5800ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff5882e1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58015e030 0x7ff5800ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff5882e1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5800ae7c0 con 0x7ff58015a0f0 2023-08-27T11:12:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff5882e1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5800af300 con 0x7ff5800cd300 2023-08-27T11:12:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff5882e1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5800aea60 con 0x7ff58015e030 2023-08-27T11:12:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58587c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58015a0f0 0x7ff5800ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58587c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58015a0f0 0x7ff5800ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52326/0 (socket says 172.21.15.45:52326) 2023-08-27T11:12:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58587c700 1 -- 172.21.15.45:0/4039469150 learned_addr learned my addr 172.21.15.45:0/4039469150 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58607d700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800cd300 0x7ff5800b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58687e700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58015e030 0x7ff5800ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58587c700 1 -- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58015e030 msgr2=0x7ff5800ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58587c700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58015e030 0x7ff5800ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58587c700 1 -- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800cd300 msgr2=0x7ff5800b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58587c700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800cd300 0x7ff5800b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58587c700 1 -- 172.21.15.45:0/4039469150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c0094a0 con 0x7ff58015a0f0 2023-08-27T11:12:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff58587c700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58015a0f0 0x7ff5800ce180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff570014040 tx=0x7ff570004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff576ffd700 1 -- 172.21.15.45:0/4039469150 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700036a0 con 0x7ff58015a0f0 2023-08-27T11:12:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff576ffd700 1 -- 172.21.15.45:0/4039469150 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57001e070 con 0x7ff58015a0f0 2023-08-27T11:12:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff576ffd700 1 -- 172.21.15.45:0/4039469150 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570003a20 con 0x7ff58015a0f0 2023-08-27T11:12:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.167+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff570009550 con 0x7ff58015a0f0 2023-08-27T11:12:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.168+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5800af020 con 0x7ff58015a0f0 2023-08-27T11:12:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.169+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5800726d0 con 0x7ff58015a0f0 2023-08-27T11:12:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.169+0000 7ff576ffd700 1 -- 172.21.15.45:0/4039469150 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff570003b80 con 0x7ff58015a0f0 2023-08-27T11:12:40.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.169+0000 7ff576ffd700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff56406a0e0 0x7ff56406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.169+0000 7ff576ffd700 1 -- 172.21.15.45:0/4039469150 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff5700897b0 con 0x7ff58015a0f0 2023-08-27T11:12:40.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.170+0000 7ff58607d700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff56406a0e0 0x7ff56406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.170+0000 7ff58607d700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff56406a0e0 0x7ff56406c5a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff580158c60 tx=0x7ff57c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:40.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.174+0000 7ff576ffd700 1 -- 172.21.15.45:0/4039469150 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff570054c40 con 0x7ff58015a0f0 2023-08-27T11:12:40.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.370+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 --> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5800cb700 con 0x7ff56406a0e0 2023-08-27T11:12:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.380+0000 7ff576ffd700 1 -- 172.21.15.45:0/4039469150 <== mgr.4100 v2:172.21.15.135:6832/4120550306 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367365 (secure 0 0 0) 0x7ff57c011860 con 0x7ff56406a0e0 2023-08-27T11:12:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:40.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff56406a0e0 msgr2=0x7ff56406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff56406a0e0 0x7ff56406c5a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff580158c60 tx=0x7ff57c008040 comp rx=0 tx=0).stop 2023-08-27T11:12:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58015a0f0 msgr2=0x7ff5800ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58015a0f0 0x7ff5800ce180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff570014040 tx=0x7ff570004750 comp rx=0 tx=0).stop 2023-08-27T11:12:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 shutdown_connections 2023-08-27T11:12:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff56406a0e0 0x7ff56406c5a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff58015e030 0x7ff5800ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800cd300 0x7ff5800b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 --2- 172.21.15.45:0/4039469150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58015a0f0 0x7ff5800ce180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 >> 172.21.15.45:0/4039469150 conn(0x7ff5800cc7e0 msgr2=0x7ff58006e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 shutdown_connections 2023-08-27T11:12:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.384+0000 7ff5882e1700 1 -- 172.21.15.45:0/4039469150 wait complete. 2023-08-27T11:12:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:12:40.398 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-08-27T11:12:38.812510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219664,"kb_used_data":3496,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755056,"statfs":{"total":773094113280,"available":772869177344,"internally_reserved":0,"allocated":3579904,"data_stored":2149224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003845"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932872+0000","last_change":"2023-08-27T11:12:08.913049+0000","last_active":"2023-08-27T11:12:13.932872+0000","last_peered":"2023-08-27T11:12:13.932872+0000","last_clean":"2023-08-27T11:12:13.932872+0000","last_became_active":"2023-08-27T11:12:08.912757+0000","last_became_peered":"2023-08-27T11:12:08.912757+0000","last_unstale":"2023-08-27T11:12:13.932872+0000","last_undegraded":"2023-08-27T11:12:13.932872+0000","last_fullsized":"2023-08-27T11:12:13.932872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:00.165033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967658+0000","last_change":"2023-08-27T11:12:08.918994+0000","last_active":"2023-08-27T11:12:14.967658+0000","last_peered":"2023-08-27T11:12:14.967658+0000","last_clean":"2023-08-27T11:12:14.967658+0000","last_became_active":"2023-08-27T11:12:08.918683+0000","last_became_peered":"2023-08-27T11:12:08.918683+0000","last_unstale":"2023-08-27T11:12:14.967658+0000","last_undegraded":"2023-08-27T11:12:14.967658+0000","last_fullsized":"2023-08-27T11:12:14.967658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:40.545799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934025+0000","last_change":"2023-08-27T11:12:08.911497+0000","last_active":"2023-08-27T11:12:13.934025+0000","last_peered":"2023-08-27T11:12:13.934025+0000","last_clean":"2023-08-27T11:12:13.934025+0000","last_became_active":"2023-08-27T11:12:08.911184+0000","last_became_peered":"2023-08-27T11:12:08.911184+0000","last_unstale":"2023-08-27T11:12:13.934025+0000","last_undegraded":"2023-08-27T11:12:13.934025+0000","last_fullsized":"2023-08-27T11:12:13.934025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:09:45.042697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934063+0000","last_change":"2023-08-27T11:12:08.914949+0000","last_active":"2023-08-27T11:12:13.934063+0000","last_peered":"2023-08-27T11:12:13.934063+0000","last_clean":"2023-08-27T11:12:13.934063+0000","last_became_active":"2023-08-27T11:12:08.914642+0000","last_became_peered":"2023-08-27T11:12:08.914642+0000","last_unstale":"2023-08-27T11:12:13.934063+0000","last_undegraded":"2023-08-27T11:12:13.934063+0000","last_fullsized":"2023-08-27T11:12:13.934063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:38.888108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726727+0000","last_change":"2023-08-27T11:12:08.929257+0000","last_active":"2023-08-27T11:12:14.726727+0000","last_peered":"2023-08-27T11:12:14.726727+0000","last_clean":"2023-08-27T11:12:14.726727+0000","last_became_active":"2023-08-27T11:12:08.928926+0000","last_became_peered":"2023-08-27T11:12:08.928926+0000","last_unstale":"2023-08-27T11:12:14.726727+0000","last_undegraded":"2023-08-27T11:12:14.726727+0000","last_fullsized":"2023-08-27T11:12:14.726727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:09.445014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934838+0000","last_change":"2023-08-27T11:12:08.916024+0000","last_active":"2023-08-27T11:12:13.934838+0000","last_peered":"2023-08-27T11:12:13.934838+0000","last_clean":"2023-08-27T11:12:13.934838+0000","last_became_active":"2023-08-27T11:12:08.913502+0000","last_became_peered":"2023-08-27T11:12:08.913502+0000","last_unstale":"2023-08-27T11:12:13.934838+0000","last_undegraded":"2023-08-27T11:12:13.934838+0000","last_fullsized":"2023-08-27T11:12:13.934838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:50:51.397518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384498+0000","last_change":"2023-08-27T11:12:08.925080+0000","last_active":"2023-08-27T11:12:14.384498+0000","last_peered":"2023-08-27T11:12:14.384498+0000","last_clean":"2023-08-27T11:12:14.384498+0000","last_became_active":"2023-08-27T11:12:08.924784+0000","last_became_peered":"2023-08-27T11:12:08.924784+0000","last_unstale":"2023-08-27T11:12:14.384498+0000","last_undegraded":"2023-08-27T11:12:14.384498+0000","last_fullsized":"2023-08-27T11:12:14.384498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967188+0000","last_change":"2023-08-27T11:12:08.917218+0000","last_active":"2023-08-27T11:12:14.967188+0000","last_peered":"2023-08-27T11:12:14.967188+0000","last_clean":"2023-08-27T11:12:14.967188+0000","last_became_active":"2023-08-27T11:12:08.916984+0000","last_became_peered":"2023-08-27T11:12:08.916984+0000","last_unstale":"2023-08-27T11:12:14.967188+0000","last_undegraded":"2023-08-27T11:12:14.967188+0000","last_fullsized":"2023-08-27T11:12:14.967188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:02.783147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726887+0000","last_change":"2023-08-27T11:12:08.918120+0000","last_active":"2023-08-27T11:12:14.726887+0000","last_peered":"2023-08-27T11:12:14.726887+0000","last_clean":"2023-08-27T11:12:14.726887+0000","last_became_active":"2023-08-27T11:12:08.917811+0000","last_became_peered":"2023-08-27T11:12:08.917811+0000","last_unstale":"2023-08-27T11:12:14.726887+0000","last_undegraded":"2023-08-27T11:12:14.726887+0000","last_fullsized":"2023-08-27T11:12:14.726887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:16.000827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967968+0000","last_change":"2023-08-27T11:12:08.912850+0000","last_active":"2023-08-27T11:12:14.967968+0000","last_peered":"2023-08-27T11:12:14.967968+0000","last_clean":"2023-08-27T11:12:14.967968+0000","last_became_active":"2023-08-27T11:12:08.912458+0000","last_became_peered":"2023-08-27T11:12:08.912458+0000","last_unstale":"2023-08-27T11:12:14.967968+0000","last_undegraded":"2023-08-27T11:12:14.967968+0000","last_fullsized":"2023-08-27T11:12:14.967968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:49.762265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933954+0000","last_change":"2023-08-27T11:12:08.922288+0000","last_active":"2023-08-27T11:12:13.933954+0000","last_peered":"2023-08-27T11:12:13.933954+0000","last_clean":"2023-08-27T11:12:13.933954+0000","last_became_active":"2023-08-27T11:12:08.921531+0000","last_became_peered":"2023-08-27T11:12:08.921531+0000","last_unstale":"2023-08-27T11:12:13.933954+0000","last_undegraded":"2023-08-27T11:12:13.933954+0000","last_fullsized":"2023-08-27T11:12:13.933954+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:20.157877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933160+0000","last_change":"2023-08-27T11:12:08.911702+0000","last_active":"2023-08-27T11:12:13.933160+0000","last_peered":"2023-08-27T11:12:13.933160+0000","last_clean":"2023-08-27T11:12:13.933160+0000","last_became_active":"2023-08-27T11:12:08.911391+0000","last_became_peered":"2023-08-27T11:12:08.911391+0000","last_unstale":"2023-08-27T11:12:13.933160+0000","last_undegraded":"2023-08-27T11:12:13.933160+0000","last_fullsized":"2023-08-27T11:12:13.933160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:53.589324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932909+0000","last_change":"2023-08-27T11:12:08.914165+0000","last_active":"2023-08-27T11:12:13.932909+0000","last_peered":"2023-08-27T11:12:13.932909+0000","last_clean":"2023-08-27T11:12:13.932909+0000","last_became_active":"2023-08-27T11:12:08.913886+0000","last_became_peered":"2023-08-27T11:12:08.913886+0000","last_unstale":"2023-08-27T11:12:13.932909+0000","last_undegraded":"2023-08-27T11:12:13.932909+0000","last_fullsized":"2023-08-27T11:12:13.932909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:12.238621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967356+0000","last_change":"2023-08-27T11:12:08.911899+0000","last_active":"2023-08-27T11:12:14.967356+0000","last_peered":"2023-08-27T11:12:14.967356+0000","last_clean":"2023-08-27T11:12:14.967356+0000","last_became_active":"2023-08-27T11:12:08.911620+0000","last_became_peered":"2023-08-27T11:12:08.911620+0000","last_unstale":"2023-08-27T11:12:14.967356+0000","last_undegraded":"2023-08-27T11:12:14.967356+0000","last_fullsized":"2023-08-27T11:12:14.967356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:07.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384118+0000","last_change":"2023-08-27T11:12:08.925181+0000","last_active":"2023-08-27T11:12:14.384118+0000","last_peered":"2023-08-27T11:12:14.384118+0000","last_clean":"2023-08-27T11:12:14.384118+0000","last_became_active":"2023-08-27T11:12:08.924909+0000","last_became_peered":"2023-08-27T11:12:08.924909+0000","last_unstale":"2023-08-27T11:12:14.384118+0000","last_undegraded":"2023-08-27T11:12:14.384118+0000","last_fullsized":"2023-08-27T11:12:14.384118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:10.885579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934515+0000","last_change":"2023-08-27T11:12:08.911662+0000","last_active":"2023-08-27T11:12:13.934515+0000","last_peered":"2023-08-27T11:12:13.934515+0000","last_clean":"2023-08-27T11:12:13.934515+0000","last_became_active":"2023-08-27T11:12:08.911345+0000","last_became_peered":"2023-08-27T11:12:08.911345+0000","last_unstale":"2023-08-27T11:12:13.934515+0000","last_undegraded":"2023-08-27T11:12:13.934515+0000","last_fullsized":"2023-08-27T11:12:13.934515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:57.932637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726927+0000","last_change":"2023-08-27T11:12:08.916813+0000","last_active":"2023-08-27T11:12:14.726927+0000","last_peered":"2023-08-27T11:12:14.726927+0000","last_clean":"2023-08-27T11:12:14.726927+0000","last_became_active":"2023-08-27T11:12:08.916478+0000","last_became_peered":"2023-08-27T11:12:08.916478+0000","last_unstale":"2023-08-27T11:12:14.726927+0000","last_undegraded":"2023-08-27T11:12:14.726927+0000","last_fullsized":"2023-08-27T11:12:14.726927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:29.674355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933703+0000","last_change":"2023-08-27T11:12:08.907326+0000","last_active":"2023-08-27T11:12:13.933703+0000","last_peered":"2023-08-27T11:12:13.933703+0000","last_clean":"2023-08-27T11:12:13.933703+0000","last_became_active":"2023-08-27T11:12:08.907047+0000","last_became_peered":"2023-08-27T11:12:08.907047+0000","last_unstale":"2023-08-27T11:12:13.933703+0000","last_undegraded":"2023-08-27T11:12:13.933703+0000","last_fullsized":"2023-08-27T11:12:13.933703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:32:17.983727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932728+0000","last_change":"2023-08-27T11:12:08.919764+0000","last_active":"2023-08-27T11:12:13.932728+0000","last_peered":"2023-08-27T11:12:13.932728+0000","last_clean":"2023-08-27T11:12:13.932728+0000","last_became_active":"2023-08-27T11:12:08.914867+0000","last_became_peered":"2023-08-27T11:12:08.914867+0000","last_unstale":"2023-08-27T11:12:13.932728+0000","last_undegraded":"2023-08-27T11:12:13.932728+0000","last_fullsized":"2023-08-27T11:12:13.932728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:05.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933919+0000","last_change":"2023-08-27T11:12:08.922111+0000","last_active":"2023-08-27T11:12:13.933919+0000","last_peered":"2023-08-27T11:12:13.933919+0000","last_clean":"2023-08-27T11:12:13.933919+0000","last_became_active":"2023-08-27T11:12:08.917591+0000","last_became_peered":"2023-08-27T11:12:08.917591+0000","last_unstale":"2023-08-27T11:12:13.933919+0000","last_undegraded":"2023-08-27T11:12:13.933919+0000","last_fullsized":"2023-08-27T11:12:13.933919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:59.881615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384562+0000","last_change":"2023-08-27T11:12:08.931238+0000","last_active":"2023-08-27T11:12:14.384562+0000","last_peered":"2023-08-27T11:12:14.384562+0000","last_clean":"2023-08-27T11:12:14.384562+0000","last_became_active":"2023-08-27T11:12:08.930972+0000","last_became_peered":"2023-08-27T11:12:08.930972+0000","last_unstale":"2023-08-27T11:12:14.384562+0000","last_undegraded":"2023-08-27T11:12:14.384562+0000","last_fullsized":"2023-08-27T11:12:14.384562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:41.570266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726461+0000","last_change":"2023-08-27T11:12:08.919835+0000","last_active":"2023-08-27T11:12:14.726461+0000","last_peered":"2023-08-27T11:12:14.726461+0000","last_clean":"2023-08-27T11:12:14.726461+0000","last_became_active":"2023-08-27T11:12:08.919439+0000","last_became_peered":"2023-08-27T11:12:08.919439+0000","last_unstale":"2023-08-27T11:12:14.726461+0000","last_undegraded":"2023-08-27T11:12:14.726461+0000","last_fullsized":"2023-08-27T11:12:14.726461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:25:20.526417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982143+0000","last_change":"2023-08-27T11:12:08.913113+0000","last_active":"2023-08-27T11:12:14.982143+0000","last_peered":"2023-08-27T11:12:14.982143+0000","last_clean":"2023-08-27T11:12:14.982143+0000","last_became_active":"2023-08-27T11:12:08.912792+0000","last_became_peered":"2023-08-27T11:12:08.912792+0000","last_unstale":"2023-08-27T11:12:14.982143+0000","last_undegraded":"2023-08-27T11:12:14.982143+0000","last_fullsized":"2023-08-27T11:12:14.982143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:53.041694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384187+0000","last_change":"2023-08-27T11:12:08.926432+0000","last_active":"2023-08-27T11:12:14.384187+0000","last_peered":"2023-08-27T11:12:14.384187+0000","last_clean":"2023-08-27T11:12:14.384187+0000","last_became_active":"2023-08-27T11:12:08.926137+0000","last_became_peered":"2023-08-27T11:12:08.926137+0000","last_unstale":"2023-08-27T11:12:14.384187+0000","last_undegraded":"2023-08-27T11:12:14.384187+0000","last_fullsized":"2023-08-27T11:12:14.384187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:07.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838528+0000","last_change":"2023-08-27T11:12:08.919071+0000","last_active":"2023-08-27T11:12:14.838528+0000","last_peered":"2023-08-27T11:12:14.838528+0000","last_clean":"2023-08-27T11:12:14.838528+0000","last_became_active":"2023-08-27T11:12:08.918772+0000","last_became_peered":"2023-08-27T11:12:08.918772+0000","last_unstale":"2023-08-27T11:12:14.838528+0000","last_undegraded":"2023-08-27T11:12:14.838528+0000","last_fullsized":"2023-08-27T11:12:14.838528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:48.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838629+0000","last_change":"2023-08-27T11:12:08.911583+0000","last_active":"2023-08-27T11:12:14.838629+0000","last_peered":"2023-08-27T11:12:14.838629+0000","last_clean":"2023-08-27T11:12:14.838629+0000","last_became_active":"2023-08-27T11:12:08.911295+0000","last_became_peered":"2023-08-27T11:12:08.911295+0000","last_unstale":"2023-08-27T11:12:14.838629+0000","last_undegraded":"2023-08-27T11:12:14.838629+0000","last_fullsized":"2023-08-27T11:12:14.838629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:22.748117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967776+0000","last_change":"2023-08-27T11:12:08.931465+0000","last_active":"2023-08-27T11:12:14.967776+0000","last_peered":"2023-08-27T11:12:14.967776+0000","last_clean":"2023-08-27T11:12:14.967776+0000","last_became_active":"2023-08-27T11:12:08.930966+0000","last_became_peered":"2023-08-27T11:12:08.930966+0000","last_unstale":"2023-08-27T11:12:14.967776+0000","last_undegraded":"2023-08-27T11:12:14.967776+0000","last_fullsized":"2023-08-27T11:12:14.967776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:41.570266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934397+0000","last_change":"2023-08-27T11:12:12.959219+0000","last_active":"2023-08-27T11:12:13.934397+0000","last_peered":"2023-08-27T11:12:13.934397+0000","last_clean":"2023-08-27T11:12:13.934397+0000","last_became_active":"2023-08-27T11:12:12.958817+0000","last_became_peered":"2023-08-27T11:12:12.958817+0000","last_unstale":"2023-08-27T11:12:13.934397+0000","last_undegraded":"2023-08-27T11:12:13.934397+0000","last_fullsized":"2023-08-27T11:12:13.934397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:24:44.882336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933845+0000","last_change":"2023-08-27T11:12:07.903534+0000","last_active":"2023-08-27T11:12:13.933845+0000","last_peered":"2023-08-27T11:12:13.933845+0000","last_clean":"2023-08-27T11:12:13.933845+0000","last_became_active":"2023-08-27T11:12:07.903216+0000","last_became_peered":"2023-08-27T11:12:07.903216+0000","last_unstale":"2023-08-27T11:12:13.933845+0000","last_undegraded":"2023-08-27T11:12:13.933845+0000","last_fullsized":"2023-08-27T11:12:13.933845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:35:15.612642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982446+0000","last_change":"2023-08-27T11:12:08.923471+0000","last_active":"2023-08-27T11:12:14.982446+0000","last_peered":"2023-08-27T11:12:14.982446+0000","last_clean":"2023-08-27T11:12:14.982446+0000","last_became_active":"2023-08-27T11:12:08.918459+0000","last_became_peered":"2023-08-27T11:12:08.918459+0000","last_unstale":"2023-08-27T11:12:14.982446+0000","last_undegraded":"2023-08-27T11:12:14.982446+0000","last_fullsized":"2023-08-27T11:12:14.982446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934474+0000","last_change":"2023-08-27T11:12:12.953626+0000","last_active":"2023-08-27T11:12:13.934474+0000","last_peered":"2023-08-27T11:12:13.934474+0000","last_clean":"2023-08-27T11:12:13.934474+0000","last_became_active":"2023-08-27T11:12:12.953231+0000","last_became_peered":"2023-08-27T11:12:12.953231+0000","last_unstale":"2023-08-27T11:12:13.934474+0000","last_undegraded":"2023-08-27T11:12:13.934474+0000","last_fullsized":"2023-08-27T11:12:13.934474+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:57.631242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837663+0000","last_change":"2023-08-27T11:12:07.895584+0000","last_active":"2023-08-27T11:12:14.837663+0000","last_peered":"2023-08-27T11:12:14.837663+0000","last_clean":"2023-08-27T11:12:14.837663+0000","last_became_active":"2023-08-27T11:12:07.895276+0000","last_became_peered":"2023-08-27T11:12:07.895276+0000","last_unstale":"2023-08-27T11:12:14.837663+0000","last_undegraded":"2023-08-27T11:12:14.837663+0000","last_fullsized":"2023-08-27T11:12:14.837663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:44.245501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726897+0000","last_change":"2023-08-27T11:12:08.928505+0000","last_active":"2023-08-27T11:12:14.726897+0000","last_peered":"2023-08-27T11:12:14.726897+0000","last_clean":"2023-08-27T11:12:14.726897+0000","last_became_active":"2023-08-27T11:12:08.926233+0000","last_became_peered":"2023-08-27T11:12:08.926233+0000","last_unstale":"2023-08-27T11:12:14.726897+0000","last_undegraded":"2023-08-27T11:12:14.726897+0000","last_fullsized":"2023-08-27T11:12:14.726897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:17.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727088+0000","last_change":"2023-08-27T11:12:12.962888+0000","last_active":"2023-08-27T11:12:14.727088+0000","last_peered":"2023-08-27T11:12:14.727088+0000","last_clean":"2023-08-27T11:12:14.727088+0000","last_became_active":"2023-08-27T11:12:12.962501+0000","last_became_peered":"2023-08-27T11:12:12.962501+0000","last_unstale":"2023-08-27T11:12:14.727088+0000","last_undegraded":"2023-08-27T11:12:14.727088+0000","last_fullsized":"2023-08-27T11:12:14.727088+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:06.825066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982310+0000","last_change":"2023-08-27T11:12:07.898500+0000","last_active":"2023-08-27T11:12:14.982310+0000","last_peered":"2023-08-27T11:12:14.982310+0000","last_clean":"2023-08-27T11:12:14.982310+0000","last_became_active":"2023-08-27T11:12:07.898197+0000","last_became_peered":"2023-08-27T11:12:07.898197+0000","last_unstale":"2023-08-27T11:12:14.982310+0000","last_undegraded":"2023-08-27T11:12:14.982310+0000","last_fullsized":"2023-08-27T11:12:14.982310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:39.541198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934586+0000","last_change":"2023-08-27T11:12:08.925037+0000","last_active":"2023-08-27T11:12:13.934586+0000","last_peered":"2023-08-27T11:12:13.934586+0000","last_clean":"2023-08-27T11:12:13.934586+0000","last_became_active":"2023-08-27T11:12:08.924740+0000","last_became_peered":"2023-08-27T11:12:08.924740+0000","last_unstale":"2023-08-27T11:12:13.934586+0000","last_undegraded":"2023-08-27T11:12:13.934586+0000","last_fullsized":"2023-08-27T11:12:13.934586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:21:21.628536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933622+0000","last_change":"2023-08-27T11:12:12.956101+0000","last_active":"2023-08-27T11:12:13.933622+0000","last_peered":"2023-08-27T11:12:13.933622+0000","last_clean":"2023-08-27T11:12:13.933622+0000","last_became_active":"2023-08-27T11:12:12.952220+0000","last_became_peered":"2023-08-27T11:12:12.952220+0000","last_unstale":"2023-08-27T11:12:13.933622+0000","last_undegraded":"2023-08-27T11:12:13.933622+0000","last_fullsized":"2023-08-27T11:12:13.933622+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:16.280540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727033+0000","last_change":"2023-08-27T11:12:07.902491+0000","last_active":"2023-08-27T11:12:14.727033+0000","last_peered":"2023-08-27T11:12:14.727033+0000","last_clean":"2023-08-27T11:12:14.727033+0000","last_became_active":"2023-08-27T11:12:07.902083+0000","last_became_peered":"2023-08-27T11:12:07.902083+0000","last_unstale":"2023-08-27T11:12:14.727033+0000","last_undegraded":"2023-08-27T11:12:14.727033+0000","last_fullsized":"2023-08-27T11:12:14.727033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:19.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982268+0000","last_change":"2023-08-27T11:12:08.918017+0000","last_active":"2023-08-27T11:12:14.982268+0000","last_peered":"2023-08-27T11:12:14.982268+0000","last_clean":"2023-08-27T11:12:14.982268+0000","last_became_active":"2023-08-27T11:12:08.917728+0000","last_became_peered":"2023-08-27T11:12:08.917728+0000","last_unstale":"2023-08-27T11:12:14.982268+0000","last_undegraded":"2023-08-27T11:12:14.982268+0000","last_fullsized":"2023-08-27T11:12:14.982268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:51.458617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384060+0000","last_change":"2023-08-27T11:12:12.958663+0000","last_active":"2023-08-27T11:12:14.384060+0000","last_peered":"2023-08-27T11:12:14.384060+0000","last_clean":"2023-08-27T11:12:14.384060+0000","last_became_active":"2023-08-27T11:12:12.958275+0000","last_became_peered":"2023-08-27T11:12:12.958275+0000","last_unstale":"2023-08-27T11:12:14.384060+0000","last_undegraded":"2023-08-27T11:12:14.384060+0000","last_fullsized":"2023-08-27T11:12:14.384060+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:16.222119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967840+0000","last_change":"2023-08-27T11:12:07.909123+0000","last_active":"2023-08-27T11:12:14.967840+0000","last_peered":"2023-08-27T11:12:14.967840+0000","last_clean":"2023-08-27T11:12:14.967840+0000","last_became_active":"2023-08-27T11:12:07.908788+0000","last_became_peered":"2023-08-27T11:12:07.908788+0000","last_unstale":"2023-08-27T11:12:14.967840+0000","last_undegraded":"2023-08-27T11:12:14.967840+0000","last_fullsized":"2023-08-27T11:12:14.967840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:40.858637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982172+0000","last_change":"2023-08-27T11:12:08.927810+0000","last_active":"2023-08-27T11:12:14.982172+0000","last_peered":"2023-08-27T11:12:14.982172+0000","last_clean":"2023-08-27T11:12:14.982172+0000","last_became_active":"2023-08-27T11:12:08.927231+0000","last_became_peered":"2023-08-27T11:12:08.927231+0000","last_unstale":"2023-08-27T11:12:14.982172+0000","last_undegraded":"2023-08-27T11:12:14.982172+0000","last_fullsized":"2023-08-27T11:12:14.982172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:22.748117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982828+0000","last_change":"2023-08-27T11:12:12.954999+0000","last_active":"2023-08-27T11:12:14.982828+0000","last_peered":"2023-08-27T11:12:14.982828+0000","last_clean":"2023-08-27T11:12:14.982828+0000","last_became_active":"2023-08-27T11:12:12.954572+0000","last_became_peered":"2023-08-27T11:12:12.954572+0000","last_unstale":"2023-08-27T11:12:14.982828+0000","last_undegraded":"2023-08-27T11:12:14.982828+0000","last_fullsized":"2023-08-27T11:12:14.982828+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:38.482531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384847+0000","last_change":"2023-08-27T11:12:07.908859+0000","last_active":"2023-08-27T11:12:14.384847+0000","last_peered":"2023-08-27T11:12:14.384847+0000","last_clean":"2023-08-27T11:12:14.384847+0000","last_became_active":"2023-08-27T11:12:07.908605+0000","last_became_peered":"2023-08-27T11:12:07.908605+0000","last_unstale":"2023-08-27T11:12:14.384847+0000","last_undegraded":"2023-08-27T11:12:14.384847+0000","last_fullsized":"2023-08-27T11:12:14.384847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:20.463561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838554+0000","last_change":"2023-08-27T11:12:08.925960+0000","last_active":"2023-08-27T11:12:14.838554+0000","last_peered":"2023-08-27T11:12:14.838554+0000","last_clean":"2023-08-27T11:12:14.838554+0000","last_became_active":"2023-08-27T11:12:08.925641+0000","last_became_peered":"2023-08-27T11:12:08.925641+0000","last_unstale":"2023-08-27T11:12:14.838554+0000","last_undegraded":"2023-08-27T11:12:14.838554+0000","last_fullsized":"2023-08-27T11:12:14.838554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:53.041694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837918+0000","last_change":"2023-08-27T11:12:12.949220+0000","last_active":"2023-08-27T11:12:14.837918+0000","last_peered":"2023-08-27T11:12:14.837918+0000","last_clean":"2023-08-27T11:12:14.837918+0000","last_became_active":"2023-08-27T11:12:12.948926+0000","last_became_peered":"2023-08-27T11:12:12.948926+0000","last_unstale":"2023-08-27T11:12:14.837918+0000","last_undegraded":"2023-08-27T11:12:14.837918+0000","last_fullsized":"2023-08-27T11:12:14.837918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:20.042745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'10","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:24.935040+0000","last_change":"2023-08-27T11:12:07.908014+0000","last_active":"2023-08-27T11:12:24.935040+0000","last_peered":"2023-08-27T11:12:24.935040+0000","last_clean":"2023-08-27T11:12:24.935040+0000","last_became_active":"2023-08-27T11:12:07.907722+0000","last_became_peered":"2023-08-27T11:12:07.907722+0000","last_unstale":"2023-08-27T11:12:24.935040+0000","last_undegraded":"2023-08-27T11:12:24.935040+0000","last_fullsized":"2023-08-27T11:12:24.935040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:55:11.313074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933433+0000","last_change":"2023-08-27T11:12:08.923838+0000","last_active":"2023-08-27T11:12:13.933433+0000","last_peered":"2023-08-27T11:12:13.933433+0000","last_clean":"2023-08-27T11:12:13.933433+0000","last_became_active":"2023-08-27T11:12:08.919224+0000","last_became_peered":"2023-08-27T11:12:08.919224+0000","last_unstale":"2023-08-27T11:12:13.933433+0000","last_undegraded":"2023-08-27T11:12:13.933433+0000","last_fullsized":"2023-08-27T11:12:13.933433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:49.585442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967969+0000","last_change":"2023-08-27T11:12:12.953888+0000","last_active":"2023-08-27T11:12:14.967969+0000","last_peered":"2023-08-27T11:12:14.967969+0000","last_clean":"2023-08-27T11:12:14.967969+0000","last_became_active":"2023-08-27T11:12:12.953499+0000","last_became_peered":"2023-08-27T11:12:12.953499+0000","last_unstale":"2023-08-27T11:12:14.967969+0000","last_undegraded":"2023-08-27T11:12:14.967969+0000","last_fullsized":"2023-08-27T11:12:14.967969+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:42.319064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384452+0000","last_change":"2023-08-27T11:12:07.907713+0000","last_active":"2023-08-27T11:12:14.384452+0000","last_peered":"2023-08-27T11:12:14.384452+0000","last_clean":"2023-08-27T11:12:14.384452+0000","last_became_active":"2023-08-27T11:12:07.907429+0000","last_became_peered":"2023-08-27T11:12:07.907429+0000","last_unstale":"2023-08-27T11:12:14.384452+0000","last_undegraded":"2023-08-27T11:12:14.384452+0000","last_fullsized":"2023-08-27T11:12:14.384452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:29.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982602+0000","last_change":"2023-08-27T11:12:08.923764+0000","last_active":"2023-08-27T11:12:14.982602+0000","last_peered":"2023-08-27T11:12:14.982602+0000","last_clean":"2023-08-27T11:12:14.982602+0000","last_became_active":"2023-08-27T11:12:08.923333+0000","last_became_peered":"2023-08-27T11:12:08.923333+0000","last_unstale":"2023-08-27T11:12:14.982602+0000","last_undegraded":"2023-08-27T11:12:14.982602+0000","last_fullsized":"2023-08-27T11:12:14.982602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:41.863238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982543+0000","last_change":"2023-08-27T11:12:12.948620+0000","last_active":"2023-08-27T11:12:14.982543+0000","last_peered":"2023-08-27T11:12:14.982543+0000","last_clean":"2023-08-27T11:12:14.982543+0000","last_became_active":"2023-08-27T11:12:12.948161+0000","last_became_peered":"2023-08-27T11:12:12.948161+0000","last_unstale":"2023-08-27T11:12:14.982543+0000","last_undegraded":"2023-08-27T11:12:14.982543+0000","last_fullsized":"2023-08-27T11:12:14.982543+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:25.510081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726415+0000","last_change":"2023-08-27T11:12:07.905444+0000","last_active":"2023-08-27T11:12:14.726415+0000","last_peered":"2023-08-27T11:12:14.726415+0000","last_clean":"2023-08-27T11:12:14.726415+0000","last_became_active":"2023-08-27T11:12:07.905102+0000","last_became_peered":"2023-08-27T11:12:07.905102+0000","last_unstale":"2023-08-27T11:12:14.726415+0000","last_undegraded":"2023-08-27T11:12:14.726415+0000","last_fullsized":"2023-08-27T11:12:14.726415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:59.453384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967752+0000","last_change":"2023-08-27T11:12:08.931630+0000","last_active":"2023-08-27T11:12:14.967752+0000","last_peered":"2023-08-27T11:12:14.967752+0000","last_clean":"2023-08-27T11:12:14.967752+0000","last_became_active":"2023-08-27T11:12:08.931239+0000","last_became_peered":"2023-08-27T11:12:08.931239+0000","last_unstale":"2023-08-27T11:12:14.967752+0000","last_undegraded":"2023-08-27T11:12:14.967752+0000","last_fullsized":"2023-08-27T11:12:14.967752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:51:42.764635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982960+0000","last_change":"2023-08-27T11:12:12.954525+0000","last_active":"2023-08-27T11:12:14.982960+0000","last_peered":"2023-08-27T11:12:14.982960+0000","last_clean":"2023-08-27T11:12:14.982960+0000","last_became_active":"2023-08-27T11:12:12.954077+0000","last_became_peered":"2023-08-27T11:12:12.954077+0000","last_unstale":"2023-08-27T11:12:14.982960+0000","last_undegraded":"2023-08-27T11:12:14.982960+0000","last_fullsized":"2023-08-27T11:12:14.982960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:07:06.449989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384221+0000","last_change":"2023-08-27T11:12:07.907504+0000","last_active":"2023-08-27T11:12:14.384221+0000","last_peered":"2023-08-27T11:12:14.384221+0000","last_clean":"2023-08-27T11:12:14.384221+0000","last_became_active":"2023-08-27T11:12:07.907206+0000","last_became_peered":"2023-08-27T11:12:07.907206+0000","last_unstale":"2023-08-27T11:12:14.384221+0000","last_undegraded":"2023-08-27T11:12:14.384221+0000","last_fullsized":"2023-08-27T11:12:14.384221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:47.521180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726958+0000","last_change":"2023-08-27T11:12:08.918640+0000","last_active":"2023-08-27T11:12:14.726958+0000","last_peered":"2023-08-27T11:12:14.726958+0000","last_clean":"2023-08-27T11:12:14.726958+0000","last_became_active":"2023-08-27T11:12:08.917846+0000","last_became_peered":"2023-08-27T11:12:08.917846+0000","last_unstale":"2023-08-27T11:12:14.726958+0000","last_undegraded":"2023-08-27T11:12:14.726958+0000","last_fullsized":"2023-08-27T11:12:14.726958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:29:50.086138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967688+0000","last_change":"2023-08-27T11:12:12.962983+0000","last_active":"2023-08-27T11:12:14.967688+0000","last_peered":"2023-08-27T11:12:14.967688+0000","last_clean":"2023-08-27T11:12:14.967688+0000","last_became_active":"2023-08-27T11:12:12.962518+0000","last_became_peered":"2023-08-27T11:12:12.962518+0000","last_unstale":"2023-08-27T11:12:14.967688+0000","last_undegraded":"2023-08-27T11:12:14.967688+0000","last_fullsized":"2023-08-27T11:12:14.967688+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:34.829806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932864+0000","last_change":"2023-08-27T11:12:07.905529+0000","last_active":"2023-08-27T11:12:13.932864+0000","last_peered":"2023-08-27T11:12:13.932864+0000","last_clean":"2023-08-27T11:12:13.932864+0000","last_became_active":"2023-08-27T11:12:07.905203+0000","last_became_peered":"2023-08-27T11:12:07.905203+0000","last_unstale":"2023-08-27T11:12:13.932864+0000","last_undegraded":"2023-08-27T11:12:13.932864+0000","last_fullsized":"2023-08-27T11:12:13.932864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:54.273519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.983012+0000","last_change":"2023-08-27T11:12:08.926915+0000","last_active":"2023-08-27T11:12:14.983012+0000","last_peered":"2023-08-27T11:12:14.983012+0000","last_clean":"2023-08-27T11:12:14.983012+0000","last_became_active":"2023-08-27T11:12:08.923355+0000","last_became_peered":"2023-08-27T11:12:08.923355+0000","last_unstale":"2023-08-27T11:12:14.983012+0000","last_undegraded":"2023-08-27T11:12:14.983012+0000","last_fullsized":"2023-08-27T11:12:14.983012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:12.215932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384503+0000","last_change":"2023-08-27T11:12:12.958000+0000","last_active":"2023-08-27T11:12:14.384503+0000","last_peered":"2023-08-27T11:12:14.384503+0000","last_clean":"2023-08-27T11:12:14.384503+0000","last_became_active":"2023-08-27T11:12:12.957611+0000","last_became_peered":"2023-08-27T11:12:12.957611+0000","last_unstale":"2023-08-27T11:12:14.384503+0000","last_undegraded":"2023-08-27T11:12:14.384503+0000","last_fullsized":"2023-08-27T11:12:14.384503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:44.587717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384852+0000","last_change":"2023-08-27T11:12:07.907385+0000","last_active":"2023-08-27T11:12:14.384852+0000","last_peered":"2023-08-27T11:12:14.384852+0000","last_clean":"2023-08-27T11:12:14.384852+0000","last_became_active":"2023-08-27T11:12:07.907116+0000","last_became_peered":"2023-08-27T11:12:07.907116+0000","last_unstale":"2023-08-27T11:12:14.384852+0000","last_undegraded":"2023-08-27T11:12:14.384852+0000","last_fullsized":"2023-08-27T11:12:14.384852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:37.272544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967084+0000","last_change":"2023-08-27T11:12:08.927597+0000","last_active":"2023-08-27T11:12:14.967084+0000","last_peered":"2023-08-27T11:12:14.967084+0000","last_clean":"2023-08-27T11:12:14.967084+0000","last_became_active":"2023-08-27T11:12:08.927306+0000","last_became_peered":"2023-08-27T11:12:08.927306+0000","last_unstale":"2023-08-27T11:12:14.967084+0000","last_undegraded":"2023-08-27T11:12:14.967084+0000","last_fullsized":"2023-08-27T11:12:14.967084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:20.782136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932501+0000","last_change":"2023-08-27T11:12:12.955644+0000","last_active":"2023-08-27T11:12:13.932501+0000","last_peered":"2023-08-27T11:12:13.932501+0000","last_clean":"2023-08-27T11:12:13.932501+0000","last_became_active":"2023-08-27T11:12:12.952322+0000","last_became_peered":"2023-08-27T11:12:12.952322+0000","last_unstale":"2023-08-27T11:12:13.932501+0000","last_undegraded":"2023-08-27T11:12:13.932501+0000","last_fullsized":"2023-08-27T11:12:13.932501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:16:14.569342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384846+0000","last_change":"2023-08-27T11:12:07.907495+0000","last_active":"2023-08-27T11:12:14.384846+0000","last_peered":"2023-08-27T11:12:14.384846+0000","last_clean":"2023-08-27T11:12:14.384846+0000","last_became_active":"2023-08-27T11:12:07.907216+0000","last_became_peered":"2023-08-27T11:12:07.907216+0000","last_unstale":"2023-08-27T11:12:14.384846+0000","last_undegraded":"2023-08-27T11:12:14.384846+0000","last_fullsized":"2023-08-27T11:12:14.384846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:01:40.858637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933350+0000","last_change":"2023-08-27T11:12:08.924297+0000","last_active":"2023-08-27T11:12:13.933350+0000","last_peered":"2023-08-27T11:12:13.933350+0000","last_clean":"2023-08-27T11:12:13.933350+0000","last_became_active":"2023-08-27T11:12:08.919603+0000","last_became_peered":"2023-08-27T11:12:08.919603+0000","last_unstale":"2023-08-27T11:12:13.933350+0000","last_undegraded":"2023-08-27T11:12:13.933350+0000","last_fullsized":"2023-08-27T11:12:13.933350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:20:35.025362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932779+0000","last_change":"2023-08-27T11:12:12.954662+0000","last_active":"2023-08-27T11:12:13.932779+0000","last_peered":"2023-08-27T11:12:13.932779+0000","last_clean":"2023-08-27T11:12:13.932779+0000","last_became_active":"2023-08-27T11:12:12.954250+0000","last_became_peered":"2023-08-27T11:12:12.954250+0000","last_unstale":"2023-08-27T11:12:13.932779+0000","last_undegraded":"2023-08-27T11:12:13.932779+0000","last_fullsized":"2023-08-27T11:12:13.932779+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:50.028006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384990+0000","last_change":"2023-08-27T11:12:07.907454+0000","last_active":"2023-08-27T11:12:14.384990+0000","last_peered":"2023-08-27T11:12:14.384990+0000","last_clean":"2023-08-27T11:12:14.384990+0000","last_became_active":"2023-08-27T11:12:07.907161+0000","last_became_peered":"2023-08-27T11:12:07.907161+0000","last_unstale":"2023-08-27T11:12:14.384990+0000","last_undegraded":"2023-08-27T11:12:14.384990+0000","last_fullsized":"2023-08-27T11:12:14.384990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:33.436011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933127+0000","last_change":"2023-08-27T11:12:08.920905+0000","last_active":"2023-08-27T11:12:13.933127+0000","last_peered":"2023-08-27T11:12:13.933127+0000","last_clean":"2023-08-27T11:12:13.933127+0000","last_became_active":"2023-08-27T11:12:08.920628+0000","last_became_peered":"2023-08-27T11:12:08.920628+0000","last_unstale":"2023-08-27T11:12:13.933127+0000","last_undegraded":"2023-08-27T11:12:13.933127+0000","last_fullsized":"2023-08-27T11:12:13.933127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:05.481805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932699+0000","last_change":"2023-08-27T11:12:12.953446+0000","last_active":"2023-08-27T11:12:13.932699+0000","last_peered":"2023-08-27T11:12:13.932699+0000","last_clean":"2023-08-27T11:12:13.932699+0000","last_became_active":"2023-08-27T11:12:12.953019+0000","last_became_peered":"2023-08-27T11:12:12.953019+0000","last_unstale":"2023-08-27T11:12:13.932699+0000","last_undegraded":"2023-08-27T11:12:13.932699+0000","last_fullsized":"2023-08-27T11:12:13.932699+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:11.191056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967365+0000","last_change":"2023-08-27T11:12:07.896745+0000","last_active":"2023-08-27T11:12:14.967365+0000","last_peered":"2023-08-27T11:12:14.967365+0000","last_clean":"2023-08-27T11:12:14.967365+0000","last_became_active":"2023-08-27T11:12:07.896471+0000","last_became_peered":"2023-08-27T11:12:07.896471+0000","last_unstale":"2023-08-27T11:12:14.967365+0000","last_undegraded":"2023-08-27T11:12:14.967365+0000","last_fullsized":"2023-08-27T11:12:14.967365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:09.470216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726539+0000","last_change":"2023-08-27T11:12:08.918280+0000","last_active":"2023-08-27T11:12:14.726539+0000","last_peered":"2023-08-27T11:12:14.726539+0000","last_clean":"2023-08-27T11:12:14.726539+0000","last_became_active":"2023-08-27T11:12:08.917786+0000","last_became_peered":"2023-08-27T11:12:08.917786+0000","last_unstale":"2023-08-27T11:12:14.726539+0000","last_undegraded":"2023-08-27T11:12:14.726539+0000","last_fullsized":"2023-08-27T11:12:14.726539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:45.655952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932983+0000","last_change":"2023-08-27T11:12:12.957044+0000","last_active":"2023-08-27T11:12:13.932983+0000","last_peered":"2023-08-27T11:12:13.932983+0000","last_clean":"2023-08-27T11:12:13.932983+0000","last_became_active":"2023-08-27T11:12:12.953571+0000","last_became_peered":"2023-08-27T11:12:12.953571+0000","last_unstale":"2023-08-27T11:12:13.932983+0000","last_undegraded":"2023-08-27T11:12:13.932983+0000","last_fullsized":"2023-08-27T11:12:13.932983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:42.930026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982654+0000","last_change":"2023-08-27T11:12:07.899209+0000","last_active":"2023-08-27T11:12:14.982654+0000","last_peered":"2023-08-27T11:12:14.982654+0000","last_clean":"2023-08-27T11:12:14.982654+0000","last_became_active":"2023-08-27T11:12:07.898806+0000","last_became_peered":"2023-08-27T11:12:07.898806+0000","last_unstale":"2023-08-27T11:12:14.982654+0000","last_undegraded":"2023-08-27T11:12:14.982654+0000","last_fullsized":"2023-08-27T11:12:14.982654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:37.272544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933574+0000","last_change":"2023-08-27T11:12:08.919614+0000","last_active":"2023-08-27T11:12:13.933574+0000","last_peered":"2023-08-27T11:12:13.933574+0000","last_clean":"2023-08-27T11:12:13.933574+0000","last_became_active":"2023-08-27T11:12:08.916338+0000","last_became_peered":"2023-08-27T11:12:08.916338+0000","last_unstale":"2023-08-27T11:12:13.933574+0000","last_undegraded":"2023-08-27T11:12:13.933574+0000","last_fullsized":"2023-08-27T11:12:13.933574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:49:01.974112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838031+0000","last_change":"2023-08-27T11:12:12.962859+0000","last_active":"2023-08-27T11:12:14.838031+0000","last_peered":"2023-08-27T11:12:14.838031+0000","last_clean":"2023-08-27T11:12:14.838031+0000","last_became_active":"2023-08-27T11:12:12.962421+0000","last_became_peered":"2023-08-27T11:12:12.962421+0000","last_unstale":"2023-08-27T11:12:14.838031+0000","last_undegraded":"2023-08-27T11:12:14.838031+0000","last_fullsized":"2023-08-27T11:12:14.838031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:21.753280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933251+0000","last_change":"2023-08-27T11:12:07.901923+0000","last_active":"2023-08-27T11:12:13.933251+0000","last_peered":"2023-08-27T11:12:13.933251+0000","last_clean":"2023-08-27T11:12:13.933251+0000","last_became_active":"2023-08-27T11:12:07.901634+0000","last_became_peered":"2023-08-27T11:12:07.901634+0000","last_unstale":"2023-08-27T11:12:13.933251+0000","last_undegraded":"2023-08-27T11:12:13.933251+0000","last_fullsized":"2023-08-27T11:12:13.933251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:32:44.981487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933505+0000","last_change":"2023-08-27T11:12:08.911380+0000","last_active":"2023-08-27T11:12:13.933505+0000","last_peered":"2023-08-27T11:12:13.933505+0000","last_clean":"2023-08-27T11:12:13.933505+0000","last_became_active":"2023-08-27T11:12:08.911077+0000","last_became_peered":"2023-08-27T11:12:08.911077+0000","last_unstale":"2023-08-27T11:12:13.933505+0000","last_undegraded":"2023-08-27T11:12:13.933505+0000","last_fullsized":"2023-08-27T11:12:13.933505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:59.416171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967787+0000","last_change":"2023-08-27T11:12:12.953058+0000","last_active":"2023-08-27T11:12:14.967787+0000","last_peered":"2023-08-27T11:12:14.967787+0000","last_clean":"2023-08-27T11:12:14.967787+0000","last_became_active":"2023-08-27T11:12:12.952571+0000","last_became_peered":"2023-08-27T11:12:12.952571+0000","last_unstale":"2023-08-27T11:12:14.967787+0000","last_undegraded":"2023-08-27T11:12:14.967787+0000","last_fullsized":"2023-08-27T11:12:14.967787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:25.510081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933016+0000","last_change":"2023-08-27T11:12:07.903262+0000","last_active":"2023-08-27T11:12:13.933016+0000","last_peered":"2023-08-27T11:12:13.933016+0000","last_clean":"2023-08-27T11:12:13.933016+0000","last_became_active":"2023-08-27T11:12:07.902549+0000","last_became_peered":"2023-08-27T11:12:07.902549+0000","last_unstale":"2023-08-27T11:12:13.933016+0000","last_undegraded":"2023-08-27T11:12:13.933016+0000","last_fullsized":"2023-08-27T11:12:13.933016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:25.089023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933460+0000","last_change":"2023-08-27T11:12:08.919554+0000","last_active":"2023-08-27T11:12:13.933460+0000","last_peered":"2023-08-27T11:12:13.933460+0000","last_clean":"2023-08-27T11:12:13.933460+0000","last_became_active":"2023-08-27T11:12:08.919252+0000","last_became_peered":"2023-08-27T11:12:08.919252+0000","last_unstale":"2023-08-27T11:12:13.933460+0000","last_undegraded":"2023-08-27T11:12:13.933460+0000","last_fullsized":"2023-08-27T11:12:13.933460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:14:35.778200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933262+0000","last_change":"2023-08-27T11:12:12.954063+0000","last_active":"2023-08-27T11:12:13.933262+0000","last_peered":"2023-08-27T11:12:13.933262+0000","last_clean":"2023-08-27T11:12:13.933262+0000","last_became_active":"2023-08-27T11:12:12.953652+0000","last_became_peered":"2023-08-27T11:12:12.953652+0000","last_unstale":"2023-08-27T11:12:13.933262+0000","last_undegraded":"2023-08-27T11:12:13.933262+0000","last_fullsized":"2023-08-27T11:12:13.933262+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:20:31.699925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982128+0000","last_change":"2023-08-27T11:12:07.902468+0000","last_active":"2023-08-27T11:12:14.982128+0000","last_peered":"2023-08-27T11:12:14.982128+0000","last_clean":"2023-08-27T11:12:14.982128+0000","last_became_active":"2023-08-27T11:12:07.902193+0000","last_became_peered":"2023-08-27T11:12:07.902193+0000","last_unstale":"2023-08-27T11:12:14.982128+0000","last_undegraded":"2023-08-27T11:12:14.982128+0000","last_fullsized":"2023-08-27T11:12:14.982128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:06.455977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838774+0000","last_change":"2023-08-27T11:12:08.921979+0000","last_active":"2023-08-27T11:12:14.838774+0000","last_peered":"2023-08-27T11:12:14.838774+0000","last_clean":"2023-08-27T11:12:14.838774+0000","last_became_active":"2023-08-27T11:12:08.921659+0000","last_became_peered":"2023-08-27T11:12:08.921659+0000","last_unstale":"2023-08-27T11:12:14.838774+0000","last_undegraded":"2023-08-27T11:12:14.838774+0000","last_fullsized":"2023-08-27T11:12:14.838774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:22:40.545799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726360+0000","last_change":"2023-08-27T11:12:12.959327+0000","last_active":"2023-08-27T11:12:14.726360+0000","last_peered":"2023-08-27T11:12:14.726360+0000","last_clean":"2023-08-27T11:12:14.726360+0000","last_became_active":"2023-08-27T11:12:12.957064+0000","last_became_peered":"2023-08-27T11:12:12.957064+0000","last_unstale":"2023-08-27T11:12:14.726360+0000","last_undegraded":"2023-08-27T11:12:14.726360+0000","last_fullsized":"2023-08-27T11:12:14.726360+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:20.210306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838189+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:12:14.838189+0000","last_peered":"2023-08-27T11:12:14.838189+0000","last_clean":"2023-08-27T11:12:14.838189+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:12:14.838189+0000","last_undegraded":"2023-08-27T11:12:14.838189+0000","last_fullsized":"2023-08-27T11:12:14.838189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:04.448471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967607+0000","last_change":"2023-08-27T11:12:07.902012+0000","last_active":"2023-08-27T11:12:14.967607+0000","last_peered":"2023-08-27T11:12:14.967607+0000","last_clean":"2023-08-27T11:12:14.967607+0000","last_became_active":"2023-08-27T11:12:07.901706+0000","last_became_peered":"2023-08-27T11:12:07.901706+0000","last_unstale":"2023-08-27T11:12:14.967607+0000","last_undegraded":"2023-08-27T11:12:14.967607+0000","last_fullsized":"2023-08-27T11:12:14.967607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:21.743516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838056+0000","last_change":"2023-08-27T11:12:08.916047+0000","last_active":"2023-08-27T11:12:14.838056+0000","last_peered":"2023-08-27T11:12:14.838056+0000","last_clean":"2023-08-27T11:12:14.838056+0000","last_became_active":"2023-08-27T11:12:08.915742+0000","last_became_peered":"2023-08-27T11:12:08.915742+0000","last_unstale":"2023-08-27T11:12:14.838056+0000","last_undegraded":"2023-08-27T11:12:14.838056+0000","last_fullsized":"2023-08-27T11:12:14.838056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:43:29.674355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726449+0000","last_change":"2023-08-27T11:12:12.952690+0000","last_active":"2023-08-27T11:12:14.726449+0000","last_peered":"2023-08-27T11:12:14.726449+0000","last_clean":"2023-08-27T11:12:14.726449+0000","last_became_active":"2023-08-27T11:12:12.952209+0000","last_became_peered":"2023-08-27T11:12:12.952209+0000","last_unstale":"2023-08-27T11:12:14.726449+0000","last_undegraded":"2023-08-27T11:12:14.726449+0000","last_fullsized":"2023-08-27T11:12:14.726449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:28.731513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967363+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:12:14.967363+0000","last_peered":"2023-08-27T11:12:14.967363+0000","last_clean":"2023-08-27T11:12:14.967363+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:12:14.967363+0000","last_undegraded":"2023-08-27T11:12:14.967363+0000","last_fullsized":"2023-08-27T11:12:14.967363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:04.859233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933791+0000","last_change":"2023-08-27T11:12:07.905363+0000","last_active":"2023-08-27T11:12:13.933791+0000","last_peered":"2023-08-27T11:12:13.933791+0000","last_clean":"2023-08-27T11:12:13.933791+0000","last_became_active":"2023-08-27T11:12:07.905002+0000","last_became_peered":"2023-08-27T11:12:07.905002+0000","last_unstale":"2023-08-27T11:12:13.933791+0000","last_undegraded":"2023-08-27T11:12:13.933791+0000","last_fullsized":"2023-08-27T11:12:13.933791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:06:55.872046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838055+0000","last_change":"2023-08-27T11:12:08.920259+0000","last_active":"2023-08-27T11:12:14.838055+0000","last_peered":"2023-08-27T11:12:14.838055+0000","last_clean":"2023-08-27T11:12:14.838055+0000","last_became_active":"2023-08-27T11:12:08.916978+0000","last_became_peered":"2023-08-27T11:12:08.916978+0000","last_unstale":"2023-08-27T11:12:14.838055+0000","last_undegraded":"2023-08-27T11:12:14.838055+0000","last_fullsized":"2023-08-27T11:12:14.838055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:29:50.086138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933007+0000","last_change":"2023-08-27T11:12:12.959735+0000","last_active":"2023-08-27T11:12:13.933007+0000","last_peered":"2023-08-27T11:12:13.933007+0000","last_clean":"2023-08-27T11:12:13.933007+0000","last_became_active":"2023-08-27T11:12:12.959275+0000","last_became_peered":"2023-08-27T11:12:12.959275+0000","last_unstale":"2023-08-27T11:12:13.933007+0000","last_undegraded":"2023-08-27T11:12:13.933007+0000","last_fullsized":"2023-08-27T11:12:13.933007+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:57.120477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932550+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:12:13.932550+0000","last_peered":"2023-08-27T11:12:13.932550+0000","last_clean":"2023-08-27T11:12:13.932550+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:12:13.932550+0000","last_undegraded":"2023-08-27T11:12:13.932550+0000","last_fullsized":"2023-08-27T11:12:13.932550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:49.251775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.837761+0000","last_change":"2023-08-27T11:12:07.902482+0000","last_active":"2023-08-27T11:12:14.837761+0000","last_peered":"2023-08-27T11:12:14.837761+0000","last_clean":"2023-08-27T11:12:14.837761+0000","last_became_active":"2023-08-27T11:12:07.902184+0000","last_became_peered":"2023-08-27T11:12:07.902184+0000","last_unstale":"2023-08-27T11:12:14.837761+0000","last_undegraded":"2023-08-27T11:12:14.837761+0000","last_fullsized":"2023-08-27T11:12:14.837761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:46.293833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838666+0000","last_change":"2023-08-27T11:12:08.919949+0000","last_active":"2023-08-27T11:12:14.838666+0000","last_peered":"2023-08-27T11:12:14.838666+0000","last_clean":"2023-08-27T11:12:14.838666+0000","last_became_active":"2023-08-27T11:12:08.918796+0000","last_became_peered":"2023-08-27T11:12:08.918796+0000","last_unstale":"2023-08-27T11:12:14.838666+0000","last_undegraded":"2023-08-27T11:12:14.838666+0000","last_fullsized":"2023-08-27T11:12:14.838666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:51.458617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384661+0000","last_change":"2023-08-27T11:12:12.965405+0000","last_active":"2023-08-27T11:12:14.384661+0000","last_peered":"2023-08-27T11:12:14.384661+0000","last_clean":"2023-08-27T11:12:14.384661+0000","last_became_active":"2023-08-27T11:12:12.965085+0000","last_became_peered":"2023-08-27T11:12:12.965085+0000","last_unstale":"2023-08-27T11:12:14.384661+0000","last_undegraded":"2023-08-27T11:12:14.384661+0000","last_fullsized":"2023-08-27T11:12:14.384661+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:45.516388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967468+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:12:14.967468+0000","last_peered":"2023-08-27T11:12:14.967468+0000","last_clean":"2023-08-27T11:12:14.967468+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:12:14.967468+0000","last_undegraded":"2023-08-27T11:12:14.967468+0000","last_fullsized":"2023-08-27T11:12:14.967468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:47.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384165+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:12:14.384165+0000","last_peered":"2023-08-27T11:12:14.384165+0000","last_clean":"2023-08-27T11:12:14.384165+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:12:14.384165+0000","last_undegraded":"2023-08-27T11:12:14.384165+0000","last_fullsized":"2023-08-27T11:12:14.384165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:10.078531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982472+0000","last_change":"2023-08-27T11:12:07.903434+0000","last_active":"2023-08-27T11:12:14.982472+0000","last_peered":"2023-08-27T11:12:14.982472+0000","last_clean":"2023-08-27T11:12:14.982472+0000","last_became_active":"2023-08-27T11:12:07.903142+0000","last_became_peered":"2023-08-27T11:12:07.903142+0000","last_unstale":"2023-08-27T11:12:14.982472+0000","last_undegraded":"2023-08-27T11:12:14.982472+0000","last_fullsized":"2023-08-27T11:12:14.982472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:42:40.565665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934388+0000","last_change":"2023-08-27T11:12:08.915968+0000","last_active":"2023-08-27T11:12:13.934388+0000","last_peered":"2023-08-27T11:12:13.934388+0000","last_clean":"2023-08-27T11:12:13.934388+0000","last_became_active":"2023-08-27T11:12:08.915646+0000","last_became_peered":"2023-08-27T11:12:08.915646+0000","last_unstale":"2023-08-27T11:12:13.934388+0000","last_undegraded":"2023-08-27T11:12:13.934388+0000","last_fullsized":"2023-08-27T11:12:13.934388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:53.078559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967776+0000","last_change":"2023-08-27T11:12:12.953539+0000","last_active":"2023-08-27T11:12:14.967776+0000","last_peered":"2023-08-27T11:12:14.967776+0000","last_clean":"2023-08-27T11:12:14.967776+0000","last_became_active":"2023-08-27T11:12:12.953154+0000","last_became_peered":"2023-08-27T11:12:12.953154+0000","last_unstale":"2023-08-27T11:12:14.967776+0000","last_undegraded":"2023-08-27T11:12:14.967776+0000","last_fullsized":"2023-08-27T11:12:14.967776+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:45.516388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384351+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:12:14.384351+0000","last_peered":"2023-08-27T11:12:14.384351+0000","last_clean":"2023-08-27T11:12:14.384351+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:12:14.384351+0000","last_undegraded":"2023-08-27T11:12:14.384351+0000","last_fullsized":"2023-08-27T11:12:14.384351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:29:47.015349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.725986+0000","last_change":"2023-08-27T11:12:07.899116+0000","last_active":"2023-08-27T11:12:14.725986+0000","last_peered":"2023-08-27T11:12:14.725986+0000","last_clean":"2023-08-27T11:12:14.725986+0000","last_became_active":"2023-08-27T11:12:07.898805+0000","last_became_peered":"2023-08-27T11:12:07.898805+0000","last_unstale":"2023-08-27T11:12:14.725986+0000","last_undegraded":"2023-08-27T11:12:14.725986+0000","last_fullsized":"2023-08-27T11:12:14.725986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:31:07.168983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726764+0000","last_change":"2023-08-27T11:12:08.918851+0000","last_active":"2023-08-27T11:12:14.726764+0000","last_peered":"2023-08-27T11:12:14.726764+0000","last_clean":"2023-08-27T11:12:14.726764+0000","last_became_active":"2023-08-27T11:12:08.918248+0000","last_became_peered":"2023-08-27T11:12:08.918248+0000","last_unstale":"2023-08-27T11:12:14.726764+0000","last_undegraded":"2023-08-27T11:12:14.726764+0000","last_fullsized":"2023-08-27T11:12:14.726764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:45.250102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933527+0000","last_change":"2023-08-27T11:12:12.959470+0000","last_active":"2023-08-27T11:12:13.933527+0000","last_peered":"2023-08-27T11:12:13.933527+0000","last_clean":"2023-08-27T11:12:13.933527+0000","last_became_active":"2023-08-27T11:12:12.959082+0000","last_became_peered":"2023-08-27T11:12:12.959082+0000","last_unstale":"2023-08-27T11:12:13.933527+0000","last_undegraded":"2023-08-27T11:12:13.933527+0000","last_fullsized":"2023-08-27T11:12:13.933527+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:04.206952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384869+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:12:14.384869+0000","last_peered":"2023-08-27T11:12:14.384869+0000","last_clean":"2023-08-27T11:12:14.384869+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:12:14.384869+0000","last_undegraded":"2023-08-27T11:12:14.384869+0000","last_fullsized":"2023-08-27T11:12:14.384869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:43.735919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838283+0000","last_change":"2023-08-27T11:12:07.903460+0000","last_active":"2023-08-27T11:12:14.838283+0000","last_peered":"2023-08-27T11:12:14.838283+0000","last_clean":"2023-08-27T11:12:14.838283+0000","last_became_active":"2023-08-27T11:12:07.903110+0000","last_became_peered":"2023-08-27T11:12:07.903110+0000","last_unstale":"2023-08-27T11:12:14.838283+0000","last_undegraded":"2023-08-27T11:12:14.838283+0000","last_fullsized":"2023-08-27T11:12:14.838283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:09.880978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726625+0000","last_change":"2023-08-27T11:12:08.913520+0000","last_active":"2023-08-27T11:12:14.726625+0000","last_peered":"2023-08-27T11:12:14.726625+0000","last_clean":"2023-08-27T11:12:14.726625+0000","last_became_active":"2023-08-27T11:12:08.913192+0000","last_became_peered":"2023-08-27T11:12:08.913192+0000","last_unstale":"2023-08-27T11:12:14.726625+0000","last_undegraded":"2023-08-27T11:12:14.726625+0000","last_fullsized":"2023-08-27T11:12:14.726625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:12:06.235129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933906+0000","last_change":"2023-08-27T11:12:12.959458+0000","last_active":"2023-08-27T11:12:13.933906+0000","last_peered":"2023-08-27T11:12:13.933906+0000","last_clean":"2023-08-27T11:12:13.933906+0000","last_became_active":"2023-08-27T11:12:12.959073+0000","last_became_peered":"2023-08-27T11:12:12.959073+0000","last_unstale":"2023-08-27T11:12:13.933906+0000","last_undegraded":"2023-08-27T11:12:13.933906+0000","last_fullsized":"2023-08-27T11:12:13.933906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:37:24.619850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.983100+0000","last_change":"2023-08-27T11:12:07.908908+0000","last_active":"2023-08-27T11:12:14.983100+0000","last_peered":"2023-08-27T11:12:14.983100+0000","last_clean":"2023-08-27T11:12:14.983100+0000","last_became_active":"2023-08-27T11:12:07.908670+0000","last_became_peered":"2023-08-27T11:12:07.908670+0000","last_unstale":"2023-08-27T11:12:14.983100+0000","last_undegraded":"2023-08-27T11:12:14.983100+0000","last_fullsized":"2023-08-27T11:12:14.983100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:06:59.757966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838348+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:12:14.838348+0000","last_peered":"2023-08-27T11:12:14.838348+0000","last_clean":"2023-08-27T11:12:14.838348+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:12:14.838348+0000","last_undegraded":"2023-08-27T11:12:14.838348+0000","last_fullsized":"2023-08-27T11:12:14.838348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:56:01.434232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982129+0000","last_change":"2023-08-27T11:12:08.917056+0000","last_active":"2023-08-27T11:12:14.982129+0000","last_peered":"2023-08-27T11:12:14.982129+0000","last_clean":"2023-08-27T11:12:14.982129+0000","last_became_active":"2023-08-27T11:12:08.916853+0000","last_became_peered":"2023-08-27T11:12:08.916853+0000","last_unstale":"2023-08-27T11:12:14.982129+0000","last_undegraded":"2023-08-27T11:12:14.982129+0000","last_fullsized":"2023-08-27T11:12:14.982129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:48.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982032+0000","last_change":"2023-08-27T11:12:12.955385+0000","last_active":"2023-08-27T11:12:14.982032+0000","last_peered":"2023-08-27T11:12:14.982032+0000","last_clean":"2023-08-27T11:12:14.982032+0000","last_became_active":"2023-08-27T11:12:12.953960+0000","last_became_peered":"2023-08-27T11:12:12.953960+0000","last_unstale":"2023-08-27T11:12:14.982032+0000","last_undegraded":"2023-08-27T11:12:14.982032+0000","last_fullsized":"2023-08-27T11:12:14.982032+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:14.927498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934591+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:12:13.934591+0000","last_peered":"2023-08-27T11:12:13.934591+0000","last_clean":"2023-08-27T11:12:13.934591+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:12:13.934591+0000","last_undegraded":"2023-08-27T11:12:13.934591+0000","last_fullsized":"2023-08-27T11:12:13.934591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:05:01.448436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726808+0000","last_change":"2023-08-27T11:12:07.902830+0000","last_active":"2023-08-27T11:12:14.726808+0000","last_peered":"2023-08-27T11:12:14.726808+0000","last_clean":"2023-08-27T11:12:14.726808+0000","last_became_active":"2023-08-27T11:12:07.902452+0000","last_became_peered":"2023-08-27T11:12:07.902452+0000","last_unstale":"2023-08-27T11:12:14.726808+0000","last_undegraded":"2023-08-27T11:12:14.726808+0000","last_fullsized":"2023-08-27T11:12:14.726808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:22:46.293833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933163+0000","last_change":"2023-08-27T11:12:08.919355+0000","last_active":"2023-08-27T11:12:13.933163+0000","last_peered":"2023-08-27T11:12:13.933163+0000","last_clean":"2023-08-27T11:12:13.933163+0000","last_became_active":"2023-08-27T11:12:08.917203+0000","last_became_peered":"2023-08-27T11:12:08.917203+0000","last_unstale":"2023-08-27T11:12:13.933163+0000","last_undegraded":"2023-08-27T11:12:13.933163+0000","last_fullsized":"2023-08-27T11:12:13.933163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:07:00.762567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838062+0000","last_change":"2023-08-27T11:12:12.963107+0000","last_active":"2023-08-27T11:12:14.838062+0000","last_peered":"2023-08-27T11:12:14.838062+0000","last_clean":"2023-08-27T11:12:14.838062+0000","last_became_active":"2023-08-27T11:12:12.962714+0000","last_became_peered":"2023-08-27T11:12:12.962714+0000","last_unstale":"2023-08-27T11:12:14.838062+0000","last_undegraded":"2023-08-27T11:12:14.838062+0000","last_fullsized":"2023-08-27T11:12:14.838062+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:14:24.824055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.726473+0000","last_change":"2023-08-27T11:12:07.900588+0000","last_active":"2023-08-27T11:12:14.726473+0000","last_peered":"2023-08-27T11:12:14.726473+0000","last_clean":"2023-08-27T11:12:14.726473+0000","last_became_active":"2023-08-27T11:12:07.900223+0000","last_became_peered":"2023-08-27T11:12:07.900223+0000","last_unstale":"2023-08-27T11:12:14.726473+0000","last_undegraded":"2023-08-27T11:12:14.726473+0000","last_fullsized":"2023-08-27T11:12:14.726473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:54.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933266+0000","last_change":"2023-08-27T11:12:08.921029+0000","last_active":"2023-08-27T11:12:13.933266+0000","last_peered":"2023-08-27T11:12:13.933266+0000","last_clean":"2023-08-27T11:12:13.933266+0000","last_became_active":"2023-08-27T11:12:08.920756+0000","last_became_peered":"2023-08-27T11:12:08.920756+0000","last_unstale":"2023-08-27T11:12:13.933266+0000","last_undegraded":"2023-08-27T11:12:13.933266+0000","last_fullsized":"2023-08-27T11:12:13.933266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:21:21.628536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.384440+0000","last_change":"2023-08-27T11:12:12.958972+0000","last_active":"2023-08-27T11:12:14.384440+0000","last_peered":"2023-08-27T11:12:14.384440+0000","last_clean":"2023-08-27T11:12:14.384440+0000","last_became_active":"2023-08-27T11:12:12.958605+0000","last_became_peered":"2023-08-27T11:12:12.958605+0000","last_unstale":"2023-08-27T11:12:14.384440+0000","last_undegraded":"2023-08-27T11:12:14.384440+0000","last_fullsized":"2023-08-27T11:12:14.384440+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:55.500535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967443+0000","last_change":"2023-08-27T11:12:07.898515+0000","last_active":"2023-08-27T11:12:14.967443+0000","last_peered":"2023-08-27T11:12:14.967443+0000","last_clean":"2023-08-27T11:12:14.967443+0000","last_became_active":"2023-08-27T11:12:07.898243+0000","last_became_peered":"2023-08-27T11:12:07.898243+0000","last_unstale":"2023-08-27T11:12:14.967443+0000","last_undegraded":"2023-08-27T11:12:14.967443+0000","last_fullsized":"2023-08-27T11:12:14.967443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:47.521180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934374+0000","last_change":"2023-08-27T11:12:08.919278+0000","last_active":"2023-08-27T11:12:13.934374+0000","last_peered":"2023-08-27T11:12:13.934374+0000","last_clean":"2023-08-27T11:12:13.934374+0000","last_became_active":"2023-08-27T11:12:08.913663+0000","last_became_peered":"2023-08-27T11:12:08.913663+0000","last_unstale":"2023-08-27T11:12:13.934374+0000","last_undegraded":"2023-08-27T11:12:13.934374+0000","last_fullsized":"2023-08-27T11:12:13.934374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:20:17.707884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967392+0000","last_change":"2023-08-27T11:12:12.955103+0000","last_active":"2023-08-27T11:12:14.967392+0000","last_peered":"2023-08-27T11:12:14.967392+0000","last_clean":"2023-08-27T11:12:14.967392+0000","last_became_active":"2023-08-27T11:12:12.954719+0000","last_became_peered":"2023-08-27T11:12:12.954719+0000","last_unstale":"2023-08-27T11:12:14.967392+0000","last_undegraded":"2023-08-27T11:12:14.967392+0000","last_fullsized":"2023-08-27T11:12:14.967392+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:16.222119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.967603+0000","last_change":"2023-08-27T11:12:07.899028+0000","last_active":"2023-08-27T11:12:14.967603+0000","last_peered":"2023-08-27T11:12:14.967603+0000","last_clean":"2023-08-27T11:12:14.967603+0000","last_became_active":"2023-08-27T11:12:07.898749+0000","last_became_peered":"2023-08-27T11:12:07.898749+0000","last_unstale":"2023-08-27T11:12:14.967603+0000","last_undegraded":"2023-08-27T11:12:14.967603+0000","last_fullsized":"2023-08-27T11:12:14.967603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:43:50.454016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933154+0000","last_change":"2023-08-27T11:12:08.917140+0000","last_active":"2023-08-27T11:12:13.933154+0000","last_peered":"2023-08-27T11:12:13.933154+0000","last_clean":"2023-08-27T11:12:13.933154+0000","last_became_active":"2023-08-27T11:12:08.916844+0000","last_became_peered":"2023-08-27T11:12:08.916844+0000","last_unstale":"2023-08-27T11:12:13.933154+0000","last_undegraded":"2023-08-27T11:12:13.933154+0000","last_fullsized":"2023-08-27T11:12:13.933154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:12.215932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933507+0000","last_change":"2023-08-27T11:12:12.952640+0000","last_active":"2023-08-27T11:12:13.933507+0000","last_peered":"2023-08-27T11:12:13.933507+0000","last_clean":"2023-08-27T11:12:13.933507+0000","last_became_active":"2023-08-27T11:12:12.952235+0000","last_became_peered":"2023-08-27T11:12:12.952235+0000","last_unstale":"2023-08-27T11:12:13.933507+0000","last_undegraded":"2023-08-27T11:12:13.933507+0000","last_fullsized":"2023-08-27T11:12:13.933507+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:44.222308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982940+0000","last_change":"2023-08-27T11:12:07.898854+0000","last_active":"2023-08-27T11:12:14.982940+0000","last_peered":"2023-08-27T11:12:14.982940+0000","last_clean":"2023-08-27T11:12:14.982940+0000","last_became_active":"2023-08-27T11:12:07.898427+0000","last_became_peered":"2023-08-27T11:12:07.898427+0000","last_unstale":"2023-08-27T11:12:14.982940+0000","last_undegraded":"2023-08-27T11:12:14.982940+0000","last_fullsized":"2023-08-27T11:12:14.982940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:25.089023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982399+0000","last_change":"2023-08-27T11:12:08.913067+0000","last_active":"2023-08-27T11:12:14.982399+0000","last_peered":"2023-08-27T11:12:14.982399+0000","last_clean":"2023-08-27T11:12:14.982399+0000","last_became_active":"2023-08-27T11:12:08.912345+0000","last_became_peered":"2023-08-27T11:12:08.912345+0000","last_unstale":"2023-08-27T11:12:14.982399+0000","last_undegraded":"2023-08-27T11:12:14.982399+0000","last_fullsized":"2023-08-27T11:12:14.982399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:15:41.474470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838060+0000","last_change":"2023-08-27T11:12:12.948294+0000","last_active":"2023-08-27T11:12:14.838060+0000","last_peered":"2023-08-27T11:12:14.838060+0000","last_clean":"2023-08-27T11:12:14.838060+0000","last_became_active":"2023-08-27T11:12:12.947742+0000","last_became_peered":"2023-08-27T11:12:12.947742+0000","last_unstale":"2023-08-27T11:12:14.838060+0000","last_undegraded":"2023-08-27T11:12:14.838060+0000","last_fullsized":"2023-08-27T11:12:14.838060+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:11.922531+0000","last_clean_scrub_stamp":"2023-08-27T11:12:11.922531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:06.825066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.727133+0000","last_change":"2023-08-27T11:12:07.902780+0000","last_active":"2023-08-27T11:12:14.727133+0000","last_peered":"2023-08-27T11:12:14.727133+0000","last_clean":"2023-08-27T11:12:14.727133+0000","last_became_active":"2023-08-27T11:12:07.902465+0000","last_became_peered":"2023-08-27T11:12:07.902465+0000","last_unstale":"2023-08-27T11:12:14.727133+0000","last_undegraded":"2023-08-27T11:12:14.727133+0000","last_fullsized":"2023-08-27T11:12:14.727133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:06.876011+0000","last_clean_scrub_stamp":"2023-08-27T11:12:06.876011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:17:09.470216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.933956+0000","last_change":"2023-08-27T11:12:08.925508+0000","last_active":"2023-08-27T11:12:13.933956+0000","last_peered":"2023-08-27T11:12:13.933956+0000","last_clean":"2023-08-27T11:12:13.933956+0000","last_became_active":"2023-08-27T11:12:08.919770+0000","last_became_peered":"2023-08-27T11:12:08.919770+0000","last_unstale":"2023-08-27T11:12:13.933956+0000","last_undegraded":"2023-08-27T11:12:13.933956+0000","last_fullsized":"2023-08-27T11:12:13.933956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:57.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.932961+0000","last_change":"2023-08-27T11:12:08.914782+0000","last_active":"2023-08-27T11:12:13.932961+0000","last_peered":"2023-08-27T11:12:13.932961+0000","last_clean":"2023-08-27T11:12:13.932961+0000","last_became_active":"2023-08-27T11:12:08.914443+0000","last_became_peered":"2023-08-27T11:12:08.914443+0000","last_unstale":"2023-08-27T11:12:13.932961+0000","last_undegraded":"2023-08-27T11:12:13.932961+0000","last_fullsized":"2023-08-27T11:12:13.932961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:11:07.149137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982607+0000","last_change":"2023-08-27T11:12:08.923263+0000","last_active":"2023-08-27T11:12:14.982607+0000","last_peered":"2023-08-27T11:12:14.982607+0000","last_clean":"2023-08-27T11:12:14.982607+0000","last_became_active":"2023-08-27T11:12:08.922904+0000","last_became_peered":"2023-08-27T11:12:08.922904+0000","last_unstale":"2023-08-27T11:12:14.982607+0000","last_undegraded":"2023-08-27T11:12:14.982607+0000","last_fullsized":"2023-08-27T11:12:14.982607+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:53:49.762265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:13.934690+0000","last_change":"2023-08-27T11:12:08.925804+0000","last_active":"2023-08-27T11:12:13.934690+0000","last_peered":"2023-08-27T11:12:13.934690+0000","last_clean":"2023-08-27T11:12:13.934690+0000","last_became_active":"2023-08-27T11:12:08.925554+0000","last_became_peered":"2023-08-27T11:12:08.925554+0000","last_unstale":"2023-08-27T11:12:13.934690+0000","last_undegraded":"2023-08-27T11:12:13.934690+0000","last_fullsized":"2023-08-27T11:12:13.934690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:04:15.724420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.982308+0000","last_change":"2023-08-27T11:12:08.929175+0000","last_active":"2023-08-27T11:12:14.982308+0000","last_peered":"2023-08-27T11:12:14.982308+0000","last_clean":"2023-08-27T11:12:14.982308+0000","last_became_active":"2023-08-27T11:12:08.928863+0000","last_became_peered":"2023-08-27T11:12:08.928863+0000","last_unstale":"2023-08-27T11:12:14.982308+0000","last_undegraded":"2023-08-27T11:12:14.982308+0000","last_fullsized":"2023-08-27T11:12:14.982308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:11:30.787887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-08-27T11:12:14.838203+0000","last_change":"2023-08-27T11:12:08.924528+0000","last_active":"2023-08-27T11:12:14.838203+0000","last_peered":"2023-08-27T11:12:14.838203+0000","last_clean":"2023-08-27T11:12:14.838203+0000","last_became_active":"2023-08-27T11:12:08.924221+0000","last_became_peered":"2023-08-27T11:12:08.924221+0000","last_unstale":"2023-08-27T11:12:14.838203+0000","last_undegraded":"2023-08-27T11:12:14.838203+0000","last_fullsized":"2023-08-27T11:12:14.838203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:07.880612+0000","last_clean_scrub_stamp":"2023-08-27T11:12:07.880612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:12.180200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":913408,"data_stored":714327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":710851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":123799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":120553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:12:40.406 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-27T11:12:40.406 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-27T11:12:40.406 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-08-27T11:12:40.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-08-27T11:12:40.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.561+0000 7fdfd0640700 1 Processor -- start 2023-08-27T11:12:40.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.561+0000 7fdfd0640700 1 -- start start 2023-08-27T11:12:40.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.562+0000 7fdfd0640700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc80ae920 0x7fdfc80cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.562+0000 7fdfd0640700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc8071970 con 0x7fdfc806a5a0 2023-08-27T11:12:40.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.562+0000 7fdfd0640700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc8073800 con 0x7fdfc806dcc0 2023-08-27T11:12:40.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.562+0000 7fdfd0640700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80b3220 con 0x7fdfc80ae920 2023-08-27T11:12:40.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.562+0000 7fdfcdbdb700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdfc806a5a0 0x7fdfc80ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39108/0 (socket says 172.21.15.45:39108) 2023-08-27T11:12:40.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.562+0000 7fdfcdbdb700 1 -- 172.21.15.45:0/3466591289 learned_addr learned my addr 172.21.15.45:0/3466591289 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:40.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.562+0000 7fdfce3dc700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc80ae920 0x7fdfc80cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.562+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307363049 0 0) 0x7fdfc8071970 con 0x7fdfc806a5a0 2023-08-27T11:12:40.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb40057c0 con 0x7fdfc806a5a0 2023-08-27T11:12:40.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821036612 0 0) 0x7fdfc8073800 con 0x7fdfc806dcc0 2023-08-27T11:12:40.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfc8071970 con 0x7fdfc806dcc0 2023-08-27T11:12:40.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1141493428 0 0) 0x7fdfb40057c0 con 0x7fdfc806a5a0 2023-08-27T11:12:40.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc80ae920 msgr2=0x7fdfc80cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:40.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc80ae920 0x7fdfc80cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 >> v1:172.21.15.135:6789/0 conn(0x7fdfc806dcc0 legacy=0x7fdfc8071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc815dc50 con 0x7fdfc806a5a0 2023-08-27T11:12:40.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fdfb8001670 con 0x7fdfc806a5a0 2023-08-27T11:12:40.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 >> v1:172.21.15.45:6789/0 conn(0x7fdfc806a5a0 legacy=0x7fdfc80ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc80ae920 0x7fdfb4005dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb4006330 0x7fdfb4006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfb4007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80b3220 con 0x7fdfb4006fa0 2023-08-27T11:12:40.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb400dd00 con 0x7fdfb4006330 2023-08-27T11:12:40.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb400dec0 con 0x7fdfc80ae920 2023-08-27T11:12:40.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcebdd700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfb4007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcdbdb700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb4006330 0x7fdfb4006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfce3dc700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc80ae920 0x7fdfb4005dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcebdd700 1 -- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc80ae920 msgr2=0x7fdfb4005dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:40.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcebdd700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc80ae920 0x7fdfb4005dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcebdd700 1 -- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb4006330 msgr2=0x7fdfb4006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:40.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcebdd700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfb4006330 0x7fdfb4006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcebdd700 1 -- 172.21.15.45:0/3466591289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb40057c0 con 0x7fdfb4006fa0 2023-08-27T11:12:40.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.563+0000 7fdfcebdd700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfb4007420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000c4c0 tx=0x7fdfb4007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffe0743fd29be60 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:40.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0003a00 con 0x7fdfb4006fa0 2023-08-27T11:12:40.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc0003b60 con 0x7fdfb4006fa0 2023-08-27T11:12:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfcd3da700 1 -- 172.21.15.45:0/3466591289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc00050d0 con 0x7fdfb4006fa0 2023-08-27T11:12:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfd0640700 1 -- 172.21.15.45:0/3466591289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 msgr2=0x7fdfb4007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfd0640700 1 --2- 172.21.15.45:0/3466591289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfb4007420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000c4c0 tx=0x7fdfb4007a90 comp rx=0 tx=0).stop 2023-08-27T11:12:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfcebdd700 1 -- 172.21.15.45:0/3466591289 reap_dead start 2023-08-27T11:12:40.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfd0640700 1 -- 172.21.15.45:0/3466591289 shutdown_connections 2023-08-27T11:12:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfd0640700 1 -- 172.21.15.45:0/3466591289 >> 172.21.15.45:0/3466591289 conn(0x7fdfc80b23d0 msgr2=0x7fdfc80b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfd0640700 1 -- 172.21.15.45:0/3466591289 shutdown_connections 2023-08-27T11:12:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.564+0000 7fdfd0640700 1 -- 172.21.15.45:0/3466591289 wait complete. 2023-08-27T11:12:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfd0640700 1 Processor -- start 2023-08-27T11:12:40.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfd0640700 1 -- start start 2023-08-27T11:12:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfd0640700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfc8153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfd0640700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc8154330 0x7fdfc81536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfd0640700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc81555b0 0x7fdfc8155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfd0640700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc8155f80 con 0x7fdfb4006fa0 2023-08-27T11:12:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfd0640700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80b09a0 con 0x7fdfc8154330 2023-08-27T11:12:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfd0640700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc806d680 con 0x7fdfc81555b0 2023-08-27T11:12:40.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfce3dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfc8153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfce3dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfc8153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52342/0 (socket says 172.21.15.45:52342) 2023-08-27T11:12:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.566+0000 7fdfce3dc700 1 -- 172.21.15.45:0/3289262055 learned_addr learned my addr 172.21.15.45:0/3289262055 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfcdbdb700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc8154330 0x7fdfc81536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfcebdd700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc81555b0 0x7fdfc8155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfce3dc700 1 -- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc81555b0 msgr2=0x7fdfc8155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfce3dc700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc81555b0 0x7fdfc8155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfce3dc700 1 -- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc8154330 msgr2=0x7fdfc81536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfce3dc700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc8154330 0x7fdfc81536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfce3dc700 1 -- 172.21.15.45:0/3289262055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8157c90 con 0x7fdfb4006fa0 2023-08-27T11:12:40.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfcdbdb700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc8154330 0x7fdfc81536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:40.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfcebdd700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc81555b0 0x7fdfc8155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:40.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfce3dc700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfc8153d70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4011aa0 tx=0x7fdfc4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:40.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfbeffd700 1 -- 172.21.15.45:0/3289262055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400f980 con 0x7fdfb4006fa0 2023-08-27T11:12:40.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfbeffd700 1 -- 172.21.15.45:0/3289262055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc400fae0 con 0x7fdfb4006fa0 2023-08-27T11:12:40.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfbeffd700 1 -- 172.21.15.45:0/3289262055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400fe90 con 0x7fdfb4006fa0 2023-08-27T11:12:40.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc8157f70 con 0x7fdfb4006fa0 2023-08-27T11:12:40.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.567+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc806d2a0 con 0x7fdfb4006fa0 2023-08-27T11:12:40.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.568+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc80b0ca0 con 0x7fdfb4006fa0 2023-08-27T11:12:40.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.568+0000 7fdfbeffd700 1 -- 172.21.15.45:0/3289262055 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdfc4014440 con 0x7fdfb4006fa0 2023-08-27T11:12:40.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.569+0000 7fdfbeffd700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdfac06a0e0 0x7fdfac06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.569+0000 7fdfbeffd700 1 -- 172.21.15.45:0/3289262055 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdfc4089040 con 0x7fdfb4006fa0 2023-08-27T11:12:40.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.569+0000 7fdfcdbdb700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdfac06a0e0 0x7fdfac06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.570+0000 7fdfcdbdb700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdfac06a0e0 0x7fdfac06c5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80017a0 tx=0x7fdfb8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:40.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.574+0000 7fdfbeffd700 1 -- 172.21.15.45:0/3289262055 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdfc4057f90 con 0x7fdfb4006fa0 2023-08-27T11:12:40.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.818+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdfc8157ad0 con 0x7fdfb4006fa0 2023-08-27T11:12:40.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.819+0000 7fdfbeffd700 1 -- 172.21.15.45:0/3289262055 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdfc4022040 con 0x7fdfb4006fa0 2023-08-27T11:12:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:40.821 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-27T11:12:40.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdfac06a0e0 msgr2=0x7fdfac06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdfac06a0e0 0x7fdfac06c5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80017a0 tx=0x7fdfb8009040 comp rx=0 tx=0).stop 2023-08-27T11:12:40.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 msgr2=0x7fdfc8153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfc8153d70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4011aa0 tx=0x7fdfc4011e60 comp rx=0 tx=0).stop 2023-08-27T11:12:40.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 shutdown_connections 2023-08-27T11:12:40.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4006fa0 0x7fdfc8153d70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdfac06a0e0 0x7fdfac06c5a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdfc81555b0 0x7fdfc8155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 --2- 172.21.15.45:0/3289262055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdfc8154330 0x7fdfc81536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 >> 172.21.15.45:0/3289262055 conn(0x7fdfc80b23d0 msgr2=0x7fdfc815ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:40.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 shutdown_connections 2023-08-27T11:12:40.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.821+0000 7fdfd0640700 1 -- 172.21.15.45:0/3289262055 wait complete. 2023-08-27T11:12:40.835 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-08-27T11:12:40.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:40.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.960+0000 7f8e40c2c700 1 Processor -- start 2023-08-27T11:12:40.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.960+0000 7f8e40c2c700 1 -- start start 2023-08-27T11:12:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e40c2c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e3c15b070 0x7f8e3c0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e40c2c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c06e980 0x7f8e3c0ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e40c2c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c15dad0 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e40c2c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3c15d8e0 con 0x7f8e3c15b070 2023-08-27T11:12:40.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e40c2c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c15ddb0 con 0x7f8e3c06e980 2023-08-27T11:12:40.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e3a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e3c15b070 0x7f8e3c0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e39d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8e3c0cdf60 0x7f8e3c06e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39126/0 (socket says 172.21.15.45:39126) 2023-08-27T11:12:40.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e39d9b700 1 -- 172.21.15.45:0/1388288635 learned_addr learned my addr 172.21.15.45:0/1388288635 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:40.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e3ad9d700 1 --2- 172.21.15.45:0/1388288635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c06e980 0x7f8e3c0ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e3a59c700 1 -- 172.21.15.45:0/1388288635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c06e980 msgr2=0x7f8e3c0ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:40.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e3a59c700 1 --2- 172.21.15.45:0/1388288635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c06e980 0x7f8e3c0ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e3a59c700 1 -- 172.21.15.45:0/1388288635 >> v1:172.21.15.45:6789/0 conn(0x7f8e3c0cdf60 legacy=0x7f8e3c06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e3a59c700 1 -- 172.21.15.45:0/1388288635 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c15e090 con 0x7f8e3c15b070 2023-08-27T11:12:40.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e3ad9d700 1 --2- 172.21.15.45:0/1388288635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c06e980 0x7f8e3c0ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:40.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.961+0000 7f8e3a59c700 1 --2- 172.21.15.45:0/1388288635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c15b070 0x7f8e3c0cda20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009840 tx=0x7f8e2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7134c78abb87ce6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:40.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e3959a700 1 -- 172.21.15.45:0/1388288635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c011830 con 0x7f8e3c15b070 2023-08-27T11:12:40.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e3959a700 1 -- 172.21.15.45:0/1388288635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c011990 con 0x7f8e3c15b070 2023-08-27T11:12:40.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e3959a700 1 -- 172.21.15.45:0/1388288635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c011d20 con 0x7f8e3c15b070 2023-08-27T11:12:40.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e40c2c700 1 -- 172.21.15.45:0/1388288635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c15b070 msgr2=0x7f8e3c0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e40c2c700 1 --2- 172.21.15.45:0/1388288635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c15b070 0x7f8e3c0cda20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009840 tx=0x7f8e2c009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:40.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e40c2c700 1 -- 172.21.15.45:0/1388288635 shutdown_connections 2023-08-27T11:12:40.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e40c2c700 1 --2- 172.21.15.45:0/1388288635 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c06e980 0x7f8e3c0ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e40c2c700 1 --2- 172.21.15.45:0/1388288635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c15b070 0x7f8e3c0cda20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e40c2c700 1 -- 172.21.15.45:0/1388288635 >> 172.21.15.45:0/1388288635 conn(0x7f8e3c0cfe50 msgr2=0x7f8e3c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:40.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e40c2c700 1 -- 172.21.15.45:0/1388288635 shutdown_connections 2023-08-27T11:12:40.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.962+0000 7f8e40c2c700 1 -- 172.21.15.45:0/1388288635 wait complete. 2023-08-27T11:12:40.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.964+0000 7f8e40c2c700 1 Processor -- start 2023-08-27T11:12:40.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.964+0000 7f8e40c2c700 1 -- start start 2023-08-27T11:12:40.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e40c2c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e3c06e980 0x7f8e3c1f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e40c2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c0cdf60 0x7f8e3c145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e40c2c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c15b070 0x7f8e3c0c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e40c2c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c145bf0 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e40c2c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1f73a0 con 0x7f8e3c06e980 2023-08-27T11:12:40.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e40c2c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1459a0 con 0x7f8e3c15b070 2023-08-27T11:12:40.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e39d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c0cdf60 0x7f8e3c145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e39d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c0cdf60 0x7f8e3c145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52356/0 (socket says 172.21.15.45:52356) 2023-08-27T11:12:40.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e39d9b700 1 -- 172.21.15.45:0/2879603816 learned_addr learned my addr 172.21.15.45:0/2879603816 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:40.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e3a59c700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e3c06e980 0x7f8e3c1f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e3ad9d700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c15b070 0x7f8e3c0c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e39d9b700 1 -- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c15b070 msgr2=0x7f8e3c0c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e39d9b700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c15b070 0x7f8e3c0c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e39d9b700 1 -- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e3c06e980 msgr2=0x7f8e3c1f6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e39d9b700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e3c06e980 0x7f8e3c1f6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.965+0000 7f8e39d9b700 1 -- 172.21.15.45:0/2879603816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c0094a0 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.966+0000 7f8e39d9b700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c0cdf60 0x7f8e3c145460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c4a0 tx=0x7f8e3000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:40.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.966+0000 7f8e2affd700 1 -- 172.21.15.45:0/2879603816 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30005d70 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.966+0000 7f8e2affd700 1 -- 172.21.15.45:0/2879603816 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e300070d0 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.966+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c145e40 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.966+0000 7f8e2affd700 1 -- 172.21.15.45:0/2879603816 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30007440 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.966+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c156ef0 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.967+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c15e350 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.967+0000 7f8e2affd700 1 -- 172.21.15.45:0/2879603816 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8e300075a0 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.968+0000 7f8e2affd700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e1c06a0e0 0x7f8e1c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.968+0000 7f8e2affd700 1 -- 172.21.15.45:0/2879603816 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8e30087bd0 con 0x7f8e3c0cdf60 2023-08-27T11:12:40.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.968+0000 7f8e3a59c700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e1c06a0e0 0x7f8e1c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:40.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.968+0000 7f8e3a59c700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e1c06a0e0 0x7f8e1c06c5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c000c00 tx=0x7f8e2c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:40.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:40.972+0000 7f8e2affd700 1 -- 172.21.15.45:0/2879603816 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8e3c15e350 con 0x7f8e3c0cdf60 2023-08-27T11:12:41.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.174+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e3c15e350 con 0x7f8e3c0cdf60 2023-08-27T11:12:41.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.175+0000 7f8e2affd700 1 -- 172.21.15.45:0/2879603816 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f8e3c15e350 con 0x7f8e3c0cdf60 2023-08-27T11:12:41.177 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:41.178 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:41.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e1c06a0e0 msgr2=0x7f8e1c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e1c06a0e0 0x7f8e1c06c5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c000c00 tx=0x7f8e2c01c040 comp rx=0 tx=0).stop 2023-08-27T11:12:41.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c0cdf60 msgr2=0x7f8e3c145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c0cdf60 0x7f8e3c145460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c4a0 tx=0x7f8e3000c860 comp rx=0 tx=0).stop 2023-08-27T11:12:41.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 shutdown_connections 2023-08-27T11:12:41.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e1c06a0e0 0x7f8e1c06c5a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e3c15b070 0x7f8e3c0c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e3c06e980 0x7f8e3c1f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 --2- 172.21.15.45:0/2879603816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e3c0cdf60 0x7f8e3c145460 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 >> 172.21.15.45:0/2879603816 conn(0x7f8e3c0cfe50 msgr2=0x7f8e3c070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:41.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 shutdown_connections 2023-08-27T11:12:41.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.177+0000 7f8e40c2c700 1 -- 172.21.15.45:0/2879603816 wait complete. 2023-08-27T11:12:41.191 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:12:41.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:12:41.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.319+0000 7ff4dcdcc700 1 Processor -- start 2023-08-27T11:12:41.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.319+0000 7ff4dcdcc700 1 -- start start 2023-08-27T11:12:41.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4dcdcc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff4d815d340 0x7ff4d8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4dcdcc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4d806a990 0x7ff4d80b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4dcdcc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d806b160 con 0x7ff4d80cd880 2023-08-27T11:12:41.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4dcdcc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d8072010 con 0x7ff4d815d340 2023-08-27T11:12:41.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4dcdcc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d80af590 con 0x7ff4d806a990 2023-08-27T11:12:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d5d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff4d80cd880 0x7ff4d806a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39140/0 (socket says 172.21.15.45:39140) 2023-08-27T11:12:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d5d9b700 1 -- 172.21.15.45:0/3531729982 learned_addr learned my addr 172.21.15.45:0/3531729982 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104259038 0 0) 0x7ff4d806b160 con 0x7ff4d80cd880 2023-08-27T11:12:41.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d6d9d700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4d806a990 0x7ff4d80b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c00035b0 con 0x7ff4d80cd880 2023-08-27T11:12:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d659c700 1 --2- 172.21.15.45:0/3531729982 >> v2:172.21.15.135:3300/0 conn(0x7ff4d815d340 0x7ff4d8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 52182527 0 0) 0x7ff4c00035b0 con 0x7ff4d80cd880 2023-08-27T11:12:41.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4d806a990 msgr2=0x7ff4d80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4d806a990 0x7ff4d80b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 >> v2:172.21.15.135:3300/0 conn(0x7ff4d815d340 msgr2=0x7ff4d8160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 --2- 172.21.15.45:0/3531729982 >> v2:172.21.15.135:3300/0 conn(0x7ff4d815d340 0x7ff4d8160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d80af2b0 con 0x7ff4d80cd880 2023-08-27T11:12:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7ff4c8001670 con 0x7ff4d80cd880 2023-08-27T11:12:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 >> v1:172.21.15.45:6789/0 conn(0x7ff4d80cd880 legacy=0x7ff4d806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c0003a00 0x7ff4c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4d806a990 0x7ff4c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0004d90 0x7ff4c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d8072010 con 0x7ff4c0004d90 2023-08-27T11:12:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d80af590 con 0x7ff4c0003a00 2023-08-27T11:12:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.320+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c000ba40 con 0x7ff4d806a990 2023-08-27T11:12:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d6d9d700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0004d90 0x7ff4c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d659c700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c0003a00 0x7ff4c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d5d9b700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4d806a990 0x7ff4c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d6d9d700 1 -- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4d806a990 msgr2=0x7ff4c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:41.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d6d9d700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4d806a990 0x7ff4c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d6d9d700 1 -- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c0003a00 msgr2=0x7ff4c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:41.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d6d9d700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4c0003a00 0x7ff4c0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d6d9d700 1 -- 172.21.15.45:0/3531729982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c00035b0 con 0x7ff4c0004d90 2023-08-27T11:12:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d6d9d700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0004d90 0x7ff4c0005210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009fc0 tx=0x7ff4c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c30d943854eb35b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000ec30 con 0x7ff4c0004d90 2023-08-27T11:12:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d000f040 con 0x7ff4c0004d90 2023-08-27T11:12:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.321+0000 7ff4d559a700 1 -- 172.21.15.45:0/3531729982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0014480 con 0x7ff4c0004d90 2023-08-27T11:12:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.322+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/3531729982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0004d90 msgr2=0x7ff4c0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.322+0000 7ff4dcdcc700 1 --2- 172.21.15.45:0/3531729982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0004d90 0x7ff4c0005210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009fc0 tx=0x7ff4c0005880 comp rx=0 tx=0).stop 2023-08-27T11:12:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.322+0000 7ff4d6d9d700 1 -- 172.21.15.45:0/3531729982 reap_dead start 2023-08-27T11:12:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.322+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/3531729982 shutdown_connections 2023-08-27T11:12:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.322+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/3531729982 >> 172.21.15.45:0/3531729982 conn(0x7ff4d80cc7c0 msgr2=0x7ff4d80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.322+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/3531729982 shutdown_connections 2023-08-27T11:12:41.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.322+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/3531729982 wait complete. 2023-08-27T11:12:41.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.323+0000 7ff4dcdcc700 1 Processor -- start 2023-08-27T11:12:41.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.323+0000 7ff4dcdcc700 1 -- start start 2023-08-27T11:12:41.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4dcdcc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4c0004d90 0x7ff4d80c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4dcdcc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4d8157120 0x7ff4d8141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4dcdcc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4d80ada30 0x7ff4d80adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4dcdcc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d80caa20 con 0x7ff4d80ada30 2023-08-27T11:12:41.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4dcdcc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d80ae3f0 con 0x7ff4d8157120 2023-08-27T11:12:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4dcdcc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d81575a0 con 0x7ff4c0004d90 2023-08-27T11:12:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4d80ada30 0x7ff4d80adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d659c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4c0004d90 0x7ff4d80c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4d80ada30 0x7ff4d80adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52378/0 (socket says 172.21.15.45:52378) 2023-08-27T11:12:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d5d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4d8157120 0x7ff4d8141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d6d9d700 1 -- 172.21.15.45:0/1906332058 learned_addr learned my addr 172.21.15.45:0/1906332058 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d6d9d700 1 -- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4c0004d90 msgr2=0x7ff4d80c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d6d9d700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4c0004d90 0x7ff4d80c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d6d9d700 1 -- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4d8157120 msgr2=0x7ff4d8141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d6d9d700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4d8157120 0x7ff4d8141ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.324+0000 7ff4d6d9d700 1 -- 172.21.15.45:0/1906332058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d80c7480 con 0x7ff4d80ada30 2023-08-27T11:12:41.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.325+0000 7ff4d6d9d700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4d80ada30 0x7ff4d80adeb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0017040 tx=0x7ff4d0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.325+0000 7ff4c6ffd700 1 -- 172.21.15.45:0/1906332058 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0007000 con 0x7ff4d80ada30 2023-08-27T11:12:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.325+0000 7ff4c6ffd700 1 -- 172.21.15.45:0/1906332058 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d0003e50 con 0x7ff4d80ada30 2023-08-27T11:12:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.325+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d80ca080 con 0x7ff4d80ada30 2023-08-27T11:12:41.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.325+0000 7ff4c6ffd700 1 -- 172.21.15.45:0/1906332058 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0014aa0 con 0x7ff4d80ada30 2023-08-27T11:12:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.325+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d80ca640 con 0x7ff4d80ada30 2023-08-27T11:12:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.326+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d80cab60 con 0x7ff4d80ada30 2023-08-27T11:12:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.326+0000 7ff4c6ffd700 1 -- 172.21.15.45:0/1906332058 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff4d0023020 con 0x7ff4d80ada30 2023-08-27T11:12:41.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.326+0000 7ff4c6ffd700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff4b806a0e0 0x7ff4b806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.327+0000 7ff4c6ffd700 1 -- 172.21.15.45:0/1906332058 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff4d0089d20 con 0x7ff4d80ada30 2023-08-27T11:12:41.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.327+0000 7ff4d659c700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff4b806a0e0 0x7ff4b806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.327+0000 7ff4d659c700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff4b806a0e0 0x7ff4b806c5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009c60 tx=0x7ff4cc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:41.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.331+0000 7ff4c6ffd700 1 -- 172.21.15.45:0/1906332058 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff4d80cab60 con 0x7ff4d80ada30 2023-08-27T11:12:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.530+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4d80cab60 con 0x7ff4d80ada30 2023-08-27T11:12:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.530+0000 7ff4c6ffd700 1 -- 172.21.15.45:0/1906332058 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4d80cab60 con 0x7ff4d80ada30 2023-08-27T11:12:41.533 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:12:41.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.532+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff4b806a0e0 msgr2=0x7ff4b806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.532+0000 7ff4dcdcc700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff4b806a0e0 0x7ff4b806c5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009c60 tx=0x7ff4cc009210 comp rx=0 tx=0).stop 2023-08-27T11:12:41.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.532+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4d80ada30 msgr2=0x7ff4d80adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.532+0000 7ff4dcdcc700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4d80ada30 0x7ff4d80adeb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0017040 tx=0x7ff4d0009750 comp rx=0 tx=0).stop 2023-08-27T11:12:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.533+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 shutdown_connections 2023-08-27T11:12:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.533+0000 7ff4dcdcc700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4d80ada30 0x7ff4d80adeb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.533+0000 7ff4dcdcc700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff4b806a0e0 0x7ff4b806c5a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.533+0000 7ff4dcdcc700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4c0004d90 0x7ff4d80c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.533+0000 7ff4dcdcc700 1 --2- 172.21.15.45:0/1906332058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4d8157120 0x7ff4d8141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.533+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 >> 172.21.15.45:0/1906332058 conn(0x7ff4d80cc7c0 msgr2=0x7ff4d80b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.533+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 shutdown_connections 2023-08-27T11:12:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.533+0000 7ff4dcdcc700 1 -- 172.21.15.45:0/1906332058 wait complete. 2023-08-27T11:12:41.547 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:12:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.672+0000 7fbc08d15700 1 Processor -- start 2023-08-27T11:12:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.672+0000 7fbc08d15700 1 -- start start 2023-08-27T11:12:41.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc08d15700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc0415d340 0x7fbc04160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc08d15700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cd880 0x7fbc0406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc08d15700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc0406b160 con 0x7fbc040cdd00 2023-08-27T11:12:41.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc08d15700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc04072010 con 0x7fbc0415d340 2023-08-27T11:12:41.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc08d15700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040af590 con 0x7fbc040cd880 2023-08-27T11:12:41.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc0259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc0415d340 0x7fbc04160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc02d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbc040cdd00 0x7fbc040b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39162/0 (socket says 172.21.15.45:39162) 2023-08-27T11:12:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc02d9d700 1 -- 172.21.15.45:0/363691510 learned_addr learned my addr 172.21.15.45:0/363691510 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc01d9b700 1 --2- 172.21.15.45:0/363691510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cd880 0x7fbc0406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc0259c700 1 -- 172.21.15.45:0/363691510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cd880 msgr2=0x7fbc0406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc0259c700 1 --2- 172.21.15.45:0/363691510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cd880 0x7fbc0406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc0259c700 1 -- 172.21.15.45:0/363691510 >> v1:172.21.15.45:6789/0 conn(0x7fbc040cdd00 legacy=0x7fbc040b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc0259c700 1 -- 172.21.15.45:0/363691510 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc040af2b0 con 0x7fbc0415d340 2023-08-27T11:12:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc0159a700 1 -- 172.21.15.45:0/363691510 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771847798 0 0) 0x7fbc0406b160 con 0x7fbc040cdd00 2023-08-27T11:12:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc0259c700 1 --2- 172.21.15.45:0/363691510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 0x7fbc04160d00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009840 tx=0x7fbbf4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2311bbcab6398f4e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.673+0000 7fbc0159a700 1 -- 172.21.15.45:0/363691510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4011830 con 0x7fbc0415d340 2023-08-27T11:12:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc0159a700 1 -- 172.21.15.45:0/363691510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4011990 con 0x7fbc0415d340 2023-08-27T11:12:41.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc0159a700 1 -- 172.21.15.45:0/363691510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4011d20 con 0x7fbc0415d340 2023-08-27T11:12:41.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc08d15700 1 -- 172.21.15.45:0/363691510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 msgr2=0x7fbc04160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc08d15700 1 --2- 172.21.15.45:0/363691510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 0x7fbc04160d00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009840 tx=0x7fbbf4009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:41.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc08d15700 1 -- 172.21.15.45:0/363691510 shutdown_connections 2023-08-27T11:12:41.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc08d15700 1 --2- 172.21.15.45:0/363691510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 0x7fbc04160d00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc08d15700 1 --2- 172.21.15.45:0/363691510 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cd880 0x7fbc0406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc08d15700 1 -- 172.21.15.45:0/363691510 >> 172.21.15.45:0/363691510 conn(0x7fbc040cc7c0 msgr2=0x7fbc040ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:41.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc08d15700 1 -- 172.21.15.45:0/363691510 shutdown_connections 2023-08-27T11:12:41.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.674+0000 7fbc08d15700 1 -- 172.21.15.45:0/363691510 wait complete. 2023-08-27T11:12:41.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.676+0000 7fbc08d15700 1 Processor -- start 2023-08-27T11:12:41.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.676+0000 7fbc08d15700 1 -- start start 2023-08-27T11:12:41.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc08d15700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc040cd880 0x7fbc040c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc08d15700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cdd00 0x7fbc04159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc08d15700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 0x7fbc040cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc08d15700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040c37c0 con 0x7fbc0415d340 2023-08-27T11:12:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc08d15700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040cbf50 con 0x7fbc040cd880 2023-08-27T11:12:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc08d15700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc040c3a10 con 0x7fbc040cdd00 2023-08-27T11:12:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc02d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 0x7fbc040cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc02d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 0x7fbc040cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52392/0 (socket says 172.21.15.45:52392) 2023-08-27T11:12:41.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc02d9d700 1 -- 172.21.15.45:0/127810106 learned_addr learned my addr 172.21.15.45:0/127810106 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:41.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc0259c700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc040cd880 0x7fbc040c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc01d9b700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cdd00 0x7fbc04159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc02d9d700 1 -- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cdd00 msgr2=0x7fbc04159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc02d9d700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cdd00 0x7fbc04159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc02d9d700 1 -- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc040cd880 msgr2=0x7fbc040c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc02d9d700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc040cd880 0x7fbc040c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc02d9d700 1 -- 172.21.15.45:0/127810106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40094a0 con 0x7fbc0415d340 2023-08-27T11:12:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.677+0000 7fbc02d9d700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 0x7fbc040cb570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00cb30 tx=0x7fbbfc00ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:41.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.678+0000 7fbbf2ffd700 1 -- 172.21.15.45:0/127810106 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc00a9e0 con 0x7fbc0415d340 2023-08-27T11:12:41.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.678+0000 7fbbf2ffd700 1 -- 172.21.15.45:0/127810106 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbfc00ab80 con 0x7fbc0415d340 2023-08-27T11:12:41.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.678+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc040c3cf0 con 0x7fbc0415d340 2023-08-27T11:12:41.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.678+0000 7fbbf2ffd700 1 -- 172.21.15.45:0/127810106 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc0070d0 con 0x7fbc0415d340 2023-08-27T11:12:41.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.678+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc04151780 con 0x7fbc0415d340 2023-08-27T11:12:41.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.679+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc04072690 con 0x7fbc0415d340 2023-08-27T11:12:41.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.679+0000 7fbbf2ffd700 1 -- 172.21.15.45:0/127810106 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbbfc007230 con 0x7fbc0415d340 2023-08-27T11:12:41.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.680+0000 7fbbf2ffd700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbe406a0e0 0x7fbbe406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:41.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.680+0000 7fbbf2ffd700 1 -- 172.21.15.45:0/127810106 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbbfc088520 con 0x7fbc0415d340 2023-08-27T11:12:41.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.680+0000 7fbc0259c700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbe406a0e0 0x7fbbe406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:41.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.680+0000 7fbc0259c700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbe406a0e0 0x7fbbe406c5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4000c00 tx=0x7fbbf401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:41.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.684+0000 7fbbf2ffd700 1 -- 172.21.15.45:0/127810106 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbbfc0539b0 con 0x7fbc0415d340 2023-08-27T11:12:41.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.882+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbc040734a0 con 0x7fbc0415d340 2023-08-27T11:12:41.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.883+0000 7fbbf2ffd700 1 -- 172.21.15.45:0/127810106 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbbfc001ba0 con 0x7fbc0415d340 2023-08-27T11:12:41.885 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:12:41.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbe406a0e0 msgr2=0x7fbbe406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbe406a0e0 0x7fbbe406c5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4000c00 tx=0x7fbbf401c040 comp rx=0 tx=0).stop 2023-08-27T11:12:41.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 msgr2=0x7fbc040cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 0x7fbc040cb570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00cb30 tx=0x7fbbfc00ce40 comp rx=0 tx=0).stop 2023-08-27T11:12:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 shutdown_connections 2023-08-27T11:12:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc0415d340 0x7fbc040cb570 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbbe406a0e0 0x7fbbe406c5a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc040cd880 0x7fbc040c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 --2- 172.21.15.45:0/127810106 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc040cdd00 0x7fbc04159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 >> 172.21.15.45:0/127810106 conn(0x7fbc040cc7c0 msgr2=0x7fbc0415bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 shutdown_connections 2023-08-27T11:12:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:41.885+0000 7fbc08d15700 1 -- 172.21.15.45:0/127810106 wait complete. 2023-08-27T11:12:41.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:12:42.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.024+0000 7fb09fcd0700 1 Processor -- start 2023-08-27T11:12:42.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.025+0000 7fb09fcd0700 1 -- start start 2023-08-27T11:12:42.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.025+0000 7fb09fcd0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb098157be0 0x7fb0980716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.025+0000 7fb09fcd0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09815b460 con 0x7fb09806d5e0 2023-08-27T11:12:42.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.025+0000 7fb09fcd0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0980b2280 con 0x7fb0980b17c0 2023-08-27T11:12:42.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.025+0000 7fb09fcd0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb098157fc0 con 0x7fb098157be0 2023-08-27T11:12:42.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09e26d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb09806d5e0 0x7fb0980cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39176/0 (socket says 172.21.15.45:39176) 2023-08-27T11:12:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09e26d700 1 -- 172.21.15.45:0/2274227780 learned_addr learned my addr 172.21.15.45:0/2274227780 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09da6c700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb098157be0 0x7fb0980716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678720421 0 0) 0x7fb09815b460 con 0x7fb09806d5e0 2023-08-27T11:12:42.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0840035b0 con 0x7fb09806d5e0 2023-08-27T11:12:42.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2943692242 0 0) 0x7fb0840035b0 con 0x7fb09806d5e0 2023-08-27T11:12:42.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb098157be0 msgr2=0x7fb0980716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:42.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb098157be0 0x7fb0980716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 >> v1:172.21.15.135:6789/0 conn(0x7fb0980b17c0 legacy=0x7fb09806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0980b2560 con 0x7fb09806d5e0 2023-08-27T11:12:42.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fb0900019f0 con 0x7fb09806d5e0 2023-08-27T11:12:42.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 >> v1:172.21.15.45:6789/0 conn(0x7fb09806d5e0 legacy=0x7fb0980cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb084003a30 0x7fb084003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb084004850 0x7fb084004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb098157be0 0x7fb084005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098157fc0 con 0x7fb084003a30 2023-08-27T11:12:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08400bac0 con 0x7fb084004850 2023-08-27T11:12:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08400bc80 con 0x7fb098157be0 2023-08-27T11:12:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09da6c700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb084003a30 0x7fb084003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09d26b700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb084004850 0x7fb084004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.026+0000 7fb09e26d700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb098157be0 0x7fb084005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09da6c700 1 -- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb098157be0 msgr2=0x7fb084005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:42.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09da6c700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb098157be0 0x7fb084005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09da6c700 1 -- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb084004850 msgr2=0x7fb084004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:42.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09da6c700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb084004850 0x7fb084004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09da6c700 1 -- 172.21.15.45:0/2274227780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0840035b0 con 0x7fb084003a30 2023-08-27T11:12:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09e26d700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb098157be0 0x7fb084005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09da6c700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb084003a30 0x7fb084003e50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb088009b30 tx=0x7fb084005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a20f5c82b4d6462 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800e6c0 con 0x7fb084003a30 2023-08-27T11:12:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08800e820 con 0x7fb084003a30 2023-08-27T11:12:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09ca6a700 1 -- 172.21.15.45:0/2274227780 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800eb90 con 0x7fb084003a30 2023-08-27T11:12:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2274227780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb084003a30 msgr2=0x7fb084003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09fcd0700 1 --2- 172.21.15.45:0/2274227780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb084003a30 0x7fb084003e50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb088009b30 tx=0x7fb084005880 comp rx=0 tx=0).stop 2023-08-27T11:12:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09e26d700 1 -- 172.21.15.45:0/2274227780 reap_dead start 2023-08-27T11:12:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2274227780 shutdown_connections 2023-08-27T11:12:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.027+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2274227780 >> 172.21.15.45:0/2274227780 conn(0x7fb0980cc7c0 msgr2=0x7fb0980ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.028+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2274227780 shutdown_connections 2023-08-27T11:12:42.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.028+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2274227780 wait complete. 2023-08-27T11:12:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.029+0000 7fb09fcd0700 1 Processor -- start 2023-08-27T11:12:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.029+0000 7fb09fcd0700 1 -- start start 2023-08-27T11:12:42.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09fcd0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb084003a30 0x7fb0981541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09fcd0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb09815cb00 0x7fb09815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09fcd0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0980b00e0 0x7fb0980b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09fcd0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981f5c50 con 0x7fb0980b00e0 2023-08-27T11:12:42.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09fcd0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09815d4c0 con 0x7fb09815cb00 2023-08-27T11:12:42.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09fcd0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981f5ea0 con 0x7fb084003a30 2023-08-27T11:12:42.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09e26d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0980b00e0 0x7fb0980b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09e26d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0980b00e0 0x7fb0980b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52412/0 (socket says 172.21.15.45:52412) 2023-08-27T11:12:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09e26d700 1 -- 172.21.15.45:0/2448424705 learned_addr learned my addr 172.21.15.45:0/2448424705 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09da6c700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb084003a30 0x7fb0981541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09d26b700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb09815cb00 0x7fb09815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09e26d700 1 -- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb084003a30 msgr2=0x7fb0981541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09e26d700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb084003a30 0x7fb0981541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09e26d700 1 -- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb09815cb00 msgr2=0x7fb09815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09e26d700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb09815cb00 0x7fb09815cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09e26d700 1 -- 172.21.15.45:0/2448424705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0981f6180 con 0x7fb0980b00e0 2023-08-27T11:12:42.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09d26b700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb09815cb00 0x7fb09815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:42.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09e26d700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0980b00e0 0x7fb0980b0560 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb098066e70 tx=0x7fb09000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:42.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb08e7fc700 1 -- 172.21.15.45:0/2448424705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090017070 con 0x7fb0980b00e0 2023-08-27T11:12:42.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb08e7fc700 1 -- 172.21.15.45:0/2448424705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09000ec70 con 0x7fb0980b00e0 2023-08-27T11:12:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb08e7fc700 1 -- 172.21.15.45:0/2448424705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000ee20 con 0x7fb0980b00e0 2023-08-27T11:12:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0981f6460 con 0x7fb0980b00e0 2023-08-27T11:12:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.030+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0980b0aa0 con 0x7fb0980b00e0 2023-08-27T11:12:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.031+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb098072690 con 0x7fb0980b00e0 2023-08-27T11:12:42.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.032+0000 7fb08e7fc700 1 -- 172.21.15.45:0/2448424705 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb090013420 con 0x7fb0980b00e0 2023-08-27T11:12:42.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.032+0000 7fb08e7fc700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb07c06a0e0 0x7fb07c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.032+0000 7fb08e7fc700 1 -- 172.21.15.45:0/2448424705 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb0900884f0 con 0x7fb0980b00e0 2023-08-27T11:12:42.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.033+0000 7fb09da6c700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb07c06a0e0 0x7fb07c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.033+0000 7fb09da6c700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb07c06a0e0 0x7fb07c06c5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0880090b0 tx=0x7fb088008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:42.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.037+0000 7fb08e7fc700 1 -- 172.21.15.45:0/2448424705 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb0900538f0 con 0x7fb0980b00e0 2023-08-27T11:12:42.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.237+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb0980734a0 con 0x7fb0980b00e0 2023-08-27T11:12:42.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.237+0000 7fb08e7fc700 1 -- 172.21.15.45:0/2448424705 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb090056f10 con 0x7fb0980b00e0 2023-08-27T11:12:42.239 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:42.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb07c06a0e0 msgr2=0x7fb07c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb07c06a0e0 0x7fb07c06c5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0880090b0 tx=0x7fb088008030 comp rx=0 tx=0).stop 2023-08-27T11:12:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0980b00e0 msgr2=0x7fb0980b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0980b00e0 0x7fb0980b0560 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb098066e70 tx=0x7fb09000e440 comp rx=0 tx=0).stop 2023-08-27T11:12:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 shutdown_connections 2023-08-27T11:12:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0980b00e0 0x7fb0980b0560 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb07c06a0e0 0x7fb07c06c5a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb084003a30 0x7fb0981541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 --2- 172.21.15.45:0/2448424705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb09815cb00 0x7fb09815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 >> 172.21.15.45:0/2448424705 conn(0x7fb0980cc7c0 msgr2=0x7fb0980af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 shutdown_connections 2023-08-27T11:12:42.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.239+0000 7fb09fcd0700 1 -- 172.21.15.45:0/2448424705 wait complete. 2023-08-27T11:12:42.253 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:12:42.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.378+0000 7f5134d8b700 1 Processor -- start 2023-08-27T11:12:42.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.378+0000 7f5134d8b700 1 -- start start 2023-08-27T11:12:42.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.379+0000 7f5134d8b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f513015a220 0x7f51300cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.379+0000 7f5134d8b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5130072080 0x7f513006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.379+0000 7f5134d8b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51300b3220 con 0x7f51300aea00 2023-08-27T11:12:42.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.379+0000 7f5134d8b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300b2c60 con 0x7f5130072080 2023-08-27T11:12:42.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.379+0000 7f5134d8b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51300b2f40 con 0x7f513015a220 2023-08-27T11:12:42.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.379+0000 7f512ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5130072080 0x7f513006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.379+0000 7f512dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f51300aea00 0x7f5130071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39194/0 (socket says 172.21.15.45:39194) 2023-08-27T11:12:42.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.379+0000 7f512dd9b700 1 -- 172.21.15.45:0/3667347237 learned_addr learned my addr 172.21.15.45:0/3667347237 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:42.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512e59c700 1 --2- 172.21.15.45:0/3667347237 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f513015a220 0x7f51300cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512ed9d700 1 -- 172.21.15.45:0/3667347237 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f513015a220 msgr2=0x7f51300cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:42.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512ed9d700 1 --2- 172.21.15.45:0/3667347237 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f513015a220 0x7f51300cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512ed9d700 1 -- 172.21.15.45:0/3667347237 >> v1:172.21.15.45:6789/0 conn(0x7f51300aea00 legacy=0x7f5130071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512ed9d700 1 -- 172.21.15.45:0/3667347237 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300cec70 con 0x7f5130072080 2023-08-27T11:12:42.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512d59a700 1 -- 172.21.15.45:0/3667347237 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693517695 0 0) 0x7f51300b3220 con 0x7f51300aea00 2023-08-27T11:12:42.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512ed9d700 1 --2- 172.21.15.45:0/3667347237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5130072080 0x7f513006a280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5128009dc0 tx=0x7f512800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cabff424d81c17 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:42.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512d59a700 1 -- 172.21.15.45:0/3667347237 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800e070 con 0x7f5130072080 2023-08-27T11:12:42.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512d59a700 1 -- 172.21.15.45:0/3667347237 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512800bb90 con 0x7f5130072080 2023-08-27T11:12:42.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.380+0000 7f512d59a700 1 -- 172.21.15.45:0/3667347237 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128012430 con 0x7f5130072080 2023-08-27T11:12:42.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.381+0000 7f5134d8b700 1 -- 172.21.15.45:0/3667347237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5130072080 msgr2=0x7f513006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.381+0000 7f5134d8b700 1 --2- 172.21.15.45:0/3667347237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5130072080 0x7f513006a280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5128009dc0 tx=0x7f512800b330 comp rx=0 tx=0).stop 2023-08-27T11:12:42.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.381+0000 7f5134d8b700 1 -- 172.21.15.45:0/3667347237 shutdown_connections 2023-08-27T11:12:42.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.381+0000 7f5134d8b700 1 --2- 172.21.15.45:0/3667347237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5130072080 0x7f513006a280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.381+0000 7f5134d8b700 1 --2- 172.21.15.45:0/3667347237 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f513015a220 0x7f51300cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.381+0000 7f5134d8b700 1 -- 172.21.15.45:0/3667347237 >> 172.21.15.45:0/3667347237 conn(0x7f51300b4650 msgr2=0x7f51300b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:42.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.381+0000 7f5134d8b700 1 -- 172.21.15.45:0/3667347237 shutdown_connections 2023-08-27T11:12:42.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.381+0000 7f5134d8b700 1 -- 172.21.15.45:0/3667347237 wait complete. 2023-08-27T11:12:42.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.382+0000 7f5134d8b700 1 Processor -- start 2023-08-27T11:12:42.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.383+0000 7f5134d8b700 1 -- start start 2023-08-27T11:12:42.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.383+0000 7f5134d8b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5130072080 0x7f513006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.383+0000 7f5134d8b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51300aea00 0x7f513006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.383+0000 7f5134d8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f513015a220 0x7f51300c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.383+0000 7f5134d8b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51300c2310 con 0x7f513015a220 2023-08-27T11:12:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f5134d8b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130152ea0 con 0x7f51300aea00 2023-08-27T11:12:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f5134d8b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5130153180 con 0x7f5130072080 2023-08-27T11:12:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f513015a220 0x7f51300c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f513015a220 0x7f51300c1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52426/0 (socket says 172.21.15.45:52426) 2023-08-27T11:12:42.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512ed9d700 1 -- 172.21.15.45:0/698388796 learned_addr learned my addr 172.21.15.45:0/698388796 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512dd9b700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51300aea00 0x7f513006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512e59c700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5130072080 0x7f513006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512ed9d700 1 -- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5130072080 msgr2=0x7f513006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512ed9d700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5130072080 0x7f513006ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512ed9d700 1 -- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51300aea00 msgr2=0x7f513006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512ed9d700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51300aea00 0x7f513006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512ed9d700 1 -- 172.21.15.45:0/698388796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128009a70 con 0x7f513015a220 2023-08-27T11:12:42.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512dd9b700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51300aea00 0x7f513006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:42.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f512ed9d700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f513015a220 0x7f51300c1dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5128014040 tx=0x7f5128004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:42.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f511effd700 1 -- 172.21.15.45:0/698388796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128012430 con 0x7f513015a220 2023-08-27T11:12:42.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f511effd700 1 -- 172.21.15.45:0/698388796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128020070 con 0x7f513015a220 2023-08-27T11:12:42.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f511effd700 1 -- 172.21.15.45:0/698388796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128004280 con 0x7f513015a220 2023-08-27T11:12:42.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.384+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5130153460 con 0x7f513015a220 2023-08-27T11:12:42.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.385+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5130153a20 con 0x7f513015a220 2023-08-27T11:12:42.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.385+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51300ccf10 con 0x7f513015a220 2023-08-27T11:12:42.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.386+0000 7f511effd700 1 -- 172.21.15.45:0/698388796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f51280043e0 con 0x7f513015a220 2023-08-27T11:12:42.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.386+0000 7f511effd700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f511006a0e0 0x7f511006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.386+0000 7f511effd700 1 -- 172.21.15.45:0/698388796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5128022070 con 0x7f513015a220 2023-08-27T11:12:42.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.387+0000 7f512e59c700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f511006a0e0 0x7f511006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.387+0000 7f512e59c700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f511006a0e0 0x7f511006c5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5124006b20 tx=0x7f5124008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:42.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.391+0000 7f511effd700 1 -- 172.21.15.45:0/698388796 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5128055da0 con 0x7f513015a220 2023-08-27T11:12:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.590+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f513009d480 con 0x7f513015a220 2023-08-27T11:12:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.590+0000 7f511effd700 1 -- 172.21.15.45:0/698388796 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51280593c0 con 0x7f513015a220 2023-08-27T11:12:42.593 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:12:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f511006a0e0 msgr2=0x7f511006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f511006a0e0 0x7f511006c5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5124006b20 tx=0x7f5124008040 comp rx=0 tx=0).stop 2023-08-27T11:12:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f513015a220 msgr2=0x7f51300c1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f513015a220 0x7f51300c1dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5128014040 tx=0x7f5128004ad0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 shutdown_connections 2023-08-27T11:12:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f513015a220 0x7f51300c1dd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f511006a0e0 0x7f511006c5a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5130072080 0x7f513006ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 --2- 172.21.15.45:0/698388796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51300aea00 0x7f513006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 >> 172.21.15.45:0/698388796 conn(0x7f51300b4650 msgr2=0x7f51301548c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 shutdown_connections 2023-08-27T11:12:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.592+0000 7f5134d8b700 1 -- 172.21.15.45:0/698388796 wait complete. 2023-08-27T11:12:42.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:12:42.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.732+0000 7ff37062a700 1 Processor -- start 2023-08-27T11:12:42.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.732+0000 7ff37062a700 1 -- start start 2023-08-27T11:12:42.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff37062a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff36806a280 0x7ff36806a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff37062a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff368071400 0x7ff368071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff37062a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff36806ced0 0x7ff36806d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff37062a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3680b2290 con 0x7ff36806ced0 2023-08-27T11:12:42.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff37062a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff36806aba0 con 0x7ff368071400 2023-08-27T11:12:42.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff37062a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680b1960 con 0x7ff36806a280 2023-08-27T11:12:42.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36ebc7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff36806ced0 0x7ff36806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36ebc7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff36806ced0 0x7ff36806d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52434/0 (socket says 172.21.15.45:52434) 2023-08-27T11:12:42.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36ebc7700 1 -- 172.21.15.45:0/2030125820 learned_addr learned my addr 172.21.15.45:0/2030125820 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:42.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36e3c6700 1 --2- 172.21.15.45:0/2030125820 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff36806a280 0x7ff36806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36dbc5700 1 --2- 172.21.15.45:0/2030125820 >> v2:172.21.15.135:3300/0 conn(0x7ff368071400 0x7ff368071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36ebc7700 1 -- 172.21.15.45:0/2030125820 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff36806a280 msgr2=0x7ff36806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:42.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36ebc7700 1 --2- 172.21.15.45:0/2030125820 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff36806a280 0x7ff36806a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36ebc7700 1 -- 172.21.15.45:0/2030125820 >> v2:172.21.15.135:3300/0 conn(0x7ff368071400 msgr2=0x7ff368071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:42.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36ebc7700 1 --2- 172.21.15.45:0/2030125820 >> v2:172.21.15.135:3300/0 conn(0x7ff368071400 0x7ff368071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.733+0000 7ff36ebc7700 1 -- 172.21.15.45:0/2030125820 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3680b1c40 con 0x7ff36806ced0 2023-08-27T11:12:42.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff36ebc7700 1 --2- 172.21.15.45:0/2030125820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 0x7ff36806d360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3680733d0 tx=0x7ff36000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451dd9b041074a5e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:42.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff36d3c4700 1 -- 172.21.15.45:0/2030125820 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36000e070 con 0x7ff36806ced0 2023-08-27T11:12:42.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff36d3c4700 1 -- 172.21.15.45:0/2030125820 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36000b9c0 con 0x7ff36806ced0 2023-08-27T11:12:42.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff36d3c4700 1 -- 172.21.15.45:0/2030125820 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36000bd70 con 0x7ff36806ced0 2023-08-27T11:12:42.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff37062a700 1 -- 172.21.15.45:0/2030125820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 msgr2=0x7ff36806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff37062a700 1 --2- 172.21.15.45:0/2030125820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 0x7ff36806d360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3680733d0 tx=0x7ff36000b330 comp rx=0 tx=0).stop 2023-08-27T11:12:42.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff37062a700 1 -- 172.21.15.45:0/2030125820 shutdown_connections 2023-08-27T11:12:42.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff37062a700 1 --2- 172.21.15.45:0/2030125820 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff36806a280 0x7ff36806a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff37062a700 1 --2- 172.21.15.45:0/2030125820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 0x7ff36806d360 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff37062a700 1 --2- 172.21.15.45:0/2030125820 >> v2:172.21.15.135:3300/0 conn(0x7ff368071400 0x7ff368071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff37062a700 1 -- 172.21.15.45:0/2030125820 >> 172.21.15.45:0/2030125820 conn(0x7ff3680cc7c0 msgr2=0x7ff3680ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:42.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff37062a700 1 -- 172.21.15.45:0/2030125820 shutdown_connections 2023-08-27T11:12:42.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.734+0000 7ff37062a700 1 -- 172.21.15.45:0/2030125820 wait complete. 2023-08-27T11:12:42.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff37062a700 1 Processor -- start 2023-08-27T11:12:42.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff37062a700 1 -- start start 2023-08-27T11:12:42.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff37062a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff36806a280 0x7ff3681f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff37062a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 0x7ff3680b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff37062a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368071400 0x7ff3680b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff37062a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36814a170 con 0x7ff36806ced0 2023-08-27T11:12:42.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff37062a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff368153950 con 0x7ff36806a280 2023-08-27T11:12:42.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff37062a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff368153c30 con 0x7ff368071400 2023-08-27T11:12:42.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff36dbc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 0x7ff3680b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff36dbc5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 0x7ff3680b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52438/0 (socket says 172.21.15.45:52438) 2023-08-27T11:12:42.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.736+0000 7ff36dbc5700 1 -- 172.21.15.45:0/2160148405 learned_addr learned my addr 172.21.15.45:0/2160148405 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:42.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff36e3c6700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff36806a280 0x7ff3681f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff36ebc7700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368071400 0x7ff3680b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff36dbc5700 1 -- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368071400 msgr2=0x7ff3680b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff36dbc5700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368071400 0x7ff3680b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff36dbc5700 1 -- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff36806a280 msgr2=0x7ff3681f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff36dbc5700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff36806a280 0x7ff3681f00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff36dbc5700 1 -- 172.21.15.45:0/2160148405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360009b30 con 0x7ff36806ced0 2023-08-27T11:12:42.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff36dbc5700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 0x7ff3680b0af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff36400d800 tx=0x7ff36400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:42.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff35effd700 1 -- 172.21.15.45:0/2160148405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3640125f0 con 0x7ff36806ced0 2023-08-27T11:12:42.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff35effd700 1 -- 172.21.15.45:0/2160148405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff364012750 con 0x7ff36806ced0 2023-08-27T11:12:42.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3680c9200 con 0x7ff36806ced0 2023-08-27T11:12:42.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff35effd700 1 -- 172.21.15.45:0/2160148405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364012b00 con 0x7ff36806ced0 2023-08-27T11:12:42.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.737+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3680c8e20 con 0x7ff36806ced0 2023-08-27T11:12:42.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.738+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff368072690 con 0x7ff36806ced0 2023-08-27T11:12:42.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.739+0000 7ff35effd700 1 -- 172.21.15.45:0/2160148405 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff364012c60 con 0x7ff36806ced0 2023-08-27T11:12:42.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.739+0000 7ff35effd700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff34c06a0e0 0x7ff34c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:42.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.739+0000 7ff35effd700 1 -- 172.21.15.45:0/2160148405 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff364088b10 con 0x7ff36806ced0 2023-08-27T11:12:42.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.739+0000 7ff36e3c6700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff34c06a0e0 0x7ff34c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:42.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.740+0000 7ff36e3c6700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff34c06a0e0 0x7ff34c06c5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3580096f0 tx=0x7ff358008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:42.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.743+0000 7ff35effd700 1 -- 172.21.15.45:0/2160148405 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff364053fa0 con 0x7ff36806ced0 2023-08-27T11:12:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.942+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff3680734a0 con 0x7ff36806ced0 2023-08-27T11:12:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.942+0000 7ff35effd700 1 -- 172.21.15.45:0/2160148405 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff3640575c0 con 0x7ff36806ced0 2023-08-27T11:12:42.945 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff34c06a0e0 msgr2=0x7ff34c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff34c06a0e0 0x7ff34c06c5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3580096f0 tx=0x7ff358008040 comp rx=0 tx=0).stop 2023-08-27T11:12:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 msgr2=0x7ff3680b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 0x7ff3680b0af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff36400d800 tx=0x7ff36400dbc0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 shutdown_connections 2023-08-27T11:12:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff34c06a0e0 0x7ff34c06c5a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff368071400 0x7ff3680b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff36806a280 0x7ff3681f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 --2- 172.21.15.45:0/2160148405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff36806ced0 0x7ff3680b0af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 >> 172.21.15.45:0/2160148405 conn(0x7ff3680cc7c0 msgr2=0x7ff36815c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 shutdown_connections 2023-08-27T11:12:42.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:42.944+0000 7ff37062a700 1 -- 172.21.15.45:0/2160148405 wait complete. 2023-08-27T11:12:42.958 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:12:43.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.082+0000 7fe974a70700 1 Processor -- start 2023-08-27T11:12:43.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.082+0000 7fe974a70700 1 -- start start 2023-08-27T11:12:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe974a70700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe970155660 0x7fe97006e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe974a70700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970070e80 0x7fe97015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe974a70700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9700cbf00 con 0x7fe97006a280 2023-08-27T11:12:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe974a70700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97015c760 con 0x7fe970155660 2023-08-27T11:12:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe974a70700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9700cc180 con 0x7fe970070e80 2023-08-27T11:12:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe970155660 0x7fe97006e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96ed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe97006a280 0x7fe97015ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39224/0 (socket says 172.21.15.45:39224) 2023-08-27T11:12:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96ed9d700 1 -- 172.21.15.45:0/1583613571 learned_addr learned my addr 172.21.15.45:0/1583613571 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96e59c700 1 -- 172.21.15.45:0/1583613571 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970070e80 msgr2=0x7fe97015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96e59c700 1 --2- 172.21.15.45:0/1583613571 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970070e80 0x7fe97015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96e59c700 1 -- 172.21.15.45:0/1583613571 >> v1:172.21.15.45:6789/0 conn(0x7fe97006a280 legacy=0x7fe97015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96e59c700 1 -- 172.21.15.45:0/1583613571 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9700caa60 con 0x7fe970155660 2023-08-27T11:12:43.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96d59a700 1 -- 172.21.15.45:0/1583613571 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265588046 0 0) 0x7fe9700cbf00 con 0x7fe97006a280 2023-08-27T11:12:43.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96e59c700 1 --2- 172.21.15.45:0/1583613571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe970155660 0x7fe97006e850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe960009840 tx=0x7fe960009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af013a84374f36db server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:43.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96d59a700 1 -- 172.21.15.45:0/1583613571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960011830 con 0x7fe970155660 2023-08-27T11:12:43.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96d59a700 1 -- 172.21.15.45:0/1583613571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960011990 con 0x7fe970155660 2023-08-27T11:12:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.083+0000 7fe96d59a700 1 -- 172.21.15.45:0/1583613571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960011d20 con 0x7fe970155660 2023-08-27T11:12:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.084+0000 7fe974a70700 1 -- 172.21.15.45:0/1583613571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe970155660 msgr2=0x7fe97006e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.084+0000 7fe974a70700 1 --2- 172.21.15.45:0/1583613571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe970155660 0x7fe97006e850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe960009840 tx=0x7fe960009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.084+0000 7fe974a70700 1 -- 172.21.15.45:0/1583613571 shutdown_connections 2023-08-27T11:12:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.084+0000 7fe974a70700 1 --2- 172.21.15.45:0/1583613571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe970155660 0x7fe97006e850 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.084+0000 7fe974a70700 1 --2- 172.21.15.45:0/1583613571 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970070e80 0x7fe97015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.084+0000 7fe974a70700 1 -- 172.21.15.45:0/1583613571 >> 172.21.15.45:0/1583613571 conn(0x7fe9700cfe50 msgr2=0x7fe9700d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.084+0000 7fe974a70700 1 -- 172.21.15.45:0/1583613571 shutdown_connections 2023-08-27T11:12:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.084+0000 7fe974a70700 1 -- 172.21.15.45:0/1583613571 wait complete. 2023-08-27T11:12:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.085+0000 7fe974a70700 1 Processor -- start 2023-08-27T11:12:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.085+0000 7fe974a70700 1 -- start start 2023-08-27T11:12:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe974a70700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe97006a280 0x7fe970149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe974a70700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe970070e80 0x7fe970153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe974a70700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970155660 0x7fe9700c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe974a70700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701587c0 con 0x7fe97006a280 2023-08-27T11:12:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe974a70700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701497c0 con 0x7fe970070e80 2023-08-27T11:12:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe974a70700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9701540d0 con 0x7fe970155660 2023-08-27T11:12:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe96e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe97006a280 0x7fe970149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe96e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe97006a280 0x7fe970149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52450/0 (socket says 172.21.15.45:52450) 2023-08-27T11:12:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe96e59c700 1 -- 172.21.15.45:0/2751068633 learned_addr learned my addr 172.21.15.45:0/2751068633 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe96dd9b700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe970070e80 0x7fe970153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.086+0000 7fe96ed9d700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970155660 0x7fe9700c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe96e59c700 1 -- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970155660 msgr2=0x7fe9700c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe96e59c700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970155660 0x7fe9700c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe96e59c700 1 -- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe970070e80 msgr2=0x7fe970153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe96e59c700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe970070e80 0x7fe970153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe96e59c700 1 -- 172.21.15.45:0/2751068633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600094a0 con 0x7fe97006a280 2023-08-27T11:12:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe96dd9b700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe970070e80 0x7fe970153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe96ed9d700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970155660 0x7fe9700c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe96e59c700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe97006a280 0x7fe970149280 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe960009b90 tx=0x7fe960004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe95effd700 1 -- 172.21.15.45:0/2751068633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600038a0 con 0x7fe97006a280 2023-08-27T11:12:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe95effd700 1 -- 172.21.15.45:0/2751068633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96001e070 con 0x7fe97006a280 2023-08-27T11:12:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9701543b0 con 0x7fe97006a280 2023-08-27T11:12:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe95effd700 1 -- 172.21.15.45:0/2751068633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960003c20 con 0x7fe97006a280 2023-08-27T11:12:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.087+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe970154970 con 0x7fe97006a280 2023-08-27T11:12:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.088+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe970154b60 con 0x7fe97006a280 2023-08-27T11:12:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.088+0000 7fe95effd700 1 -- 172.21.15.45:0/2751068633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe96002d450 con 0x7fe97006a280 2023-08-27T11:12:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.089+0000 7fe95effd700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe95006a0e0 0x7fe95006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.089+0000 7fe95effd700 1 -- 172.21.15.45:0/2751068633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe9600899e0 con 0x7fe97006a280 2023-08-27T11:12:43.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.089+0000 7fe96dd9b700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe95006a0e0 0x7fe95006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.090+0000 7fe96dd9b700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe95006a0e0 0x7fe95006c5a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe97006b150 tx=0x7fe964008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:43.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.094+0000 7fe95effd700 1 -- 172.21.15.45:0/2751068633 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe960054e70 con 0x7fe97006a280 2023-08-27T11:12:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.341+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe970072810 con 0x7fe97006a280 2023-08-27T11:12:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.342+0000 7fe95effd700 1 -- 172.21.15.45:0/2751068633 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fe960058490 con 0x7fe97006a280 2023-08-27T11:12:43.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:43.344 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:33.983169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:12:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe95006a0e0 msgr2=0x7fe95006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe95006a0e0 0x7fe95006c5a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe97006b150 tx=0x7fe964008040 comp rx=0 tx=0).stop 2023-08-27T11:12:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe97006a280 msgr2=0x7fe970149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe97006a280 0x7fe970149280 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe960009b90 tx=0x7fe960004750 comp rx=0 tx=0).stop 2023-08-27T11:12:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 shutdown_connections 2023-08-27T11:12:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe97006a280 0x7fe970149280 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe95006a0e0 0x7fe95006c5a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe970155660 0x7fe9700c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 --2- 172.21.15.45:0/2751068633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe970070e80 0x7fe970153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 >> 172.21.15.45:0/2751068633 conn(0x7fe9700cfe50 msgr2=0x7fe9700af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 shutdown_connections 2023-08-27T11:12:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.344+0000 7fe974a70700 1 -- 172.21.15.45:0/2751068633 wait complete. 2023-08-27T11:12:43.349 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-08-27T11:12:43.358 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:43.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.481+0000 7fbdb279e700 1 Processor -- start 2023-08-27T11:12:43.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.482+0000 7fbdb279e700 1 -- start start 2023-08-27T11:12:43.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.482+0000 7fbdb279e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbdac15d400 0x7fbdac0cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.482+0000 7fbdb279e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbdac0cd2a0 0x7fbdac06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.482+0000 7fbdb279e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac06a7c0 0x7fbdac0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.482+0000 7fbdb279e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdac06ac50 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.482+0000 7fbdb279e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdac0cb300 con 0x7fbdac15d400 2023-08-27T11:12:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.482+0000 7fbdb279e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0ca6f0 con 0x7fbdac06a7c0 2023-08-27T11:12:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb0f9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbdac0cd2a0 0x7fbdac06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb0f9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbdac0cd2a0 0x7fbdac06a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52458/0 (socket says 172.21.15.45:52458) 2023-08-27T11:12:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb0f9b700 1 -- 172.21.15.45:0/139486970 learned_addr learned my addr 172.21.15.45:0/139486970 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb179c700 1 --2- 172.21.15.45:0/139486970 >> v2:172.21.15.135:3300/0 conn(0x7fbdac15d400 0x7fbdac0cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb1f9d700 1 --2- 172.21.15.45:0/139486970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac06a7c0 0x7fbdac0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb0f9b700 1 -- 172.21.15.45:0/139486970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac06a7c0 msgr2=0x7fbdac0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb0f9b700 1 --2- 172.21.15.45:0/139486970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac06a7c0 0x7fbdac0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb0f9b700 1 -- 172.21.15.45:0/139486970 >> v2:172.21.15.135:3300/0 conn(0x7fbdac15d400 msgr2=0x7fbdac0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:43.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb0f9b700 1 --2- 172.21.15.45:0/139486970 >> v2:172.21.15.135:3300/0 conn(0x7fbdac15d400 0x7fbdac0cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb0f9b700 1 -- 172.21.15.45:0/139486970 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdac0ca9b0 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb1f9d700 1 --2- 172.21.15.45:0/139486970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac06a7c0 0x7fbdac0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:43.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbdb0f9b700 1 --2- 172.21.15.45:0/139486970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac06a280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbdac0704c0 tx=0x7fbda8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b4747edfc8158e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:43.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbda3fff700 1 -- 172.21.15.45:0/139486970 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8011830 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbda3fff700 1 -- 172.21.15.45:0/139486970 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda8011990 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.483+0000 7fbda3fff700 1 -- 172.21.15.45:0/139486970 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8011d20 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.484+0000 7fbdb279e700 1 -- 172.21.15.45:0/139486970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 msgr2=0x7fbdac06a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.484+0000 7fbdb279e700 1 --2- 172.21.15.45:0/139486970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac06a280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbdac0704c0 tx=0x7fbda8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.484+0000 7fbdb279e700 1 -- 172.21.15.45:0/139486970 shutdown_connections 2023-08-27T11:12:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.484+0000 7fbdb279e700 1 --2- 172.21.15.45:0/139486970 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac06a7c0 0x7fbdac0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.484+0000 7fbdb279e700 1 --2- 172.21.15.45:0/139486970 >> v2:172.21.15.135:3300/0 conn(0x7fbdac15d400 0x7fbdac0cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.484+0000 7fbdb279e700 1 --2- 172.21.15.45:0/139486970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac06a280 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.484+0000 7fbdb279e700 1 -- 172.21.15.45:0/139486970 >> 172.21.15.45:0/139486970 conn(0x7fbdac0cc780 msgr2=0x7fbdac0ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:43.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.484+0000 7fbdb279e700 1 -- 172.21.15.45:0/139486970 shutdown_connections 2023-08-27T11:12:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.484+0000 7fbdb279e700 1 -- 172.21.15.45:0/139486970 wait complete. 2023-08-27T11:12:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.485+0000 7fbdb279e700 1 Processor -- start 2023-08-27T11:12:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.485+0000 7fbdb279e700 1 -- start start 2023-08-27T11:12:43.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb279e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdac06a7c0 0x7fbdac152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb279e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb279e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac15d400 0x7fbdac15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb279e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0c3c40 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb279e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac15d7e0 con 0x7fbdac06a7c0 2023-08-27T11:12:43.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb279e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac0c95d0 con 0x7fbdac15d400 2023-08-27T11:12:43.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb0f9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb0f9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac15ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52466/0 (socket says 172.21.15.45:52466) 2023-08-27T11:12:43.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb0f9b700 1 -- 172.21.15.45:0/757644503 learned_addr learned my addr 172.21.15.45:0/757644503 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:43.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb179c700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdac06a7c0 0x7fbdac152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb1f9d700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac15d400 0x7fbdac15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb0f9b700 1 -- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac15d400 msgr2=0x7fbdac15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb0f9b700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac15d400 0x7fbdac15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb0f9b700 1 -- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdac06a7c0 msgr2=0x7fbdac152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb0f9b700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdac06a7c0 0x7fbdac152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.486+0000 7fbdb0f9b700 1 -- 172.21.15.45:0/757644503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda8009550 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.487+0000 7fbdb0f9b700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac15ad40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbda8000c00 tx=0x7fbda8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:43.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.487+0000 7fbda1ffb700 1 -- 172.21.15.45:0/757644503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8004240 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.487+0000 7fbda1ffb700 1 -- 172.21.15.45:0/757644503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda801f070 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.487+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdac0c98b0 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.487+0000 7fbda1ffb700 1 -- 172.21.15.45:0/757644503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda80045e0 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.487+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdac0c38d0 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.488+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdac0c3db0 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.488+0000 7fbda1ffb700 1 -- 172.21.15.45:0/757644503 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbda8021030 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.489+0000 7fbda1ffb700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd9006a0e0 0x7fbd9006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.489+0000 7fbda1ffb700 1 -- 172.21.15.45:0/757644503 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbda80898f0 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.489+0000 7fbdb179c700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd9006a0e0 0x7fbd9006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.489+0000 7fbdb179c700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd9006a0e0 0x7fbd9006c5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009ac0 tx=0x7fbd9c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:43.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.493+0000 7fbda1ffb700 1 -- 172.21.15.45:0/757644503 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbda8054d80 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.694+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdac0726a0 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.696+0000 7fbda1ffb700 1 -- 172.21.15.45:0/757644503 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7fbda80583a0 con 0x7fbdac0cd2a0 2023-08-27T11:12:43.698 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:43.698 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd9006a0e0 msgr2=0x7fbd9006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd9006a0e0 0x7fbd9006c5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009ac0 tx=0x7fbd9c009210 comp rx=0 tx=0).stop 2023-08-27T11:12:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 msgr2=0x7fbdac15ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac15ad40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbda8000c00 tx=0x7fbda8004750 comp rx=0 tx=0).stop 2023-08-27T11:12:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 shutdown_connections 2023-08-27T11:12:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbd9006a0e0 0x7fbd9006c5a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdac15d400 0x7fbdac15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdac06a7c0 0x7fbdac152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 --2- 172.21.15.45:0/757644503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdac0cd2a0 0x7fbdac15ad40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 >> 172.21.15.45:0/757644503 conn(0x7fbdac0cc780 msgr2=0x7fbdac071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 shutdown_connections 2023-08-27T11:12:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.698+0000 7fbdb279e700 1 -- 172.21.15.45:0/757644503 wait complete. 2023-08-27T11:12:43.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:12:43.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:12:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.840+0000 7fef6f835700 1 Processor -- start 2023-08-27T11:12:43.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.840+0000 7fef6f835700 1 -- start start 2023-08-27T11:12:43.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.841+0000 7fef6f835700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fef68157be0 0x7fef680716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.841+0000 7fef6f835700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6815b460 con 0x7fef6806d5e0 2023-08-27T11:12:43.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.841+0000 7fef6f835700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef680b2280 con 0x7fef680b17c0 2023-08-27T11:12:43.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.841+0000 7fef6f835700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef68157fc0 con 0x7fef68157be0 2023-08-27T11:12:43.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.841+0000 7fef6ddd2700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef6806d5e0 0x7fef680cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39252/0 (socket says 172.21.15.45:39252) 2023-08-27T11:12:43.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.841+0000 7fef6ddd2700 1 -- 172.21.15.45:0/2718007057 learned_addr learned my addr 172.21.15.45:0/2718007057 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:43.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef6d5d1700 1 --2- 172.21.15.45:0/2718007057 >> v2:172.21.15.135:3300/0 conn(0x7fef68157be0 0x7fef680716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361556592 0 0) 0x7fef6815b460 con 0x7fef6806d5e0 2023-08-27T11:12:43.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef540057c0 con 0x7fef6806d5e0 2023-08-27T11:12:43.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1670206509 0 0) 0x7fef540057c0 con 0x7fef6806d5e0 2023-08-27T11:12:43.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 >> v2:172.21.15.135:3300/0 conn(0x7fef68157be0 msgr2=0x7fef680716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:43.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 --2- 172.21.15.45:0/2718007057 >> v2:172.21.15.135:3300/0 conn(0x7fef68157be0 0x7fef680716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 >> v1:172.21.15.135:6789/0 conn(0x7fef680b17c0 legacy=0x7fef6806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680b2560 con 0x7fef6806d5e0 2023-08-27T11:12:43.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fef600019f0 con 0x7fef6806d5e0 2023-08-27T11:12:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 >> v1:172.21.15.45:6789/0 conn(0x7fef6806d5e0 legacy=0x7fef680cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef54005c10 0x7fef54006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef54006a60 0x7fef54006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef5400dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68157fc0 con 0x7fef54007420 2023-08-27T11:12:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5400e2c0 con 0x7fef54005c10 2023-08-27T11:12:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5400e480 con 0x7fef54006a60 2023-08-27T11:12:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef6d5d1700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef54005c10 0x7fef54006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef6ddd2700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef5400dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef6cdd0700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef54006a60 0x7fef54006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef6ddd2700 1 -- 172.21.15.45:0/2718007057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef54006a60 msgr2=0x7fef54006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:43.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef6ddd2700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef54006a60 0x7fef54006ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef6ddd2700 1 -- 172.21.15.45:0/2718007057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef54005c10 msgr2=0x7fef54006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:43.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.842+0000 7fef6ddd2700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef54005c10 0x7fef54006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef6ddd2700 1 -- 172.21.15.45:0/2718007057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef540057c0 con 0x7fef54007420 2023-08-27T11:12:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef6ddd2700 1 -- 172.21.15.45:0/2718007057 reap_dead start 2023-08-27T11:12:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef6ddd2700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef5400dc20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef6000a410 tx=0x7fef54007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c1a3af0cbd9c91 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60011880 con 0x7fef54007420 2023-08-27T11:12:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60011a00 con 0x7fef54007420 2023-08-27T11:12:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef5ffff700 1 -- 172.21.15.45:0/2718007057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60011db0 con 0x7fef54007420 2023-08-27T11:12:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef6f835700 1 -- 172.21.15.45:0/2718007057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 msgr2=0x7fef5400dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef6f835700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef5400dc20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef6000a410 tx=0x7fef54007a90 comp rx=0 tx=0).stop 2023-08-27T11:12:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef6f835700 1 -- 172.21.15.45:0/2718007057 shutdown_connections 2023-08-27T11:12:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef6f835700 1 --2- 172.21.15.45:0/2718007057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef5400dc20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.843+0000 7fef6f835700 1 -- 172.21.15.45:0/2718007057 >> 172.21.15.45:0/2718007057 conn(0x7fef680cc7c0 msgr2=0x7fef680ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.844+0000 7fef6f835700 1 -- 172.21.15.45:0/2718007057 shutdown_connections 2023-08-27T11:12:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.844+0000 7fef6f835700 1 -- 172.21.15.45:0/2718007057 wait complete. 2023-08-27T11:12:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.845+0000 7fef6f835700 1 Processor -- start 2023-08-27T11:12:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.845+0000 7fef6f835700 1 -- start start 2023-08-27T11:12:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6f835700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef68147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6f835700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef680b2c90 0x7fef6815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6f835700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef6815c570 0x7fef680b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6f835700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6815d000 con 0x7fef54007420 2023-08-27T11:12:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6f835700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68154350 con 0x7fef6815c570 2023-08-27T11:12:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6f835700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef681ee3a0 con 0x7fef680b2c90 2023-08-27T11:12:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6d5d1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef68147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6d5d1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef68147fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52484/0 (socket says 172.21.15.45:52484) 2023-08-27T11:12:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6d5d1700 1 -- 172.21.15.45:0/3267715053 learned_addr learned my addr 172.21.15.45:0/3267715053 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6cdd0700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef680b2c90 0x7fef6815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6ddd2700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef6815c570 0x7fef680b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6d5d1700 1 -- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef680b2c90 msgr2=0x7fef6815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6d5d1700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef680b2c90 0x7fef6815c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6d5d1700 1 -- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef6815c570 msgr2=0x7fef680b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:43.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6d5d1700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef6815c570 0x7fef680b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:43.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6d5d1700 1 -- 172.21.15.45:0/3267715053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6815ca40 con 0x7fef54007420 2023-08-27T11:12:43.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6ddd2700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef6815c570 0x7fef680b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6d5d1700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef68147fa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef580119a0 tx=0x7fef58011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef5dffb700 1 -- 172.21.15.45:0/3267715053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800f940 con 0x7fef54007420 2023-08-27T11:12:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef5dffb700 1 -- 172.21.15.45:0/3267715053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5800faa0 con 0x7fef54007420 2023-08-27T11:12:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6815cd20 con 0x7fef54007420 2023-08-27T11:12:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef5dffb700 1 -- 172.21.15.45:0/3267715053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800fe30 con 0x7fef54007420 2023-08-27T11:12:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.846+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef681484e0 con 0x7fef54007420 2023-08-27T11:12:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.847+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef68154630 con 0x7fef54007420 2023-08-27T11:12:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.848+0000 7fef5dffb700 1 -- 172.21.15.45:0/3267715053 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fef58014440 con 0x7fef54007420 2023-08-27T11:12:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.848+0000 7fef5dffb700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fef4c06a0e0 0x7fef4c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.848+0000 7fef5dffb700 1 -- 172.21.15.45:0/3267715053 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fef58088fe0 con 0x7fef54007420 2023-08-27T11:12:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.848+0000 7fef6cdd0700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fef4c06a0e0 0x7fef4c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.849+0000 7fef6cdd0700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fef4c06a0e0 0x7fef4c06c5a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef680ca260 tx=0x7fef64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:43.853+0000 7fef5dffb700 1 -- 172.21.15.45:0/3267715053 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fef58057e60 con 0x7fef54007420 2023-08-27T11:12:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.053+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef68072690 con 0x7fef54007420 2023-08-27T11:12:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.053+0000 7fef5dffb700 1 -- 172.21.15.45:0/3267715053 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef580579f0 con 0x7fef54007420 2023-08-27T11:12:44.055 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:12:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fef4c06a0e0 msgr2=0x7fef4c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fef4c06a0e0 0x7fef4c06c5a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef680ca260 tx=0x7fef64009040 comp rx=0 tx=0).stop 2023-08-27T11:12:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 msgr2=0x7fef68147fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef68147fa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef580119a0 tx=0x7fef58011d60 comp rx=0 tx=0).stop 2023-08-27T11:12:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 shutdown_connections 2023-08-27T11:12:44.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef54007420 0x7fef68147fa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fef4c06a0e0 0x7fef4c06c5a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef6815c570 0x7fef680b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 --2- 172.21.15.45:0/3267715053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef680b2c90 0x7fef6815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.055+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 >> 172.21.15.45:0/3267715053 conn(0x7fef680cc7c0 msgr2=0x7fef680ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:44.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.056+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 shutdown_connections 2023-08-27T11:12:44.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.056+0000 7fef6f835700 1 -- 172.21.15.45:0/3267715053 wait complete. 2023-08-27T11:12:44.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:12:44.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.196+0000 7fc8cc937700 1 Processor -- start 2023-08-27T11:12:44.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.196+0000 7fc8cc937700 1 -- start start 2023-08-27T11:12:44.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8cc937700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8c815a700 0x7fc8c815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8cc937700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 0x7fc8c815af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8cc937700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c80cb060 con 0x7fc8c806ef10 2023-08-27T11:12:44.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8cc937700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c806e320 con 0x7fc8c815a700 2023-08-27T11:12:44.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8cc937700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c815d7d0 con 0x7fc8c815aae0 2023-08-27T11:12:44.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8c815a700 0x7fc8c815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8c815a700 0x7fc8c815b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52492/0 (socket says 172.21.15.45:52492) 2023-08-27T11:12:44.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c659c700 1 -- 172.21.15.45:0/19775073 learned_addr learned my addr 172.21.15.45:0/19775073 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:44.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c5d9b700 1 --2- 172.21.15.45:0/19775073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 0x7fc8c815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c659c700 1 -- 172.21.15.45:0/19775073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 msgr2=0x7fc8c815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:44.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c659c700 1 --2- 172.21.15.45:0/19775073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 0x7fc8c815af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c659c700 1 -- 172.21.15.45:0/19775073 >> v1:172.21.15.135:6789/0 conn(0x7fc8c806ef10 legacy=0x7fc8c80cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:44.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c659c700 1 -- 172.21.15.45:0/19775073 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c80ca450 con 0x7fc8c815a700 2023-08-27T11:12:44.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c5d9b700 1 --2- 172.21.15.45:0/19775073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 0x7fc8c815af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:44.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c659c700 1 --2- 172.21.15.45:0/19775073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c815a700 0x7fc8c815b5e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80098f0 tx=0x7fc8b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0c8c8d9b8eb21e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:44.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c559a700 1 -- 172.21.15.45:0/19775073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8004230 con 0x7fc8c815a700 2023-08-27T11:12:44.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c559a700 1 -- 172.21.15.45:0/19775073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b8004390 con 0x7fc8c815a700 2023-08-27T11:12:44.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.197+0000 7fc8c559a700 1 -- 172.21.15.45:0/19775073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80036a0 con 0x7fc8c815a700 2023-08-27T11:12:44.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.198+0000 7fc8cc937700 1 -- 172.21.15.45:0/19775073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c815a700 msgr2=0x7fc8c815b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.198+0000 7fc8cc937700 1 --2- 172.21.15.45:0/19775073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c815a700 0x7fc8c815b5e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80098f0 tx=0x7fc8b8009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:44.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.198+0000 7fc8cc937700 1 -- 172.21.15.45:0/19775073 shutdown_connections 2023-08-27T11:12:44.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.198+0000 7fc8cc937700 1 --2- 172.21.15.45:0/19775073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c815a700 0x7fc8c815b5e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.198+0000 7fc8cc937700 1 --2- 172.21.15.45:0/19775073 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 0x7fc8c815af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.198+0000 7fc8cc937700 1 -- 172.21.15.45:0/19775073 >> 172.21.15.45:0/19775073 conn(0x7fc8c80cc7e0 msgr2=0x7fc8c80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.198+0000 7fc8cc937700 1 -- 172.21.15.45:0/19775073 shutdown_connections 2023-08-27T11:12:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.198+0000 7fc8cc937700 1 -- 172.21.15.45:0/19775073 wait complete. 2023-08-27T11:12:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.200+0000 7fc8cc937700 1 Processor -- start 2023-08-27T11:12:44.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.200+0000 7fc8cc937700 1 -- start start 2023-08-27T11:12:44.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8cc937700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c806ef10 0x7fc8c8152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8cc937700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8c815a700 0x7fc8c806a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8cc937700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 0x7fc8c806a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8cc937700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80c8e00 con 0x7fc8c806ef10 2023-08-27T11:12:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8cc937700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c81f6b80 con 0x7fc8c815a700 2023-08-27T11:12:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8cc937700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c81f66f0 con 0x7fc8c815aae0 2023-08-27T11:12:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c806ef10 0x7fc8c8152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c806ef10 0x7fc8c8152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52500/0 (socket says 172.21.15.45:52500) 2023-08-27T11:12:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c659c700 1 -- 172.21.15.45:0/4068160086 learned_addr learned my addr 172.21.15.45:0/4068160086 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c5d9b700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8c815a700 0x7fc8c806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c6d9d700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 0x7fc8c806a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c659c700 1 -- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 msgr2=0x7fc8c806a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c659c700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 0x7fc8c806a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c659c700 1 -- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8c815a700 msgr2=0x7fc8c806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c659c700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8c815a700 0x7fc8c806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.201+0000 7fc8c659c700 1 -- 172.21.15.45:0/4068160086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8009550 con 0x7fc8c806ef10 2023-08-27T11:12:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.202+0000 7fc8c659c700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c806ef10 0x7fc8c8152e70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b0e0 tx=0x7fc8b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.202+0000 7fc8b6ffd700 1 -- 172.21.15.45:0/4068160086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800b660 con 0x7fc8c806ef10 2023-08-27T11:12:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.202+0000 7fc8b6ffd700 1 -- 172.21.15.45:0/4068160086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b800b7c0 con 0x7fc8c806ef10 2023-08-27T11:12:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.202+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c81533b0 con 0x7fc8c806ef10 2023-08-27T11:12:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.202+0000 7fc8b6ffd700 1 -- 172.21.15.45:0/4068160086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800bb70 con 0x7fc8c806ef10 2023-08-27T11:12:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.202+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c81f60a0 con 0x7fc8c806ef10 2023-08-27T11:12:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.203+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c81f69d0 con 0x7fc8c806ef10 2023-08-27T11:12:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.203+0000 7fc8b6ffd700 1 -- 172.21.15.45:0/4068160086 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc8b800bcd0 con 0x7fc8c806ef10 2023-08-27T11:12:44.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.204+0000 7fc8b6ffd700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc8a806a0e0 0x7fc8a806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.204+0000 7fc8b6ffd700 1 -- 172.21.15.45:0/4068160086 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc8b808a030 con 0x7fc8c806ef10 2023-08-27T11:12:44.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.204+0000 7fc8c5d9b700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc8a806a0e0 0x7fc8a806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.204+0000 7fc8c5d9b700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc8a806a0e0 0x7fc8a806c5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80c6de0 tx=0x7fc8bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:44.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.208+0000 7fc8b6ffd700 1 -- 172.21.15.45:0/4068160086 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc8b80554c0 con 0x7fc8c806ef10 2023-08-27T11:12:44.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.409+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc8c80726d0 con 0x7fc8c806ef10 2023-08-27T11:12:44.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.409+0000 7fc8b6ffd700 1 -- 172.21.15.45:0/4068160086 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc8b801f020 con 0x7fc8c806ef10 2023-08-27T11:12:44.412 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:12:44.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.411+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc8a806a0e0 msgr2=0x7fc8a806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.411+0000 7fc8cc937700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc8a806a0e0 0x7fc8a806c5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80c6de0 tx=0x7fc8bc008040 comp rx=0 tx=0).stop 2023-08-27T11:12:44.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.411+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c806ef10 msgr2=0x7fc8c8152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.411+0000 7fc8cc937700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c806ef10 0x7fc8c8152e70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b0e0 tx=0x7fc8b8004750 comp rx=0 tx=0).stop 2023-08-27T11:12:44.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.411+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 shutdown_connections 2023-08-27T11:12:44.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.411+0000 7fc8cc937700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8c806ef10 0x7fc8c8152e70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.411+0000 7fc8cc937700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc8a806a0e0 0x7fc8a806c5a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.411+0000 7fc8cc937700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc8c815aae0 0x7fc8c806a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.411+0000 7fc8cc937700 1 --2- 172.21.15.45:0/4068160086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8c815a700 0x7fc8c806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.412+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 >> 172.21.15.45:0/4068160086 conn(0x7fc8c80cc7e0 msgr2=0x7fc8c80cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:44.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.412+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 shutdown_connections 2023-08-27T11:12:44.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.412+0000 7fc8cc937700 1 -- 172.21.15.45:0/4068160086 wait complete. 2023-08-27T11:12:44.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:12:44.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.551+0000 7fc280d04700 1 Processor -- start 2023-08-27T11:12:44.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.551+0000 7fc280d04700 1 -- start start 2023-08-27T11:12:44.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.552+0000 7fc280d04700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc27c161a00 0x7fc27c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.552+0000 7fc280d04700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 0x7fc27c15d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.552+0000 7fc280d04700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27c161f00 con 0x7fc27c15d470 2023-08-27T11:12:44.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.552+0000 7fc280d04700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27c0ce2b0 con 0x7fc27c161a00 2023-08-27T11:12:44.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.552+0000 7fc280d04700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c06d410 con 0x7fc27c0cde80 2023-08-27T11:12:44.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.552+0000 7fc279d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc27c161a00 0x7fc27c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.552+0000 7fc279d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc27c161a00 0x7fc27c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52508/0 (socket says 172.21.15.45:52508) 2023-08-27T11:12:44.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.552+0000 7fc279d9b700 1 -- 172.21.15.45:0/1017834184 learned_addr learned my addr 172.21.15.45:0/1017834184 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:44.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc27ad9d700 1 --2- 172.21.15.45:0/1017834184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 0x7fc27c15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc279d9b700 1 -- 172.21.15.45:0/1017834184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 msgr2=0x7fc27c15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:44.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc279d9b700 1 --2- 172.21.15.45:0/1017834184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 0x7fc27c15d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc279d9b700 1 -- 172.21.15.45:0/1017834184 >> v1:172.21.15.135:6789/0 conn(0x7fc27c15d470 legacy=0x7fc27c0d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc279d9b700 1 -- 172.21.15.45:0/1017834184 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c0cec30 con 0x7fc27c161a00 2023-08-27T11:12:44.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc27ad9d700 1 --2- 172.21.15.45:0/1017834184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 0x7fc27c15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:44.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc279d9b700 1 --2- 172.21.15.45:0/1017834184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 0x7fc27c0cd940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0098f0 tx=0x7fc26c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82168aefa39f08b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:44.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc27959a700 1 -- 172.21.15.45:0/1017834184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00b8a0 con 0x7fc27c161a00 2023-08-27T11:12:44.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc27959a700 1 -- 172.21.15.45:0/1017834184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c00ba00 con 0x7fc27c161a00 2023-08-27T11:12:44.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.553+0000 7fc27959a700 1 -- 172.21.15.45:0/1017834184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00bd90 con 0x7fc27c161a00 2023-08-27T11:12:44.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.554+0000 7fc280d04700 1 -- 172.21.15.45:0/1017834184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 msgr2=0x7fc27c0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.554+0000 7fc280d04700 1 --2- 172.21.15.45:0/1017834184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 0x7fc27c0cd940 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0098f0 tx=0x7fc26c009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:44.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.554+0000 7fc280d04700 1 -- 172.21.15.45:0/1017834184 shutdown_connections 2023-08-27T11:12:44.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.554+0000 7fc280d04700 1 --2- 172.21.15.45:0/1017834184 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 0x7fc27c15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.554+0000 7fc280d04700 1 --2- 172.21.15.45:0/1017834184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 0x7fc27c0cd940 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.554+0000 7fc280d04700 1 -- 172.21.15.45:0/1017834184 >> 172.21.15.45:0/1017834184 conn(0x7fc27c06fa40 msgr2=0x7fc27c06fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:44.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.554+0000 7fc280d04700 1 -- 172.21.15.45:0/1017834184 shutdown_connections 2023-08-27T11:12:44.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.554+0000 7fc280d04700 1 -- 172.21.15.45:0/1017834184 wait complete. 2023-08-27T11:12:44.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.555+0000 7fc280d04700 1 Processor -- start 2023-08-27T11:12:44.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.555+0000 7fc280d04700 1 -- start start 2023-08-27T11:12:44.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc280d04700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 0x7fc27c0b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc280d04700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c15d470 0x7fc27c157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc280d04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 0x7fc27c158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc280d04700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0b1830 con 0x7fc27c161a00 2023-08-27T11:12:44.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc280d04700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c158590 con 0x7fc27c15d470 2023-08-27T11:12:44.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc280d04700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0b0f20 con 0x7fc27c0cde80 2023-08-27T11:12:44.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc27ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 0x7fc27c158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc27ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 0x7fc27c158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52516/0 (socket says 172.21.15.45:52516) 2023-08-27T11:12:44.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc27ad9d700 1 -- 172.21.15.45:0/3789046196 learned_addr learned my addr 172.21.15.45:0/3789046196 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:44.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc279d9b700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c15d470 0x7fc27c157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc27a59c700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 0x7fc27c0b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc27ad9d700 1 -- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 msgr2=0x7fc27c0b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc27ad9d700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 0x7fc27c0b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc27ad9d700 1 -- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c15d470 msgr2=0x7fc27c157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc27ad9d700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c15d470 0x7fc27c157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc27ad9d700 1 -- 172.21.15.45:0/3789046196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c009550 con 0x7fc27c161a00 2023-08-27T11:12:44.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.556+0000 7fc279d9b700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c15d470 0x7fc27c157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:44.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.557+0000 7fc27ad9d700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 0x7fc27c158050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc274007df0 tx=0x7fc27400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:44.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.557+0000 7fc26affd700 1 -- 172.21.15.45:0/3789046196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400ddd0 con 0x7fc27c161a00 2023-08-27T11:12:44.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.557+0000 7fc26affd700 1 -- 172.21.15.45:0/3789046196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27400f040 con 0x7fc27c161a00 2023-08-27T11:12:44.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.557+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27c0b3130 con 0x7fc27c161a00 2023-08-27T11:12:44.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.557+0000 7fc26affd700 1 -- 172.21.15.45:0/3789046196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274003c60 con 0x7fc27c161a00 2023-08-27T11:12:44.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.557+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c0b1450 con 0x7fc27c161a00 2023-08-27T11:12:44.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.558+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27c0b1970 con 0x7fc27c161a00 2023-08-27T11:12:44.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.558+0000 7fc26affd700 1 -- 172.21.15.45:0/3789046196 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc274004030 con 0x7fc27c161a00 2023-08-27T11:12:44.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.559+0000 7fc26affd700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc25c06a0e0 0x7fc25c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.559+0000 7fc26affd700 1 -- 172.21.15.45:0/3789046196 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc2740889b0 con 0x7fc27c161a00 2023-08-27T11:12:44.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.559+0000 7fc27a59c700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc25c06a0e0 0x7fc25c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.559+0000 7fc27a59c700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc25c06a0e0 0x7fc25c06c5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0b2b20 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:44.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.563+0000 7fc26affd700 1 -- 172.21.15.45:0/3789046196 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc27c0b1970 con 0x7fc27c161a00 2023-08-27T11:12:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.764+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc27c0b1970 con 0x7fc27c161a00 2023-08-27T11:12:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.765+0000 7fc26affd700 1 -- 172.21.15.45:0/3789046196 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc27c0b1970 con 0x7fc27c161a00 2023-08-27T11:12:44.767 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:44.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc25c06a0e0 msgr2=0x7fc25c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc25c06a0e0 0x7fc25c06c5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0b2b20 tx=0x7fc270009040 comp rx=0 tx=0).stop 2023-08-27T11:12:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 msgr2=0x7fc27c158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 0x7fc27c158050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc274007df0 tx=0x7fc27400d3b0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 shutdown_connections 2023-08-27T11:12:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc27c161a00 0x7fc27c158050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc25c06a0e0 0x7fc25c06c5a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc27c0cde80 0x7fc27c0b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 --2- 172.21.15.45:0/3789046196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c15d470 0x7fc27c157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 >> 172.21.15.45:0/3789046196 conn(0x7fc27c06fa40 msgr2=0x7fc27c0b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 shutdown_connections 2023-08-27T11:12:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.767+0000 7fc280d04700 1 -- 172.21.15.45:0/3789046196 wait complete. 2023-08-27T11:12:44.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:12:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.907+0000 7fdbe49dc700 1 Processor -- start 2023-08-27T11:12:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.907+0000 7fdbe49dc700 1 -- start start 2023-08-27T11:12:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.908+0000 7fdbe49dc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdbe00b65d0 0x7fdbe00ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.908+0000 7fdbe49dc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cb5c0 0x7fdbe015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.908+0000 7fdbe49dc700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe015ec50 con 0x7fdbe00cfe10 2023-08-27T11:12:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.908+0000 7fdbe49dc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe015c620 con 0x7fdbe00b65d0 2023-08-27T11:12:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.908+0000 7fdbe49dc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe015eea0 con 0x7fdbe00cb5c0 2023-08-27T11:12:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbddd9b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cb5c0 0x7fdbe015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbde59c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdbe00b65d0 0x7fdbe00ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbde59c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdbe00b65d0 0x7fdbe00ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41376/0 (socket says 172.21.15.45:41376) 2023-08-27T11:12:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbddd9b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cb5c0 0x7fdbe015c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50430/0 (socket says 172.21.15.45:50430) 2023-08-27T11:12:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbde59c700 1 -- 172.21.15.45:0/1387072303 learned_addr learned my addr 172.21.15.45:0/1387072303 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbddd9b700 1 -- 172.21.15.45:0/1387072303 >> v2:172.21.15.135:3300/0 conn(0x7fdbe00b65d0 msgr2=0x7fdbe00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbddd9b700 1 --2- 172.21.15.45:0/1387072303 >> v2:172.21.15.135:3300/0 conn(0x7fdbe00b65d0 0x7fdbe00ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbddd9b700 1 -- 172.21.15.45:0/1387072303 >> v1:172.21.15.135:6789/0 conn(0x7fdbe00cfe10 legacy=0x7fdbe00cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbddd9b700 1 -- 172.21.15.45:0/1387072303 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe015f180 con 0x7fdbe00cb5c0 2023-08-27T11:12:44.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbde59c700 1 --2- 172.21.15.45:0/1387072303 >> v2:172.21.15.135:3300/0 conn(0x7fdbe00b65d0 0x7fdbe00ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:44.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbddd9b700 1 --2- 172.21.15.45:0/1387072303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cb5c0 0x7fdbe015c0e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009840 tx=0x7fdbd0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8afc986d6f320a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:44.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.909+0000 7fdbdd59a700 1 -- 172.21.15.45:0/1387072303 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0011830 con 0x7fdbe00cb5c0 2023-08-27T11:12:44.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbdd59a700 1 -- 172.21.15.45:0/1387072303 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd0011990 con 0x7fdbe00cb5c0 2023-08-27T11:12:44.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbdd59a700 1 -- 172.21.15.45:0/1387072303 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0011d20 con 0x7fdbe00cb5c0 2023-08-27T11:12:44.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbe49dc700 1 -- 172.21.15.45:0/1387072303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cb5c0 msgr2=0x7fdbe015c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbe49dc700 1 --2- 172.21.15.45:0/1387072303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cb5c0 0x7fdbe015c0e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009840 tx=0x7fdbd0009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:44.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbe49dc700 1 -- 172.21.15.45:0/1387072303 shutdown_connections 2023-08-27T11:12:44.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbe49dc700 1 --2- 172.21.15.45:0/1387072303 >> v2:172.21.15.135:3300/0 conn(0x7fdbe00b65d0 0x7fdbe00ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbe49dc700 1 --2- 172.21.15.45:0/1387072303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cb5c0 0x7fdbe015c0e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbe49dc700 1 -- 172.21.15.45:0/1387072303 >> 172.21.15.45:0/1387072303 conn(0x7fdbe006f960 msgr2=0x7fdbe006fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:44.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbe49dc700 1 -- 172.21.15.45:0/1387072303 shutdown_connections 2023-08-27T11:12:44.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.910+0000 7fdbe49dc700 1 -- 172.21.15.45:0/1387072303 wait complete. 2023-08-27T11:12:44.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.911+0000 7fdbe49dc700 1 Processor -- start 2023-08-27T11:12:44.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbe49dc700 1 -- start start 2023-08-27T11:12:44.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbe49dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbe00b65d0 0x7fdbe006cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbe49dc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe00cb5c0 0x7fdbe00c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbe49dc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cfe10 0x7fdbe00b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbe49dc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00c9330 con 0x7fdbe00b65d0 2023-08-27T11:12:44.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbe49dc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe006d430 con 0x7fdbe00cb5c0 2023-08-27T11:12:44.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbe49dc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00c8480 con 0x7fdbe00cfe10 2023-08-27T11:12:44.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbde59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbe00b65d0 0x7fdbe006cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbde59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbe00b65d0 0x7fdbe006cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52524/0 (socket says 172.21.15.45:52524) 2023-08-27T11:12:44.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbde59c700 1 -- 172.21.15.45:0/3565931896 learned_addr learned my addr 172.21.15.45:0/3565931896 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:44.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbddd9b700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe00cb5c0 0x7fdbe00c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.912+0000 7fdbded9d700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cfe10 0x7fdbe00b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbde59c700 1 -- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cfe10 msgr2=0x7fdbe00b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbde59c700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cfe10 0x7fdbe00b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbde59c700 1 -- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe00cb5c0 msgr2=0x7fdbe00c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:44.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbde59c700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe00cb5c0 0x7fdbe00c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:44.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbde59c700 1 -- 172.21.15.45:0/3565931896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd4009550 con 0x7fdbe00b65d0 2023-08-27T11:12:44.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbde59c700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbe00b65d0 0x7fdbe006cef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400ebe0 tx=0x7fdbd400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:44.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbceffd700 1 -- 172.21.15.45:0/3565931896 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400c850 con 0x7fdbe00b65d0 2023-08-27T11:12:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbceffd700 1 -- 172.21.15.45:0/3565931896 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd400c9b0 con 0x7fdbe00b65d0 2023-08-27T11:12:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd00094a0 con 0x7fdbe00b65d0 2023-08-27T11:12:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbceffd700 1 -- 172.21.15.45:0/3565931896 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400cd40 con 0x7fdbe00b65d0 2023-08-27T11:12:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.913+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe00c8990 con 0x7fdbe00b65d0 2023-08-27T11:12:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.914+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe00c95a0 con 0x7fdbe00b65d0 2023-08-27T11:12:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.915+0000 7fdbceffd700 1 -- 172.21.15.45:0/3565931896 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdbd4013470 con 0x7fdbe00b65d0 2023-08-27T11:12:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.915+0000 7fdbceffd700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbc006a0e0 0x7fdbc006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.915+0000 7fdbceffd700 1 -- 172.21.15.45:0/3565931896 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdbd4088910 con 0x7fdbe00b65d0 2023-08-27T11:12:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.915+0000 7fdbddd9b700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbc006a0e0 0x7fdbc006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.916+0000 7fdbddd9b700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbc006a0e0 0x7fdbc006c5a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0000c00 tx=0x7fdbd0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:44.919+0000 7fdbceffd700 1 -- 172.21.15.45:0/3565931896 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdbd4053da0 con 0x7fdbe00b65d0 2023-08-27T11:12:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.124+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdbe015f440 con 0x7fdbe00b65d0 2023-08-27T11:12:45.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.125+0000 7fdbceffd700 1 -- 172.21.15.45:0/3565931896 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdbe015f440 con 0x7fdbe00b65d0 2023-08-27T11:12:45.127 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:12:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbc006a0e0 msgr2=0x7fdbc006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbc006a0e0 0x7fdbc006c5a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0000c00 tx=0x7fdbd0008030 comp rx=0 tx=0).stop 2023-08-27T11:12:45.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbe00b65d0 msgr2=0x7fdbe006cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbe00b65d0 0x7fdbe006cef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400ebe0 tx=0x7fdbd400eef0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 shutdown_connections 2023-08-27T11:12:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbe00b65d0 0x7fdbe006cef0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbc006a0e0 0x7fdbc006c5a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdbe00cfe10 0x7fdbe00b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 --2- 172.21.15.45:0/3565931896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe00cb5c0 0x7fdbe00c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 >> 172.21.15.45:0/3565931896 conn(0x7fdbe006f960 msgr2=0x7fdbe0154340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 shutdown_connections 2023-08-27T11:12:45.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.127+0000 7fdbe49dc700 1 -- 172.21.15.45:0/3565931896 wait complete. 2023-08-27T11:12:45.141 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:12:45.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.267+0000 7f5885676700 1 Processor -- start 2023-08-27T11:12:45.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.267+0000 7f5885676700 1 -- start start 2023-08-27T11:12:45.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.268+0000 7f5885676700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588015e570 0x7f58800d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.268+0000 7f5885676700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58800cd220 con 0x7f58800cce40 2023-08-27T11:12:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.268+0000 7f5885676700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58800d0ad0 con 0x7f588006c8d0 2023-08-27T11:12:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.268+0000 7f5885676700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5880161d90 con 0x7f588015e570 2023-08-27T11:12:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.268+0000 7f587effd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f58800cce40 0x7f58800cea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39308/0 (socket says 172.21.15.45:39308) 2023-08-27T11:12:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.268+0000 7f587effd700 1 -- 172.21.15.45:0/2277245440 learned_addr learned my addr 172.21.15.45:0/2277245440 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.268+0000 7f587e7fc700 1 --2- 172.21.15.45:0/2277245440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588015e570 0x7f58800d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.268+0000 7f587dffb700 1 -- 172.21.15.45:0/2277245440 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192467792 0 0) 0x7f58800cd220 con 0x7f58800cce40 2023-08-27T11:12:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.268+0000 7f587dffb700 1 -- 172.21.15.45:0/2277245440 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58680035b0 con 0x7f58800cce40 2023-08-27T11:12:45.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.269+0000 7f587dffb700 1 -- 172.21.15.45:0/2277245440 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762680267 0 0) 0x7f58800d0ad0 con 0x7f588006c8d0 2023-08-27T11:12:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.269+0000 7f587e7fc700 1 -- 172.21.15.45:0/2277245440 >> v1:172.21.15.135:6789/0 conn(0x7f588006c8d0 legacy=0x7f588015caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.269+0000 7f587e7fc700 1 -- 172.21.15.45:0/2277245440 >> v1:172.21.15.45:6789/0 conn(0x7f58800cce40 legacy=0x7f58800cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.269+0000 7f587e7fc700 1 -- 172.21.15.45:0/2277245440 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588015da90 con 0x7f588015e570 2023-08-27T11:12:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.269+0000 7f587e7fc700 1 --2- 172.21.15.45:0/2277245440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588015e570 0x7f58800d04d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5870009870 tx=0x7f5870009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c25350fad1500b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.269+0000 7f587dffb700 1 -- 172.21.15.45:0/2277245440 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874001670 con 0x7f588015e570 2023-08-27T11:12:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.269+0000 7f587dffb700 1 -- 172.21.15.45:0/2277245440 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5870004030 con 0x7f588015e570 2023-08-27T11:12:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.269+0000 7f587dffb700 1 -- 172.21.15.45:0/2277245440 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58700043c0 con 0x7f588015e570 2023-08-27T11:12:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.270+0000 7f5885676700 1 -- 172.21.15.45:0/2277245440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588015e570 msgr2=0x7f58800d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.270+0000 7f5885676700 1 --2- 172.21.15.45:0/2277245440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588015e570 0x7f58800d04d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5870009870 tx=0x7f5870009b80 comp rx=0 tx=0).stop 2023-08-27T11:12:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.270+0000 7f5885676700 1 -- 172.21.15.45:0/2277245440 shutdown_connections 2023-08-27T11:12:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.270+0000 7f5885676700 1 --2- 172.21.15.45:0/2277245440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588015e570 0x7f58800d04d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.270+0000 7f5885676700 1 -- 172.21.15.45:0/2277245440 >> 172.21.15.45:0/2277245440 conn(0x7f58800cd900 msgr2=0x7f58800cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.270+0000 7f5885676700 1 -- 172.21.15.45:0/2277245440 shutdown_connections 2023-08-27T11:12:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.270+0000 7f5885676700 1 -- 172.21.15.45:0/2277245440 wait complete. 2023-08-27T11:12:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.271+0000 7f5885676700 1 Processor -- start 2023-08-27T11:12:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.271+0000 7f5885676700 1 -- start start 2023-08-27T11:12:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f5885676700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588006c8d0 0x7f58800aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f5885676700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58800cce40 0x7f58801f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f5885676700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f588015e570 0x7f58801f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f5885676700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800b5360 con 0x7f58800cce40 2023-08-27T11:12:45.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f5885676700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58801f3720 con 0x7f588015e570 2023-08-27T11:12:45.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f5885676700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58800b49c0 con 0x7f588006c8d0 2023-08-27T11:12:45.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58800cce40 0x7f58801f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58800cce40 0x7f58801f27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52540/0 (socket says 172.21.15.45:52540) 2023-08-27T11:12:45.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587e7fc700 1 -- 172.21.15.45:0/3929671095 learned_addr learned my addr 172.21.15.45:0/3929671095 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:45.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587f7fe700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f588015e570 0x7f58801f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587effd700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588006c8d0 0x7f58800aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587e7fc700 1 -- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588006c8d0 msgr2=0x7f58800aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587e7fc700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588006c8d0 0x7f58800aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587e7fc700 1 -- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f588015e570 msgr2=0x7f58801f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587e7fc700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f588015e570 0x7f58801f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587e7fc700 1 -- 172.21.15.45:0/3929671095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700094a0 con 0x7f58800cce40 2023-08-27T11:12:45.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587f7fe700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f588015e570 0x7f58801f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:45.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587effd700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588006c8d0 0x7f58800aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:45.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f587e7fc700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58800cce40 0x7f58801f27a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f58700097c0 tx=0x7f5870004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:45.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.272+0000 7f58677fe700 1 -- 172.21.15.45:0/3929671095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870010590 con 0x7f58800cce40 2023-08-27T11:12:45.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.273+0000 7f58677fe700 1 -- 172.21.15.45:0/3929671095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700106f0 con 0x7f58800cce40 2023-08-27T11:12:45.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.273+0000 7f58677fe700 1 -- 172.21.15.45:0/3929671095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870010aa0 con 0x7f58800cce40 2023-08-27T11:12:45.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.273+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58800b4ca0 con 0x7f58800cce40 2023-08-27T11:12:45.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.273+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58800af3b0 con 0x7f58800cce40 2023-08-27T11:12:45.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.273+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58800b54a0 con 0x7f58800cce40 2023-08-27T11:12:45.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.274+0000 7f58677fe700 1 -- 172.21.15.45:0/3929671095 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5870010c00 con 0x7f58800cce40 2023-08-27T11:12:45.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.274+0000 7f58677fe700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f586c06a0e0 0x7f586c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.274+0000 7f58677fe700 1 -- 172.21.15.45:0/3929671095 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5870089db0 con 0x7f58800cce40 2023-08-27T11:12:45.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.275+0000 7f587effd700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f586c06a0e0 0x7f586c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.275+0000 7f587effd700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f586c06a0e0 0x7f586c06c5a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58740011e0 tx=0x7f5874009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:45.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.279+0000 7f58677fe700 1 -- 172.21.15.45:0/3929671095 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f58800b54a0 con 0x7f58800cce40 2023-08-27T11:12:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.480+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f58800b54a0 con 0x7f58800cce40 2023-08-27T11:12:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.480+0000 7f58677fe700 1 -- 172.21.15.45:0/3929671095 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f58800b54a0 con 0x7f58800cce40 2023-08-27T11:12:45.483 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.482+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f586c06a0e0 msgr2=0x7f586c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.482+0000 7f5885676700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f586c06a0e0 0x7f586c06c5a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58740011e0 tx=0x7f5874009040 comp rx=0 tx=0).stop 2023-08-27T11:12:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.482+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58800cce40 msgr2=0x7f58801f27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.482+0000 7f5885676700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58800cce40 0x7f58801f27a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f58700097c0 tx=0x7f5870004750 comp rx=0 tx=0).stop 2023-08-27T11:12:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.483+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 shutdown_connections 2023-08-27T11:12:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.483+0000 7f5885676700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f586c06a0e0 0x7f586c06c5a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.483+0000 7f5885676700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f588015e570 0x7f58801f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.483+0000 7f5885676700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f588006c8d0 0x7f58800aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.483+0000 7f5885676700 1 --2- 172.21.15.45:0/3929671095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58800cce40 0x7f58801f27a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.483+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 >> 172.21.15.45:0/3929671095 conn(0x7f58800cd900 msgr2=0x7f58800d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.483+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 shutdown_connections 2023-08-27T11:12:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.483+0000 7f5885676700 1 -- 172.21.15.45:0/3929671095 wait complete. 2023-08-27T11:12:45.497 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:12:45.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.619+0000 7fa3cacd8700 1 Processor -- start 2023-08-27T11:12:45.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.620+0000 7fa3cacd8700 1 -- start start 2023-08-27T11:12:45.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.620+0000 7fa3cacd8700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa3c415a240 0x7fa3c40cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.620+0000 7fa3cacd8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3c40cbe70 0x7fa3c40ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.620+0000 7fa3cacd8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c406d690 con 0x7fa3c40ce9d0 2023-08-27T11:12:45.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.620+0000 7fa3cacd8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c40cede0 con 0x7fa3c40cbe70 2023-08-27T11:12:45.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.620+0000 7fa3cacd8700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c40cc440 con 0x7fa3c415a240 2023-08-27T11:12:45.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.620+0000 7fa3c3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3c40cbe70 0x7fa3c40ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.620+0000 7fa3c9275700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa3c40ce9d0 0x7fa3c4070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39326/0 (socket says 172.21.15.45:39326) 2023-08-27T11:12:45.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.620+0000 7fa3c9275700 1 -- 172.21.15.45:0/477122227 learned_addr learned my addr 172.21.15.45:0/477122227 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:45.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c8a74700 1 --2- 172.21.15.45:0/477122227 >> v2:172.21.15.135:3300/0 conn(0x7fa3c415a240 0x7fa3c40cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c3fff700 1 -- 172.21.15.45:0/477122227 >> v2:172.21.15.135:3300/0 conn(0x7fa3c415a240 msgr2=0x7fa3c40cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:45.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c3fff700 1 --2- 172.21.15.45:0/477122227 >> v2:172.21.15.135:3300/0 conn(0x7fa3c415a240 0x7fa3c40cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c3fff700 1 -- 172.21.15.45:0/477122227 >> v1:172.21.15.45:6789/0 conn(0x7fa3c40ce9d0 legacy=0x7fa3c4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c3fff700 1 -- 172.21.15.45:0/477122227 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c406d040 con 0x7fa3c40cbe70 2023-08-27T11:12:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c37fe700 1 -- 172.21.15.45:0/477122227 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898333068 0 0) 0x7fa3c406d690 con 0x7fa3c40ce9d0 2023-08-27T11:12:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c3fff700 1 --2- 172.21.15.45:0/477122227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 0x7fa3c40ce490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40b2950 tx=0x7fa3b0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa82a0fe2261b76 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c37fe700 1 -- 172.21.15.45:0/477122227 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0011830 con 0x7fa3c40cbe70 2023-08-27T11:12:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c37fe700 1 -- 172.21.15.45:0/477122227 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b0011990 con 0x7fa3c40cbe70 2023-08-27T11:12:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.621+0000 7fa3c37fe700 1 -- 172.21.15.45:0/477122227 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0011d20 con 0x7fa3c40cbe70 2023-08-27T11:12:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.622+0000 7fa3cacd8700 1 -- 172.21.15.45:0/477122227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 msgr2=0x7fa3c40ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.622+0000 7fa3cacd8700 1 --2- 172.21.15.45:0/477122227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 0x7fa3c40ce490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40b2950 tx=0x7fa3b0009b20 comp rx=0 tx=0).stop 2023-08-27T11:12:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.622+0000 7fa3cacd8700 1 -- 172.21.15.45:0/477122227 shutdown_connections 2023-08-27T11:12:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.622+0000 7fa3cacd8700 1 --2- 172.21.15.45:0/477122227 >> v2:172.21.15.135:3300/0 conn(0x7fa3c415a240 0x7fa3c40cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.622+0000 7fa3cacd8700 1 --2- 172.21.15.45:0/477122227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 0x7fa3c40ce490 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.622+0000 7fa3cacd8700 1 -- 172.21.15.45:0/477122227 >> 172.21.15.45:0/477122227 conn(0x7fa3c40cfe50 msgr2=0x7fa3c40d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:45.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.622+0000 7fa3cacd8700 1 -- 172.21.15.45:0/477122227 shutdown_connections 2023-08-27T11:12:45.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.622+0000 7fa3cacd8700 1 -- 172.21.15.45:0/477122227 wait complete. 2023-08-27T11:12:45.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.623+0000 7fa3cacd8700 1 Processor -- start 2023-08-27T11:12:45.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.623+0000 7fa3cacd8700 1 -- start start 2023-08-27T11:12:45.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3cacd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 0x7fa3c415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3cacd8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c40ce9d0 0x7fa3c40c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3cacd8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415a240 0x7fa3c414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3cacd8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40c5290 con 0x7fa3c40cbe70 2023-08-27T11:12:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3cacd8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c415b990 con 0x7fa3c415a240 2023-08-27T11:12:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3cacd8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40c5550 con 0x7fa3c40ce9d0 2023-08-27T11:12:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c8a74700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 0x7fa3c415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c8a74700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 0x7fa3c415b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52552/0 (socket says 172.21.15.45:52552) 2023-08-27T11:12:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c8a74700 1 -- 172.21.15.45:0/3178860511 learned_addr learned my addr 172.21.15.45:0/3178860511 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c9275700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415a240 0x7fa3c414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c3fff700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c40ce9d0 0x7fa3c40c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c8a74700 1 -- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c40ce9d0 msgr2=0x7fa3c40c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c8a74700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c40ce9d0 0x7fa3c40c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c8a74700 1 -- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415a240 msgr2=0x7fa3c414f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c8a74700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415a240 0x7fa3c414f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c8a74700 1 -- 172.21.15.45:0/3178860511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b00094a0 con 0x7fa3c40cbe70 2023-08-27T11:12:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.624+0000 7fa3c8a74700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 0x7fa3c415b2d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800e870 tx=0x7fa3b800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.625+0000 7fa3c17fa700 1 -- 172.21.15.45:0/3178860511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8009590 con 0x7fa3c40cbe70 2023-08-27T11:12:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.625+0000 7fa3c17fa700 1 -- 172.21.15.45:0/3178860511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b8013680 con 0x7fa3c40cbe70 2023-08-27T11:12:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.625+0000 7fa3c17fa700 1 -- 172.21.15.45:0/3178860511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80139f0 con 0x7fa3c40cbe70 2023-08-27T11:12:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.625+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c40c5830 con 0x7fa3c40cbe70 2023-08-27T11:12:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.625+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c414f660 con 0x7fa3c40cbe70 2023-08-27T11:12:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.626+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c40c5d50 con 0x7fa3c40cbe70 2023-08-27T11:12:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.626+0000 7fa3c17fa700 1 -- 172.21.15.45:0/3178860511 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa3b8013b50 con 0x7fa3c40cbe70 2023-08-27T11:12:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.627+0000 7fa3c17fa700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa3a806a0e0 0x7fa3a806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.627+0000 7fa3c17fa700 1 -- 172.21.15.45:0/3178860511 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa3b80886a0 con 0x7fa3c40cbe70 2023-08-27T11:12:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.627+0000 7fa3c3fff700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa3a806a0e0 0x7fa3a806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.627+0000 7fa3c3fff700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa3a806a0e0 0x7fa3a806c5a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00090b0 tx=0x7fa3b0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:45.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.631+0000 7fa3c17fa700 1 -- 172.21.15.45:0/3178860511 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa3c40c5d50 con 0x7fa3c40cbe70 2023-08-27T11:12:45.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.876+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3c40c5d50 con 0x7fa3c40cbe70 2023-08-27T11:12:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.877+0000 7fa3c17fa700 1 -- 172.21.15.45:0/3178860511 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fa3c40c5d50 con 0x7fa3c40cbe70 2023-08-27T11:12:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:45.879 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:33.983169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:12:45.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa3a806a0e0 msgr2=0x7fa3a806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa3a806a0e0 0x7fa3a806c5a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00090b0 tx=0x7fa3b0008030 comp rx=0 tx=0).stop 2023-08-27T11:12:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 msgr2=0x7fa3c415b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 0x7fa3c415b2d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800e870 tx=0x7fa3b800eb80 comp rx=0 tx=0).stop 2023-08-27T11:12:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 shutdown_connections 2023-08-27T11:12:45.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c40cbe70 0x7fa3c415b2d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa3a806a0e0 0x7fa3a806c5a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3c415a240 0x7fa3c414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 --2- 172.21.15.45:0/3178860511 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa3c40ce9d0 0x7fa3c40c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 >> 172.21.15.45:0/3178860511 conn(0x7fa3c40cfe50 msgr2=0x7fa3c40af880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 shutdown_connections 2023-08-27T11:12:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:45.879+0000 7fa3cacd8700 1 -- 172.21.15.45:0/3178860511 wait complete. 2023-08-27T11:12:45.884 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-08-27T11:12:45.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:46.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.017+0000 7f0551f81700 1 Processor -- start 2023-08-27T11:12:46.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.017+0000 7f0551f81700 1 -- start start 2023-08-27T11:12:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f0551f81700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f054c15b070 0x7f054c0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f0551f81700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c0cdf60 0x7f054c06e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f0551f81700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f054c06cec0 0x7f054c06e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f0551f81700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c15d8e0 con 0x7f054c15b070 2023-08-27T11:12:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f0551f81700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c15dad0 con 0x7f054c06cec0 2023-08-27T11:12:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f0551f81700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c15ddb0 con 0x7f054c0cdf60 2023-08-27T11:12:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f054b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f054c15b070 0x7f054c0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f054b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f054c15b070 0x7f054c0cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52560/0 (socket says 172.21.15.45:52560) 2023-08-27T11:12:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f054b7fe700 1 -- 172.21.15.45:0/1884120953 learned_addr learned my addr 172.21.15.45:0/1884120953 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f0542ffd700 1 --2- 172.21.15.45:0/1884120953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c0cdf60 0x7f054c06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.018+0000 7f054bfff700 1 --2- 172.21.15.45:0/1884120953 >> v2:172.21.15.135:3300/0 conn(0x7f054c06cec0 0x7f054c06e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f054b7fe700 1 -- 172.21.15.45:0/1884120953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c0cdf60 msgr2=0x7f054c06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f054b7fe700 1 --2- 172.21.15.45:0/1884120953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c0cdf60 0x7f054c06e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f054b7fe700 1 -- 172.21.15.45:0/1884120953 >> v2:172.21.15.135:3300/0 conn(0x7f054c06cec0 msgr2=0x7f054c06e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:46.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f054b7fe700 1 --2- 172.21.15.45:0/1884120953 >> v2:172.21.15.135:3300/0 conn(0x7f054c06cec0 0x7f054c06e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f054b7fe700 1 -- 172.21.15.45:0/1884120953 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c15e090 con 0x7f054c15b070 2023-08-27T11:12:46.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f0542ffd700 1 --2- 172.21.15.45:0/1884120953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c0cdf60 0x7f054c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:46.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f054b7fe700 1 --2- 172.21.15.45:0/1884120953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 0x7f054c0cda20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f054c072300 tx=0x7f053c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40f3a88575e9170 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:46.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f054affd700 1 -- 172.21.15.45:0/1884120953 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00b870 con 0x7f054c15b070 2023-08-27T11:12:46.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f054affd700 1 -- 172.21.15.45:0/1884120953 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c00b9d0 con 0x7f054c15b070 2023-08-27T11:12:46.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f054affd700 1 -- 172.21.15.45:0/1884120953 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00bd60 con 0x7f054c15b070 2023-08-27T11:12:46.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f0551f81700 1 -- 172.21.15.45:0/1884120953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 msgr2=0x7f054c0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f0551f81700 1 --2- 172.21.15.45:0/1884120953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 0x7f054c0cda20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f054c072300 tx=0x7f053c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f0551f81700 1 -- 172.21.15.45:0/1884120953 shutdown_connections 2023-08-27T11:12:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f0551f81700 1 --2- 172.21.15.45:0/1884120953 >> v2:172.21.15.135:3300/0 conn(0x7f054c06cec0 0x7f054c06e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f0551f81700 1 --2- 172.21.15.45:0/1884120953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 0x7f054c0cda20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f0551f81700 1 --2- 172.21.15.45:0/1884120953 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c0cdf60 0x7f054c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.019+0000 7f0551f81700 1 -- 172.21.15.45:0/1884120953 >> 172.21.15.45:0/1884120953 conn(0x7f054c0cfe50 msgr2=0x7f054c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.020+0000 7f0551f81700 1 -- 172.21.15.45:0/1884120953 shutdown_connections 2023-08-27T11:12:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.020+0000 7f0551f81700 1 -- 172.21.15.45:0/1884120953 wait complete. 2023-08-27T11:12:46.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.021+0000 7f0551f81700 1 Processor -- start 2023-08-27T11:12:46.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.021+0000 7f0551f81700 1 -- start start 2023-08-27T11:12:46.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.022+0000 7f0551f81700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c06cec0 0x7f054c160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.022+0000 7f0551f81700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f054c0cdf60 0x7f054c14f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.022+0000 7f0551f81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 0x7f054c14f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.022+0000 7f0551f81700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1607d0 con 0x7f054c15b070 2023-08-27T11:12:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.022+0000 7f0551f81700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0c4cd0 con 0x7f054c0cdf60 2023-08-27T11:12:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.022+0000 7f0551f81700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0c40c0 con 0x7f054c06cec0 2023-08-27T11:12:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.022+0000 7f054bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 0x7f054c14f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.022+0000 7f054bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 0x7f054c14f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52570/0 (socket says 172.21.15.45:52570) 2023-08-27T11:12:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.022+0000 7f054bfff700 1 -- 172.21.15.45:0/3080177699 learned_addr learned my addr 172.21.15.45:0/3080177699 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f0542ffd700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f054c0cdf60 0x7f054c14f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f054b7fe700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c06cec0 0x7f054c160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f054bfff700 1 -- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c06cec0 msgr2=0x7f054c160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f054bfff700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c06cec0 0x7f054c160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f054bfff700 1 -- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f054c0cdf60 msgr2=0x7f054c14f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f054bfff700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f054c0cdf60 0x7f054c14f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f054bfff700 1 -- 172.21.15.45:0/3080177699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c009550 con 0x7f054c15b070 2023-08-27T11:12:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f054bfff700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 0x7f054c14f860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0544007e60 tx=0x7f054400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:46.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f0548ff9700 1 -- 172.21.15.45:0/3080177699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054400ede0 con 0x7f054c15b070 2023-08-27T11:12:46.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f0548ff9700 1 -- 172.21.15.45:0/3080177699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544014420 con 0x7f054c15b070 2023-08-27T11:12:46.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c0c43a0 con 0x7f054c15b070 2023-08-27T11:12:46.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f0548ff9700 1 -- 172.21.15.45:0/3080177699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440147d0 con 0x7f054c15b070 2023-08-27T11:12:46.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.023+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c0c4960 con 0x7f054c15b070 2023-08-27T11:12:46.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.024+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c072810 con 0x7f054c15b070 2023-08-27T11:12:46.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.025+0000 7f0548ff9700 1 -- 172.21.15.45:0/3080177699 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0544014930 con 0x7f054c15b070 2023-08-27T11:12:46.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.025+0000 7f0548ff9700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052c06a0e0 0x7f052c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.025+0000 7f0548ff9700 1 -- 172.21.15.45:0/3080177699 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0544088fe0 con 0x7f054c15b070 2023-08-27T11:12:46.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.025+0000 7f054b7fe700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052c06a0e0 0x7f052c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.026+0000 7f054b7fe700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052c06a0e0 0x7f052c06c5a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f053c014040 tx=0x7f053c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.029+0000 7f0548ff9700 1 -- 172.21.15.45:0/3080177699 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f05440543f0 con 0x7f054c15b070 2023-08-27T11:12:46.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.232+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f054c0734a0 con 0x7f054c15b070 2023-08-27T11:12:46.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.234+0000 7f0548ff9700 1 -- 172.21.15.45:0/3080177699 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+20132 (secure 0 0 0) 0x7f0544057a10 con 0x7f054c15b070 2023-08-27T11:12:46.236 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:46.236 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:13.360479+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052c06a0e0 msgr2=0x7f052c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052c06a0e0 0x7f052c06c5a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f053c014040 tx=0x7f053c008030 comp rx=0 tx=0).stop 2023-08-27T11:12:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 msgr2=0x7f054c14f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 0x7f054c14f860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0544007e60 tx=0x7f054400e3c0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 shutdown_connections 2023-08-27T11:12:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054c15b070 0x7f054c14f860 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052c06a0e0 0x7f052c06c5a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f054c06cec0 0x7f054c160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 --2- 172.21.15.45:0/3080177699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f054c0cdf60 0x7f054c14f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 >> 172.21.15.45:0/3080177699 conn(0x7f054c0cfe50 msgr2=0x7f054c06e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 shutdown_connections 2023-08-27T11:12:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.236+0000 7f0551f81700 1 -- 172.21.15.45:0/3080177699 wait complete. 2023-08-27T11:12:46.250 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:12:46.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.373+0000 7fae103eb700 1 Processor -- start 2023-08-27T11:12:46.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.373+0000 7fae103eb700 1 -- start start 2023-08-27T11:12:46.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.374+0000 7fae103eb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0815b070 0x7fae080cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.374+0000 7fae103eb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fae080cdf60 0x7fae0806e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.374+0000 7fae103eb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0815dad0 con 0x7fae0806cec0 2023-08-27T11:12:46.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.374+0000 7fae103eb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0815d8e0 con 0x7fae080cdf60 2023-08-27T11:12:46.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.374+0000 7fae103eb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0815ddb0 con 0x7fae0815b070 2023-08-27T11:12:46.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.374+0000 7fae0d986700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fae080cdf60 0x7fae0806e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.374+0000 7fae0e988700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fae0806cec0 0x7fae080ae8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39354/0 (socket says 172.21.15.45:39354) 2023-08-27T11:12:46.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.374+0000 7fae0e988700 1 -- 172.21.15.45:0/695558889 learned_addr learned my addr 172.21.15.45:0/695558889 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:46.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.374+0000 7fae0e187700 1 --2- 172.21.15.45:0/695558889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0815b070 0x7fae080cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.375+0000 7fae0d986700 1 -- 172.21.15.45:0/695558889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0815b070 msgr2=0x7fae080cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.375+0000 7fae0d986700 1 --2- 172.21.15.45:0/695558889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0815b070 0x7fae080cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.375+0000 7fae0d986700 1 -- 172.21.15.45:0/695558889 >> v1:172.21.15.45:6789/0 conn(0x7fae0806cec0 legacy=0x7fae080ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.375+0000 7fae0d986700 1 -- 172.21.15.45:0/695558889 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0815e090 con 0x7fae080cdf60 2023-08-27T11:12:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.375+0000 7fae0d185700 1 -- 172.21.15.45:0/695558889 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074886190 0 0) 0x7fae0815dad0 con 0x7fae0806cec0 2023-08-27T11:12:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.375+0000 7fae0d986700 1 --2- 172.21.15.45:0/695558889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 0x7fae0806e270 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae04009840 tx=0x7fae04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd88351a3c8f5ce3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.375+0000 7fae0d185700 1 -- 172.21.15.45:0/695558889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04011830 con 0x7fae080cdf60 2023-08-27T11:12:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.375+0000 7fae0d185700 1 -- 172.21.15.45:0/695558889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae04011990 con 0x7fae080cdf60 2023-08-27T11:12:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.375+0000 7fae0d185700 1 -- 172.21.15.45:0/695558889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04011d20 con 0x7fae080cdf60 2023-08-27T11:12:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.376+0000 7fae103eb700 1 -- 172.21.15.45:0/695558889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 msgr2=0x7fae0806e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.376+0000 7fae103eb700 1 --2- 172.21.15.45:0/695558889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 0x7fae0806e270 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae04009840 tx=0x7fae04009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.376+0000 7fae103eb700 1 -- 172.21.15.45:0/695558889 shutdown_connections 2023-08-27T11:12:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.376+0000 7fae103eb700 1 --2- 172.21.15.45:0/695558889 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0815b070 0x7fae080cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.376+0000 7fae103eb700 1 --2- 172.21.15.45:0/695558889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 0x7fae0806e270 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.376+0000 7fae103eb700 1 -- 172.21.15.45:0/695558889 >> 172.21.15.45:0/695558889 conn(0x7fae080cfe50 msgr2=0x7fae080d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.376+0000 7fae103eb700 1 -- 172.21.15.45:0/695558889 shutdown_connections 2023-08-27T11:12:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.376+0000 7fae103eb700 1 -- 172.21.15.45:0/695558889 wait complete. 2023-08-27T11:12:46.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.377+0000 7fae103eb700 1 Processor -- start 2023-08-27T11:12:46.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.377+0000 7fae103eb700 1 -- start start 2023-08-27T11:12:46.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae103eb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0806cec0 0x7fae08154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae103eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 0x7fae08155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae103eb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae0815b070 0x7fae080c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae103eb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08154ab0 con 0x7fae080cdf60 2023-08-27T11:12:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae103eb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08153ea0 con 0x7fae0815b070 2023-08-27T11:12:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae103eb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae08154180 con 0x7fae0806cec0 2023-08-27T11:12:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0d986700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 0x7fae08155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0d986700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 0x7fae08155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52582/0 (socket says 172.21.15.45:52582) 2023-08-27T11:12:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0d986700 1 -- 172.21.15.45:0/2446471576 learned_addr learned my addr 172.21.15.45:0/2446471576 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0e187700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0806cec0 0x7fae08154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0e988700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae0815b070 0x7fae080c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0d986700 1 -- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0806cec0 msgr2=0x7fae08154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0d986700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0806cec0 0x7fae08154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0d986700 1 -- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae0815b070 msgr2=0x7fae080c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0d986700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae0815b070 0x7fae080c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0d986700 1 -- 172.21.15.45:0/2446471576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae040094a0 con 0x7fae080cdf60 2023-08-27T11:12:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0e187700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0806cec0 0x7fae08154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0e988700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae0815b070 0x7fae080c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae0d986700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 0x7fae08155480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae04009fd0 tx=0x7fae04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fadfeffd700 1 -- 172.21.15.45:0/2446471576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040038a0 con 0x7fae080cdf60 2023-08-27T11:12:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fadfeffd700 1 -- 172.21.15.45:0/2446471576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0401f070 con 0x7fae080cdf60 2023-08-27T11:12:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fadfeffd700 1 -- 172.21.15.45:0/2446471576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04003c40 con 0x7fae080cdf60 2023-08-27T11:12:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae08154460 con 0x7fae080cdf60 2023-08-27T11:12:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.378+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae080c6180 con 0x7fae080cdf60 2023-08-27T11:12:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.379+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae08154c60 con 0x7fae080cdf60 2023-08-27T11:12:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.380+0000 7fadfeffd700 1 -- 172.21.15.45:0/2446471576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fae04026480 con 0x7fae080cdf60 2023-08-27T11:12:46.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.380+0000 7fadfeffd700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fadec06a110 0x7fadec06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:46.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.380+0000 7fadfeffd700 1 -- 172.21.15.45:0/2446471576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fae04021070 con 0x7fae080cdf60 2023-08-27T11:12:46.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.381+0000 7fae0e187700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fadec06a110 0x7fadec06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:46.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.381+0000 7fae0e187700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fadec06a110 0x7fadec06c5d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009ac0 tx=0x7fadf8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:46.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.385+0000 7fadfeffd700 1 -- 172.21.15.45:0/2446471576 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fae08154c60 con 0x7fae080cdf60 2023-08-27T11:12:46.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.627+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae080cf8d0 con 0x7fae080cdf60 2023-08-27T11:12:46.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.628+0000 7fadfeffd700 1 -- 172.21.15.45:0/2446471576 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fae0405c320 con 0x7fae080cdf60 2023-08-27T11:12:46.630 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:46.630 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:33.983169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:12:46.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.629+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fadec06a110 msgr2=0x7fadec06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.629+0000 7fae103eb700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fadec06a110 0x7fadec06c5d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009ac0 tx=0x7fadf8008040 comp rx=0 tx=0).stop 2023-08-27T11:12:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.629+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 msgr2=0x7fae08155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.629+0000 7fae103eb700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 0x7fae08155480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae04009fd0 tx=0x7fae04004750 comp rx=0 tx=0).stop 2023-08-27T11:12:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.630+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 shutdown_connections 2023-08-27T11:12:46.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.630+0000 7fae103eb700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fadec06a110 0x7fadec06c5d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.630+0000 7fae103eb700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae0815b070 0x7fae080c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.630+0000 7fae103eb700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae0806cec0 0x7fae08154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.630+0000 7fae103eb700 1 --2- 172.21.15.45:0/2446471576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae080cdf60 0x7fae08155480 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:46.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.630+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 >> 172.21.15.45:0/2446471576 conn(0x7fae080cfe50 msgr2=0x7fae080d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:46.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.630+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 shutdown_connections 2023-08-27T11:12:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:46.630+0000 7fae103eb700 1 -- 172.21.15.45:0/2446471576 wait complete. 2023-08-27T11:12:46.634 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-08-27T11:12:47.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:12:08.309320+0000', 'modified': '2023-08-27T11:12:33.983169+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6836/739639682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 739639682}, {'type': 'v1', 'addr': '172.21.15.135:6836', 'nonce': 739639682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:12:47.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:12:47.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-08-27T11:12:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.178+0000 7f16c82d3700 1 Processor -- start 2023-08-27T11:12:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.178+0000 7f16c82d3700 1 -- start start 2023-08-27T11:12:47.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.179+0000 7f16c82d3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f16c00aed60 0x7f16c0161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.179+0000 7f16c82d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c00af420 0x7f16c006ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.179+0000 7f16c82d3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c0071de0 con 0x7f16c015d3f0 2023-08-27T11:12:47.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.179+0000 7f16c82d3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c00c6ca0 con 0x7f16c00aed60 2023-08-27T11:12:47.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.179+0000 7f16c82d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c015d7d0 con 0x7f16c00af420 2023-08-27T11:12:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.179+0000 7f16c606f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f16c015d3f0 0x7f16c0160e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39364/0 (socket says 172.21.15.45:39364) 2023-08-27T11:12:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.179+0000 7f16c606f700 1 -- 172.21.15.45:0/479991801 learned_addr learned my addr 172.21.15.45:0/479991801 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c586e700 1 --2- 172.21.15.45:0/479991801 >> v2:172.21.15.135:3300/0 conn(0x7f16c00aed60 0x7f16c0161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c6870700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c00af420 0x7f16c006ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349930990 0 0) 0x7f16c0071de0 con 0x7f16c015d3f0 2023-08-27T11:12:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16ac0035b0 con 0x7f16c015d3f0 2023-08-27T11:12:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 752002085 0 0) 0x7f16ac0035b0 con 0x7f16c015d3f0 2023-08-27T11:12:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c00af420 msgr2=0x7f16c006ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c00af420 0x7f16c006ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 >> v2:172.21.15.135:3300/0 conn(0x7f16c00aed60 msgr2=0x7f16c0161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 --2- 172.21.15.45:0/479991801 >> v2:172.21.15.135:3300/0 conn(0x7f16c00aed60 0x7f16c0161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00c5310 con 0x7f16c015d3f0 2023-08-27T11:12:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f16bc001670 con 0x7f16c015d3f0 2023-08-27T11:12:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 >> v1:172.21.15.45:6789/0 conn(0x7f16c015d3f0 legacy=0x7f16c0160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c00af420 0x7f16ac003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac003fb0 0x7f16ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16ac004d90 0x7f16ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00c6ca0 con 0x7f16ac004d90 2023-08-27T11:12:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c015d7d0 con 0x7f16ac003fb0 2023-08-27T11:12:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac00ba90 con 0x7f16c00af420 2023-08-27T11:12:47.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c6870700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16ac004d90 0x7f16ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c606f700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c00af420 0x7f16ac003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.180+0000 7f16c586e700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac003fb0 0x7f16ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c6870700 1 -- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c00af420 msgr2=0x7f16ac003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c6870700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c00af420 0x7f16ac003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c6870700 1 -- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac003fb0 msgr2=0x7f16ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c6870700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac003fb0 0x7f16ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c6870700 1 -- 172.21.15.45:0/479991801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac0035b0 con 0x7f16ac004d90 2023-08-27T11:12:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c606f700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c00af420 0x7f16ac003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c586e700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac003fb0 0x7f16ac004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c6870700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16ac004d90 0x7f16ac005210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009fc0 tx=0x7f16ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6cc2e78b2d8ca5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800ec30 con 0x7f16ac004d90 2023-08-27T11:12:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800f040 con 0x7f16ac004d90 2023-08-27T11:12:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c506d700 1 -- 172.21.15.45:0/479991801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8014480 con 0x7f16ac004d90 2023-08-27T11:12:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c82d3700 1 -- 172.21.15.45:0/479991801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16ac004d90 msgr2=0x7f16ac005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c82d3700 1 --2- 172.21.15.45:0/479991801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16ac004d90 0x7f16ac005210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009fc0 tx=0x7f16ac005880 comp rx=0 tx=0).stop 2023-08-27T11:12:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.181+0000 7f16c6870700 1 -- 172.21.15.45:0/479991801 reap_dead start 2023-08-27T11:12:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.182+0000 7f16c82d3700 1 -- 172.21.15.45:0/479991801 shutdown_connections 2023-08-27T11:12:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.182+0000 7f16c82d3700 1 -- 172.21.15.45:0/479991801 >> 172.21.15.45:0/479991801 conn(0x7f16c00cfdd0 msgr2=0x7f16c00d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.182+0000 7f16c82d3700 1 -- 172.21.15.45:0/479991801 shutdown_connections 2023-08-27T11:12:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.182+0000 7f16c82d3700 1 -- 172.21.15.45:0/479991801 wait complete. 2023-08-27T11:12:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.183+0000 7f16c82d3700 1 Processor -- start 2023-08-27T11:12:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.183+0000 7f16c82d3700 1 -- start start 2023-08-27T11:12:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c82d3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac004d90 0x7f16c00b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c82d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c0154cf0 0x7f16c00b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c82d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00b4800 0x7f16c00b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c82d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00b51c0 con 0x7f16c00b4800 2023-08-27T11:12:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c82d3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00cdb10 con 0x7f16ac004d90 2023-08-27T11:12:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c82d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c01f31c0 con 0x7f16c0154cf0 2023-08-27T11:12:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c6870700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00b4800 0x7f16c00b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c6870700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00b4800 0x7f16c00b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52604/0 (socket says 172.21.15.45:52604) 2023-08-27T11:12:47.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c6870700 1 -- 172.21.15.45:0/2435076235 learned_addr learned my addr 172.21.15.45:0/2435076235 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:47.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c606f700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac004d90 0x7f16c00b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c586e700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c0154cf0 0x7f16c00b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c6870700 1 -- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c0154cf0 msgr2=0x7f16c00b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c6870700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c0154cf0 0x7f16c00b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c6870700 1 -- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac004d90 msgr2=0x7f16c00b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c6870700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac004d90 0x7f16c00b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c6870700 1 -- 172.21.15.45:0/2435076235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00b34c0 con 0x7f16c00b4800 2023-08-27T11:12:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c586e700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c0154cf0 0x7f16c00b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:47.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c6870700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00b4800 0x7f16c00b4c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009f90 tx=0x7f16b8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16b2ffd700 1 -- 172.21.15.45:0/2435076235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8007000 con 0x7f16c00b4800 2023-08-27T11:12:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16b2ffd700 1 -- 172.21.15.45:0/2435076235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8022070 con 0x7f16c00b4800 2023-08-27T11:12:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c82d3700 1 -- 172.21.15.45:0/2435076235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c00b37a0 con 0x7f16c00b4800 2023-08-27T11:12:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16b2ffd700 1 -- 172.21.15.45:0/2435076235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8029480 con 0x7f16c00b4800 2023-08-27T11:12:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.184+0000 7f16c82d3700 1 -- 172.21.15.45:0/2435076235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c00cd770 con 0x7f16c00b4800 2023-08-27T11:12:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.186+0000 7f16b2ffd700 1 -- 172.21.15.45:0/2435076235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f16b801e430 con 0x7f16c00b4800 2023-08-27T11:12:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.186+0000 7f16b2ffd700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f16a406a0e0 0x7f16a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.186+0000 7f16b2ffd700 1 -- 172.21.15.45:0/2435076235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f16b8024070 con 0x7f16c00b4800 2023-08-27T11:12:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.186+0000 7f16c606f700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f16a406a0e0 0x7f16a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.187+0000 7f16c606f700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f16a406a0e0 0x7f16a406c5a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0016d0 tx=0x7f16bc007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.188+0000 7f16d4495b80 1 Processor -- start 2023-08-27T11:12:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.188+0000 7f16d4495b80 1 -- start start 2023-08-27T11:12:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb02320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 0x56156cafeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16d4495b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156ca30730 con 0x56156cad53d0 2023-08-27T11:12:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16d4495b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156ca2e050 con 0x56156cb13960 2023-08-27T11:12:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16d4495b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x56156ca29940 con 0x56156cb01f40 2023-08-27T11:12:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c6870700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c6870700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52610/0 (socket says 172.21.15.45:52610) 2023-08-27T11:12:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c6870700 1 -- 172.21.15.45:0/3317926016 learned_addr learned my addr 172.21.15.45:0/3317926016 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c606f700 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb02320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c586e700 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 0x56156cafeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c6870700 1 -- 172.21.15.45:0/3317926016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 msgr2=0x56156cb02320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c6870700 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb02320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c6870700 1 -- 172.21.15.45:0/3317926016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 msgr2=0x56156cafeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c6870700 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 0x56156cafeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c6870700 1 -- 172.21.15.45:0/3317926016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56156c8a95e0 con 0x56156cad53d0 2023-08-27T11:12:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.189+0000 7f16c586e700 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 0x56156cafeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16c6870700 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caff0b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f16b800e940 tx=0x7f16b80589d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a145543b1b638a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16a3fff700 1 -- 172.21.15.45:0/3317926016 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8060c50 con 0x56156cad53d0 2023-08-27T11:12:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16a3fff700 1 -- 172.21.15.45:0/3317926016 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b80572e0 con 0x56156cad53d0 2023-08-27T11:12:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16a3fff700 1 -- 172.21.15.45:0/3317926016 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b80607c0 con 0x56156cad53d0 2023-08-27T11:12:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16d4495b80 1 -- 172.21.15.45:0/3317926016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 msgr2=0x56156caff0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16d4495b80 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caff0b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f16b800e940 tx=0x7f16b80589d0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16d4495b80 1 -- 172.21.15.45:0/3317926016 shutdown_connections 2023-08-27T11:12:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16d4495b80 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caff0b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16d4495b80 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb02320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16d4495b80 1 --2- 172.21.15.45:0/3317926016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 0x56156cafeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16d4495b80 1 -- 172.21.15.45:0/3317926016 >> 172.21.15.45:0/3317926016 conn(0x56156ca35c30 msgr2=0x56156cb1cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16d4495b80 1 -- 172.21.15.45:0/3317926016 shutdown_connections 2023-08-27T11:12:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.190+0000 7f16d4495b80 1 -- 172.21.15.45:0/3317926016 wait complete. 2023-08-27T11:12:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.191+0000 7f16d4495b80 1 -- start start 2023-08-27T11:12:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.191+0000 7f16d4495b80 1 Processor -- start 2023-08-27T11:12:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caf79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb1c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 0x56156cb1cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16d4495b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156c9c7b60 con 0x56156cad53d0 2023-08-27T11:12:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16d4495b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156c9c7860 con 0x56156cb13960 2023-08-27T11:12:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16d4495b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x56156c8ab8d0 con 0x56156cb01f40 2023-08-27T11:12:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16d4495b80 10 client.0 _ll_register_callbacks cb 0x56156c9c5630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:12:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c606f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caf79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c606f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caf79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52614/0 (socket says 172.21.15.45:52614) 2023-08-27T11:12:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c606f700 1 -- 172.21.15.45:0/2206119467 learned_addr learned my addr 172.21.15.45:0/2206119467 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c6870700 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 0x56156cb1cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c586e700 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb1c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c606f700 1 -- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 msgr2=0x56156cb1c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c606f700 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb1c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c606f700 1 -- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 msgr2=0x56156cb1cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c606f700 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 0x56156cb1cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.192+0000 7f16c606f700 1 -- 172.21.15.45:0/2206119467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c00cd770 con 0x56156cad53d0 2023-08-27T11:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16c606f700 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caf79b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16bc015a20 tx=0x7f16bc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2da835d84a04fb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:12:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16a17fa700 1 -- 172.21.15.45:0/2206119467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc01a040 con 0x56156cad53d0 2023-08-27T11:12:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16a17fa700 1 -- 172.21.15.45:0/2206119467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f16bc0145b0 con 0x56156cad53d0 2023-08-27T11:12:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16d4495b80 1 -- 172.21.15.45:0/2206119467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56156caca4f0 con 0x56156cad53d0 2023-08-27T11:12:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16a17fa700 1 -- 172.21.15.45:0/2206119467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16bc014770 con 0x56156cad53d0 2023-08-27T11:12:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16d4495b80 10 client.4875 fetch_fsmap learned FSMap version 9 2023-08-27T11:12:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16d4495b80 1 -- 172.21.15.45:0/2206119467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x56156ca41750 con 0x56156cad53d0 2023-08-27T11:12:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16a17fa700 1 -- 172.21.15.45:0/2206119467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56156caca4f0 con 0x56156cad53d0 2023-08-27T11:12:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16a17fa700 1 -- 172.21.15.45:0/2206119467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f16bc0148d0 con 0x56156cad53d0 2023-08-27T11:12:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16d4495b80 10 client.4875 fetch_fsmap finished waiting for FSMap version 9 2023-08-27T11:12:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16d4495b80 10 client.4875 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-27T11:12:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16d4495b80 20 client.4875 populate_metadata read hostname 'smithi045' 2023-08-27T11:12:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16d4495b80 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x56156cb1e580 0x56156cb1e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16d4495b80 4 client.4875 mds_command: new command op to 4303 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:12:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.193+0000 7f16d4495b80 1 -- 172.21.15.45:0/2206119467 --> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56156ca22920 con 0x56156cb1e580 2023-08-27T11:12:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.194+0000 7f16c586e700 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x56156cb1e580 0x56156cb1e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.194+0000 7f16a17fa700 10 client.4875 ms_handle_connect on v2:172.21.15.135:6834/739639682 2023-08-27T11:12:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.194+0000 7f16c586e700 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x56156cb1e580 0x56156cb1e960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead4b210b89dad48 server_cookie=d6c31b836b7dab4b in_seq=0 out_seq=0 2023-08-27T11:12:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16a17fa700 1 -- 172.21.15.45:0/2206119467 <== mds.0 v2:172.21.15.135:6834/739639682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56156ca22920 con 0x56156cb1e580 2023-08-27T11:12:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16a17fa700 10 client.4875 handle_command_reply: tid=0 2023-08-27T11:12:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 client.4875 shutdown 2023-08-27T11:12:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 -- 172.21.15.45:0/2206119467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 msgr2=0x56156caf79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caf79b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16bc015a20 tx=0x7f16bc001a00 comp rx=0 tx=0).stop 2023-08-27T11:12:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 -- 172.21.15.45:0/2206119467 shutdown_connections 2023-08-27T11:12:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x56156cb1e580 0x56156cb1e960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156caf79b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb13960 0x56156cb1cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 --2- 172.21.15.45:0/2206119467 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb1c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16a17fa700 0 client.4875 ms_handle_reset on v2:172.21.15.135:6834/739639682 2023-08-27T11:12:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 -- 172.21.15.45:0/2206119467 >> 172.21.15.45:0/2206119467 conn(0x56156ca35c30 msgr2=0x56156cad67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 -- 172.21.15.45:0/2206119467 shutdown_connections 2023-08-27T11:12:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 1 -- 172.21.15.45:0/2206119467 wait complete. 2023-08-27T11:12:47.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.195+0000 7f16d4495b80 20 client.4875 trim_cache size 0 max 16384 2023-08-27T11:12:47.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.213+0000 7f16d4495b80 1 Processor -- start 2023-08-27T11:12:47.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.213+0000 7f16d4495b80 1 -- start start 2023-08-27T11:12:47.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.213+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cad53d0 0x56156cad80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.213+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.213+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cb13960 0x56156cb00600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.213+0000 7f16d4495b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156cace4e0 con 0x56156cb13960 2023-08-27T11:12:47.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.213+0000 7f16d4495b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156cb01140 con 0x56156cad53d0 2023-08-27T11:12:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.213+0000 7f16d4495b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x56156cb02b20 con 0x56156cb01f40 2023-08-27T11:12:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c6870700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cb13960 0x56156cb00600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c6870700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cb13960 0x56156cb00600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52626/0 (socket says 172.21.15.45:52626) 2023-08-27T11:12:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c6870700 1 -- 172.21.15.45:0/755079804 learned_addr learned my addr 172.21.15.45:0/755079804 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c606f700 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cad53d0 0x56156cad80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c586e700 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c6870700 1 -- 172.21.15.45:0/755079804 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 msgr2=0x56156cb000c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c6870700 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb000c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c6870700 1 -- 172.21.15.45:0/755079804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cad53d0 msgr2=0x56156cad80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c6870700 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cad53d0 0x56156cad80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c6870700 1 -- 172.21.15.45:0/755079804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56156c855d90 con 0x56156cb13960 2023-08-27T11:12:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c606f700 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cad53d0 0x56156cad80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16c6870700 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cb13960 0x56156cb00600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16b800e940 tx=0x7f16b806e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3957ffce9b83b88 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16b0d9e700 1 -- 172.21.15.45:0/755079804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8058d10 con 0x56156cb13960 2023-08-27T11:12:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16b0d9e700 1 -- 172.21.15.45:0/755079804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8084040 con 0x56156cb13960 2023-08-27T11:12:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.214+0000 7f16b0d9e700 1 -- 172.21.15.45:0/755079804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8060c50 con 0x56156cb13960 2023-08-27T11:12:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 -- 172.21.15.45:0/755079804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cb13960 msgr2=0x56156cb00600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cb13960 0x56156cb00600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16b800e940 tx=0x7f16b806e000 comp rx=0 tx=0).stop 2023-08-27T11:12:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 -- 172.21.15.45:0/755079804 shutdown_connections 2023-08-27T11:12:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cb13960 0x56156cb00600 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cad53d0 0x56156cad80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 --2- 172.21.15.45:0/755079804 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb01f40 0x56156cb000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 -- 172.21.15.45:0/755079804 >> 172.21.15.45:0/755079804 conn(0x56156ca35c30 msgr2=0x56156cad43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 -- 172.21.15.45:0/755079804 shutdown_connections 2023-08-27T11:12:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 -- 172.21.15.45:0/755079804 wait complete. 2023-08-27T11:12:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.215+0000 7f16d4495b80 1 -- start start 2023-08-27T11:12:47.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.216+0000 7f16d4495b80 1 Processor -- start 2023-08-27T11:12:47.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156cb126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb01f40 0x56156cb12c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16d4495b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb13960 0x56156caf5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16d4495b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156caf6890 con 0x56156cad53d0 2023-08-27T11:12:47.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16d4495b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x56156cad0c80 con 0x56156cb01f40 2023-08-27T11:12:47.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16d4495b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x56156cad0ed0 con 0x56156cb13960 2023-08-27T11:12:47.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16d4495b80 10 client.0 _ll_register_callbacks cb 0x56156c9c5630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:12:47.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c606f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156cb126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c606f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156cb126e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52628/0 (socket says 172.21.15.45:52628) 2023-08-27T11:12:47.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c606f700 1 -- 172.21.15.45:0/1879327005 learned_addr learned my addr 172.21.15.45:0/1879327005 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:47.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c586e700 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb01f40 0x56156cb12c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c6870700 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb13960 0x56156caf5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c606f700 1 -- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb13960 msgr2=0x56156caf5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c606f700 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb13960 0x56156caf5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c606f700 1 -- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb01f40 msgr2=0x56156cb12c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:47.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c606f700 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb01f40 0x56156cb12c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.217+0000 7f16c606f700 1 -- 172.21.15.45:0/1879327005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56156c8a95e0 con 0x56156cad53d0 2023-08-27T11:12:47.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16c606f700 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156cb126e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005f80 tx=0x7f16bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7909b8ecf6a91f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:12:47.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16a17fa700 1 -- 172.21.15.45:0/1879327005 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc015720 con 0x56156cad53d0 2023-08-27T11:12:47.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16a17fa700 1 -- 172.21.15.45:0/1879327005 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f16bc0077e0 con 0x56156cad53d0 2023-08-27T11:12:47.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16d4495b80 1 -- 172.21.15.45:0/1879327005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16b80572e0 con 0x56156cad53d0 2023-08-27T11:12:47.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16a17fa700 1 -- 172.21.15.45:0/1879327005 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16bc02a350 con 0x56156cad53d0 2023-08-27T11:12:47.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16d4495b80 10 client.4881 fetch_fsmap learned FSMap version 9 2023-08-27T11:12:47.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16d4495b80 1 -- 172.21.15.45:0/1879327005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x56156c83ec10 con 0x56156cad53d0 2023-08-27T11:12:47.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16a17fa700 1 -- 172.21.15.45:0/1879327005 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b80572e0 con 0x56156cad53d0 2023-08-27T11:12:47.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16a17fa700 1 -- 172.21.15.45:0/1879327005 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f16bc02a740 con 0x56156cad53d0 2023-08-27T11:12:47.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16d4495b80 10 client.4881 fetch_fsmap finished waiting for FSMap version 9 2023-08-27T11:12:47.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16d4495b80 10 client.4881 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-27T11:12:47.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16d4495b80 20 client.4881 populate_metadata read hostname 'smithi045' 2023-08-27T11:12:47.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16d4495b80 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x56156caf5e80 0x56156cafe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16d4495b80 4 client.4881 mds_command: new command op to 4303 tid=0[{"prefix": "status"}] 2023-08-27T11:12:47.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16d4495b80 1 -- 172.21.15.45:0/1879327005 --> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56156c9e3000 con 0x56156caf5e80 2023-08-27T11:12:47.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.218+0000 7f16c586e700 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x56156caf5e80 0x56156cafe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16a17fa700 10 client.4881 ms_handle_connect on v2:172.21.15.135:6834/739639682 2023-08-27T11:12:47.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16c586e700 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x56156caf5e80 0x56156cafe690 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca649250f7f51e82 server_cookie=22bc0e4ec5aef51e in_seq=0 out_seq=0 2023-08-27T11:12:47.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16a17fa700 1 -- 172.21.15.45:0/1879327005 <== mds.0 v2:172.21.15.135:6834/739639682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x56156c9e3000 con 0x56156caf5e80 2023-08-27T11:12:47.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16a17fa700 10 client.4881 handle_command_reply: tid=0 2023-08-27T11:12:47.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 client.4881 shutdown 2023-08-27T11:12:47.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 -- 172.21.15.45:0/1879327005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 msgr2=0x56156cb126e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156cb126e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005f80 tx=0x7f16bc016000 comp rx=0 tx=0).stop 2023-08-27T11:12:47.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 -- 172.21.15.45:0/1879327005 shutdown_connections 2023-08-27T11:12:47.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:6834/739639682,v1:172.21.15.135:6836/739639682] conn(0x56156caf5e80 0x56156cafe690 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56156cad53d0 0x56156cb126e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x56156cb13960 0x56156caf5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 --2- 172.21.15.45:0/1879327005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56156cb01f40 0x56156cb12c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16a17fa700 0 client.4881 ms_handle_reset on v2:172.21.15.135:6834/739639682 2023-08-27T11:12:47.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 -- 172.21.15.45:0/1879327005 >> 172.21.15.45:0/1879327005 conn(0x56156ca35c30 msgr2=0x56156cb000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:47.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 -- 172.21.15.45:0/1879327005 shutdown_connections 2023-08-27T11:12:47.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 1 -- 172.21.15.45:0/1879327005 wait complete. 2023-08-27T11:12:47.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.219+0000 7f16d4495b80 20 client.4881 trim_cache size 0 max 16384 2023-08-27T11:12:47.244 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:12:47.244 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "8ea1ae34-1697-4b79-b2ce-3f741cd3c828", 2023-08-27T11:12:47.244 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-08-27T11:12:47.245 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4303, 2023-08-27T11:12:47.245 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-08-27T11:12:47.245 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-08-27T11:12:47.245 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-08-27T11:12:47.245 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 38.308188317000003, 2023-08-27T11:12:47.246 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 9, 2023-08-27T11:12:47.246 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 27, 2023-08-27T11:12:47.246 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 27, 2023-08-27T11:12:47.246 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 44.237786159000002 2023-08-27T11:12:47.246 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:12:47.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 -- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f16a406a0e0 msgr2=0x7f16a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f16a406a0e0 0x7f16a406c5a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0016d0 tx=0x7f16bc007ce0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 -- 172.21.15.45:0/2435076235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00b4800 msgr2=0x7f16c00b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00b4800 0x7f16c00b4c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009f90 tx=0x7f16b8009750 comp rx=0 tx=0).stop 2023-08-27T11:12:47.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 -- 172.21.15.45:0/2435076235 shutdown_connections 2023-08-27T11:12:47.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16c00b4800 0x7f16c00b4c80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f16a406a0e0 0x7f16a406c5a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16ac004d90 0x7f16c00b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 --2- 172.21.15.45:0/2435076235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f16c0154cf0 0x7f16c00b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 -- 172.21.15.45:0/2435076235 >> 172.21.15.45:0/2435076235 conn(0x7f16c00cfdd0 msgr2=0x7f16c0157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:47.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 -- 172.21.15.45:0/2435076235 shutdown_connections 2023-08-27T11:12:47.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.220+0000 7f16c82d3700 1 -- 172.21.15.45:0/2435076235 wait complete. 2023-08-27T11:12:47.251 INFO:teuthology.run_tasks:Running task kclient... 2023-08-27T11:12:47.289 INFO:tasks.kclient:Mounting kernel clients... 2023-08-27T11:12:47.290 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-08-27T11:12:47.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:47.291 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:12:47.308 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-08-27T11:12:47.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:47.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-27T11:12:47.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:47.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-08-27T11:12:47.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:47.499 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:12:47.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:47.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2023-08-27T11:12:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2023-08-27T11:12:47.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:12:47.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:47.589 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2023-08-27T11:12:47.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:47.610 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2023-08-27T11:12:47.685 INFO:teuthology.orchestra.run.smithi135.stderr:Cannot find device "ceph-brx" 2023-08-27T11:12:47.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:12:47.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:47.687 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2023-08-27T11:12:47.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:47.747 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2023-08-27T11:12:47.818 INFO:teuthology.orchestra.run.smithi135.stderr:Cannot find device "ceph-brx" 2023-08-27T11:12:47.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-27T11:12:47.820 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-08-27T11:12:47.820 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-27T11:12:47.821 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:12:47.821 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:12:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.946+0000 7fe330e9f700 1 Processor -- start 2023-08-27T11:12:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.946+0000 7fe330e9f700 1 -- start start 2023-08-27T11:12:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe330e9f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 0x7fe32c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe330e9f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c070090 con 0x7fe32c06c4d0 2023-08-27T11:12:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe330e9f700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c070370 con 0x7fe32c0cb920 2023-08-27T11:12:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe330e9f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c15ed10 con 0x7fe32c15e930 2023-08-27T11:12:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe32ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe32c06c4d0 0x7fe32c0b1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39416/0 (socket says 172.21.15.45:39416) 2023-08-27T11:12:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe32ad9d700 1 -- 172.21.15.45:0/1170262077 learned_addr learned my addr 172.21.15.45:0/1170262077 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe32a59c700 1 --2- 172.21.15.45:0/1170262077 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 0x7fe32c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe32959a700 1 -- 172.21.15.45:0/1170262077 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221351804 0 0) 0x7fe32c070090 con 0x7fe32c06c4d0 2023-08-27T11:12:47.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe32959a700 1 -- 172.21.15.45:0/1170262077 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3140035b0 con 0x7fe32c06c4d0 2023-08-27T11:12:47.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe32959a700 1 -- 172.21.15.45:0/1170262077 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145496895 0 0) 0x7fe32c070370 con 0x7fe32c0cb920 2023-08-27T11:12:47.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.947+0000 7fe32959a700 1 -- 172.21.15.45:0/1170262077 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c070090 con 0x7fe32c0cb920 2023-08-27T11:12:47.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.948+0000 7fe32a59c700 1 -- 172.21.15.45:0/1170262077 >> v1:172.21.15.135:6789/0 conn(0x7fe32c0cb920 legacy=0x7fe32c06bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.948+0000 7fe32a59c700 1 -- 172.21.15.45:0/1170262077 >> v1:172.21.15.45:6789/0 conn(0x7fe32c06c4d0 legacy=0x7fe32c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.948+0000 7fe32a59c700 1 -- 172.21.15.45:0/1170262077 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c0cae40 con 0x7fe32c15e930 2023-08-27T11:12:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.948+0000 7fe32a59c700 1 --2- 172.21.15.45:0/1170262077 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 0x7fe32c0c9c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe320009870 tx=0x7fe320009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b19d6692e4a4b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.948+0000 7fe32959a700 1 -- 172.21.15.45:0/1170262077 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320011830 con 0x7fe32c15e930 2023-08-27T11:12:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.948+0000 7fe32959a700 1 -- 172.21.15.45:0/1170262077 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe320011990 con 0x7fe32c15e930 2023-08-27T11:12:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.948+0000 7fe32959a700 1 -- 172.21.15.45:0/1170262077 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320011d00 con 0x7fe32c15e930 2023-08-27T11:12:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.948+0000 7fe330e9f700 1 -- 172.21.15.45:0/1170262077 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 msgr2=0x7fe32c0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.948+0000 7fe330e9f700 1 --2- 172.21.15.45:0/1170262077 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 0x7fe32c0c9c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe320009870 tx=0x7fe320009b80 comp rx=0 tx=0).stop 2023-08-27T11:12:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.949+0000 7fe330e9f700 1 -- 172.21.15.45:0/1170262077 shutdown_connections 2023-08-27T11:12:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.949+0000 7fe330e9f700 1 --2- 172.21.15.45:0/1170262077 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 0x7fe32c0c9c90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.949+0000 7fe330e9f700 1 -- 172.21.15.45:0/1170262077 >> 172.21.15.45:0/1170262077 conn(0x7fe32c0b07c0 msgr2=0x7fe32c0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.949+0000 7fe330e9f700 1 -- 172.21.15.45:0/1170262077 shutdown_connections 2023-08-27T11:12:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.949+0000 7fe330e9f700 1 -- 172.21.15.45:0/1170262077 wait complete. 2023-08-27T11:12:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.950+0000 7fe330e9f700 1 Processor -- start 2023-08-27T11:12:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.950+0000 7fe330e9f700 1 -- start start 2023-08-27T11:12:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.951+0000 7fe330e9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe32c06c4d0 0x7fe32c0ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.951+0000 7fe330e9f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe32c0cb920 0x7fe32c0adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.951+0000 7fe330e9f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 0x7fe32c15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.951+0000 7fe330e9f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c0cd1a0 con 0x7fe32c06c4d0 2023-08-27T11:12:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.951+0000 7fe330e9f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c0ae1d0 con 0x7fe32c0cb920 2023-08-27T11:12:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.951+0000 7fe330e9f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c15b8f0 con 0x7fe32c15e930 2023-08-27T11:12:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.951+0000 7fe32a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe32c06c4d0 0x7fe32c0ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe32a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe32c06c4d0 0x7fe32c0ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52642/0 (socket says 172.21.15.45:52642) 2023-08-27T11:12:47.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe32a59c700 1 -- 172.21.15.45:0/3908438915 learned_addr learned my addr 172.21.15.45:0/3908438915 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:47.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe329d9b700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe32c0cb920 0x7fe32c0adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe32ad9d700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 0x7fe32c15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe32a59c700 1 -- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 msgr2=0x7fe32c15b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe32a59c700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 0x7fe32c15b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe32a59c700 1 -- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe32c0cb920 msgr2=0x7fe32c0adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe32a59c700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe32c0cb920 0x7fe32c0adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe32a59c700 1 -- 172.21.15.45:0/3908438915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3200094a0 con 0x7fe32c06c4d0 2023-08-27T11:12:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe32a59c700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe32c06c4d0 0x7fe32c0ad450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe3200097c0 tx=0x7fe320004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe31affd700 1 -- 172.21.15.45:0/3908438915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320004550 con 0x7fe32c06c4d0 2023-08-27T11:12:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe31affd700 1 -- 172.21.15.45:0/3908438915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32001f070 con 0x7fe32c06c4d0 2023-08-27T11:12:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c0ae3c0 con 0x7fe32c06c4d0 2023-08-27T11:12:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe31affd700 1 -- 172.21.15.45:0/3908438915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3200038e0 con 0x7fe32c06c4d0 2023-08-27T11:12:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.952+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c0ccae0 con 0x7fe32c06c4d0 2023-08-27T11:12:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.953+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c0b4c40 con 0x7fe32c06c4d0 2023-08-27T11:12:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.954+0000 7fe31affd700 1 -- 172.21.15.45:0/3908438915 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe320003a40 con 0x7fe32c06c4d0 2023-08-27T11:12:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.954+0000 7fe31affd700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe30c06a0e0 0x7fe30c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.954+0000 7fe31affd700 1 -- 172.21.15.45:0/3908438915 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe320021070 con 0x7fe32c06c4d0 2023-08-27T11:12:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.954+0000 7fe329d9b700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe30c06a0e0 0x7fe30c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.955+0000 7fe329d9b700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe30c06a0e0 0x7fe30c06c5a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0adbe0 tx=0x7fe31c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:47.959+0000 7fe31affd700 1 -- 172.21.15.45:0/3908438915 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe32008e070 con 0x7fe32c06c4d0 2023-08-27T11:12:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.204+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe32c0ae550 con 0x7fe32c06c4d0 2023-08-27T11:12:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.205+0000 7fe31affd700 1 -- 172.21.15.45:0/3908438915 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe320015020 con 0x7fe32c06c4d0 2023-08-27T11:12:48.207 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-27T11:12:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe30c06a0e0 msgr2=0x7fe30c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe30c06a0e0 0x7fe30c06c5a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0adbe0 tx=0x7fe31c009040 comp rx=0 tx=0).stop 2023-08-27T11:12:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe32c06c4d0 msgr2=0x7fe32c0ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe32c06c4d0 0x7fe32c0ad450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe3200097c0 tx=0x7fe320004750 comp rx=0 tx=0).stop 2023-08-27T11:12:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 shutdown_connections 2023-08-27T11:12:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe32c06c4d0 0x7fe32c0ad450 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe30c06a0e0 0x7fe30c06c5a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe32c15e930 0x7fe32c15b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:48.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 --2- 172.21.15.45:0/3908438915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe32c0cb920 0x7fe32c0adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:48.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 >> 172.21.15.45:0/3908438915 conn(0x7fe32c0b07c0 msgr2=0x7fe32c0c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:48.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 shutdown_connections 2023-08-27T11:12:48.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:48.207+0000 7fe330e9f700 1 -- 172.21.15.45:0/3908438915 wait complete. 2023-08-27T11:12:48.220 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-27T11:12:48.221 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:12:48.221 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-08-27T11:12:48.221 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:12:48.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:12:48.222 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-27T11:12:48.222 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-27T11:12:48.222 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:12:48.223 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:12:48.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.223 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-08-27T11:12:48.240 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:12:48.241 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:12:48.241 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:12:48.241 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:48.241 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:12:48.241 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:48.242 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:48.242 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:48.242 INFO:teuthology.orchestra.run.smithi045.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:12:48.242 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:48.242 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-27T11:12:48.243 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42373sec preferred_lft 42373sec 2023-08-27T11:12:48.243 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-08-27T11:12:48.243 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:48.243 INFO:teuthology.orchestra.run.smithi045.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:48.243 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:48.244 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:48.244 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:48.244 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-27T11:12:48.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.245 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:48.245 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:48.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2023-08-27T11:12:48.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2023-08-27T11:12:48.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2023-08-27T11:12:48.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-27T11:12:48.246 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:48.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.359 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:12:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-08-27T11:12:48.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.388 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-08-27T11:12:48.447 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-08-27T11:12:48.447 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:12:48.447 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-27T11:12:48.448 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-27T11:12:48.448 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:12:48.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.449 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:48.449 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:48.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-27T11:12:48.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-27T11:12:48.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-27T11:12:48.450 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:48.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.555 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:12:48.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.572 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2023-08-27T11:12:48.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.629 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:48.629 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:48.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:12:48.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-08-27T11:12:48.630 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:48.723 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-08-27T11:12:48.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.724 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:48.724 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:48.724 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-08-27T11:12:48.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-27T11:12:48.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-08-27T11:12:48.727 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-08-27T11:12:48.727 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-08-27T11:12:48.727 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:48.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:48.958 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:48.958 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:48.958 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-08-27T11:12:48.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2023-08-27T11:12:48.959 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:49.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:49.084 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:12:49.108 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:12:49.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:49.108 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:12:49.165 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:12:49.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:49.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:12:49.255 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:12:49.255 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:12:49.255 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:12:49.370 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:12:49.370 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:12:49.371 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:12:49.371 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:12:49.371 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:12:49.443 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:12:49.444 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:12:49.444 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:12:49.445 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:12:49.445 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:12:49.445 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790" 2023-08-27T11:12:49.446 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2023-08-27T11:12:49.446 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8ea1ae34-1697-4b79-b2ce-3f741cd3c828" will pass to kernel 2023-08-27T11:12:49.448 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:12:49.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:49.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:12:49.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:49.476 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:49.548 INFO:teuthology.orchestra.run.smithi045.stdout:client4673 2023-08-27T11:12:49.549 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:12:49.549 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:12:49.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:49.606 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:49.679 INFO:teuthology.orchestra.run.smithi045.stdout:client4673 2023-08-27T11:12:49.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:49.680 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:49.753 INFO:teuthology.orchestra.run.smithi045.stdout:client4673 2023-08-27T11:12:49.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:49.754 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8ea1ae34-1697-4b79-b2ce-3f741cd3c828.client4673/status') 2023-08-27T11:12:49.835 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.4673 (3)172.21.15.45:0/462868862 2023-08-27T11:12:49.836 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2023-08-27T11:12:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2023-08-27T11:12:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2023-08-27T11:12:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:69 bytes copied, 2.4437e-05 s, 2.8 MB/s 2023-08-27T11:12:49.838 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-08-27T11:12:49.838 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-27T11:12:49.838 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-27T11:12:49.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:12:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.001+0000 7f053e394700 1 Processor -- start 2023-08-27T11:12:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.001+0000 7f053e394700 1 -- start start 2023-08-27T11:12:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f053e394700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f053806ce30 0x7f053806d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f053e394700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05380ce240 con 0x7f05380c4cc0 2023-08-27T11:12:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f053e394700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05380ce520 con 0x7f0538143820 2023-08-27T11:12:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f053e394700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05380ce050 con 0x7f053806ce30 2023-08-27T11:12:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f05377fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f05380c4cc0 0x7f05380cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39430/0 (socket says 172.21.15.45:39430) 2023-08-27T11:12:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f05377fe700 1 -- 172.21.15.45:0/4144489076 learned_addr learned my addr 172.21.15.45:0/4144489076 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f053c931700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f053806ce30 0x7f053806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033020281 0 0) 0x7f05380ce240 con 0x7f05380c4cc0 2023-08-27T11:12:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05200035b0 con 0x7f05380c4cc0 2023-08-27T11:12:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 132238312 0 0) 0x7f05200035b0 con 0x7f05380c4cc0 2023-08-27T11:12:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f053806ce30 msgr2=0x7f053806d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f053806ce30 0x7f053806d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 >> v1:172.21.15.135:6789/0 conn(0x7f0538143820 legacy=0x7f05380cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:50.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380ce800 con 0x7f05380c4cc0 2023-08-27T11:12:50.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f052c001670 con 0x7f05380c4cc0 2023-08-27T11:12:50.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 >> v1:172.21.15.45:6789/0 conn(0x7f05380c4cc0 legacy=0x7f05380cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:50.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f053806ce30 0x7f0520003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:50.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0520003fb0 0x7f0520004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:50.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0520004d90 0x7f0520005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052000bac0 con 0x7f0520004d90 2023-08-27T11:12:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052000bc30 con 0x7f0520003fb0 2023-08-27T11:12:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.002+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052000bdf0 con 0x7f053806ce30 2023-08-27T11:12:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f053c931700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0520004d90 0x7f0520005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f0537fff700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f053806ce30 0x7f0520003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f05377fe700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0520003fb0 0x7f0520004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f053c931700 1 -- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f053806ce30 msgr2=0x7f0520003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f053c931700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f053806ce30 0x7f0520003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f053c931700 1 -- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0520003fb0 msgr2=0x7f0520004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f053c931700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0520003fb0 0x7f0520004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f053c931700 1 -- 172.21.15.45:0/4144489076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200035b0 con 0x7f0520004d90 2023-08-27T11:12:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f053c931700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0520004d90 0x7f0520005210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0530009fc0 tx=0x7f0520005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3064dde31ab4018e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053000ec30 con 0x7f0520004d90 2023-08-27T11:12:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053000f040 con 0x7f0520004d90 2023-08-27T11:12:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.003+0000 7f0536ffd700 1 -- 172.21.15.45:0/4144489076 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530014480 con 0x7f0520004d90 2023-08-27T11:12:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.004+0000 7f053e394700 1 -- 172.21.15.45:0/4144489076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0520004d90 msgr2=0x7f0520005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.004+0000 7f053e394700 1 --2- 172.21.15.45:0/4144489076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0520004d90 0x7f0520005210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0530009fc0 tx=0x7f0520005880 comp rx=0 tx=0).stop 2023-08-27T11:12:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.004+0000 7f053c931700 1 -- 172.21.15.45:0/4144489076 reap_dead start 2023-08-27T11:12:50.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.004+0000 7f053e394700 1 -- 172.21.15.45:0/4144489076 shutdown_connections 2023-08-27T11:12:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.004+0000 7f053e394700 1 -- 172.21.15.45:0/4144489076 >> 172.21.15.45:0/4144489076 conn(0x7f05380cfd70 msgr2=0x7f05380d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.004+0000 7f053e394700 1 -- 172.21.15.45:0/4144489076 shutdown_connections 2023-08-27T11:12:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.004+0000 7f053e394700 1 -- 172.21.15.45:0/4144489076 wait complete. 2023-08-27T11:12:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.005+0000 7f053e394700 1 Processor -- start 2023-08-27T11:12:50.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.005+0000 7f053e394700 1 -- start start 2023-08-27T11:12:50.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053e394700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0538153200 0x7f053806d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:50.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053e394700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05380c8e00 0x7f0538133520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:50.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053e394700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0538133a60 0x7f05380c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:50.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053e394700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0538133ef0 con 0x7f0538133a60 2023-08-27T11:12:50.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053e394700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053806f560 con 0x7f0538153200 2023-08-27T11:12:50.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053e394700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05380c8a00 con 0x7f05380c8e00 2023-08-27T11:12:50.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053c931700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0538133a60 0x7f05380c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:50.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053c931700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0538133a60 0x7f05380c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52670/0 (socket says 172.21.15.45:52670) 2023-08-27T11:12:50.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053c931700 1 -- 172.21.15.45:0/3437749722 learned_addr learned my addr 172.21.15.45:0/3437749722 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:50.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f0537fff700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0538153200 0x7f053806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:50.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f05377fe700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05380c8e00 0x7f0538133520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:50.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053c931700 1 -- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05380c8e00 msgr2=0x7f0538133520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:50.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053c931700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05380c8e00 0x7f0538133520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053c931700 1 -- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0538153200 msgr2=0x7f053806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053c931700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0538153200 0x7f053806d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.006+0000 7f053c931700 1 -- 172.21.15.45:0/3437749722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053806d820 con 0x7f0538133a60 2023-08-27T11:12:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.007+0000 7f053c931700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0538133a60 0x7f05380c6e40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0530016000 tx=0x7f0530009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.007+0000 7f0534ff9700 1 -- 172.21.15.45:0/3437749722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530007000 con 0x7f0538133a60 2023-08-27T11:12:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.007+0000 7f0534ff9700 1 -- 172.21.15.45:0/3437749722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0530022070 con 0x7f0538133a60 2023-08-27T11:12:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.007+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053806db00 con 0x7f0538133a60 2023-08-27T11:12:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.007+0000 7f0534ff9700 1 -- 172.21.15.45:0/3437749722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530015e70 con 0x7f0538133a60 2023-08-27T11:12:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.007+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053806e0c0 con 0x7f0538133a60 2023-08-27T11:12:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.008+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05380727c0 con 0x7f0538133a60 2023-08-27T11:12:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.008+0000 7f0534ff9700 1 -- 172.21.15.45:0/3437749722 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f053001e430 con 0x7f0538133a60 2023-08-27T11:12:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.009+0000 7f0534ff9700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052406a0e0 0x7f052406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.009+0000 7f0534ff9700 1 -- 172.21.15.45:0/3437749722 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0530024070 con 0x7f0538133a60 2023-08-27T11:12:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.009+0000 7f0537fff700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052406a0e0 0x7f052406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.009+0000 7f0537fff700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052406a0e0 0x7f052406c5a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f0528009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:50.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.013+0000 7f0534ff9700 1 -- 172.21.15.45:0/3437749722 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0530055740 con 0x7f0538133a60 2023-08-27T11:12:50.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.264+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f05380734a0 con 0x7f0538133a60 2023-08-27T11:12:50.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.264+0000 7f0534ff9700 1 -- 172.21.15.45:0/3437749722 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0530058d60 con 0x7f0538133a60 2023-08-27T11:12:50.267 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-27T11:12:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052406a0e0 msgr2=0x7f052406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:50.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052406a0e0 0x7f052406c5a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f0528009040 comp rx=0 tx=0).stop 2023-08-27T11:12:50.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0538133a60 msgr2=0x7f05380c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:50.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0538133a60 0x7f05380c6e40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0530016000 tx=0x7f0530009750 comp rx=0 tx=0).stop 2023-08-27T11:12:50.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 shutdown_connections 2023-08-27T11:12:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0538133a60 0x7f05380c6e40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f052406a0e0 0x7f052406c5a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:50.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0538153200 0x7f053806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:50.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 --2- 172.21.15.45:0/3437749722 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05380c8e00 0x7f0538133520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:50.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 >> 172.21.15.45:0/3437749722 conn(0x7f05380cfd70 msgr2=0x7f05380c5130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:50.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 shutdown_connections 2023-08-27T11:12:50.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:50.266+0000 7f053e394700 1 -- 172.21.15.45:0/3437749722 wait complete. 2023-08-27T11:12:50.280 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-27T11:12:50.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:12:50.281 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-08-27T11:12:50.281 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-08-27T11:12:50.281 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-27T11:12:50.282 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-27T11:12:50.282 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-27T11:12:50.282 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:12:50.282 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-08-27T11:12:50.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.283 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-08-27T11:12:50.301 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:12:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:12:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:12:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:12:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.302 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:50.302 INFO:teuthology.orchestra.run.smithi045.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:12:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-27T11:12:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42371sec preferred_lft 42371sec 2023-08-27T11:12:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-08-27T11:12:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.304 INFO:teuthology.orchestra.run.smithi045.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:50.304 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:50.304 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:12:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether a6:e4:29:97:ce:d1 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:12:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a41d:a1ff:fed9:3bf2/64 scope link tentative 2023-08-27T11:12:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:12:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether a6:e4:29:97:ce:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:12:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a4e4:29ff:fe97:ced1/64 scope link tentative 2023-08-27T11:12:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.307 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:12:50.366 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-08-27T11:12:50.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.369 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-08-27T11:12:50.430 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-08-27T11:12:50.430 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:12:50.430 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-27T11:12:50.431 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-27T11:12:50.431 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:12:50.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.432 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:50.432 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:50.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-27T11:12:50.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-27T11:12:50.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-27T11:12:50.433 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:50.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.538 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:12:50.556 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:12:50.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.557 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2023-08-27T11:12:50.613 INFO:teuthology.orchestra.run.smithi045.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-08-27T11:12:50.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.614 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:50.615 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:50.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:12:50.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-08-27T11:12:50.616 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:50.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-27T11:12:50.746 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:12:50.746 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:12:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:12:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:12:50.747 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.747 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:12:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether e6:c3:27:f0:04:39 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-27T11:12:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-27T11:12:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::e4c3:27ff:fef0:439/64 scope link 2023-08-27T11:12:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.758 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-27T11:12:50.839 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:12:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:12:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:12:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:12:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.841 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:12:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether e6:c3:27:f0:04:39 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-27T11:12:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-27T11:12:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::e4c3:27ff:fef0:439/64 scope link 2023-08-27T11:12:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:50.853 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-08-27T11:12:50.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:50.853 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:50.854 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:50.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-08-27T11:12:50.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-27T11:12:50.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-08-27T11:12:50.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-08-27T11:12:50.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-08-27T11:12:50.855 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:51.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:51.108 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:12:51.109 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:12:51.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.1 up 2023-08-27T11:12:51.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.1 master ceph-brx 2023-08-27T11:12:51.110 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:12:51.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:51.229 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-08-27T11:12:51.250 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-08-27T11:12:51.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:51.251 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-08-27T11:12:51.306 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:12:51.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:51.307 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-27T11:12:51.393 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-27T11:12:51.393 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-08-27T11:12:51.393 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:12:51.402 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-27T11:12:51.402 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-08-27T11:12:51.402 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:12:51.403 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:12:51.403 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:12:51.476 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-27T11:12:51.476 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-08-27T11:12:51.476 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:12:51.476 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:12:51.477 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:12:51.477 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790" 2023-08-27T11:12:51.477 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2023-08-27T11:12:51.477 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=8ea1ae34-1697-4b79-b2ce-3f741cd3c828" will pass to kernel 2023-08-27T11:12:51.478 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:12:51.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:51.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-08-27T11:12:51.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:51.506 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:51.579 INFO:teuthology.orchestra.run.smithi045.stdout:client4896 2023-08-27T11:12:51.579 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:12:51.580 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:12:51.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:51.636 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:client4896 2023-08-27T11:12:51.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:51.710 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:51.785 INFO:teuthology.orchestra.run.smithi045.stdout:client4896 2023-08-27T11:12:51.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:51.786 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8ea1ae34-1697-4b79-b2ce-3f741cd3c828.client4896/status') 2023-08-27T11:12:51.867 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.4896 (3)192.168.0.2:0/2178287623 2023-08-27T11:12:51.868 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2023-08-27T11:12:51.869 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2023-08-27T11:12:51.869 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2023-08-27T11:12:51.869 INFO:teuthology.orchestra.run.smithi045.stderr:69 bytes copied, 2.214e-05 s, 3.1 MB/s 2023-08-27T11:12:51.869 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-08-27T11:12:51.870 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-27T11:12:51.870 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:12:51.870 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:12:52.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.032+0000 7f55b4cfc700 1 Processor -- start 2023-08-27T11:12:52.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.032+0000 7f55b4cfc700 1 -- start start 2023-08-27T11:12:52.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.033+0000 7f55b4cfc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55b00cbba0 0x7f55b0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:52.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.033+0000 7f55b4cfc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b0071280 0x7f55b00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:52.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.033+0000 7f55b4cfc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b00cc9a0 con 0x7f55b015a1f0 2023-08-27T11:12:52.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.033+0000 7f55b4cfc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b0072080 con 0x7f55b00cbba0 2023-08-27T11:12:52.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.033+0000 7f55b4cfc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0071ac0 con 0x7f55b0071280 2023-08-27T11:12:52.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.033+0000 7f55ae59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f55b015a1f0 0x7f55b00cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39454/0 (socket says 172.21.15.45:39454) 2023-08-27T11:12:52.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.033+0000 7f55ae59c700 1 -- 172.21.15.45:0/2471113081 learned_addr learned my addr 172.21.15.45:0/2471113081 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:52.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55aed9d700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b0071280 0x7f55b00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:52.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55add9b700 1 --2- 172.21.15.45:0/2471113081 >> v2:172.21.15.135:3300/0 conn(0x7f55b00cbba0 0x7f55b0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:52.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595537552 0 0) 0x7f55b00cc9a0 con 0x7f55b015a1f0 2023-08-27T11:12:52.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f559c0035b0 con 0x7f55b015a1f0 2023-08-27T11:12:52.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2040552863 0 0) 0x7f559c0035b0 con 0x7f55b015a1f0 2023-08-27T11:12:52.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b0071280 msgr2=0x7f55b00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:52.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b0071280 0x7f55b00cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 >> v2:172.21.15.135:3300/0 conn(0x7f55b00cbba0 msgr2=0x7f55b0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:52.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 --2- 172.21.15.45:0/2471113081 >> v2:172.21.15.135:3300/0 conn(0x7f55b00cbba0 0x7f55b0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b0071da0 con 0x7f55b015a1f0 2023-08-27T11:12:52.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f55a0001670 con 0x7f55b015a1f0 2023-08-27T11:12:52.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 >> v1:172.21.15.45:6789/0 conn(0x7f55b015a1f0 legacy=0x7f55b00cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:52.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c003a00 0x7f559c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:52.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c004850 0x7f559c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:52.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f559c005210 0x7f559c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:52.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0072080 con 0x7f559c004850 2023-08-27T11:12:52.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0071ac0 con 0x7f559c003a00 2023-08-27T11:12:52.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c00bf50 con 0x7f559c005210 2023-08-27T11:12:52.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55add9b700 1 --2- 172.21.15.45:0/2471113081 >> v2:172.21.15.135:3300/0 conn(0x7f55b00cbba0 0x7f55b0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:52.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55aed9d700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b0071280 0x7f55b00cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:52.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55add9b700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c004850 0x7f559c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:52.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55ae59c700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c003a00 0x7f559c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:52.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.034+0000 7f55aed9d700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f559c005210 0x7f559c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:52.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55add9b700 1 -- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f559c005210 msgr2=0x7f559c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:52.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55add9b700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f559c005210 0x7f559c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55add9b700 1 -- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c003a00 msgr2=0x7f559c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:52.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55add9b700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c003a00 0x7f559c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55add9b700 1 -- 172.21.15.45:0/2471113081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c0035b0 con 0x7f559c004850 2023-08-27T11:12:52.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55add9b700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c004850 0x7f559c004cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5598009d10 tx=0x7f559c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4a886a1b5c5b2c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:52.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598009e60 con 0x7f559c004850 2023-08-27T11:12:52.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5598004030 con 0x7f559c004850 2023-08-27T11:12:52.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55ad59a700 1 -- 172.21.15.45:0/2471113081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55980043a0 con 0x7f559c004850 2023-08-27T11:12:52.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55b4cfc700 1 -- 172.21.15.45:0/2471113081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c004850 msgr2=0x7f559c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:52.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55b4cfc700 1 --2- 172.21.15.45:0/2471113081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c004850 0x7f559c004cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5598009d10 tx=0x7f559c005880 comp rx=0 tx=0).stop 2023-08-27T11:12:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.035+0000 7f55aed9d700 1 -- 172.21.15.45:0/2471113081 reap_dead start 2023-08-27T11:12:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.036+0000 7f55b4cfc700 1 -- 172.21.15.45:0/2471113081 shutdown_connections 2023-08-27T11:12:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.036+0000 7f55b4cfc700 1 -- 172.21.15.45:0/2471113081 >> 172.21.15.45:0/2471113081 conn(0x7f55b00b07c0 msgr2=0x7f55b00b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.036+0000 7f55b4cfc700 1 -- 172.21.15.45:0/2471113081 shutdown_connections 2023-08-27T11:12:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.036+0000 7f55b4cfc700 1 -- 172.21.15.45:0/2471113081 wait complete. 2023-08-27T11:12:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.037+0000 7f55b4cfc700 1 Processor -- start 2023-08-27T11:12:52.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.037+0000 7f55b4cfc700 1 -- start start 2023-08-27T11:12:52.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55b4cfc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c004850 0x7f55b01552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:52.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55b4cfc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55b014d9f0 0x7f55b014d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:52.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55b4cfc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b00c1bf0 0x7f55b00c2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:52.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55b4cfc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0151110 con 0x7f55b014d9f0 2023-08-27T11:12:52.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55b4cfc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00c25b0 con 0x7f559c004850 2023-08-27T11:12:52.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55b4cfc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b014d660 con 0x7f55b00c1bf0 2023-08-27T11:12:52.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55add9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55b014d9f0 0x7f55b014d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:52.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55add9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55b014d9f0 0x7f55b014d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52692/0 (socket says 172.21.15.45:52692) 2023-08-27T11:12:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55add9b700 1 -- 172.21.15.45:0/3183752474 learned_addr learned my addr 172.21.15.45:0/3183752474 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55aed9d700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b00c1bf0 0x7f55b00c2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55ae59c700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c004850 0x7f55b01552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55add9b700 1 -- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b00c1bf0 msgr2=0x7f55b00c2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55add9b700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b00c1bf0 0x7f55b00c2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55add9b700 1 -- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c004850 msgr2=0x7f55b01552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55add9b700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c004850 0x7f55b01552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55add9b700 1 -- 172.21.15.45:0/3183752474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b014ea00 con 0x7f55b014d9f0 2023-08-27T11:12:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55aed9d700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b00c1bf0 0x7f55b00c2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55ae59c700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c004850 0x7f55b01552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55add9b700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55b014d9f0 0x7f55b014d120 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f55980090b0 tx=0x7f55980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55a6ffd700 1 -- 172.21.15.45:0/3183752474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800ed50 con 0x7f55b014d9f0 2023-08-27T11:12:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55a6ffd700 1 -- 172.21.15.45:0/3183752474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559800eef0 con 0x7f55b014d9f0 2023-08-27T11:12:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b014e160 con 0x7f55b014d9f0 2023-08-27T11:12:52.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.038+0000 7f55a6ffd700 1 -- 172.21.15.45:0/3183752474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55980038e0 con 0x7f55b014d9f0 2023-08-27T11:12:52.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.039+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b014e720 con 0x7f55b014d9f0 2023-08-27T11:12:52.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.039+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b0151380 con 0x7f55b014d9f0 2023-08-27T11:12:52.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.040+0000 7f55a6ffd700 1 -- 172.21.15.45:0/3183752474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5598003a40 con 0x7f55b014d9f0 2023-08-27T11:12:52.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.040+0000 7f55a6ffd700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f559006a0e0 0x7f559006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:52.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.040+0000 7f55a6ffd700 1 -- 172.21.15.45:0/3183752474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f559808c380 con 0x7f55b014d9f0 2023-08-27T11:12:52.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.041+0000 7f55ae59c700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f559006a0e0 0x7f559006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:52.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.041+0000 7f55ae59c700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f559006a0e0 0x7f559006c5a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55a0001050 tx=0x7f55a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:52.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.045+0000 7f55a6ffd700 1 -- 172.21.15.45:0/3183752474 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f559805b380 con 0x7f55b014d9f0 2023-08-27T11:12:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.293+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f55b00ca9e0 con 0x7f55b014d9f0 2023-08-27T11:12:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.293+0000 7f55a6ffd700 1 -- 172.21.15.45:0/3183752474 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f559805af10 con 0x7f55b014d9f0 2023-08-27T11:12:52.296 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-27T11:12:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f559006a0e0 msgr2=0x7f559006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:52.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f559006a0e0 0x7f559006c5a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55a0001050 tx=0x7f55a0009040 comp rx=0 tx=0).stop 2023-08-27T11:12:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55b014d9f0 msgr2=0x7f55b014d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:52.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55b014d9f0 0x7f55b014d120 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f55980090b0 tx=0x7f55980047c0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 shutdown_connections 2023-08-27T11:12:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f559006a0e0 0x7f559006c5a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55b00c1bf0 0x7f55b00c2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f559c004850 0x7f55b01552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 --2- 172.21.15.45:0/3183752474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55b014d9f0 0x7f55b014d120 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:52.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 >> 172.21.15.45:0/3183752474 conn(0x7f55b00b07c0 msgr2=0x7f55b00b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 shutdown_connections 2023-08-27T11:12:52.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:52.295+0000 7f55b4cfc700 1 -- 172.21.15.45:0/3183752474 wait complete. 2023-08-27T11:12:52.310 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-27T11:12:52.310 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:12:52.311 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi135.front.sepia.ceph.com 2023-08-27T11:12:52.311 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-27T11:12:52.312 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-27T11:12:52.312 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-27T11:12:52.312 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-27T11:12:52.313 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:12:52.313 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:12:52.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:52.315 DEBUG:teuthology.orchestra.run.smithi135:> ip addr 2023-08-27T11:12:52.336 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:12:52.337 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:12:52.337 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:12:52.338 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:52.338 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2023-08-27T11:12:52.338 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:52.338 INFO:teuthology.orchestra.run.smithi135.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:52.339 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:76 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:52.339 INFO:teuthology.orchestra.run.smithi135.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:52.339 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:77 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:52.340 INFO:teuthology.orchestra.run.smithi135.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:52.340 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:82 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:52.341 INFO:teuthology.orchestra.run.smithi135.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:12:52.341 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:83 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:52.341 INFO:teuthology.orchestra.run.smithi135.stdout: inet 172.21.15.135/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:12:52.341 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft 42375sec preferred_lft 42375sec 2023-08-27T11:12:52.342 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::87f2:7855:4d85:d55b/64 scope link noprefixroute 2023-08-27T11:12:52.342 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:52.342 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-27T11:12:52.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:52.343 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec bash -c ' 2023-08-27T11:12:52.343 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2023-08-27T11:12:52.344 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add name ceph-brx type bridge 2023-08-27T11:12:52.344 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr flush dev ceph-brx 2023-08-27T11:12:52.344 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set ceph-brx up 2023-08-27T11:12:52.344 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-27T11:12:52.345 DEBUG:teuthology.orchestra.run.smithi135:> ') 2023-08-27T11:12:52.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:52.527 DEBUG:teuthology.orchestra.run.smithi135:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:12:52.555 INFO:teuthology.orchestra.run.smithi135.stdout:1 2023-08-27T11:12:52.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:52.557 DEBUG:teuthology.orchestra.run.smithi135:> route 2023-08-27T11:12:52.632 INFO:teuthology.orchestra.run.smithi135.stdout:Kernel IP routing table 2023-08-27T11:12:52.632 INFO:teuthology.orchestra.run.smithi135.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:12:52.633 INFO:teuthology.orchestra.run.smithi135.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:12:52.633 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:12:52.633 INFO:teuthology.orchestra.run.smithi135.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:12:52.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:52.634 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec bash -c ' 2023-08-27T11:12:52.634 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2023-08-27T11:12:52.634 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:12:52.635 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:12:52.635 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:12:52.635 DEBUG:teuthology.orchestra.run.smithi135:> ') 2023-08-27T11:12:52.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:52.874 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2023-08-27T11:12:52.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:52.892 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list-id 2023-08-27T11:12:52.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:52.949 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec bash -c ' 2023-08-27T11:12:52.949 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2023-08-27T11:12:52.950 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:12:52.950 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-08-27T11:12:52.950 DEBUG:teuthology.orchestra.run.smithi135:> ') 2023-08-27T11:12:53.043 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-08-27T11:12:53.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:53.044 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec bash -c ' 2023-08-27T11:12:53.044 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2023-08-27T11:12:53.045 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-08-27T11:12:53.045 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-27T11:12:53.045 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-08-27T11:12:53.045 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-08-27T11:12:53.046 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-08-27T11:12:53.046 DEBUG:teuthology.orchestra.run.smithi135:> ') 2023-08-27T11:12:53.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:53.244 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec bash -c ' 2023-08-27T11:12:53.244 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2023-08-27T11:12:53.245 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set brx.0 up 2023-08-27T11:12:53.245 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set dev brx.0 master ceph-brx 2023-08-27T11:12:53.245 DEBUG:teuthology.orchestra.run.smithi135:> ') 2023-08-27T11:12:53.347 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:53.347 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-27T11:12:53.366 INFO:teuthology.orchestra.run.smithi135.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-27T11:12:53.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:53.368 DEBUG:teuthology.orchestra.run.smithi135:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:12:53.423 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:12:53.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:53.423 DEBUG:teuthology.orchestra.run.smithi135:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:12:53.519 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:12:53.519 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:12:53.520 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2023-08-27T11:12:53.613 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:12:53.613 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:12:53.613 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2023-08-27T11:12:53.614 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:12:53.614 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:12:53.645 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-27T11:12:53.645 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-27T11:12:53.646 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2023-08-27T11:12:53.646 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:12:53.646 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:12:53.647 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790" 2023-08-27T11:12:53.647 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2023-08-27T11:12:53.647 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8ea1ae34-1697-4b79-b2ce-3f741cd3c828" will pass to kernel 2023-08-27T11:12:53.649 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:12:53.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:53.650 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-27T11:12:53.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:53.682 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:53.753 INFO:teuthology.orchestra.run.smithi135.stdout:client4688 2023-08-27T11:12:53.754 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:12:53.755 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:12:53.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:53.812 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:53.884 INFO:teuthology.orchestra.run.smithi135.stdout:client4688 2023-08-27T11:12:53.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:53.885 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:53.956 INFO:teuthology.orchestra.run.smithi135.stdout:client4688 2023-08-27T11:12:53.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:53.956 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8ea1ae34-1697-4b79-b2ce-3f741cd3c828.client4688/status') 2023-08-27T11:12:54.037 INFO:teuthology.orchestra.run.smithi135.stdout:instance: client.4688 (3)192.168.0.1:0/3519603882 2023-08-27T11:12:54.037 INFO:teuthology.orchestra.run.smithi135.stdout:blocklisted: false 2023-08-27T11:12:54.038 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records in 2023-08-27T11:12:54.039 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records out 2023-08-27T11:12:54.039 INFO:teuthology.orchestra.run.smithi135.stderr:69 bytes copied, 2.2196e-05 s, 3.1 MB/s 2023-08-27T11:12:54.039 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-08-27T11:12:54.040 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-27T11:12:54.040 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-27T11:12:54.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:12:54.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.178+0000 7f9a803ba700 1 Processor -- start 2023-08-27T11:12:54.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.178+0000 7f9a803ba700 1 -- start start 2023-08-27T11:12:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.179+0000 7f9a803ba700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a7815d3d0 0x7f9a781619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.179+0000 7f9a803ba700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a780af040 0x7f9a780cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.179+0000 7f9a803ba700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a78161550 con 0x7f9a7806e3a0 2023-08-27T11:12:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.179+0000 7f9a803ba700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780cdf90 con 0x7f9a7815d3d0 2023-08-27T11:12:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.179+0000 7f9a803ba700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78161f20 con 0x7f9a780af040 2023-08-27T11:12:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.179+0000 7f9a7e156700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a7815d3d0 0x7f9a781619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.179+0000 7f9a7e156700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a7815d3d0 0x7f9a781619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52704/0 (socket says 172.21.15.45:52704) 2023-08-27T11:12:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.179+0000 7f9a7e156700 1 -- 172.21.15.45:0/2310980684 learned_addr learned my addr 172.21.15.45:0/2310980684 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.179+0000 7f9a7d955700 1 --2- 172.21.15.45:0/2310980684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a780af040 0x7f9a780cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.180+0000 7f9a7e156700 1 -- 172.21.15.45:0/2310980684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a780af040 msgr2=0x7f9a780cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.180+0000 7f9a7e156700 1 --2- 172.21.15.45:0/2310980684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a780af040 0x7f9a780cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.180+0000 7f9a7e156700 1 -- 172.21.15.45:0/2310980684 >> v1:172.21.15.135:6789/0 conn(0x7f9a7806e3a0 legacy=0x7f9a78070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.180+0000 7f9a7e156700 1 -- 172.21.15.45:0/2310980684 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780cd9d0 con 0x7f9a7815d3d0 2023-08-27T11:12:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.180+0000 7f9a7e156700 1 --2- 172.21.15.45:0/2310980684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7815d3d0 0x7f9a781619e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9a680098f0 tx=0x7f9a68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d02dd929a449ec server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.180+0000 7f9a7d154700 1 -- 172.21.15.45:0/2310980684 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68011830 con 0x7f9a7815d3d0 2023-08-27T11:12:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.180+0000 7f9a7d154700 1 -- 172.21.15.45:0/2310980684 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a68011990 con 0x7f9a7815d3d0 2023-08-27T11:12:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.180+0000 7f9a7d154700 1 -- 172.21.15.45:0/2310980684 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68011d20 con 0x7f9a7815d3d0 2023-08-27T11:12:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.181+0000 7f9a803ba700 1 -- 172.21.15.45:0/2310980684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7815d3d0 msgr2=0x7f9a781619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.181+0000 7f9a803ba700 1 --2- 172.21.15.45:0/2310980684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7815d3d0 0x7f9a781619e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9a680098f0 tx=0x7f9a68009c00 comp rx=0 tx=0).stop 2023-08-27T11:12:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.181+0000 7f9a803ba700 1 -- 172.21.15.45:0/2310980684 shutdown_connections 2023-08-27T11:12:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.181+0000 7f9a803ba700 1 --2- 172.21.15.45:0/2310980684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7815d3d0 0x7f9a781619e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.181+0000 7f9a803ba700 1 --2- 172.21.15.45:0/2310980684 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a780af040 0x7f9a780cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.181+0000 7f9a803ba700 1 -- 172.21.15.45:0/2310980684 >> 172.21.15.45:0/2310980684 conn(0x7f9a780cfd60 msgr2=0x7f9a780d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.181+0000 7f9a803ba700 1 -- 172.21.15.45:0/2310980684 shutdown_connections 2023-08-27T11:12:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.181+0000 7f9a803ba700 1 -- 172.21.15.45:0/2310980684 wait complete. 2023-08-27T11:12:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.182+0000 7f9a803ba700 1 Processor -- start 2023-08-27T11:12:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.183+0000 7f9a803ba700 1 -- start start 2023-08-27T11:12:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.183+0000 7f9a803ba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7806e3a0 0x7f9a78155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.183+0000 7f9a803ba700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a780af040 0x7f9a780ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.183+0000 7f9a803ba700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a7815d3d0 0x7f9a780acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.183+0000 7f9a803ba700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780b3210 con 0x7f9a7806e3a0 2023-08-27T11:12:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.183+0000 7f9a803ba700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780b34f0 con 0x7f9a780af040 2023-08-27T11:12:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.183+0000 7f9a803ba700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780b37d0 con 0x7f9a7815d3d0 2023-08-27T11:12:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e156700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7806e3a0 0x7f9a78155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e156700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7806e3a0 0x7f9a78155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52710/0 (socket says 172.21.15.45:52710) 2023-08-27T11:12:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e156700 1 -- 172.21.15.45:0/18596631 learned_addr learned my addr 172.21.15.45:0/18596631 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7d955700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a780af040 0x7f9a780ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e957700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a7815d3d0 0x7f9a780acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e156700 1 -- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a7815d3d0 msgr2=0x7f9a780acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e156700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a7815d3d0 0x7f9a780acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e156700 1 -- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a780af040 msgr2=0x7f9a780ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e156700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a780af040 0x7f9a780ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e156700 1 -- 172.21.15.45:0/18596631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a68009550 con 0x7f9a7806e3a0 2023-08-27T11:12:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a7e156700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7806e3a0 0x7f9a78155e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9a68000c00 tx=0x7f9a68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a6effd700 1 -- 172.21.15.45:0/18596631 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68004480 con 0x7f9a7806e3a0 2023-08-27T11:12:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a6effd700 1 -- 172.21.15.45:0/18596631 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6801f070 con 0x7f9a7806e3a0 2023-08-27T11:12:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a780b3ab0 con 0x7f9a7806e3a0 2023-08-27T11:12:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a6effd700 1 -- 172.21.15.45:0/18596631 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a680038c0 con 0x7f9a7806e3a0 2023-08-27T11:12:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.184+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a780b2e50 con 0x7f9a7806e3a0 2023-08-27T11:12:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.185+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a780b2fe0 con 0x7f9a7806e3a0 2023-08-27T11:12:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.186+0000 7f9a6effd700 1 -- 172.21.15.45:0/18596631 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9a68021030 con 0x7f9a7806e3a0 2023-08-27T11:12:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.186+0000 7f9a6effd700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a5c06a0e0 0x7f9a5c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.186+0000 7f9a6effd700 1 -- 172.21.15.45:0/18596631 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9a68089d70 con 0x7f9a7806e3a0 2023-08-27T11:12:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.186+0000 7f9a7d955700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a5c06a0e0 0x7f9a5c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:54.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.187+0000 7f9a7d955700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a5c06a0e0 0x7f9a5c06c5a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a780cf930 tx=0x7f9a74009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:54.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.191+0000 7f9a6effd700 1 -- 172.21.15.45:0/18596631 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9a680552e0 con 0x7f9a7806e3a0 2023-08-27T11:12:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.437+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9a780b3fd0 con 0x7f9a7806e3a0 2023-08-27T11:12:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.437+0000 7f9a6effd700 1 -- 172.21.15.45:0/18596631 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9a780b3fd0 con 0x7f9a7806e3a0 2023-08-27T11:12:54.440 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-27T11:12:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a5c06a0e0 msgr2=0x7f9a5c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a5c06a0e0 0x7f9a5c06c5a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a780cf930 tx=0x7f9a74009210 comp rx=0 tx=0).stop 2023-08-27T11:12:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7806e3a0 msgr2=0x7f9a78155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7806e3a0 0x7f9a78155e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9a68000c00 tx=0x7f9a68004750 comp rx=0 tx=0).stop 2023-08-27T11:12:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 shutdown_connections 2023-08-27T11:12:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a7806e3a0 0x7f9a78155e10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a5c06a0e0 0x7f9a5c06c5a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a7815d3d0 0x7f9a780acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 --2- 172.21.15.45:0/18596631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a780af040 0x7f9a780ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 >> 172.21.15.45:0/18596631 conn(0x7f9a780cfd60 msgr2=0x7f9a780d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.439+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 shutdown_connections 2023-08-27T11:12:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:54.440+0000 7f9a803ba700 1 -- 172.21.15.45:0/18596631 wait complete. 2023-08-27T11:12:54.453 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-27T11:12:54.454 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:12:54.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi135.front.sepia.ceph.com 2023-08-27T11:12:54.454 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-08-27T11:12:54.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-27T11:12:54.455 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-27T11:12:54.455 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-27T11:12:54.455 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:12:54.456 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-08-27T11:12:54.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:54.456 DEBUG:teuthology.orchestra.run.smithi135:> ip addr 2023-08-27T11:12:54.475 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:12:54.475 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:12:54.475 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:12:54.476 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.476 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2023-08-27T11:12:54.476 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.476 INFO:teuthology.orchestra.run.smithi135.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:54.476 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:76 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:54.477 INFO:teuthology.orchestra.run.smithi135.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:54.477 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:77 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:54.477 INFO:teuthology.orchestra.run.smithi135.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:12:54.477 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:82 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:54.478 INFO:teuthology.orchestra.run.smithi135.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:12:54.478 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:83 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:54.478 INFO:teuthology.orchestra.run.smithi135.stdout: inet 172.21.15.135/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-27T11:12:54.478 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft 42373sec preferred_lft 42373sec 2023-08-27T11:12:54.478 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::87f2:7855:4d85:d55b/64 scope link noprefixroute 2023-08-27T11:12:54.478 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.479 INFO:teuthology.orchestra.run.smithi135.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:12:54.479 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 2e:fb:65:ec:2e:9f brd ff:ff:ff:ff:ff:ff 2023-08-27T11:12:54.479 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:12:54.479 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.479 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::94af:dfff:fec6:b602/64 scope link 2023-08-27T11:12:54.480 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.480 INFO:teuthology.orchestra.run.smithi135.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:12:54.480 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 2e:fb:65:ec:2e:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-27T11:12:54.480 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::2cfb:65ff:feec:2e9f/64 scope link tentative 2023-08-27T11:12:54.480 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:54.481 DEBUG:teuthology.orchestra.run.smithi135:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:12:54.541 INFO:teuthology.orchestra.run.smithi135.stdout:1 2023-08-27T11:12:54.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:54.543 DEBUG:teuthology.orchestra.run.smithi135:> route 2023-08-27T11:12:54.602 INFO:teuthology.orchestra.run.smithi135.stdout:Kernel IP routing table 2023-08-27T11:12:54.603 INFO:teuthology.orchestra.run.smithi135.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:12:54.603 INFO:teuthology.orchestra.run.smithi135.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-27T11:12:54.604 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-27T11:12:54.604 INFO:teuthology.orchestra.run.smithi135.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:12:54.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:54.605 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec bash -c ' 2023-08-27T11:12:54.605 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2023-08-27T11:12:54.606 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-27T11:12:54.606 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-27T11:12:54.606 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-27T11:12:54.607 DEBUG:teuthology.orchestra.run.smithi135:> ') 2023-08-27T11:12:54.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:54.708 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2023-08-27T11:12:54.724 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-27T11:12:54.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:54.725 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list-id 2023-08-27T11:12:54.781 INFO:teuthology.orchestra.run.smithi135.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-08-27T11:12:54.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:54.782 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec bash -c ' 2023-08-27T11:12:54.782 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2023-08-27T11:12:54.782 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-27T11:12:54.782 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-08-27T11:12:54.783 DEBUG:teuthology.orchestra.run.smithi135:> ') 2023-08-27T11:12:54.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:54.875 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-27T11:12:54.915 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:12:54.915 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:12:54.915 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:12:54.915 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.916 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2023-08-27T11:12:54.916 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.916 INFO:teuthology.orchestra.run.smithi135.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:12:54.916 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 1e:a4:a0:94:6f:dd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-27T11:12:54.916 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-27T11:12:54.917 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.917 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::1ca4:a0ff:fe94:6fdd/64 scope link 2023-08-27T11:12:54.917 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:54.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:54.928 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-27T11:12:55.005 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:12:55.005 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:12:55.005 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:12:55.005 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:55.006 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2023-08-27T11:12:55.006 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:55.006 INFO:teuthology.orchestra.run.smithi135.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:12:55.006 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 1e:a4:a0:94:6f:dd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-27T11:12:55.007 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-27T11:12:55.007 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:55.007 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::1ca4:a0ff:fe94:6fdd/64 scope link 2023-08-27T11:12:55.007 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:12:55.022 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-08-27T11:12:55.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:55.023 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec bash -c ' 2023-08-27T11:12:55.023 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2023-08-27T11:12:55.023 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-08-27T11:12:55.024 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-27T11:12:55.024 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-08-27T11:12:55.024 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-08-27T11:12:55.025 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-08-27T11:12:55.025 DEBUG:teuthology.orchestra.run.smithi135:> ') 2023-08-27T11:12:55.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:55.260 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec bash -c ' 2023-08-27T11:12:55.260 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2023-08-27T11:12:55.261 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set brx.1 up 2023-08-27T11:12:55.261 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set dev brx.1 master ceph-brx 2023-08-27T11:12:55.261 DEBUG:teuthology.orchestra.run.smithi135:> ') 2023-08-27T11:12:55.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:55.379 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-08-27T11:12:55.400 INFO:teuthology.orchestra.run.smithi135.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-08-27T11:12:55.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:12:55.401 DEBUG:teuthology.orchestra.run.smithi135:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-08-27T11:12:55.457 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:12:55.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:55.458 DEBUG:teuthology.orchestra.run.smithi135:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-27T11:12:55.542 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-27T11:12:55.543 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-08-27T11:12:55.543 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2023-08-27T11:12:55.551 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-27T11:12:55.551 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-08-27T11:12:55.552 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2023-08-27T11:12:55.552 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:12:55.552 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:12:55.584 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-27T11:12:55.585 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-08-27T11:12:55.585 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2023-08-27T11:12:55.585 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:12:55.585 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:12:55.585 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790" 2023-08-27T11:12:55.586 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2023-08-27T11:12:55.586 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=8ea1ae34-1697-4b79-b2ce-3f741cd3c828" will pass to kernel 2023-08-27T11:12:55.588 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:12:55.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:55.588 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-08-27T11:12:55.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:55.622 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:55.694 INFO:teuthology.orchestra.run.smithi135.stdout:client4697 2023-08-27T11:12:55.694 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:12:55.695 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:12:55.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:55.752 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:55.824 INFO:teuthology.orchestra.run.smithi135.stdout:client4697 2023-08-27T11:12:55.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:55.825 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:12:55.895 INFO:teuthology.orchestra.run.smithi135.stdout:client4697 2023-08-27T11:12:55.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:12:55.896 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8ea1ae34-1697-4b79-b2ce-3f741cd3c828.client4697/status') 2023-08-27T11:12:55.975 INFO:teuthology.orchestra.run.smithi135.stdout:instance: client.4697 (3)192.168.0.2:0/51809840 2023-08-27T11:12:55.975 INFO:teuthology.orchestra.run.smithi135.stdout:blocklisted: false 2023-08-27T11:12:55.976 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records in 2023-08-27T11:12:55.976 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records out 2023-08-27T11:12:55.976 INFO:teuthology.orchestra.run.smithi135.stderr:67 bytes copied, 2.1791e-05 s, 3.1 MB/s 2023-08-27T11:12:55.977 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-08-27T11:12:56.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.422+0000 7f0a5109d700 1 Processor -- start 2023-08-27T11:12:56.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.423+0000 7f0a5109d700 1 -- start start 2023-08-27T11:12:56.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.423+0000 7f0a5109d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0a4c15e990 0x7f0a4c0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.423+0000 7f0a5109d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c0cc650 0x7f0a4c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.423+0000 7f0a5109d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4c0ccb80 con 0x7f0a4c0ce620 2023-08-27T11:12:56.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.423+0000 7f0a5109d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4c0cce60 con 0x7f0a4c15e990 2023-08-27T11:12:56.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.423+0000 7f0a5109d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0cd600 con 0x7f0a4c0cc650 2023-08-27T11:12:56.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4ad9d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0a4c15e990 0x7f0a4c0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4a59c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c0cc650 0x7f0a4c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4ad9d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0a4c15e990 0x7f0a4c0ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41574/0 (socket says 172.21.15.45:41574) 2023-08-27T11:12:56.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4ad9d700 1 -- 172.21.15.45:0/60931603 learned_addr learned my addr 172.21.15.45:0/60931603 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:56.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4a59c700 1 -- 172.21.15.45:0/60931603 >> v2:172.21.15.135:3300/0 conn(0x7f0a4c15e990 msgr2=0x7f0a4c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:56.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4a59c700 1 --2- 172.21.15.45:0/60931603 >> v2:172.21.15.135:3300/0 conn(0x7f0a4c15e990 0x7f0a4c0ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4a59c700 1 -- 172.21.15.45:0/60931603 >> v1:172.21.15.135:6789/0 conn(0x7f0a4c0ce620 legacy=0x7f0a4c0adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4a59c700 1 -- 172.21.15.45:0/60931603 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c0cd8e0 con 0x7f0a4c0cc650 2023-08-27T11:12:56.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4ad9d700 1 --2- 172.21.15.45:0/60931603 >> v2:172.21.15.135:3300/0 conn(0x7f0a4c15e990 0x7f0a4c0ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:56.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.424+0000 7f0a4a59c700 1 --2- 172.21.15.45:0/60931603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c0cc650 0x7f0a4c0ce0e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009840 tx=0x7f0a34009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f50a2736870406bd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:56.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a49d9b700 1 -- 172.21.15.45:0/60931603 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34011830 con 0x7f0a4c0cc650 2023-08-27T11:12:56.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a49d9b700 1 -- 172.21.15.45:0/60931603 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a34011990 con 0x7f0a4c0cc650 2023-08-27T11:12:56.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a49d9b700 1 -- 172.21.15.45:0/60931603 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34011d20 con 0x7f0a4c0cc650 2023-08-27T11:12:56.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a5109d700 1 -- 172.21.15.45:0/60931603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c0cc650 msgr2=0x7f0a4c0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a5109d700 1 --2- 172.21.15.45:0/60931603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c0cc650 0x7f0a4c0ce0e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009840 tx=0x7f0a34009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:56.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a5109d700 1 -- 172.21.15.45:0/60931603 shutdown_connections 2023-08-27T11:12:56.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a5109d700 1 --2- 172.21.15.45:0/60931603 >> v2:172.21.15.135:3300/0 conn(0x7f0a4c15e990 0x7f0a4c0ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a5109d700 1 --2- 172.21.15.45:0/60931603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c0cc650 0x7f0a4c0ce0e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a5109d700 1 -- 172.21.15.45:0/60931603 >> 172.21.15.45:0/60931603 conn(0x7f0a4c0d0f60 msgr2=0x7f0a4c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:56.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a5109d700 1 -- 172.21.15.45:0/60931603 shutdown_connections 2023-08-27T11:12:56.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.425+0000 7f0a5109d700 1 -- 172.21.15.45:0/60931603 wait complete. 2023-08-27T11:12:56.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.427+0000 7f0a5109d700 1 Processor -- start 2023-08-27T11:12:56.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.427+0000 7f0a5109d700 1 -- start start 2023-08-27T11:12:56.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.427+0000 7f0a5109d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a4c0cc650 0x7f0a4c0b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.427+0000 7f0a5109d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a4c0ce620 0x7f0a4c0c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a5109d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c15e990 0x7f0a4c0c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a5109d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0b4790 con 0x7f0a4c0ce620 2023-08-27T11:12:56.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a5109d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0b12b0 con 0x7f0a4c0cc650 2023-08-27T11:12:56.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a5109d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0b1590 con 0x7f0a4c15e990 2023-08-27T11:12:56.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a4c0ce620 0x7f0a4c0c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a4c0ce620 0x7f0a4c0c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52726/0 (socket says 172.21.15.45:52726) 2023-08-27T11:12:56.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4a59c700 1 -- 172.21.15.45:0/3781868027 learned_addr learned my addr 172.21.15.45:0/3781868027 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:56.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4b59e700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c15e990 0x7f0a4c0c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4ad9d700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a4c0cc650 0x7f0a4c0b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4a59c700 1 -- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c15e990 msgr2=0x7f0a4c0c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4a59c700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c15e990 0x7f0a4c0c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4a59c700 1 -- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a4c0cc650 msgr2=0x7f0a4c0b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4a59c700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a4c0cc650 0x7f0a4c0b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4a59c700 1 -- 172.21.15.45:0/3781868027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c009550 con 0x7f0a4c0ce620 2023-08-27T11:12:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a4a59c700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a4c0ce620 0x7f0a4c0c8030 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a340090b0 tx=0x7f0a34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a437fe700 1 -- 172.21.15.45:0/3781868027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a340036a0 con 0x7f0a4c0ce620 2023-08-27T11:12:56.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a437fe700 1 -- 172.21.15.45:0/3781868027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3401e070 con 0x7f0a4c0ce620 2023-08-27T11:12:56.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a340094a0 con 0x7f0a4c0ce620 2023-08-27T11:12:56.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.428+0000 7f0a437fe700 1 -- 172.21.15.45:0/3781868027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34003a60 con 0x7f0a4c0ce620 2023-08-27T11:12:56.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.429+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c0b1b50 con 0x7f0a4c0ce620 2023-08-27T11:12:56.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.429+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c0b16d0 con 0x7f0a4c0ce620 2023-08-27T11:12:56.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.430+0000 7f0a437fe700 1 -- 172.21.15.45:0/3781868027 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0a34003bc0 con 0x7f0a4c0ce620 2023-08-27T11:12:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.430+0000 7f0a437fe700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0a2c06a0e0 0x7f0a2c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.430+0000 7f0a437fe700 1 -- 172.21.15.45:0/3781868027 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0a340898b0 con 0x7f0a4c0ce620 2023-08-27T11:12:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.430+0000 7f0a4ad9d700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0a2c06a0e0 0x7f0a2c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.431+0000 7f0a4ad9d700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0a2c06a0e0 0x7f0a2c06c5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006d30 tx=0x7f0a3c0092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:56.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.435+0000 7f0a437fe700 1 -- 172.21.15.45:0/3781868027 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0a4c0b16d0 con 0x7f0a4c0ce620 2023-08-27T11:12:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.637+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a4c0b16d0 con 0x7f0a4c0ce620 2023-08-27T11:12:56.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.639+0000 7f0a437fe700 1 -- 172.21.15.45:0/3781868027 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7f0a4c0b16d0 con 0x7f0a4c0ce620 2023-08-27T11:12:56.641 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:56.641 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:56.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0a2c06a0e0 msgr2=0x7f0a2c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0a2c06a0e0 0x7f0a2c06c5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006d30 tx=0x7f0a3c0092c0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a4c0ce620 msgr2=0x7f0a4c0c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a4c0ce620 0x7f0a4c0c8030 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a340090b0 tx=0x7f0a34004750 comp rx=0 tx=0).stop 2023-08-27T11:12:56.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 shutdown_connections 2023-08-27T11:12:56.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0a2c06a0e0 0x7f0a2c06c5a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0a4c15e990 0x7f0a4c0c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a4c0cc650 0x7f0a4c0b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 --2- 172.21.15.45:0/3781868027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a4c0ce620 0x7f0a4c0c8030 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 >> 172.21.15.45:0/3781868027 conn(0x7f0a4c0d0f60 msgr2=0x7f0a4c15c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 shutdown_connections 2023-08-27T11:12:56.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.641+0000 7f0a5109d700 1 -- 172.21.15.45:0/3781868027 wait complete. 2023-08-27T11:12:56.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:12:56.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:12:56.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.782+0000 7f5466d8f700 1 Processor -- start 2023-08-27T11:12:56.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.782+0000 7f5466d8f700 1 -- start start 2023-08-27T11:12:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f5466d8f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f546006c260 0x7f54600cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f5466d8f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54600cdd40 0x7f546015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f5466d8f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54600d04f0 con 0x7f54600b02c0 2023-08-27T11:12:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f5466d8f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54600ce1d0 con 0x7f546006c260 2023-08-27T11:12:56.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f5466d8f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54600b06a0 con 0x7f54600cdd40 2023-08-27T11:12:56.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f5464b2b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f54600b02c0 0x7f54600cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39508/0 (socket says 172.21.15.45:39508) 2023-08-27T11:12:56.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f5464b2b700 1 -- 172.21.15.45:0/224855057 learned_addr learned my addr 172.21.15.45:0/224855057 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:56.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f545ffff700 1 --2- 172.21.15.45:0/224855057 >> v2:172.21.15.135:3300/0 conn(0x7f546006c260 0x7f54600cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f546532c700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54600cdd40 0x7f546015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508698252 0 0) 0x7f54600d04f0 con 0x7f54600b02c0 2023-08-27T11:12:56.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.783+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54500035b0 con 0x7f54600b02c0 2023-08-27T11:12:56.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 460499204 0 0) 0x7f54500035b0 con 0x7f54600b02c0 2023-08-27T11:12:56.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54600cdd40 msgr2=0x7f546015ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:56.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54600cdd40 0x7f546015ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 >> v2:172.21.15.135:3300/0 conn(0x7f546006c260 msgr2=0x7f54600cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 --2- 172.21.15.45:0/224855057 >> v2:172.21.15.135:3300/0 conn(0x7f546006c260 0x7f54600cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600cf8e0 con 0x7f54600b02c0 2023-08-27T11:12:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f5454001670 con 0x7f54600b02c0 2023-08-27T11:12:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 >> v1:172.21.15.45:6789/0 conn(0x7f54600b02c0 legacy=0x7f54600cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f5450003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54600cdd40 0x7f5450004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004d90 0x7f5450005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54600ce1d0 con 0x7f5450003a00 2023-08-27T11:12:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54600b06a0 con 0x7f5450004d90 2023-08-27T11:12:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545000ba10 con 0x7f54600cdd40 2023-08-27T11:12:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545ffff700 1 --2- 172.21.15.45:0/224855057 >> v2:172.21.15.135:3300/0 conn(0x7f546006c260 0x7f54600cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f5464b2b700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f5450003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f546532c700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004d90 0x7f5450005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545ffff700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54600cdd40 0x7f5450004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f5464b2b700 1 -- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54600cdd40 msgr2=0x7f5450004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f5464b2b700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54600cdd40 0x7f5450004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f5464b2b700 1 -- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004d90 msgr2=0x7f5450005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f5464b2b700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004d90 0x7f5450005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f5464b2b700 1 -- 172.21.15.45:0/224855057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500035b0 con 0x7f5450003a00 2023-08-27T11:12:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f546532c700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5450004d90 0x7f5450005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:56.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f5464b2b700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f5450003e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f545400c350 tx=0x7f5450005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891260488697469a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:56.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545400ce50 con 0x7f5450003a00 2023-08-27T11:12:56.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454005960 con 0x7f5450003a00 2023-08-27T11:12:56.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.784+0000 7f545f7fe700 1 -- 172.21.15.45:0/224855057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454005d10 con 0x7f5450003a00 2023-08-27T11:12:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.785+0000 7f5466d8f700 1 -- 172.21.15.45:0/224855057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 msgr2=0x7f5450003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.785+0000 7f5466d8f700 1 --2- 172.21.15.45:0/224855057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f5450003e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f545400c350 tx=0x7f5450005880 comp rx=0 tx=0).stop 2023-08-27T11:12:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.785+0000 7f546532c700 1 -- 172.21.15.45:0/224855057 reap_dead start 2023-08-27T11:12:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.785+0000 7f5466d8f700 1 -- 172.21.15.45:0/224855057 shutdown_connections 2023-08-27T11:12:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.785+0000 7f5466d8f700 1 -- 172.21.15.45:0/224855057 >> 172.21.15.45:0/224855057 conn(0x7f546009a1a0 msgr2=0x7f54600d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.785+0000 7f5466d8f700 1 -- 172.21.15.45:0/224855057 shutdown_connections 2023-08-27T11:12:56.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.785+0000 7f5466d8f700 1 -- 172.21.15.45:0/224855057 wait complete. 2023-08-27T11:12:56.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.787+0000 7f5466d8f700 1 Processor -- start 2023-08-27T11:12:56.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.787+0000 7f5466d8f700 1 -- start start 2023-08-27T11:12:56.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5466d8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f546015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5466d8f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546009d680 0x7f54600cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5466d8f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54600ccdc0 0x7f546014fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5466d8f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546009c470 con 0x7f5450003a00 2023-08-27T11:12:56.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5466d8f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546009c750 con 0x7f54600ccdc0 2023-08-27T11:12:56.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5466d8f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546009ca30 con 0x7f546009d680 2023-08-27T11:12:56.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5464b2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f546015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5464b2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f546015ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52744/0 (socket says 172.21.15.45:52744) 2023-08-27T11:12:56.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5464b2b700 1 -- 172.21.15.45:0/2224926727 learned_addr learned my addr 172.21.15.45:0/2224926727 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:56.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f546532c700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54600ccdc0 0x7f546014fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f545ffff700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546009d680 0x7f54600cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5464b2b700 1 -- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546009d680 msgr2=0x7f54600cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5464b2b700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546009d680 0x7f54600cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5464b2b700 1 -- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54600ccdc0 msgr2=0x7f546014fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:56.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5464b2b700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54600ccdc0 0x7f546014fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:56.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5464b2b700 1 -- 172.21.15.45:0/2224926727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546009cd10 con 0x7f5450003a00 2023-08-27T11:12:56.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.788+0000 7f5464b2b700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f546015ee80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5454008fd0 tx=0x7f5454001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:56.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.789+0000 7f545d7fa700 1 -- 172.21.15.45:0/2224926727 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454001680 con 0x7f5450003a00 2023-08-27T11:12:56.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.789+0000 7f545d7fa700 1 -- 172.21.15.45:0/2224926727 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454004a80 con 0x7f5450003a00 2023-08-27T11:12:56.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.789+0000 7f545d7fa700 1 -- 172.21.15.45:0/2224926727 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454004e30 con 0x7f5450003a00 2023-08-27T11:12:56.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.789+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546009cff0 con 0x7f5450003a00 2023-08-27T11:12:56.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.789+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546015f3c0 con 0x7f5450003a00 2023-08-27T11:12:56.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.790+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546015f5b0 con 0x7f5450003a00 2023-08-27T11:12:56.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.790+0000 7f545d7fa700 1 -- 172.21.15.45:0/2224926727 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f54540070d0 con 0x7f5450003a00 2023-08-27T11:12:56.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.790+0000 7f545d7fa700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544406a0e0 0x7f544406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:56.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.791+0000 7f545d7fa700 1 -- 172.21.15.45:0/2224926727 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5454089ea0 con 0x7f5450003a00 2023-08-27T11:12:56.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.791+0000 7f545ffff700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544406a0e0 0x7f544406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:56.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.791+0000 7f545ffff700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544406a0e0 0x7f544406c5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f544c009be0 tx=0x7f544c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:56.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.795+0000 7f545d7fa700 1 -- 172.21.15.45:0/2224926727 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f54540554d0 con 0x7f5450003a00 2023-08-27T11:12:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.998+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f546009eda0 con 0x7f5450003a00 2023-08-27T11:12:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:56.999+0000 7f545d7fa700 1 -- 172.21.15.45:0/2224926727 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5454007da0 con 0x7f5450003a00 2023-08-27T11:12:57.002 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:12:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544406a0e0 msgr2=0x7f544406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544406a0e0 0x7f544406c5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f544c009be0 tx=0x7f544c008040 comp rx=0 tx=0).stop 2023-08-27T11:12:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 msgr2=0x7f546015ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f546015ee80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5454008fd0 tx=0x7f5454001a00 comp rx=0 tx=0).stop 2023-08-27T11:12:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 shutdown_connections 2023-08-27T11:12:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5450003a00 0x7f546015ee80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f544406a0e0 0x7f544406c5a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54600ccdc0 0x7f546014fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 --2- 172.21.15.45:0/2224926727 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f546009d680 0x7f54600cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 >> 172.21.15.45:0/2224926727 conn(0x7f546009a1a0 msgr2=0x7f54600cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 shutdown_connections 2023-08-27T11:12:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.001+0000 7f5466d8f700 1 -- 172.21.15.45:0/2224926727 wait complete. 2023-08-27T11:12:57.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:12:57.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.154+0000 7f8f64b3d700 1 Processor -- start 2023-08-27T11:12:57.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.154+0000 7f8f64b3d700 1 -- start start 2023-08-27T11:12:57.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f64b3d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8f6015d340 0x7f8f60160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f64b3d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8f600cd880 0x7f8f6006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f64b3d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6006b160 con 0x7f8f600cdd00 2023-08-27T11:12:57.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f64b3d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f60072010 con 0x7f8f6015d340 2023-08-27T11:12:57.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f64b3d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f600af590 con 0x7f8f600cd880 2023-08-27T11:12:57.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f5e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8f6015d340 0x7f8f60160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f5e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8f6015d340 0x7f8f60160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52752/0 (socket says 172.21.15.45:52752) 2023-08-27T11:12:57.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f5e59c700 1 -- 172.21.15.45:0/1477518607 learned_addr learned my addr 172.21.15.45:0/1477518607 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:57.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f5dd9b700 1 --2- 172.21.15.45:0/1477518607 >> v2:172.21.15.135:3300/0 conn(0x7f8f600cd880 0x7f8f6006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f5e59c700 1 -- 172.21.15.45:0/1477518607 >> v2:172.21.15.135:3300/0 conn(0x7f8f600cd880 msgr2=0x7f8f6006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:57.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f5e59c700 1 --2- 172.21.15.45:0/1477518607 >> v2:172.21.15.135:3300/0 conn(0x7f8f600cd880 0x7f8f6006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f5e59c700 1 -- 172.21.15.45:0/1477518607 >> v1:172.21.15.135:6789/0 conn(0x7f8f600cdd00 legacy=0x7f8f600b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:57.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f5e59c700 1 -- 172.21.15.45:0/1477518607 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600af2b0 con 0x7f8f6015d340 2023-08-27T11:12:57.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.155+0000 7f8f5e59c700 1 --2- 172.21.15.45:0/1477518607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 0x7f8f60160d00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f600cd7f0 tx=0x7f8f50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936182c4801cbd05 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:57.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f5d59a700 1 -- 172.21.15.45:0/1477518607 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50004230 con 0x7f8f6015d340 2023-08-27T11:12:57.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f5d59a700 1 -- 172.21.15.45:0/1477518607 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f50004390 con 0x7f8f6015d340 2023-08-27T11:12:57.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f5d59a700 1 -- 172.21.15.45:0/1477518607 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f500036a0 con 0x7f8f6015d340 2023-08-27T11:12:57.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f64b3d700 1 -- 172.21.15.45:0/1477518607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 msgr2=0x7f8f60160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f64b3d700 1 --2- 172.21.15.45:0/1477518607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 0x7f8f60160d00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f600cd7f0 tx=0x7f8f50009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f64b3d700 1 -- 172.21.15.45:0/1477518607 shutdown_connections 2023-08-27T11:12:57.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f64b3d700 1 --2- 172.21.15.45:0/1477518607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 0x7f8f60160d00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f64b3d700 1 --2- 172.21.15.45:0/1477518607 >> v2:172.21.15.135:3300/0 conn(0x7f8f600cd880 0x7f8f6006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f64b3d700 1 -- 172.21.15.45:0/1477518607 >> 172.21.15.45:0/1477518607 conn(0x7f8f600cc7c0 msgr2=0x7f8f600ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:57.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f64b3d700 1 -- 172.21.15.45:0/1477518607 shutdown_connections 2023-08-27T11:12:57.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.156+0000 7f8f64b3d700 1 -- 172.21.15.45:0/1477518607 wait complete. 2023-08-27T11:12:57.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.157+0000 7f8f64b3d700 1 Processor -- start 2023-08-27T11:12:57.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f64b3d700 1 -- start start 2023-08-27T11:12:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f64b3d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cd880 0x7f8f60154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f64b3d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600cdd00 0x7f8f600c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f64b3d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 0x7f8f60147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f64b3d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c8630 con 0x7f8f6015d340 2023-08-27T11:12:57.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f64b3d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c72c0 con 0x7f8f600cdd00 2023-08-27T11:12:57.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f64b3d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c7590 con 0x7f8f600cd880 2023-08-27T11:12:57.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f5ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 0x7f8f60147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f5ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 0x7f8f60147fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52762/0 (socket says 172.21.15.45:52762) 2023-08-27T11:12:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.158+0000 7f8f5ed9d700 1 -- 172.21.15.45:0/624797004 learned_addr learned my addr 172.21.15.45:0/624797004 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f5e59c700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cd880 0x7f8f60154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f5dd9b700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600cdd00 0x7f8f600c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f5ed9d700 1 -- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cd880 msgr2=0x7f8f60154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f5ed9d700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cd880 0x7f8f60154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f5ed9d700 1 -- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600cdd00 msgr2=0x7f8f600c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f5ed9d700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600cdd00 0x7f8f600c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f5ed9d700 1 -- 172.21.15.45:0/624797004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f50009550 con 0x7f8f6015d340 2023-08-27T11:12:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f5ed9d700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 0x7f8f60147fc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800b9e0 tx=0x7f8f5800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:57.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f4effd700 1 -- 172.21.15.45:0/624797004 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58003c00 con 0x7f8f6015d340 2023-08-27T11:12:57.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f4effd700 1 -- 172.21.15.45:0/624797004 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f58003da0 con 0x7f8f6015d340 2023-08-27T11:12:57.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f600c7870 con 0x7f8f6015d340 2023-08-27T11:12:57.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f4effd700 1 -- 172.21.15.45:0/624797004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800d920 con 0x7f8f6015d340 2023-08-27T11:12:57.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.159+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f600c7e30 con 0x7f8f6015d340 2023-08-27T11:12:57.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.160+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f60072690 con 0x7f8f6015d340 2023-08-27T11:12:57.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.161+0000 7f8f4effd700 1 -- 172.21.15.45:0/624797004 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8f5800da80 con 0x7f8f6015d340 2023-08-27T11:12:57.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.161+0000 7f8f4effd700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8f4006a0e0 0x7f8f4006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.161+0000 7f8f4effd700 1 -- 172.21.15.45:0/624797004 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8f58011070 con 0x7f8f6015d340 2023-08-27T11:12:57.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.161+0000 7f8f5e59c700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8f4006a0e0 0x7f8f4006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.162+0000 7f8f5e59c700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8f4006a0e0 0x7f8f4006c5a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009130 tx=0x7f8f50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:57.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.165+0000 7f8f4effd700 1 -- 172.21.15.45:0/624797004 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8f58053ad0 con 0x7f8f6015d340 2023-08-27T11:12:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.364+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8f600734a0 con 0x7f8f6015d340 2023-08-27T11:12:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.365+0000 7f8f4effd700 1 -- 172.21.15.45:0/624797004 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8f580570f0 con 0x7f8f6015d340 2023-08-27T11:12:57.367 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:12:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8f4006a0e0 msgr2=0x7f8f4006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8f4006a0e0 0x7f8f4006c5a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009130 tx=0x7f8f50008030 comp rx=0 tx=0).stop 2023-08-27T11:12:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 msgr2=0x7f8f60147fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 0x7f8f60147fc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800b9e0 tx=0x7f8f5800bcf0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 shutdown_connections 2023-08-27T11:12:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6015d340 0x7f8f60147fc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8f4006a0e0 0x7f8f4006c5a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f600cd880 0x7f8f60154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 --2- 172.21.15.45:0/624797004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f600cdd00 0x7f8f600c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.366+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 >> 172.21.15.45:0/624797004 conn(0x7f8f600cc7c0 msgr2=0x7f8f6015e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:57.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.367+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 shutdown_connections 2023-08-27T11:12:57.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.367+0000 7f8f64b3d700 1 -- 172.21.15.45:0/624797004 wait complete. 2023-08-27T11:12:57.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:12:57.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.508+0000 7f55fb899700 1 Processor -- start 2023-08-27T11:12:57.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.508+0000 7f55fb899700 1 -- start start 2023-08-27T11:12:57.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55fb899700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55f406f620 0x7f55f40cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55fb899700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f40cb060 con 0x7f55f415b5e0 2023-08-27T11:12:57.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55fb899700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f415d7d0 con 0x7f55f415a700 2023-08-27T11:12:57.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55fb899700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f406e320 con 0x7f55f406f620 2023-08-27T11:12:57.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55f8e34700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f55f415b5e0 0x7f55f406ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39542/0 (socket says 172.21.15.45:39542) 2023-08-27T11:12:57.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55f8e34700 1 -- 172.21.15.45:0/1754054623 learned_addr learned my addr 172.21.15.45:0/1754054623 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:12:57.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55f9e36700 1 --2- 172.21.15.45:0/1754054623 >> v2:172.21.15.135:3300/0 conn(0x7f55f406f620 0x7f55f40cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55ebfff700 1 -- 172.21.15.45:0/1754054623 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953870181 0 0) 0x7f55f40cb060 con 0x7f55f415b5e0 2023-08-27T11:12:57.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55ebfff700 1 -- 172.21.15.45:0/1754054623 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e00035b0 con 0x7f55f415b5e0 2023-08-27T11:12:57.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55ebfff700 1 -- 172.21.15.45:0/1754054623 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3940874858 0 0) 0x7f55e00035b0 con 0x7f55f415b5e0 2023-08-27T11:12:57.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55f9e36700 1 -- 172.21.15.45:0/1754054623 >> v1:172.21.15.135:6789/0 conn(0x7f55f415a700 legacy=0x7f55f415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55f9e36700 1 -- 172.21.15.45:0/1754054623 >> v1:172.21.15.45:6789/0 conn(0x7f55f415b5e0 legacy=0x7f55f406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.509+0000 7f55f9e36700 1 -- 172.21.15.45:0/1754054623 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40ca450 con 0x7f55f406f620 2023-08-27T11:12:57.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55f9e36700 1 --2- 172.21.15.45:0/1754054623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55f406f620 0x7f55f40cb4f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009d90 tx=0x7f55ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6529a10f8ba041f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:57.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55ebfff700 1 -- 172.21.15.45:0/1754054623 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00e070 con 0x7f55f406f620 2023-08-27T11:12:57.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55ebfff700 1 -- 172.21.15.45:0/1754054623 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec003a20 con 0x7f55f406f620 2023-08-27T11:12:57.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55ebfff700 1 -- 172.21.15.45:0/1754054623 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0001670 con 0x7f55f406f620 2023-08-27T11:12:57.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55fb899700 1 -- 172.21.15.45:0/1754054623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55f406f620 msgr2=0x7f55f40cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55fb899700 1 --2- 172.21.15.45:0/1754054623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55f406f620 0x7f55f40cb4f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009d90 tx=0x7f55ec00b330 comp rx=0 tx=0).stop 2023-08-27T11:12:57.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55fb899700 1 -- 172.21.15.45:0/1754054623 shutdown_connections 2023-08-27T11:12:57.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55fb899700 1 --2- 172.21.15.45:0/1754054623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55f406f620 0x7f55f40cb4f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55fb899700 1 -- 172.21.15.45:0/1754054623 >> 172.21.15.45:0/1754054623 conn(0x7f55f40cc7e0 msgr2=0x7f55f40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:57.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55fb899700 1 -- 172.21.15.45:0/1754054623 shutdown_connections 2023-08-27T11:12:57.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.510+0000 7f55fb899700 1 -- 172.21.15.45:0/1754054623 wait complete. 2023-08-27T11:12:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55fb899700 1 Processor -- start 2023-08-27T11:12:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55fb899700 1 -- start start 2023-08-27T11:12:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55fb899700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f406f620 0x7f55f40b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55fb899700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55f415a700 0x7f55f41470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55fb899700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55f415b5e0 0x7f55f4143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55fb899700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40c6f90 con 0x7f55f406f620 2023-08-27T11:12:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55fb899700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40b56f0 con 0x7f55f415b5e0 2023-08-27T11:12:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55fb899700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40b2fa0 con 0x7f55f415a700 2023-08-27T11:12:57.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55f9635700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f406f620 0x7f55f40b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55f9635700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f406f620 0x7f55f40b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52772/0 (socket says 172.21.15.45:52772) 2023-08-27T11:12:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.512+0000 7f55f9635700 1 -- 172.21.15.45:0/1503709942 learned_addr learned my addr 172.21.15.45:0/1503709942 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55f8e34700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55f415a700 0x7f55f41470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55f9635700 1 -- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55f415a700 msgr2=0x7f55f41470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55f9635700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55f415a700 0x7f55f41470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55f9635700 1 -- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55f415b5e0 msgr2=0x7f55f4143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55f9635700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55f415b5e0 0x7f55f4143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55f9635700 1 -- 172.21.15.45:0/1503709942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec009a70 con 0x7f55f406f620 2023-08-27T11:12:57.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55f9635700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f406f620 0x7f55f40b29c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55e400c490 tx=0x7f55e400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:57.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55e9ffb700 1 -- 172.21.15.45:0/1503709942 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e40052d0 con 0x7f55f406f620 2023-08-27T11:12:57.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55e9ffb700 1 -- 172.21.15.45:0/1503709942 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e4005450 con 0x7f55f406f620 2023-08-27T11:12:57.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40b5990 con 0x7f55f406f620 2023-08-27T11:12:57.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55e9ffb700 1 -- 172.21.15.45:0/1503709942 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4006aa0 con 0x7f55f406f620 2023-08-27T11:12:57.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.513+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f4143870 con 0x7f55f406f620 2023-08-27T11:12:57.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.514+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f4143a00 con 0x7f55f406f620 2023-08-27T11:12:57.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.514+0000 7f55e9ffb700 1 -- 172.21.15.45:0/1503709942 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f55e4003680 con 0x7f55f406f620 2023-08-27T11:12:57.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.515+0000 7f55e9ffb700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f55d806a0e0 0x7f55d806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.515+0000 7f55e9ffb700 1 -- 172.21.15.45:0/1503709942 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f55e4087ad0 con 0x7f55f406f620 2023-08-27T11:12:57.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.515+0000 7f55f8e34700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f55d806a0e0 0x7f55d806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.516+0000 7f55f8e34700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f55d806a0e0 0x7f55d806c5a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f55f0001eb0 tx=0x7f55f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:57.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.519+0000 7f55e9ffb700 1 -- 172.21.15.45:0/1503709942 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f55e4052f20 con 0x7f55f406f620 2023-08-27T11:12:57.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.720+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f55f40726d0 con 0x7f55f406f620 2023-08-27T11:12:57.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.720+0000 7f55e9ffb700 1 -- 172.21.15.45:0/1503709942 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f55e4056540 con 0x7f55f406f620 2023-08-27T11:12:57.723 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:57.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.722+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f55d806a0e0 msgr2=0x7f55d806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.722+0000 7f55fb899700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f55d806a0e0 0x7f55d806c5a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f55f0001eb0 tx=0x7f55f0009040 comp rx=0 tx=0).stop 2023-08-27T11:12:57.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f406f620 msgr2=0x7f55f40b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f406f620 0x7f55f40b29c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55e400c490 tx=0x7f55e400c850 comp rx=0 tx=0).stop 2023-08-27T11:12:57.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 shutdown_connections 2023-08-27T11:12:57.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55f406f620 0x7f55f40b29c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f55d806a0e0 0x7f55d806c5a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55f415b5e0 0x7f55f4143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 --2- 172.21.15.45:0/1503709942 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55f415a700 0x7f55f41470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 >> 172.21.15.45:0/1503709942 conn(0x7f55f40cc7e0 msgr2=0x7f55f40af920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 shutdown_connections 2023-08-27T11:12:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.723+0000 7f55fb899700 1 -- 172.21.15.45:0/1503709942 wait complete. 2023-08-27T11:12:57.737 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:12:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.867+0000 7f4bbeeea700 1 Processor -- start 2023-08-27T11:12:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.867+0000 7f4bbeeea700 1 -- start start 2023-08-27T11:12:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.867+0000 7f4bbeeea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 0x7f4bb806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.867+0000 7f4bbeeea700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4bb806a7c0 0x7f4bb80b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.867+0000 7f4bbeeea700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb806b160 con 0x7f4bb815d340 2023-08-27T11:12:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.867+0000 7f4bbeeea700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb8072010 con 0x7f4bb806a7c0 2023-08-27T11:12:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.867+0000 7f4bbeeea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb806ac50 con 0x7f4bb8160d00 2023-08-27T11:12:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.868+0000 7f4bbd487700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4bb806a7c0 0x7f4bb80b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.868+0000 7f4bb7fff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 0x7f4bb806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.868+0000 7f4bbd487700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4bb806a7c0 0x7f4bb80b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41640/0 (socket says 172.21.15.45:41640) 2023-08-27T11:12:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.868+0000 7f4bbd487700 1 -- 172.21.15.45:0/3398888872 learned_addr learned my addr 172.21.15.45:0/3398888872 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.868+0000 7f4bbd487700 1 -- 172.21.15.45:0/3398888872 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 msgr2=0x7f4bb806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.868+0000 7f4bbd487700 1 --2- 172.21.15.45:0/3398888872 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 0x7f4bb806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.868+0000 7f4bbd487700 1 -- 172.21.15.45:0/3398888872 >> v1:172.21.15.135:6789/0 conn(0x7f4bb815d340 legacy=0x7f4bb80cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.868+0000 7f4bbd487700 1 -- 172.21.15.45:0/3398888872 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80cdf90 con 0x7f4bb806a7c0 2023-08-27T11:12:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.868+0000 7f4bbd487700 1 --2- 172.21.15.45:0/3398888872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb806a7c0 0x7f4bb80b18a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009e50 tx=0x7f4bb000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9a34f419adcfb6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bb77fe700 1 -- 172.21.15.45:0/3398888872 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000e070 con 0x7f4bb806a7c0 2023-08-27T11:12:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bb77fe700 1 -- 172.21.15.45:0/3398888872 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb000b910 con 0x7f4bb806a7c0 2023-08-27T11:12:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bb77fe700 1 -- 172.21.15.45:0/3398888872 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000bcc0 con 0x7f4bb806a7c0 2023-08-27T11:12:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bbeeea700 1 -- 172.21.15.45:0/3398888872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb806a7c0 msgr2=0x7f4bb80b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bbeeea700 1 --2- 172.21.15.45:0/3398888872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb806a7c0 0x7f4bb80b18a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009e50 tx=0x7f4bb000b330 comp rx=0 tx=0).stop 2023-08-27T11:12:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bbeeea700 1 -- 172.21.15.45:0/3398888872 shutdown_connections 2023-08-27T11:12:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bbeeea700 1 --2- 172.21.15.45:0/3398888872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb806a7c0 0x7f4bb80b18a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bbeeea700 1 --2- 172.21.15.45:0/3398888872 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 0x7f4bb806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bbeeea700 1 -- 172.21.15.45:0/3398888872 >> 172.21.15.45:0/3398888872 conn(0x7f4bb80cc7c0 msgr2=0x7f4bb80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bbeeea700 1 -- 172.21.15.45:0/3398888872 shutdown_connections 2023-08-27T11:12:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.869+0000 7f4bbeeea700 1 -- 172.21.15.45:0/3398888872 wait complete. 2023-08-27T11:12:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.870+0000 7f4bbeeea700 1 Processor -- start 2023-08-27T11:12:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.870+0000 7f4bbeeea700 1 -- start start 2023-08-27T11:12:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.871+0000 7f4bbeeea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bb806a7c0 0x7f4bb80ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.871+0000 7f4bbeeea700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815d340 0x7f4bb80c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.871+0000 7f4bbeeea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 0x7f4bb8148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.871+0000 7f4bbeeea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80adb30 con 0x7f4bb806a7c0 2023-08-27T11:12:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.871+0000 7f4bbeeea700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80acf20 con 0x7f4bb815d340 2023-08-27T11:12:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.871+0000 7f4bbeeea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80ad200 con 0x7f4bb8160d00 2023-08-27T11:12:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.871+0000 7f4bbcc86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bb806a7c0 0x7f4bb80ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.871+0000 7f4bbcc86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bb806a7c0 0x7f4bb80ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52786/0 (socket says 172.21.15.45:52786) 2023-08-27T11:12:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.871+0000 7f4bbcc86700 1 -- 172.21.15.45:0/451512231 learned_addr learned my addr 172.21.15.45:0/451512231 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bb7fff700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815d340 0x7f4bb80c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bbd487700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 0x7f4bb8148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bbcc86700 1 -- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 msgr2=0x7f4bb8148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bbcc86700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 0x7f4bb8148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bbcc86700 1 -- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815d340 msgr2=0x7f4bb80c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bbcc86700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815d340 0x7f4bb80c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bbcc86700 1 -- 172.21.15.45:0/451512231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0009b30 con 0x7f4bb806a7c0 2023-08-27T11:12:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bb7fff700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815d340 0x7f4bb80c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bbcc86700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bb806a7c0 0x7f4bb80ac8b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00c4e0 tx=0x7f4bac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bb57fa700 1 -- 172.21.15.45:0/451512231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0052d0 con 0x7f4bb806a7c0 2023-08-27T11:12:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bb57fa700 1 -- 172.21.15.45:0/451512231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac005430 con 0x7f4bb806a7c0 2023-08-27T11:12:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb80ad4e0 con 0x7f4bb806a7c0 2023-08-27T11:12:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bb57fa700 1 -- 172.21.15.45:0/451512231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0038c0 con 0x7f4bb806a7c0 2023-08-27T11:12:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.872+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb80c37a0 con 0x7f4bb806a7c0 2023-08-27T11:12:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.873+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb80cb310 con 0x7f4bb806a7c0 2023-08-27T11:12:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.873+0000 7f4bb57fa700 1 -- 172.21.15.45:0/451512231 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4bac003a20 con 0x7f4bb806a7c0 2023-08-27T11:12:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.874+0000 7f4bb57fa700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b9c06a0e0 0x7f4b9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.874+0000 7f4bb57fa700 1 -- 172.21.15.45:0/451512231 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4bac087cd0 con 0x7f4bb806a7c0 2023-08-27T11:12:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.874+0000 7f4bb7fff700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b9c06a0e0 0x7f4b9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:57.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.874+0000 7f4bb7fff700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b9c06a0e0 0x7f4b9c06c5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8156a30 tx=0x7f4ba4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:57.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:57.878+0000 7f4bb57fa700 1 -- 172.21.15.45:0/451512231 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4bac053120 con 0x7f4bb806a7c0 2023-08-27T11:12:58.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.079+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4bb8072690 con 0x7f4bb806a7c0 2023-08-27T11:12:58.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.080+0000 7f4bb57fa700 1 -- 172.21.15.45:0/451512231 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4bac017020 con 0x7f4bb806a7c0 2023-08-27T11:12:58.083 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:12:58.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b9c06a0e0 msgr2=0x7f4b9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b9c06a0e0 0x7f4b9c06c5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8156a30 tx=0x7f4ba4008040 comp rx=0 tx=0).stop 2023-08-27T11:12:58.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bb806a7c0 msgr2=0x7f4bb80ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bb806a7c0 0x7f4bb80ac8b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00c4e0 tx=0x7f4bac00c8a0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 shutdown_connections 2023-08-27T11:12:58.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bb806a7c0 0x7f4bb80ac8b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b9c06a0e0 0x7f4b9c06c5a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4bb8160d00 0x7f4bb8148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 --2- 172.21.15.45:0/451512231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb815d340 0x7f4bb80c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 >> 172.21.15.45:0/451512231 conn(0x7f4bb80cc7c0 msgr2=0x7f4bb815dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:58.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 shutdown_connections 2023-08-27T11:12:58.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.082+0000 7f4bbeeea700 1 -- 172.21.15.45:0/451512231 wait complete. 2023-08-27T11:12:58.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:12:58.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.223+0000 7f186567e700 1 Processor -- start 2023-08-27T11:12:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.223+0000 7f186567e700 1 -- start start 2023-08-27T11:12:58.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.224+0000 7f186567e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1860157be0 0x7f18600716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.224+0000 7f186567e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f186006d5e0 0x7f18600b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.224+0000 7f186567e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186015b460 con 0x7f18600b17c0 2023-08-27T11:12:58.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.224+0000 7f186567e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1860157fc0 con 0x7f1860157be0 2023-08-27T11:12:58.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.224+0000 7f186567e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600b2280 con 0x7f186006d5e0 2023-08-27T11:12:58.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185f7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f186006d5e0 0x7f18600b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185effd700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1860157be0 0x7f18600716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185f7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f186006d5e0 0x7f18600b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50708/0 (socket says 172.21.15.45:50708) 2023-08-27T11:12:58.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185effd700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1860157be0 0x7f18600716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41650/0 (socket says 172.21.15.45:41650) 2023-08-27T11:12:58.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185f7fe700 1 -- 172.21.15.45:0/2032178546 learned_addr learned my addr 172.21.15.45:0/2032178546 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:58.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185effd700 1 -- 172.21.15.45:0/2032178546 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f186006d5e0 msgr2=0x7f18600b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:58.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185effd700 1 --2- 172.21.15.45:0/2032178546 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f186006d5e0 0x7f18600b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185effd700 1 -- 172.21.15.45:0/2032178546 >> v1:172.21.15.135:6789/0 conn(0x7f18600b17c0 legacy=0x7f186006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185effd700 1 -- 172.21.15.45:0/2032178546 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600b2560 con 0x7f1860157be0 2023-08-27T11:12:58.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185f7fe700 1 --2- 172.21.15.45:0/2032178546 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f186006d5e0 0x7f18600b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:12:58.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185effd700 1 --2- 172.21.15.45:0/2032178546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1860157be0 0x7f18600716b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1854009840 tx=0x7f1854009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4286c54a326648 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:58.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185dffb700 1 -- 172.21.15.45:0/2032178546 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854011830 con 0x7f1860157be0 2023-08-27T11:12:58.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185dffb700 1 -- 172.21.15.45:0/2032178546 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1854011990 con 0x7f1860157be0 2023-08-27T11:12:58.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.225+0000 7f185dffb700 1 -- 172.21.15.45:0/2032178546 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854011d20 con 0x7f1860157be0 2023-08-27T11:12:58.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.226+0000 7f186567e700 1 -- 172.21.15.45:0/2032178546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1860157be0 msgr2=0x7f18600716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.226+0000 7f186567e700 1 --2- 172.21.15.45:0/2032178546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1860157be0 0x7f18600716b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1854009840 tx=0x7f1854009b50 comp rx=0 tx=0).stop 2023-08-27T11:12:58.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.226+0000 7f186567e700 1 -- 172.21.15.45:0/2032178546 shutdown_connections 2023-08-27T11:12:58.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.226+0000 7f186567e700 1 --2- 172.21.15.45:0/2032178546 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f186006d5e0 0x7f18600b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.226+0000 7f186567e700 1 --2- 172.21.15.45:0/2032178546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1860157be0 0x7f18600716b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.226+0000 7f186567e700 1 -- 172.21.15.45:0/2032178546 >> 172.21.15.45:0/2032178546 conn(0x7f18600cc7c0 msgr2=0x7f18600ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:58.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.226+0000 7f186567e700 1 -- 172.21.15.45:0/2032178546 shutdown_connections 2023-08-27T11:12:58.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.226+0000 7f186567e700 1 -- 172.21.15.45:0/2032178546 wait complete. 2023-08-27T11:12:58.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.228+0000 7f186567e700 1 Processor -- start 2023-08-27T11:12:58.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.228+0000 7f186567e700 1 -- start start 2023-08-27T11:12:58.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f186567e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f186006d5e0 0x7f18601f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f186567e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18600b17c0 0x7f18600ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f186567e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860157be0 0x7f1860159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f186567e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1860159200 con 0x7f1860157be0 2023-08-27T11:12:58.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f186567e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1860158610 con 0x7f186006d5e0 2023-08-27T11:12:58.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f186567e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1860158860 con 0x7f18600b17c0 2023-08-27T11:12:58.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860157be0 0x7f1860159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860157be0 0x7f1860159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52804/0 (socket says 172.21.15.45:52804) 2023-08-27T11:12:58.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185f7fe700 1 -- 172.21.15.45:0/1543883768 learned_addr learned my addr 172.21.15.45:0/1543883768 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:58.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185effd700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f186006d5e0 0x7f18601f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185e7fc700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18600b17c0 0x7f18600ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185f7fe700 1 -- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18600b17c0 msgr2=0x7f18600ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185f7fe700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18600b17c0 0x7f18600ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185f7fe700 1 -- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f186006d5e0 msgr2=0x7f18601f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185f7fe700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f186006d5e0 0x7f18601f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185f7fe700 1 -- 172.21.15.45:0/1543883768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1858009b30 con 0x7f1860157be0 2023-08-27T11:12:58.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f185f7fe700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860157be0 0x7f1860159620 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f185800efc0 tx=0x7f185800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:58.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.229+0000 7f18477fe700 1 -- 172.21.15.45:0/1543883768 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185800ca80 con 0x7f1860157be0 2023-08-27T11:12:58.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.230+0000 7f18477fe700 1 -- 172.21.15.45:0/1543883768 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185800f040 con 0x7f1860157be0 2023-08-27T11:12:58.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.230+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18540094a0 con 0x7f1860157be0 2023-08-27T11:12:58.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.230+0000 7f18477fe700 1 -- 172.21.15.45:0/1543883768 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185800ce40 con 0x7f1860157be0 2023-08-27T11:12:58.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.230+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1860158e20 con 0x7f1860157be0 2023-08-27T11:12:58.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.231+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1860159340 con 0x7f1860157be0 2023-08-27T11:12:58.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.231+0000 7f18477fe700 1 -- 172.21.15.45:0/1543883768 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1858014480 con 0x7f1860157be0 2023-08-27T11:12:58.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.231+0000 7f18477fe700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f184c06a0e0 0x7f184c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.232+0000 7f18477fe700 1 -- 172.21.15.45:0/1543883768 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1858088d40 con 0x7f1860157be0 2023-08-27T11:12:58.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.232+0000 7f185effd700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f184c06a0e0 0x7f184c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.232+0000 7f185effd700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f184c06a0e0 0x7f184c06c5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1854000c00 tx=0x7f1854008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:58.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.236+0000 7f18477fe700 1 -- 172.21.15.45:0/1543883768 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1860159340 con 0x7f1860157be0 2023-08-27T11:12:58.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.438+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1860159340 con 0x7f1860157be0 2023-08-27T11:12:58.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.439+0000 7f18477fe700 1 -- 172.21.15.45:0/1543883768 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1860159340 con 0x7f1860157be0 2023-08-27T11:12:58.441 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:58.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f184c06a0e0 msgr2=0x7f184c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f184c06a0e0 0x7f184c06c5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1854000c00 tx=0x7f1854008030 comp rx=0 tx=0).stop 2023-08-27T11:12:58.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860157be0 msgr2=0x7f1860159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860157be0 0x7f1860159620 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f185800efc0 tx=0x7f185800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 shutdown_connections 2023-08-27T11:12:58.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860157be0 0x7f1860159620 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f184c06a0e0 0x7f184c06c5a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f186006d5e0 0x7f18601f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 --2- 172.21.15.45:0/1543883768 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18600b17c0 0x7f18600ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 >> 172.21.15.45:0/1543883768 conn(0x7f18600cc7c0 msgr2=0x7f18600af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:58.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 shutdown_connections 2023-08-27T11:12:58.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.441+0000 7f186567e700 1 -- 172.21.15.45:0/1543883768 wait complete. 2023-08-27T11:12:58.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:12:58.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.581+0000 7f5173a53700 1 Processor -- start 2023-08-27T11:12:58.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.581+0000 7f5173a53700 1 -- start start 2023-08-27T11:12:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5173a53700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0c9b00 0x7f516c0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5173a53700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f516c0cdef0 0x7f516c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5173a53700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516c0cd350 con 0x7f516c1599e0 2023-08-27T11:12:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5173a53700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c06bdc0 con 0x7f516c0cdef0 2023-08-27T11:12:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5173a53700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0ce8c0 con 0x7f516c0c9b00 2023-08-27T11:12:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5171ff0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f516c0cdef0 0x7f516c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5171ff0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f516c0cdef0 0x7f516c0ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52812/0 (socket says 172.21.15.45:52812) 2023-08-27T11:12:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f51717ef700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f516c1599e0 0x7f516c0700b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39584/0 (socket says 172.21.15.45:39584) 2023-08-27T11:12:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5171ff0700 1 -- 172.21.15.45:0/2702524813 learned_addr learned my addr 172.21.15.45:0/2702524813 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5170fee700 1 --2- 172.21.15.45:0/2702524813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0c9b00 0x7f516c0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5171ff0700 1 -- 172.21.15.45:0/2702524813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0c9b00 msgr2=0x7f516c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5171ff0700 1 --2- 172.21.15.45:0/2702524813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0c9b00 0x7f516c0c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5171ff0700 1 -- 172.21.15.45:0/2702524813 >> v1:172.21.15.45:6789/0 conn(0x7f516c1599e0 legacy=0x7f516c0700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5171ff0700 1 -- 172.21.15.45:0/2702524813 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c0ca4a0 con 0x7f516c0cdef0 2023-08-27T11:12:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.582+0000 7f5171ff0700 1 --2- 172.21.15.45:0/2702524813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0cdef0 0x7f516c0ce380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5164009e50 tx=0x7f516400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61e3f715f38d76e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5163fff700 1 -- 172.21.15.45:0/2702524813 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516400e070 con 0x7f516c0cdef0 2023-08-27T11:12:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5163fff700 1 -- 172.21.15.45:0/2702524813 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516400b9c0 con 0x7f516c0cdef0 2023-08-27T11:12:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5163fff700 1 -- 172.21.15.45:0/2702524813 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516400bd70 con 0x7f516c0cdef0 2023-08-27T11:12:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5173a53700 1 -- 172.21.15.45:0/2702524813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0cdef0 msgr2=0x7f516c0ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5173a53700 1 --2- 172.21.15.45:0/2702524813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0cdef0 0x7f516c0ce380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5164009e50 tx=0x7f516400b330 comp rx=0 tx=0).stop 2023-08-27T11:12:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5173a53700 1 -- 172.21.15.45:0/2702524813 shutdown_connections 2023-08-27T11:12:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5173a53700 1 --2- 172.21.15.45:0/2702524813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0cdef0 0x7f516c0ce380 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5173a53700 1 --2- 172.21.15.45:0/2702524813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0c9b00 0x7f516c0c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5173a53700 1 -- 172.21.15.45:0/2702524813 >> 172.21.15.45:0/2702524813 conn(0x7f516c0b07c0 msgr2=0x7f516c0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5173a53700 1 -- 172.21.15.45:0/2702524813 shutdown_connections 2023-08-27T11:12:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.583+0000 7f5173a53700 1 -- 172.21.15.45:0/2702524813 wait complete. 2023-08-27T11:12:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.584+0000 7f5173a53700 1 Processor -- start 2023-08-27T11:12:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f5173a53700 1 -- start start 2023-08-27T11:12:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f5173a53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0c9b00 0x7f516c0c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f5173a53700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0cdef0 0x7f516c155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f5173a53700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f516c1599e0 0x7f516c158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f5173a53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0c8140 con 0x7f516c0c9b00 2023-08-27T11:12:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f5173a53700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1e2dc0 con 0x7f516c1599e0 2023-08-27T11:12:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f5173a53700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0c7290 con 0x7f516c0cdef0 2023-08-27T11:12:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f51717ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0c9b00 0x7f516c0c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f51717ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0c9b00 0x7f516c0c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52814/0 (socket says 172.21.15.45:52814) 2023-08-27T11:12:58.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.585+0000 7f51717ef700 1 -- 172.21.15.45:0/2579359586 learned_addr learned my addr 172.21.15.45:0/2579359586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:58.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f5170fee700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0cdef0 0x7f516c155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f5171ff0700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f516c1599e0 0x7f516c158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f51717ef700 1 -- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0cdef0 msgr2=0x7f516c155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f51717ef700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0cdef0 0x7f516c155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f51717ef700 1 -- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f516c1599e0 msgr2=0x7f516c158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f51717ef700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f516c1599e0 0x7f516c158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f51717ef700 1 -- 172.21.15.45:0/2579359586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164009b30 con 0x7f516c0c9b00 2023-08-27T11:12:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f5170fee700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0cdef0 0x7f516c155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f51717ef700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0c9b00 0x7f516c0c6d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f515c00c500 tx=0x7f515c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:58.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f5161ffb700 1 -- 172.21.15.45:0/2579359586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c005cb0 con 0x7f516c0c9b00 2023-08-27T11:12:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f5161ffb700 1 -- 172.21.15.45:0/2579359586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c005e10 con 0x7f516c0c9b00 2023-08-27T11:12:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c0c74c0 con 0x7f516c0c9b00 2023-08-27T11:12:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f5161ffb700 1 -- 172.21.15.45:0/2579359586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0072d0 con 0x7f516c0c9b00 2023-08-27T11:12:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.586+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c0c7a80 con 0x7f516c0c9b00 2023-08-27T11:12:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.587+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c04e620 con 0x7f516c0c9b00 2023-08-27T11:12:58.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.588+0000 7f5161ffb700 1 -- 172.21.15.45:0/2579359586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f515c007430 con 0x7f516c0c9b00 2023-08-27T11:12:58.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.588+0000 7f5161ffb700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f515006a0e0 0x7f515006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.588+0000 7f5161ffb700 1 -- 172.21.15.45:0/2579359586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f515c087b90 con 0x7f516c0c9b00 2023-08-27T11:12:58.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.589+0000 7f5170fee700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f515006a0e0 0x7f515006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.589+0000 7f5170fee700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f515006a0e0 0x7f515006c5a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f516c15c550 tx=0x7f5168008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:58.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.593+0000 7f5161ffb700 1 -- 172.21.15.45:0/2579359586 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f515c052f80 con 0x7f516c0c9b00 2023-08-27T11:12:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.793+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f516c071080 con 0x7f516c0c9b00 2023-08-27T11:12:58.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.794+0000 7f5161ffb700 1 -- 172.21.15.45:0/2579359586 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7f515c0565a0 con 0x7f516c0c9b00 2023-08-27T11:12:58.796 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:12:58.797 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:12:58.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.796+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f515006a0e0 msgr2=0x7f515006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.796+0000 7f5173a53700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f515006a0e0 0x7f515006c5a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f516c15c550 tx=0x7f5168008040 comp rx=0 tx=0).stop 2023-08-27T11:12:58.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.796+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0c9b00 msgr2=0x7f516c0c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.796+0000 7f5173a53700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0c9b00 0x7f516c0c6d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f515c00c500 tx=0x7f515c00c810 comp rx=0 tx=0).stop 2023-08-27T11:12:58.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.796+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 shutdown_connections 2023-08-27T11:12:58.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.796+0000 7f5173a53700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f516c0c9b00 0x7f516c0c6d50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.797+0000 7f5173a53700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f515006a0e0 0x7f515006c5a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.797+0000 7f5173a53700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f516c1599e0 0x7f516c158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.797+0000 7f5173a53700 1 --2- 172.21.15.45:0/2579359586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f516c0cdef0 0x7f516c155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.797+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 >> 172.21.15.45:0/2579359586 conn(0x7f516c0b07c0 msgr2=0x7f516c06cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.797+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 shutdown_connections 2023-08-27T11:12:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.797+0000 7f5173a53700 1 -- 172.21.15.45:0/2579359586 wait complete. 2023-08-27T11:12:58.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:12:58.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:12:58.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.938+0000 7f23b996e700 1 Processor -- start 2023-08-27T11:12:58.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.938+0000 7f23b996e700 1 -- start start 2023-08-27T11:12:58.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.938+0000 7f23b996e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f23b40b23c0 0x7f23b40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.938+0000 7f23b996e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23b40b1e70 0x7f23b40b6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.938+0000 7f23b996e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b40ce0d0 con 0x7f23b406a8b0 2023-08-27T11:12:58.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.938+0000 7f23b996e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40b9040 con 0x7f23b40b1e70 2023-08-27T11:12:58.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.938+0000 7f23b996e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40d2a10 con 0x7f23b40b23c0 2023-08-27T11:12:58.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23b40b1e70 0x7f23b40b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23b40b1e70 0x7f23b40b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52826/0 (socket says 172.21.15.45:52826) 2023-08-27T11:12:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b37fe700 1 -- 172.21.15.45:0/2365555648 learned_addr learned my addr 172.21.15.45:0/2365555648 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b27fc700 1 --2- 172.21.15.45:0/2365555648 >> v2:172.21.15.135:3300/0 conn(0x7f23b40b23c0 0x7f23b40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b37fe700 1 -- 172.21.15.45:0/2365555648 >> v2:172.21.15.135:3300/0 conn(0x7f23b40b23c0 msgr2=0x7f23b40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:58.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b37fe700 1 --2- 172.21.15.45:0/2365555648 >> v2:172.21.15.135:3300/0 conn(0x7f23b40b23c0 0x7f23b40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b37fe700 1 -- 172.21.15.45:0/2365555648 >> v1:172.21.15.135:6789/0 conn(0x7f23b406a8b0 legacy=0x7f23b406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:58.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b37fe700 1 -- 172.21.15.45:0/2365555648 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40cd8d0 con 0x7f23b40b1e70 2023-08-27T11:12:58.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b27fc700 1 --2- 172.21.15.45:0/2365555648 >> v2:172.21.15.135:3300/0 conn(0x7f23b40b23c0 0x7f23b40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:58.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b37fe700 1 --2- 172.21.15.45:0/2365555648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 0x7f23b40b6380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23b40733b0 tx=0x7f23ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f61367af103dae server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:58.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b1ffb700 1 -- 172.21.15.45:0/2365555648 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac00e070 con 0x7f23b40b1e70 2023-08-27T11:12:58.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b1ffb700 1 -- 172.21.15.45:0/2365555648 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac003a20 con 0x7f23b40b1e70 2023-08-27T11:12:58.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.939+0000 7f23b1ffb700 1 -- 172.21.15.45:0/2365555648 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac003dd0 con 0x7f23b40b1e70 2023-08-27T11:12:58.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.940+0000 7f23b996e700 1 -- 172.21.15.45:0/2365555648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 msgr2=0x7f23b40b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.940+0000 7f23b996e700 1 --2- 172.21.15.45:0/2365555648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 0x7f23b40b6380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23b40733b0 tx=0x7f23ac00b330 comp rx=0 tx=0).stop 2023-08-27T11:12:58.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.940+0000 7f23b996e700 1 -- 172.21.15.45:0/2365555648 shutdown_connections 2023-08-27T11:12:58.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.940+0000 7f23b996e700 1 --2- 172.21.15.45:0/2365555648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 0x7f23b40b6380 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.940+0000 7f23b996e700 1 --2- 172.21.15.45:0/2365555648 >> v2:172.21.15.135:3300/0 conn(0x7f23b40b23c0 0x7f23b40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.940+0000 7f23b996e700 1 -- 172.21.15.45:0/2365555648 >> 172.21.15.45:0/2365555648 conn(0x7f23b40d3070 msgr2=0x7f23b40d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:58.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.940+0000 7f23b996e700 1 -- 172.21.15.45:0/2365555648 shutdown_connections 2023-08-27T11:12:58.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.940+0000 7f23b996e700 1 -- 172.21.15.45:0/2365555648 wait complete. 2023-08-27T11:12:58.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.941+0000 7f23b996e700 1 Processor -- start 2023-08-27T11:12:58.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.942+0000 7f23b996e700 1 -- start start 2023-08-27T11:12:58.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.942+0000 7f23b996e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23b406a8b0 0x7f23b40b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b996e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 0x7f23b40ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b996e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23b40b23c0 0x7f23b4151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b996e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40caf50 con 0x7f23b40b1e70 2023-08-27T11:12:58.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b996e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b41f6550 con 0x7f23b40b23c0 2023-08-27T11:12:58.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b996e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b41f67f0 con 0x7f23b406a8b0 2023-08-27T11:12:58.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 0x7f23b40ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 0x7f23b40ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52830/0 (socket says 172.21.15.45:52830) 2023-08-27T11:12:58.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b27fc700 1 -- 172.21.15.45:0/2211123555 learned_addr learned my addr 172.21.15.45:0/2211123555 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:58.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b37fe700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23b40b23c0 0x7f23b4151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b2ffd700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23b406a8b0 0x7f23b40b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b27fc700 1 -- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23b406a8b0 msgr2=0x7f23b40b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b27fc700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23b406a8b0 0x7f23b40b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b27fc700 1 -- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23b40b23c0 msgr2=0x7f23b4151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:58.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b27fc700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23b40b23c0 0x7f23b4151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:58.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b27fc700 1 -- 172.21.15.45:0/2211123555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac009b30 con 0x7f23b40b1e70 2023-08-27T11:12:58.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b37fe700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23b40b23c0 0x7f23b4151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:58.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b27fc700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 0x7f23b40ca3b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c7f0 tx=0x7f23a800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:58.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f239bfff700 1 -- 172.21.15.45:0/2211123555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8011800 con 0x7f23b40b1e70 2023-08-27T11:12:58.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f239bfff700 1 -- 172.21.15.45:0/2211123555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8011960 con 0x7f23b40b1e70 2023-08-27T11:12:58.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b40b2c70 con 0x7f23b40b1e70 2023-08-27T11:12:58.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f239bfff700 1 -- 172.21.15.45:0/2211123555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8011d10 con 0x7f23b40b1e70 2023-08-27T11:12:58.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.943+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b41563b0 con 0x7f23b40b1e70 2023-08-27T11:12:58.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.944+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b41f6a10 con 0x7f23b40b1e70 2023-08-27T11:12:58.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.945+0000 7f239bfff700 1 -- 172.21.15.45:0/2211123555 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f23a8018360 con 0x7f23b40b1e70 2023-08-27T11:12:58.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.945+0000 7f239bfff700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23a006a0e0 0x7f23a006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:58.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.945+0000 7f239bfff700 1 -- 172.21.15.45:0/2211123555 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f23a8088870 con 0x7f23b40b1e70 2023-08-27T11:12:58.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.945+0000 7f23b2ffd700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23a006a0e0 0x7f23a006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:58.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.946+0000 7f23b2ffd700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23a006a0e0 0x7f23a006c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f23b40b4c70 tx=0x7f23a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:58.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:58.950+0000 7f239bfff700 1 -- 172.21.15.45:0/2211123555 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f23a80576b0 con 0x7f23b40b1e70 2023-08-27T11:12:59.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.151+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23b40d2ca0 con 0x7f23b40b1e70 2023-08-27T11:12:59.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.151+0000 7f239bfff700 1 -- 172.21.15.45:0/2211123555 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23a8057240 con 0x7f23b40b1e70 2023-08-27T11:12:59.154 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:12:59.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23a006a0e0 msgr2=0x7f23a006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23a006a0e0 0x7f23a006c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f23b40b4c70 tx=0x7f23a4009040 comp rx=0 tx=0).stop 2023-08-27T11:12:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 msgr2=0x7f23b40ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 0x7f23b40ca3b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c7f0 tx=0x7f23a800cbb0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 shutdown_connections 2023-08-27T11:12:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23a006a0e0 0x7f23a006c5a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23b40b23c0 0x7f23b4151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23b406a8b0 0x7f23b40b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 --2- 172.21.15.45:0/2211123555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23b40b1e70 0x7f23b40ca3b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 >> 172.21.15.45:0/2211123555 conn(0x7f23b40d3070 msgr2=0x7f23b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 shutdown_connections 2023-08-27T11:12:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.153+0000 7f23b996e700 1 -- 172.21.15.45:0/2211123555 wait complete. 2023-08-27T11:12:59.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:12:59.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.294+0000 7f376399c700 1 Processor -- start 2023-08-27T11:12:59.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.294+0000 7f376399c700 1 -- start start 2023-08-27T11:12:59.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f376399c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 0x7f375c0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f376399c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f375c0b17c0 0x7f375c0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f376399c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f375c06ced0 0x7f375c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f376399c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c157fc0 con 0x7f375c0b17c0 2023-08-27T11:12:59.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f376399c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c06d860 con 0x7f375c06ced0 2023-08-27T11:12:59.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f376399c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c15b460 con 0x7f375c157be0 2023-08-27T11:12:59.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3760f37700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f375c0b17c0 0x7f375c0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3760f37700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f375c0b17c0 0x7f375c0b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52838/0 (socket says 172.21.15.45:52838) 2023-08-27T11:12:59.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3760f37700 1 -- 172.21.15.45:0/3932830036 learned_addr learned my addr 172.21.15.45:0/3932830036 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:59.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3761738700 1 --2- 172.21.15.45:0/3932830036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 0x7f375c0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3761f39700 1 --2- 172.21.15.45:0/3932830036 >> v2:172.21.15.135:3300/0 conn(0x7f375c06ced0 0x7f375c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3760f37700 1 -- 172.21.15.45:0/3932830036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 msgr2=0x7f375c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3760f37700 1 --2- 172.21.15.45:0/3932830036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 0x7f375c0716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3760f37700 1 -- 172.21.15.45:0/3932830036 >> v2:172.21.15.135:3300/0 conn(0x7f375c06ced0 msgr2=0x7f375c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3760f37700 1 --2- 172.21.15.45:0/3932830036 >> v2:172.21.15.135:3300/0 conn(0x7f375c06ced0 0x7f375c06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3760f37700 1 -- 172.21.15.45:0/3932830036 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c0b2280 con 0x7f375c0b17c0 2023-08-27T11:12:59.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.295+0000 7f3761f39700 1 --2- 172.21.15.45:0/3932830036 >> v2:172.21.15.135:3300/0 conn(0x7f375c06ced0 0x7f375c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:12:59.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f3760f37700 1 --2- 172.21.15.45:0/3932830036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c0b17c0 0x7f375c0b1c40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f375c0cd3f0 tx=0x7f3758009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e76047b9269d547 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:59.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f3753fff700 1 -- 172.21.15.45:0/3932830036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375800b870 con 0x7f375c0b17c0 2023-08-27T11:12:59.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f3753fff700 1 -- 172.21.15.45:0/3932830036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375800b9d0 con 0x7f375c0b17c0 2023-08-27T11:12:59.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f3753fff700 1 -- 172.21.15.45:0/3932830036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375800bd60 con 0x7f375c0b17c0 2023-08-27T11:12:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f376399c700 1 -- 172.21.15.45:0/3932830036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c0b17c0 msgr2=0x7f375c0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f376399c700 1 --2- 172.21.15.45:0/3932830036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c0b17c0 0x7f375c0b1c40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f375c0cd3f0 tx=0x7f3758009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f376399c700 1 -- 172.21.15.45:0/3932830036 shutdown_connections 2023-08-27T11:12:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f376399c700 1 --2- 172.21.15.45:0/3932830036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 0x7f375c0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f376399c700 1 --2- 172.21.15.45:0/3932830036 >> v2:172.21.15.135:3300/0 conn(0x7f375c06ced0 0x7f375c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f376399c700 1 --2- 172.21.15.45:0/3932830036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c0b17c0 0x7f375c0b1c40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f376399c700 1 -- 172.21.15.45:0/3932830036 >> 172.21.15.45:0/3932830036 conn(0x7f375c0cc7c0 msgr2=0x7f375c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f376399c700 1 -- 172.21.15.45:0/3932830036 shutdown_connections 2023-08-27T11:12:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.296+0000 7f376399c700 1 -- 172.21.15.45:0/3932830036 wait complete. 2023-08-27T11:12:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.298+0000 7f376399c700 1 Processor -- start 2023-08-27T11:12:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.298+0000 7f376399c700 1 -- start start 2023-08-27T11:12:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f376399c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06ced0 0x7f375c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f376399c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0b17c0 0x7f375c0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f376399c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 0x7f375c1f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f376399c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c15b5a0 con 0x7f375c06ced0 2023-08-27T11:12:59.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f376399c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c15b880 con 0x7f375c0b17c0 2023-08-27T11:12:59.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f376399c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c15bb60 con 0x7f375c157be0 2023-08-27T11:12:59.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3761738700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06ced0 0x7f375c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3761738700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06ced0 0x7f375c0b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52842/0 (socket says 172.21.15.45:52842) 2023-08-27T11:12:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3761738700 1 -- 172.21.15.45:0/2530649672 learned_addr learned my addr 172.21.15.45:0/2530649672 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3760f37700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0b17c0 0x7f375c0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3761f39700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 0x7f375c1f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3761738700 1 -- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 msgr2=0x7f375c1f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3761738700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 0x7f375c1f3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3761738700 1 -- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0b17c0 msgr2=0x7f375c0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3761738700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0b17c0 0x7f375c0b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.299+0000 7f3761738700 1 -- 172.21.15.45:0/2530649672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3758009550 con 0x7f375c06ced0 2023-08-27T11:12:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.300+0000 7f3761738700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06ced0 0x7f375c0b2e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f374c00da80 tx=0x7f374c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.300+0000 7f3751ffb700 1 -- 172.21.15.45:0/2530649672 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c00aa90 con 0x7f375c06ced0 2023-08-27T11:12:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.300+0000 7f3751ffb700 1 -- 172.21.15.45:0/2530649672 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c00abf0 con 0x7f375c06ced0 2023-08-27T11:12:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.300+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c15be40 con 0x7f375c06ced0 2023-08-27T11:12:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.300+0000 7f3751ffb700 1 -- 172.21.15.45:0/2530649672 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c019480 con 0x7f375c06ced0 2023-08-27T11:12:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.300+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c0b3340 con 0x7f375c06ced0 2023-08-27T11:12:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.301+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c0b3530 con 0x7f375c06ced0 2023-08-27T11:12:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.301+0000 7f3751ffb700 1 -- 172.21.15.45:0/2530649672 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f374c0195e0 con 0x7f375c06ced0 2023-08-27T11:12:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.302+0000 7f3751ffb700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f374006a0e0 0x7f374006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.302+0000 7f3751ffb700 1 -- 172.21.15.45:0/2530649672 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f374c014070 con 0x7f375c06ced0 2023-08-27T11:12:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.302+0000 7f3760f37700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f374006a0e0 0x7f374006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.302+0000 7f3760f37700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f374006a0e0 0x7f374006c5a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3758000c00 tx=0x7f3758008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:59.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.306+0000 7f3751ffb700 1 -- 172.21.15.45:0/2530649672 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f374c0542c0 con 0x7f375c06ced0 2023-08-27T11:12:59.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.504+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f375c072690 con 0x7f375c06ced0 2023-08-27T11:12:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.505+0000 7f3751ffb700 1 -- 172.21.15.45:0/2530649672 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f374c0578e0 con 0x7f375c06ced0 2023-08-27T11:12:59.507 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f374006a0e0 msgr2=0x7f374006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f374006a0e0 0x7f374006c5a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3758000c00 tx=0x7f3758008030 comp rx=0 tx=0).stop 2023-08-27T11:12:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06ced0 msgr2=0x7f375c0b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06ced0 0x7f375c0b2e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f374c00da80 tx=0x7f374c00de40 comp rx=0 tx=0).stop 2023-08-27T11:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 shutdown_connections 2023-08-27T11:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f375c06ced0 0x7f375c0b2e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f374006a0e0 0x7f374006c5a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f375c157be0 0x7f375c1f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 --2- 172.21.15.45:0/2530649672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0b17c0 0x7f375c0b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 >> 172.21.15.45:0/2530649672 conn(0x7f375c0cc7c0 msgr2=0x7f375c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:59.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 shutdown_connections 2023-08-27T11:12:59.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.507+0000 7f376399c700 1 -- 172.21.15.45:0/2530649672 wait complete. 2023-08-27T11:12:59.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:12:59.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.647+0000 7fa7a86ac700 1 Processor -- start 2023-08-27T11:12:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.647+0000 7fa7a86ac700 1 -- start start 2023-08-27T11:12:59.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.647+0000 7fa7a86ac700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7a015d480 0x7fa7a00cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.647+0000 7fa7a86ac700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa7a015d860 0x7fa7a00cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a86ac700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a0071d70 con 0x7fa7a006ac90 2023-08-27T11:12:59.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a86ac700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a0072050 con 0x7fa7a015d480 2023-08-27T11:12:59.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a86ac700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a0072330 con 0x7fa7a015d860 2023-08-27T11:12:59.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a6448700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7a015d480 0x7fa7a00cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a6448700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7a015d480 0x7fa7a00cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52850/0 (socket says 172.21.15.45:52850) 2023-08-27T11:12:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a6448700 1 -- 172.21.15.45:0/4209383651 learned_addr learned my addr 172.21.15.45:0/4209383651 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a5c47700 1 --2- 172.21.15.45:0/4209383651 >> v2:172.21.15.135:3300/0 conn(0x7fa7a015d860 0x7fa7a00cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a6448700 1 -- 172.21.15.45:0/4209383651 >> v2:172.21.15.135:3300/0 conn(0x7fa7a015d860 msgr2=0x7fa7a00cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:59.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a6448700 1 --2- 172.21.15.45:0/4209383651 >> v2:172.21.15.135:3300/0 conn(0x7fa7a015d860 0x7fa7a00cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a6448700 1 -- 172.21.15.45:0/4209383651 >> v1:172.21.15.135:6789/0 conn(0x7fa7a006ac90 legacy=0x7fa7a00b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:12:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a6448700 1 -- 172.21.15.45:0/4209383651 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a0072610 con 0x7fa7a015d480 2023-08-27T11:12:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a5c47700 1 --2- 172.21.15.45:0/4209383651 >> v2:172.21.15.135:3300/0 conn(0x7fa7a015d860 0x7fa7a00cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a6448700 1 --2- 172.21.15.45:0/4209383651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a015d480 0x7fa7a00cbce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0071810 tx=0x7fa79c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508770e015ea99f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:59.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a5446700 1 -- 172.21.15.45:0/4209383651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c005e30 con 0x7fa7a015d480 2023-08-27T11:12:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a5446700 1 -- 172.21.15.45:0/4209383651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c006030 con 0x7fa7a015d480 2023-08-27T11:12:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.648+0000 7fa7a5446700 1 -- 172.21.15.45:0/4209383651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0063e0 con 0x7fa7a015d480 2023-08-27T11:12:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.649+0000 7fa7a86ac700 1 -- 172.21.15.45:0/4209383651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a015d480 msgr2=0x7fa7a00cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.649+0000 7fa7a86ac700 1 --2- 172.21.15.45:0/4209383651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a015d480 0x7fa7a00cbce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0071810 tx=0x7fa79c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.649+0000 7fa7a86ac700 1 -- 172.21.15.45:0/4209383651 shutdown_connections 2023-08-27T11:12:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.649+0000 7fa7a86ac700 1 --2- 172.21.15.45:0/4209383651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a015d480 0x7fa7a00cbce0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.649+0000 7fa7a86ac700 1 --2- 172.21.15.45:0/4209383651 >> v2:172.21.15.135:3300/0 conn(0x7fa7a015d860 0x7fa7a00cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.649+0000 7fa7a86ac700 1 -- 172.21.15.45:0/4209383651 >> 172.21.15.45:0/4209383651 conn(0x7fa7a00b2440 msgr2=0x7fa7a00b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.649+0000 7fa7a86ac700 1 -- 172.21.15.45:0/4209383651 shutdown_connections 2023-08-27T11:12:59.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.649+0000 7fa7a86ac700 1 -- 172.21.15.45:0/4209383651 wait complete. 2023-08-27T11:12:59.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.650+0000 7fa7a86ac700 1 Processor -- start 2023-08-27T11:12:59.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.650+0000 7fa7a86ac700 1 -- start start 2023-08-27T11:12:59.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a86ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a006ac90 0x7fa7a01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a86ac700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7a015d480 0x7fa7a00c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a86ac700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7a015d860 0x7fa7a00afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a86ac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00af710 con 0x7fa7a006ac90 2023-08-27T11:12:59.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a86ac700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a015afe0 con 0x7fa7a015d860 2023-08-27T11:12:59.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a86ac700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00af430 con 0x7fa7a015d480 2023-08-27T11:12:59.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6448700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a006ac90 0x7fa7a01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6448700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a006ac90 0x7fa7a01593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52858/0 (socket says 172.21.15.45:52858) 2023-08-27T11:12:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6448700 1 -- 172.21.15.45:0/476419261 learned_addr learned my addr 172.21.15.45:0/476419261 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:12:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6c49700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7a015d860 0x7fa7a00afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a5c47700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7a015d480 0x7fa7a00c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6448700 1 -- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7a015d480 msgr2=0x7fa7a00c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6448700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7a015d480 0x7fa7a00c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6448700 1 -- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7a015d860 msgr2=0x7fa7a00afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6448700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7a015d860 0x7fa7a00afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6448700 1 -- 172.21.15.45:0/476419261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c009500 con 0x7fa7a006ac90 2023-08-27T11:12:59.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a5c47700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7a015d480 0x7fa7a00c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:12:59.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.651+0000 7fa7a6448700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a006ac90 0x7fa7a01593b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa79c006000 tx=0x7fa79c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:59.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.652+0000 7fa796ffd700 1 -- 172.21.15.45:0/476419261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c006880 con 0x7fa7a006ac90 2023-08-27T11:12:59.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.652+0000 7fa796ffd700 1 -- 172.21.15.45:0/476419261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c020070 con 0x7fa7a006ac90 2023-08-27T11:12:59.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.652+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a015a990 con 0x7fa7a006ac90 2023-08-27T11:12:59.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.652+0000 7fa796ffd700 1 -- 172.21.15.45:0/476419261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c025670 con 0x7fa7a006ac90 2023-08-27T11:12:59.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.652+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a00c5d70 con 0x7fa7a006ac90 2023-08-27T11:12:59.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.653+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a009d4b0 con 0x7fa7a006ac90 2023-08-27T11:12:59.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.653+0000 7fa796ffd700 1 -- 172.21.15.45:0/476419261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa79c022030 con 0x7fa7a006ac90 2023-08-27T11:12:59.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.654+0000 7fa796ffd700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa78406a0e0 0x7fa78406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:12:59.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.654+0000 7fa796ffd700 1 -- 172.21.15.45:0/476419261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa79c089d50 con 0x7fa7a006ac90 2023-08-27T11:12:59.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.654+0000 7fa7a5c47700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa78406a0e0 0x7fa78406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:12:59.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.655+0000 7fa7a5c47700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa78406a0e0 0x7fa78406c5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00b1ff0 tx=0x7fa790008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:12:59.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.658+0000 7fa796ffd700 1 -- 172.21.15.45:0/476419261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa79c08d030 con 0x7fa7a006ac90 2023-08-27T11:12:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.859+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7a0073d30 con 0x7fa7a006ac90 2023-08-27T11:12:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.859+0000 7fa796ffd700 1 -- 172.21.15.45:0/476419261 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa79c058c30 con 0x7fa7a006ac90 2023-08-27T11:12:59.862 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:12:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa78406a0e0 msgr2=0x7fa78406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa78406a0e0 0x7fa78406c5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00b1ff0 tx=0x7fa790008040 comp rx=0 tx=0).stop 2023-08-27T11:12:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a006ac90 msgr2=0x7fa7a01593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:12:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a006ac90 0x7fa7a01593b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa79c006000 tx=0x7fa79c004750 comp rx=0 tx=0).stop 2023-08-27T11:12:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 shutdown_connections 2023-08-27T11:12:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7a006ac90 0x7fa7a01593b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa78406a0e0 0x7fa78406c5a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7a015d860 0x7fa7a00afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 --2- 172.21.15.45:0/476419261 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa7a015d480 0x7fa7a00c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:12:59.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 >> 172.21.15.45:0/476419261 conn(0x7fa7a00b2440 msgr2=0x7fa7a00ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 shutdown_connections 2023-08-27T11:12:59.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:12:59.861+0000 7fa7a86ac700 1 -- 172.21.15.45:0/476419261 wait complete. 2023-08-27T11:12:59.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:13:00.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.002+0000 7ffb74780700 1 Processor -- start 2023-08-27T11:13:00.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.002+0000 7ffb74780700 1 -- start start 2023-08-27T11:13:00.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.003+0000 7ffb74780700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb6c15b200 0x7ffb6c15f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.003+0000 7ffb74780700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c06cf40 0x7ffb6c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.003+0000 7ffb74780700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c15ecb0 con 0x7ffb6c0b5920 2023-08-27T11:13:00.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.003+0000 7ffb74780700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c15c5f0 con 0x7ffb6c15b200 2023-08-27T11:13:00.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.003+0000 7ffb74780700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c06d4b0 con 0x7ffb6c06cf40 2023-08-27T11:13:00.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.003+0000 7ffb7251c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb6c15b200 0x7ffb6c15f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.003+0000 7ffb7251c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb6c15b200 0x7ffb6c15f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52864/0 (socket says 172.21.15.45:52864) 2023-08-27T11:13:00.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.003+0000 7ffb7251c700 1 -- 172.21.15.45:0/1235806982 learned_addr learned my addr 172.21.15.45:0/1235806982 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.003+0000 7ffb71d1b700 1 --2- 172.21.15.45:0/1235806982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c06cf40 0x7ffb6c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb7251c700 1 -- 172.21.15.45:0/1235806982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c06cf40 msgr2=0x7ffb6c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb7251c700 1 --2- 172.21.15.45:0/1235806982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c06cf40 0x7ffb6c06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb7251c700 1 -- 172.21.15.45:0/1235806982 >> v1:172.21.15.135:6789/0 conn(0x7ffb6c0b5920 legacy=0x7ffb6c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb7251c700 1 -- 172.21.15.45:0/1235806982 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c06d790 con 0x7ffb6c15b200 2023-08-27T11:13:00.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb7251c700 1 --2- 172.21.15.45:0/1235806982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c15b200 0x7ffb6c15f700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0098f0 tx=0x7ffb5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36edd74615e9c7e0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:00.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb7151a700 1 -- 172.21.15.45:0/1235806982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00b8a0 con 0x7ffb6c15b200 2023-08-27T11:13:00.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb7151a700 1 -- 172.21.15.45:0/1235806982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5c00ba00 con 0x7ffb6c15b200 2023-08-27T11:13:00.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb7151a700 1 -- 172.21.15.45:0/1235806982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00bd90 con 0x7ffb6c15b200 2023-08-27T11:13:00.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb74780700 1 -- 172.21.15.45:0/1235806982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c15b200 msgr2=0x7ffb6c15f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb74780700 1 --2- 172.21.15.45:0/1235806982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c15b200 0x7ffb6c15f700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0098f0 tx=0x7ffb5c009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:00.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb74780700 1 -- 172.21.15.45:0/1235806982 shutdown_connections 2023-08-27T11:13:00.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb74780700 1 --2- 172.21.15.45:0/1235806982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c15b200 0x7ffb6c15f700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb74780700 1 --2- 172.21.15.45:0/1235806982 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c06cf40 0x7ffb6c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.004+0000 7ffb74780700 1 -- 172.21.15.45:0/1235806982 >> 172.21.15.45:0/1235806982 conn(0x7ffb6c0cc7e0 msgr2=0x7ffb6c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:00.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.005+0000 7ffb74780700 1 -- 172.21.15.45:0/1235806982 shutdown_connections 2023-08-27T11:13:00.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.005+0000 7ffb74780700 1 -- 172.21.15.45:0/1235806982 wait complete. 2023-08-27T11:13:00.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.006+0000 7ffb74780700 1 Processor -- start 2023-08-27T11:13:00.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.006+0000 7ffb74780700 1 -- start start 2023-08-27T11:13:00.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.006+0000 7ffb74780700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c06cf40 0x7ffb6c14b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.006+0000 7ffb74780700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c0b5920 0x7ffb6c0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.006+0000 7ffb74780700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c15b200 0x7ffb6c0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.006+0000 7ffb74780700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0cb0f0 con 0x7ffb6c0b5920 2023-08-27T11:13:00.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.006+0000 7ffb74780700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0cb3d0 con 0x7ffb6c06cf40 2023-08-27T11:13:00.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.006+0000 7ffb74780700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0cb6b0 con 0x7ffb6c15b200 2023-08-27T11:13:00.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb71d1b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c0b5920 0x7ffb6c0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb71d1b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c0b5920 0x7ffb6c0ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52872/0 (socket says 172.21.15.45:52872) 2023-08-27T11:13:00.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb71d1b700 1 -- 172.21.15.45:0/1682127751 learned_addr learned my addr 172.21.15.45:0/1682127751 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:00.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb7251c700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c06cf40 0x7ffb6c14b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb72d1d700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c15b200 0x7ffb6c0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb71d1b700 1 -- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c15b200 msgr2=0x7ffb6c0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb71d1b700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c15b200 0x7ffb6c0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb71d1b700 1 -- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c06cf40 msgr2=0x7ffb6c14b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb71d1b700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c06cf40 0x7ffb6c14b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb71d1b700 1 -- 172.21.15.45:0/1682127751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c009550 con 0x7ffb6c0b5920 2023-08-27T11:13:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb71d1b700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c0b5920 0x7ffb6c0ca570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800d800 tx=0x7ffb6800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb62ffd700 1 -- 172.21.15.45:0/1682127751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb680125f0 con 0x7ffb6c0b5920 2023-08-27T11:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb62ffd700 1 -- 172.21.15.45:0/1682127751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb68012750 con 0x7ffb6c0b5920 2023-08-27T11:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c0cb990 con 0x7ffb6c0b5920 2023-08-27T11:13:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb62ffd700 1 -- 172.21.15.45:0/1682127751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68012b00 con 0x7ffb6c0b5920 2023-08-27T11:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.007+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c158ce0 con 0x7ffb6c0b5920 2023-08-27T11:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.008+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c158e70 con 0x7ffb6c0b5920 2023-08-27T11:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.009+0000 7ffb62ffd700 1 -- 172.21.15.45:0/1682127751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffb68012c60 con 0x7ffb6c0b5920 2023-08-27T11:13:00.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.009+0000 7ffb62ffd700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb5006a0e0 0x7ffb5006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.009+0000 7ffb62ffd700 1 -- 172.21.15.45:0/1682127751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ffb68088ba0 con 0x7ffb6c0b5920 2023-08-27T11:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.009+0000 7ffb7251c700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb5006a0e0 0x7ffb5006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.010+0000 7ffb7251c700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb5006a0e0 0x7ffb5006c5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009c40 tx=0x7ffb5c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:00.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.013+0000 7ffb62ffd700 1 -- 172.21.15.45:0/1682127751 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffb68053f60 con 0x7ffb6c0b5920 2023-08-27T11:13:00.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.215+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb6c0c3610 con 0x7ffb6c0b5920 2023-08-27T11:13:00.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.216+0000 7ffb62ffd700 1 -- 172.21.15.45:0/1682127751 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb68057580 con 0x7ffb6c0b5920 2023-08-27T11:13:00.218 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb5006a0e0 msgr2=0x7ffb5006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb5006a0e0 0x7ffb5006c5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009c40 tx=0x7ffb5c008030 comp rx=0 tx=0).stop 2023-08-27T11:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c0b5920 msgr2=0x7ffb6c0ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c0b5920 0x7ffb6c0ca570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800d800 tx=0x7ffb6800db10 comp rx=0 tx=0).stop 2023-08-27T11:13:00.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 shutdown_connections 2023-08-27T11:13:00.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ffb5006a0e0 0x7ffb5006c5a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffb6c15b200 0x7ffb6c0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb6c06cf40 0x7ffb6c14b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 --2- 172.21.15.45:0/1682127751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6c0b5920 0x7ffb6c0ca570 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 >> 172.21.15.45:0/1682127751 conn(0x7ffb6c0cc7e0 msgr2=0x7ffb6c0ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:00.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 shutdown_connections 2023-08-27T11:13:00.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.218+0000 7ffb74780700 1 -- 172.21.15.45:0/1682127751 wait complete. 2023-08-27T11:13:00.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:13:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.359+0000 7f29e2174700 1 Processor -- start 2023-08-27T11:13:00.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.359+0000 7f29e2174700 1 -- start start 2023-08-27T11:13:00.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29e2174700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc0b17c0 0x7f29dc0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29e2174700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f29dc06ced0 0x7f29dc06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29e2174700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc06d860 con 0x7f29dc157be0 2023-08-27T11:13:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29e2174700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29dc157fc0 con 0x7f29dc06ced0 2023-08-27T11:13:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29e2174700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc071dc0 con 0x7f29dc0b17c0 2023-08-27T11:13:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29db7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f29dc157be0 0x7f29dc0716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39654/0 (socket says 172.21.15.45:39654) 2023-08-27T11:13:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29db7fe700 1 -- 172.21.15.45:0/1345341708 learned_addr learned my addr 172.21.15.45:0/1345341708 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29daffd700 1 --2- 172.21.15.45:0/1345341708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc0b17c0 0x7f29dc0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29dbfff700 1 --2- 172.21.15.45:0/1345341708 >> v2:172.21.15.135:3300/0 conn(0x7f29dc06ced0 0x7f29dc06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29da7fc700 1 -- 172.21.15.45:0/1345341708 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457253236 0 0) 0x7f29cc001110 con 0x7f29dc157be0 2023-08-27T11:13:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.360+0000 7f29da7fc700 1 -- 172.21.15.45:0/1345341708 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c80035b0 con 0x7f29dc157be0 2023-08-27T11:13:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29da7fc700 1 -- 172.21.15.45:0/1345341708 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3703394070 0 0) 0x7f29c80035b0 con 0x7f29dc157be0 2023-08-27T11:13:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29daffd700 1 -- 172.21.15.45:0/1345341708 >> v2:172.21.15.135:3300/0 conn(0x7f29dc06ced0 msgr2=0x7f29dc06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29daffd700 1 --2- 172.21.15.45:0/1345341708 >> v2:172.21.15.135:3300/0 conn(0x7f29dc06ced0 0x7f29dc06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29daffd700 1 -- 172.21.15.45:0/1345341708 >> v1:172.21.15.45:6789/0 conn(0x7f29dc157be0 legacy=0x7f29dc0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29daffd700 1 -- 172.21.15.45:0/1345341708 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc15b460 con 0x7f29dc0b17c0 2023-08-27T11:13:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29dbfff700 1 --2- 172.21.15.45:0/1345341708 >> v2:172.21.15.135:3300/0 conn(0x7f29dc06ced0 0x7f29dc06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29daffd700 1 --2- 172.21.15.45:0/1345341708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc0b17c0 0x7f29dc0b1c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009840 tx=0x7f29c4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d0e8fd63a088df server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29da7fc700 1 -- 172.21.15.45:0/1345341708 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc001800 con 0x7f29dc0b17c0 2023-08-27T11:13:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29da7fc700 1 -- 172.21.15.45:0/1345341708 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c4004030 con 0x7f29dc0b17c0 2023-08-27T11:13:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.361+0000 7f29da7fc700 1 -- 172.21.15.45:0/1345341708 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c40043c0 con 0x7f29dc0b17c0 2023-08-27T11:13:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.362+0000 7f29e2174700 1 -- 172.21.15.45:0/1345341708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc0b17c0 msgr2=0x7f29dc0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.362+0000 7f29e2174700 1 --2- 172.21.15.45:0/1345341708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc0b17c0 0x7f29dc0b1c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009840 tx=0x7f29c4009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.362+0000 7f29e2174700 1 -- 172.21.15.45:0/1345341708 shutdown_connections 2023-08-27T11:13:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.362+0000 7f29e2174700 1 --2- 172.21.15.45:0/1345341708 >> v2:172.21.15.135:3300/0 conn(0x7f29dc06ced0 0x7f29dc06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.362+0000 7f29e2174700 1 --2- 172.21.15.45:0/1345341708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc0b17c0 0x7f29dc0b1c40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.362+0000 7f29e2174700 1 -- 172.21.15.45:0/1345341708 >> 172.21.15.45:0/1345341708 conn(0x7f29dc0cc7c0 msgr2=0x7f29dc0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.362+0000 7f29e2174700 1 -- 172.21.15.45:0/1345341708 shutdown_connections 2023-08-27T11:13:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.362+0000 7f29e2174700 1 -- 172.21.15.45:0/1345341708 wait complete. 2023-08-27T11:13:00.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.363+0000 7f29e2174700 1 Processor -- start 2023-08-27T11:13:00.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.363+0000 7f29e2174700 1 -- start start 2023-08-27T11:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29e2174700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc06ced0 0x7f29dc156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29e2174700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29dc0b17c0 0x7f29dc156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29e2174700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29dc157be0 0x7f29dc14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29e2174700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc159ff0 con 0x7f29dc157be0 2023-08-27T11:13:00.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29e2174700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0c4990 con 0x7f29dc0b17c0 2023-08-27T11:13:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29e2174700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0c4c70 con 0x7f29dc06ced0 2023-08-27T11:13:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29dbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29dc157be0 0x7f29dc14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29dbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29dc157be0 0x7f29dc14da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52888/0 (socket says 172.21.15.45:52888) 2023-08-27T11:13:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29dbfff700 1 -- 172.21.15.45:0/2485243745 learned_addr learned my addr 172.21.15.45:0/2485243745 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29daffd700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29dc0b17c0 0x7f29dc156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29db7fe700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc06ced0 0x7f29dc156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29dbfff700 1 -- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc06ced0 msgr2=0x7f29dc156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29dbfff700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc06ced0 0x7f29dc156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29dbfff700 1 -- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29dc0b17c0 msgr2=0x7f29dc156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29dbfff700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29dc0b17c0 0x7f29dc156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29dbfff700 1 -- 172.21.15.45:0/2485243745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4009a70 con 0x7f29dc157be0 2023-08-27T11:13:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29db7fe700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc06ced0 0x7f29dc156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.364+0000 7f29dbfff700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29dc157be0 0x7f29dc14da90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f29d400efc0 tx=0x7f29d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.365+0000 7f29d3fff700 1 -- 172.21.15.45:0/2485243745 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4009cd0 con 0x7f29dc157be0 2023-08-27T11:13:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.365+0000 7f29d3fff700 1 -- 172.21.15.45:0/2485243745 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d400f040 con 0x7f29dc157be0 2023-08-27T11:13:00.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.365+0000 7f29d3fff700 1 -- 172.21.15.45:0/2485243745 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d400cd60 con 0x7f29dc157be0 2023-08-27T11:13:00.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.365+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c40094a0 con 0x7f29dc157be0 2023-08-27T11:13:00.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.365+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc0c5230 con 0x7f29dc157be0 2023-08-27T11:13:00.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.366+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc0c6ba0 con 0x7f29dc157be0 2023-08-27T11:13:00.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.366+0000 7f29d3fff700 1 -- 172.21.15.45:0/2485243745 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f29d4014480 con 0x7f29dc157be0 2023-08-27T11:13:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.366+0000 7f29d3fff700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f29bc06a0e0 0x7f29bc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.366+0000 7f29d3fff700 1 -- 172.21.15.45:0/2485243745 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f29d4088f90 con 0x7f29dc157be0 2023-08-27T11:13:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.367+0000 7f29db7fe700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f29bc06a0e0 0x7f29bc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.367+0000 7f29db7fe700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f29bc06a0e0 0x7f29bc06c5a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000f60 tx=0x7f29cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:00.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.371+0000 7f29d3fff700 1 -- 172.21.15.45:0/2485243745 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29d40bb990 con 0x7f29dc157be0 2023-08-27T11:13:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.569+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f29dc0c5450 con 0x7f29dc157be0 2023-08-27T11:13:00.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.570+0000 7f29d3fff700 1 -- 172.21.15.45:0/2485243745 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f29d4057de0 con 0x7f29dc157be0 2023-08-27T11:13:00.572 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f29bc06a0e0 msgr2=0x7f29bc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f29bc06a0e0 0x7f29bc06c5a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000f60 tx=0x7f29cc009040 comp rx=0 tx=0).stop 2023-08-27T11:13:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29dc157be0 msgr2=0x7f29dc14da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29dc157be0 0x7f29dc14da90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f29d400efc0 tx=0x7f29d400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 shutdown_connections 2023-08-27T11:13:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29dc157be0 0x7f29dc14da90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f29bc06a0e0 0x7f29bc06c5a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29dc06ced0 0x7f29dc156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 --2- 172.21.15.45:0/2485243745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29dc0b17c0 0x7f29dc156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 >> 172.21.15.45:0/2485243745 conn(0x7f29dc0cc7c0 msgr2=0x7f29dc0b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 shutdown_connections 2023-08-27T11:13:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.572+0000 7f29e2174700 1 -- 172.21.15.45:0/2485243745 wait complete. 2023-08-27T11:13:00.586 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:00.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.708+0000 7fa05a447700 1 Processor -- start 2023-08-27T11:13:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.708+0000 7fa05a447700 1 -- start start 2023-08-27T11:13:00.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.709+0000 7fa05a447700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa05415dbc0 0x7fa05415a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.709+0000 7fa05a447700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa05415a750 0x7fa0540cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.709+0000 7fa05a447700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540caef0 con 0x7fa054070e40 2023-08-27T11:13:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.709+0000 7fa05a447700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0540cac50 con 0x7fa05415dbc0 2023-08-27T11:13:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.709+0000 7fa05a447700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0540ce000 con 0x7fa05415a750 2023-08-27T11:13:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.709+0000 7fa0537fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa05415dbc0 0x7fa05415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.709+0000 7fa0537fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa05415dbc0 0x7fa05415a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52894/0 (socket says 172.21.15.45:52894) 2023-08-27T11:13:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.709+0000 7fa0537fe700 1 -- 172.21.15.45:0/2710383581 learned_addr learned my addr 172.21.15.45:0/2710383581 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.710+0000 7fa0589e4700 1 --2- 172.21.15.45:0/2710383581 >> v2:172.21.15.135:3300/0 conn(0x7fa05415a750 0x7fa0540cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.710+0000 7fa0537fe700 1 -- 172.21.15.45:0/2710383581 >> v2:172.21.15.135:3300/0 conn(0x7fa05415a750 msgr2=0x7fa0540cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.710+0000 7fa0537fe700 1 --2- 172.21.15.45:0/2710383581 >> v2:172.21.15.135:3300/0 conn(0x7fa05415a750 0x7fa0540cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.710+0000 7fa0537fe700 1 -- 172.21.15.45:0/2710383581 >> v1:172.21.15.135:6789/0 conn(0x7fa054070e40 legacy=0x7fa054071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.710+0000 7fa0537fe700 1 -- 172.21.15.45:0/2710383581 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05415b5c0 con 0x7fa05415dbc0 2023-08-27T11:13:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.710+0000 7fa0537fe700 1 --2- 172.21.15.45:0/2710383581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415dbc0 0x7fa05415a210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098f0 tx=0x7fa048009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c64600ad22b2c05 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.710+0000 7fa052ffd700 1 -- 172.21.15.45:0/2710383581 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048011830 con 0x7fa05415dbc0 2023-08-27T11:13:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.710+0000 7fa052ffd700 1 -- 172.21.15.45:0/2710383581 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa048011990 con 0x7fa05415dbc0 2023-08-27T11:13:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.710+0000 7fa052ffd700 1 -- 172.21.15.45:0/2710383581 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048011d20 con 0x7fa05415dbc0 2023-08-27T11:13:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.711+0000 7fa05a447700 1 -- 172.21.15.45:0/2710383581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415dbc0 msgr2=0x7fa05415a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.711+0000 7fa05a447700 1 --2- 172.21.15.45:0/2710383581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415dbc0 0x7fa05415a210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098f0 tx=0x7fa048009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.711+0000 7fa05a447700 1 -- 172.21.15.45:0/2710383581 shutdown_connections 2023-08-27T11:13:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.711+0000 7fa05a447700 1 --2- 172.21.15.45:0/2710383581 >> v2:172.21.15.135:3300/0 conn(0x7fa05415a750 0x7fa0540cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.711+0000 7fa05a447700 1 --2- 172.21.15.45:0/2710383581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415dbc0 0x7fa05415a210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.711+0000 7fa05a447700 1 -- 172.21.15.45:0/2710383581 >> 172.21.15.45:0/2710383581 conn(0x7fa0540cfd60 msgr2=0x7fa0540d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.711+0000 7fa05a447700 1 -- 172.21.15.45:0/2710383581 shutdown_connections 2023-08-27T11:13:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.711+0000 7fa05a447700 1 -- 172.21.15.45:0/2710383581 wait complete. 2023-08-27T11:13:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.712+0000 7fa05a447700 1 Processor -- start 2023-08-27T11:13:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.713+0000 7fa05a447700 1 -- start start 2023-08-27T11:13:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.713+0000 7fa05a447700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa054070e40 0x7fa0540add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa05a447700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415a750 0x7fa0540ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa05a447700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa05415dbc0 0x7fa05406a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa05a447700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540b4630 con 0x7fa05415a750 2023-08-27T11:13:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa05a447700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540aef80 con 0x7fa05415dbc0 2023-08-27T11:13:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa05a447700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540ae300 con 0x7fa054070e40 2023-08-27T11:13:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415a750 0x7fa0540ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415a750 0x7fa0540ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52900/0 (socket says 172.21.15.45:52900) 2023-08-27T11:13:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0537fe700 1 -- 172.21.15.45:0/1686126474 learned_addr learned my addr 172.21.15.45:0/1686126474 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0589e4700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa05415dbc0 0x7fa05406a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa053fff700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa054070e40 0x7fa0540add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0537fe700 1 -- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa054070e40 msgr2=0x7fa0540add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0537fe700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa054070e40 0x7fa0540add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0537fe700 1 -- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa05415dbc0 msgr2=0x7fa05406a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0537fe700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa05415dbc0 0x7fa05406a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0537fe700 1 -- 172.21.15.45:0/1686126474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048009550 con 0x7fa05415a750 2023-08-27T11:13:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa053fff700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa054070e40 0x7fa0540add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa0537fe700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415a750 0x7fa0540ad3a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa048013000 tx=0x7fa048004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa050ff9700 1 -- 172.21.15.45:0/1686126474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048004590 con 0x7fa05415a750 2023-08-27T11:13:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa050ff9700 1 -- 172.21.15.45:0/1686126474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0480036a0 con 0x7fa05415a750 2023-08-27T11:13:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa050ff9700 1 -- 172.21.15.45:0/1686126474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048003a10 con 0x7fa05415a750 2023-08-27T11:13:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0540ae5e0 con 0x7fa05415a750 2023-08-27T11:13:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0540ad8e0 con 0x7fa05415a750 2023-08-27T11:13:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.715+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa054072180 con 0x7fa05415a750 2023-08-27T11:13:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.716+0000 7fa050ff9700 1 -- 172.21.15.45:0/1686126474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa048003b70 con 0x7fa05415a750 2023-08-27T11:13:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.716+0000 7fa050ff9700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa04006a0e0 0x7fa04006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.716+0000 7fa050ff9700 1 -- 172.21.15.45:0/1686126474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa048089640 con 0x7fa05415a750 2023-08-27T11:13:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.716+0000 7fa053fff700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa04006a0e0 0x7fa04006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.717+0000 7fa053fff700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa04006a0e0 0x7fa04006c5a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa044000f60 tx=0x7fa044009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.721+0000 7fa050ff9700 1 -- 172.21.15.45:0/1686126474 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa048054b40 con 0x7fa05415a750 2023-08-27T11:13:00.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.922+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0540aea00 con 0x7fa05415a750 2023-08-27T11:13:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.923+0000 7fa050ff9700 1 -- 172.21.15.45:0/1686126474 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7fa0540aea00 con 0x7fa05415a750 2023-08-27T11:13:00.925 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:00.925 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa04006a0e0 msgr2=0x7fa04006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa04006a0e0 0x7fa04006c5a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa044000f60 tx=0x7fa044009040 comp rx=0 tx=0).stop 2023-08-27T11:13:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415a750 msgr2=0x7fa0540ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415a750 0x7fa0540ad3a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa048013000 tx=0x7fa048004750 comp rx=0 tx=0).stop 2023-08-27T11:13:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 shutdown_connections 2023-08-27T11:13:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa04006a0e0 0x7fa04006c5a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa05415dbc0 0x7fa05406a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa054070e40 0x7fa0540add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 --2- 172.21.15.45:0/1686126474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa05415a750 0x7fa0540ad3a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 >> 172.21.15.45:0/1686126474 conn(0x7fa0540cfd60 msgr2=0x7fa05406ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 shutdown_connections 2023-08-27T11:13:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:00.925+0000 7fa05a447700 1 -- 172.21.15.45:0/1686126474 wait complete. 2023-08-27T11:13:00.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:13:00.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:13:01.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.068+0000 7fb171192700 1 Processor -- start 2023-08-27T11:13:01.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.068+0000 7fb171192700 1 -- start start 2023-08-27T11:13:01.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb171192700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 0x7fb16c160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb171192700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb16c0aed60 0x7fb16c161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb171192700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16c071de0 con 0x7fb16c0af420 2023-08-27T11:13:01.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb171192700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16c0c6ca0 con 0x7fb16c0aed60 2023-08-27T11:13:01.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb171192700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c15d7d0 con 0x7fb16c15d3f0 2023-08-27T11:13:01.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb16a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb16c0aed60 0x7fb16c161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb16b59e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb16c0af420 0x7fb16c06a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39686/0 (socket says 172.21.15.45:39686) 2023-08-27T11:13:01.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb16a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb16c0aed60 0x7fb16c161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52908/0 (socket says 172.21.15.45:52908) 2023-08-27T11:13:01.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb16b59e700 1 -- 172.21.15.45:0/695856170 learned_addr learned my addr 172.21.15.45:0/695856170 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:01.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.069+0000 7fb16ad9d700 1 --2- 172.21.15.45:0/695856170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 0x7fb16c160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb16a59c700 1 -- 172.21.15.45:0/695856170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 msgr2=0x7fb16c160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:01.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb16a59c700 1 --2- 172.21.15.45:0/695856170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 0x7fb16c160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb16a59c700 1 -- 172.21.15.45:0/695856170 >> v1:172.21.15.45:6789/0 conn(0x7fb16c0af420 legacy=0x7fb16c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb16a59c700 1 -- 172.21.15.45:0/695856170 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c0c5310 con 0x7fb16c0aed60 2023-08-27T11:13:01.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb16a59c700 1 --2- 172.21.15.45:0/695856170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 0x7fb16c161390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb160009840 tx=0x7fb160009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e04343f542991e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:01.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb169d9b700 1 -- 172.21.15.45:0/695856170 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160011830 con 0x7fb16c0aed60 2023-08-27T11:13:01.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb169d9b700 1 -- 172.21.15.45:0/695856170 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb160011990 con 0x7fb16c0aed60 2023-08-27T11:13:01.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb169d9b700 1 -- 172.21.15.45:0/695856170 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160011d20 con 0x7fb16c0aed60 2023-08-27T11:13:01.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb171192700 1 -- 172.21.15.45:0/695856170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 msgr2=0x7fb16c161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb171192700 1 --2- 172.21.15.45:0/695856170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 0x7fb16c161390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb160009840 tx=0x7fb160009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:01.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb171192700 1 -- 172.21.15.45:0/695856170 shutdown_connections 2023-08-27T11:13:01.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb171192700 1 --2- 172.21.15.45:0/695856170 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 0x7fb16c160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb171192700 1 --2- 172.21.15.45:0/695856170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 0x7fb16c161390 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.070+0000 7fb171192700 1 -- 172.21.15.45:0/695856170 >> 172.21.15.45:0/695856170 conn(0x7fb16c0cfdd0 msgr2=0x7fb16c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:01.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.071+0000 7fb171192700 1 -- 172.21.15.45:0/695856170 shutdown_connections 2023-08-27T11:13:01.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.071+0000 7fb171192700 1 -- 172.21.15.45:0/695856170 wait complete. 2023-08-27T11:13:01.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.072+0000 7fb171192700 1 Processor -- start 2023-08-27T11:13:01.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.072+0000 7fb171192700 1 -- start start 2023-08-27T11:13:01.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb171192700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 0x7fb16c13eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb171192700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb16c0af420 0x7fb16c0b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb171192700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 0x7fb16c0b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb171192700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c0cf750 con 0x7fb16c0aed60 2023-08-27T11:13:01.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb171192700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c141bf0 con 0x7fb16c0af420 2023-08-27T11:13:01.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb171192700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c0b4860 con 0x7fb16c15d3f0 2023-08-27T11:13:01.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 0x7fb16c13eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 0x7fb16c13eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52912/0 (socket says 172.21.15.45:52912) 2023-08-27T11:13:01.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16ad9d700 1 -- 172.21.15.45:0/1331187 learned_addr learned my addr 172.21.15.45:0/1331187 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:01.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16a59c700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb16c0af420 0x7fb16c0b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16b59e700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 0x7fb16c0b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16ad9d700 1 -- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 msgr2=0x7fb16c0b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16ad9d700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 0x7fb16c0b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16ad9d700 1 -- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb16c0af420 msgr2=0x7fb16c0b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16ad9d700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb16c0af420 0x7fb16c0b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16ad9d700 1 -- 172.21.15.45:0/1331187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1600094a0 con 0x7fb16c0aed60 2023-08-27T11:13:01.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.073+0000 7fb16ad9d700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 0x7fb16c13eb90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00d830 tx=0x7fb15c00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:01.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.074+0000 7fb1537fe700 1 -- 172.21.15.45:0/1331187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0125f0 con 0x7fb16c0aed60 2023-08-27T11:13:01.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.074+0000 7fb1537fe700 1 -- 172.21.15.45:0/1331187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15c012750 con 0x7fb16c0aed60 2023-08-27T11:13:01.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.074+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c13f0d0 con 0x7fb16c0aed60 2023-08-27T11:13:01.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.074+0000 7fb1537fe700 1 -- 172.21.15.45:0/1331187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c012b00 con 0x7fb16c0aed60 2023-08-27T11:13:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.074+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c1415a0 con 0x7fb16c0aed60 2023-08-27T11:13:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.075+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c0727c0 con 0x7fb16c0aed60 2023-08-27T11:13:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.075+0000 7fb1537fe700 1 -- 172.21.15.45:0/1331187 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb15c012c60 con 0x7fb16c0aed60 2023-08-27T11:13:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.075+0000 7fb1537fe700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb15806a0e0 0x7fb15806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.076+0000 7fb1537fe700 1 -- 172.21.15.45:0/1331187 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb15c088c70 con 0x7fb16c0aed60 2023-08-27T11:13:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.076+0000 7fb16a59c700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb15806a0e0 0x7fb15806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.076+0000 7fb16a59c700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb15806a0e0 0x7fb15806c5a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0af1a0 tx=0x7fb160008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:01.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.081+0000 7fb1537fe700 1 -- 172.21.15.45:0/1331187 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb15c054040 con 0x7fb16c0aed60 2023-08-27T11:13:01.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.281+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb16c0b5ca0 con 0x7fb16c0aed60 2023-08-27T11:13:01.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.282+0000 7fb1537fe700 1 -- 172.21.15.45:0/1331187 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb15c057660 con 0x7fb16c0aed60 2023-08-27T11:13:01.284 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:13:01.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb15806a0e0 msgr2=0x7fb15806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb15806a0e0 0x7fb15806c5a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0af1a0 tx=0x7fb160008030 comp rx=0 tx=0).stop 2023-08-27T11:13:01.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 msgr2=0x7fb16c13eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 0x7fb16c13eb90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00d830 tx=0x7fb15c00dbf0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 shutdown_connections 2023-08-27T11:13:01.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb16c0aed60 0x7fb16c13eb90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb15806a0e0 0x7fb15806c5a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb16c15d3f0 0x7fb16c0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 --2- 172.21.15.45:0/1331187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb16c0af420 0x7fb16c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 >> 172.21.15.45:0/1331187 conn(0x7fb16c0cfdd0 msgr2=0x7fb16c0b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:01.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 shutdown_connections 2023-08-27T11:13:01.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.284+0000 7fb171192700 1 -- 172.21.15.45:0/1331187 wait complete. 2023-08-27T11:13:01.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:13:01.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.424+0000 7f7ee9fd3700 1 Processor -- start 2023-08-27T11:13:01.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.424+0000 7f7ee9fd3700 1 -- start start 2023-08-27T11:13:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.425+0000 7f7ee9fd3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee406a280 0x7f7ee406a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.425+0000 7f7ee9fd3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7ee415b5b0 0x7f7ee415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.425+0000 7f7ee9fd3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee4071f90 con 0x7f7ee4071400 2023-08-27T11:13:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.425+0000 7f7ee9fd3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee40b2290 con 0x7f7ee415b5b0 2023-08-27T11:13:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.425+0000 7f7ee9fd3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee406aba0 con 0x7f7ee406a280 2023-08-27T11:13:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee3fff700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7ee415b5b0 0x7f7ee415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee37fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee406a280 0x7f7ee406a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee37fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee406a280 0x7f7ee406a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50830/0 (socket says 172.21.15.45:50830) 2023-08-27T11:13:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee37fe700 1 -- 172.21.15.45:0/1306871691 learned_addr learned my addr 172.21.15.45:0/1306871691 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee3fff700 1 -- 172.21.15.45:0/1306871691 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee406a280 msgr2=0x7f7ee406a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee3fff700 1 --2- 172.21.15.45:0/1306871691 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee406a280 0x7f7ee406a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee3fff700 1 -- 172.21.15.45:0/1306871691 >> v1:172.21.15.135:6789/0 conn(0x7f7ee4071400 legacy=0x7f7ee4071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee3fff700 1 -- 172.21.15.45:0/1306871691 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee40b1960 con 0x7f7ee415b5b0 2023-08-27T11:13:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee37fe700 1 --2- 172.21.15.45:0/1306871691 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee406a280 0x7f7ee406a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee3fff700 1 --2- 172.21.15.45:0/1306871691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee415b5b0 0x7f7ee415ba40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009d90 tx=0x7f7edc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee7f0787f1968c16 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee27fc700 1 -- 172.21.15.45:0/1306871691 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00e070 con 0x7f7ee415b5b0 2023-08-27T11:13:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee27fc700 1 -- 172.21.15.45:0/1306871691 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc00bb20 con 0x7f7ee415b5b0 2023-08-27T11:13:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.426+0000 7f7ee27fc700 1 -- 172.21.15.45:0/1306871691 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc012430 con 0x7f7ee415b5b0 2023-08-27T11:13:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.427+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/1306871691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee415b5b0 msgr2=0x7f7ee415ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.427+0000 7f7ee9fd3700 1 --2- 172.21.15.45:0/1306871691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee415b5b0 0x7f7ee415ba40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009d90 tx=0x7f7edc00b330 comp rx=0 tx=0).stop 2023-08-27T11:13:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.427+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/1306871691 shutdown_connections 2023-08-27T11:13:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.427+0000 7f7ee9fd3700 1 --2- 172.21.15.45:0/1306871691 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee406a280 0x7f7ee406a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.427+0000 7f7ee9fd3700 1 --2- 172.21.15.45:0/1306871691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee415b5b0 0x7f7ee415ba40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.427+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/1306871691 >> 172.21.15.45:0/1306871691 conn(0x7f7ee40cc7c0 msgr2=0x7f7ee40ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.427+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/1306871691 shutdown_connections 2023-08-27T11:13:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.427+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/1306871691 wait complete. 2023-08-27T11:13:01.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.428+0000 7f7ee9fd3700 1 Processor -- start 2023-08-27T11:13:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.428+0000 7f7ee9fd3700 1 -- start start 2023-08-27T11:13:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee9fd3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee406a280 0x7f7ee41f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee9fd3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee4071400 0x7f7ee415dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee9fd3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee415b5b0 0x7f7ee415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee9fd3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40b4d80 con 0x7f7ee406a280 2023-08-27T11:13:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee9fd3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee41573d0 con 0x7f7ee4071400 2023-08-27T11:13:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee9fd3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40b4aa0 con 0x7f7ee415b5b0 2023-08-27T11:13:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee406a280 0x7f7ee41f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee406a280 0x7f7ee41f3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52924/0 (socket says 172.21.15.45:52924) 2023-08-27T11:13:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee37fe700 1 -- 172.21.15.45:0/3986557837 learned_addr learned my addr 172.21.15.45:0/3986557837 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee3fff700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee415b5b0 0x7f7ee415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.429+0000 7f7ee2ffd700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee4071400 0x7f7ee415dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ee37fe700 1 -- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee415b5b0 msgr2=0x7f7ee415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ee37fe700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee415b5b0 0x7f7ee415e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ee37fe700 1 -- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee4071400 msgr2=0x7f7ee415dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ee37fe700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee4071400 0x7f7ee415dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ee37fe700 1 -- 172.21.15.45:0/3986557837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed4009550 con 0x7f7ee406a280 2023-08-27T11:13:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ee2ffd700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee4071400 0x7f7ee415dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ee37fe700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee406a280 0x7f7ee41f3190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400e9b0 tx=0x7f7ed400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ecbfff700 1 -- 172.21.15.45:0/3986557837 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed400c850 con 0x7f7ee406a280 2023-08-27T11:13:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ecbfff700 1 -- 172.21.15.45:0/3986557837 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed400c9b0 con 0x7f7ee406a280 2023-08-27T11:13:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ecbfff700 1 -- 172.21.15.45:0/3986557837 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed400cd20 con 0x7f7ee406a280 2023-08-27T11:13:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc009a70 con 0x7f7ee406a280 2023-08-27T11:13:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.430+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee4157950 con 0x7f7ee406a280 2023-08-27T11:13:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.431+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee4157cc0 con 0x7f7ee406a280 2023-08-27T11:13:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.431+0000 7f7ecbfff700 1 -- 172.21.15.45:0/3986557837 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ed4013470 con 0x7f7ee406a280 2023-08-27T11:13:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.432+0000 7f7ecbfff700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ed006a0e0 0x7f7ed006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.432+0000 7f7ecbfff700 1 -- 172.21.15.45:0/3986557837 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7ed4088910 con 0x7f7ee406a280 2023-08-27T11:13:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.432+0000 7f7ee2ffd700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ed006a0e0 0x7f7ed006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.433+0000 7f7ee2ffd700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ed006a0e0 0x7f7ed006c5a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40c3600 tx=0x7f7ed8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.437+0000 7f7ecbfff700 1 -- 172.21.15.45:0/3986557837 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7ed4053ce0 con 0x7f7ee406a280 2023-08-27T11:13:01.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.637+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7ee4072690 con 0x7f7ee406a280 2023-08-27T11:13:01.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.638+0000 7f7ecbfff700 1 -- 172.21.15.45:0/3986557837 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7ed4057300 con 0x7f7ee406a280 2023-08-27T11:13:01.639 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:13:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ed006a0e0 msgr2=0x7f7ed006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ed006a0e0 0x7f7ed006c5a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40c3600 tx=0x7f7ed8009040 comp rx=0 tx=0).stop 2023-08-27T11:13:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee406a280 msgr2=0x7f7ee41f3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee406a280 0x7f7ee41f3190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400e9b0 tx=0x7f7ed400ecc0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 shutdown_connections 2023-08-27T11:13:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ee406a280 0x7f7ee41f3190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f7ed006a0e0 0x7f7ed006c5a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7ee415b5b0 0x7f7ee415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 --2- 172.21.15.45:0/3986557837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee4071400 0x7f7ee415dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 >> 172.21.15.45:0/3986557837 conn(0x7f7ee40cc7c0 msgr2=0x7f7ee415c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 shutdown_connections 2023-08-27T11:13:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.640+0000 7f7ee9fd3700 1 -- 172.21.15.45:0/3986557837 wait complete. 2023-08-27T11:13:01.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:13:01.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.780+0000 7f91be0e5700 1 Processor -- start 2023-08-27T11:13:01.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.780+0000 7f91be0e5700 1 -- start start 2023-08-27T11:13:01.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91be0e5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b815d380 0x7f91b80ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91be0e5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91b8073a10 0x7f91b80cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91be0e5700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80ce3a0 con 0x7f91b806a280 2023-08-27T11:13:01.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91be0e5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80d0230 con 0x7f91b8073a10 2023-08-27T11:13:01.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91be0e5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80b3210 con 0x7f91b815d380 2023-08-27T11:13:01.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91b6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91b8073a10 0x7f91b80cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91b6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91b8073a10 0x7f91b80cfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52932/0 (socket says 172.21.15.45:52932) 2023-08-27T11:13:01.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91b6ffd700 1 -- 172.21.15.45:0/1976549159 learned_addr learned my addr 172.21.15.45:0/1976549159 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:01.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91b77fe700 1 --2- 172.21.15.45:0/1976549159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b815d380 0x7f91b80ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91b6ffd700 1 -- 172.21.15.45:0/1976549159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b815d380 msgr2=0x7f91b80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:01.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91b6ffd700 1 --2- 172.21.15.45:0/1976549159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b815d380 0x7f91b80ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91b6ffd700 1 -- 172.21.15.45:0/1976549159 >> v1:172.21.15.135:6789/0 conn(0x7f91b806a280 legacy=0x7f91b806c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91b6ffd700 1 -- 172.21.15.45:0/1976549159 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80cd7d0 con 0x7f91b8073a10 2023-08-27T11:13:01.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.781+0000 7f91b6ffd700 1 --2- 172.21.15.45:0/1976549159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b8073a10 0x7f91b80cfae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91a00098f0 tx=0x7f91a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0b1e8301d64572 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:01.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91b67fc700 1 -- 172.21.15.45:0/1976549159 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0011830 con 0x7f91b8073a10 2023-08-27T11:13:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91b67fc700 1 -- 172.21.15.45:0/1976549159 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a0011990 con 0x7f91b8073a10 2023-08-27T11:13:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91b67fc700 1 -- 172.21.15.45:0/1976549159 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0011d20 con 0x7f91b8073a10 2023-08-27T11:13:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91be0e5700 1 -- 172.21.15.45:0/1976549159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b8073a10 msgr2=0x7f91b80cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91be0e5700 1 --2- 172.21.15.45:0/1976549159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b8073a10 0x7f91b80cfae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91a00098f0 tx=0x7f91a0009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91be0e5700 1 -- 172.21.15.45:0/1976549159 shutdown_connections 2023-08-27T11:13:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91be0e5700 1 --2- 172.21.15.45:0/1976549159 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b815d380 0x7f91b80ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91be0e5700 1 --2- 172.21.15.45:0/1976549159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b8073a10 0x7f91b80cfae0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91be0e5700 1 -- 172.21.15.45:0/1976549159 >> 172.21.15.45:0/1976549159 conn(0x7f91b80b2360 msgr2=0x7f91b80b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91be0e5700 1 -- 172.21.15.45:0/1976549159 shutdown_connections 2023-08-27T11:13:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.782+0000 7f91be0e5700 1 -- 172.21.15.45:0/1976549159 wait complete. 2023-08-27T11:13:01.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.783+0000 7f91be0e5700 1 Processor -- start 2023-08-27T11:13:01.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.784+0000 7f91be0e5700 1 -- start start 2023-08-27T11:13:01.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.784+0000 7f91be0e5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b806a280 0x7f91b80ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.784+0000 7f91be0e5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b8073a10 0x7f91b80c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.784+0000 7f91be0e5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b815d380 0x7f91b8156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.784+0000 7f91be0e5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815d970 con 0x7f91b806a280 2023-08-27T11:13:01.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.784+0000 7f91be0e5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815dd90 con 0x7f91b815d380 2023-08-27T11:13:01.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.784+0000 7f91be0e5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80b0da0 con 0x7f91b8073a10 2023-08-27T11:13:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b806a280 0x7f91b80ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b806a280 0x7f91b80ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52938/0 (socket says 172.21.15.45:52938) 2023-08-27T11:13:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b77fe700 1 -- 172.21.15.45:0/1737010233 learned_addr learned my addr 172.21.15.45:0/1737010233 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:01.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b6ffd700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b8073a10 0x7f91b80c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b77fe700 1 -- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b8073a10 msgr2=0x7f91b80c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b77fe700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b8073a10 0x7f91b80c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b77fe700 1 -- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b815d380 msgr2=0x7f91b8156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b7fff700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b815d380 0x7f91b8156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b77fe700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b815d380 0x7f91b8156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b77fe700 1 -- 172.21.15.45:0/1737010233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a0009550 con 0x7f91b806a280 2023-08-27T11:13:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91b77fe700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b806a280 0x7f91b80ad750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e8a0 tx=0x7f91a800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:01.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91affff700 1 -- 172.21.15.45:0/1737010233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8009590 con 0x7f91b806a280 2023-08-27T11:13:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91affff700 1 -- 172.21.15.45:0/1737010233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a800eea0 con 0x7f91b806a280 2023-08-27T11:13:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b80c5d50 con 0x7f91b806a280 2023-08-27T11:13:01.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91affff700 1 -- 172.21.15.45:0/1737010233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a800ca50 con 0x7f91b806a280 2023-08-27T11:13:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.785+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b80b0ac0 con 0x7f91b806a280 2023-08-27T11:13:01.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.786+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b809d520 con 0x7f91b806a280 2023-08-27T11:13:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.787+0000 7f91affff700 1 -- 172.21.15.45:0/1737010233 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f91a800cbb0 con 0x7f91b806a280 2023-08-27T11:13:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.787+0000 7f91affff700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f919806a110 0x7f919806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:01.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.787+0000 7f91affff700 1 -- 172.21.15.45:0/1737010233 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f91a8088750 con 0x7f91b806a280 2023-08-27T11:13:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.787+0000 7f91b6ffd700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f919806a110 0x7f919806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.788+0000 7f91b6ffd700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f919806a110 0x7f919806c5d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91b81f77a0 tx=0x7f91a0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:01.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.791+0000 7f91affff700 1 -- 172.21.15.45:0/1737010233 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91a8057630 con 0x7f91b806a280 2023-08-27T11:13:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.990+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91b815dbe0 con 0x7f91b806a280 2023-08-27T11:13:01.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.991+0000 7f91affff700 1 -- 172.21.15.45:0/1737010233 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f91a80571c0 con 0x7f91b806a280 2023-08-27T11:13:01.992 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:01.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f919806a110 msgr2=0x7f919806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f919806a110 0x7f919806c5d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91b81f77a0 tx=0x7f91a0008030 comp rx=0 tx=0).stop 2023-08-27T11:13:01.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b806a280 msgr2=0x7f91b80ad750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:01.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b806a280 0x7f91b80ad750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e8a0 tx=0x7f91a800ec60 comp rx=0 tx=0).stop 2023-08-27T11:13:01.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 shutdown_connections 2023-08-27T11:13:01.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b806a280 0x7f91b80ad750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f919806a110 0x7f919806c5d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91b815d380 0x7f91b8156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 --2- 172.21.15.45:0/1737010233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91b8073a10 0x7f91b80c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 >> 172.21.15.45:0/1737010233 conn(0x7f91b80b2360 msgr2=0x7f91b806a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 shutdown_connections 2023-08-27T11:13:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:01.993+0000 7f91be0e5700 1 -- 172.21.15.45:0/1737010233 wait complete. 2023-08-27T11:13:02.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.133+0000 7f449408f700 1 Processor -- start 2023-08-27T11:13:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.134+0000 7f449408f700 1 -- start start 2023-08-27T11:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.134+0000 7f449408f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f448c0b17c0 0x7f448c0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.134+0000 7f449408f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c06ced0 0x7f448c06d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.134+0000 7f449408f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448c15b460 con 0x7f448c157be0 2023-08-27T11:13:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.134+0000 7f449408f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448c157fc0 con 0x7f448c0b17c0 2023-08-27T11:13:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.134+0000 7f449408f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c071dc0 con 0x7f448c06ced0 2023-08-27T11:13:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f4491e2b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f448c157be0 0x7f448c0716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39722/0 (socket says 172.21.15.45:39722) 2023-08-27T11:13:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f4491e2b700 1 -- 172.21.15.45:0/1810783434 learned_addr learned my addr 172.21.15.45:0/1810783434 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f449162a700 1 --2- 172.21.15.45:0/1810783434 >> v2:172.21.15.135:3300/0 conn(0x7f448c0b17c0 0x7f448c0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f449262c700 1 --2- 172.21.15.45:0/1810783434 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c06ced0 0x7f448c06d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f4490e29700 1 -- 172.21.15.45:0/1810783434 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665595689 0 0) 0x7f448c15b460 con 0x7f448c157be0 2023-08-27T11:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f4490e29700 1 -- 172.21.15.45:0/1810783434 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44780035b0 con 0x7f448c157be0 2023-08-27T11:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f449262c700 1 -- 172.21.15.45:0/1810783434 >> v2:172.21.15.135:3300/0 conn(0x7f448c0b17c0 msgr2=0x7f448c0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f449262c700 1 --2- 172.21.15.45:0/1810783434 >> v2:172.21.15.135:3300/0 conn(0x7f448c0b17c0 0x7f448c0b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f449262c700 1 -- 172.21.15.45:0/1810783434 >> v1:172.21.15.45:6789/0 conn(0x7f448c157be0 legacy=0x7f448c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f4490e29700 1 -- 172.21.15.45:0/1810783434 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3095613518 0 0) 0x7f44780035b0 con 0x7f448c157be0 2023-08-27T11:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f449262c700 1 -- 172.21.15.45:0/1810783434 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c0b2280 con 0x7f448c06ced0 2023-08-27T11:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f449162a700 1 --2- 172.21.15.45:0/1810783434 >> v2:172.21.15.135:3300/0 conn(0x7f448c0b17c0 0x7f448c0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f449262c700 1 --2- 172.21.15.45:0/1810783434 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c06ced0 0x7f448c06d340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4484009d90 tx=0x7f448400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bddfdc1ff05cd134 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.135+0000 7f4490e29700 1 -- 172.21.15.45:0/1810783434 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448400e070 con 0x7f448c06ced0 2023-08-27T11:13:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f4490e29700 1 -- 172.21.15.45:0/1810783434 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484003a20 con 0x7f448c06ced0 2023-08-27T11:13:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f4490e29700 1 -- 172.21.15.45:0/1810783434 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488001670 con 0x7f448c06ced0 2023-08-27T11:13:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f449408f700 1 -- 172.21.15.45:0/1810783434 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c06ced0 msgr2=0x7f448c06d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f449408f700 1 --2- 172.21.15.45:0/1810783434 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c06ced0 0x7f448c06d340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4484009d90 tx=0x7f448400b330 comp rx=0 tx=0).stop 2023-08-27T11:13:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f449408f700 1 -- 172.21.15.45:0/1810783434 shutdown_connections 2023-08-27T11:13:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f449408f700 1 --2- 172.21.15.45:0/1810783434 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c06ced0 0x7f448c06d340 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f449408f700 1 --2- 172.21.15.45:0/1810783434 >> v2:172.21.15.135:3300/0 conn(0x7f448c0b17c0 0x7f448c0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f449408f700 1 -- 172.21.15.45:0/1810783434 >> 172.21.15.45:0/1810783434 conn(0x7f448c0cc7c0 msgr2=0x7f448c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f449408f700 1 -- 172.21.15.45:0/1810783434 shutdown_connections 2023-08-27T11:13:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.136+0000 7f449408f700 1 -- 172.21.15.45:0/1810783434 wait complete. 2023-08-27T11:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.137+0000 7f449408f700 1 Processor -- start 2023-08-27T11:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.137+0000 7f449408f700 1 -- start start 2023-08-27T11:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.138+0000 7f449408f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f448c06ced0 0x7f448c156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.138+0000 7f449408f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c0b17c0 0x7f448c0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.138+0000 7f449408f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c157be0 0x7f448c159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.138+0000 7f449408f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0c8150 con 0x7f448c0b17c0 2023-08-27T11:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.138+0000 7f449408f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0b3d80 con 0x7f448c06ced0 2023-08-27T11:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.138+0000 7f449408f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0c75d0 con 0x7f448c157be0 2023-08-27T11:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.138+0000 7f449162a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c0b17c0 0x7f448c0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.138+0000 7f449162a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c0b17c0 0x7f448c0c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52954/0 (socket says 172.21.15.45:52954) 2023-08-27T11:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.138+0000 7f449162a700 1 -- 172.21.15.45:0/1634965509 learned_addr learned my addr 172.21.15.45:0/1634965509 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f4491e2b700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f448c06ced0 0x7f448c156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449262c700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c157be0 0x7f448c159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449162a700 1 -- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c157be0 msgr2=0x7f448c159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449162a700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c157be0 0x7f448c159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449162a700 1 -- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f448c06ced0 msgr2=0x7f448c156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449162a700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f448c06ced0 0x7f448c156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449162a700 1 -- 172.21.15.45:0/1634965509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44800094a0 con 0x7f448c0b17c0 2023-08-27T11:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449262c700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c157be0 0x7f448c159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449162a700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c0b17c0 0x7f448c0c6d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f448000ebe0 tx=0x7f448000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f447e7fc700 1 -- 172.21.15.45:0/1634965509 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000c850 con 0x7f448c0b17c0 2023-08-27T11:13:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f447e7fc700 1 -- 172.21.15.45:0/1634965509 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448000c9b0 con 0x7f448c0b17c0 2023-08-27T11:13:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4484009a70 con 0x7f448c0b17c0 2023-08-27T11:13:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f447e7fc700 1 -- 172.21.15.45:0/1634965509 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000cd40 con 0x7f448c0b17c0 2023-08-27T11:13:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.139+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c0c7b90 con 0x7f448c0b17c0 2023-08-27T11:13:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.140+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c0c8570 con 0x7f448c0b17c0 2023-08-27T11:13:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.141+0000 7f447e7fc700 1 -- 172.21.15.45:0/1634965509 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4480013470 con 0x7f448c0b17c0 2023-08-27T11:13:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.141+0000 7f447e7fc700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f447006a0e0 0x7f447006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.141+0000 7f447e7fc700 1 -- 172.21.15.45:0/1634965509 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4480089090 con 0x7f448c0b17c0 2023-08-27T11:13:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.142+0000 7f4491e2b700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f447006a0e0 0x7f447006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.142+0000 7f4491e2b700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f447006a0e0 0x7f447006c5a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f448c0cdaf0 tx=0x7f4488001e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.145+0000 7f447e7fc700 1 -- 172.21.15.45:0/1634965509 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4480054530 con 0x7f448c0b17c0 2023-08-27T11:13:02.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.344+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f448c072690 con 0x7f448c0b17c0 2023-08-27T11:13:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.345+0000 7f447e7fc700 1 -- 172.21.15.45:0/1634965509 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44800170a0 con 0x7f448c0b17c0 2023-08-27T11:13:02.346 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:13:02.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f447006a0e0 msgr2=0x7f447006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f447006a0e0 0x7f447006c5a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f448c0cdaf0 tx=0x7f4488001e40 comp rx=0 tx=0).stop 2023-08-27T11:13:02.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c0b17c0 msgr2=0x7f448c0c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c0b17c0 0x7f448c0c6d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f448000ebe0 tx=0x7f448000eef0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 shutdown_connections 2023-08-27T11:13:02.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f447006a0e0 0x7f447006c5a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f448c157be0 0x7f448c159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f448c06ced0 0x7f448c156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 --2- 172.21.15.45:0/1634965509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c0b17c0 0x7f448c0c6d80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 >> 172.21.15.45:0/1634965509 conn(0x7f448c0cc7c0 msgr2=0x7f448c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:02.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 shutdown_connections 2023-08-27T11:13:02.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.347+0000 7f449408f700 1 -- 172.21.15.45:0/1634965509 wait complete. 2023-08-27T11:13:02.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:13:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.493+0000 7f5e4d84a700 1 Processor -- start 2023-08-27T11:13:02.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.493+0000 7f5e4d84a700 1 -- start start 2023-08-27T11:13:02.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.494+0000 7f5e4d84a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e480b8c20 0x7f5e480cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.494+0000 7f5e4d84a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4815d6a0 con 0x7f5e480ce2e0 2023-08-27T11:13:02.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.494+0000 7f5e4d84a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480d2c90 con 0x7f5e4815d2c0 2023-08-27T11:13:02.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.494+0000 7f5e477fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5e480ce2e0 0x7f5e480b6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39740/0 (socket says 172.21.15.45:39740) 2023-08-27T11:13:02.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.494+0000 7f5e477fe700 1 -- 172.21.15.45:0/2409953100 learned_addr learned my addr 172.21.15.45:0/2409953100 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:02.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.494+0000 7f5e4d84a700 1 -- 172.21.15.45:0/2409953100 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480ce770 con 0x7f5e480b8c20 2023-08-27T11:13:02.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.494+0000 7f5e467fc700 1 --2- 172.21.15.45:0/2409953100 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e480b8c20 0x7f5e480cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.494+0000 7f5e45ffb700 1 -- 172.21.15.45:0/2409953100 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934834017 0 0) 0x7f5e40001490 con 0x7f5e480ce2e0 2023-08-27T11:13:02.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.494+0000 7f5e45ffb700 1 -- 172.21.15.45:0/2409953100 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e300035b0 con 0x7f5e480ce2e0 2023-08-27T11:13:02.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.495+0000 7f5e45ffb700 1 -- 172.21.15.45:0/2409953100 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1515808318 0 0) 0x7f5e300035b0 con 0x7f5e480ce2e0 2023-08-27T11:13:02.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.495+0000 7f5e467fc700 1 -- 172.21.15.45:0/2409953100 >> v1:172.21.15.135:6789/0 conn(0x7f5e4815d2c0 legacy=0x7f5e480d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.495+0000 7f5e467fc700 1 -- 172.21.15.45:0/2409953100 >> v1:172.21.15.45:6789/0 conn(0x7f5e480ce2e0 legacy=0x7f5e480b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.495+0000 7f5e467fc700 1 -- 172.21.15.45:0/2409953100 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480d1bf0 con 0x7f5e480b8c20 2023-08-27T11:13:02.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.495+0000 7f5e467fc700 1 --2- 172.21.15.45:0/2409953100 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e480b8c20 0x7f5e480cdda0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e38009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=290bdb6a6eb599b0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:02.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.495+0000 7f5e45ffb700 1 -- 172.21.15.45:0/2409953100 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40001b80 con 0x7f5e480b8c20 2023-08-27T11:13:02.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.495+0000 7f5e45ffb700 1 -- 172.21.15.45:0/2409953100 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e38004030 con 0x7f5e480b8c20 2023-08-27T11:13:02.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.495+0000 7f5e45ffb700 1 -- 172.21.15.45:0/2409953100 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380043c0 con 0x7f5e480b8c20 2023-08-27T11:13:02.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.496+0000 7f5e4d84a700 1 -- 172.21.15.45:0/2409953100 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e480b8c20 msgr2=0x7f5e480cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.496+0000 7f5e4d84a700 1 --2- 172.21.15.45:0/2409953100 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e480b8c20 0x7f5e480cdda0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e38009b80 comp rx=0 tx=0).stop 2023-08-27T11:13:02.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.496+0000 7f5e4d84a700 1 -- 172.21.15.45:0/2409953100 shutdown_connections 2023-08-27T11:13:02.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.496+0000 7f5e4d84a700 1 --2- 172.21.15.45:0/2409953100 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e480b8c20 0x7f5e480cdda0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.496+0000 7f5e4d84a700 1 -- 172.21.15.45:0/2409953100 >> 172.21.15.45:0/2409953100 conn(0x7f5e480d2fd0 msgr2=0x7f5e480d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:02.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.496+0000 7f5e4d84a700 1 -- 172.21.15.45:0/2409953100 shutdown_connections 2023-08-27T11:13:02.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.496+0000 7f5e4d84a700 1 -- 172.21.15.45:0/2409953100 wait complete. 2023-08-27T11:13:02.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.497+0000 7f5e4d84a700 1 Processor -- start 2023-08-27T11:13:02.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.497+0000 7f5e4d84a700 1 -- start start 2023-08-27T11:13:02.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e4d84a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e480b8c20 0x7f5e481519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e4d84a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e480ce2e0 0x7f5e480cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e4d84a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e4815d2c0 0x7f5e481500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e4d84a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480c7fa0 con 0x7f5e480b8c20 2023-08-27T11:13:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e4d84a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480cc310 con 0x7f5e480ce2e0 2023-08-27T11:13:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e4d84a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48152080 con 0x7f5e4815d2c0 2023-08-27T11:13:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e46ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e480b8c20 0x7f5e481519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e46ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e480b8c20 0x7f5e481519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52966/0 (socket says 172.21.15.45:52966) 2023-08-27T11:13:02.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e46ffd700 1 -- 172.21.15.45:0/620108647 learned_addr learned my addr 172.21.15.45:0/620108647 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e467fc700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e480ce2e0 0x7f5e480cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e477fe700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e4815d2c0 0x7f5e481500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e46ffd700 1 -- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e4815d2c0 msgr2=0x7f5e481500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e46ffd700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e4815d2c0 0x7f5e481500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e46ffd700 1 -- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e480ce2e0 msgr2=0x7f5e480cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e46ffd700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e480ce2e0 0x7f5e480cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e46ffd700 1 -- 172.21.15.45:0/620108647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380094a0 con 0x7f5e480b8c20 2023-08-27T11:13:02.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.498+0000 7f5e477fe700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e4815d2c0 0x7f5e481500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.499+0000 7f5e46ffd700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e480b8c20 0x7f5e481519f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c750 tx=0x7f5e3c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.499+0000 7f5e2ffff700 1 -- 172.21.15.45:0/620108647 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c005480 con 0x7f5e480b8c20 2023-08-27T11:13:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.499+0000 7f5e2ffff700 1 -- 172.21.15.45:0/620108647 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c003680 con 0x7f5e480b8c20 2023-08-27T11:13:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.499+0000 7f5e2ffff700 1 -- 172.21.15.45:0/620108647 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c003a30 con 0x7f5e480b8c20 2023-08-27T11:13:02.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.499+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e480c7600 con 0x7f5e480b8c20 2023-08-27T11:13:02.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.499+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e480c83c0 con 0x7f5e480b8c20 2023-08-27T11:13:02.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.500+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e480c80e0 con 0x7f5e480b8c20 2023-08-27T11:13:02.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.500+0000 7f5e2ffff700 1 -- 172.21.15.45:0/620108647 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5e3c003b90 con 0x7f5e480b8c20 2023-08-27T11:13:02.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.500+0000 7f5e2ffff700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5e3406a0e0 0x7f5e3406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.501+0000 7f5e2ffff700 1 -- 172.21.15.45:0/620108647 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5e3c0882d0 con 0x7f5e480b8c20 2023-08-27T11:13:02.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.501+0000 7f5e467fc700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5e3406a0e0 0x7f5e3406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.501+0000 7f5e467fc700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5e3406a0e0 0x7f5e3406c5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5e380097c0 tx=0x7f5e38008000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:02.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.505+0000 7f5e2ffff700 1 -- 172.21.15.45:0/620108647 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5e3c0537e0 con 0x7f5e480b8c20 2023-08-27T11:13:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.706+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5e480734b0 con 0x7f5e480b8c20 2023-08-27T11:13:02.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.706+0000 7f5e2ffff700 1 -- 172.21.15.45:0/620108647 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5e3c056e00 con 0x7f5e480b8c20 2023-08-27T11:13:02.708 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.708+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5e3406a0e0 msgr2=0x7f5e3406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.708+0000 7f5e4d84a700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5e3406a0e0 0x7f5e3406c5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5e380097c0 tx=0x7f5e38008000 comp rx=0 tx=0).stop 2023-08-27T11:13:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.708+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e480b8c20 msgr2=0x7f5e481519f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.708+0000 7f5e4d84a700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e480b8c20 0x7f5e481519f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c750 tx=0x7f5e3c00cb10 comp rx=0 tx=0).stop 2023-08-27T11:13:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.709+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 shutdown_connections 2023-08-27T11:13:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.709+0000 7f5e4d84a700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e480b8c20 0x7f5e481519f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.709+0000 7f5e4d84a700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5e3406a0e0 0x7f5e3406c5a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.709+0000 7f5e4d84a700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e4815d2c0 0x7f5e481500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.709+0000 7f5e4d84a700 1 --2- 172.21.15.45:0/620108647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e480ce2e0 0x7f5e480cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.709+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 >> 172.21.15.45:0/620108647 conn(0x7f5e480d2fd0 msgr2=0x7f5e480b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.709+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 shutdown_connections 2023-08-27T11:13:02.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.709+0000 7f5e4d84a700 1 -- 172.21.15.45:0/620108647 wait complete. 2023-08-27T11:13:02.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.855+0000 7f01ba837700 1 Processor -- start 2023-08-27T11:13:02.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.855+0000 7f01ba837700 1 -- start start 2023-08-27T11:13:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01ba837700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b415a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01ba837700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40caef0 con 0x7f01b415a750 2023-08-27T11:13:02.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01ba837700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40ce000 con 0x7f01b4070e40 2023-08-27T11:13:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01ba837700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40cac50 con 0x7f01b415dbc0 2023-08-27T11:13:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01b8dd4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f01b415a750 0x7f01b40cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39754/0 (socket says 172.21.15.45:39754) 2023-08-27T11:13:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01b8dd4700 1 -- 172.21.15.45:0/21882003 learned_addr learned my addr 172.21.15.45:0/21882003 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01b37fe700 1 --2- 172.21.15.45:0/21882003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01b37fe700 1 -- 172.21.15.45:0/21882003 >> v1:172.21.15.135:6789/0 conn(0x7f01b4070e40 legacy=0x7f01b4071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01b37fe700 1 -- 172.21.15.45:0/21882003 >> v1:172.21.15.45:6789/0 conn(0x7f01b415a750 legacy=0x7f01b40cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.856+0000 7f01b37fe700 1 -- 172.21.15.45:0/21882003 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b415b5c0 con 0x7f01b415dbc0 2023-08-27T11:13:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.857+0000 7f01b37fe700 1 --2- 172.21.15.45:0/21882003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b415a210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009870 tx=0x7f01a8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479fcac2f8f3b56f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:02.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.857+0000 7f01b2ffd700 1 -- 172.21.15.45:0/21882003 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8011830 con 0x7f01b415dbc0 2023-08-27T11:13:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.857+0000 7f01b2ffd700 1 -- 172.21.15.45:0/21882003 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a8011990 con 0x7f01b415dbc0 2023-08-27T11:13:02.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.857+0000 7f01b2ffd700 1 -- 172.21.15.45:0/21882003 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8011d20 con 0x7f01b415dbc0 2023-08-27T11:13:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.858+0000 7f01ba837700 1 -- 172.21.15.45:0/21882003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 msgr2=0x7f01b415a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.858+0000 7f01ba837700 1 --2- 172.21.15.45:0/21882003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b415a210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009870 tx=0x7f01a8009b80 comp rx=0 tx=0).stop 2023-08-27T11:13:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.858+0000 7f01ba837700 1 -- 172.21.15.45:0/21882003 shutdown_connections 2023-08-27T11:13:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.858+0000 7f01ba837700 1 --2- 172.21.15.45:0/21882003 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b415a210 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.858+0000 7f01ba837700 1 -- 172.21.15.45:0/21882003 >> 172.21.15.45:0/21882003 conn(0x7f01b40cfd60 msgr2=0x7f01b40d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.858+0000 7f01ba837700 1 -- 172.21.15.45:0/21882003 shutdown_connections 2023-08-27T11:13:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.858+0000 7f01ba837700 1 -- 172.21.15.45:0/21882003 wait complete. 2023-08-27T11:13:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.859+0000 7f01ba837700 1 Processor -- start 2023-08-27T11:13:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01ba837700 1 -- start start 2023-08-27T11:13:02.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01ba837700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01b4070e40 0x7f01b40b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01ba837700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01b415a750 0x7f01b4145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01ba837700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b40c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01ba837700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c8570 con 0x7f01b415a750 2023-08-27T11:13:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01ba837700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c8850 con 0x7f01b4070e40 2023-08-27T11:13:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01ba837700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c8b30 con 0x7f01b415dbc0 2023-08-27T11:13:02.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01b37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01b415a750 0x7f01b4145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01b37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01b415a750 0x7f01b4145020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52982/0 (socket says 172.21.15.45:52982) 2023-08-27T11:13:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01b37fe700 1 -- 172.21.15.45:0/2515739229 learned_addr learned my addr 172.21.15.45:0/2515739229 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01b3fff700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01b4070e40 0x7f01b40b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.860+0000 7f01b8dd4700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b40c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b37fe700 1 -- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 msgr2=0x7f01b40c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b37fe700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b40c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b37fe700 1 -- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01b4070e40 msgr2=0x7f01b40b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b37fe700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01b4070e40 0x7f01b40b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b37fe700 1 -- 172.21.15.45:0/2515739229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80094a0 con 0x7f01b415a750 2023-08-27T11:13:02.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b8dd4700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b40c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b37fe700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01b415a750 0x7f01b4145020 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f01a80090b0 tx=0x7f01a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b0ff9700 1 -- 172.21.15.45:0/2515739229 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8020070 con 0x7f01b415a750 2023-08-27T11:13:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b0ff9700 1 -- 172.21.15.45:0/2515739229 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a8004280 con 0x7f01b415a750 2023-08-27T11:13:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01b0ff9700 1 -- 172.21.15.45:0/2515739229 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80036a0 con 0x7f01b415a750 2023-08-27T11:13:02.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b40c8e10 con 0x7f01b415a750 2023-08-27T11:13:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.861+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b40b2ec0 con 0x7f01b415a750 2023-08-27T11:13:02.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.862+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b4072180 con 0x7f01b415a750 2023-08-27T11:13:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.863+0000 7f01b0ff9700 1 -- 172.21.15.45:0/2515739229 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f01a8024070 con 0x7f01b415a750 2023-08-27T11:13:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.863+0000 7f01b0ff9700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f01a006a110 0x7f01a006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.863+0000 7f01b0ff9700 1 -- 172.21.15.45:0/2515739229 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f01a808b2e0 con 0x7f01b415a750 2023-08-27T11:13:02.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.863+0000 7f01b3fff700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f01a006a110 0x7f01a006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.864+0000 7f01b3fff700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f01a006a110 0x7f01a006c5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007fd0 tx=0x7f01a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:02.868+0000 7f01b0ff9700 1 -- 172.21.15.45:0/2515739229 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f01a8056850 con 0x7f01b415a750 2023-08-27T11:13:03.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.067+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01b40c9330 con 0x7f01b415a750 2023-08-27T11:13:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.069+0000 7f01b0ff9700 1 -- 172.21.15.45:0/2515739229 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7f01b40c9330 con 0x7f01b415a750 2023-08-27T11:13:03.074 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:03.074 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f01a006a110 msgr2=0x7f01a006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f01a006a110 0x7f01a006c5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007fd0 tx=0x7f01a4009040 comp rx=0 tx=0).stop 2023-08-27T11:13:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01b415a750 msgr2=0x7f01b4145020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01b415a750 0x7f01b4145020 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f01a80090b0 tx=0x7f01a8004750 comp rx=0 tx=0).stop 2023-08-27T11:13:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 shutdown_connections 2023-08-27T11:13:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f01a006a110 0x7f01a006c5d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01b415dbc0 0x7f01b40c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01b4070e40 0x7f01b40b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 --2- 172.21.15.45:0/2515739229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01b415a750 0x7f01b4145020 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 >> 172.21.15.45:0/2515739229 conn(0x7f01b40cfd60 msgr2=0x7f01b4156d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 shutdown_connections 2023-08-27T11:13:03.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.071+0000 7f01ba837700 1 -- 172.21.15.45:0/2515739229 wait complete. 2023-08-27T11:13:03.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:13:03.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:13:03.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.220+0000 7f3ec37b1700 1 Processor -- start 2023-08-27T11:13:03.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.220+0000 7f3ec37b1700 1 -- start start 2023-08-27T11:13:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec37b1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3ebc071e10 0x7f3ebc071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec37b1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc15ea60 con 0x7f3ebc0c5740 2023-08-27T11:13:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec37b1700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc15dec0 con 0x7f3ebc15a000 2023-08-27T11:13:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec37b1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ebc15b3f0 con 0x7f3ebc071e10 2023-08-27T11:13:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec0d4c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3ebc071e10 0x7f3ebc071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec1d4e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3ebc0c5740 0x7f3ebc06ce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39766/0 (socket says 172.21.15.45:39766) 2023-08-27T11:13:03.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec1d4e700 1 -- 172.21.15.45:0/416280090 learned_addr learned my addr 172.21.15.45:0/416280090 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:03.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec0d4c700 1 -- 172.21.15.45:0/416280090 >> v1:172.21.15.135:6789/0 conn(0x7f3ebc15a000 legacy=0x7f3ebc15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:03.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec0d4c700 1 -- 172.21.15.45:0/416280090 >> v1:172.21.15.45:6789/0 conn(0x7f3ebc0c5740 legacy=0x7f3ebc06ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3ec0d4c700 1 -- 172.21.15.45:0/416280090 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0c64a0 con 0x7f3ebc071e10 2023-08-27T11:13:03.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.221+0000 7f3eb3fff700 1 -- 172.21.15.45:0/416280090 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826996241 0 0) 0x7f3eb4001490 con 0x7f3ebc0c5740 2023-08-27T11:13:03.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3ec0d4c700 1 --2- 172.21.15.45:0/416280090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc071e10 0x7f3ebc071820 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009840 tx=0x7f3eb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a918dc6e99fefe15 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:03.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3eb3fff700 1 -- 172.21.15.45:0/416280090 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8004230 con 0x7f3ebc071e10 2023-08-27T11:13:03.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3eb3fff700 1 -- 172.21.15.45:0/416280090 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb8004390 con 0x7f3ebc071e10 2023-08-27T11:13:03.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3eb3fff700 1 -- 172.21.15.45:0/416280090 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb80036a0 con 0x7f3ebc071e10 2023-08-27T11:13:03.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3ec37b1700 1 -- 172.21.15.45:0/416280090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc071e10 msgr2=0x7f3ebc071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3ec37b1700 1 --2- 172.21.15.45:0/416280090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc071e10 0x7f3ebc071820 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009840 tx=0x7f3eb8009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:03.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3ec37b1700 1 -- 172.21.15.45:0/416280090 shutdown_connections 2023-08-27T11:13:03.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3ec37b1700 1 --2- 172.21.15.45:0/416280090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc071e10 0x7f3ebc071820 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3ec37b1700 1 -- 172.21.15.45:0/416280090 >> 172.21.15.45:0/416280090 conn(0x7f3ebc0cfdd0 msgr2=0x7f3ebc0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:03.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.222+0000 7f3ec37b1700 1 -- 172.21.15.45:0/416280090 shutdown_connections 2023-08-27T11:13:03.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.223+0000 7f3ec37b1700 1 -- 172.21.15.45:0/416280090 wait complete. 2023-08-27T11:13:03.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.224+0000 7f3ec37b1700 1 Processor -- start 2023-08-27T11:13:03.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.224+0000 7f3ec37b1700 1 -- start start 2023-08-27T11:13:03.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.224+0000 7f3ec37b1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ebc071e10 0x7f3ebc14aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.224+0000 7f3ec37b1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ebc0c5740 0x7f3ebc0cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.224+0000 7f3ec37b1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc15a000 0x7f3ebc070d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.224+0000 7f3ec37b1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0b2f70 con 0x7f3ebc15a000 2023-08-27T11:13:03.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.224+0000 7f3ec37b1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc1537e0 con 0x7f3ebc0c5740 2023-08-27T11:13:03.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.224+0000 7f3ec37b1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc156110 con 0x7f3ebc071e10 2023-08-27T11:13:03.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec1d4e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc15a000 0x7f3ebc070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec1d4e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc15a000 0x7f3ebc070d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52998/0 (socket says 172.21.15.45:52998) 2023-08-27T11:13:03.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec1d4e700 1 -- 172.21.15.45:0/35684283 learned_addr learned my addr 172.21.15.45:0/35684283 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:03.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec0d4c700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ebc0c5740 0x7f3ebc0cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec154d700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ebc071e10 0x7f3ebc14aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec1d4e700 1 -- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ebc071e10 msgr2=0x7f3ebc14aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec1d4e700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ebc071e10 0x7f3ebc14aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec1d4e700 1 -- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ebc0c5740 msgr2=0x7f3ebc0cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec1d4e700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ebc0c5740 0x7f3ebc0cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec1d4e700 1 -- 172.21.15.45:0/35684283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb80094a0 con 0x7f3ebc15a000 2023-08-27T11:13:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec154d700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ebc071e10 0x7f3ebc14aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec1d4e700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc15a000 0x7f3ebc070d10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400cca0 tx=0x7f3eb400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:03.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3eb1ffb700 1 -- 172.21.15.45:0/35684283 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400ac20 con 0x7f3ebc15a000 2023-08-27T11:13:03.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3eb1ffb700 1 -- 172.21.15.45:0/35684283 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb400ad80 con 0x7f3ebc15a000 2023-08-27T11:13:03.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3eb1ffb700 1 -- 172.21.15.45:0/35684283 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4007310 con 0x7f3ebc15a000 2023-08-27T11:13:03.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.225+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc1f7390 con 0x7f3ebc15a000 2023-08-27T11:13:03.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.226+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc0b2920 con 0x7f3ebc15a000 2023-08-27T11:13:03.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.226+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc073b10 con 0x7f3ebc15a000 2023-08-27T11:13:03.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.227+0000 7f3eb1ffb700 1 -- 172.21.15.45:0/35684283 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3eb4007470 con 0x7f3ebc15a000 2023-08-27T11:13:03.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.227+0000 7f3eb1ffb700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ea006a0e0 0x7f3ea006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.227+0000 7f3eb1ffb700 1 -- 172.21.15.45:0/35684283 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3eb4088290 con 0x7f3ebc15a000 2023-08-27T11:13:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.227+0000 7f3ec154d700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ea006a0e0 0x7f3ea006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.228+0000 7f3ec154d700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ea006a0e0 0x7f3ea006c5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3eac003fc0 tx=0x7f3eac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.231+0000 7f3eb1ffb700 1 -- 172.21.15.45:0/35684283 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3eb40536e0 con 0x7f3ebc15a000 2023-08-27T11:13:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.432+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ebc15ea60 con 0x7f3ebc15a000 2023-08-27T11:13:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.432+0000 7f3eb1ffb700 1 -- 172.21.15.45:0/35684283 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ebc15ea60 con 0x7f3ebc15a000 2023-08-27T11:13:03.434 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ea006a0e0 msgr2=0x7f3ea006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ea006a0e0 0x7f3ea006c5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3eac003fc0 tx=0x7f3eac009040 comp rx=0 tx=0).stop 2023-08-27T11:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc15a000 msgr2=0x7f3ebc070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc15a000 0x7f3ebc070d10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400cca0 tx=0x7f3eb400a060 comp rx=0 tx=0).stop 2023-08-27T11:13:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 shutdown_connections 2023-08-27T11:13:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ebc15a000 0x7f3ebc070d10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ea006a0e0 0x7f3ea006c5a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ebc071e10 0x7f3ebc14aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 --2- 172.21.15.45:0/35684283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ebc0c5740 0x7f3ebc0cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 >> 172.21.15.45:0/35684283 conn(0x7f3ebc0cfdd0 msgr2=0x7f3ebc06dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 shutdown_connections 2023-08-27T11:13:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.434+0000 7f3ec37b1700 1 -- 172.21.15.45:0/35684283 wait complete. 2023-08-27T11:13:03.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:13:03.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:13:03.506 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:13:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.580+0000 7f3418800700 1 Processor -- start 2023-08-27T11:13:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.580+0000 7f3418800700 1 -- start start 2023-08-27T11:13:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f3418800700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f34140cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f3418800700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f34140cde30 0x7f341415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f3418800700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34140cd0d0 con 0x7f34140cb2d0 2023-08-27T11:13:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f3418800700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34140cd3b0 con 0x7f34140cde30 2023-08-27T11:13:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f3418800700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34140ce850 con 0x7f341415b100 2023-08-27T11:13:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f341359e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f34140cde30 0x7f341415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f3412d9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f34140cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f3412d9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f34140cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:50912/0 (socket says 172.21.15.45:50912) 2023-08-27T11:13:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f341359e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f34140cde30 0x7f341415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:41862/0 (socket says 172.21.15.45:41862) 2023-08-27T11:13:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.581+0000 7f3412d9d700 1 -- 172.21.15.45:0/1930792318 learned_addr learned my addr 172.21.15.45:0/1930792318 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f3412d9d700 1 -- 172.21.15.45:0/1930792318 >> v2:172.21.15.135:3300/0 conn(0x7f34140cde30 msgr2=0x7f341415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f3412d9d700 1 --2- 172.21.15.45:0/1930792318 >> v2:172.21.15.135:3300/0 conn(0x7f34140cde30 0x7f341415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f3412d9d700 1 -- 172.21.15.45:0/1930792318 >> v1:172.21.15.135:6789/0 conn(0x7f34140cb2d0 legacy=0x7f34140cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f3412d9d700 1 -- 172.21.15.45:0/1930792318 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34140ceb30 con 0x7f341415b100 2023-08-27T11:13:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f3411d9b700 1 -- 172.21.15.45:0/1930792318 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258840896 0 0) 0x7f34140cd0d0 con 0x7f34140cb2d0 2023-08-27T11:13:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f341359e700 1 --2- 172.21.15.45:0/1930792318 >> v2:172.21.15.135:3300/0 conn(0x7f34140cde30 0x7f341415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f3412d9d700 1 --2- 172.21.15.45:0/1930792318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f34140cf330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34040098f0 tx=0x7f3404009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c05d760d702930 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f3411d9b700 1 -- 172.21.15.45:0/1930792318 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404011830 con 0x7f341415b100 2023-08-27T11:13:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f3411d9b700 1 -- 172.21.15.45:0/1930792318 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3404011990 con 0x7f341415b100 2023-08-27T11:13:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.582+0000 7f3411d9b700 1 -- 172.21.15.45:0/1930792318 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404011d20 con 0x7f341415b100 2023-08-27T11:13:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.583+0000 7f3418800700 1 -- 172.21.15.45:0/1930792318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 msgr2=0x7f34140cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.583+0000 7f3418800700 1 --2- 172.21.15.45:0/1930792318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f34140cf330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34040098f0 tx=0x7f3404009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.583+0000 7f3418800700 1 -- 172.21.15.45:0/1930792318 shutdown_connections 2023-08-27T11:13:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.583+0000 7f3418800700 1 --2- 172.21.15.45:0/1930792318 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f34140cf330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.583+0000 7f3418800700 1 --2- 172.21.15.45:0/1930792318 >> v2:172.21.15.135:3300/0 conn(0x7f34140cde30 0x7f341415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.583+0000 7f3418800700 1 -- 172.21.15.45:0/1930792318 >> 172.21.15.45:0/1930792318 conn(0x7f341406ddd0 msgr2=0x7f341406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.583+0000 7f3418800700 1 -- 172.21.15.45:0/1930792318 shutdown_connections 2023-08-27T11:13:03.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.583+0000 7f3418800700 1 -- 172.21.15.45:0/1930792318 wait complete. 2023-08-27T11:13:03.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.584+0000 7f3418800700 1 Processor -- start 2023-08-27T11:13:03.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.584+0000 7f3418800700 1 -- start start 2023-08-27T11:13:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f3418800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34140cb2d0 0x7f341414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f3418800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34140cde30 0x7f34140b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f3418800700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f341414b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f3418800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341415cb40 con 0x7f34140cde30 2023-08-27T11:13:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f3418800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34140b58a0 con 0x7f34140cb2d0 2023-08-27T11:13:03.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f3418800700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341414b790 con 0x7f341415b100 2023-08-27T11:13:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f341259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34140cde30 0x7f34140b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f341259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34140cde30 0x7f34140b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53010/0 (socket says 172.21.15.45:53010) 2023-08-27T11:13:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f341259c700 1 -- 172.21.15.45:0/2148456729 learned_addr learned my addr 172.21.15.45:0/2148456729 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f3412d9d700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34140cb2d0 0x7f341414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f341359e700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f341414b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f341259c700 1 -- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 msgr2=0x7f341414b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f341259c700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f341414b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f341259c700 1 -- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34140cb2d0 msgr2=0x7f341414dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f341259c700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34140cb2d0 0x7f341414dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f341259c700 1 -- 172.21.15.45:0/2148456729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c009a70 con 0x7f34140cde30 2023-08-27T11:13:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.585+0000 7f3412d9d700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34140cb2d0 0x7f341414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.586+0000 7f341259c700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34140cde30 0x7f34140b5360 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f340800c750 tx=0x7f340800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.586+0000 7f33fb7fe700 1 -- 172.21.15.45:0/2148456729 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34080054e0 con 0x7f34140cde30 2023-08-27T11:13:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.586+0000 7f33fb7fe700 1 -- 172.21.15.45:0/2148456729 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3408003680 con 0x7f34140cde30 2023-08-27T11:13:03.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.586+0000 7f33fb7fe700 1 -- 172.21.15.45:0/2148456729 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408003a30 con 0x7f34140cde30 2023-08-27T11:13:03.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.586+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3404009550 con 0x7f34140cde30 2023-08-27T11:13:03.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.586+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34140b5b20 con 0x7f34140cde30 2023-08-27T11:13:03.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.587+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341415c910 con 0x7f34140cde30 2023-08-27T11:13:03.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.587+0000 7f33fb7fe700 1 -- 172.21.15.45:0/2148456729 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3408003b90 con 0x7f34140cde30 2023-08-27T11:13:03.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.587+0000 7f33fb7fe700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f340006a0e0 0x7f340006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.588+0000 7f33fb7fe700 1 -- 172.21.15.45:0/2148456729 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3408087cb0 con 0x7f34140cde30 2023-08-27T11:13:03.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.588+0000 7f3412d9d700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f340006a0e0 0x7f340006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.588+0000 7f3412d9d700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f340006a0e0 0x7f340006c5a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3404009c40 tx=0x7f3404008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:03.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.592+0000 7f33fb7fe700 1 -- 172.21.15.45:0/2148456729 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3408053130 con 0x7f34140cde30 2023-08-27T11:13:03.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.790+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f34140d0900 con 0x7f34140cde30 2023-08-27T11:13:03.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.790+0000 7f33fb7fe700 1 -- 172.21.15.45:0/2148456729 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f34080170a0 con 0x7f34140cde30 2023-08-27T11:13:03.792 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:13:03.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f340006a0e0 msgr2=0x7f340006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f340006a0e0 0x7f340006c5a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3404009c40 tx=0x7f3404008030 comp rx=0 tx=0).stop 2023-08-27T11:13:03.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34140cde30 msgr2=0x7f34140b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34140cde30 0x7f34140b5360 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f340800c750 tx=0x7f340800ca60 comp rx=0 tx=0).stop 2023-08-27T11:13:03.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 shutdown_connections 2023-08-27T11:13:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f340006a0e0 0x7f340006c5a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f341415b100 0x7f341414b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34140cb2d0 0x7f341414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 --2- 172.21.15.45:0/2148456729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34140cde30 0x7f34140b5360 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 >> 172.21.15.45:0/2148456729 conn(0x7f341406ddd0 msgr2=0x7f34140c70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 shutdown_connections 2023-08-27T11:13:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.792+0000 7f3418800700 1 -- 172.21.15.45:0/2148456729 wait complete. 2023-08-27T11:13:03.805 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:13:03.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.974+0000 7f29b06fe700 1 Processor -- start 2023-08-27T11:13:03.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.974+0000 7f29b06fe700 1 -- start start 2023-08-27T11:13:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29b06fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a8071400 0x7f29a8071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29b06fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29a815b5b0 0x7f29a815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29b06fe700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a80b2290 con 0x7f29a806a280 2023-08-27T11:13:03.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29b06fe700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a80b1960 con 0x7f29a815b5b0 2023-08-27T11:13:03.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29b06fe700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80b1c40 con 0x7f29a8071400 2023-08-27T11:13:03.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29aec9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29a815b5b0 0x7f29a815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29aec9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29a815b5b0 0x7f29a815ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53020/0 (socket says 172.21.15.45:53020) 2023-08-27T11:13:03.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29aec9b700 1 -- 172.21.15.45:0/35693529 learned_addr learned my addr 172.21.15.45:0/35693529 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29adc99700 1 --2- 172.21.15.45:0/35693529 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a8071400 0x7f29a8071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29aec9b700 1 -- 172.21.15.45:0/35693529 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a8071400 msgr2=0x7f29a8071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29aec9b700 1 --2- 172.21.15.45:0/35693529 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a8071400 0x7f29a8071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29aec9b700 1 -- 172.21.15.45:0/35693529 >> v1:172.21.15.135:6789/0 conn(0x7f29a806a280 legacy=0x7f29a806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29aec9b700 1 -- 172.21.15.45:0/35693529 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80b1f20 con 0x7f29a815b5b0 2023-08-27T11:13:03.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29adc99700 1 --2- 172.21.15.45:0/35693529 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a8071400 0x7f29a8071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:03.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.975+0000 7f29aec9b700 1 --2- 172.21.15.45:0/35693529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a815b5b0 0x7f29a815ba40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009e50 tx=0x7f29a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4e80ab6e8a5b68 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:03.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29ad498700 1 -- 172.21.15.45:0/35693529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000e070 con 0x7f29a815b5b0 2023-08-27T11:13:03.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29ad498700 1 -- 172.21.15.45:0/35693529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a000b9c0 con 0x7f29a815b5b0 2023-08-27T11:13:03.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29ad498700 1 -- 172.21.15.45:0/35693529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000bd70 con 0x7f29a815b5b0 2023-08-27T11:13:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29b06fe700 1 -- 172.21.15.45:0/35693529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a815b5b0 msgr2=0x7f29a815ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29b06fe700 1 --2- 172.21.15.45:0/35693529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a815b5b0 0x7f29a815ba40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009e50 tx=0x7f29a000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29b06fe700 1 -- 172.21.15.45:0/35693529 shutdown_connections 2023-08-27T11:13:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29b06fe700 1 --2- 172.21.15.45:0/35693529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a815b5b0 0x7f29a815ba40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29b06fe700 1 --2- 172.21.15.45:0/35693529 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a8071400 0x7f29a8071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29b06fe700 1 -- 172.21.15.45:0/35693529 >> 172.21.15.45:0/35693529 conn(0x7f29a80cc7c0 msgr2=0x7f29a80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:03.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29b06fe700 1 -- 172.21.15.45:0/35693529 shutdown_connections 2023-08-27T11:13:03.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.976+0000 7f29b06fe700 1 -- 172.21.15.45:0/35693529 wait complete. 2023-08-27T11:13:03.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.977+0000 7f29b06fe700 1 Processor -- start 2023-08-27T11:13:03.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.977+0000 7f29b06fe700 1 -- start start 2023-08-27T11:13:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29b06fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a806a280 0x7f29a8152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29b06fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a8071400 0x7f29a80c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29b06fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29a815b5b0 0x7f29a80c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29b06fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8152c90 con 0x7f29a8071400 2023-08-27T11:13:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29b06fe700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8157020 con 0x7f29a815b5b0 2023-08-27T11:13:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29b06fe700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80b3ee0 con 0x7f29a806a280 2023-08-27T11:13:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29adc99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a8071400 0x7f29a80c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29adc99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a8071400 0x7f29a80c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53024/0 (socket says 172.21.15.45:53024) 2023-08-27T11:13:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29adc99700 1 -- 172.21.15.45:0/1936704373 learned_addr learned my addr 172.21.15.45:0/1936704373 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29ae49a700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a806a280 0x7f29a8152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.978+0000 7f29aec9b700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29a815b5b0 0x7f29a80c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f29adc99700 1 -- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a806a280 msgr2=0x7f29a8152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f29adc99700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a806a280 0x7f29a8152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f29adc99700 1 -- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29a815b5b0 msgr2=0x7f29a80c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f29adc99700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29a815b5b0 0x7f29a80c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f29adc99700 1 -- 172.21.15.45:0/1936704373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009b30 con 0x7f29a8071400 2023-08-27T11:13:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f29aec9b700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29a815b5b0 0x7f29a80c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f29adc99700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a8071400 0x7f29a80c37e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f299800e810 tx=0x7f299800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f299effd700 1 -- 172.21.15.45:0/1936704373 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998004230 con 0x7f29a8071400 2023-08-27T11:13:03.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f299effd700 1 -- 172.21.15.45:0/1936704373 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29980043b0 con 0x7f29a8071400 2023-08-27T11:13:03.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a80c4350 con 0x7f29a8071400 2023-08-27T11:13:03.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f299effd700 1 -- 172.21.15.45:0/1936704373 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980038e0 con 0x7f29a8071400 2023-08-27T11:13:03.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.979+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a80c4910 con 0x7f29a8071400 2023-08-27T11:13:03.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.980+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a8072690 con 0x7f29a8071400 2023-08-27T11:13:03.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.980+0000 7f299effd700 1 -- 172.21.15.45:0/1936704373 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2998003a40 con 0x7f29a8071400 2023-08-27T11:13:03.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.981+0000 7f299effd700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f298c06a0e0 0x7f298c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:03.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.981+0000 7f299effd700 1 -- 172.21.15.45:0/1936704373 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2998088200 con 0x7f29a8071400 2023-08-27T11:13:03.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.981+0000 7f29ae49a700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f298c06a0e0 0x7f298c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:03.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.982+0000 7f29ae49a700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f298c06a0e0 0x7f298c06c5a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f29a4007fd0 tx=0x7f29a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:03.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:03.985+0000 7f299effd700 1 -- 172.21.15.45:0/1936704373 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29980570e0 con 0x7f29a8071400 2023-08-27T11:13:04.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.184+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29a80734a0 con 0x7f29a8071400 2023-08-27T11:13:04.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.184+0000 7f299effd700 1 -- 172.21.15.45:0/1936704373 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29980094c0 con 0x7f29a8071400 2023-08-27T11:13:04.186 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f298c06a0e0 msgr2=0x7f298c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f298c06a0e0 0x7f298c06c5a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f29a4007fd0 tx=0x7f29a4009040 comp rx=0 tx=0).stop 2023-08-27T11:13:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a8071400 msgr2=0x7f29a80c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a8071400 0x7f29a80c37e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f299800e810 tx=0x7f299800eb20 comp rx=0 tx=0).stop 2023-08-27T11:13:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 shutdown_connections 2023-08-27T11:13:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f298c06a0e0 0x7f298c06c5a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29a815b5b0 0x7f29a80c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29a806a280 0x7f29a8152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 --2- 172.21.15.45:0/1936704373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29a8071400 0x7f29a80c37e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 >> 172.21.15.45:0/1936704373 conn(0x7f29a80cc7c0 msgr2=0x7f29a81552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 shutdown_connections 2023-08-27T11:13:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.187+0000 7f29b06fe700 1 -- 172.21.15.45:0/1936704373 wait complete. 2023-08-27T11:13:04.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:13:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.327+0000 7f63b4d79700 1 Processor -- start 2023-08-27T11:13:04.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.327+0000 7f63b4d79700 1 -- start start 2023-08-27T11:13:04.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.328+0000 7f63b4d79700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b00d0520 0x7f63b00ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.328+0000 7f63b4d79700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b015ec50 con 0x7f63b00cb5c0 2023-08-27T11:13:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.328+0000 7f63b4d79700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b015eea0 con 0x7f63b00b65d0 2023-08-27T11:13:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.328+0000 7f63b4d79700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b015f180 con 0x7f63b00d0520 2023-08-27T11:13:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.328+0000 7f63aed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f63b00cb5c0 0x7f63b00cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39810/0 (socket says 172.21.15.45:39810) 2023-08-27T11:13:04.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.328+0000 7f63aed9d700 1 -- 172.21.15.45:0/72625202 learned_addr learned my addr 172.21.15.45:0/72625202 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.328+0000 7f63add9b700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b00d0520 0x7f63b00ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.328+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175200173 0 0) 0x7f63b015ec50 con 0x7f63b00cb5c0 2023-08-27T11:13:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.328+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63980035b0 con 0x7f63b00cb5c0 2023-08-27T11:13:04.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 430930696 0 0) 0x7f63980035b0 con 0x7f63b00cb5c0 2023-08-27T11:13:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b00d0520 msgr2=0x7f63b00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b00d0520 0x7f63b00ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 >> v1:172.21.15.135:6789/0 conn(0x7f63b00b65d0 legacy=0x7f63b00cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b015f460 con 0x7f63b00cb5c0 2023-08-27T11:13:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f63a80019f0 con 0x7f63b00cb5c0 2023-08-27T11:13:04.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 >> v1:172.21.15.45:6789/0 conn(0x7f63b00cb5c0 legacy=0x7f63b00cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b00d0520 0x7f6398003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f6398004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6398004d90 0x7f6398005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b015f180 con 0x7f6398003fb0 2023-08-27T11:13:04.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639800bba0 con 0x7f6398004d90 2023-08-27T11:13:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639800bd60 con 0x7f63b00d0520 2023-08-27T11:13:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63add9b700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f6398004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ae59c700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b00d0520 0x7f6398003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63aed9d700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6398004d90 0x7f6398005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63add9b700 1 -- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b00d0520 msgr2=0x7f6398003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63add9b700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b00d0520 0x7f6398003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63add9b700 1 -- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6398004d90 msgr2=0x7f6398005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63add9b700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6398004d90 0x7f6398005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63add9b700 1 -- 172.21.15.45:0/72625202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63980035b0 con 0x7f6398003fb0 2023-08-27T11:13:04.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63add9b700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f6398004850 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009b60 tx=0x7f6398005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92fa196c77773bc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000e6c0 con 0x7f6398003fb0 2023-08-27T11:13:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a000e820 con 0x7f6398003fb0 2023-08-27T11:13:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.329+0000 7f63ad59a700 1 -- 172.21.15.45:0/72625202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000eb90 con 0x7f6398003fb0 2023-08-27T11:13:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.330+0000 7f63b4d79700 1 -- 172.21.15.45:0/72625202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 msgr2=0x7f6398004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.330+0000 7f63b4d79700 1 --2- 172.21.15.45:0/72625202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f6398004850 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009b60 tx=0x7f6398005880 comp rx=0 tx=0).stop 2023-08-27T11:13:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.330+0000 7f63aed9d700 1 -- 172.21.15.45:0/72625202 reap_dead start 2023-08-27T11:13:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.330+0000 7f63b4d79700 1 -- 172.21.15.45:0/72625202 shutdown_connections 2023-08-27T11:13:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.330+0000 7f63b4d79700 1 -- 172.21.15.45:0/72625202 >> 172.21.15.45:0/72625202 conn(0x7f63b006f960 msgr2=0x7f63b006fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.330+0000 7f63b4d79700 1 -- 172.21.15.45:0/72625202 shutdown_connections 2023-08-27T11:13:04.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.330+0000 7f63b4d79700 1 -- 172.21.15.45:0/72625202 wait complete. 2023-08-27T11:13:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.331+0000 7f63b4d79700 1 Processor -- start 2023-08-27T11:13:04.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.331+0000 7f63b4d79700 1 -- start start 2023-08-27T11:13:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63b4d79700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f63b01536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63b4d79700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b01f2c00 0x7f63b006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63b4d79700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63b006d000 0x7f63b00ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63b4d79700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00b0320 con 0x7f6398003fb0 2023-08-27T11:13:04.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63b4d79700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b0153c00 con 0x7f63b006d000 2023-08-27T11:13:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63b4d79700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00af7c0 con 0x7f63b01f2c00 2023-08-27T11:13:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f63b01536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63ae59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f63b01536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53042/0 (socket says 172.21.15.45:53042) 2023-08-27T11:13:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63ae59c700 1 -- 172.21.15.45:0/366159441 learned_addr learned my addr 172.21.15.45:0/366159441 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63add9b700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b01f2c00 0x7f63b006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63aed9d700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63b006d000 0x7f63b00ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63ae59c700 1 -- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b01f2c00 msgr2=0x7f63b006cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63ae59c700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b01f2c00 0x7f63b006cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63ae59c700 1 -- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63b006d000 msgr2=0x7f63b00ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.332+0000 7f63ae59c700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63b006d000 0x7f63b00ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.333+0000 7f63ae59c700 1 -- 172.21.15.45:0/366159441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00afa80 con 0x7f6398003fb0 2023-08-27T11:13:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.333+0000 7f63ae59c700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f63b01536c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e7b0 tx=0x7f63a400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.333+0000 7f639effd700 1 -- 172.21.15.45:0/366159441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40052e0 con 0x7f6398003fb0 2023-08-27T11:13:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.333+0000 7f639effd700 1 -- 172.21.15.45:0/366159441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a4005440 con 0x7f6398003fb0 2023-08-27T11:13:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.333+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b00afd60 con 0x7f6398003fb0 2023-08-27T11:13:04.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.333+0000 7f639effd700 1 -- 172.21.15.45:0/366159441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40038a0 con 0x7f6398003fb0 2023-08-27T11:13:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.333+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b006c7e0 con 0x7f6398003fb0 2023-08-27T11:13:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.334+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b006f4a0 con 0x7f6398003fb0 2023-08-27T11:13:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.334+0000 7f639effd700 1 -- 172.21.15.45:0/366159441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f63a4003a00 con 0x7f6398003fb0 2023-08-27T11:13:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.335+0000 7f639effd700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f639006a0e0 0x7f639006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.335+0000 7f639effd700 1 -- 172.21.15.45:0/366159441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f63a4087f10 con 0x7f6398003fb0 2023-08-27T11:13:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.335+0000 7f63add9b700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f639006a0e0 0x7f639006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.336+0000 7f63add9b700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f639006a0e0 0x7f639006c5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f63b00cb400 tx=0x7f63a0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.339+0000 7f639effd700 1 -- 172.21.15.45:0/366159441 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f63a4053410 con 0x7f6398003fb0 2023-08-27T11:13:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.539+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63b01f63b0 con 0x7f6398003fb0 2023-08-27T11:13:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.539+0000 7f639effd700 1 -- 172.21.15.45:0/366159441 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63a4019020 con 0x7f6398003fb0 2023-08-27T11:13:04.541 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:13:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.541+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f639006a0e0 msgr2=0x7f639006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.541+0000 7f63b4d79700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f639006a0e0 0x7f639006c5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f63b00cb400 tx=0x7f63a0008030 comp rx=0 tx=0).stop 2023-08-27T11:13:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.541+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 msgr2=0x7f63b01536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.541+0000 7f63b4d79700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f63b01536c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e7b0 tx=0x7f63a400eb70 comp rx=0 tx=0).stop 2023-08-27T11:13:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.541+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 shutdown_connections 2023-08-27T11:13:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.541+0000 7f63b4d79700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6398003fb0 0x7f63b01536c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.541+0000 7f63b4d79700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f639006a0e0 0x7f639006c5a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.541+0000 7f63b4d79700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63b006d000 0x7f63b00ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.541+0000 7f63b4d79700 1 --2- 172.21.15.45:0/366159441 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63b01f2c00 0x7f63b006cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.542+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 >> 172.21.15.45:0/366159441 conn(0x7f63b006f960 msgr2=0x7f63b015dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.542+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 shutdown_connections 2023-08-27T11:13:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.542+0000 7f63b4d79700 1 -- 172.21.15.45:0/366159441 wait complete. 2023-08-27T11:13:04.554 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:13:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.682+0000 7f94489f3700 1 Processor -- start 2023-08-27T11:13:04.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.682+0000 7f94489f3700 1 -- start start 2023-08-27T11:13:04.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f94489f3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f944415a2c0 0x7f9444073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f94489f3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f94440b2f60 0x7f944406b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f94489f3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9444073d40 con 0x7f944406b890 2023-08-27T11:13:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f94489f3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94440cf360 con 0x7f944415a2c0 2023-08-27T11:13:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f94489f3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944415a6a0 con 0x7f94440b2f60 2023-08-27T11:13:04.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f944259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f944415a2c0 0x7f9444073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f944259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f944415a2c0 0x7f9444073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53050/0 (socket says 172.21.15.45:53050) 2023-08-27T11:13:04.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f944259c700 1 -- 172.21.15.45:0/620486171 learned_addr learned my addr 172.21.15.45:0/620486171 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:04.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f9441d9b700 1 --2- 172.21.15.45:0/620486171 >> v2:172.21.15.135:3300/0 conn(0x7f94440b2f60 0x7f944406b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f944259c700 1 -- 172.21.15.45:0/620486171 >> v2:172.21.15.135:3300/0 conn(0x7f94440b2f60 msgr2=0x7f944406b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:04.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f944259c700 1 --2- 172.21.15.45:0/620486171 >> v2:172.21.15.135:3300/0 conn(0x7f94440b2f60 0x7f944406b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f944259c700 1 -- 172.21.15.45:0/620486171 >> v1:172.21.15.135:6789/0 conn(0x7f944406b890 legacy=0x7f944406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:04.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f944259c700 1 -- 172.21.15.45:0/620486171 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94440ceda0 con 0x7f944415a2c0 2023-08-27T11:13:04.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.683+0000 7f944259c700 1 --2- 172.21.15.45:0/620486171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944415a2c0 0x7f9444073800 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9444071810 tx=0x7f9434009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557b4c4a4daf8b51 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:04.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f944159a700 1 -- 172.21.15.45:0/620486171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434005e30 con 0x7f944415a2c0 2023-08-27T11:13:04.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f944159a700 1 -- 172.21.15.45:0/620486171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434006030 con 0x7f944415a2c0 2023-08-27T11:13:04.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f944159a700 1 -- 172.21.15.45:0/620486171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94340063e0 con 0x7f944415a2c0 2023-08-27T11:13:04.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f94489f3700 1 -- 172.21.15.45:0/620486171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944415a2c0 msgr2=0x7f9444073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f94489f3700 1 --2- 172.21.15.45:0/620486171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944415a2c0 0x7f9444073800 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9444071810 tx=0x7f9434009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f94489f3700 1 -- 172.21.15.45:0/620486171 shutdown_connections 2023-08-27T11:13:04.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f94489f3700 1 --2- 172.21.15.45:0/620486171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f944415a2c0 0x7f9444073800 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f94489f3700 1 --2- 172.21.15.45:0/620486171 >> v2:172.21.15.135:3300/0 conn(0x7f94440b2f60 0x7f944406b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f94489f3700 1 -- 172.21.15.45:0/620486171 >> 172.21.15.45:0/620486171 conn(0x7f94440b2440 msgr2=0x7f94440b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f94489f3700 1 -- 172.21.15.45:0/620486171 shutdown_connections 2023-08-27T11:13:04.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.684+0000 7f94489f3700 1 -- 172.21.15.45:0/620486171 wait complete. 2023-08-27T11:13:04.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f94489f3700 1 Processor -- start 2023-08-27T11:13:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f94489f3700 1 -- start start 2023-08-27T11:13:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f94489f3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f944406b890 0x7f9444167980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f94489f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94440b2f60 0x7f94440ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f94489f3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944415a2c0 0x7f94440cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f94489f3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94440af8f0 con 0x7f94440b2f60 2023-08-27T11:13:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f94489f3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944406d870 con 0x7f944406b890 2023-08-27T11:13:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f94489f3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94440afae0 con 0x7f944415a2c0 2023-08-27T11:13:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f9441d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94440b2f60 0x7f94440ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f9441d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94440b2f60 0x7f94440ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53058/0 (socket says 172.21.15.45:53058) 2023-08-27T11:13:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.686+0000 7f9441d9b700 1 -- 172.21.15.45:0/3007362221 learned_addr learned my addr 172.21.15.45:0/3007362221 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f944259c700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f944406b890 0x7f9444167980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9442d9d700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944415a2c0 0x7f94440cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9441d9b700 1 -- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944415a2c0 msgr2=0x7f94440cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9441d9b700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944415a2c0 0x7f94440cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9441d9b700 1 -- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f944406b890 msgr2=0x7f9444167980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9441d9b700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f944406b890 0x7f9444167980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9441d9b700 1 -- 172.21.15.45:0/3007362221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9434009500 con 0x7f94440b2f60 2023-08-27T11:13:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9442d9d700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944415a2c0 0x7f94440cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9441d9b700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94440b2f60 0x7f94440ca940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f943800e810 tx=0x7f943800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9432ffd700 1 -- 172.21.15.45:0/3007362221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438013610 con 0x7f94440b2f60 2023-08-27T11:13:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9432ffd700 1 -- 172.21.15.45:0/3007362221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438013770 con 0x7f94440b2f60 2023-08-27T11:13:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94440afdc0 con 0x7f94440b2f60 2023-08-27T11:13:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f9432ffd700 1 -- 172.21.15.45:0/3007362221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438013b20 con 0x7f94440b2f60 2023-08-27T11:13:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.687+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94440b0380 con 0x7f94440b2f60 2023-08-27T11:13:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.688+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94440cea50 con 0x7f94440b2f60 2023-08-27T11:13:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.689+0000 7f9432ffd700 1 -- 172.21.15.45:0/3007362221 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9438013c80 con 0x7f94440b2f60 2023-08-27T11:13:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.689+0000 7f9432ffd700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f942406a0e0 0x7f942406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.689+0000 7f9432ffd700 1 -- 172.21.15.45:0/3007362221 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f94380884e0 con 0x7f94440b2f60 2023-08-27T11:13:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.689+0000 7f944259c700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f942406a0e0 0x7f942406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.690+0000 7f944259c700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f942406a0e0 0x7f942406c5a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9444071810 tx=0x7f9434008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:04.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.693+0000 7f9432ffd700 1 -- 172.21.15.45:0/3007362221 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9438057340 con 0x7f94440b2f60 2023-08-27T11:13:04.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.892+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f944409d4b0 con 0x7f94440b2f60 2023-08-27T11:13:04.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.892+0000 7f9432ffd700 1 -- 172.21.15.45:0/3007362221 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9438056ed0 con 0x7f94440b2f60 2023-08-27T11:13:04.893 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:04.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f942406a0e0 msgr2=0x7f942406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f942406a0e0 0x7f942406c5a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9444071810 tx=0x7f9434008070 comp rx=0 tx=0).stop 2023-08-27T11:13:04.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94440b2f60 msgr2=0x7f94440ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:04.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94440b2f60 0x7f94440ca940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f943800e810 tx=0x7f943800eb20 comp rx=0 tx=0).stop 2023-08-27T11:13:04.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 shutdown_connections 2023-08-27T11:13:04.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f942406a0e0 0x7f942406c5a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944415a2c0 0x7f94440cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f944406b890 0x7f9444167980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 --2- 172.21.15.45:0/3007362221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94440b2f60 0x7f94440ca940 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:04.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 >> 172.21.15.45:0/3007362221 conn(0x7f94440b2440 msgr2=0x7f94440adbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:04.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 shutdown_connections 2023-08-27T11:13:04.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:04.894+0000 7f94489f3700 1 -- 172.21.15.45:0/3007362221 wait complete. 2023-08-27T11:13:04.936 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-08-27T11:13:04.937 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-27T11:13:04.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-08-27T11:13:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.062+0000 7f8ada76f700 1 Processor -- start 2023-08-27T11:13:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.062+0000 7f8ada76f700 1 -- start start 2023-08-27T11:13:05.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ada76f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ad415d3d0 0x7f8ad41619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:05.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ada76f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8ad40af040 0x7f8ad40cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:05.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ada76f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad4161550 con 0x7f8ad406e3a0 2023-08-27T11:13:05.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ada76f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad40cdf90 con 0x7f8ad415d3d0 2023-08-27T11:13:05.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ada76f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad4161f20 con 0x7f8ad40af040 2023-08-27T11:13:05.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ad415d3d0 0x7f8ad41619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:05.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ad415d3d0 0x7f8ad41619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53064/0 (socket says 172.21.15.45:53064) 2023-08-27T11:13:05.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad3fff700 1 -- 172.21.15.45:0/2075201760 learned_addr learned my addr 172.21.15.45:0/2075201760 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad8d0c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8ad406e3a0 0x7f8ad4070e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39842/0 (socket says 172.21.15.45:39842) 2023-08-27T11:13:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad37fe700 1 --2- 172.21.15.45:0/2075201760 >> v2:172.21.15.135:3300/0 conn(0x7f8ad40af040 0x7f8ad40cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad3fff700 1 -- 172.21.15.45:0/2075201760 >> v2:172.21.15.135:3300/0 conn(0x7f8ad40af040 msgr2=0x7f8ad40cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad3fff700 1 --2- 172.21.15.45:0/2075201760 >> v2:172.21.15.135:3300/0 conn(0x7f8ad40af040 0x7f8ad40cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad3fff700 1 -- 172.21.15.45:0/2075201760 >> v1:172.21.15.45:6789/0 conn(0x7f8ad406e3a0 legacy=0x7f8ad4070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad3fff700 1 -- 172.21.15.45:0/2075201760 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40cd9d0 con 0x7f8ad415d3d0 2023-08-27T11:13:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.063+0000 7f8ad2ffd700 1 -- 172.21.15.45:0/2075201760 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196448543 0 0) 0x7f8ad4161550 con 0x7f8ad406e3a0 2023-08-27T11:13:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ad3fff700 1 --2- 172.21.15.45:0/2075201760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad415d3d0 0x7f8ad41619e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40c35c0 tx=0x7f8ac4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef37b532a7dd273 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ad2ffd700 1 -- 172.21.15.45:0/2075201760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac400b870 con 0x7f8ad415d3d0 2023-08-27T11:13:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ad2ffd700 1 -- 172.21.15.45:0/2075201760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac400b9d0 con 0x7f8ad415d3d0 2023-08-27T11:13:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ad2ffd700 1 -- 172.21.15.45:0/2075201760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac400bd60 con 0x7f8ad415d3d0 2023-08-27T11:13:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ada76f700 1 -- 172.21.15.45:0/2075201760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad415d3d0 msgr2=0x7f8ad41619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ada76f700 1 --2- 172.21.15.45:0/2075201760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad415d3d0 0x7f8ad41619e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40c35c0 tx=0x7f8ac4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ada76f700 1 -- 172.21.15.45:0/2075201760 shutdown_connections 2023-08-27T11:13:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ada76f700 1 --2- 172.21.15.45:0/2075201760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad415d3d0 0x7f8ad41619e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ada76f700 1 --2- 172.21.15.45:0/2075201760 >> v2:172.21.15.135:3300/0 conn(0x7f8ad40af040 0x7f8ad40cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ada76f700 1 -- 172.21.15.45:0/2075201760 >> 172.21.15.45:0/2075201760 conn(0x7f8ad40cfd60 msgr2=0x7f8ad40d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ada76f700 1 -- 172.21.15.45:0/2075201760 shutdown_connections 2023-08-27T11:13:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.064+0000 7f8ada76f700 1 -- 172.21.15.45:0/2075201760 wait complete. 2023-08-27T11:13:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.066+0000 7f8ada76f700 1 Processor -- start 2023-08-27T11:13:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.066+0000 7f8ada76f700 1 -- start start 2023-08-27T11:13:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ada76f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad406e3a0 0x7f8ad4155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ada76f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ad40af040 0x7f8ad40c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ada76f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad415d3d0 0x7f8ad406ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:05.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ada76f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40af8d0 con 0x7f8ad406e3a0 2023-08-27T11:13:05.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ada76f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40c6800 con 0x7f8ad415d3d0 2023-08-27T11:13:05.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ada76f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad415d7b0 con 0x7f8ad40af040 2023-08-27T11:13:05.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad406e3a0 0x7f8ad4155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:05.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad406e3a0 0x7f8ad4155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53068/0 (socket says 172.21.15.45:53068) 2023-08-27T11:13:05.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad3fff700 1 -- 172.21.15.45:0/2614956586 learned_addr learned my addr 172.21.15.45:0/2614956586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:05.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad37fe700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ad40af040 0x7f8ad40c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:05.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad8d0c700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad415d3d0 0x7f8ad406ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:05.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad3fff700 1 -- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ad40af040 msgr2=0x7f8ad40c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:05.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad3fff700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ad40af040 0x7f8ad40c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:05.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad3fff700 1 -- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad415d3d0 msgr2=0x7f8ad406ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:05.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad3fff700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad415d3d0 0x7f8ad406ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:05.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad3fff700 1 -- 172.21.15.45:0/2614956586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4009550 con 0x7f8ad406e3a0 2023-08-27T11:13:05.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad3fff700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad406e3a0 0x7f8ad4155be0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009c10 tx=0x7f8ac4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:05.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad0ff9700 1 -- 172.21.15.45:0/2614956586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4004310 con 0x7f8ad406e3a0 2023-08-27T11:13:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad0ff9700 1 -- 172.21.15.45:0/2614956586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac401f070 con 0x7f8ad406e3a0 2023-08-27T11:13:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ad0ff9700 1 -- 172.21.15.45:0/2614956586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40036a0 con 0x7f8ad406e3a0 2023-08-27T11:13:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad40b5210 con 0x7f8ad406e3a0 2023-08-27T11:13:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.067+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad40b57d0 con 0x7f8ad406e3a0 2023-08-27T11:13:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.068+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad4072780 con 0x7f8ad406e3a0 2023-08-27T11:13:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.069+0000 7f8ad0ff9700 1 -- 172.21.15.45:0/2614956586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8ac4021030 con 0x7f8ad406e3a0 2023-08-27T11:13:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.069+0000 7f8ad0ff9700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8ac006a0e0 0x7f8ac006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.069+0000 7f8ad0ff9700 1 -- 172.21.15.45:0/2614956586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8ac4089b70 con 0x7f8ad406e3a0 2023-08-27T11:13:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.069+0000 7f8ad37fe700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8ac006a0e0 0x7f8ac006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.070+0000 7f8ad37fe700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8ac006a0e0 0x7f8ac006c5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40c6250 tx=0x7f8ac8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:05.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.074+0000 7f8ad0ff9700 1 -- 172.21.15.45:0/2614956586 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8ac4054fc0 con 0x7f8ad406e3a0 2023-08-27T11:13:05.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:05.362+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f8ad40734a0 con 0x7f8ad406e3a0 2023-08-27T11:13:06.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.119+0000 7f8ad0ff9700 1 -- 172.21.15.45:0/2614956586 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8ac40150e0 con 0x7f8ad406e3a0 2023-08-27T11:13:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8ac006a0e0 msgr2=0x7f8ac006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8ac006a0e0 0x7f8ac006c5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40c6250 tx=0x7f8ac8008040 comp rx=0 tx=0).stop 2023-08-27T11:13:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad406e3a0 msgr2=0x7f8ad4155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad406e3a0 0x7f8ad4155be0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009c10 tx=0x7f8ac4004750 comp rx=0 tx=0).stop 2023-08-27T11:13:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 shutdown_connections 2023-08-27T11:13:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad406e3a0 0x7f8ad4155be0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8ac006a0e0 0x7f8ac006c5a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad415d3d0 0x7f8ad406ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 --2- 172.21.15.45:0/2614956586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ad40af040 0x7f8ad40c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 >> 172.21.15.45:0/2614956586 conn(0x7f8ad40cfd60 msgr2=0x7f8ad40b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:06.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.122+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 shutdown_connections 2023-08-27T11:13:06.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.123+0000 7f8ada76f700 1 -- 172.21.15.45:0/2614956586 wait complete. 2023-08-27T11:13:06.139 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:06.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.270+0000 7fbfa47ea700 1 Processor -- start 2023-08-27T11:13:06.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.270+0000 7fbfa47ea700 1 -- start start 2023-08-27T11:13:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa47ea700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbf9c15a240 0x7fbf9c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa47ea700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf9c0cbe70 0x7fbf9c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa47ea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ce9d0 0x7fbf9c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa47ea700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c06d690 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa47ea700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c0cee20 con 0x7fbf9c15a240 2023-08-27T11:13:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa47ea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0cc440 con 0x7fbf9c0ce9d0 2023-08-27T11:13:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa1d85700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf9c0cbe70 0x7fbf9c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa1d85700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf9c0cbe70 0x7fbf9c0ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53076/0 (socket says 172.21.15.45:53076) 2023-08-27T11:13:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa1d85700 1 -- 172.21.15.45:0/4019945234 learned_addr learned my addr 172.21.15.45:0/4019945234 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa2d87700 1 --2- 172.21.15.45:0/4019945234 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ce9d0 0x7fbf9c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.271+0000 7fbfa2586700 1 --2- 172.21.15.45:0/4019945234 >> v2:172.21.15.135:3300/0 conn(0x7fbf9c15a240 0x7fbf9c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa1d85700 1 -- 172.21.15.45:0/4019945234 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ce9d0 msgr2=0x7fbf9c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:06.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa1d85700 1 --2- 172.21.15.45:0/4019945234 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ce9d0 0x7fbf9c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa1d85700 1 -- 172.21.15.45:0/4019945234 >> v2:172.21.15.135:3300/0 conn(0x7fbf9c15a240 msgr2=0x7fbf9c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa1d85700 1 --2- 172.21.15.45:0/4019945234 >> v2:172.21.15.135:3300/0 conn(0x7fbf9c15a240 0x7fbf9c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa1d85700 1 -- 172.21.15.45:0/4019945234 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c06d040 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa1d85700 1 --2- 172.21.15.45:0/4019945234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 0x7fbf9c0ce490 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0b2950 tx=0x7fbf98009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df3b5b23e6ef89a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:06.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa1584700 1 -- 172.21.15.45:0/4019945234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800b870 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa1584700 1 -- 172.21.15.45:0/4019945234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9800b9d0 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa1584700 1 -- 172.21.15.45:0/4019945234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800bd60 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa47ea700 1 -- 172.21.15.45:0/4019945234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 msgr2=0x7fbf9c0ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.272+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/4019945234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 0x7fbf9c0ce490 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0b2950 tx=0x7fbf98009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.273+0000 7fbfa47ea700 1 -- 172.21.15.45:0/4019945234 shutdown_connections 2023-08-27T11:13:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.273+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/4019945234 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ce9d0 0x7fbf9c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.273+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/4019945234 >> v2:172.21.15.135:3300/0 conn(0x7fbf9c15a240 0x7fbf9c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.273+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/4019945234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 0x7fbf9c0ce490 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.273+0000 7fbfa47ea700 1 -- 172.21.15.45:0/4019945234 >> 172.21.15.45:0/4019945234 conn(0x7fbf9c0cfe50 msgr2=0x7fbf9c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.273+0000 7fbfa47ea700 1 -- 172.21.15.45:0/4019945234 shutdown_connections 2023-08-27T11:13:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.273+0000 7fbfa47ea700 1 -- 172.21.15.45:0/4019945234 wait complete. 2023-08-27T11:13:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.274+0000 7fbfa47ea700 1 Processor -- start 2023-08-27T11:13:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.274+0000 7fbfa47ea700 1 -- start start 2023-08-27T11:13:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa47ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 0x7fbf9c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa47ea700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0ce9d0 0x7fbf9c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa47ea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c15a240 0x7fbf9c0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa47ea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c154690 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa47ea700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0c54d0 con 0x7fbf9c0ce9d0 2023-08-27T11:13:06.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa47ea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0c87c0 con 0x7fbf9c15a240 2023-08-27T11:13:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2586700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 0x7fbf9c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2586700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 0x7fbf9c150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53082/0 (socket says 172.21.15.45:53082) 2023-08-27T11:13:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2586700 1 -- 172.21.15.45:0/926209053 learned_addr learned my addr 172.21.15.45:0/926209053 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa1d85700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0ce9d0 0x7fbf9c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2d87700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c15a240 0x7fbf9c0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2586700 1 -- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c15a240 msgr2=0x7fbf9c0c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2586700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c15a240 0x7fbf9c0c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2586700 1 -- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0ce9d0 msgr2=0x7fbf9c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2586700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0ce9d0 0x7fbf9c154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2586700 1 -- 172.21.15.45:0/926209053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf98009550 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa1d85700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0ce9d0 0x7fbf9c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa2586700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 0x7fbf9c150ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00d800 tx=0x7fbf8c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbf92ffd700 1 -- 172.21.15.45:0/926209053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0125f0 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbf92ffd700 1 -- 172.21.15.45:0/926209053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c012750 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c0c4ba0 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbf92ffd700 1 -- 172.21.15.45:0/926209053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c012b00 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.275+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c0c5160 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.276+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c072810 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.277+0000 7fbf92ffd700 1 -- 172.21.15.45:0/926209053 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbf8c012c60 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.277+0000 7fbf92ffd700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.277+0000 7fbf92ffd700 1 -- 172.21.15.45:0/926209053 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbf8c088b50 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.277+0000 7fbfa1d85700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.278+0000 7fbfa1d85700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009130 tx=0x7fbf98008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.282+0000 7fbf92ffd700 1 -- 172.21.15.45:0/926209053 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf8c053fa0 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.484+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf9c0734a0 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.485+0000 7fbf92ffd700 1 -- 172.21.15.45:0/926209053 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7fbf8c0575c0 con 0x7fbf9c0cbe70 2023-08-27T11:13:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:06.486 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 msgr2=0x7fbf8006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009130 tx=0x7fbf98008030 comp rx=0 tx=0).stop 2023-08-27T11:13:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 msgr2=0x7fbf9c150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 0x7fbf9c150ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00d800 tx=0x7fbf8c00dbc0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 shutdown_connections 2023-08-27T11:13:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0cbe70 0x7fbf9c150ab0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c15a240 0x7fbf9c0c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 --2- 172.21.15.45:0/926209053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0ce9d0 0x7fbf9c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 >> 172.21.15.45:0/926209053 conn(0x7fbf9c0cfe50 msgr2=0x7fbf9c0b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 shutdown_connections 2023-08-27T11:13:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.487+0000 7fbfa47ea700 1 -- 172.21.15.45:0/926209053 wait complete. 2023-08-27T11:13:06.500 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:13:06.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:13:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.628+0000 7f5495cbb700 1 Processor -- start 2023-08-27T11:13:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.628+0000 7f5495cbb700 1 -- start start 2023-08-27T11:13:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f5495cbb700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f54900ca3b0 0x7f54900ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f5495cbb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f549015d600 0x7f5490160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f5495cbb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 0x7f5490161430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f5495cbb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54900cacd0 con 0x7f549015d600 2023-08-27T11:13:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f5495cbb700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54900af270 con 0x7f54900ca3b0 2023-08-27T11:13:06.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f5495cbb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900af550 con 0x7f54900aed20 2023-08-27T11:13:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f549015d600 0x7f5490160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f549015d600 0x7f5490160ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53092/0 (socket says 172.21.15.45:53092) 2023-08-27T11:13:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548effd700 1 -- 172.21.15.45:0/399373828 learned_addr learned my addr 172.21.15.45:0/399373828 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548ffff700 1 --2- 172.21.15.45:0/399373828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 0x7f5490161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548f7fe700 1 --2- 172.21.15.45:0/399373828 >> v2:172.21.15.135:3300/0 conn(0x7f54900ca3b0 0x7f54900ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548effd700 1 -- 172.21.15.45:0/399373828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 msgr2=0x7f5490161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548effd700 1 --2- 172.21.15.45:0/399373828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 0x7f5490161430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548effd700 1 -- 172.21.15.45:0/399373828 >> v2:172.21.15.135:3300/0 conn(0x7f54900ca3b0 msgr2=0x7f54900ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548effd700 1 --2- 172.21.15.45:0/399373828 >> v2:172.21.15.135:3300/0 conn(0x7f54900ca3b0 0x7f54900ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.629+0000 7f548effd700 1 -- 172.21.15.45:0/399373828 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549006ac90 con 0x7f549015d600 2023-08-27T11:13:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f548effd700 1 --2- 172.21.15.45:0/399373828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f549015d600 0x7f5490160ef0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f54900cd8e0 tx=0x7f5480009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e4bb21f3e1de95 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f548e7fc700 1 -- 172.21.15.45:0/399373828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000b870 con 0x7f549015d600 2023-08-27T11:13:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f548e7fc700 1 -- 172.21.15.45:0/399373828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548000b9d0 con 0x7f549015d600 2023-08-27T11:13:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f548e7fc700 1 -- 172.21.15.45:0/399373828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000bd60 con 0x7f549015d600 2023-08-27T11:13:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f5495cbb700 1 -- 172.21.15.45:0/399373828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f549015d600 msgr2=0x7f5490160ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f5495cbb700 1 --2- 172.21.15.45:0/399373828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f549015d600 0x7f5490160ef0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f54900cd8e0 tx=0x7f5480009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f5495cbb700 1 -- 172.21.15.45:0/399373828 shutdown_connections 2023-08-27T11:13:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f5495cbb700 1 --2- 172.21.15.45:0/399373828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 0x7f5490161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f5495cbb700 1 --2- 172.21.15.45:0/399373828 >> v2:172.21.15.135:3300/0 conn(0x7f54900ca3b0 0x7f54900ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f5495cbb700 1 --2- 172.21.15.45:0/399373828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f549015d600 0x7f5490160ef0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f5495cbb700 1 -- 172.21.15.45:0/399373828 >> 172.21.15.45:0/399373828 conn(0x7f54900cc7e0 msgr2=0x7f54900ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f5495cbb700 1 -- 172.21.15.45:0/399373828 shutdown_connections 2023-08-27T11:13:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.630+0000 7f5495cbb700 1 -- 172.21.15.45:0/399373828 wait complete. 2023-08-27T11:13:06.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.632+0000 7f5495cbb700 1 Processor -- start 2023-08-27T11:13:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.632+0000 7f5495cbb700 1 -- start start 2023-08-27T11:13:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.632+0000 7f5495cbb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 0x7f54901f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.632+0000 7f5495cbb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54900ca3b0 0x7f54900c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.632+0000 7f5495cbb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54900c7840 0x7f54901f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.632+0000 7f5495cbb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54900c8930 con 0x7f54900ca3b0 2023-08-27T11:13:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.632+0000 7f5495cbb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54901f0830 con 0x7f54900c7840 2023-08-27T11:13:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.632+0000 7f5495cbb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900c7db0 con 0x7f54900aed20 2023-08-27T11:13:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54900ca3b0 0x7f54900c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54900ca3b0 0x7f54900c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53098/0 (socket says 172.21.15.45:53098) 2023-08-27T11:13:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548effd700 1 -- 172.21.15.45:0/2634100139 learned_addr learned my addr 172.21.15.45:0/2634100139 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548ffff700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54900c7840 0x7f54901f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548f7fe700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 0x7f54901f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548effd700 1 -- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 msgr2=0x7f54901f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548effd700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 0x7f54901f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548effd700 1 -- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54900c7840 msgr2=0x7f54901f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548effd700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54900c7840 0x7f54901f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548effd700 1 -- 172.21.15.45:0/2634100139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480009550 con 0x7f54900ca3b0 2023-08-27T11:13:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548ffff700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54900c7840 0x7f54901f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f548effd700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54900ca3b0 0x7f54900c8b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5480000c00 tx=0x7f5480004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f5487fff700 1 -- 172.21.15.45:0/2634100139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800129c0 con 0x7f54900ca3b0 2023-08-27T11:13:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f5487fff700 1 -- 172.21.15.45:0/2634100139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548001e070 con 0x7f54900ca3b0 2023-08-27T11:13:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f5487fff700 1 -- 172.21.15.45:0/2634100139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480012d40 con 0x7f54900ca3b0 2023-08-27T11:13:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54900c8090 con 0x7f54900ca3b0 2023-08-27T11:13:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.633+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54900c8650 con 0x7f54900ca3b0 2023-08-27T11:13:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.634+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54900726d0 con 0x7f54900ca3b0 2023-08-27T11:13:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.635+0000 7f5487fff700 1 -- 172.21.15.45:0/2634100139 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f548002d450 con 0x7f54900ca3b0 2023-08-27T11:13:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.635+0000 7f5487fff700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f547006a0e0 0x7f547006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.635+0000 7f5487fff700 1 -- 172.21.15.45:0/2634100139 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5480089910 con 0x7f54900ca3b0 2023-08-27T11:13:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.635+0000 7f548f7fe700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f547006a0e0 0x7f547006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.636+0000 7f548f7fe700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f547006a0e0 0x7f547006c5a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5478009950 tx=0x7f5478008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.640+0000 7f5487fff700 1 -- 172.21.15.45:0/2634100139 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5480054d60 con 0x7f54900ca3b0 2023-08-27T11:13:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.841+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54900734a0 con 0x7f54900ca3b0 2023-08-27T11:13:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.841+0000 7f5487fff700 1 -- 172.21.15.45:0/2634100139 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5480058380 con 0x7f54900ca3b0 2023-08-27T11:13:06.842 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:13:06.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f547006a0e0 msgr2=0x7f547006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f547006a0e0 0x7f547006c5a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5478009950 tx=0x7f5478008040 comp rx=0 tx=0).stop 2023-08-27T11:13:06.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54900ca3b0 msgr2=0x7f54900c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54900ca3b0 0x7f54900c8b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5480000c00 tx=0x7f5480004750 comp rx=0 tx=0).stop 2023-08-27T11:13:06.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 shutdown_connections 2023-08-27T11:13:06.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f547006a0e0 0x7f547006c5a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54900c7840 0x7f54901f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54900aed20 0x7f54901f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 --2- 172.21.15.45:0/2634100139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54900ca3b0 0x7f54900c8b70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 >> 172.21.15.45:0/2634100139 conn(0x7f54900cc7e0 msgr2=0x7f54900afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:06.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 shutdown_connections 2023-08-27T11:13:06.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.843+0000 7f5495cbb700 1 -- 172.21.15.45:0/2634100139 wait complete. 2023-08-27T11:13:06.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:13:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.983+0000 7f23e7cda700 1 Processor -- start 2023-08-27T11:13:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.983+0000 7f23e7cda700 1 -- start start 2023-08-27T11:13:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.984+0000 7f23e7cda700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f23e0157be0 0x7f23e00716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.984+0000 7f23e7cda700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23e006d5e0 0x7f23e00b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.984+0000 7f23e7cda700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e015b460 con 0x7f23e00b17c0 2023-08-27T11:13:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.984+0000 7f23e7cda700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e0157fc0 con 0x7f23e006d5e0 2023-08-27T11:13:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.984+0000 7f23e7cda700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e00b2280 con 0x7f23e0157be0 2023-08-27T11:13:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.984+0000 7f23e6277700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23e006d5e0 0x7f23e00b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.984+0000 7f23e6277700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23e006d5e0 0x7f23e00b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53106/0 (socket says 172.21.15.45:53106) 2023-08-27T11:13:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.984+0000 7f23e6277700 1 -- 172.21.15.45:0/69550798 learned_addr learned my addr 172.21.15.45:0/69550798 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.985+0000 7f23e5a76700 1 --2- 172.21.15.45:0/69550798 >> v2:172.21.15.135:3300/0 conn(0x7f23e0157be0 0x7f23e00716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.985+0000 7f23e6277700 1 -- 172.21.15.45:0/69550798 >> v2:172.21.15.135:3300/0 conn(0x7f23e0157be0 msgr2=0x7f23e00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.985+0000 7f23e6277700 1 --2- 172.21.15.45:0/69550798 >> v2:172.21.15.135:3300/0 conn(0x7f23e0157be0 0x7f23e00716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.985+0000 7f23e6277700 1 -- 172.21.15.45:0/69550798 >> v1:172.21.15.135:6789/0 conn(0x7f23e00b17c0 legacy=0x7f23e006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.985+0000 7f23e6277700 1 -- 172.21.15.45:0/69550798 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e00b2560 con 0x7f23e006d5e0 2023-08-27T11:13:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.985+0000 7f23e6277700 1 --2- 172.21.15.45:0/69550798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e006d5e0 0x7f23e00b1c70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f23e0066e70 tx=0x7f23d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5841f257423e84f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.985+0000 7f23e4a74700 1 -- 172.21.15.45:0/69550798 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800e070 con 0x7f23e006d5e0 2023-08-27T11:13:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.985+0000 7f23e4a74700 1 -- 172.21.15.45:0/69550798 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d800b9c0 con 0x7f23e006d5e0 2023-08-27T11:13:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.985+0000 7f23e4a74700 1 -- 172.21.15.45:0/69550798 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800bd70 con 0x7f23e006d5e0 2023-08-27T11:13:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.986+0000 7f23e7cda700 1 -- 172.21.15.45:0/69550798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e006d5e0 msgr2=0x7f23e00b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.986+0000 7f23e7cda700 1 --2- 172.21.15.45:0/69550798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e006d5e0 0x7f23e00b1c70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f23e0066e70 tx=0x7f23d800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.986+0000 7f23e7cda700 1 -- 172.21.15.45:0/69550798 shutdown_connections 2023-08-27T11:13:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.986+0000 7f23e7cda700 1 --2- 172.21.15.45:0/69550798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e006d5e0 0x7f23e00b1c70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.986+0000 7f23e7cda700 1 --2- 172.21.15.45:0/69550798 >> v2:172.21.15.135:3300/0 conn(0x7f23e0157be0 0x7f23e00716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.986+0000 7f23e7cda700 1 -- 172.21.15.45:0/69550798 >> 172.21.15.45:0/69550798 conn(0x7f23e00cc7c0 msgr2=0x7f23e00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:06.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.986+0000 7f23e7cda700 1 -- 172.21.15.45:0/69550798 shutdown_connections 2023-08-27T11:13:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.986+0000 7f23e7cda700 1 -- 172.21.15.45:0/69550798 wait complete. 2023-08-27T11:13:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.987+0000 7f23e7cda700 1 Processor -- start 2023-08-27T11:13:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.987+0000 7f23e7cda700 1 -- start start 2023-08-27T11:13:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e7cda700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23e006d5e0 0x7f23e00c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e7cda700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e00b17c0 0x7f23e01e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e7cda700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23e0157be0 0x7f23e01e7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e7cda700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00c4a20 con 0x7f23e00b17c0 2023-08-27T11:13:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e7cda700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00c4d00 con 0x7f23e006d5e0 2023-08-27T11:13:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e7cda700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00c4fe0 con 0x7f23e0157be0 2023-08-27T11:13:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5275700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e00b17c0 0x7f23e01e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5275700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e00b17c0 0x7f23e01e6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53110/0 (socket says 172.21.15.45:53110) 2023-08-27T11:13:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5275700 1 -- 172.21.15.45:0/2955165891 learned_addr learned my addr 172.21.15.45:0/2955165891 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5a76700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23e006d5e0 0x7f23e00c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e6277700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23e0157be0 0x7f23e01e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5275700 1 -- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23e0157be0 msgr2=0x7f23e01e7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5275700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23e0157be0 0x7f23e01e7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5275700 1 -- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23e006d5e0 msgr2=0x7f23e00c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5275700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23e006d5e0 0x7f23e00c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5275700 1 -- 172.21.15.45:0/2955165891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8009b30 con 0x7f23e00b17c0 2023-08-27T11:13:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5a76700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23e006d5e0 0x7f23e00c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23e5275700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e00b17c0 0x7f23e01e6b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00c450 tx=0x7f23dc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.988+0000 7f23d67fc700 1 -- 172.21.15.45:0/2955165891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc0052d0 con 0x7f23e00b17c0 2023-08-27T11:13:07.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.989+0000 7f23d67fc700 1 -- 172.21.15.45:0/2955165891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23dc005430 con 0x7f23e00b17c0 2023-08-27T11:13:07.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.989+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e00c52c0 con 0x7f23e00b17c0 2023-08-27T11:13:07.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.989+0000 7f23d67fc700 1 -- 172.21.15.45:0/2955165891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc0038c0 con 0x7f23e00b17c0 2023-08-27T11:13:07.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.989+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e0154e50 con 0x7f23e00b17c0 2023-08-27T11:13:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.990+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e00c57e0 con 0x7f23e00b17c0 2023-08-27T11:13:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.991+0000 7f23d67fc700 1 -- 172.21.15.45:0/2955165891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f23dc003a20 con 0x7f23e00b17c0 2023-08-27T11:13:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.991+0000 7f23d67fc700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23c406a0e0 0x7f23c406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.991+0000 7f23d67fc700 1 -- 172.21.15.45:0/2955165891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f23dc087eb0 con 0x7f23e00b17c0 2023-08-27T11:13:07.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.992+0000 7f23e5a76700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23c406a0e0 0x7f23c406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.992+0000 7f23e5a76700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23c406a0e0 0x7f23c406c5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f23e0072640 tx=0x7f23d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:07.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:06.995+0000 7f23d67fc700 1 -- 172.21.15.45:0/2955165891 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f23dc053350 con 0x7f23e00b17c0 2023-08-27T11:13:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.196+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f23e00c57e0 con 0x7f23e00b17c0 2023-08-27T11:13:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.196+0000 7f23d67fc700 1 -- 172.21.15.45:0/2955165891 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f23e00c57e0 con 0x7f23e00b17c0 2023-08-27T11:13:07.198 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:13:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23c406a0e0 msgr2=0x7f23c406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23c406a0e0 0x7f23c406c5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f23e0072640 tx=0x7f23d0008040 comp rx=0 tx=0).stop 2023-08-27T11:13:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e00b17c0 msgr2=0x7f23e01e6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e00b17c0 0x7f23e01e6b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00c450 tx=0x7f23dc00c810 comp rx=0 tx=0).stop 2023-08-27T11:13:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 shutdown_connections 2023-08-27T11:13:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f23c406a0e0 0x7f23c406c5a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23e0157be0 0x7f23e01e7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23e006d5e0 0x7f23e00c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 --2- 172.21.15.45:0/2955165891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e00b17c0 0x7f23e01e6b10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 >> 172.21.15.45:0/2955165891 conn(0x7f23e00cc7c0 msgr2=0x7f23e00b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 shutdown_connections 2023-08-27T11:13:07.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.198+0000 7f23e7cda700 1 -- 172.21.15.45:0/2955165891 wait complete. 2023-08-27T11:13:07.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:13:07.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.339+0000 7f9a98eb9700 1 Processor -- start 2023-08-27T11:13:07.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.339+0000 7f9a98eb9700 1 -- start start 2023-08-27T11:13:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a98eb9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a9406a280 0x7f9a940cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a98eb9700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9a940ac1a0 0x7f9a940ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a98eb9700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9415b930 con 0x7f9a9406a660 2023-08-27T11:13:07.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a98eb9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a940cdc80 con 0x7f9a9406a280 2023-08-27T11:13:07.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a98eb9700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a940cdf20 con 0x7f9a940ac1a0 2023-08-27T11:13:07.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a9259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a9406a280 0x7f9a940cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a9259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a9406a280 0x7f9a940cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53116/0 (socket says 172.21.15.45:53116) 2023-08-27T11:13:07.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a9259c700 1 -- 172.21.15.45:0/1896623170 learned_addr learned my addr 172.21.15.45:0/1896623170 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:07.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a9259c700 1 -- 172.21.15.45:0/1896623170 >> v2:172.21.15.135:3300/0 conn(0x7f9a940ac1a0 msgr2=0x7f9a940ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:07.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a92d9d700 1 --2- 172.21.15.45:0/1896623170 >> v2:172.21.15.135:3300/0 conn(0x7f9a940ac1a0 0x7f9a940ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a9259c700 1 --2- 172.21.15.45:0/1896623170 >> v2:172.21.15.135:3300/0 conn(0x7f9a940ac1a0 0x7f9a940ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a9259c700 1 -- 172.21.15.45:0/1896623170 >> v1:172.21.15.135:6789/0 conn(0x7f9a9406a660 legacy=0x7f9a940cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a9259c700 1 -- 172.21.15.45:0/1896623170 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940ce200 con 0x7f9a9406a280 2023-08-27T11:13:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a92d9d700 1 --2- 172.21.15.45:0/1896623170 >> v2:172.21.15.135:3300/0 conn(0x7f9a940ac1a0 0x7f9a940ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.340+0000 7f9a9259c700 1 --2- 172.21.15.45:0/1896623170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9406a280 0x7f9a940cd600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9a940c6510 tx=0x7f9a84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa863faf1da1f85 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:07.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a9159a700 1 -- 172.21.15.45:0/1896623170 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004230 con 0x7f9a9406a280 2023-08-27T11:13:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a9159a700 1 -- 172.21.15.45:0/1896623170 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a84004390 con 0x7f9a9406a280 2023-08-27T11:13:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a9159a700 1 -- 172.21.15.45:0/1896623170 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a840036a0 con 0x7f9a9406a280 2023-08-27T11:13:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a98eb9700 1 -- 172.21.15.45:0/1896623170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9406a280 msgr2=0x7f9a940cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a98eb9700 1 --2- 172.21.15.45:0/1896623170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9406a280 0x7f9a940cd600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9a940c6510 tx=0x7f9a84009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a98eb9700 1 -- 172.21.15.45:0/1896623170 shutdown_connections 2023-08-27T11:13:07.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a98eb9700 1 --2- 172.21.15.45:0/1896623170 >> v2:172.21.15.135:3300/0 conn(0x7f9a940ac1a0 0x7f9a940ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a98eb9700 1 --2- 172.21.15.45:0/1896623170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9406a280 0x7f9a940cd600 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a98eb9700 1 -- 172.21.15.45:0/1896623170 >> 172.21.15.45:0/1896623170 conn(0x7f9a940b23e0 msgr2=0x7f9a940b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:07.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a98eb9700 1 -- 172.21.15.45:0/1896623170 shutdown_connections 2023-08-27T11:13:07.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.341+0000 7f9a98eb9700 1 -- 172.21.15.45:0/1896623170 wait complete. 2023-08-27T11:13:07.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.343+0000 7f9a98eb9700 1 Processor -- start 2023-08-27T11:13:07.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.343+0000 7f9a98eb9700 1 -- start start 2023-08-27T11:13:07.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.344+0000 7f9a98eb9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a9406a280 0x7f9a9409b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.344+0000 7f9a98eb9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a9406a660 0x7f9a9409ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.344+0000 7f9a98eb9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a940ac1a0 0x7f9a941583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.344+0000 7f9a98eb9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941f19d0 con 0x7f9a940ac1a0 2023-08-27T11:13:07.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.344+0000 7f9a98eb9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940b1820 con 0x7f9a9406a660 2023-08-27T11:13:07.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.344+0000 7f9a98eb9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941f0e20 con 0x7f9a9406a280 2023-08-27T11:13:07.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.344+0000 7f9a92d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a940ac1a0 0x7f9a941583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.344+0000 7f9a92d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a940ac1a0 0x7f9a941583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53126/0 (socket says 172.21.15.45:53126) 2023-08-27T11:13:07.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.344+0000 7f9a92d9d700 1 -- 172.21.15.45:0/4068111552 learned_addr learned my addr 172.21.15.45:0/4068111552 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:07.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a9259c700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a9406a280 0x7f9a9409b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a91d9b700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a9406a660 0x7f9a9409ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a92d9d700 1 -- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a9406a280 msgr2=0x7f9a9409b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a92d9d700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a9406a280 0x7f9a9409b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a92d9d700 1 -- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a9406a660 msgr2=0x7f9a9409ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a92d9d700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a9406a660 0x7f9a9409ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a92d9d700 1 -- 172.21.15.45:0/4068111552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84009550 con 0x7f9a940ac1a0 2023-08-27T11:13:07.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a91d9b700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a9406a660 0x7f9a9409ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:07.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a92d9d700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a940ac1a0 0x7f9a941583b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00cef0 tx=0x7f9a8c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:07.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a82ffd700 1 -- 172.21.15.45:0/4068111552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00ae50 con 0x7f9a940ac1a0 2023-08-27T11:13:07.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a82ffd700 1 -- 172.21.15.45:0/4068111552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c015440 con 0x7f9a940ac1a0 2023-08-27T11:13:07.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a941f1df0 con 0x7f9a940ac1a0 2023-08-27T11:13:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a82ffd700 1 -- 172.21.15.45:0/4068111552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0157f0 con 0x7f9a940ac1a0 2023-08-27T11:13:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.345+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a941f1310 con 0x7f9a940ac1a0 2023-08-27T11:13:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.346+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9406fb30 con 0x7f9a940ac1a0 2023-08-27T11:13:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.346+0000 7f9a82ffd700 1 -- 172.21.15.45:0/4068111552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9a8c00d430 con 0x7f9a940ac1a0 2023-08-27T11:13:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.347+0000 7f9a82ffd700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a7406a0e0 0x7f9a7406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.347+0000 7f9a82ffd700 1 -- 172.21.15.45:0/4068111552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9a8c088e90 con 0x7f9a940ac1a0 2023-08-27T11:13:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.347+0000 7f9a9259c700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a7406a0e0 0x7f9a7406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.348+0000 7f9a9259c700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a7406a0e0 0x7f9a7406c5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9a941597c0 tx=0x7f9a84008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.351+0000 7f9a82ffd700 1 -- 172.21.15.45:0/4068111552 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9a8c054240 con 0x7f9a940ac1a0 2023-08-27T11:13:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.551+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a9406aa90 con 0x7f9a940ac1a0 2023-08-27T11:13:07.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.552+0000 7f9a82ffd700 1 -- 172.21.15.45:0/4068111552 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a8c01a020 con 0x7f9a940ac1a0 2023-08-27T11:13:07.553 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.553+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a7406a0e0 msgr2=0x7f9a7406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.553+0000 7f9a98eb9700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a7406a0e0 0x7f9a7406c5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9a941597c0 tx=0x7f9a84008030 comp rx=0 tx=0).stop 2023-08-27T11:13:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.553+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a940ac1a0 msgr2=0x7f9a941583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.553+0000 7f9a98eb9700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a940ac1a0 0x7f9a941583b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00cef0 tx=0x7f9a8c00a340 comp rx=0 tx=0).stop 2023-08-27T11:13:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.553+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 shutdown_connections 2023-08-27T11:13:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.553+0000 7f9a98eb9700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a940ac1a0 0x7f9a941583b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.553+0000 7f9a98eb9700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9a7406a0e0 0x7f9a7406c5a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.553+0000 7f9a98eb9700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9a9406a280 0x7f9a9409b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.553+0000 7f9a98eb9700 1 --2- 172.21.15.45:0/4068111552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a9406a660 0x7f9a9409ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.554+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 >> 172.21.15.45:0/4068111552 conn(0x7f9a940b23e0 msgr2=0x7f9a940cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.554+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 shutdown_connections 2023-08-27T11:13:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.554+0000 7f9a98eb9700 1 -- 172.21.15.45:0/4068111552 wait complete. 2023-08-27T11:13:07.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:13:07.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.693+0000 7fb481632700 1 Processor -- start 2023-08-27T11:13:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.694+0000 7fb481632700 1 -- start start 2023-08-27T11:13:07.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.694+0000 7fb481632700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb47c0b23c0 0x7fb47c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.694+0000 7fb481632700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47c0ce0d0 con 0x7fb47c06a8b0 2023-08-27T11:13:07.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.694+0000 7fb481632700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47c0b9040 con 0x7fb47c0b1e70 2023-08-27T11:13:07.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.694+0000 7fb481632700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47c0d2a10 con 0x7fb47c0b23c0 2023-08-27T11:13:07.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb47a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb47c0b23c0 0x7fb47c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb47affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb47c06a8b0 0x7fb47c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39906/0 (socket says 172.21.15.45:39906) 2023-08-27T11:13:07.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb47a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb47c0b23c0 0x7fb47c0ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53132/0 (socket says 172.21.15.45:53132) 2023-08-27T11:13:07.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb47affd700 1 -- 172.21.15.45:0/3723791630 learned_addr learned my addr 172.21.15.45:0/3723791630 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:07.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb47a7fc700 1 -- 172.21.15.45:0/3723791630 >> v1:172.21.15.135:6789/0 conn(0x7fb47c0b1e70 legacy=0x7fb47c0b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:07.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb479ffb700 1 -- 172.21.15.45:0/3723791630 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624634824 0 0) 0x7fb47c0ce0d0 con 0x7fb47c06a8b0 2023-08-27T11:13:07.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb47a7fc700 1 -- 172.21.15.45:0/3723791630 >> v1:172.21.15.45:6789/0 conn(0x7fb47c06a8b0 legacy=0x7fb47c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb47a7fc700 1 -- 172.21.15.45:0/3723791630 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c0cd8d0 con 0x7fb47c0b23c0 2023-08-27T11:13:07.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb47a7fc700 1 --2- 172.21.15.45:0/3723791630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b23c0 0x7fb47c0ce560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009840 tx=0x7fb46c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ce482a0a9212df server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:07.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb479ffb700 1 -- 172.21.15.45:0/3723791630 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c011830 con 0x7fb47c0b23c0 2023-08-27T11:13:07.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb479ffb700 1 -- 172.21.15.45:0/3723791630 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c011990 con 0x7fb47c0b23c0 2023-08-27T11:13:07.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.695+0000 7fb479ffb700 1 -- 172.21.15.45:0/3723791630 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c011d20 con 0x7fb47c0b23c0 2023-08-27T11:13:07.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.696+0000 7fb481632700 1 -- 172.21.15.45:0/3723791630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b23c0 msgr2=0x7fb47c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.696+0000 7fb481632700 1 --2- 172.21.15.45:0/3723791630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b23c0 0x7fb47c0ce560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009840 tx=0x7fb46c009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:07.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.696+0000 7fb481632700 1 -- 172.21.15.45:0/3723791630 shutdown_connections 2023-08-27T11:13:07.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.696+0000 7fb481632700 1 --2- 172.21.15.45:0/3723791630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b23c0 0x7fb47c0ce560 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.696+0000 7fb481632700 1 -- 172.21.15.45:0/3723791630 >> 172.21.15.45:0/3723791630 conn(0x7fb47c0d3070 msgr2=0x7fb47c0d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:07.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.696+0000 7fb481632700 1 -- 172.21.15.45:0/3723791630 shutdown_connections 2023-08-27T11:13:07.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.696+0000 7fb481632700 1 -- 172.21.15.45:0/3723791630 wait complete. 2023-08-27T11:13:07.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.697+0000 7fb481632700 1 Processor -- start 2023-08-27T11:13:07.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.697+0000 7fb481632700 1 -- start start 2023-08-27T11:13:07.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb481632700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb47c06a8b0 0x7fb47c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb481632700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b1e70 0x7fb47c150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb481632700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb47c0b23c0 0x7fb47c150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb481632700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c150ac0 con 0x7fb47c0b1e70 2023-08-27T11:13:07.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb481632700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0ca1e0 con 0x7fb47c0b23c0 2023-08-27T11:13:07.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb481632700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0ca480 con 0x7fb47c06a8b0 2023-08-27T11:13:07.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b1e70 0x7fb47c150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b1e70 0x7fb47c150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53138/0 (socket says 172.21.15.45:53138) 2023-08-27T11:13:07.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47a7fc700 1 -- 172.21.15.45:0/132412594 learned_addr learned my addr 172.21.15.45:0/132412594 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:07.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47affd700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb47c06a8b0 0x7fb47c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47b7fe700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb47c0b23c0 0x7fb47c150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47a7fc700 1 -- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb47c06a8b0 msgr2=0x7fb47c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47a7fc700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb47c06a8b0 0x7fb47c0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47a7fc700 1 -- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb47c0b23c0 msgr2=0x7fb47c150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47a7fc700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb47c0b23c0 0x7fb47c150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47a7fc700 1 -- 172.21.15.45:0/132412594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0094a0 con 0x7fb47c0b1e70 2023-08-27T11:13:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47b7fe700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb47c0b23c0 0x7fb47c150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47affd700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb47c06a8b0 0x7fb47c0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.698+0000 7fb47a7fc700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b1e70 0x7fb47c150040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb46c014040 tx=0x7fb46c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:07.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.699+0000 7fb4637fe700 1 -- 172.21.15.45:0/132412594 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c01a9c0 con 0x7fb47c0b1e70 2023-08-27T11:13:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.699+0000 7fb4637fe700 1 -- 172.21.15.45:0/132412594 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c01ab20 con 0x7fb47c0b1e70 2023-08-27T11:13:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.699+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47c0ca760 con 0x7fb47c0b1e70 2023-08-27T11:13:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.699+0000 7fb4637fe700 1 -- 172.21.15.45:0/132412594 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c024420 con 0x7fb47c0b1e70 2023-08-27T11:13:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.699+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47c0cad20 con 0x7fb47c0b1e70 2023-08-27T11:13:07.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.700+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47c04e620 con 0x7fb47c0b1e70 2023-08-27T11:13:07.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.700+0000 7fb4637fe700 1 -- 172.21.15.45:0/132412594 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb46c022070 con 0x7fb47c0b1e70 2023-08-27T11:13:07.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.700+0000 7fb4637fe700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb46806a0e0 0x7fb46806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:07.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.701+0000 7fb4637fe700 1 -- 172.21.15.45:0/132412594 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb46c089b00 con 0x7fb47c0b1e70 2023-08-27T11:13:07.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.701+0000 7fb47affd700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb46806a0e0 0x7fb46806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:07.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.701+0000 7fb47affd700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb46806a0e0 0x7fb46806c5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb470000f60 tx=0x7fb470007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:07.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.705+0000 7fb4637fe700 1 -- 172.21.15.45:0/132412594 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb46c054f50 con 0x7fb47c0b1e70 2023-08-27T11:13:07.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.903+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb47c15d000 con 0x7fb47c0b1e70 2023-08-27T11:13:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.903+0000 7fb4637fe700 1 -- 172.21.15.45:0/132412594 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb46c029020 con 0x7fb47c0b1e70 2023-08-27T11:13:07.905 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:13:07.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb46806a0e0 msgr2=0x7fb46806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb46806a0e0 0x7fb46806c5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb470000f60 tx=0x7fb470007ce0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b1e70 msgr2=0x7fb47c150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b1e70 0x7fb47c150040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb46c014040 tx=0x7fb46c004750 comp rx=0 tx=0).stop 2023-08-27T11:13:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 shutdown_connections 2023-08-27T11:13:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb46806a0e0 0x7fb46806c5a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb47c0b23c0 0x7fb47c150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb47c06a8b0 0x7fb47c0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 --2- 172.21.15.45:0/132412594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb47c0b1e70 0x7fb47c150040 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 >> 172.21.15.45:0/132412594 conn(0x7fb47c0d3070 msgr2=0x7fb47c15dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 shutdown_connections 2023-08-27T11:13:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:07.905+0000 7fb481632700 1 -- 172.21.15.45:0/132412594 wait complete. 2023-08-27T11:13:07.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:13:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.045+0000 7f54269bb700 1 Processor -- start 2023-08-27T11:13:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.045+0000 7f54269bb700 1 -- start start 2023-08-27T11:13:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f54269bb700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f54200ca620 0x7f54200cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f54269bb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200af200 0x7f54200ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f54269bb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54200caa00 0x7f54200acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f54269bb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54200ce340 con 0x7f54200caa00 2023-08-27T11:13:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f54269bb700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54200ceec0 con 0x7f54200ca620 2023-08-27T11:13:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f54269bb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542015d7c0 con 0x7f54200af200 2023-08-27T11:13:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f5424f58700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54200caa00 0x7f54200acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f5424f58700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54200caa00 0x7f54200acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53148/0 (socket says 172.21.15.45:53148) 2023-08-27T11:13:08.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f5424f58700 1 -- 172.21.15.45:0/2202357308 learned_addr learned my addr 172.21.15.45:0/2202357308 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:08.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f541f7fe700 1 --2- 172.21.15.45:0/2202357308 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200af200 0x7f54200ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f541ffff700 1 --2- 172.21.15.45:0/2202357308 >> v2:172.21.15.135:3300/0 conn(0x7f54200ca620 0x7f54200cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f5424f58700 1 -- 172.21.15.45:0/2202357308 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200af200 msgr2=0x7f54200ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f5424f58700 1 --2- 172.21.15.45:0/2202357308 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200af200 0x7f54200ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f5424f58700 1 -- 172.21.15.45:0/2202357308 >> v2:172.21.15.135:3300/0 conn(0x7f54200ca620 msgr2=0x7f54200cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f5424f58700 1 --2- 172.21.15.45:0/2202357308 >> v2:172.21.15.135:3300/0 conn(0x7f54200ca620 0x7f54200cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f5424f58700 1 -- 172.21.15.45:0/2202357308 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200cc570 con 0x7f54200caa00 2023-08-27T11:13:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.046+0000 7f5424f58700 1 --2- 172.21.15.45:0/2202357308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200caa00 0x7f54200acdf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5420079e70 tx=0x7f541800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d99f8d5d8d7cfe0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f541effd700 1 -- 172.21.15.45:0/2202357308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800e070 con 0x7f54200caa00 2023-08-27T11:13:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f541effd700 1 -- 172.21.15.45:0/2202357308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541800b910 con 0x7f54200caa00 2023-08-27T11:13:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f541effd700 1 -- 172.21.15.45:0/2202357308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800bcc0 con 0x7f54200caa00 2023-08-27T11:13:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f54269bb700 1 -- 172.21.15.45:0/2202357308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200caa00 msgr2=0x7f54200acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f54269bb700 1 --2- 172.21.15.45:0/2202357308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200caa00 0x7f54200acdf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5420079e70 tx=0x7f541800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f54269bb700 1 -- 172.21.15.45:0/2202357308 shutdown_connections 2023-08-27T11:13:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f54269bb700 1 --2- 172.21.15.45:0/2202357308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200caa00 0x7f54200acdf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f54269bb700 1 --2- 172.21.15.45:0/2202357308 >> v2:172.21.15.135:3300/0 conn(0x7f54200ca620 0x7f54200cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f54269bb700 1 --2- 172.21.15.45:0/2202357308 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200af200 0x7f54200ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f54269bb700 1 -- 172.21.15.45:0/2202357308 >> 172.21.15.45:0/2202357308 conn(0x7f54200cc7e0 msgr2=0x7f54200ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:08.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f54269bb700 1 -- 172.21.15.45:0/2202357308 shutdown_connections 2023-08-27T11:13:08.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.047+0000 7f54269bb700 1 -- 172.21.15.45:0/2202357308 wait complete. 2023-08-27T11:13:08.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.048+0000 7f54269bb700 1 Processor -- start 2023-08-27T11:13:08.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.049+0000 7f54269bb700 1 -- start start 2023-08-27T11:13:08.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.049+0000 7f54269bb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54200af200 0x7f542015aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.049+0000 7f54269bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200ca620 0x7f54200b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.049+0000 7f54269bb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200caa00 0x7f54201f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.049+0000 7f54269bb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200c4af0 con 0x7f54200ca620 2023-08-27T11:13:08.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.049+0000 7f54269bb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200c4dd0 con 0x7f54200af200 2023-08-27T11:13:08.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.049+0000 7f54269bb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200c50b0 con 0x7f54200caa00 2023-08-27T11:13:08.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.049+0000 7f541f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200ca620 0x7f54200b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.049+0000 7f541f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200ca620 0x7f54200b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53150/0 (socket says 172.21.15.45:53150) 2023-08-27T11:13:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541f7fe700 1 -- 172.21.15.45:0/2074138914 learned_addr learned my addr 172.21.15.45:0/2074138914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f5424f58700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200caa00 0x7f54201f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541ffff700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54200af200 0x7f542015aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541f7fe700 1 -- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200caa00 msgr2=0x7f54201f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541f7fe700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200caa00 0x7f54201f29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541f7fe700 1 -- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54200af200 msgr2=0x7f542015aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541f7fe700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54200af200 0x7f542015aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541f7fe700 1 -- 172.21.15.45:0/2074138914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418009b30 con 0x7f54200ca620 2023-08-27T11:13:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f5424f58700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200caa00 0x7f54201f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541ffff700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54200af200 0x7f542015aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:08.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541f7fe700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200ca620 0x7f54200b33d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f540800e900 tx=0x7f540800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541cff9700 1 -- 172.21.15.45:0/2074138914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408009690 con 0x7f54200ca620 2023-08-27T11:13:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541cff9700 1 -- 172.21.15.45:0/2074138914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540800c850 con 0x7f54200ca620 2023-08-27T11:13:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f541cff9700 1 -- 172.21.15.45:0/2074138914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540800cbe0 con 0x7f54200ca620 2023-08-27T11:13:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54200c5390 con 0x7f54200ca620 2023-08-27T11:13:08.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.050+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54200b3af0 con 0x7f54200ca620 2023-08-27T11:13:08.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.051+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54200c5700 con 0x7f54200ca620 2023-08-27T11:13:08.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.052+0000 7f541cff9700 1 -- 172.21.15.45:0/2074138914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f540800cd40 con 0x7f54200ca620 2023-08-27T11:13:08.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.052+0000 7f541cff9700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f540006a0e0 0x7f540006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.052+0000 7f541cff9700 1 -- 172.21.15.45:0/2074138914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f54080884c0 con 0x7f54200ca620 2023-08-27T11:13:08.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.052+0000 7f541ffff700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f540006a0e0 0x7f540006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.053+0000 7f541ffff700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f540006a0e0 0x7f540006c5a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5410009b60 tx=0x7f5410008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.056+0000 7f541cff9700 1 -- 172.21.15.45:0/2074138914 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f54080573a0 con 0x7f54200ca620 2023-08-27T11:13:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.259+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f54200ce570 con 0x7f54200ca620 2023-08-27T11:13:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.259+0000 7f541cff9700 1 -- 172.21.15.45:0/2074138914 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f54200ce570 con 0x7f54200ca620 2023-08-27T11:13:08.260 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f540006a0e0 msgr2=0x7f540006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f540006a0e0 0x7f540006c5a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5410009b60 tx=0x7f5410008040 comp rx=0 tx=0).stop 2023-08-27T11:13:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200ca620 msgr2=0x7f54200b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200ca620 0x7f54200b33d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f540800e900 tx=0x7f540800ecc0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 shutdown_connections 2023-08-27T11:13:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f540006a0e0 0x7f540006c5a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f54200caa00 0x7f54201f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54200af200 0x7f542015aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 --2- 172.21.15.45:0/2074138914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54200ca620 0x7f54200b33d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 >> 172.21.15.45:0/2074138914 conn(0x7f54200cc7e0 msgr2=0x7f54200ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 shutdown_connections 2023-08-27T11:13:08.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.261+0000 7f54269bb700 1 -- 172.21.15.45:0/2074138914 wait complete. 2023-08-27T11:13:08.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:13:08.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.401+0000 7fbfa2ff9700 1 Processor -- start 2023-08-27T11:13:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.401+0000 7fbfa2ff9700 1 -- start start 2023-08-27T11:13:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.402+0000 7fbfa2ff9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ca620 0x7fbf9c0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.402+0000 7fbfa2ff9700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbf9c0caa00 0x7fbf9c0acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.402+0000 7fbfa2ff9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c0ce340 con 0x7fbf9c0af200 2023-08-27T11:13:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.402+0000 7fbfa2ff9700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c0ceec0 con 0x7fbf9c0caa00 2023-08-27T11:13:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.402+0000 7fbfa2ff9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c15d7c0 con 0x7fbf9c0ca620 2023-08-27T11:13:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.402+0000 7fbf9bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbf9c0af200 0x7fbf9c0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39934/0 (socket says 172.21.15.45:39934) 2023-08-27T11:13:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.402+0000 7fbf9bfff700 1 -- 172.21.15.45:0/684443817 learned_addr learned my addr 172.21.15.45:0/684443817 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:08.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbfa1596700 1 --2- 172.21.15.45:0/684443817 >> v2:172.21.15.135:3300/0 conn(0x7fbf9c0caa00 0x7fbf9c0acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbfa0d95700 1 --2- 172.21.15.45:0/684443817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ca620 0x7fbf9c0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbf9b7fe700 1 -- 172.21.15.45:0/684443817 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691053869 0 0) 0x7fbf9c0ce340 con 0x7fbf9c0af200 2023-08-27T11:13:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbf9b7fe700 1 -- 172.21.15.45:0/684443817 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf8c0035b0 con 0x7fbf9c0af200 2023-08-27T11:13:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbfa1596700 1 -- 172.21.15.45:0/684443817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ca620 msgr2=0x7fbf9c0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbfa1596700 1 --2- 172.21.15.45:0/684443817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ca620 0x7fbf9c0cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbfa1596700 1 -- 172.21.15.45:0/684443817 >> v1:172.21.15.45:6789/0 conn(0x7fbf9c0af200 legacy=0x7fbf9c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbfa1596700 1 -- 172.21.15.45:0/684443817 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0cc570 con 0x7fbf9c0caa00 2023-08-27T11:13:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbfa1596700 1 --2- 172.21.15.45:0/684443817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c0acfc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009d90 tx=0x7fbf9400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=285b3aa15960023b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbf9b7fe700 1 -- 172.21.15.45:0/684443817 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400e070 con 0x7fbf9c0caa00 2023-08-27T11:13:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbf9b7fe700 1 -- 172.21.15.45:0/684443817 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9400bb20 con 0x7fbf9c0caa00 2023-08-27T11:13:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.403+0000 7fbf9b7fe700 1 -- 172.21.15.45:0/684443817 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94012430 con 0x7fbf9c0caa00 2023-08-27T11:13:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.404+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/684443817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 msgr2=0x7fbf9c0acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.404+0000 7fbfa2ff9700 1 --2- 172.21.15.45:0/684443817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c0acfc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009d90 tx=0x7fbf9400b330 comp rx=0 tx=0).stop 2023-08-27T11:13:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.404+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/684443817 shutdown_connections 2023-08-27T11:13:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.404+0000 7fbfa2ff9700 1 --2- 172.21.15.45:0/684443817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c0acfc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.404+0000 7fbfa2ff9700 1 --2- 172.21.15.45:0/684443817 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0ca620 0x7fbf9c0cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.404+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/684443817 >> 172.21.15.45:0/684443817 conn(0x7fbf9c0cc7e0 msgr2=0x7fbf9c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.404+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/684443817 shutdown_connections 2023-08-27T11:13:08.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.404+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/684443817 wait complete. 2023-08-27T11:13:08.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.405+0000 7fbfa2ff9700 1 Processor -- start 2023-08-27T11:13:08.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.405+0000 7fbfa2ff9700 1 -- start start 2023-08-27T11:13:08.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbfa2ff9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0af200 0x7fbf9c1544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbfa2ff9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0ca620 0x7fbf9c150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbfa2ff9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c14db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbfa2ff9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0ca120 con 0x7fbf9c0ca620 2023-08-27T11:13:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbfa2ff9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0c9080 con 0x7fbf9c0caa00 2023-08-27T11:13:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbfa2ff9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0c9360 con 0x7fbf9c0af200 2023-08-27T11:13:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbf9bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0ca620 0x7fbf9c150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbf9bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0ca620 0x7fbf9c150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53162/0 (socket says 172.21.15.45:53162) 2023-08-27T11:13:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbf9bfff700 1 -- 172.21.15.45:0/98385191 learned_addr learned my addr 172.21.15.45:0/98385191 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbfa0d95700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0af200 0x7fbf9c1544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.406+0000 7fbfa1596700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c14db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbf9bfff700 1 -- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0af200 msgr2=0x7fbf9c1544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0af200 0x7fbf9c1544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbf9bfff700 1 -- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 msgr2=0x7fbf9c14db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c14db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbf9bfff700 1 -- 172.21.15.45:0/98385191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94009a70 con 0x7fbf9c0ca620 2023-08-27T11:13:08.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbfa0d95700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0af200 0x7fbf9c1544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:08.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbfa1596700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c14db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:08.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbf9bfff700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0ca620 0x7fbf9c150c60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbf88005910 tx=0x7fbf8800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:08.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbf997fa700 1 -- 172.21.15.45:0/98385191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88005d70 con 0x7fbf9c0ca620 2023-08-27T11:13:08.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbf997fa700 1 -- 172.21.15.45:0/98385191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf880070d0 con 0x7fbf9c0ca620 2023-08-27T11:13:08.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbf997fa700 1 -- 172.21.15.45:0/98385191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88007460 con 0x7fbf9c0ca620 2023-08-27T11:13:08.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c0c9640 con 0x7fbf9c0ca620 2023-08-27T11:13:08.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.407+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c0c9c00 con 0x7fbf9c0ca620 2023-08-27T11:13:08.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.408+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c0cbce0 con 0x7fbf9c0ca620 2023-08-27T11:13:08.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.408+0000 7fbf997fa700 1 -- 172.21.15.45:0/98385191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbf880075c0 con 0x7fbf9c0ca620 2023-08-27T11:13:08.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.409+0000 7fbf997fa700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:08.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.409+0000 7fbf997fa700 1 -- 172.21.15.45:0/98385191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbf88087dc0 con 0x7fbf9c0ca620 2023-08-27T11:13:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.409+0000 7fbfa0d95700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.410+0000 7fbfa0d95700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009c30 tx=0x7fbf90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.413+0000 7fbf997fa700 1 -- 172.21.15.45:0/98385191 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf88053210 con 0x7fbf9c0ca620 2023-08-27T11:13:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.616+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbf9c0aca50 con 0x7fbf9c0ca620 2023-08-27T11:13:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.619+0000 7fbf997fa700 1 -- 172.21.15.45:0/98385191 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbf88056830 con 0x7fbf9c0ca620 2023-08-27T11:13:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 msgr2=0x7fbf8006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009c30 tx=0x7fbf90008040 comp rx=0 tx=0).stop 2023-08-27T11:13:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0ca620 msgr2=0x7fbf9c150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0ca620 0x7fbf9c150c60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbf88005910 tx=0x7fbf8800ca90 comp rx=0 tx=0).stop 2023-08-27T11:13:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 shutdown_connections 2023-08-27T11:13:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbf8006a0e0 0x7fbf8006c5a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c14db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf9c0af200 0x7fbf9c1544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 --2- 172.21.15.45:0/98385191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c0ca620 0x7fbf9c150c60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 >> 172.21.15.45:0/98385191 conn(0x7fbf9c0cc7e0 msgr2=0x7fbf9c0cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 shutdown_connections 2023-08-27T11:13:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:08.621+0000 7fbfa2ff9700 1 -- 172.21.15.45:0/98385191 wait complete. 2023-08-27T11:13:08.634 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:13:08.635 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:13:08.677 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:13:08.722 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:13:08.723 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:13:08.723 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:08.741 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:13:08.742 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:08.799 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:13:08.799 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:08.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:09.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.022+0000 7f68b4bf6700 1 Processor -- start 2023-08-27T11:13:09.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.022+0000 7f68b4bf6700 1 -- start start 2023-08-27T11:13:09.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68b4bf6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b015a240 0x7f68b00cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68b4bf6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68b00ceba0 0x7f68b006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68b4bf6700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b006d690 con 0x7f68b00cbe70 2023-08-27T11:13:09.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68b4bf6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b00cc440 con 0x7f68b00ceba0 2023-08-27T11:13:09.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68b4bf6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b006d040 con 0x7f68b015a240 2023-08-27T11:13:09.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68aed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68b00ceba0 0x7f68b006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68aed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68b00ceba0 0x7f68b006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53172/0 (socket says 172.21.15.45:53172) 2023-08-27T11:13:09.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68aed9d700 1 -- 172.21.15.45:0/3888040166 learned_addr learned my addr 172.21.15.45:0/3888040166 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:09.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68ae59c700 1 --2- 172.21.15.45:0/3888040166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b015a240 0x7f68b00cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68aed9d700 1 -- 172.21.15.45:0/3888040166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b015a240 msgr2=0x7f68b00cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:09.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68aed9d700 1 --2- 172.21.15.45:0/3888040166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b015a240 0x7f68b00cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68aed9d700 1 -- 172.21.15.45:0/3888040166 >> v1:172.21.15.135:6789/0 conn(0x7f68b00cbe70 legacy=0x7f68b00ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68aed9d700 1 -- 172.21.15.45:0/3888040166 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b006d320 con 0x7f68b00ceba0 2023-08-27T11:13:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68aed9d700 1 --2- 172.21.15.45:0/3888040166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b00ceba0 0x7f68b006db20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009e50 tx=0x7f68a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31fd13acf47151b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68ad59a700 1 -- 172.21.15.45:0/3888040166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a800e070 con 0x7f68b00ceba0 2023-08-27T11:13:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.023+0000 7f68ad59a700 1 -- 172.21.15.45:0/3888040166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a800b9c0 con 0x7f68b00ceba0 2023-08-27T11:13:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.024+0000 7f68ad59a700 1 -- 172.21.15.45:0/3888040166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a800bd70 con 0x7f68b00ceba0 2023-08-27T11:13:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.024+0000 7f68b4bf6700 1 -- 172.21.15.45:0/3888040166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b00ceba0 msgr2=0x7f68b006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.024+0000 7f68b4bf6700 1 --2- 172.21.15.45:0/3888040166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b00ceba0 0x7f68b006db20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009e50 tx=0x7f68a800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.024+0000 7f68b4bf6700 1 -- 172.21.15.45:0/3888040166 shutdown_connections 2023-08-27T11:13:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.024+0000 7f68b4bf6700 1 --2- 172.21.15.45:0/3888040166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b015a240 0x7f68b00cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.024+0000 7f68b4bf6700 1 --2- 172.21.15.45:0/3888040166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b00ceba0 0x7f68b006db20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.024+0000 7f68b4bf6700 1 -- 172.21.15.45:0/3888040166 >> 172.21.15.45:0/3888040166 conn(0x7f68b00cfe50 msgr2=0x7f68b00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.024+0000 7f68b4bf6700 1 -- 172.21.15.45:0/3888040166 shutdown_connections 2023-08-27T11:13:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.024+0000 7f68b4bf6700 1 -- 172.21.15.45:0/3888040166 wait complete. 2023-08-27T11:13:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.025+0000 7f68b4bf6700 1 Processor -- start 2023-08-27T11:13:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68b4bf6700 1 -- start start 2023-08-27T11:13:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68b4bf6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b00cbe70 0x7f68b014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68b4bf6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68b00ceba0 0x7f68b015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68b4bf6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b015a240 0x7f68b00a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68b4bf6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b014cf30 con 0x7f68b015a240 2023-08-27T11:13:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68b4bf6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b014b950 con 0x7f68b00ceba0 2023-08-27T11:13:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68b4bf6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b014bc30 con 0x7f68b00cbe70 2023-08-27T11:13:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68aed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b015a240 0x7f68b00a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68aed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b015a240 0x7f68b00a8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53178/0 (socket says 172.21.15.45:53178) 2023-08-27T11:13:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.026+0000 7f68aed9d700 1 -- 172.21.15.45:0/676970994 learned_addr learned my addr 172.21.15.45:0/676970994 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68add9b700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68b00ceba0 0x7f68b015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68ae59c700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b00cbe70 0x7f68b014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68aed9d700 1 -- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b00cbe70 msgr2=0x7f68b014c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68aed9d700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b00cbe70 0x7f68b014c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68aed9d700 1 -- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68b00ceba0 msgr2=0x7f68b015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68aed9d700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68b00ceba0 0x7f68b015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68aed9d700 1 -- 172.21.15.45:0/676970994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a8009b30 con 0x7f68b015a240 2023-08-27T11:13:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68add9b700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68b00ceba0 0x7f68b015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68aed9d700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b015a240 0x7f68b00a8580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f68a8012000 tx=0x7f68a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f689effd700 1 -- 172.21.15.45:0/676970994 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8004960 con 0x7f68b015a240 2023-08-27T11:13:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f689effd700 1 -- 172.21.15.45:0/676970994 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a80099b0 con 0x7f68b015a240 2023-08-27T11:13:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b014bf10 con 0x7f68b015a240 2023-08-27T11:13:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f689effd700 1 -- 172.21.15.45:0/676970994 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a801c680 con 0x7f68b015a240 2023-08-27T11:13:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.027+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b014c4d0 con 0x7f68b015a240 2023-08-27T11:13:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.028+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b0072810 con 0x7f68b015a240 2023-08-27T11:13:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.029+0000 7f689effd700 1 -- 172.21.15.45:0/676970994 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f68a801c7e0 con 0x7f68b015a240 2023-08-27T11:13:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.029+0000 7f689effd700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f689006a0e0 0x7f689006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.029+0000 7f689effd700 1 -- 172.21.15.45:0/676970994 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f68a808a230 con 0x7f68b015a240 2023-08-27T11:13:09.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.029+0000 7f68ae59c700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f689006a0e0 0x7f689006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.030+0000 7f68ae59c700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f689006a0e0 0x7f689006c5a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68b00b0510 tx=0x7f68a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:09.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.034+0000 7f689effd700 1 -- 172.21.15.45:0/676970994 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f68a8055680 con 0x7f68b015a240 2023-08-27T11:13:09.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.238+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68b00734a0 con 0x7f68b015a240 2023-08-27T11:13:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.239+0000 7f689effd700 1 -- 172.21.15.45:0/676970994 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7f68a8058ca0 con 0x7f68b015a240 2023-08-27T11:13:09.240 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:09.241 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:09.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.241+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f689006a0e0 msgr2=0x7f689006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.241+0000 7f68b4bf6700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f689006a0e0 0x7f689006c5a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68b00b0510 tx=0x7f68a0008040 comp rx=0 tx=0).stop 2023-08-27T11:13:09.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.241+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b015a240 msgr2=0x7f68b00a8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.241+0000 7f68b4bf6700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b015a240 0x7f68b00a8580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f68a8012000 tx=0x7f68a8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.242+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 shutdown_connections 2023-08-27T11:13:09.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.242+0000 7f68b4bf6700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68b015a240 0x7f68b00a8580 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.242+0000 7f68b4bf6700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f689006a0e0 0x7f689006c5a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.242+0000 7f68b4bf6700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68b00cbe70 0x7f68b014c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.242+0000 7f68b4bf6700 1 --2- 172.21.15.45:0/676970994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68b00ceba0 0x7f68b015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.242+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 >> 172.21.15.45:0/676970994 conn(0x7f68b00cfe50 msgr2=0x7f68b0156e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:09.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.242+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 shutdown_connections 2023-08-27T11:13:09.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.242+0000 7f68b4bf6700 1 -- 172.21.15.45:0/676970994 wait complete. 2023-08-27T11:13:09.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:13:09.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:13:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.383+0000 7f1f5e01e700 1 Processor -- start 2023-08-27T11:13:09.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.383+0000 7f1f5e01e700 1 -- start start 2023-08-27T11:13:09.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.384+0000 7f1f5e01e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1f58157be0 0x7f1f580716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.384+0000 7f1f5e01e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1f580b17c0 0x7f1f580b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.384+0000 7f1f5e01e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5815b460 con 0x7f1f5806ced0 2023-08-27T11:13:09.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.384+0000 7f1f5e01e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f58157fc0 con 0x7f1f58157be0 2023-08-27T11:13:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.384+0000 7f1f5e01e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f580b2280 con 0x7f1f580b17c0 2023-08-27T11:13:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.384+0000 7f1f577fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1f58157be0 0x7f1f580716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.384+0000 7f1f57fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1f5806ced0 0x7f1f580cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39962/0 (socket says 172.21.15.45:39962) 2023-08-27T11:13:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.384+0000 7f1f57fff700 1 -- 172.21.15.45:0/1350995258 learned_addr learned my addr 172.21.15.45:0/1350995258 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:09.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.384+0000 7f1f56ffd700 1 --2- 172.21.15.45:0/1350995258 >> v2:172.21.15.135:3300/0 conn(0x7f1f580b17c0 0x7f1f580b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f577fe700 1 -- 172.21.15.45:0/1350995258 >> v2:172.21.15.135:3300/0 conn(0x7f1f580b17c0 msgr2=0x7f1f580b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f577fe700 1 --2- 172.21.15.45:0/1350995258 >> v2:172.21.15.135:3300/0 conn(0x7f1f580b17c0 0x7f1f580b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f577fe700 1 -- 172.21.15.45:0/1350995258 >> v1:172.21.15.45:6789/0 conn(0x7f1f5806ced0 legacy=0x7f1f580cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f567fc700 1 -- 172.21.15.45:0/1350995258 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691227201 0 0) 0x7f1f5815b460 con 0x7f1f5806ced0 2023-08-27T11:13:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f577fe700 1 -- 172.21.15.45:0/1350995258 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f580b2560 con 0x7f1f58157be0 2023-08-27T11:13:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f56ffd700 1 --2- 172.21.15.45:0/1350995258 >> v2:172.21.15.135:3300/0 conn(0x7f1f580b17c0 0x7f1f580b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:09.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f577fe700 1 --2- 172.21.15.45:0/1350995258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 0x7f1f580716b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1f580cd3f0 tx=0x7f1f40009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f76fe3e1c86d87 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:09.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f567fc700 1 -- 172.21.15.45:0/1350995258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40011830 con 0x7f1f58157be0 2023-08-27T11:13:09.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f567fc700 1 -- 172.21.15.45:0/1350995258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f40011990 con 0x7f1f58157be0 2023-08-27T11:13:09.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f567fc700 1 -- 172.21.15.45:0/1350995258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40011d20 con 0x7f1f58157be0 2023-08-27T11:13:09.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f5e01e700 1 -- 172.21.15.45:0/1350995258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 msgr2=0x7f1f580716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.385+0000 7f1f5e01e700 1 --2- 172.21.15.45:0/1350995258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 0x7f1f580716b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1f580cd3f0 tx=0x7f1f40009b20 comp rx=0 tx=0).stop 2023-08-27T11:13:09.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.386+0000 7f1f5e01e700 1 -- 172.21.15.45:0/1350995258 shutdown_connections 2023-08-27T11:13:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.386+0000 7f1f5e01e700 1 --2- 172.21.15.45:0/1350995258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 0x7f1f580716b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.386+0000 7f1f5e01e700 1 --2- 172.21.15.45:0/1350995258 >> v2:172.21.15.135:3300/0 conn(0x7f1f580b17c0 0x7f1f580b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.386+0000 7f1f5e01e700 1 -- 172.21.15.45:0/1350995258 >> 172.21.15.45:0/1350995258 conn(0x7f1f580cc7c0 msgr2=0x7f1f580ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.386+0000 7f1f5e01e700 1 -- 172.21.15.45:0/1350995258 shutdown_connections 2023-08-27T11:13:09.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.386+0000 7f1f5e01e700 1 -- 172.21.15.45:0/1350995258 wait complete. 2023-08-27T11:13:09.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.387+0000 7f1f5e01e700 1 Processor -- start 2023-08-27T11:13:09.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.387+0000 7f1f5e01e700 1 -- start start 2023-08-27T11:13:09.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f5e01e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f5806ced0 0x7f1f58155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f5e01e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f580b17c0 0x7f1f580b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f5e01e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 0x7f1f580af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f5e01e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f580ae220 con 0x7f1f58157be0 2023-08-27T11:13:09.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f5e01e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f581563a0 con 0x7f1f5806ced0 2023-08-27T11:13:09.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f5e01e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f580ad2c0 con 0x7f1f580b17c0 2023-08-27T11:13:09.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f57fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 0x7f1f580af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f57fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 0x7f1f580af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53192/0 (socket says 172.21.15.45:53192) 2023-08-27T11:13:09.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f57fff700 1 -- 172.21.15.45:0/820731440 learned_addr learned my addr 172.21.15.45:0/820731440 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:09.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f577fe700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f5806ced0 0x7f1f58155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f56ffd700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f580b17c0 0x7f1f580b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f57fff700 1 -- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f580b17c0 msgr2=0x7f1f580b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f57fff700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f580b17c0 0x7f1f580b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f57fff700 1 -- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f5806ced0 msgr2=0x7f1f58155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f57fff700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f5806ced0 0x7f1f58155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f57fff700 1 -- 172.21.15.45:0/820731440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f400094a0 con 0x7f1f58157be0 2023-08-27T11:13:09.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f577fe700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f5806ced0 0x7f1f58155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:09.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f57fff700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 0x7f1f580af720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000cba0 tx=0x7f1f5000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:09.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f4ffff700 1 -- 172.21.15.45:0/820731440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000ab80 con 0x7f1f58157be0 2023-08-27T11:13:09.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f4ffff700 1 -- 172.21.15.45:0/820731440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5000ad00 con 0x7f1f58157be0 2023-08-27T11:13:09.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f580ad5a0 con 0x7f1f58157be0 2023-08-27T11:13:09.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.388+0000 7f1f4ffff700 1 -- 172.21.15.45:0/820731440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50007310 con 0x7f1f58157be0 2023-08-27T11:13:09.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.389+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f580adb60 con 0x7f1f58157be0 2023-08-27T11:13:09.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.389+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f580ae360 con 0x7f1f58157be0 2023-08-27T11:13:09.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.390+0000 7f1f4ffff700 1 -- 172.21.15.45:0/820731440 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1f50007470 con 0x7f1f58157be0 2023-08-27T11:13:09.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.390+0000 7f1f4ffff700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1f3806a0e0 0x7f1f3806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.390+0000 7f1f4ffff700 1 -- 172.21.15.45:0/820731440 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1f500883c0 con 0x7f1f58157be0 2023-08-27T11:13:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.391+0000 7f1f577fe700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1f3806a0e0 0x7f1f3806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.391+0000 7f1f577fe700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1f3806a0e0 0x7f1f3806c5a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1f400090b0 tx=0x7f1f40008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.395+0000 7f1f4ffff700 1 -- 172.21.15.45:0/820731440 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1f500536e0 con 0x7f1f58157be0 2023-08-27T11:13:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.594+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f580ae540 con 0x7f1f58157be0 2023-08-27T11:13:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.594+0000 7f1f4ffff700 1 -- 172.21.15.45:0/820731440 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f50056d00 con 0x7f1f58157be0 2023-08-27T11:13:09.596 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:13:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1f3806a0e0 msgr2=0x7f1f3806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1f3806a0e0 0x7f1f3806c5a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1f400090b0 tx=0x7f1f40008030 comp rx=0 tx=0).stop 2023-08-27T11:13:09.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 msgr2=0x7f1f580af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 0x7f1f580af720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000cba0 tx=0x7f1f5000cf60 comp rx=0 tx=0).stop 2023-08-27T11:13:09.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 shutdown_connections 2023-08-27T11:13:09.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f58157be0 0x7f1f580af720 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1f3806a0e0 0x7f1f3806c5a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f5806ced0 0x7f1f58155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 --2- 172.21.15.45:0/820731440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f580b17c0 0x7f1f580b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 >> 172.21.15.45:0/820731440 conn(0x7f1f580cc7c0 msgr2=0x7f1f580b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:09.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.596+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 shutdown_connections 2023-08-27T11:13:09.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.597+0000 7f1f5e01e700 1 -- 172.21.15.45:0/820731440 wait complete. 2023-08-27T11:13:09.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:13:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.737+0000 7fe5dda8f700 1 Processor -- start 2023-08-27T11:13:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.737+0000 7fe5dda8f700 1 -- start start 2023-08-27T11:13:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5dda8f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe5d80b65d0 0x7fe5d80ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5dda8f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 0x7fe5d815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5dda8f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5d80cfe10 0x7fe5d80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5dda8f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d815c620 con 0x7fe5d80cfe10 2023-08-27T11:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5dda8f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d80d07c0 con 0x7fe5d80b65d0 2023-08-27T11:13:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5dda8f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d815ec50 con 0x7fe5d80cb5c0 2023-08-27T11:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5d80cfe10 0x7fe5d80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5d80cfe10 0x7fe5d80d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53200/0 (socket says 172.21.15.45:53200) 2023-08-27T11:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d7fff700 1 -- 172.21.15.45:0/4070857746 learned_addr learned my addr 172.21.15.45:0/4070857746 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d6ffd700 1 --2- 172.21.15.45:0/4070857746 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 0x7fe5d815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d77fe700 1 --2- 172.21.15.45:0/4070857746 >> v2:172.21.15.135:3300/0 conn(0x7fe5d80b65d0 0x7fe5d80ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d7fff700 1 -- 172.21.15.45:0/4070857746 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 msgr2=0x7fe5d815c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d7fff700 1 --2- 172.21.15.45:0/4070857746 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 0x7fe5d815c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d7fff700 1 -- 172.21.15.45:0/4070857746 >> v2:172.21.15.135:3300/0 conn(0x7fe5d80b65d0 msgr2=0x7fe5d80ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d7fff700 1 --2- 172.21.15.45:0/4070857746 >> v2:172.21.15.135:3300/0 conn(0x7fe5d80b65d0 0x7fe5d80ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d7fff700 1 -- 172.21.15.45:0/4070857746 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d815eea0 con 0x7fe5d80cfe10 2023-08-27T11:13:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.738+0000 7fe5d77fe700 1 --2- 172.21.15.45:0/4070857746 >> v2:172.21.15.135:3300/0 conn(0x7fe5d80b65d0 0x7fe5d80ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5d7fff700 1 --2- 172.21.15.45:0/4070857746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80cfe10 0x7fe5d80d0280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5d806ad60 tx=0x7fe5d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bceac832c4dc8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5d67fc700 1 -- 172.21.15.45:0/4070857746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000e070 con 0x7fe5d80cfe10 2023-08-27T11:13:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5d67fc700 1 -- 172.21.15.45:0/4070857746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d000b910 con 0x7fe5d80cfe10 2023-08-27T11:13:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5d67fc700 1 -- 172.21.15.45:0/4070857746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000bcc0 con 0x7fe5d80cfe10 2023-08-27T11:13:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5dda8f700 1 -- 172.21.15.45:0/4070857746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80cfe10 msgr2=0x7fe5d80d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/4070857746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80cfe10 0x7fe5d80d0280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5d806ad60 tx=0x7fe5d000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5dda8f700 1 -- 172.21.15.45:0/4070857746 shutdown_connections 2023-08-27T11:13:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/4070857746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80cfe10 0x7fe5d80d0280 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/4070857746 >> v2:172.21.15.135:3300/0 conn(0x7fe5d80b65d0 0x7fe5d80ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/4070857746 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 0x7fe5d815c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5dda8f700 1 -- 172.21.15.45:0/4070857746 >> 172.21.15.45:0/4070857746 conn(0x7fe5d806f960 msgr2=0x7fe5d806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5dda8f700 1 -- 172.21.15.45:0/4070857746 shutdown_connections 2023-08-27T11:13:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.739+0000 7fe5dda8f700 1 -- 172.21.15.45:0/4070857746 wait complete. 2023-08-27T11:13:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.740+0000 7fe5dda8f700 1 Processor -- start 2023-08-27T11:13:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.741+0000 7fe5dda8f700 1 -- start start 2023-08-27T11:13:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.741+0000 7fe5dda8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80b65d0 0x7fe5d80c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.741+0000 7fe5dda8f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 0x7fe5d815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.741+0000 7fe5dda8f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5d80cfe10 0x7fe5d815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.741+0000 7fe5dda8f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d814ec20 con 0x7fe5d80b65d0 2023-08-27T11:13:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.741+0000 7fe5dda8f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d814ede0 con 0x7fe5d80cfe10 2023-08-27T11:13:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.741+0000 7fe5dda8f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d814fc40 con 0x7fe5d80cb5c0 2023-08-27T11:13:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80b65d0 0x7fe5d80c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80b65d0 0x7fe5d80c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53202/0 (socket says 172.21.15.45:53202) 2023-08-27T11:13:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d77fe700 1 -- 172.21.15.45:0/1474873691 learned_addr learned my addr 172.21.15.45:0/1474873691 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d6ffd700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 0x7fe5d815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d7fff700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5d80cfe10 0x7fe5d815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d77fe700 1 -- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 msgr2=0x7fe5d815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d77fe700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 0x7fe5d815a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d77fe700 1 -- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5d80cfe10 msgr2=0x7fe5d815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d77fe700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5d80cfe10 0x7fe5d815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d77fe700 1 -- 172.21.15.45:0/1474873691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0009b30 con 0x7fe5d80b65d0 2023-08-27T11:13:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5d77fe700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80b65d0 0x7fe5d80c8b30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800d800 tx=0x7fe5c800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5bffff700 1 -- 172.21.15.45:0/1474873691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8004030 con 0x7fe5d80b65d0 2023-08-27T11:13:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5bffff700 1 -- 172.21.15.45:0/1474873691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c80041b0 con 0x7fe5d80b65d0 2023-08-27T11:13:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d814f030 con 0x7fe5d80b65d0 2023-08-27T11:13:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5bffff700 1 -- 172.21.15.45:0/1474873691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8004560 con 0x7fe5d80b65d0 2023-08-27T11:13:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.742+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d814f5f0 con 0x7fe5d80b65d0 2023-08-27T11:13:09.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.743+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d814ff20 con 0x7fe5d80b65d0 2023-08-27T11:13:09.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.744+0000 7fe5bffff700 1 -- 172.21.15.45:0/1474873691 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe5c80036a0 con 0x7fe5d80b65d0 2023-08-27T11:13:09.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.744+0000 7fe5bffff700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5c406a0e0 0x7fe5c406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:09.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.744+0000 7fe5bffff700 1 -- 172.21.15.45:0/1474873691 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe5c8087db0 con 0x7fe5d80b65d0 2023-08-27T11:13:09.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.744+0000 7fe5d6ffd700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5c406a0e0 0x7fe5c406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:09.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.745+0000 7fe5d6ffd700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5c406a0e0 0x7fe5c406c5a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5d806c6e0 tx=0x7fe5cc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:09.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.749+0000 7fe5bffff700 1 -- 172.21.15.45:0/1474873691 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe5c8056cb0 con 0x7fe5d80b65d0 2023-08-27T11:13:09.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.948+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe5d806f4a0 con 0x7fe5d80b65d0 2023-08-27T11:13:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.948+0000 7fe5bffff700 1 -- 172.21.15.45:0/1474873691 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe5c80180a0 con 0x7fe5d80b65d0 2023-08-27T11:13:09.949 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:13:09.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5c406a0e0 msgr2=0x7fe5c406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5c406a0e0 0x7fe5c406c5a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5d806c6e0 tx=0x7fe5cc009210 comp rx=0 tx=0).stop 2023-08-27T11:13:09.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80b65d0 msgr2=0x7fe5d80c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:09.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80b65d0 0x7fe5d80c8b30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800d800 tx=0x7fe5c800db10 comp rx=0 tx=0).stop 2023-08-27T11:13:09.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 shutdown_connections 2023-08-27T11:13:09.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5d80b65d0 0x7fe5d80c8b30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5c406a0e0 0x7fe5c406c5a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5d80cfe10 0x7fe5d815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 --2- 172.21.15.45:0/1474873691 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5d80cb5c0 0x7fe5d815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:09.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 >> 172.21.15.45:0/1474873691 conn(0x7fe5d806f960 msgr2=0x7fe5d8154780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:09.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 shutdown_connections 2023-08-27T11:13:09.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:09.950+0000 7fe5dda8f700 1 -- 172.21.15.45:0/1474873691 wait complete. 2023-08-27T11:13:09.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:13:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.090+0000 7fdd5b189700 1 Processor -- start 2023-08-27T11:13:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.090+0000 7fdd5b189700 1 -- start start 2023-08-27T11:13:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.091+0000 7fdd5b189700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdd5415a0f0 0x7fdd540cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.091+0000 7fdd5b189700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd540cd300 0x7fdd5415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.091+0000 7fdd5b189700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd540cfae0 con 0x7fdd5415e030 2023-08-27T11:13:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.091+0000 7fdd5b189700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5415e900 con 0x7fdd5415a0f0 2023-08-27T11:13:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.091+0000 7fdd5b189700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540cae60 con 0x7fdd540cd300 2023-08-27T11:13:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.091+0000 7fdd58f25700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdd5415a0f0 0x7fdd540cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.091+0000 7fdd58f25700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdd5415a0f0 0x7fdd540cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53210/0 (socket says 172.21.15.45:53210) 2023-08-27T11:13:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.091+0000 7fdd58f25700 1 -- 172.21.15.45:0/3799122175 learned_addr learned my addr 172.21.15.45:0/3799122175 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.091+0000 7fdd53fff700 1 --2- 172.21.15.45:0/3799122175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd540cd300 0x7fdd5415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.092+0000 7fdd58f25700 1 -- 172.21.15.45:0/3799122175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd540cd300 msgr2=0x7fdd5415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.092+0000 7fdd58f25700 1 --2- 172.21.15.45:0/3799122175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd540cd300 0x7fdd5415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.092+0000 7fdd58f25700 1 -- 172.21.15.45:0/3799122175 >> v1:172.21.15.135:6789/0 conn(0x7fdd5415e030 legacy=0x7fdd540cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.092+0000 7fdd58f25700 1 -- 172.21.15.45:0/3799122175 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd540cb140 con 0x7fdd5415a0f0 2023-08-27T11:13:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.092+0000 7fdd58f25700 1 --2- 172.21.15.45:0/3799122175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5415a0f0 0x7fdd540cbc20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098f0 tx=0x7fdd48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b9b60b83d72ddf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.092+0000 7fdd537fe700 1 -- 172.21.15.45:0/3799122175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48011830 con 0x7fdd5415a0f0 2023-08-27T11:13:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.092+0000 7fdd537fe700 1 -- 172.21.15.45:0/3799122175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd48011990 con 0x7fdd5415a0f0 2023-08-27T11:13:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.092+0000 7fdd537fe700 1 -- 172.21.15.45:0/3799122175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48011d20 con 0x7fdd5415a0f0 2023-08-27T11:13:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.093+0000 7fdd5b189700 1 -- 172.21.15.45:0/3799122175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5415a0f0 msgr2=0x7fdd540cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.093+0000 7fdd5b189700 1 --2- 172.21.15.45:0/3799122175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5415a0f0 0x7fdd540cbc20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdd480098f0 tx=0x7fdd48009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.093+0000 7fdd5b189700 1 -- 172.21.15.45:0/3799122175 shutdown_connections 2023-08-27T11:13:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.093+0000 7fdd5b189700 1 --2- 172.21.15.45:0/3799122175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd5415a0f0 0x7fdd540cbc20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.093+0000 7fdd5b189700 1 --2- 172.21.15.45:0/3799122175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd540cd300 0x7fdd5415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.093+0000 7fdd5b189700 1 -- 172.21.15.45:0/3799122175 >> 172.21.15.45:0/3799122175 conn(0x7fdd540cc7e0 msgr2=0x7fdd540ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.093+0000 7fdd5b189700 1 -- 172.21.15.45:0/3799122175 shutdown_connections 2023-08-27T11:13:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.093+0000 7fdd5b189700 1 -- 172.21.15.45:0/3799122175 wait complete. 2023-08-27T11:13:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.094+0000 7fdd5b189700 1 Processor -- start 2023-08-27T11:13:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.094+0000 7fdd5b189700 1 -- start start 2023-08-27T11:13:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd5b189700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd540cd300 0x7fdd54149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd5b189700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd5415a0f0 0x7fdd5414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd5b189700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd5415e030 0x7fdd540b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd5b189700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540b54c0 con 0x7fdd540cd300 2023-08-27T11:13:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd5b189700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54149f50 con 0x7fdd5415a0f0 2023-08-27T11:13:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd5b189700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540b4840 con 0x7fdd5415e030 2023-08-27T11:13:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd58f25700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd540cd300 0x7fdd54149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd58f25700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd540cd300 0x7fdd54149a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53216/0 (socket says 172.21.15.45:53216) 2023-08-27T11:13:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd58f25700 1 -- 172.21.15.45:0/3886801851 learned_addr learned my addr 172.21.15.45:0/3886801851 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd58f25700 1 -- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd5415e030 msgr2=0x7fdd540b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd59726700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd5415e030 0x7fdd540b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd53fff700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd5415a0f0 0x7fdd5414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd58f25700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd5415e030 0x7fdd540b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd58f25700 1 -- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd5415a0f0 msgr2=0x7fdd5414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd58f25700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd5415a0f0 0x7fdd5414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd58f25700 1 -- 172.21.15.45:0/3886801851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009550 con 0x7fdd540cd300 2023-08-27T11:13:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd59726700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd5415e030 0x7fdd540b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd53fff700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd5415a0f0 0x7fdd5414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.095+0000 7fdd58f25700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd540cd300 0x7fdd54149a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdd48014040 tx=0x7fdd48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.096+0000 7fdd517fa700 1 -- 172.21.15.45:0/3886801851 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480036a0 con 0x7fdd540cd300 2023-08-27T11:13:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.096+0000 7fdd517fa700 1 -- 172.21.15.45:0/3886801851 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4801e070 con 0x7fdd540cd300 2023-08-27T11:13:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.096+0000 7fdd517fa700 1 -- 172.21.15.45:0/3886801851 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48003a40 con 0x7fdd540cd300 2023-08-27T11:13:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.096+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd540b4b20 con 0x7fdd540cd300 2023-08-27T11:13:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.096+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd540b50e0 con 0x7fdd540cd300 2023-08-27T11:13:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.097+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd540726d0 con 0x7fdd540cd300 2023-08-27T11:13:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.097+0000 7fdd517fa700 1 -- 172.21.15.45:0/3886801851 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdd48003ba0 con 0x7fdd540cd300 2023-08-27T11:13:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.098+0000 7fdd517fa700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdd3806a0e0 0x7fdd3806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.098+0000 7fdd517fa700 1 -- 172.21.15.45:0/3886801851 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdd48089940 con 0x7fdd540cd300 2023-08-27T11:13:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.098+0000 7fdd53fff700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdd3806a0e0 0x7fdd3806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.098+0000 7fdd53fff700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdd3806a0e0 0x7fdd3806c5a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd540c9ab0 tx=0x7fdd40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.102+0000 7fdd517fa700 1 -- 172.21.15.45:0/3886801851 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdd48054d00 con 0x7fdd540cd300 2023-08-27T11:13:10.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.302+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd540734a0 con 0x7fdd540cd300 2023-08-27T11:13:10.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.303+0000 7fdd517fa700 1 -- 172.21.15.45:0/3886801851 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd48058320 con 0x7fdd540cd300 2023-08-27T11:13:10.304 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:10.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdd3806a0e0 msgr2=0x7fdd3806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdd3806a0e0 0x7fdd3806c5a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd540c9ab0 tx=0x7fdd40008040 comp rx=0 tx=0).stop 2023-08-27T11:13:10.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd540cd300 msgr2=0x7fdd54149a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd540cd300 0x7fdd54149a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdd48014040 tx=0x7fdd48004750 comp rx=0 tx=0).stop 2023-08-27T11:13:10.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 shutdown_connections 2023-08-27T11:13:10.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd540cd300 0x7fdd54149a10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdd3806a0e0 0x7fdd3806c5a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdd5415e030 0x7fdd540b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 --2- 172.21.15.45:0/3886801851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd5415a0f0 0x7fdd5414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 >> 172.21.15.45:0/3886801851 conn(0x7fdd540cc7e0 msgr2=0x7fdd540716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 shutdown_connections 2023-08-27T11:13:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.305+0000 7fdd5b189700 1 -- 172.21.15.45:0/3886801851 wait complete. 2023-08-27T11:13:10.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:13:10.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.446+0000 7f70c1cb8700 1 Processor -- start 2023-08-27T11:13:10.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.446+0000 7f70c1cb8700 1 -- start start 2023-08-27T11:13:10.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70c1cb8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0b02c0 0x7f70bc0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70c1cb8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70bc06c260 0x7f70bc0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70c1cb8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc0d04f0 con 0x7f70bc0cdd40 2023-08-27T11:13:10.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70c1cb8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70bc0ce1d0 con 0x7f70bc06c260 2023-08-27T11:13:10.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70c1cb8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0b06a0 con 0x7f70bc0b02c0 2023-08-27T11:13:10.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70baffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70bc06c260 0x7f70bc0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70baffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70bc06c260 0x7f70bc0cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53226/0 (socket says 172.21.15.45:53226) 2023-08-27T11:13:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70baffd700 1 -- 172.21.15.45:0/4193990410 learned_addr learned my addr 172.21.15.45:0/4193990410 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70bb7fe700 1 --2- 172.21.15.45:0/4193990410 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0b02c0 0x7f70bc0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70baffd700 1 -- 172.21.15.45:0/4193990410 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0b02c0 msgr2=0x7f70bc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70baffd700 1 --2- 172.21.15.45:0/4193990410 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0b02c0 0x7f70bc0cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70baffd700 1 -- 172.21.15.45:0/4193990410 >> v1:172.21.15.135:6789/0 conn(0x7f70bc0cdd40 legacy=0x7f70bc160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70baffd700 1 -- 172.21.15.45:0/4193990410 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc0cf620 con 0x7f70bc06c260 2023-08-27T11:13:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.447+0000 7f70baffd700 1 --2- 172.21.15.45:0/4193990410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc06c260 0x7f70bc0cd800 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f70a40098f0 tx=0x7f70a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5320a999be3e3da server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70ba7fc700 1 -- 172.21.15.45:0/4193990410 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400b8a0 con 0x7f70bc06c260 2023-08-27T11:13:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70ba7fc700 1 -- 172.21.15.45:0/4193990410 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a400ba00 con 0x7f70bc06c260 2023-08-27T11:13:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70ba7fc700 1 -- 172.21.15.45:0/4193990410 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400bd90 con 0x7f70bc06c260 2023-08-27T11:13:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4193990410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc06c260 msgr2=0x7f70bc0cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70c1cb8700 1 --2- 172.21.15.45:0/4193990410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc06c260 0x7f70bc0cd800 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f70a40098f0 tx=0x7f70a4009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4193990410 shutdown_connections 2023-08-27T11:13:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70c1cb8700 1 --2- 172.21.15.45:0/4193990410 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0b02c0 0x7f70bc0cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70c1cb8700 1 --2- 172.21.15.45:0/4193990410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc06c260 0x7f70bc0cd800 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4193990410 >> 172.21.15.45:0/4193990410 conn(0x7f70bc09a1a0 msgr2=0x7f70bc0d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:10.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4193990410 shutdown_connections 2023-08-27T11:13:10.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.448+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4193990410 wait complete. 2023-08-27T11:13:10.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.449+0000 7f70c1cb8700 1 Processor -- start 2023-08-27T11:13:10.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70c1cb8700 1 -- start start 2023-08-27T11:13:10.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70c1cb8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70bc06c260 0x7f70bc0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70c1cb8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc0b02c0 0x7f70bc1467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70c1cb8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0cdd40 0x7f70bc1f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70c1cb8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc158090 con 0x7f70bc0b02c0 2023-08-27T11:13:10.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70c1cb8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc146ce0 con 0x7f70bc06c260 2023-08-27T11:13:10.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70c1cb8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc143cb0 con 0x7f70bc0cdd40 2023-08-27T11:13:10.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70baffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc0b02c0 0x7f70bc1467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70baffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc0b02c0 0x7f70bc1467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53232/0 (socket says 172.21.15.45:53232) 2023-08-27T11:13:10.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70baffd700 1 -- 172.21.15.45:0/4216939850 learned_addr learned my addr 172.21.15.45:0/4216939850 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:10.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.450+0000 7f70bb7fe700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70bc06c260 0x7f70bc0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70bbfff700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0cdd40 0x7f70bc1f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70baffd700 1 -- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0cdd40 msgr2=0x7f70bc1f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70baffd700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0cdd40 0x7f70bc1f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70baffd700 1 -- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70bc06c260 msgr2=0x7f70bc0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70baffd700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70bc06c260 0x7f70bc0c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70baffd700 1 -- 172.21.15.45:0/4216939850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4009550 con 0x7f70bc0b02c0 2023-08-27T11:13:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70bbfff700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0cdd40 0x7f70bc1f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70baffd700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc0b02c0 0x7f70bc1467a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70a4014040 tx=0x7f70a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70b3fff700 1 -- 172.21.15.45:0/4216939850 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4004590 con 0x7f70bc0b02c0 2023-08-27T11:13:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70b3fff700 1 -- 172.21.15.45:0/4216939850 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a401e070 con 0x7f70bc0b02c0 2023-08-27T11:13:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70bc144550 con 0x7f70bc0b02c0 2023-08-27T11:13:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70b3fff700 1 -- 172.21.15.45:0/4216939850 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40038e0 con 0x7f70bc0b02c0 2023-08-27T11:13:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.451+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc1f5b60 con 0x7f70bc0b02c0 2023-08-27T11:13:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.452+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70bc09eda0 con 0x7f70bc0b02c0 2023-08-27T11:13:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.452+0000 7f70b3fff700 1 -- 172.21.15.45:0/4216939850 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f70a4003a40 con 0x7f70bc0b02c0 2023-08-27T11:13:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.453+0000 7f70b3fff700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f709c06a0e0 0x7f709c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.453+0000 7f70b3fff700 1 -- 172.21.15.45:0/4216939850 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f70a4020070 con 0x7f70bc0b02c0 2023-08-27T11:13:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.453+0000 7f70bb7fe700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f709c06a0e0 0x7f709c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.454+0000 7f70bb7fe700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f709c06a0e0 0x7f709c06c5a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0d1210 tx=0x7f70ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.457+0000 7f70b3fff700 1 -- 172.21.15.45:0/4216939850 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f70a40550d0 con 0x7f70bc0b02c0 2023-08-27T11:13:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.659+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70bc0734a0 con 0x7f70bc0b02c0 2023-08-27T11:13:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.659+0000 7f70b3fff700 1 -- 172.21.15.45:0/4216939850 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70a402ee80 con 0x7f70bc0b02c0 2023-08-27T11:13:10.661 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:13:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.661+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f709c06a0e0 msgr2=0x7f709c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.661+0000 7f70c1cb8700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f709c06a0e0 0x7f709c06c5a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0d1210 tx=0x7f70ac008040 comp rx=0 tx=0).stop 2023-08-27T11:13:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.661+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc0b02c0 msgr2=0x7f70bc1467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.661+0000 7f70c1cb8700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc0b02c0 0x7f70bc1467a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70a4014040 tx=0x7f70a4004750 comp rx=0 tx=0).stop 2023-08-27T11:13:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.662+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 shutdown_connections 2023-08-27T11:13:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.662+0000 7f70c1cb8700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f709c06a0e0 0x7f709c06c5a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.662+0000 7f70c1cb8700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70bc0cdd40 0x7f70bc1f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.662+0000 7f70c1cb8700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70bc06c260 0x7f70bc0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.662+0000 7f70c1cb8700 1 --2- 172.21.15.45:0/4216939850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70bc0b02c0 0x7f70bc1467a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.662+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 >> 172.21.15.45:0/4216939850 conn(0x7f70bc09a1a0 msgr2=0x7f70bc0af300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.662+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 shutdown_connections 2023-08-27T11:13:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.662+0000 7f70c1cb8700 1 -- 172.21.15.45:0/4216939850 wait complete. 2023-08-27T11:13:10.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:13:10.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.803+0000 7fe016b9f700 1 Processor -- start 2023-08-27T11:13:10.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.803+0000 7fe016b9f700 1 -- start start 2023-08-27T11:13:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe016b9f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe010157be0 0x7fe0100716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe016b9f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe0100b17c0 0x7fe0100b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe016b9f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01015b460 con 0x7fe01006ced0 2023-08-27T11:13:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe016b9f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe010157fc0 con 0x7fe0100b17c0 2023-08-27T11:13:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe016b9f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0100b2280 con 0x7fe010157be0 2023-08-27T11:13:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe00ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe0100b17c0 0x7fe0100b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe00ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe0100b17c0 0x7fe0100b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53240/0 (socket says 172.21.15.45:53240) 2023-08-27T11:13:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe00ffff700 1 -- 172.21.15.45:0/3904101254 learned_addr learned my addr 172.21.15.45:0/3904101254 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe01513c700 1 --1- 172.21.15.45:0/3904101254 >> v1:172.21.15.45:6789/0 conn(0x7fe01006ced0 0x7fe0100cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40016/0 (socket says 172.21.15.45:40016) 2023-08-27T11:13:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe01493b700 1 --2- 172.21.15.45:0/3904101254 >> v2:172.21.15.135:3300/0 conn(0x7fe010157be0 0x7fe0100716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe00ffff700 1 -- 172.21.15.45:0/3904101254 >> v2:172.21.15.135:3300/0 conn(0x7fe010157be0 msgr2=0x7fe0100716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe00ffff700 1 --2- 172.21.15.45:0/3904101254 >> v2:172.21.15.135:3300/0 conn(0x7fe010157be0 0x7fe0100716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe00ffff700 1 -- 172.21.15.45:0/3904101254 >> v1:172.21.15.45:6789/0 conn(0x7fe01006ced0 legacy=0x7fe0100cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe00f7fe700 1 -- 172.21.15.45:0/3904101254 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861426848 0 0) 0x7fe01015b460 con 0x7fe01006ced0 2023-08-27T11:13:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.804+0000 7fe00ffff700 1 -- 172.21.15.45:0/3904101254 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100b2560 con 0x7fe0100b17c0 2023-08-27T11:13:10.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe01493b700 1 --2- 172.21.15.45:0/3904101254 >> v2:172.21.15.135:3300/0 conn(0x7fe010157be0 0x7fe0100716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe00ffff700 1 --2- 172.21.15.45:0/3904101254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 0x7fe0100b1c70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe010066e70 tx=0x7fdffc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344e7067c2326f04 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe00f7fe700 1 -- 172.21.15.45:0/3904101254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc00b870 con 0x7fe0100b17c0 2023-08-27T11:13:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe00f7fe700 1 -- 172.21.15.45:0/3904101254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc00b9d0 con 0x7fe0100b17c0 2023-08-27T11:13:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe00f7fe700 1 -- 172.21.15.45:0/3904101254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc00bd60 con 0x7fe0100b17c0 2023-08-27T11:13:10.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe016b9f700 1 -- 172.21.15.45:0/3904101254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 msgr2=0x7fe0100b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe016b9f700 1 --2- 172.21.15.45:0/3904101254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 0x7fe0100b1c70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe010066e70 tx=0x7fdffc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe016b9f700 1 -- 172.21.15.45:0/3904101254 shutdown_connections 2023-08-27T11:13:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe016b9f700 1 --2- 172.21.15.45:0/3904101254 >> v2:172.21.15.135:3300/0 conn(0x7fe010157be0 0x7fe0100716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe016b9f700 1 --2- 172.21.15.45:0/3904101254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 0x7fe0100b1c70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe016b9f700 1 -- 172.21.15.45:0/3904101254 >> 172.21.15.45:0/3904101254 conn(0x7fe0100cc7c0 msgr2=0x7fe0100ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe016b9f700 1 -- 172.21.15.45:0/3904101254 shutdown_connections 2023-08-27T11:13:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.805+0000 7fe016b9f700 1 -- 172.21.15.45:0/3904101254 wait complete. 2023-08-27T11:13:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.807+0000 7fe016b9f700 1 Processor -- start 2023-08-27T11:13:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.807+0000 7fe016b9f700 1 -- start start 2023-08-27T11:13:10.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe016b9f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe01006ced0 0x7fe010154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe016b9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 0x7fe0100c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe016b9f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe010157be0 0x7fe0101560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe016b9f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101f2840 con 0x7fe0100b17c0 2023-08-27T11:13:10.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe016b9f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100c3bc0 con 0x7fe010157be0 2023-08-27T11:13:10.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe016b9f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101f2a90 con 0x7fe01006ced0 2023-08-27T11:13:10.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 0x7fe0100c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 0x7fe0100c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53244/0 (socket says 172.21.15.45:53244) 2023-08-27T11:13:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00ffff700 1 -- 172.21.15.45:0/3530221084 learned_addr learned my addr 172.21.15.45:0/3530221084 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe01513c700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe010157be0 0x7fe0101560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe01493b700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe01006ced0 0x7fe010154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00ffff700 1 -- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe01006ced0 msgr2=0x7fe010154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00ffff700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe01006ced0 0x7fe010154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00ffff700 1 -- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe010157be0 msgr2=0x7fe0101560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:10.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00ffff700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe010157be0 0x7fe0101560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:10.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00ffff700 1 -- 172.21.15.45:0/3530221084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc009550 con 0x7fe0100b17c0 2023-08-27T11:13:10.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe01493b700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe01006ced0 0x7fe010154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:10.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe01513c700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe010157be0 0x7fe0101560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:10.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00ffff700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 0x7fe0100c6440 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdffc014040 tx=0x7fdffc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:10.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00d7fa700 1 -- 172.21.15.45:0/3530221084 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc004520 con 0x7fe0100b17c0 2023-08-27T11:13:10.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00d7fa700 1 -- 172.21.15.45:0/3530221084 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc01e070 con 0x7fe0100b17c0 2023-08-27T11:13:10.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0101f2d70 con 0x7fe0100b17c0 2023-08-27T11:13:10.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.808+0000 7fe00d7fa700 1 -- 172.21.15.45:0/3530221084 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0038a0 con 0x7fe0100b17c0 2023-08-27T11:13:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.809+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0101f3330 con 0x7fe0100b17c0 2023-08-27T11:13:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.809+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0101f3520 con 0x7fe0100b17c0 2023-08-27T11:13:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.810+0000 7fe00d7fa700 1 -- 172.21.15.45:0/3530221084 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdffc003a00 con 0x7fe0100b17c0 2023-08-27T11:13:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.810+0000 7fe00d7fa700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdff406a0e0 0x7fdff406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.810+0000 7fe00d7fa700 1 -- 172.21.15.45:0/3530221084 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdffc020070 con 0x7fe0100b17c0 2023-08-27T11:13:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.811+0000 7fe01493b700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdff406a0e0 0x7fdff406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.811+0000 7fe01493b700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdff406a0e0 0x7fdff406c5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0100cef10 tx=0x7fe004008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:10.815+0000 7fe00d7fa700 1 -- 172.21.15.45:0/3530221084 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdffc0551c0 con 0x7fe0100b17c0 2023-08-27T11:13:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.015+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe010072690 con 0x7fe0100b17c0 2023-08-27T11:13:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.015+0000 7fe00d7fa700 1 -- 172.21.15.45:0/3530221084 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdffc003cb0 con 0x7fe0100b17c0 2023-08-27T11:13:11.017 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.017+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdff406a0e0 msgr2=0x7fdff406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.017+0000 7fe016b9f700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdff406a0e0 0x7fdff406c5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0100cef10 tx=0x7fe004008040 comp rx=0 tx=0).stop 2023-08-27T11:13:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.017+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 msgr2=0x7fe0100c6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.017+0000 7fe016b9f700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 0x7fe0100c6440 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdffc014040 tx=0x7fdffc004750 comp rx=0 tx=0).stop 2023-08-27T11:13:11.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.018+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 shutdown_connections 2023-08-27T11:13:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.018+0000 7fe016b9f700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdff406a0e0 0x7fdff406c5a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.018+0000 7fe016b9f700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe010157be0 0x7fe0101560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.018+0000 7fe016b9f700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe01006ced0 0x7fe010154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.018+0000 7fe016b9f700 1 --2- 172.21.15.45:0/3530221084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0100b17c0 0x7fe0100c6440 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.018+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 >> 172.21.15.45:0/3530221084 conn(0x7fe0100cc7c0 msgr2=0x7fe0100ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.018+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 shutdown_connections 2023-08-27T11:13:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.018+0000 7fe016b9f700 1 -- 172.21.15.45:0/3530221084 wait complete. 2023-08-27T11:13:11.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:13:11.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.158+0000 7f8e84d57700 1 Processor -- start 2023-08-27T11:13:11.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.158+0000 7f8e84d57700 1 -- start start 2023-08-27T11:13:11.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.159+0000 7f8e84d57700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e800ca620 0x7f8e800cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.159+0000 7f8e84d57700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800caa00 0x7f8e800acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.159+0000 7f8e84d57700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e800ce340 con 0x7f8e800af200 2023-08-27T11:13:11.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.159+0000 7f8e84d57700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e800ceec0 con 0x7f8e800ca620 2023-08-27T11:13:11.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.159+0000 7f8e84d57700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8015d7c0 con 0x7f8e800caa00 2023-08-27T11:13:11.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.159+0000 7f8e7e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e800ca620 0x7f8e800cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8e800af200 0x7f8e800ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40026/0 (socket says 172.21.15.45:40026) 2023-08-27T11:13:11.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e800ca620 0x7f8e800cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53250/0 (socket says 172.21.15.45:53250) 2023-08-27T11:13:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7dd9b700 1 -- 172.21.15.45:0/2926911971 learned_addr learned my addr 172.21.15.45:0/2926911971 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7e59c700 1 -- 172.21.15.45:0/2926911971 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800caa00 msgr2=0x7f8e800acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7e59c700 1 --2- 172.21.15.45:0/2926911971 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800caa00 0x7f8e800acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7e59c700 1 -- 172.21.15.45:0/2926911971 >> v1:172.21.15.45:6789/0 conn(0x7f8e800af200 legacy=0x7f8e800ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7d59a700 1 -- 172.21.15.45:0/2926911971 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76161504 0 0) 0x7f8e800ce340 con 0x7f8e800af200 2023-08-27T11:13:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7e59c700 1 -- 172.21.15.45:0/2926911971 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e800cc570 con 0x7f8e800ca620 2023-08-27T11:13:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7e59c700 1 --2- 172.21.15.45:0/2926911971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 0x7f8e800cf350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009840 tx=0x7f8e74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadbab284c18b49a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7d59a700 1 -- 172.21.15.45:0/2926911971 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74011830 con 0x7f8e800ca620 2023-08-27T11:13:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7d59a700 1 -- 172.21.15.45:0/2926911971 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e74011990 con 0x7f8e800ca620 2023-08-27T11:13:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.160+0000 7f8e7d59a700 1 -- 172.21.15.45:0/2926911971 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74011d20 con 0x7f8e800ca620 2023-08-27T11:13:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.161+0000 7f8e84d57700 1 -- 172.21.15.45:0/2926911971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 msgr2=0x7f8e800cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.161+0000 7f8e84d57700 1 --2- 172.21.15.45:0/2926911971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 0x7f8e800cf350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009840 tx=0x7f8e74009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.161+0000 7f8e84d57700 1 -- 172.21.15.45:0/2926911971 shutdown_connections 2023-08-27T11:13:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.161+0000 7f8e84d57700 1 --2- 172.21.15.45:0/2926911971 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800caa00 0x7f8e800acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.161+0000 7f8e84d57700 1 --2- 172.21.15.45:0/2926911971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 0x7f8e800cf350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.161+0000 7f8e84d57700 1 -- 172.21.15.45:0/2926911971 >> 172.21.15.45:0/2926911971 conn(0x7f8e800cc7e0 msgr2=0x7f8e800ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.161+0000 7f8e84d57700 1 -- 172.21.15.45:0/2926911971 shutdown_connections 2023-08-27T11:13:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.161+0000 7f8e84d57700 1 -- 172.21.15.45:0/2926911971 wait complete. 2023-08-27T11:13:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.163+0000 7f8e84d57700 1 Processor -- start 2023-08-27T11:13:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.163+0000 7f8e84d57700 1 -- start start 2023-08-27T11:13:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e84d57700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800af200 0x7f8e800c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e84d57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 0x7f8e80157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e84d57700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e800caa00 0x7f8e800c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e84d57700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800b2e50 con 0x7f8e800ca620 2023-08-27T11:13:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e84d57700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8015b180 con 0x7f8e800caa00 2023-08-27T11:13:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e84d57700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8015b460 con 0x7f8e800af200 2023-08-27T11:13:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 0x7f8e80157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 0x7f8e80157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53258/0 (socket says 172.21.15.45:53258) 2023-08-27T11:13:11.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7dd9b700 1 -- 172.21.15.45:0/1325295459 learned_addr learned my addr 172.21.15.45:0/1325295459 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7e59c700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800af200 0x7f8e800c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7ed9d700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e800caa00 0x7f8e800c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7dd9b700 1 -- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800af200 msgr2=0x7f8e800c33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7dd9b700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800af200 0x7f8e800c33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7dd9b700 1 -- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e800caa00 msgr2=0x7f8e800c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7dd9b700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e800caa00 0x7f8e800c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7dd9b700 1 -- 172.21.15.45:0/1325295459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740094a0 con 0x7f8e800ca620 2023-08-27T11:13:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7e59c700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800af200 0x7f8e800c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7ed9d700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e800caa00 0x7f8e800c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:11.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e7dd9b700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 0x7f8e80157010 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000c540 tx=0x7f8e7000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e6effd700 1 -- 172.21.15.45:0/1325295459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70004c80 con 0x7f8e800ca620 2023-08-27T11:13:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e6effd700 1 -- 172.21.15.45:0/1325295459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e70004de0 con 0x7f8e800ca620 2023-08-27T11:13:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e6effd700 1 -- 172.21.15.45:0/1325295459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70007310 con 0x7f8e800ca620 2023-08-27T11:13:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8015b740 con 0x7f8e800ca620 2023-08-27T11:13:11.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.164+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e800c3900 con 0x7f8e800ca620 2023-08-27T11:13:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.165+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8015b960 con 0x7f8e800ca620 2023-08-27T11:13:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.166+0000 7f8e6effd700 1 -- 172.21.15.45:0/1325295459 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8e70007470 con 0x7f8e800ca620 2023-08-27T11:13:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.166+0000 7f8e6effd700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e6006a0e0 0x7f8e6006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.166+0000 7f8e6effd700 1 -- 172.21.15.45:0/1325295459 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8e70088670 con 0x7f8e800ca620 2023-08-27T11:13:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.166+0000 7f8e7e59c700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e6006a0e0 0x7f8e6006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.167+0000 7f8e7e59c700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e6006a0e0 0x7f8e6006c5a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8e80159410 tx=0x7f8e74008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:11.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.171+0000 7f8e6effd700 1 -- 172.21.15.45:0/1325295459 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8e70053a30 con 0x7f8e800ca620 2023-08-27T11:13:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.392+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8e800cbce0 con 0x7f8e800ca620 2023-08-27T11:13:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.392+0000 7f8e6effd700 1 -- 172.21.15.45:0/1325295459 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8e70057050 con 0x7f8e800ca620 2023-08-27T11:13:11.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e6006a0e0 msgr2=0x7f8e6006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e6006a0e0 0x7f8e6006c5a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8e80159410 tx=0x7f8e74008030 comp rx=0 tx=0).stop 2023-08-27T11:13:11.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 msgr2=0x7f8e80157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 0x7f8e80157010 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000c540 tx=0x7f8e7000c900 comp rx=0 tx=0).stop 2023-08-27T11:13:11.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 shutdown_connections 2023-08-27T11:13:11.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f8e6006a0e0 0x7f8e6006c5a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e800caa00 0x7f8e800c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8e800af200 0x7f8e800c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 --2- 172.21.15.45:0/1325295459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e800ca620 0x7f8e80157010 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 >> 172.21.15.45:0/1325295459 conn(0x7f8e800cc7e0 msgr2=0x7f8e800ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 shutdown_connections 2023-08-27T11:13:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.394+0000 7f8e84d57700 1 -- 172.21.15.45:0/1325295459 wait complete. 2023-08-27T11:13:11.398 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2023-08-27T11:13:11.407 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:13:11.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:11.408 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-08-27T11:13:11.450 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:13:11.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:11.451 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:13:11.508 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:13:11.508 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:11.530 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:13:11.530 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:11.590 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:13:11.590 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:11.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:11.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.774+0000 7f1d6cae1700 1 Processor -- start 2023-08-27T11:13:11.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.774+0000 7f1d6cae1700 1 -- start start 2023-08-27T11:13:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.775+0000 7f1d6cae1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d68155660 0x7f1d6806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.775+0000 7f1d6cae1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680cbf00 con 0x7f1d68070e80 2023-08-27T11:13:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.775+0000 7f1d6cae1700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680cc180 con 0x7f1d6806a990 2023-08-27T11:13:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.775+0000 7f1d6cae1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680caa60 con 0x7f1d68155660 2023-08-27T11:13:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.775+0000 7f1d65d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1d68070e80 0x7f1d6806a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40040/0 (socket says 172.21.15.45:40040) 2023-08-27T11:13:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.775+0000 7f1d65d9b700 1 -- 172.21.15.45:0/3414644831 learned_addr learned my addr 172.21.15.45:0/3414644831 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.775+0000 7f1d6659c700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d68155660 0x7f1d6806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.775+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227174895 0 0) 0x7f1d680cbf00 con 0x7f1d68070e80 2023-08-27T11:13:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.775+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d540035b0 con 0x7f1d68070e80 2023-08-27T11:13:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619905718 0 0) 0x7f1d680cc180 con 0x7f1d6806a990 2023-08-27T11:13:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d680cbf00 con 0x7f1d6806a990 2023-08-27T11:13:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2744535294 0 0) 0x7f1d540035b0 con 0x7f1d68070e80 2023-08-27T11:13:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d68155660 msgr2=0x7f1d6806e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d68155660 0x7f1d6806e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 >> v1:172.21.15.135:6789/0 conn(0x7f1d6806a990 legacy=0x7f1d6806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d680ca4a0 con 0x7f1d68070e80 2023-08-27T11:13:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f1d50001670 con 0x7f1d68070e80 2023-08-27T11:13:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 >> v1:172.21.15.45:6789/0 conn(0x7f1d68070e80 legacy=0x7f1d6806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d54003bc0 0x7f1d54003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d68155660 0x7f1d54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d54004d90 0x7f1d54005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680caa60 con 0x7f1d54004d90 2023-08-27T11:13:11.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5400bba0 con 0x7f1d54003bc0 2023-08-27T11:13:11.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5400bd30 con 0x7f1d68155660 2023-08-27T11:13:11.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d66d9d700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d54004d90 0x7f1d54005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d6659c700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d54003bc0 0x7f1d54003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d65d9b700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d68155660 0x7f1d54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d66d9d700 1 -- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d68155660 msgr2=0x7f1d54004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d66d9d700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d68155660 0x7f1d54004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d66d9d700 1 -- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d54003bc0 msgr2=0x7f1d54003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d66d9d700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d54003bc0 0x7f1d54003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d66d9d700 1 -- 172.21.15.45:0/3414644831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540035b0 con 0x7f1d54004d90 2023-08-27T11:13:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.776+0000 7f1d66d9d700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d54004d90 0x7f1d54005210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000c320 tx=0x7f1d54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc3aa016690b6b9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60001f30 con 0x7f1d54004d90 2023-08-27T11:13:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6000ce80 con 0x7f1d54004d90 2023-08-27T11:13:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d6559a700 1 -- 172.21.15.45:0/3414644831 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60003c40 con 0x7f1d54004d90 2023-08-27T11:13:11.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d6cae1700 1 -- 172.21.15.45:0/3414644831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d54004d90 msgr2=0x7f1d54005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d6cae1700 1 --2- 172.21.15.45:0/3414644831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d54004d90 0x7f1d54005210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000c320 tx=0x7f1d54005880 comp rx=0 tx=0).stop 2023-08-27T11:13:11.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d66d9d700 1 -- 172.21.15.45:0/3414644831 reap_dead start 2023-08-27T11:13:11.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d6cae1700 1 -- 172.21.15.45:0/3414644831 shutdown_connections 2023-08-27T11:13:11.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d6cae1700 1 -- 172.21.15.45:0/3414644831 >> 172.21.15.45:0/3414644831 conn(0x7f1d680cfe50 msgr2=0x7f1d680d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d6cae1700 1 -- 172.21.15.45:0/3414644831 shutdown_connections 2023-08-27T11:13:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.777+0000 7f1d6cae1700 1 -- 172.21.15.45:0/3414644831 wait complete. 2023-08-27T11:13:11.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.779+0000 7f1d6cae1700 1 Processor -- start 2023-08-27T11:13:11.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.779+0000 7f1d6cae1700 1 -- start start 2023-08-27T11:13:11.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.779+0000 7f1d6cae1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d54004d90 0x7f1d68152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.779+0000 7f1d6cae1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d680aefd0 0x7f1d680c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.779+0000 7f1d6cae1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d680ac3d0 0x7f1d680ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.779+0000 7f1d6cae1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680c58c0 con 0x7f1d680ac3d0 2023-08-27T11:13:11.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.779+0000 7f1d6cae1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680acd90 con 0x7f1d680aefd0 2023-08-27T11:13:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.779+0000 7f1d6cae1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d681f25c0 con 0x7f1d54004d90 2023-08-27T11:13:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d66d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d680ac3d0 0x7f1d680ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d66d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d680ac3d0 0x7f1d680ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53280/0 (socket says 172.21.15.45:53280) 2023-08-27T11:13:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d66d9d700 1 -- 172.21.15.45:0/950355615 learned_addr learned my addr 172.21.15.45:0/950355615 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d65d9b700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d680aefd0 0x7f1d680c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d6659c700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d54004d90 0x7f1d68152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d66d9d700 1 -- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d54004d90 msgr2=0x7f1d68152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d66d9d700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d54004d90 0x7f1d68152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d66d9d700 1 -- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d680aefd0 msgr2=0x7f1d680c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d66d9d700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d680aefd0 0x7f1d680c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d66d9d700 1 -- 172.21.15.45:0/950355615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d681f1c90 con 0x7f1d680ac3d0 2023-08-27T11:13:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d6659c700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d54004d90 0x7f1d68152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d66d9d700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d680ac3d0 0x7f1d680ac850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d60004000 tx=0x7f1d60001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d5effd700 1 -- 172.21.15.45:0/950355615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000cc80 con 0x7f1d680ac3d0 2023-08-27T11:13:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d5effd700 1 -- 172.21.15.45:0/950355615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d600050d0 con 0x7f1d680ac3d0 2023-08-27T11:13:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d681f1f70 con 0x7f1d680ac3d0 2023-08-27T11:13:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d5effd700 1 -- 172.21.15.45:0/950355615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60005460 con 0x7f1d680ac3d0 2023-08-27T11:13:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.780+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d68152790 con 0x7f1d680ac3d0 2023-08-27T11:13:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.781+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d68152920 con 0x7f1d680ac3d0 2023-08-27T11:13:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.782+0000 7f1d5effd700 1 -- 172.21.15.45:0/950355615 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1d60019070 con 0x7f1d680ac3d0 2023-08-27T11:13:11.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.782+0000 7f1d5effd700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1d4806a0e0 0x7f1d4806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:11.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.782+0000 7f1d5effd700 1 -- 172.21.15.45:0/950355615 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1d60089390 con 0x7f1d680ac3d0 2023-08-27T11:13:11.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.782+0000 7f1d6659c700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1d4806a0e0 0x7f1d4806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:11.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.783+0000 7f1d6659c700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1d4806a0e0 0x7f1d4806c5a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1d68152ad0 tx=0x7f1d58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:11.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.787+0000 7f1d5effd700 1 -- 172.21.15.45:0/950355615 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1d600547e0 con 0x7f1d680ac3d0 2023-08-27T11:13:11.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.989+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d68072810 con 0x7f1d680ac3d0 2023-08-27T11:13:11.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.991+0000 7f1d5effd700 1 -- 172.21.15.45:0/950355615 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7f1d600011c0 con 0x7f1d680ac3d0 2023-08-27T11:13:11.992 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:11.992 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:11.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1d4806a0e0 msgr2=0x7f1d4806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1d4806a0e0 0x7f1d4806c5a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1d68152ad0 tx=0x7f1d58008040 comp rx=0 tx=0).stop 2023-08-27T11:13:11.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d680ac3d0 msgr2=0x7f1d680ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d680ac3d0 0x7f1d680ac850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d60004000 tx=0x7f1d60001650 comp rx=0 tx=0).stop 2023-08-27T11:13:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 shutdown_connections 2023-08-27T11:13:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d680ac3d0 0x7f1d680ac850 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1d4806a0e0 0x7f1d4806c5a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1d54004d90 0x7f1d68152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 --2- 172.21.15.45:0/950355615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d680aefd0 0x7f1d680c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:11.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 >> 172.21.15.45:0/950355615 conn(0x7f1d680cfe50 msgr2=0x7f1d6806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:11.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 shutdown_connections 2023-08-27T11:13:11.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:11.993+0000 7f1d6cae1700 1 -- 172.21.15.45:0/950355615 wait complete. 2023-08-27T11:13:12.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:13:12.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:13:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.135+0000 7f470de71700 1 Processor -- start 2023-08-27T11:13:12.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.135+0000 7f470de71700 1 -- start start 2023-08-27T11:13:12.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.135+0000 7f470de71700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 0x7f47080cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.135+0000 7f470de71700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47080cdd40 0x7f470815ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.135+0000 7f470de71700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47080d04f0 con 0x7f47080b02c0 2023-08-27T11:13:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.135+0000 7f470de71700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47080ce1d0 con 0x7f47080cdd40 2023-08-27T11:13:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.135+0000 7f470de71700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47080b06a0 con 0x7f470806c260 2023-08-27T11:13:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.135+0000 7f4707fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47080cdd40 0x7f470815ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f4707fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47080cdd40 0x7f470815ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53288/0 (socket says 172.21.15.45:53288) 2023-08-27T11:13:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f4707fff700 1 -- 172.21.15.45:0/959146025 learned_addr learned my addr 172.21.15.45:0/959146025 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f4706ffd700 1 --2- 172.21.15.45:0/959146025 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 0x7f47080cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f4707fff700 1 -- 172.21.15.45:0/959146025 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 msgr2=0x7f47080cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:12.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f4707fff700 1 --2- 172.21.15.45:0/959146025 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 0x7f47080cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f4707fff700 1 -- 172.21.15.45:0/959146025 >> v1:172.21.15.135:6789/0 conn(0x7f47080b02c0 legacy=0x7f47080cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f4707fff700 1 -- 172.21.15.45:0/959146025 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47080cf8e0 con 0x7f47080cdd40 2023-08-27T11:13:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f4706ffd700 1 --2- 172.21.15.45:0/959146025 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 0x7f47080cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f4707fff700 1 --2- 172.21.15.45:0/959146025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080cdd40 0x7f470815ca80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4700009e50 tx=0x7f470000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c392e81cb579b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:12.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f47067fc700 1 -- 172.21.15.45:0/959146025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000e070 con 0x7f47080cdd40 2023-08-27T11:13:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f47067fc700 1 -- 172.21.15.45:0/959146025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470000b9c0 con 0x7f47080cdd40 2023-08-27T11:13:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.136+0000 7f47067fc700 1 -- 172.21.15.45:0/959146025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000bd70 con 0x7f47080cdd40 2023-08-27T11:13:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.137+0000 7f470de71700 1 -- 172.21.15.45:0/959146025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080cdd40 msgr2=0x7f470815ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.137+0000 7f470de71700 1 --2- 172.21.15.45:0/959146025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080cdd40 0x7f470815ca80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4700009e50 tx=0x7f470000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.137+0000 7f470de71700 1 -- 172.21.15.45:0/959146025 shutdown_connections 2023-08-27T11:13:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.137+0000 7f470de71700 1 --2- 172.21.15.45:0/959146025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080cdd40 0x7f470815ca80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.137+0000 7f470de71700 1 --2- 172.21.15.45:0/959146025 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 0x7f47080cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.137+0000 7f470de71700 1 -- 172.21.15.45:0/959146025 >> 172.21.15.45:0/959146025 conn(0x7f470809a1a0 msgr2=0x7f47080d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.137+0000 7f470de71700 1 -- 172.21.15.45:0/959146025 shutdown_connections 2023-08-27T11:13:12.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.137+0000 7f470de71700 1 -- 172.21.15.45:0/959146025 wait complete. 2023-08-27T11:13:12.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.138+0000 7f470de71700 1 Processor -- start 2023-08-27T11:13:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.138+0000 7f470de71700 1 -- start start 2023-08-27T11:13:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f470de71700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 0x7f47080c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f470de71700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080b02c0 0x7f470809ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f470de71700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080cdd40 0x7f47081523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f470de71700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470809b270 con 0x7f47080b02c0 2023-08-27T11:13:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f470de71700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708152900 con 0x7f47080cdd40 2023-08-27T11:13:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f470de71700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470809ce90 con 0x7f470806c260 2023-08-27T11:13:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f4706ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080b02c0 0x7f470809ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f4706ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080b02c0 0x7f470809ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53292/0 (socket says 172.21.15.45:53292) 2023-08-27T11:13:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f4706ffd700 1 -- 172.21.15.45:0/1591284856 learned_addr learned my addr 172.21.15.45:0/1591284856 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f4707fff700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080cdd40 0x7f47081523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f47077fe700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 0x7f47080c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f4706ffd700 1 -- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 msgr2=0x7f47080c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f4706ffd700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 0x7f47080c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f4706ffd700 1 -- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080cdd40 msgr2=0x7f47081523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f4706ffd700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080cdd40 0x7f47081523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.139+0000 7f4706ffd700 1 -- 172.21.15.45:0/1591284856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4700009b30 con 0x7f47080b02c0 2023-08-27T11:13:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.140+0000 7f4706ffd700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080b02c0 0x7f470809ad30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00e810 tx=0x7f46fc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:12.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.140+0000 7f46effff700 1 -- 172.21.15.45:0/1591284856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00c070 con 0x7f47080b02c0 2023-08-27T11:13:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.140+0000 7f46effff700 1 -- 172.21.15.45:0/1591284856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc004230 con 0x7f47080b02c0 2023-08-27T11:13:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.140+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470809d170 con 0x7f47080b02c0 2023-08-27T11:13:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.140+0000 7f46effff700 1 -- 172.21.15.45:0/1591284856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0036a0 con 0x7f47080b02c0 2023-08-27T11:13:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.140+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470809d730 con 0x7f47080b02c0 2023-08-27T11:13:12.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.141+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470809dd80 con 0x7f47080b02c0 2023-08-27T11:13:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.141+0000 7f46effff700 1 -- 172.21.15.45:0/1591284856 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f46fc003800 con 0x7f47080b02c0 2023-08-27T11:13:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.141+0000 7f46effff700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f46f406a0e0 0x7f46f406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.141+0000 7f46effff700 1 -- 172.21.15.45:0/1591284856 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f46fc088510 con 0x7f47080b02c0 2023-08-27T11:13:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.142+0000 7f47077fe700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f46f406a0e0 0x7f46f406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.142+0000 7f47077fe700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f46f406a0e0 0x7f46f406c5a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f46f80018a0 tx=0x7f46f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:12.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.146+0000 7f46effff700 1 -- 172.21.15.45:0/1591284856 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f46fc053960 con 0x7f47080b02c0 2023-08-27T11:13:12.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.350+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f470809eda0 con 0x7f47080b02c0 2023-08-27T11:13:12.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.350+0000 7f46effff700 1 -- 172.21.15.45:0/1591284856 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46fc056f80 con 0x7f47080b02c0 2023-08-27T11:13:12.351 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:13:12.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f46f406a0e0 msgr2=0x7f46f406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f46f406a0e0 0x7f46f406c5a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f46f80018a0 tx=0x7f46f8009040 comp rx=0 tx=0).stop 2023-08-27T11:13:12.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080b02c0 msgr2=0x7f470809ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080b02c0 0x7f470809ad30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00e810 tx=0x7f46fc00eb20 comp rx=0 tx=0).stop 2023-08-27T11:13:12.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 shutdown_connections 2023-08-27T11:13:12.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f46f406a0e0 0x7f46f406c5a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47080cdd40 0x7f47081523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470806c260 0x7f47080c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 --2- 172.21.15.45:0/1591284856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47080b02c0 0x7f470809ad30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 >> 172.21.15.45:0/1591284856 conn(0x7f470809a1a0 msgr2=0x7f47080d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:12.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 shutdown_connections 2023-08-27T11:13:12.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.352+0000 7f470de71700 1 -- 172.21.15.45:0/1591284856 wait complete. 2023-08-27T11:13:12.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:13:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.492+0000 7fdce5ca4700 1 Processor -- start 2023-08-27T11:13:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.493+0000 7fdce5ca4700 1 -- start start 2023-08-27T11:13:12.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.493+0000 7fdce5ca4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdce00cd300 0x7fdce00cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.493+0000 7fdce5ca4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce00cfae0 con 0x7fdce015a0f0 2023-08-27T11:13:12.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.493+0000 7fdce5ca4700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce015e900 con 0x7fdce006ed90 2023-08-27T11:13:12.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.493+0000 7fdce5ca4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce00cae60 con 0x7fdce00cd300 2023-08-27T11:13:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.493+0000 7fdcdeffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdce00cd300 0x7fdce00cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.493+0000 7fdcdf7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdce015a0f0 0x7fdce015daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40074/0 (socket says 172.21.15.45:40074) 2023-08-27T11:13:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.493+0000 7fdcdf7fe700 1 -- 172.21.15.45:0/1895233847 learned_addr learned my addr 172.21.15.45:0/1895233847 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:12.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.494+0000 7fdcdeffd700 1 -- 172.21.15.45:0/1895233847 >> v1:172.21.15.135:6789/0 conn(0x7fdce006ed90 legacy=0x7fdce00cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.494+0000 7fdcdeffd700 1 -- 172.21.15.45:0/1895233847 >> v1:172.21.15.45:6789/0 conn(0x7fdce015a0f0 legacy=0x7fdce015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.494+0000 7fdcdeffd700 1 -- 172.21.15.45:0/1895233847 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce00cb140 con 0x7fdce00cd300 2023-08-27T11:13:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.494+0000 7fdcde7fc700 1 -- 172.21.15.45:0/1895233847 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784243255 0 0) 0x7fdce00cfae0 con 0x7fdce015a0f0 2023-08-27T11:13:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.494+0000 7fdcdeffd700 1 --2- 172.21.15.45:0/1895233847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce00cd300 0x7fdce00cbc20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009840 tx=0x7fdcd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02f212c0b9ce339 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:12.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.494+0000 7fdcde7fc700 1 -- 172.21.15.45:0/1895233847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0011830 con 0x7fdce00cd300 2023-08-27T11:13:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.494+0000 7fdcde7fc700 1 -- 172.21.15.45:0/1895233847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0011990 con 0x7fdce00cd300 2023-08-27T11:13:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.494+0000 7fdcde7fc700 1 -- 172.21.15.45:0/1895233847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0011d20 con 0x7fdce00cd300 2023-08-27T11:13:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.495+0000 7fdce5ca4700 1 -- 172.21.15.45:0/1895233847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce00cd300 msgr2=0x7fdce00cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.495+0000 7fdce5ca4700 1 --2- 172.21.15.45:0/1895233847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce00cd300 0x7fdce00cbc20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009840 tx=0x7fdcd0009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:12.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.495+0000 7fdce5ca4700 1 -- 172.21.15.45:0/1895233847 shutdown_connections 2023-08-27T11:13:12.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.495+0000 7fdce5ca4700 1 --2- 172.21.15.45:0/1895233847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce00cd300 0x7fdce00cbc20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.495+0000 7fdce5ca4700 1 -- 172.21.15.45:0/1895233847 >> 172.21.15.45:0/1895233847 conn(0x7fdce00cc7e0 msgr2=0x7fdce00ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:12.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.495+0000 7fdce5ca4700 1 -- 172.21.15.45:0/1895233847 shutdown_connections 2023-08-27T11:13:12.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.495+0000 7fdce5ca4700 1 -- 172.21.15.45:0/1895233847 wait complete. 2023-08-27T11:13:12.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.496+0000 7fdce5ca4700 1 Processor -- start 2023-08-27T11:13:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.496+0000 7fdce5ca4700 1 -- start start 2023-08-27T11:13:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdce5ca4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdce006ed90 0x7fdce00c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdce5ca4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdce00cd300 0x7fdce00c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdce5ca4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce015a0f0 0x7fdce01eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdce5ca4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00c7d80 con 0x7fdce015a0f0 2023-08-27T11:13:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdce5ca4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00c8830 con 0x7fdce006ed90 2023-08-27T11:13:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdce5ca4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00c7f90 con 0x7fdce00cd300 2023-08-27T11:13:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce015a0f0 0x7fdce01eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce015a0f0 0x7fdce01eeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53308/0 (socket says 172.21.15.45:53308) 2023-08-27T11:13:12.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdffff700 1 -- 172.21.15.45:0/185020026 learned_addr learned my addr 172.21.15.45:0/185020026 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdf7fe700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdce006ed90 0x7fdce00c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdeffd700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdce00cd300 0x7fdce00c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdffff700 1 -- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdce00cd300 msgr2=0x7fdce00c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdffff700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdce00cd300 0x7fdce00c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdffff700 1 -- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdce006ed90 msgr2=0x7fdce00c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdffff700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdce006ed90 0x7fdce00c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.497+0000 7fdcdffff700 1 -- 172.21.15.45:0/185020026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd00094a0 con 0x7fdce015a0f0 2023-08-27T11:13:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.498+0000 7fdcdffff700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce015a0f0 0x7fdce01eeec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80021c0 tx=0x7fdcd80021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.498+0000 7fdcd7fff700 1 -- 172.21.15.45:0/185020026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8003a00 con 0x7fdce015a0f0 2023-08-27T11:13:12.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.498+0000 7fdcd7fff700 1 -- 172.21.15.45:0/185020026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd8003b60 con 0x7fdce015a0f0 2023-08-27T11:13:12.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.498+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce00c8270 con 0x7fdce015a0f0 2023-08-27T11:13:12.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.498+0000 7fdcd7fff700 1 -- 172.21.15.45:0/185020026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80050d0 con 0x7fdce015a0f0 2023-08-27T11:13:12.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.498+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce00b2b60 con 0x7fdce015a0f0 2023-08-27T11:13:12.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.499+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce00cf850 con 0x7fdce015a0f0 2023-08-27T11:13:12.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.499+0000 7fdcd7fff700 1 -- 172.21.15.45:0/185020026 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdcd8005230 con 0x7fdce015a0f0 2023-08-27T11:13:12.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.500+0000 7fdcd7fff700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdcc006a0e0 0x7fdcc006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.500+0000 7fdcd7fff700 1 -- 172.21.15.45:0/185020026 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdcd808afb0 con 0x7fdce015a0f0 2023-08-27T11:13:12.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.500+0000 7fdcdf7fe700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdcc006a0e0 0x7fdcc006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.500+0000 7fdcdf7fe700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdcc006a0e0 0x7fdcc006c5a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8000f60 tx=0x7fdcc8004a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:12.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.504+0000 7fdcd7fff700 1 -- 172.21.15.45:0/185020026 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdcd80bd990 con 0x7fdce015a0f0 2023-08-27T11:13:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.705+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdce00726d0 con 0x7fdce015a0f0 2023-08-27T11:13:12.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.705+0000 7fdcd7fff700 1 -- 172.21.15.45:0/185020026 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdcd80562f0 con 0x7fdce015a0f0 2023-08-27T11:13:12.706 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:13:12.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdcc006a0e0 msgr2=0x7fdcc006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdcc006a0e0 0x7fdcc006c5a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8000f60 tx=0x7fdcc8004a50 comp rx=0 tx=0).stop 2023-08-27T11:13:12.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce015a0f0 msgr2=0x7fdce01eeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce015a0f0 0x7fdce01eeec0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80021c0 tx=0x7fdcd80021f0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 shutdown_connections 2023-08-27T11:13:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce015a0f0 0x7fdce01eeec0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdcc006a0e0 0x7fdcc006c5a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdce006ed90 0x7fdce00c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 --2- 172.21.15.45:0/185020026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdce00cd300 0x7fdce00c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.707+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 >> 172.21.15.45:0/185020026 conn(0x7fdce00cc7e0 msgr2=0x7fdce00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.708+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 shutdown_connections 2023-08-27T11:13:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.708+0000 7fdce5ca4700 1 -- 172.21.15.45:0/185020026 wait complete. 2023-08-27T11:13:12.721 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:13:12.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.849+0000 7fe3ae21b700 1 Processor -- start 2023-08-27T11:13:12.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.849+0000 7fe3ae21b700 1 -- start start 2023-08-27T11:13:12.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3ae21b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a80aed20 0x7fe3a80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3ae21b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80af270 con 0x7fe3a80ca3b0 2023-08-27T11:13:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3ae21b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80af550 con 0x7fe3a815d600 2023-08-27T11:13:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3ae21b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a806ac90 con 0x7fe3a80aed20 2023-08-27T11:13:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a77fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe3a80ca3b0 0x7fe3a80ca790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40088/0 (socket says 172.21.15.45:40088) 2023-08-27T11:13:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a77fe700 1 -- 172.21.15.45:0/1422086476 learned_addr learned my addr 172.21.15.45:0/1422086476 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a7fff700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a80aed20 0x7fe3a80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554617257 0 0) 0x7fe3a80af270 con 0x7fe3a80ca3b0 2023-08-27T11:13:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3940035b0 con 0x7fe3a80ca3b0 2023-08-27T11:13:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186156567 0 0) 0x7fe3a80af550 con 0x7fe3a815d600 2023-08-27T11:13:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3a80af270 con 0x7fe3a815d600 2023-08-27T11:13:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1500660297 0 0) 0x7fe3940035b0 con 0x7fe3a80ca3b0 2023-08-27T11:13:12.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a80aed20 msgr2=0x7fe3a80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:12.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a80aed20 0x7fe3a80ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 >> v1:172.21.15.135:6789/0 conn(0x7fe3a815d600 legacy=0x7fe3a80cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a806af70 con 0x7fe3a80ca3b0 2023-08-27T11:13:12.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fe398001670 con 0x7fe3a80ca3b0 2023-08-27T11:13:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a7fff700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a80aed20 0x7fe3a80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.850+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 >> v1:172.21.15.45:6789/0 conn(0x7fe3a80ca3b0 legacy=0x7fe3a80ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a67fc700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe394003bc0 0x7fe394003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a67fc700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a80aed20 0x7fe394004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a67fc700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe394004d90 0x7fe394005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a806ac90 con 0x7fe394004d90 2023-08-27T11:13:12.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39400bac0 con 0x7fe394003bc0 2023-08-27T11:13:12.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39400bc30 con 0x7fe3a80aed20 2023-08-27T11:13:12.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a7fff700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe394004d90 0x7fe394005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a77fe700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe394003bc0 0x7fe394003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a6ffd700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a80aed20 0x7fe394004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a7fff700 1 -- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a80aed20 msgr2=0x7fe394004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a7fff700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a80aed20 0x7fe394004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a7fff700 1 -- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe394003bc0 msgr2=0x7fe394003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a7fff700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe394003bc0 0x7fe394003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a7fff700 1 -- 172.21.15.45:0/1422086476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0009a70 con 0x7fe394004d90 2023-08-27T11:13:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a7fff700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe394004d90 0x7fe394005210 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000e350 tx=0x7fe394005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fe695f7777b0dc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000ed90 con 0x7fe394004d90 2023-08-27T11:13:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a000f040 con 0x7fe394004d90 2023-08-27T11:13:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.851+0000 7fe3a67fc700 1 -- 172.21.15.45:0/1422086476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a00146a0 con 0x7fe394004d90 2023-08-27T11:13:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.852+0000 7fe3ae21b700 1 -- 172.21.15.45:0/1422086476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe394004d90 msgr2=0x7fe394005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.852+0000 7fe3ae21b700 1 --2- 172.21.15.45:0/1422086476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe394004d90 0x7fe394005210 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000e350 tx=0x7fe394005880 comp rx=0 tx=0).stop 2023-08-27T11:13:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.852+0000 7fe3a7fff700 1 -- 172.21.15.45:0/1422086476 reap_dead start 2023-08-27T11:13:12.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.852+0000 7fe3ae21b700 1 -- 172.21.15.45:0/1422086476 shutdown_connections 2023-08-27T11:13:12.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.852+0000 7fe3ae21b700 1 -- 172.21.15.45:0/1422086476 >> 172.21.15.45:0/1422086476 conn(0x7fe3a80cc7e0 msgr2=0x7fe3a80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:12.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.852+0000 7fe3ae21b700 1 -- 172.21.15.45:0/1422086476 shutdown_connections 2023-08-27T11:13:12.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.852+0000 7fe3ae21b700 1 -- 172.21.15.45:0/1422086476 wait complete. 2023-08-27T11:13:12.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.853+0000 7fe3ae21b700 1 Processor -- start 2023-08-27T11:13:12.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.853+0000 7fe3ae21b700 1 -- start start 2023-08-27T11:13:12.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3ae21b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a814c2a0 0x7fe3a80c3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3ae21b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a814c680 0x7fe3a80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3ae21b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a80c4f00 0x7fe3a80c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3ae21b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80c4ae0 con 0x7fe3a80c4f00 2023-08-27T11:13:12.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3ae21b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80b3a60 con 0x7fe3a814c2a0 2023-08-27T11:13:12.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3ae21b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80c3e60 con 0x7fe3a814c680 2023-08-27T11:13:12.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a80c4f00 0x7fe3a80c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3a7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a80c4f00 0x7fe3a80c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53328/0 (socket says 172.21.15.45:53328) 2023-08-27T11:13:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3a7fff700 1 -- 172.21.15.45:0/2858571682 learned_addr learned my addr 172.21.15.45:0/2858571682 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3a77fe700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a814c2a0 0x7fe3a80c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3a6ffd700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a814c680 0x7fe3a80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.854+0000 7fe3a7fff700 1 -- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a814c680 msgr2=0x7fe3a80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe3a7fff700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a814c680 0x7fe3a80b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe3a7fff700 1 -- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a814c2a0 msgr2=0x7fe3a80c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe3a7fff700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a814c2a0 0x7fe3a80c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe3a7fff700 1 -- 172.21.15.45:0/2858571682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a80c4140 con 0x7fe3a80c4f00 2023-08-27T11:13:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe3a6ffd700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a814c680 0x7fe3a80b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:12.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe3a7fff700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a80c4f00 0x7fe3a80c5ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0016000 tx=0x7fe3a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe39ffff700 1 -- 172.21.15.45:0/2858571682 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0007000 con 0x7fe3a80c4f00 2023-08-27T11:13:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe39ffff700 1 -- 172.21.15.45:0/2858571682 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0009410 con 0x7fe3a80c4f00 2023-08-27T11:13:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a80c4420 con 0x7fe3a80c4f00 2023-08-27T11:13:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe39ffff700 1 -- 172.21.15.45:0/2858571682 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0027480 con 0x7fe3a80c4f00 2023-08-27T11:13:12.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.855+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a80c6010 con 0x7fe3a80c4f00 2023-08-27T11:13:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.856+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a80c4c20 con 0x7fe3a80c4f00 2023-08-27T11:13:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.856+0000 7fe39ffff700 1 -- 172.21.15.45:0/2858571682 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe3a001d420 con 0x7fe3a80c4f00 2023-08-27T11:13:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.857+0000 7fe39ffff700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe38806a0e0 0x7fe38806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.857+0000 7fe39ffff700 1 -- 172.21.15.45:0/2858571682 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe3a008a2a0 con 0x7fe3a80c4f00 2023-08-27T11:13:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.857+0000 7fe3a77fe700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe38806a0e0 0x7fe38806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:12.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.857+0000 7fe3a77fe700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe38806a0e0 0x7fe38806c5a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe398000c00 tx=0x7fe398001e20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:12.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:12.863+0000 7fe39ffff700 1 -- 172.21.15.45:0/2858571682 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe3a80c4c20 con 0x7fe3a80c4f00 2023-08-27T11:13:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.066+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3a80c4c20 con 0x7fe3a80c4f00 2023-08-27T11:13:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.066+0000 7fe39ffff700 1 -- 172.21.15.45:0/2858571682 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3a80c4c20 con 0x7fe3a80c4f00 2023-08-27T11:13:13.068 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe38806a0e0 msgr2=0x7fe38806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe38806a0e0 0x7fe38806c5a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe398000c00 tx=0x7fe398001e20 comp rx=0 tx=0).stop 2023-08-27T11:13:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a80c4f00 msgr2=0x7fe3a80c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a80c4f00 0x7fe3a80c5ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0016000 tx=0x7fe3a0009750 comp rx=0 tx=0).stop 2023-08-27T11:13:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 shutdown_connections 2023-08-27T11:13:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a80c4f00 0x7fe3a80c5ad0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe38806a0e0 0x7fe38806c5a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a814c2a0 0x7fe3a80c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 --2- 172.21.15.45:0/2858571682 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe3a814c680 0x7fe3a80b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 >> 172.21.15.45:0/2858571682 conn(0x7fe3a80cc7e0 msgr2=0x7fe3a80b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 shutdown_connections 2023-08-27T11:13:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.069+0000 7fe3ae21b700 1 -- 172.21.15.45:0/2858571682 wait complete. 2023-08-27T11:13:13.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:13:13.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.210+0000 7fce5c105700 1 Processor -- start 2023-08-27T11:13:13.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.210+0000 7fce5c105700 1 -- start start 2023-08-27T11:13:13.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.211+0000 7fce5c105700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce540b17c0 0x7fce540b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.211+0000 7fce5c105700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5415b460 con 0x7fce5406ced0 2023-08-27T11:13:13.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.211+0000 7fce5c105700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce540b2280 con 0x7fce54157be0 2023-08-27T11:13:13.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.211+0000 7fce5c105700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce54157fc0 con 0x7fce540b17c0 2023-08-27T11:13:13.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.211+0000 7fce596a0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce540b17c0 0x7fce540b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.211+0000 7fce5a6a2700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fce5406ced0 0x7fce540cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40112/0 (socket says 172.21.15.45:40112) 2023-08-27T11:13:13.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.211+0000 7fce596a0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce540b17c0 0x7fce540b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53334/0 (socket says 172.21.15.45:53334) 2023-08-27T11:13:13.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.211+0000 7fce5a6a2700 1 -- 172.21.15.45:0/2077537257 learned_addr learned my addr 172.21.15.45:0/2077537257 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:13.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.211+0000 7fce596a0700 1 -- 172.21.15.45:0/2077537257 >> v1:172.21.15.135:6789/0 conn(0x7fce54157be0 legacy=0x7fce540716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:13.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce596a0700 1 -- 172.21.15.45:0/2077537257 >> v1:172.21.15.45:6789/0 conn(0x7fce5406ced0 legacy=0x7fce540cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce58e9f700 1 -- 172.21.15.45:0/2077537257 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120234114 0 0) 0x7fce5415b460 con 0x7fce5406ced0 2023-08-27T11:13:13.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce596a0700 1 -- 172.21.15.45:0/2077537257 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540b2560 con 0x7fce540b17c0 2023-08-27T11:13:13.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce596a0700 1 --2- 172.21.15.45:0/2077537257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce540b17c0 0x7fce540b1c70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fce50009840 tx=0x7fce50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddeb14a571c3594 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:13.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce58e9f700 1 -- 172.21.15.45:0/2077537257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50004230 con 0x7fce540b17c0 2023-08-27T11:13:13.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce58e9f700 1 -- 172.21.15.45:0/2077537257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50004390 con 0x7fce540b17c0 2023-08-27T11:13:13.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce58e9f700 1 -- 172.21.15.45:0/2077537257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500036a0 con 0x7fce540b17c0 2023-08-27T11:13:13.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce5c105700 1 -- 172.21.15.45:0/2077537257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce540b17c0 msgr2=0x7fce540b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce5c105700 1 --2- 172.21.15.45:0/2077537257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce540b17c0 0x7fce540b1c70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fce50009840 tx=0x7fce50009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:13.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce5c105700 1 -- 172.21.15.45:0/2077537257 shutdown_connections 2023-08-27T11:13:13.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce5c105700 1 --2- 172.21.15.45:0/2077537257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce540b17c0 0x7fce540b1c70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.212+0000 7fce5c105700 1 -- 172.21.15.45:0/2077537257 >> 172.21.15.45:0/2077537257 conn(0x7fce540cc7c0 msgr2=0x7fce540ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:13.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.213+0000 7fce5c105700 1 -- 172.21.15.45:0/2077537257 shutdown_connections 2023-08-27T11:13:13.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.213+0000 7fce5c105700 1 -- 172.21.15.45:0/2077537257 wait complete. 2023-08-27T11:13:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.214+0000 7fce5c105700 1 Processor -- start 2023-08-27T11:13:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.214+0000 7fce5c105700 1 -- start start 2023-08-27T11:13:13.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce5c105700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce5406ced0 0x7fce541544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce5c105700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fce540b17c0 0x7fce540ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce5c105700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce54157be0 0x7fce540b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce5c105700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541f17e0 con 0x7fce54157be0 2023-08-27T11:13:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce5c105700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540ce590 con 0x7fce5406ced0 2023-08-27T11:13:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce5c105700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce541f0c60 con 0x7fce540b17c0 2023-08-27T11:13:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce5a6a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce54157be0 0x7fce540b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce5a6a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce54157be0 0x7fce540b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53342/0 (socket says 172.21.15.45:53342) 2023-08-27T11:13:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce5a6a2700 1 -- 172.21.15.45:0/2535865914 learned_addr learned my addr 172.21.15.45:0/2535865914 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce59ea1700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce5406ced0 0x7fce541544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.215+0000 7fce596a0700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fce540b17c0 0x7fce540ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce5a6a2700 1 -- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fce540b17c0 msgr2=0x7fce540ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce5a6a2700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fce540b17c0 0x7fce540ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce5a6a2700 1 -- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce5406ced0 msgr2=0x7fce541544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce5a6a2700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce5406ced0 0x7fce541544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce5a6a2700 1 -- 172.21.15.45:0/2535865914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce500094a0 con 0x7fce54157be0 2023-08-27T11:13:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce5a6a2700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce54157be0 0x7fce540b4da0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00cb60 tx=0x7fce4c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce4a7fc700 1 -- 172.21.15.45:0/2535865914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00a9e0 con 0x7fce54157be0 2023-08-27T11:13:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce4a7fc700 1 -- 172.21.15.45:0/2535865914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c00ab40 con 0x7fce54157be0 2023-08-27T11:13:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce541f0f40 con 0x7fce54157be0 2023-08-27T11:13:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce4a7fc700 1 -- 172.21.15.45:0/2535865914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0070d0 con 0x7fce54157be0 2023-08-27T11:13:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.216+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce541f1500 con 0x7fce54157be0 2023-08-27T11:13:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.217+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce54072690 con 0x7fce54157be0 2023-08-27T11:13:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.217+0000 7fce4a7fc700 1 -- 172.21.15.45:0/2535865914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fce4c007230 con 0x7fce54157be0 2023-08-27T11:13:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.218+0000 7fce4a7fc700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fce3806a0e0 0x7fce3806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.218+0000 7fce4a7fc700 1 -- 172.21.15.45:0/2535865914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fce4c0884d0 con 0x7fce54157be0 2023-08-27T11:13:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.218+0000 7fce59ea1700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fce3806a0e0 0x7fce3806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.218+0000 7fce59ea1700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fce3806a0e0 0x7fce3806c5a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce44007fd0 tx=0x7fce44007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:13.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.222+0000 7fce4a7fc700 1 -- 172.21.15.45:0/2535865914 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fce4c053920 con 0x7fce54157be0 2023-08-27T11:13:13.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.424+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce540734a0 con 0x7fce54157be0 2023-08-27T11:13:13.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.424+0000 7fce4a7fc700 1 -- 172.21.15.45:0/2535865914 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce4c056f40 con 0x7fce54157be0 2023-08-27T11:13:13.426 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:13:13.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fce3806a0e0 msgr2=0x7fce3806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fce3806a0e0 0x7fce3806c5a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce44007fd0 tx=0x7fce44007ec0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce54157be0 msgr2=0x7fce540b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce54157be0 0x7fce540b4da0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00cb60 tx=0x7fce4c00ce70 comp rx=0 tx=0).stop 2023-08-27T11:13:13.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 shutdown_connections 2023-08-27T11:13:13.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce54157be0 0x7fce540b4da0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fce3806a0e0 0x7fce3806c5a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce5406ced0 0x7fce541544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 --2- 172.21.15.45:0/2535865914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fce540b17c0 0x7fce540ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 >> 172.21.15.45:0/2535865914 conn(0x7fce540cc7c0 msgr2=0x7fce54158c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:13.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 shutdown_connections 2023-08-27T11:13:13.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.426+0000 7fce5c105700 1 -- 172.21.15.45:0/2535865914 wait complete. 2023-08-27T11:13:13.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:13:13.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.567+0000 7f3bd5bbc700 1 Processor -- start 2023-08-27T11:13:13.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.567+0000 7f3bd5bbc700 1 -- start start 2023-08-27T11:13:13.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.568+0000 7f3bd5bbc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3bd0160d00 0x7f3bd006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.568+0000 7f3bd5bbc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 0x7f3bd00b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.568+0000 7f3bd5bbc700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd006b160 con 0x7f3bd015d340 2023-08-27T11:13:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.568+0000 7f3bd5bbc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd0072010 con 0x7f3bd0160d00 2023-08-27T11:13:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.568+0000 7f3bd5bbc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd006ac50 con 0x7f3bd006a7c0 2023-08-27T11:13:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.568+0000 7f3bceffd700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3bd0160d00 0x7f3bd006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.568+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 0x7f3bd00b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.568+0000 7f3bceffd700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3bd0160d00 0x7f3bd006a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:42204/0 (socket says 172.21.15.45:42204) 2023-08-27T11:13:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.568+0000 7f3bceffd700 1 -- 172.21.15.45:0/947874928 learned_addr learned my addr 172.21.15.45:0/947874928 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.569+0000 7f3bceffd700 1 -- 172.21.15.45:0/947874928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 msgr2=0x7f3bd00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.569+0000 7f3bceffd700 1 --2- 172.21.15.45:0/947874928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 0x7f3bd00b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.569+0000 7f3bceffd700 1 -- 172.21.15.45:0/947874928 >> v1:172.21.15.135:6789/0 conn(0x7f3bd015d340 legacy=0x7f3bd00cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.569+0000 7f3bceffd700 1 -- 172.21.15.45:0/947874928 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00cdf90 con 0x7f3bd0160d00 2023-08-27T11:13:13.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.569+0000 7f3bcffff700 1 --2- 172.21.15.45:0/947874928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 0x7f3bd00b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:13.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.569+0000 7f3bceffd700 1 --2- 172.21.15.45:0/947874928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd0160d00 0x7f3bd006a280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80098f0 tx=0x7f3bb8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9871e230d7acdd14 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:13.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.569+0000 7f3bce7fc700 1 -- 172.21.15.45:0/947874928 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8011830 con 0x7f3bd0160d00 2023-08-27T11:13:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.569+0000 7f3bce7fc700 1 -- 172.21.15.45:0/947874928 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8011990 con 0x7f3bd0160d00 2023-08-27T11:13:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.569+0000 7f3bce7fc700 1 -- 172.21.15.45:0/947874928 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8011d20 con 0x7f3bd0160d00 2023-08-27T11:13:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.570+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/947874928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd0160d00 msgr2=0x7f3bd006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.570+0000 7f3bd5bbc700 1 --2- 172.21.15.45:0/947874928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd0160d00 0x7f3bd006a280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80098f0 tx=0x7f3bb8009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.570+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/947874928 shutdown_connections 2023-08-27T11:13:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.570+0000 7f3bd5bbc700 1 --2- 172.21.15.45:0/947874928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 0x7f3bd00b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.570+0000 7f3bd5bbc700 1 --2- 172.21.15.45:0/947874928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd0160d00 0x7f3bd006a280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.570+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/947874928 >> 172.21.15.45:0/947874928 conn(0x7f3bd00cc7c0 msgr2=0x7f3bd00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.570+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/947874928 shutdown_connections 2023-08-27T11:13:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.570+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/947874928 wait complete. 2023-08-27T11:13:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.571+0000 7f3bd5bbc700 1 Processor -- start 2023-08-27T11:13:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.571+0000 7f3bd5bbc700 1 -- start start 2023-08-27T11:13:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bd5bbc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 0x7f3bd014b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bd5bbc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd015d340 0x7f3bd00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bd5bbc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd0160d00 0x7f3bd00b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bd5bbc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01e6b40 con 0x7f3bd0160d00 2023-08-27T11:13:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bd5bbc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd014b6d0 con 0x7f3bd015d340 2023-08-27T11:13:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bd5bbc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01e6d90 con 0x7f3bd006a7c0 2023-08-27T11:13:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd0160d00 0x7f3bd00b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd0160d00 0x7f3bd00b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53356/0 (socket says 172.21.15.45:53356) 2023-08-27T11:13:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcffff700 1 -- 172.21.15.45:0/1800454575 learned_addr learned my addr 172.21.15.45:0/1800454575 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bceffd700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd015d340 0x7f3bd00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcf7fe700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 0x7f3bd014b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcffff700 1 -- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 msgr2=0x7f3bd014b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcffff700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 0x7f3bd014b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcffff700 1 -- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd015d340 msgr2=0x7f3bd00b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcffff700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd015d340 0x7f3bd00b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcffff700 1 -- 172.21.15.45:0/1800454575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009a70 con 0x7f3bd0160d00 2023-08-27T11:13:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bceffd700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd015d340 0x7f3bd00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bcffff700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd0160d00 0x7f3bd00b0c00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800efc0 tx=0x7f3bc800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.572+0000 7f3bc7fff700 1 -- 172.21.15.45:0/1800454575 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8009cd0 con 0x7f3bd0160d00 2023-08-27T11:13:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.573+0000 7f3bc7fff700 1 -- 172.21.15.45:0/1800454575 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc800f040 con 0x7f3bd0160d00 2023-08-27T11:13:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.573+0000 7f3bc7fff700 1 -- 172.21.15.45:0/1800454575 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800cd60 con 0x7f3bd0160d00 2023-08-27T11:13:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.573+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb8009550 con 0x7f3bd0160d00 2023-08-27T11:13:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.573+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd01e7350 con 0x7f3bd0160d00 2023-08-27T11:13:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.573+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd0072690 con 0x7f3bd0160d00 2023-08-27T11:13:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.574+0000 7f3bc7fff700 1 -- 172.21.15.45:0/1800454575 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3bc8014480 con 0x7f3bd0160d00 2023-08-27T11:13:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.574+0000 7f3bc7fff700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3bb006a0e0 0x7f3bb006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.574+0000 7f3bc7fff700 1 -- 172.21.15.45:0/1800454575 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3bc8088f00 con 0x7f3bd0160d00 2023-08-27T11:13:13.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.575+0000 7f3bcf7fe700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3bb006a0e0 0x7f3bb006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.575+0000 7f3bcf7fe700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3bb006a0e0 0x7f3bb006c5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0000f60 tx=0x7f3bc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:13.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.579+0000 7f3bc7fff700 1 -- 172.21.15.45:0/1800454575 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3bc8054350 con 0x7f3bd0160d00 2023-08-27T11:13:13.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.779+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3bd00734a0 con 0x7f3bd0160d00 2023-08-27T11:13:13.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.779+0000 7f3bc7fff700 1 -- 172.21.15.45:0/1800454575 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3bc8057970 con 0x7f3bd0160d00 2023-08-27T11:13:13.780 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3bb006a0e0 msgr2=0x7f3bb006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3bb006a0e0 0x7f3bb006c5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0000f60 tx=0x7f3bc0009040 comp rx=0 tx=0).stop 2023-08-27T11:13:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd0160d00 msgr2=0x7f3bd00b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd0160d00 0x7f3bd00b0c00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800efc0 tx=0x7f3bc800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 shutdown_connections 2023-08-27T11:13:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd0160d00 0x7f3bd00b0c00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3bb006a0e0 0x7f3bb006c5a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bd006a7c0 0x7f3bd014b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 --2- 172.21.15.45:0/1800454575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bd015d340 0x7f3bd00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 >> 172.21.15.45:0/1800454575 conn(0x7f3bd00cc7c0 msgr2=0x7f3bd00afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 shutdown_connections 2023-08-27T11:13:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.781+0000 7f3bd5bbc700 1 -- 172.21.15.45:0/1800454575 wait complete. 2023-08-27T11:13:13.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:13:13.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.919+0000 7f9d3a429700 1 Processor -- start 2023-08-27T11:13:13.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.919+0000 7f9d3a429700 1 -- start start 2023-08-27T11:13:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.920+0000 7f9d3a429700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9d340af200 0x7f9d340cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.920+0000 7f9d3a429700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9d340caa00 0x7f9d340acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.920+0000 7f9d3a429700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d340ce340 con 0x7f9d340ca620 2023-08-27T11:13:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.920+0000 7f9d3a429700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d340ceec0 con 0x7f9d340af200 2023-08-27T11:13:13.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.920+0000 7f9d3a429700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3415d7c0 con 0x7f9d340caa00 2023-08-27T11:13:13.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.920+0000 7f9d337fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9d340af200 0x7f9d340cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.920+0000 7f9d33fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9d340ca620 0x7f9d340ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40136/0 (socket says 172.21.15.45:40136) 2023-08-27T11:13:13.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.920+0000 7f9d33fff700 1 -- 172.21.15.45:0/3535168624 learned_addr learned my addr 172.21.15.45:0/3535168624 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:13.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d389c6700 1 --2- 172.21.15.45:0/3535168624 >> v2:172.21.15.135:3300/0 conn(0x7f9d340caa00 0x7f9d340acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d337fe700 1 -- 172.21.15.45:0/3535168624 >> v2:172.21.15.135:3300/0 conn(0x7f9d340caa00 msgr2=0x7f9d340acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:13.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d337fe700 1 --2- 172.21.15.45:0/3535168624 >> v2:172.21.15.135:3300/0 conn(0x7f9d340caa00 0x7f9d340acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d337fe700 1 -- 172.21.15.45:0/3535168624 >> v1:172.21.15.45:6789/0 conn(0x7f9d340ca620 legacy=0x7f9d340ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d32ffd700 1 -- 172.21.15.45:0/3535168624 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807174036 0 0) 0x7f9d340ce340 con 0x7f9d340ca620 2023-08-27T11:13:13.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d337fe700 1 -- 172.21.15.45:0/3535168624 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d340cc570 con 0x7f9d340af200 2023-08-27T11:13:13.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d389c6700 1 --2- 172.21.15.45:0/3535168624 >> v2:172.21.15.135:3300/0 conn(0x7f9d340caa00 0x7f9d340acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:13.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d337fe700 1 --2- 172.21.15.45:0/3535168624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340af200 0x7f9d340cf350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9d3406ccd0 tx=0x7f9d28009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54681e28ad88ce62 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:13.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d32ffd700 1 -- 172.21.15.45:0/3535168624 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28011830 con 0x7f9d340af200 2023-08-27T11:13:13.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d32ffd700 1 -- 172.21.15.45:0/3535168624 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d28011990 con 0x7f9d340af200 2023-08-27T11:13:13.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.921+0000 7f9d32ffd700 1 -- 172.21.15.45:0/3535168624 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28011d20 con 0x7f9d340af200 2023-08-27T11:13:13.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.922+0000 7f9d3a429700 1 -- 172.21.15.45:0/3535168624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340af200 msgr2=0x7f9d340cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.922+0000 7f9d3a429700 1 --2- 172.21.15.45:0/3535168624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340af200 0x7f9d340cf350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9d3406ccd0 tx=0x7f9d28009b20 comp rx=0 tx=0).stop 2023-08-27T11:13:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.922+0000 7f9d3a429700 1 -- 172.21.15.45:0/3535168624 shutdown_connections 2023-08-27T11:13:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.922+0000 7f9d3a429700 1 --2- 172.21.15.45:0/3535168624 >> v2:172.21.15.135:3300/0 conn(0x7f9d340caa00 0x7f9d340acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.922+0000 7f9d3a429700 1 --2- 172.21.15.45:0/3535168624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340af200 0x7f9d340cf350 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.922+0000 7f9d3a429700 1 -- 172.21.15.45:0/3535168624 >> 172.21.15.45:0/3535168624 conn(0x7f9d340cc7e0 msgr2=0x7f9d340ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:13.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.922+0000 7f9d3a429700 1 -- 172.21.15.45:0/3535168624 shutdown_connections 2023-08-27T11:13:13.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.922+0000 7f9d3a429700 1 -- 172.21.15.45:0/3535168624 wait complete. 2023-08-27T11:13:13.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.923+0000 7f9d3a429700 1 Processor -- start 2023-08-27T11:13:13.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.923+0000 7f9d3a429700 1 -- start start 2023-08-27T11:13:13.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d3a429700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d340af200 0x7f9d340c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d3a429700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340ca620 0x7f9d340c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d3a429700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d340caa00 0x7f9d3406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d3a429700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340c5fa0 con 0x7f9d340ca620 2023-08-27T11:13:13.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d3a429700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340c4a60 con 0x7f9d340caa00 2023-08-27T11:13:13.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d3a429700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340c5320 con 0x7f9d340af200 2023-08-27T11:13:13.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340ca620 0x7f9d340c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340ca620 0x7f9d340c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53368/0 (socket says 172.21.15.45:53368) 2023-08-27T11:13:13.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d337fe700 1 -- 172.21.15.45:0/3902434006 learned_addr learned my addr 172.21.15.45:0/3902434006 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:13.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d33fff700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d340af200 0x7f9d340c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d389c6700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d340caa00 0x7f9d3406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d337fe700 1 -- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d340af200 msgr2=0x7f9d340c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d337fe700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d340af200 0x7f9d340c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d337fe700 1 -- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d340caa00 msgr2=0x7f9d3406e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:13.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d337fe700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d340caa00 0x7f9d3406e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:13.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d337fe700 1 -- 172.21.15.45:0/3902434006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d280094a0 con 0x7f9d340ca620 2023-08-27T11:13:13.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d33fff700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d340af200 0x7f9d340c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:13.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.924+0000 7f9d337fe700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340ca620 0x7f9d340c4c40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d28000c00 tx=0x7f9d28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:13.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.925+0000 7f9d30ff9700 1 -- 172.21.15.45:0/3902434006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280036a0 con 0x7f9d340ca620 2023-08-27T11:13:13.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.925+0000 7f9d30ff9700 1 -- 172.21.15.45:0/3902434006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2801f070 con 0x7f9d340ca620 2023-08-27T11:13:13.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.925+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d340c5600 con 0x7f9d340ca620 2023-08-27T11:13:13.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.925+0000 7f9d30ff9700 1 -- 172.21.15.45:0/3902434006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28003a60 con 0x7f9d340ca620 2023-08-27T11:13:13.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.925+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d340c5bc0 con 0x7f9d340ca620 2023-08-27T11:13:13.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.926+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d340c60e0 con 0x7f9d340ca620 2023-08-27T11:13:13.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.926+0000 7f9d30ff9700 1 -- 172.21.15.45:0/3902434006 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9d28003bc0 con 0x7f9d340ca620 2023-08-27T11:13:13.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.926+0000 7f9d30ff9700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d2006a0e0 0x7f9d2006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:13.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.927+0000 7f9d30ff9700 1 -- 172.21.15.45:0/3902434006 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9d28021070 con 0x7f9d340ca620 2023-08-27T11:13:13.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.927+0000 7f9d33fff700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d2006a0e0 0x7f9d2006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:13.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.927+0000 7f9d33fff700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d2006a0e0 0x7f9d2006c5a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9d24000f60 tx=0x7f9d24007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:13.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:13.931+0000 7f9d30ff9700 1 -- 172.21.15.45:0/3902434006 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9d340c60e0 con 0x7f9d340ca620 2023-08-27T11:13:14.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.146+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d340c60e0 con 0x7f9d340ca620 2023-08-27T11:13:14.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.147+0000 7f9d30ff9700 1 -- 172.21.15.45:0/3902434006 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9d340c60e0 con 0x7f9d340ca620 2023-08-27T11:13:14.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d2006a0e0 msgr2=0x7f9d2006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d2006a0e0 0x7f9d2006c5a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9d24000f60 tx=0x7f9d24007ca0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340ca620 msgr2=0x7f9d340c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340ca620 0x7f9d340c4c40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9d28000c00 tx=0x7f9d28004750 comp rx=0 tx=0).stop 2023-08-27T11:13:14.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 shutdown_connections 2023-08-27T11:13:14.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d2006a0e0 0x7f9d2006c5a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d340caa00 0x7f9d3406e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d340af200 0x7f9d340c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 --2- 172.21.15.45:0/3902434006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d340ca620 0x7f9d340c4c40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 >> 172.21.15.45:0/3902434006 conn(0x7f9d340cc7e0 msgr2=0x7f9d340cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:14.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 shutdown_connections 2023-08-27T11:13:14.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.149+0000 7f9d3a429700 1 -- 172.21.15.45:0/3902434006 wait complete. 2023-08-27T11:13:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2023-08-27T11:13:14.162 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-27T11:13:14.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:14.163 DEBUG:teuthology.orchestra.run.smithi135:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-08-27T11:13:14.223 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2023-08-27T11:13:14.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:14.224 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-27T11:13:14.265 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:13:14.265 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:14.283 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:13:14.284 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:14.341 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:13:14.342 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:14.399 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:14.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.563+0000 7fc9fc7b3700 1 Processor -- start 2023-08-27T11:13:14.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.563+0000 7fc9fc7b3700 1 -- start start 2023-08-27T11:13:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.564+0000 7fc9fc7b3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9f406cec0 0x7fc9f406e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.564+0000 7fc9fc7b3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406e7b0 0x7fc9f40ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.564+0000 7fc9fc7b3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f415dad0 con 0x7fc9f415b070 2023-08-27T11:13:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.564+0000 7fc9fc7b3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f415d8e0 con 0x7fc9f406cec0 2023-08-27T11:13:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.564+0000 7fc9fc7b3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f406ec20 con 0x7fc9f406e7b0 2023-08-27T11:13:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f9d4e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9f406cec0 0x7fc9f406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9fa54f700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc9f415b070 0x7fc9f40cda20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40150/0 (socket says 172.21.15.45:40150) 2023-08-27T11:13:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f9d4e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9f406cec0 0x7fc9f406e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53376/0 (socket says 172.21.15.45:53376) 2023-08-27T11:13:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9fa54f700 1 -- 172.21.15.45:0/778977704 learned_addr learned my addr 172.21.15.45:0/778977704 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9fad50700 1 --2- 172.21.15.45:0/778977704 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406e7b0 0x7fc9f40ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f9d4e700 1 -- 172.21.15.45:0/778977704 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406e7b0 msgr2=0x7fc9f40ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f9d4e700 1 --2- 172.21.15.45:0/778977704 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406e7b0 0x7fc9f40ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f9d4e700 1 -- 172.21.15.45:0/778977704 >> v1:172.21.15.45:6789/0 conn(0x7fc9f415b070 legacy=0x7fc9f40cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f9d4e700 1 -- 172.21.15.45:0/778977704 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f415ddb0 con 0x7fc9f406cec0 2023-08-27T11:13:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f954d700 1 -- 172.21.15.45:0/778977704 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274312278 0 0) 0x7fc9f415dad0 con 0x7fc9f415b070 2023-08-27T11:13:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f9d4e700 1 --2- 172.21.15.45:0/778977704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f406cec0 0x7fc9f406e270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009870 tx=0x7fc9f0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b571971595e133 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f954d700 1 -- 172.21.15.45:0/778977704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0011830 con 0x7fc9f406cec0 2023-08-27T11:13:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f954d700 1 -- 172.21.15.45:0/778977704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f0011990 con 0x7fc9f406cec0 2023-08-27T11:13:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.565+0000 7fc9f954d700 1 -- 172.21.15.45:0/778977704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0011d20 con 0x7fc9f406cec0 2023-08-27T11:13:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.566+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/778977704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f406cec0 msgr2=0x7fc9f406e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.566+0000 7fc9fc7b3700 1 --2- 172.21.15.45:0/778977704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f406cec0 0x7fc9f406e270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009870 tx=0x7fc9f0009b80 comp rx=0 tx=0).stop 2023-08-27T11:13:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.566+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/778977704 shutdown_connections 2023-08-27T11:13:14.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.566+0000 7fc9fc7b3700 1 --2- 172.21.15.45:0/778977704 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406e7b0 0x7fc9f40ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.566+0000 7fc9fc7b3700 1 --2- 172.21.15.45:0/778977704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f406cec0 0x7fc9f406e270 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.566+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/778977704 >> 172.21.15.45:0/778977704 conn(0x7fc9f40cfe50 msgr2=0x7fc9f40d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.566+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/778977704 shutdown_connections 2023-08-27T11:13:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.566+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/778977704 wait complete. 2023-08-27T11:13:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.567+0000 7fc9fc7b3700 1 Processor -- start 2023-08-27T11:13:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.567+0000 7fc9fc7b3700 1 -- start start 2023-08-27T11:13:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fc7b3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406cec0 0x7fc9f41f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fc7b3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9f406e7b0 0x7fc9f41f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fc7b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f415b070 0x7fc9f40c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fc7b3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f40c7490 con 0x7fc9f415b070 2023-08-27T11:13:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fc7b3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f40c7770 con 0x7fc9f406e7b0 2023-08-27T11:13:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fc7b3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f40c7a50 con 0x7fc9f406cec0 2023-08-27T11:13:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fad50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f415b070 0x7fc9f40c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fad50700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f415b070 0x7fc9f40c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53384/0 (socket says 172.21.15.45:53384) 2023-08-27T11:13:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fad50700 1 -- 172.21.15.45:0/3178351586 learned_addr learned my addr 172.21.15.45:0/3178351586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fa54f700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406cec0 0x7fc9f41f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9f9d4e700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9f406e7b0 0x7fc9f41f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fad50700 1 -- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406cec0 msgr2=0x7fc9f41f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fad50700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406cec0 0x7fc9f41f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fad50700 1 -- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9f406e7b0 msgr2=0x7fc9f41f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fad50700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9f406e7b0 0x7fc9f41f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9fad50700 1 -- 172.21.15.45:0/3178351586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f00094a0 con 0x7fc9f415b070 2023-08-27T11:13:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.568+0000 7fc9f9d4e700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9f406e7b0 0x7fc9f41f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.569+0000 7fc9fad50700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f415b070 0x7fc9f40c8840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00cd70 tx=0x7fc9ec00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.569+0000 7fc9eaffd700 1 -- 172.21.15.45:0/3178351586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec003a20 con 0x7fc9f415b070 2023-08-27T11:13:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.569+0000 7fc9eaffd700 1 -- 172.21.15.45:0/3178351586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec003b80 con 0x7fc9f415b070 2023-08-27T11:13:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.569+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f40c7d30 con 0x7fc9f415b070 2023-08-27T11:13:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.569+0000 7fc9eaffd700 1 -- 172.21.15.45:0/3178351586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec004020 con 0x7fc9f415b070 2023-08-27T11:13:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.569+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f41f3b30 con 0x7fc9f415b070 2023-08-27T11:13:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.570+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f40c8250 con 0x7fc9f415b070 2023-08-27T11:13:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.570+0000 7fc9eaffd700 1 -- 172.21.15.45:0/3178351586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc9ec004180 con 0x7fc9f415b070 2023-08-27T11:13:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.571+0000 7fc9eaffd700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc9d806a0e0 0x7fc9d806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.571+0000 7fc9eaffd700 1 -- 172.21.15.45:0/3178351586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc9ec012070 con 0x7fc9f415b070 2023-08-27T11:13:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.571+0000 7fc9fa54f700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc9d806a0e0 0x7fc9d806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.571+0000 7fc9fa54f700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc9d806a0e0 0x7fc9d806c5a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4000f60 tx=0x7fc9e4007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.575+0000 7fc9eaffd700 1 -- 172.21.15.45:0/3178351586 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc9f40c8250 con 0x7fc9f415b070 2023-08-27T11:13:14.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.777+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9f40c8250 con 0x7fc9f415b070 2023-08-27T11:13:14.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.778+0000 7fc9eaffd700 1 -- 172.21.15.45:0/3178351586 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7fc9f40c8250 con 0x7fc9f415b070 2023-08-27T11:13:14.779 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:14.780 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:14.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc9d806a0e0 msgr2=0x7fc9d806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc9d806a0e0 0x7fc9d806c5a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4000f60 tx=0x7fc9e4007e70 comp rx=0 tx=0).stop 2023-08-27T11:13:14.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f415b070 msgr2=0x7fc9f40c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f415b070 0x7fc9f40c8840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00cd70 tx=0x7fc9ec00a570 comp rx=0 tx=0).stop 2023-08-27T11:13:14.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 shutdown_connections 2023-08-27T11:13:14.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f415b070 0x7fc9f40c8840 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc9d806a0e0 0x7fc9d806c5a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9f406cec0 0x7fc9f41f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 --2- 172.21.15.45:0/3178351586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9f406e7b0 0x7fc9f41f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.780+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 >> 172.21.15.45:0/3178351586 conn(0x7fc9f40cfe50 msgr2=0x7fc9f4065fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:14.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.781+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 shutdown_connections 2023-08-27T11:13:14.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.781+0000 7fc9fc7b3700 1 -- 172.21.15.45:0/3178351586 wait complete. 2023-08-27T11:13:14.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:13:14.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:13:14.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.922+0000 7f4b359be700 1 Processor -- start 2023-08-27T11:13:14.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.922+0000 7f4b359be700 1 -- start start 2023-08-27T11:13:14.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b359be700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b300ce4b0 0x7f4b30167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b359be700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3015d6a0 con 0x7f4b3015d2c0 2023-08-27T11:13:14.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b359be700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b300d2c90 con 0x7f4b300b8c20 2023-08-27T11:13:14.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b359be700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300d1bf0 con 0x7f4b300ce4b0 2023-08-27T11:13:14.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2effd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4b3015d2c0 0x7f4b300d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40164/0 (socket says 172.21.15.45:40164) 2023-08-27T11:13:14.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2effd700 1 -- 172.21.15.45:0/1835723194 learned_addr learned my addr 172.21.15.45:0/1835723194 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:14.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189225252 0 0) 0x7f4b24001110 con 0x7f4b3015d2c0 2023-08-27T11:13:14.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2f7fe700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b300ce4b0 0x7f4b30167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b180035b0 con 0x7f4b3015d2c0 2023-08-27T11:13:14.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 502819266 0 0) 0x7f4b180035b0 con 0x7f4b3015d2c0 2023-08-27T11:13:14.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b300ce4b0 msgr2=0x7f4b30167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:14.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b300ce4b0 0x7f4b30167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 >> v1:172.21.15.135:6789/0 conn(0x7f4b300b8c20 legacy=0x7f4b300cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300d1ed0 con 0x7f4b3015d2c0 2023-08-27T11:13:14.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f4b24001800 con 0x7f4b3015d2c0 2023-08-27T11:13:14.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 >> v1:172.21.15.45:6789/0 conn(0x7f4b3015d2c0 legacy=0x7f4b300d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b18003a30 0x7f4b18003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b180043e0 0x7f4b180048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b300ce4b0 0x7f4b18005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300d1bf0 con 0x7f4b18003a30 2023-08-27T11:13:14.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1800ba40 con 0x7f4b180043e0 2023-08-27T11:13:14.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.923+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1800bc00 con 0x7f4b300ce4b0 2023-08-27T11:13:14.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2effd700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b18003a30 0x7f4b18003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2e7fc700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b180043e0 0x7f4b180048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2f7fe700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b300ce4b0 0x7f4b18005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2effd700 1 -- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b300ce4b0 msgr2=0x7f4b18005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:14.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2effd700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b300ce4b0 0x7f4b18005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2effd700 1 -- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b180043e0 msgr2=0x7f4b180048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:14.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2effd700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b180043e0 0x7f4b180048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2effd700 1 -- 172.21.15.45:0/1835723194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b180035b0 con 0x7f4b18003a30 2023-08-27T11:13:14.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2e7fc700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b180043e0 0x7f4b180048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:14.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2effd700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b18003a30 0x7f4b18003e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400c240 tx=0x7f4b18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4970a4c2dee0bb4c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:14.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2400cde0 con 0x7f4b18003a30 2023-08-27T11:13:14.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b24005960 con 0x7f4b18003a30 2023-08-27T11:13:14.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.924+0000 7f4b2dffb700 1 -- 172.21.15.45:0/1835723194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24005cf0 con 0x7f4b18003a30 2023-08-27T11:13:14.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.925+0000 7f4b359be700 1 -- 172.21.15.45:0/1835723194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b18003a30 msgr2=0x7f4b18003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.925+0000 7f4b359be700 1 --2- 172.21.15.45:0/1835723194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b18003a30 0x7f4b18003e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400c240 tx=0x7f4b18005880 comp rx=0 tx=0).stop 2023-08-27T11:13:14.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.925+0000 7f4b2f7fe700 1 -- 172.21.15.45:0/1835723194 reap_dead start 2023-08-27T11:13:14.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.925+0000 7f4b359be700 1 -- 172.21.15.45:0/1835723194 shutdown_connections 2023-08-27T11:13:14.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.925+0000 7f4b359be700 1 -- 172.21.15.45:0/1835723194 >> 172.21.15.45:0/1835723194 conn(0x7f4b300d2fd0 msgr2=0x7f4b300d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:14.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.925+0000 7f4b359be700 1 -- 172.21.15.45:0/1835723194 shutdown_connections 2023-08-27T11:13:14.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.925+0000 7f4b359be700 1 -- 172.21.15.45:0/1835723194 wait complete. 2023-08-27T11:13:14.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.926+0000 7f4b359be700 1 Processor -- start 2023-08-27T11:13:14.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.926+0000 7f4b359be700 1 -- start start 2023-08-27T11:13:14.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b359be700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b18003a30 0x7f4b300b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b359be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b3015f6e0 0x7f4b3015fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b359be700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b301f38b0 0x7f4b301f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b359be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b301600a0 con 0x7f4b3015f6e0 2023-08-27T11:13:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b359be700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b301f32f0 con 0x7f4b301f38b0 2023-08-27T11:13:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b359be700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b301f35d0 con 0x7f4b18003a30 2023-08-27T11:13:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b3015f6e0 0x7f4b3015fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b3015f6e0 0x7f4b3015fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53402/0 (socket says 172.21.15.45:53402) 2023-08-27T11:13:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2e7fc700 1 -- 172.21.15.45:0/2521206439 learned_addr learned my addr 172.21.15.45:0/2521206439 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2effd700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b18003a30 0x7f4b300b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2f7fe700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b301f38b0 0x7f4b301f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2e7fc700 1 -- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b18003a30 msgr2=0x7f4b300b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2e7fc700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b18003a30 0x7f4b300b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2e7fc700 1 -- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b301f38b0 msgr2=0x7f4b301f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:14.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2e7fc700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b301f38b0 0x7f4b301f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:14.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2e7fc700 1 -- 172.21.15.45:0/2521206439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300b4b40 con 0x7f4b3015f6e0 2023-08-27T11:13:14.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.927+0000 7f4b2e7fc700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b3015f6e0 0x7f4b3015fb60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000e800 tx=0x7f4b2000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:14.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.928+0000 7f4b17fff700 1 -- 172.21.15.45:0/2521206439 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b200052e0 con 0x7f4b3015f6e0 2023-08-27T11:13:14.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.928+0000 7f4b17fff700 1 -- 172.21.15.45:0/2521206439 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b20005440 con 0x7f4b3015f6e0 2023-08-27T11:13:14.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.928+0000 7f4b17fff700 1 -- 172.21.15.45:0/2521206439 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b20003880 con 0x7f4b3015f6e0 2023-08-27T11:13:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.928+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b300b4e20 con 0x7f4b3015f6e0 2023-08-27T11:13:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.928+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b300b5880 con 0x7f4b3015f6e0 2023-08-27T11:13:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.928+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b300b5060 con 0x7f4b3015f6e0 2023-08-27T11:13:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.929+0000 7f4b17fff700 1 -- 172.21.15.45:0/2521206439 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4b200039e0 con 0x7f4b3015f6e0 2023-08-27T11:13:14.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.929+0000 7f4b17fff700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b1c06a0e0 0x7f4b1c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.929+0000 7f4b17fff700 1 -- 172.21.15.45:0/2521206439 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4b20087ef0 con 0x7f4b3015f6e0 2023-08-27T11:13:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.930+0000 7f4b2effd700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b1c06a0e0 0x7f4b1c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.930+0000 7f4b2effd700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b1c06a0e0 0x7f4b1c06c5a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b24015040 tx=0x7f4b24007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:14.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:14.934+0000 7f4b17fff700 1 -- 172.21.15.45:0/2521206439 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4b300b5060 con 0x7f4b3015f6e0 2023-08-27T11:13:15.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.133+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b300b5060 con 0x7f4b3015f6e0 2023-08-27T11:13:15.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.134+0000 7f4b17fff700 1 -- 172.21.15.45:0/2521206439 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b300b5060 con 0x7f4b3015f6e0 2023-08-27T11:13:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:13:15.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b1c06a0e0 msgr2=0x7f4b1c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b1c06a0e0 0x7f4b1c06c5a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b24015040 tx=0x7f4b24007000 comp rx=0 tx=0).stop 2023-08-27T11:13:15.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b3015f6e0 msgr2=0x7f4b3015fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b3015f6e0 0x7f4b3015fb60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000e800 tx=0x7f4b2000eb10 comp rx=0 tx=0).stop 2023-08-27T11:13:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 shutdown_connections 2023-08-27T11:13:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4b1c06a0e0 0x7f4b1c06c5a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b301f38b0 0x7f4b301f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b18003a30 0x7f4b300b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 --2- 172.21.15.45:0/2521206439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b3015f6e0 0x7f4b3015fb60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 >> 172.21.15.45:0/2521206439 conn(0x7f4b300d2fd0 msgr2=0x7f4b3015c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 shutdown_connections 2023-08-27T11:13:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.136+0000 7f4b359be700 1 -- 172.21.15.45:0/2521206439 wait complete. 2023-08-27T11:13:15.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:13:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.277+0000 7f31c12c8700 1 Processor -- start 2023-08-27T11:13:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.277+0000 7f31c12c8700 1 -- start start 2023-08-27T11:13:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.278+0000 7f31c12c8700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f31bc157be0 0x7f31bc0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.278+0000 7f31c12c8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31bc15b460 con 0x7f31bc0b17c0 2023-08-27T11:13:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.278+0000 7f31c12c8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31bc0b2280 con 0x7f31bc06d5e0 2023-08-27T11:13:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.278+0000 7f31c12c8700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31bc157fc0 con 0x7f31bc157be0 2023-08-27T11:13:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.278+0000 7f31ba7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f31bc0b17c0 0x7f31bc06ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40184/0 (socket says 172.21.15.45:40184) 2023-08-27T11:13:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.278+0000 7f31ba7fc700 1 -- 172.21.15.45:0/2085320310 learned_addr learned my addr 172.21.15.45:0/2085320310 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31baffd700 1 --2- 172.21.15.45:0/2085320310 >> v2:172.21.15.135:3300/0 conn(0x7f31bc157be0 0x7f31bc0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31b9ffb700 1 -- 172.21.15.45:0/2085320310 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466973808 0 0) 0x7f31bc15b460 con 0x7f31bc0b17c0 2023-08-27T11:13:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31b9ffb700 1 -- 172.21.15.45:0/2085320310 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a40035b0 con 0x7f31bc0b17c0 2023-08-27T11:13:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31baffd700 1 -- 172.21.15.45:0/2085320310 >> v1:172.21.15.135:6789/0 conn(0x7f31bc06d5e0 legacy=0x7f31bc0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31baffd700 1 -- 172.21.15.45:0/2085320310 >> v1:172.21.15.45:6789/0 conn(0x7f31bc0b17c0 legacy=0x7f31bc06ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31baffd700 1 -- 172.21.15.45:0/2085320310 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc0b2560 con 0x7f31bc157be0 2023-08-27T11:13:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31baffd700 1 --2- 172.21.15.45:0/2085320310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31bc157be0 0x7f31bc0716b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009840 tx=0x7f31b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2158b8fd0a4f95a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31b9ffb700 1 -- 172.21.15.45:0/2085320310 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0011830 con 0x7f31bc157be0 2023-08-27T11:13:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31b9ffb700 1 -- 172.21.15.45:0/2085320310 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b0011990 con 0x7f31bc157be0 2023-08-27T11:13:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.279+0000 7f31b9ffb700 1 -- 172.21.15.45:0/2085320310 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0011d20 con 0x7f31bc157be0 2023-08-27T11:13:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.280+0000 7f31c12c8700 1 -- 172.21.15.45:0/2085320310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31bc157be0 msgr2=0x7f31bc0716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.280+0000 7f31c12c8700 1 --2- 172.21.15.45:0/2085320310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31bc157be0 0x7f31bc0716b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009840 tx=0x7f31b0009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.280+0000 7f31c12c8700 1 -- 172.21.15.45:0/2085320310 shutdown_connections 2023-08-27T11:13:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.280+0000 7f31c12c8700 1 --2- 172.21.15.45:0/2085320310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31bc157be0 0x7f31bc0716b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.280+0000 7f31c12c8700 1 -- 172.21.15.45:0/2085320310 >> 172.21.15.45:0/2085320310 conn(0x7f31bc0cc7c0 msgr2=0x7f31bc0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.280+0000 7f31c12c8700 1 -- 172.21.15.45:0/2085320310 shutdown_connections 2023-08-27T11:13:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.280+0000 7f31c12c8700 1 -- 172.21.15.45:0/2085320310 wait complete. 2023-08-27T11:13:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.281+0000 7f31c12c8700 1 Processor -- start 2023-08-27T11:13:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.281+0000 7f31c12c8700 1 -- start start 2023-08-27T11:13:15.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31c12c8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31bc06d5e0 0x7f31bc0c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31c12c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc0b17c0 0x7f31bc154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31c12c8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31bc157be0 0x7f31bc1e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31c12c8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0c6b80 con 0x7f31bc0b17c0 2023-08-27T11:13:15.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31c12c8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0b4800 con 0x7f31bc06d5e0 2023-08-27T11:13:15.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31c12c8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0c63a0 con 0x7f31bc157be0 2023-08-27T11:13:15.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31ba7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc0b17c0 0x7f31bc154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31ba7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc0b17c0 0x7f31bc154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53416/0 (socket says 172.21.15.45:53416) 2023-08-27T11:13:15.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31ba7fc700 1 -- 172.21.15.45:0/919465133 learned_addr learned my addr 172.21.15.45:0/919465133 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:15.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31baffd700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31bc06d5e0 0x7f31bc0c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.282+0000 7f31bb7fe700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31bc157be0 0x7f31bc1e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31ba7fc700 1 -- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31bc157be0 msgr2=0x7f31bc1e6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31ba7fc700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31bc157be0 0x7f31bc1e6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31ba7fc700 1 -- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31bc06d5e0 msgr2=0x7f31bc0c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31ba7fc700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31bc06d5e0 0x7f31bc0c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31ba7fc700 1 -- 172.21.15.45:0/919465133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b00094a0 con 0x7f31bc0b17c0 2023-08-27T11:13:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31ba7fc700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc0b17c0 0x7f31bc154420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c700 tx=0x7f31ac00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31a37fe700 1 -- 172.21.15.45:0/919465133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac005d70 con 0x7f31bc0b17c0 2023-08-27T11:13:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31a37fe700 1 -- 172.21.15.45:0/919465133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac0070d0 con 0x7f31bc0b17c0 2023-08-27T11:13:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc0b4aa0 con 0x7f31bc0b17c0 2023-08-27T11:13:15.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31a37fe700 1 -- 172.21.15.45:0/919465133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac007460 con 0x7f31bc0b17c0 2023-08-27T11:13:15.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.283+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc0c7240 con 0x7f31bc0b17c0 2023-08-27T11:13:15.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.284+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc072690 con 0x7f31bc0b17c0 2023-08-27T11:13:15.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.284+0000 7f31a37fe700 1 -- 172.21.15.45:0/919465133 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f31ac0075c0 con 0x7f31bc0b17c0 2023-08-27T11:13:15.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.285+0000 7f31a37fe700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f31a806a0e0 0x7f31a806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.285+0000 7f31a37fe700 1 -- 172.21.15.45:0/919465133 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f31ac087c00 con 0x7f31bc0b17c0 2023-08-27T11:13:15.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.285+0000 7f31baffd700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f31a806a0e0 0x7f31a806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.286+0000 7f31baffd700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f31a806a0e0 0x7f31a806c5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f31b00090b0 tx=0x7f31b0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.289+0000 7f31a37fe700 1 -- 172.21.15.45:0/919465133 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f31ac08e080 con 0x7f31bc0b17c0 2023-08-27T11:13:15.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.487+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f31bc15a790 con 0x7f31bc0b17c0 2023-08-27T11:13:15.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.488+0000 7f31a37fe700 1 -- 172.21.15.45:0/919465133 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f31ac017070 con 0x7f31bc0b17c0 2023-08-27T11:13:15.489 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:13:15.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f31a806a0e0 msgr2=0x7f31a806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f31a806a0e0 0x7f31a806c5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f31b00090b0 tx=0x7f31b0008030 comp rx=0 tx=0).stop 2023-08-27T11:13:15.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc0b17c0 msgr2=0x7f31bc154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc0b17c0 0x7f31bc154420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c700 tx=0x7f31ac00ca10 comp rx=0 tx=0).stop 2023-08-27T11:13:15.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 shutdown_connections 2023-08-27T11:13:15.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f31a806a0e0 0x7f31a806c5a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f31bc157be0 0x7f31bc1e6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31bc06d5e0 0x7f31bc0c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 --2- 172.21.15.45:0/919465133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc0b17c0 0x7f31bc154420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 >> 172.21.15.45:0/919465133 conn(0x7f31bc0cc7c0 msgr2=0x7f31bc0cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 shutdown_connections 2023-08-27T11:13:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.490+0000 7f31c12c8700 1 -- 172.21.15.45:0/919465133 wait complete. 2023-08-27T11:13:15.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:13:15.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.630+0000 7f36cc8ab700 1 Processor -- start 2023-08-27T11:13:15.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.630+0000 7f36cc8ab700 1 -- start start 2023-08-27T11:13:15.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.631+0000 7f36cc8ab700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f36c806d5e0 0x7f36c80b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.631+0000 7f36cc8ab700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c815b460 con 0x7f36c8157be0 2023-08-27T11:13:15.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.631+0000 7f36cc8ab700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c80b2280 con 0x7f36c80b17c0 2023-08-27T11:13:15.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.631+0000 7f36cc8ab700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c8157fc0 con 0x7f36c806d5e0 2023-08-27T11:13:15.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f36c8157be0 0x7f36c80716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40198/0 (socket says 172.21.15.45:40198) 2023-08-27T11:13:15.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c659c700 1 -- 172.21.15.45:0/1557720384 learned_addr learned my addr 172.21.15.45:0/1557720384 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:15.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684910541 0 0) 0x7f36c815b460 con 0x7f36c8157be0 2023-08-27T11:13:15.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c6d9d700 1 --2- 172.21.15.45:0/1557720384 >> v2:172.21.15.135:3300/0 conn(0x7f36c806d5e0 0x7f36c80b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36b40035b0 con 0x7f36c8157be0 2023-08-27T11:13:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 693608313 0 0) 0x7f36b40035b0 con 0x7f36c8157be0 2023-08-27T11:13:15.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 >> v2:172.21.15.135:3300/0 conn(0x7f36c806d5e0 msgr2=0x7f36c80b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 --2- 172.21.15.45:0/1557720384 >> v2:172.21.15.135:3300/0 conn(0x7f36c806d5e0 0x7f36c80b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 >> v1:172.21.15.135:6789/0 conn(0x7f36c80b17c0 legacy=0x7f36c806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c80b2560 con 0x7f36c8157be0 2023-08-27T11:13:15.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f36c0001670 con 0x7f36c8157be0 2023-08-27T11:13:15.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 >> v1:172.21.15.45:6789/0 conn(0x7f36c8157be0 legacy=0x7f36c80716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36b4003a00 0x7f36b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36b4004850 0x7f36b4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4005210 0x7f36b400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c8157fc0 con 0x7f36b4005210 2023-08-27T11:13:15.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b400c0b0 con 0x7f36b4004850 2023-08-27T11:13:15.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b400c270 con 0x7f36b4003a00 2023-08-27T11:13:15.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c6d9d700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4005210 0x7f36b400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c5d9b700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36b4004850 0x7f36b4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c659c700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36b4003a00 0x7f36b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c6d9d700 1 -- 172.21.15.45:0/1557720384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36b4003a00 msgr2=0x7f36b4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:15.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c6d9d700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36b4003a00 0x7f36b4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c6d9d700 1 -- 172.21.15.45:0/1557720384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36b4004850 msgr2=0x7f36b4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:15.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c6d9d700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36b4004850 0x7f36b4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c6d9d700 1 -- 172.21.15.45:0/1557720384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b40035b0 con 0x7f36b4005210 2023-08-27T11:13:15.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.632+0000 7f36c5d9b700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36b4004850 0x7f36b4004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:15.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36c6d9d700 1 -- 172.21.15.45:0/1557720384 reap_dead start 2023-08-27T11:13:15.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36c6d9d700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4005210 0x7f36b400ba10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009fc0 tx=0x7f36b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2dc80f7e9c1f7c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8003a20 con 0x7f36b4005210 2023-08-27T11:13:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b800f040 con 0x7f36b4005210 2023-08-27T11:13:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36c559a700 1 -- 172.21.15.45:0/1557720384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8003de0 con 0x7f36b4005210 2023-08-27T11:13:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36cc8ab700 1 -- 172.21.15.45:0/1557720384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4005210 msgr2=0x7f36b400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36cc8ab700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4005210 0x7f36b400ba10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009fc0 tx=0x7f36b4005880 comp rx=0 tx=0).stop 2023-08-27T11:13:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36cc8ab700 1 -- 172.21.15.45:0/1557720384 shutdown_connections 2023-08-27T11:13:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36cc8ab700 1 --2- 172.21.15.45:0/1557720384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4005210 0x7f36b400ba10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36cc8ab700 1 -- 172.21.15.45:0/1557720384 >> 172.21.15.45:0/1557720384 conn(0x7f36c80cc7c0 msgr2=0x7f36c80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36cc8ab700 1 -- 172.21.15.45:0/1557720384 shutdown_connections 2023-08-27T11:13:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.633+0000 7f36cc8ab700 1 -- 172.21.15.45:0/1557720384 wait complete. 2023-08-27T11:13:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.635+0000 7f36cc8ab700 1 Processor -- start 2023-08-27T11:13:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.635+0000 7f36cc8ab700 1 -- start start 2023-08-27T11:13:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36cc8ab700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36b4005210 0x7f36c81f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36cc8ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c8154d70 0x7f36c80c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36cc8ab700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c80b5170 0x7f36c80ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36cc8ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c81f2790 con 0x7f36c8154d70 2023-08-27T11:13:15.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36cc8ab700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80c7b20 con 0x7f36c80b5170 2023-08-27T11:13:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36cc8ab700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80c72c0 con 0x7f36b4005210 2023-08-27T11:13:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36c5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c8154d70 0x7f36c80c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36c5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c8154d70 0x7f36c80c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53436/0 (socket says 172.21.15.45:53436) 2023-08-27T11:13:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36c5d9b700 1 -- 172.21.15.45:0/961428397 learned_addr learned my addr 172.21.15.45:0/961428397 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36c659c700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36b4005210 0x7f36c81f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.636+0000 7f36c6d9d700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c80b5170 0x7f36c80ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36c5d9b700 1 -- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36b4005210 msgr2=0x7f36c81f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36c5d9b700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36b4005210 0x7f36c81f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36c5d9b700 1 -- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c80b5170 msgr2=0x7f36c80ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36c5d9b700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c80b5170 0x7f36c80ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36c5d9b700 1 -- 172.21.15.45:0/961428397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c80c7560 con 0x7f36c8154d70 2023-08-27T11:13:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36c6d9d700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c80b5170 0x7f36c80ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36c5d9b700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c8154d70 0x7f36c80c6d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f36b000e550 tx=0x7f36b000e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:15.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36beffd700 1 -- 172.21.15.45:0/961428397 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b00052e0 con 0x7f36c8154d70 2023-08-27T11:13:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36beffd700 1 -- 172.21.15.45:0/961428397 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b0005440 con 0x7f36c8154d70 2023-08-27T11:13:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36beffd700 1 -- 172.21.15.45:0/961428397 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b00038c0 con 0x7f36c8154d70 2023-08-27T11:13:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c80c7840 con 0x7f36c8154d70 2023-08-27T11:13:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.637+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c81f1d30 con 0x7f36c8154d70 2023-08-27T11:13:15.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.638+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c8072690 con 0x7f36c8154d70 2023-08-27T11:13:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.639+0000 7f36beffd700 1 -- 172.21.15.45:0/961428397 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f36b0003a20 con 0x7f36c8154d70 2023-08-27T11:13:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.639+0000 7f36beffd700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f36a806a0e0 0x7f36a806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.639+0000 7f36beffd700 1 -- 172.21.15.45:0/961428397 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f36b0087dd0 con 0x7f36c8154d70 2023-08-27T11:13:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.639+0000 7f36c659c700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f36a806a0e0 0x7f36a806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.640+0000 7f36c659c700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f36a806a0e0 0x7f36a806c5a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f36c0000fb0 tx=0x7f36c0007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.643+0000 7f36beffd700 1 -- 172.21.15.45:0/961428397 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f36b00532d0 con 0x7f36c8154d70 2023-08-27T11:13:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.844+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36c80734a0 con 0x7f36c8154d70 2023-08-27T11:13:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.845+0000 7f36beffd700 1 -- 172.21.15.45:0/961428397 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36b00568f0 con 0x7f36c8154d70 2023-08-27T11:13:15.846 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f36a806a0e0 msgr2=0x7f36a806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f36a806a0e0 0x7f36a806c5a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f36c0000fb0 tx=0x7f36c0007040 comp rx=0 tx=0).stop 2023-08-27T11:13:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c8154d70 msgr2=0x7f36c80c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c8154d70 0x7f36c80c6d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f36b000e550 tx=0x7f36b000e860 comp rx=0 tx=0).stop 2023-08-27T11:13:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 shutdown_connections 2023-08-27T11:13:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f36a806a0e0 0x7f36a806c5a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c80b5170 0x7f36c80ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36b4005210 0x7f36c81f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 --2- 172.21.15.45:0/961428397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c8154d70 0x7f36c80c6d80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 >> 172.21.15.45:0/961428397 conn(0x7f36c80cc7c0 msgr2=0x7f36c806ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 shutdown_connections 2023-08-27T11:13:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.847+0000 7f36cc8ab700 1 -- 172.21.15.45:0/961428397 wait complete. 2023-08-27T11:13:15.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:13:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.988+0000 7f112e19e700 1 Processor -- start 2023-08-27T11:13:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.988+0000 7f112e19e700 1 -- start start 2023-08-27T11:13:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112e19e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f112815d2e0 0x7f11280af300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112e19e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 0x7f1128160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112e19e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11280af130 con 0x7f11280cd9d0 2023-08-27T11:13:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112e19e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112806e810 con 0x7f112815d2e0 2023-08-27T11:13:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112e19e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280d0d30 con 0x7f112806a990 2023-08-27T11:13:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112d19c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f112815d2e0 0x7f11280af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112d19c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f112815d2e0 0x7f11280af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53442/0 (socket says 172.21.15.45:53442) 2023-08-27T11:13:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112d19c700 1 -- 172.21.15.45:0/3483199781 learned_addr learned my addr 172.21.15.45:0/3483199781 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112d99d700 1 --2- 172.21.15.45:0/3483199781 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 0x7f1128160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112d19c700 1 -- 172.21.15.45:0/3483199781 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 msgr2=0x7f1128160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112d19c700 1 --2- 172.21.15.45:0/3483199781 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 0x7f1128160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112d19c700 1 -- 172.21.15.45:0/3483199781 >> v1:172.21.15.135:6789/0 conn(0x7f11280cd9d0 legacy=0x7f112806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.989+0000 7f112d19c700 1 -- 172.21.15.45:0/3483199781 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280cde40 con 0x7f112815d2e0 2023-08-27T11:13:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f112d19c700 1 --2- 172.21.15.45:0/3483199781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 0x7f11280af300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f11180098f0 tx=0x7f1118009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65e77e96eaee46f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f111ffff700 1 -- 172.21.15.45:0/3483199781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118011830 con 0x7f112815d2e0 2023-08-27T11:13:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f111ffff700 1 -- 172.21.15.45:0/3483199781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118011990 con 0x7f112815d2e0 2023-08-27T11:13:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f111ffff700 1 -- 172.21.15.45:0/3483199781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118011d20 con 0x7f112815d2e0 2023-08-27T11:13:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f112e19e700 1 -- 172.21.15.45:0/3483199781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 msgr2=0x7f11280af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f112e19e700 1 --2- 172.21.15.45:0/3483199781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 0x7f11280af300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f11180098f0 tx=0x7f1118009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f112e19e700 1 -- 172.21.15.45:0/3483199781 shutdown_connections 2023-08-27T11:13:15.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f112e19e700 1 --2- 172.21.15.45:0/3483199781 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 0x7f1128160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f112e19e700 1 --2- 172.21.15.45:0/3483199781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 0x7f11280af300 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:15.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f112e19e700 1 -- 172.21.15.45:0/3483199781 >> 172.21.15.45:0/3483199781 conn(0x7f11280cfe00 msgr2=0x7f11280d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:15.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f112e19e700 1 -- 172.21.15.45:0/3483199781 shutdown_connections 2023-08-27T11:13:15.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.990+0000 7f112e19e700 1 -- 172.21.15.45:0/3483199781 wait complete. 2023-08-27T11:13:15.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.992+0000 7f112e19e700 1 Processor -- start 2023-08-27T11:13:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.992+0000 7f112e19e700 1 -- start start 2023-08-27T11:13:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112e19e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 0x7f1128157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112e19e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11280cd9d0 0x7f11280b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112e19e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 0x7f11281544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112e19e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11280c36d0 con 0x7f112815d2e0 2023-08-27T11:13:16.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112e19e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11281514d0 con 0x7f11280cd9d0 2023-08-27T11:13:16.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112e19e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280b3e70 con 0x7f112806a990 2023-08-27T11:13:16.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d99d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 0x7f11281544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d99d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 0x7f11281544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53452/0 (socket says 172.21.15.45:53452) 2023-08-27T11:13:16.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d99d700 1 -- 172.21.15.45:0/3131233772 learned_addr learned my addr 172.21.15.45:0/3131233772 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:16.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d19c700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 0x7f1128157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112c99b700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11280cd9d0 0x7f11280b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d99d700 1 -- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 msgr2=0x7f1128157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d99d700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 0x7f1128157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d99d700 1 -- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11280cd9d0 msgr2=0x7f11280b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d99d700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11280cd9d0 0x7f11280b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d99d700 1 -- 172.21.15.45:0/3131233772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1118009550 con 0x7f112815d2e0 2023-08-27T11:13:16.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f112d99d700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 0x7f11281544b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f112000efc0 tx=0x7f1120009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:16.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.993+0000 7f111dffb700 1 -- 172.21.15.45:0/3131233772 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120009b20 con 0x7f112815d2e0 2023-08-27T11:13:16.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.994+0000 7f111dffb700 1 -- 172.21.15.45:0/3131233772 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112000f040 con 0x7f112815d2e0 2023-08-27T11:13:16.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.994+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11280b4150 con 0x7f112815d2e0 2023-08-27T11:13:16.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.994+0000 7f111dffb700 1 -- 172.21.15.45:0/3131233772 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000cc90 con 0x7f112815d2e0 2023-08-27T11:13:16.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.994+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11280b4710 con 0x7f112815d2e0 2023-08-27T11:13:16.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.995+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11280af620 con 0x7f112815d2e0 2023-08-27T11:13:16.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.996+0000 7f111dffb700 1 -- 172.21.15.45:0/3131233772 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1120014480 con 0x7f112815d2e0 2023-08-27T11:13:16.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.996+0000 7f111dffb700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f110c06a0e0 0x7f110c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.996+0000 7f111dffb700 1 -- 172.21.15.45:0/3131233772 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1120089760 con 0x7f112815d2e0 2023-08-27T11:13:16.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.996+0000 7f112d19c700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f110c06a0e0 0x7f110c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:15.997+0000 7f112d19c700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f110c06a0e0 0x7f110c06c5a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1118009c40 tx=0x7f1118008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:16.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.000+0000 7f111dffb700 1 -- 172.21.15.45:0/3131233772 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1120054c90 con 0x7f112815d2e0 2023-08-27T11:13:16.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.199+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1128072780 con 0x7f112815d2e0 2023-08-27T11:13:16.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.200+0000 7f111dffb700 1 -- 172.21.15.45:0/3131233772 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11200582b0 con 0x7f112815d2e0 2023-08-27T11:13:16.202 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:13:16.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f110c06a0e0 msgr2=0x7f110c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f110c06a0e0 0x7f110c06c5a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1118009c40 tx=0x7f1118008030 comp rx=0 tx=0).stop 2023-08-27T11:13:16.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 msgr2=0x7f11281544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 0x7f11281544b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f112000efc0 tx=0x7f1120009f60 comp rx=0 tx=0).stop 2023-08-27T11:13:16.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 shutdown_connections 2023-08-27T11:13:16.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f112815d2e0 0x7f11281544b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f110c06a0e0 0x7f110c06c5a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f112806a990 0x7f1128157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 --2- 172.21.15.45:0/3131233772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11280cd9d0 0x7f11280b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 >> 172.21.15.45:0/3131233772 conn(0x7f11280cfe00 msgr2=0x7f11280ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:16.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 shutdown_connections 2023-08-27T11:13:16.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.202+0000 7f112e19e700 1 -- 172.21.15.45:0/3131233772 wait complete. 2023-08-27T11:13:16.215 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:13:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.343+0000 7fdc14922700 1 Processor -- start 2023-08-27T11:13:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.343+0000 7fdc14922700 1 -- start start 2023-08-27T11:13:16.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.344+0000 7fdc14922700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc100cdf90 0x7fdc1006ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.344+0000 7fdc14922700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc10161870 con 0x7fdc100d0a50 2023-08-27T11:13:16.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.344+0000 7fdc14922700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc10161da0 con 0x7fdc1015d0a0 2023-08-27T11:13:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.344+0000 7fdc14922700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1006d3f0 con 0x7fdc100cdf90 2023-08-27T11:13:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.344+0000 7fdc0dd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc100cdf90 0x7fdc1006ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.344+0000 7fdc0dd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc100cdf90 0x7fdc1006ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53458/0 (socket says 172.21.15.45:53458) 2023-08-27T11:13:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.344+0000 7fdc0ed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdc100d0a50 0x7fdc1006b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40236/0 (socket says 172.21.15.45:40236) 2023-08-27T11:13:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.344+0000 7fdc0dd9b700 1 -- 172.21.15.45:0/802370610 learned_addr learned my addr 172.21.15.45:0/802370610 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.345+0000 7fdc0dd9b700 1 -- 172.21.15.45:0/802370610 >> v1:172.21.15.135:6789/0 conn(0x7fdc1015d0a0 legacy=0x7fdc100cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.345+0000 7fdc0dd9b700 1 -- 172.21.15.45:0/802370610 >> v1:172.21.15.45:6789/0 conn(0x7fdc100d0a50 legacy=0x7fdc1006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.345+0000 7fdc0dd9b700 1 -- 172.21.15.45:0/802370610 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc10161ac0 con 0x7fdc100cdf90 2023-08-27T11:13:16.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.345+0000 7fdc0dd9b700 1 --2- 172.21.15.45:0/802370610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100cdf90 0x7fdc1006ceb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdc040098f0 tx=0x7fdc04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34567fef647942b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:16.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.345+0000 7fdc0d59a700 1 -- 172.21.15.45:0/802370610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc04004230 con 0x7fdc100cdf90 2023-08-27T11:13:16.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.345+0000 7fdc0d59a700 1 -- 172.21.15.45:0/802370610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04004390 con 0x7fdc100cdf90 2023-08-27T11:13:16.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.345+0000 7fdc0d59a700 1 -- 172.21.15.45:0/802370610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc040036a0 con 0x7fdc100cdf90 2023-08-27T11:13:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.346+0000 7fdc14922700 1 -- 172.21.15.45:0/802370610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100cdf90 msgr2=0x7fdc1006ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.346+0000 7fdc14922700 1 --2- 172.21.15.45:0/802370610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100cdf90 0x7fdc1006ceb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdc040098f0 tx=0x7fdc04009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.346+0000 7fdc14922700 1 -- 172.21.15.45:0/802370610 shutdown_connections 2023-08-27T11:13:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.346+0000 7fdc14922700 1 --2- 172.21.15.45:0/802370610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc100cdf90 0x7fdc1006ceb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.346+0000 7fdc14922700 1 -- 172.21.15.45:0/802370610 >> 172.21.15.45:0/802370610 conn(0x7fdc1006f960 msgr2=0x7fdc1006fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:16.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.346+0000 7fdc14922700 1 -- 172.21.15.45:0/802370610 shutdown_connections 2023-08-27T11:13:16.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.346+0000 7fdc14922700 1 -- 172.21.15.45:0/802370610 wait complete. 2023-08-27T11:13:16.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.347+0000 7fdc14922700 1 Processor -- start 2023-08-27T11:13:16.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.348+0000 7fdc14922700 1 -- start start 2023-08-27T11:13:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.348+0000 7fdc14922700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc100cdf90 0x7fdc10151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.348+0000 7fdc14922700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d0a50 0x7fdc100c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.348+0000 7fdc14922700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1015d0a0 0x7fdc100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.348+0000 7fdc14922700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100c5c20 con 0x7fdc1015d0a0 2023-08-27T11:13:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc14922700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc10151850 con 0x7fdc100d0a50 2023-08-27T11:13:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc14922700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100c5380 con 0x7fdc100cdf90 2023-08-27T11:13:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1015d0a0 0x7fdc100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1015d0a0 0x7fdc100b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53466/0 (socket says 172.21.15.45:53466) 2023-08-27T11:13:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0ed9d700 1 -- 172.21.15.45:0/1670022247 learned_addr learned my addr 172.21.15.45:0/1670022247 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0e59c700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc100cdf90 0x7fdc10151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0dd9b700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d0a50 0x7fdc100c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0ed9d700 1 -- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc100cdf90 msgr2=0x7fdc10151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0ed9d700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc100cdf90 0x7fdc10151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0ed9d700 1 -- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d0a50 msgr2=0x7fdc100c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0ed9d700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d0a50 0x7fdc100c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0ed9d700 1 -- 172.21.15.45:0/1670022247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc04009550 con 0x7fdc1015d0a0 2023-08-27T11:13:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc0ed9d700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1015d0a0 0x7fdc100b2000 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800cbd0 tx=0x7fdc0800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdbfeffd700 1 -- 172.21.15.45:0/1670022247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0800abb0 con 0x7fdc1015d0a0 2023-08-27T11:13:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdbfeffd700 1 -- 172.21.15.45:0/1670022247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0800ad10 con 0x7fdc1015d0a0 2023-08-27T11:13:16.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc100c5660 con 0x7fdc1015d0a0 2023-08-27T11:13:16.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdbfeffd700 1 -- 172.21.15.45:0/1670022247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08007270 con 0x7fdc1015d0a0 2023-08-27T11:13:16.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.349+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc100c5f70 con 0x7fdc1015d0a0 2023-08-27T11:13:16.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.350+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc100c5dd0 con 0x7fdc1015d0a0 2023-08-27T11:13:16.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.351+0000 7fdbfeffd700 1 -- 172.21.15.45:0/1670022247 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdc080073d0 con 0x7fdc1015d0a0 2023-08-27T11:13:16.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.351+0000 7fdbfeffd700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbf006a0e0 0x7fdbf006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.351+0000 7fdbfeffd700 1 -- 172.21.15.45:0/1670022247 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdc08088590 con 0x7fdc1015d0a0 2023-08-27T11:13:16.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.351+0000 7fdc0e59c700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbf006a0e0 0x7fdbf006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.352+0000 7fdc0e59c700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbf006a0e0 0x7fdbf006c5a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc100b6150 tx=0x7fdc00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:16.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.355+0000 7fdbfeffd700 1 -- 172.21.15.45:0/1670022247 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdc100c5dd0 con 0x7fdc1015d0a0 2023-08-27T11:13:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.559+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdc100cee30 con 0x7fdc1015d0a0 2023-08-27T11:13:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.560+0000 7fdbfeffd700 1 -- 172.21.15.45:0/1670022247 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdc100cee30 con 0x7fdc1015d0a0 2023-08-27T11:13:16.561 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbf006a0e0 msgr2=0x7fdbf006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbf006a0e0 0x7fdbf006c5a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc100b6150 tx=0x7fdc00009040 comp rx=0 tx=0).stop 2023-08-27T11:13:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1015d0a0 msgr2=0x7fdc100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1015d0a0 0x7fdc100b2000 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800cbd0 tx=0x7fdc0800cf90 comp rx=0 tx=0).stop 2023-08-27T11:13:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 shutdown_connections 2023-08-27T11:13:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc1015d0a0 0x7fdc100b2000 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdbf006a0e0 0x7fdbf006c5a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc100cdf90 0x7fdc10151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 --2- 172.21.15.45:0/1670022247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc100d0a50 0x7fdc100c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 >> 172.21.15.45:0/1670022247 conn(0x7fdc1006f960 msgr2=0x7fdc1006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 shutdown_connections 2023-08-27T11:13:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.562+0000 7fdc14922700 1 -- 172.21.15.45:0/1670022247 wait complete. 2023-08-27T11:13:16.576 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:13:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.702+0000 7fac609e0700 1 Processor -- start 2023-08-27T11:13:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.702+0000 7fac609e0700 1 -- start start 2023-08-27T11:13:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac609e0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac5c15d340 0x7fac5c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac609e0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fac5c06a990 0x7fac5c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac609e0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5c06b160 con 0x7fac5c0cd880 2023-08-27T11:13:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac609e0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac5c072010 con 0x7fac5c15d340 2023-08-27T11:13:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac609e0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac5c0af590 con 0x7fac5c06a990 2023-08-27T11:13:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac5c15d340 0x7fac5c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac5c15d340 0x7fac5c160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53470/0 (socket says 172.21.15.45:53470) 2023-08-27T11:13:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5a59c700 1 -- 172.21.15.45:0/2891256179 learned_addr learned my addr 172.21.15.45:0/2891256179 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5ad9d700 1 --2- 172.21.15.45:0/2891256179 >> v2:172.21.15.135:3300/0 conn(0x7fac5c06a990 0x7fac5c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5a59c700 1 -- 172.21.15.45:0/2891256179 >> v2:172.21.15.135:3300/0 conn(0x7fac5c06a990 msgr2=0x7fac5c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5a59c700 1 --2- 172.21.15.45:0/2891256179 >> v2:172.21.15.135:3300/0 conn(0x7fac5c06a990 0x7fac5c0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5a59c700 1 -- 172.21.15.45:0/2891256179 >> v1:172.21.15.135:6789/0 conn(0x7fac5c0cd880 legacy=0x7fac5c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5a59c700 1 -- 172.21.15.45:0/2891256179 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c0af2b0 con 0x7fac5c15d340 2023-08-27T11:13:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5ad9d700 1 --2- 172.21.15.45:0/2891256179 >> v2:172.21.15.135:3300/0 conn(0x7fac5c06a990 0x7fac5c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.703+0000 7fac5a59c700 1 --2- 172.21.15.45:0/2891256179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c15d340 0x7fac5c160d00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac5c072af0 tx=0x7fac4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a0180f989738c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac5959a700 1 -- 172.21.15.45:0/2891256179 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00b870 con 0x7fac5c15d340 2023-08-27T11:13:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac5959a700 1 -- 172.21.15.45:0/2891256179 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4c00b9d0 con 0x7fac5c15d340 2023-08-27T11:13:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac5959a700 1 -- 172.21.15.45:0/2891256179 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00bd60 con 0x7fac5c15d340 2023-08-27T11:13:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac609e0700 1 -- 172.21.15.45:0/2891256179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c15d340 msgr2=0x7fac5c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac609e0700 1 --2- 172.21.15.45:0/2891256179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c15d340 0x7fac5c160d00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac5c072af0 tx=0x7fac4c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac609e0700 1 -- 172.21.15.45:0/2891256179 shutdown_connections 2023-08-27T11:13:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac609e0700 1 --2- 172.21.15.45:0/2891256179 >> v2:172.21.15.135:3300/0 conn(0x7fac5c06a990 0x7fac5c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac609e0700 1 --2- 172.21.15.45:0/2891256179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c15d340 0x7fac5c160d00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac609e0700 1 -- 172.21.15.45:0/2891256179 >> 172.21.15.45:0/2891256179 conn(0x7fac5c0cc7c0 msgr2=0x7fac5c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac609e0700 1 -- 172.21.15.45:0/2891256179 shutdown_connections 2023-08-27T11:13:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.704+0000 7fac609e0700 1 -- 172.21.15.45:0/2891256179 wait complete. 2023-08-27T11:13:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac609e0700 1 Processor -- start 2023-08-27T11:13:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac609e0700 1 -- start start 2023-08-27T11:13:16.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac609e0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac5c06a990 0x7fac5c0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac609e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c0cd880 0x7fac5c0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac609e0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac5c15d340 0x7fac5c1f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac609e0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c0c9270 con 0x7fac5c0cd880 2023-08-27T11:13:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac609e0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c0c9480 con 0x7fac5c06a990 2023-08-27T11:13:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac609e0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c0c9760 con 0x7fac5c15d340 2023-08-27T11:13:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac59d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c0cd880 0x7fac5c0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac59d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c0cd880 0x7fac5c0c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53478/0 (socket says 172.21.15.45:53478) 2023-08-27T11:13:16.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.706+0000 7fac59d9b700 1 -- 172.21.15.45:0/2942089358 learned_addr learned my addr 172.21.15.45:0/2942089358 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:16.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac5ad9d700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac5c15d340 0x7fac5c1f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac5a59c700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac5c06a990 0x7fac5c0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac59d9b700 1 -- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac5c15d340 msgr2=0x7fac5c1f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac59d9b700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac5c15d340 0x7fac5c1f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac59d9b700 1 -- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac5c06a990 msgr2=0x7fac5c0c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac59d9b700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac5c06a990 0x7fac5c0c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac59d9b700 1 -- 172.21.15.45:0/2942089358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c009550 con 0x7fac5c0cd880 2023-08-27T11:13:16.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac59d9b700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c0cd880 0x7fac5c0c8d30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fac5000c450 tx=0x7fac5000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:16.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac4affd700 1 -- 172.21.15.45:0/2942089358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500052d0 con 0x7fac5c0cd880 2023-08-27T11:13:16.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac4affd700 1 -- 172.21.15.45:0/2942089358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac50005430 con 0x7fac5c0cd880 2023-08-27T11:13:16.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac4affd700 1 -- 172.21.15.45:0/2942089358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500038c0 con 0x7fac5c0cd880 2023-08-27T11:13:16.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5c0c9a40 con 0x7fac5c0cd880 2023-08-27T11:13:16.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.707+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5c1f6c00 con 0x7fac5c0cd880 2023-08-27T11:13:16.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.708+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5c072690 con 0x7fac5c0cd880 2023-08-27T11:13:16.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.709+0000 7fac4affd700 1 -- 172.21.15.45:0/2942089358 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fac50003a20 con 0x7fac5c0cd880 2023-08-27T11:13:16.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.709+0000 7fac4affd700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fac3c06a0e0 0x7fac3c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:16.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.709+0000 7fac4affd700 1 -- 172.21.15.45:0/2942089358 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fac50087b90 con 0x7fac5c0cd880 2023-08-27T11:13:16.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.709+0000 7fac5a59c700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fac3c06a0e0 0x7fac3c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:16.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.710+0000 7fac5a59c700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fac3c06a0e0 0x7fac3c06c5a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fac4c014040 tx=0x7fac4c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:16.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.713+0000 7fac4affd700 1 -- 172.21.15.45:0/2942089358 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fac50052f50 con 0x7fac5c0cd880 2023-08-27T11:13:16.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.929+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fac5c0734a0 con 0x7fac5c0cd880 2023-08-27T11:13:16.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.929+0000 7fac4affd700 1 -- 172.21.15.45:0/2942089358 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fac5000ce60 con 0x7fac5c0cd880 2023-08-27T11:13:16.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.931+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fac3c06a0e0 msgr2=0x7fac3c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.931+0000 7fac609e0700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fac3c06a0e0 0x7fac3c06c5a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fac4c014040 tx=0x7fac4c008030 comp rx=0 tx=0).stop 2023-08-27T11:13:16.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.931+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c0cd880 msgr2=0x7fac5c0c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:16.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.931+0000 7fac609e0700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c0cd880 0x7fac5c0c8d30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fac5000c450 tx=0x7fac5000c760 comp rx=0 tx=0).stop 2023-08-27T11:13:16.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.932+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 shutdown_connections 2023-08-27T11:13:16.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.932+0000 7fac609e0700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fac3c06a0e0 0x7fac3c06c5a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.932+0000 7fac609e0700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac5c15d340 0x7fac5c1f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.932+0000 7fac609e0700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac5c06a990 0x7fac5c0c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.932+0000 7fac609e0700 1 --2- 172.21.15.45:0/2942089358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac5c0cd880 0x7fac5c0c8d30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:16.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.932+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 >> 172.21.15.45:0/2942089358 conn(0x7fac5c0cc7c0 msgr2=0x7fac5c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:16.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.932+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 shutdown_connections 2023-08-27T11:13:16.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:16.932+0000 7fac609e0700 1 -- 172.21.15.45:0/2942089358 wait complete. 2023-08-27T11:13:16.935 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2023-08-27T11:13:16.944 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-08-27T11:13:16.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:16.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-08-27T11:13:16.989 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:13:16.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:16.990 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-27T11:13:17.047 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:13:17.047 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:17.068 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:13:17.069 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:17.129 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:13:17.130 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:17.189 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.314+0000 7f9da1316700 1 Processor -- start 2023-08-27T11:13:17.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.314+0000 7f9da1316700 1 -- start start 2023-08-27T11:13:17.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9da1316700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c0b8c20 0x7f9d9c0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9da1316700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9d9c0ce2e0 0x7f9d9c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9da1316700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9c15d6a0 con 0x7f9d9c15d2c0 2023-08-27T11:13:17.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9da1316700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9c0ce770 con 0x7f9d9c0ce2e0 2023-08-27T11:13:17.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9da1316700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0d2c90 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9d9a7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c0b8c20 0x7f9d9c0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9d9b7fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9d9c0ce2e0 0x7f9d9c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9d9a7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c0b8c20 0x7f9d9c0cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:51396/0 (socket says 172.21.15.45:51396) 2023-08-27T11:13:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9d9a7fc700 1 -- 172.21.15.45:0/3270465519 learned_addr learned my addr 172.21.15.45:0/3270465519 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9d9a7fc700 1 -- 172.21.15.45:0/3270465519 >> v2:172.21.15.135:3300/0 conn(0x7f9d9c0ce2e0 msgr2=0x7f9d9c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:17.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9d9a7fc700 1 --2- 172.21.15.45:0/3270465519 >> v2:172.21.15.135:3300/0 conn(0x7f9d9c0ce2e0 0x7f9d9c0b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9d9a7fc700 1 -- 172.21.15.45:0/3270465519 >> v1:172.21.15.135:6789/0 conn(0x7f9d9c15d2c0 legacy=0x7f9d9c0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9d9a7fc700 1 -- 172.21.15.45:0/3270465519 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c0d1bf0 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.315+0000 7f9d9b7fe700 1 --2- 172.21.15.45:0/3270465519 >> v2:172.21.15.135:3300/0 conn(0x7f9d9c0ce2e0 0x7f9d9c0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:17.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9d9a7fc700 1 --2- 172.21.15.45:0/3270465519 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c0b8c20 0x7f9d9c0cdda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0098f0 tx=0x7f9d8c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a86c31dd8ccd7c7a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9d99ffb700 1 -- 172.21.15.45:0/3270465519 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c011830 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9d99ffb700 1 -- 172.21.15.45:0/3270465519 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c011990 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9d99ffb700 1 -- 172.21.15.45:0/3270465519 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c011d20 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9da1316700 1 -- 172.21.15.45:0/3270465519 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c0b8c20 msgr2=0x7f9d9c0cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9da1316700 1 --2- 172.21.15.45:0/3270465519 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c0b8c20 0x7f9d9c0cdda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0098f0 tx=0x7f9d8c009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9da1316700 1 -- 172.21.15.45:0/3270465519 shutdown_connections 2023-08-27T11:13:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9da1316700 1 --2- 172.21.15.45:0/3270465519 >> v2:172.21.15.135:3300/0 conn(0x7f9d9c0ce2e0 0x7f9d9c0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9da1316700 1 --2- 172.21.15.45:0/3270465519 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c0b8c20 0x7f9d9c0cdda0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9da1316700 1 -- 172.21.15.45:0/3270465519 >> 172.21.15.45:0/3270465519 conn(0x7f9d9c0d2fd0 msgr2=0x7f9d9c0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.316+0000 7f9da1316700 1 -- 172.21.15.45:0/3270465519 shutdown_connections 2023-08-27T11:13:17.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.317+0000 7f9da1316700 1 -- 172.21.15.45:0/3270465519 wait complete. 2023-08-27T11:13:17.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.318+0000 7f9da1316700 1 Processor -- start 2023-08-27T11:13:17.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.318+0000 7f9da1316700 1 -- start start 2023-08-27T11:13:17.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9da1316700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d9c0b8c20 0x7f9d9c14d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9da1316700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d9c0ce2e0 0x7f9d9c15f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9da1316700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c15d2c0 0x7f9d9c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9da1316700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0b7800 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9da1316700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0ca1d0 con 0x7f9d9c0ce2e0 2023-08-27T11:13:17.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9da1316700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0b7520 con 0x7f9d9c15d2c0 2023-08-27T11:13:17.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d9c0b8c20 0x7f9d9c14d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d9c0b8c20 0x7f9d9c14d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53490/0 (socket says 172.21.15.45:53490) 2023-08-27T11:13:17.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9affd700 1 -- 172.21.15.45:0/1456229351 learned_addr learned my addr 172.21.15.45:0/1456229351 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:17.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9a7fc700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d9c0ce2e0 0x7f9d9c15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9b7fe700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c15d2c0 0x7f9d9c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9affd700 1 -- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c15d2c0 msgr2=0x7f9d9c15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9affd700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c15d2c0 0x7f9d9c15fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9affd700 1 -- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d9c0ce2e0 msgr2=0x7f9d9c15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9affd700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d9c0ce2e0 0x7f9d9c15f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9affd700 1 -- 172.21.15.45:0/1456229351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94009a70 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d9affd700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d9c0b8c20 0x7f9d9c14d7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c490 tx=0x7f9d9000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:17.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d837fe700 1 -- 172.21.15.45:0/1456229351 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900052d0 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d837fe700 1 -- 172.21.15.45:0/1456229351 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d90005450 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c009550 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9d837fe700 1 -- 172.21.15.45:0/1456229351 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90006aa0 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.319+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c0ca750 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.320+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9c04e620 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.321+0000 7f9d837fe700 1 -- 172.21.15.45:0/1456229351 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9d90003680 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.321+0000 7f9d837fe700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d8806a0e0 0x7f9d8806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.321+0000 7f9d837fe700 1 -- 172.21.15.45:0/1456229351 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9d90087950 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.321+0000 7f9d9a7fc700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d8806a0e0 0x7f9d8806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.322+0000 7f9d9a7fc700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d8806a0e0 0x7f9d8806c5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c000c00 tx=0x7f9d8c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:17.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.326+0000 7f9d837fe700 1 -- 172.21.15.45:0/1456229351 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9d90052da0 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.531+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d9c15d6a0 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.532+0000 7f9d837fe700 1 -- 172.21.15.45:0/1456229351 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7f9d9c15d6a0 con 0x7f9d9c0b8c20 2023-08-27T11:13:17.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:17.534 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:17.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d8806a0e0 msgr2=0x7f9d8806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d8806a0e0 0x7f9d8806c5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c000c00 tx=0x7f9d8c008030 comp rx=0 tx=0).stop 2023-08-27T11:13:17.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d9c0b8c20 msgr2=0x7f9d9c14d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d9c0b8c20 0x7f9d9c14d7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c490 tx=0x7f9d9000c850 comp rx=0 tx=0).stop 2023-08-27T11:13:17.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 shutdown_connections 2023-08-27T11:13:17.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d9c0b8c20 0x7f9d9c14d7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f9d8806a0e0 0x7f9d8806c5a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9d9c15d2c0 0x7f9d9c15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 --2- 172.21.15.45:0/1456229351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d9c0ce2e0 0x7f9d9c15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 >> 172.21.15.45:0/1456229351 conn(0x7f9d9c0d2fd0 msgr2=0x7f9d9c153db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:17.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 shutdown_connections 2023-08-27T11:13:17.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.534+0000 7f9da1316700 1 -- 172.21.15.45:0/1456229351 wait complete. 2023-08-27T11:13:17.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:13:17.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:13:17.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.676+0000 7f4dee327700 1 Processor -- start 2023-08-27T11:13:17.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.676+0000 7f4dee327700 1 -- start start 2023-08-27T11:13:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.677+0000 7f4dee327700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4de815d340 0x7f4de8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.677+0000 7f4dee327700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4de806a990 0x7f4de80b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.677+0000 7f4dee327700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de806b160 con 0x7f4de80cd880 2023-08-27T11:13:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.677+0000 7f4dee327700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de8072010 con 0x7f4de815d340 2023-08-27T11:13:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.677+0000 7f4dee327700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de80af590 con 0x7f4de806a990 2023-08-27T11:13:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.677+0000 7f4de7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4de815d340 0x7f4de8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.677+0000 7f4de77fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4de80cd880 0x7f4de806a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40274/0 (socket says 172.21.15.45:40274) 2023-08-27T11:13:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.677+0000 7f4de77fe700 1 -- 172.21.15.45:0/183869490 learned_addr learned my addr 172.21.15.45:0/183869490 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4dec8c4700 1 --2- 172.21.15.45:0/183869490 >> v2:172.21.15.135:3300/0 conn(0x7f4de806a990 0x7f4de80b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4de7fff700 1 -- 172.21.15.45:0/183869490 >> v2:172.21.15.135:3300/0 conn(0x7f4de806a990 msgr2=0x7f4de80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4de7fff700 1 --2- 172.21.15.45:0/183869490 >> v2:172.21.15.135:3300/0 conn(0x7f4de806a990 0x7f4de80b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4de7fff700 1 -- 172.21.15.45:0/183869490 >> v1:172.21.15.45:6789/0 conn(0x7f4de80cd880 legacy=0x7f4de806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4de7fff700 1 -- 172.21.15.45:0/183869490 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de80af2b0 con 0x7f4de815d340 2023-08-27T11:13:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4de6ffd700 1 -- 172.21.15.45:0/183869490 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976442160 0 0) 0x7f4de806b160 con 0x7f4de80cd880 2023-08-27T11:13:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4de7fff700 1 --2- 172.21.15.45:0/183869490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de815d340 0x7f4de8160d00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4de8072af0 tx=0x7f4dd8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62dab4687dc6488 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4de6ffd700 1 -- 172.21.15.45:0/183869490 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8011830 con 0x7f4de815d340 2023-08-27T11:13:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4de6ffd700 1 -- 172.21.15.45:0/183869490 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd8011990 con 0x7f4de815d340 2023-08-27T11:13:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4de6ffd700 1 -- 172.21.15.45:0/183869490 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8011d20 con 0x7f4de815d340 2023-08-27T11:13:17.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4dee327700 1 -- 172.21.15.45:0/183869490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de815d340 msgr2=0x7f4de8160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.678+0000 7f4dee327700 1 --2- 172.21.15.45:0/183869490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de815d340 0x7f4de8160d00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4de8072af0 tx=0x7f4dd8009b20 comp rx=0 tx=0).stop 2023-08-27T11:13:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.679+0000 7f4dee327700 1 -- 172.21.15.45:0/183869490 shutdown_connections 2023-08-27T11:13:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.679+0000 7f4dee327700 1 --2- 172.21.15.45:0/183869490 >> v2:172.21.15.135:3300/0 conn(0x7f4de806a990 0x7f4de80b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.679+0000 7f4dee327700 1 --2- 172.21.15.45:0/183869490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de815d340 0x7f4de8160d00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.679+0000 7f4dee327700 1 -- 172.21.15.45:0/183869490 >> 172.21.15.45:0/183869490 conn(0x7f4de80cc7c0 msgr2=0x7f4de80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.679+0000 7f4dee327700 1 -- 172.21.15.45:0/183869490 shutdown_connections 2023-08-27T11:13:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.679+0000 7f4dee327700 1 -- 172.21.15.45:0/183869490 wait complete. 2023-08-27T11:13:17.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.680+0000 7f4dee327700 1 Processor -- start 2023-08-27T11:13:17.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.680+0000 7f4dee327700 1 -- start start 2023-08-27T11:13:17.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4dee327700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4de806a990 0x7f4de81f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4dee327700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de80cd880 0x7f4de815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4dee327700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4de815d340 0x7f4de8151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4dee327700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80cb040 con 0x7f4de80cd880 2023-08-27T11:13:17.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4dee327700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81f3f00 con 0x7f4de806a990 2023-08-27T11:13:17.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4dee327700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80ca3c0 con 0x7f4de815d340 2023-08-27T11:13:17.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de80cd880 0x7f4de815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de80cd880 0x7f4de815a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53506/0 (socket says 172.21.15.45:53506) 2023-08-27T11:13:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de77fe700 1 -- 172.21.15.45:0/3601830915 learned_addr learned my addr 172.21.15.45:0/3601830915 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4dec8c4700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4de815d340 0x7f4de8151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de7fff700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4de806a990 0x7f4de81f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de77fe700 1 -- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4de815d340 msgr2=0x7f4de8151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de77fe700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4de815d340 0x7f4de8151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de77fe700 1 -- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4de806a990 msgr2=0x7f4de81f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de77fe700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4de806a990 0x7f4de81f39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de77fe700 1 -- 172.21.15.45:0/3601830915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80094a0 con 0x7f4de80cd880 2023-08-27T11:13:17.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de7fff700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4de806a990 0x7f4de81f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:17.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.681+0000 7f4de77fe700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de80cd880 0x7f4de815a1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c550 tx=0x7f4dd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:17.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.682+0000 7f4de4ff9700 1 -- 172.21.15.45:0/3601830915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0004c80 con 0x7f4de80cd880 2023-08-27T11:13:17.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.682+0000 7f4de4ff9700 1 -- 172.21.15.45:0/3601830915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd0004e00 con 0x7f4de80cd880 2023-08-27T11:13:17.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.682+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de80ca6a0 con 0x7f4de80cd880 2023-08-27T11:13:17.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.682+0000 7f4de4ff9700 1 -- 172.21.15.45:0/3601830915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0007310 con 0x7f4de80cd880 2023-08-27T11:13:17.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.682+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de80cac60 con 0x7f4de80cd880 2023-08-27T11:13:17.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.683+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de80cdf30 con 0x7f4de80cd880 2023-08-27T11:13:17.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.683+0000 7f4de4ff9700 1 -- 172.21.15.45:0/3601830915 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4dd0007470 con 0x7f4de80cd880 2023-08-27T11:13:17.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.683+0000 7f4de4ff9700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4dc806a0e0 0x7f4dc806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:17.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.683+0000 7f4de4ff9700 1 -- 172.21.15.45:0/3601830915 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4dd0088590 con 0x7f4de80cd880 2023-08-27T11:13:17.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.684+0000 7f4de7fff700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4dc806a0e0 0x7f4dc806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:17.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.684+0000 7f4de7fff700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4dc806a0e0 0x7f4dc806c5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80090b0 tx=0x7f4dd801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:17.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.688+0000 7f4de4ff9700 1 -- 172.21.15.45:0/3601830915 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4dd0053940 con 0x7f4de80cd880 2023-08-27T11:13:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.889+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4de80734a0 con 0x7f4de80cd880 2023-08-27T11:13:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.889+0000 7f4de4ff9700 1 -- 172.21.15.45:0/3601830915 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4dd0056f60 con 0x7f4de80cd880 2023-08-27T11:13:17.891 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:13:17.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4dc806a0e0 msgr2=0x7f4dc806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4dc806a0e0 0x7f4dc806c5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80090b0 tx=0x7f4dd801c040 comp rx=0 tx=0).stop 2023-08-27T11:13:17.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de80cd880 msgr2=0x7f4de815a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:17.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de80cd880 0x7f4de815a1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c550 tx=0x7f4dd000c910 comp rx=0 tx=0).stop 2023-08-27T11:13:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 shutdown_connections 2023-08-27T11:13:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4dc806a0e0 0x7f4dc806c5a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4de815d340 0x7f4de8151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4de806a990 0x7f4de81f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 --2- 172.21.15.45:0/3601830915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4de80cd880 0x7f4de815a1a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:17.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 >> 172.21.15.45:0/3601830915 conn(0x7f4de80cc7c0 msgr2=0x7f4de815bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:17.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 shutdown_connections 2023-08-27T11:13:17.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:17.892+0000 7f4dee327700 1 -- 172.21.15.45:0/3601830915 wait complete. 2023-08-27T11:13:17.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:13:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.032+0000 7fe189030700 1 Processor -- start 2023-08-27T11:13:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.032+0000 7fe189030700 1 -- start start 2023-08-27T11:13:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.033+0000 7fe189030700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe18415a2c0 0x7fe184073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.033+0000 7fe189030700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1840b2f60 0x7fe18406b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.033+0000 7fe189030700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe184073d40 con 0x7fe18406b890 2023-08-27T11:13:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.033+0000 7fe189030700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1840cf360 con 0x7fe1840b2f60 2023-08-27T11:13:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.033+0000 7fe189030700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18415a6a0 con 0x7fe18415a2c0 2023-08-27T11:13:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.033+0000 7fe18259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1840b2f60 0x7fe18406b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.033+0000 7fe18359e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe18406b890 0x7fe18406bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40292/0 (socket says 172.21.15.45:40292) 2023-08-27T11:13:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.033+0000 7fe18359e700 1 -- 172.21.15.45:0/3246478746 learned_addr learned my addr 172.21.15.45:0/3246478746 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe182d9d700 1 --2- 172.21.15.45:0/3246478746 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe18415a2c0 0x7fe184073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe18259c700 1 -- 172.21.15.45:0/3246478746 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe18415a2c0 msgr2=0x7fe184073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe18259c700 1 --2- 172.21.15.45:0/3246478746 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe18415a2c0 0x7fe184073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe18259c700 1 -- 172.21.15.45:0/3246478746 >> v1:172.21.15.45:6789/0 conn(0x7fe18406b890 legacy=0x7fe18406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe18259c700 1 -- 172.21.15.45:0/3246478746 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1840ceda0 con 0x7fe1840b2f60 2023-08-27T11:13:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe181d9b700 1 -- 172.21.15.45:0/3246478746 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242768604 0 0) 0x7fe184073d40 con 0x7fe18406b890 2023-08-27T11:13:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe18259c700 1 --2- 172.21.15.45:0/3246478746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1840b2f60 0x7fe18406b350 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009840 tx=0x7fe16c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56d3f53f8ffa548 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe181d9b700 1 -- 172.21.15.45:0/3246478746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c011830 con 0x7fe1840b2f60 2023-08-27T11:13:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe181d9b700 1 -- 172.21.15.45:0/3246478746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c011990 con 0x7fe1840b2f60 2023-08-27T11:13:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.034+0000 7fe181d9b700 1 -- 172.21.15.45:0/3246478746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c011d20 con 0x7fe1840b2f60 2023-08-27T11:13:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.035+0000 7fe189030700 1 -- 172.21.15.45:0/3246478746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1840b2f60 msgr2=0x7fe18406b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.035+0000 7fe189030700 1 --2- 172.21.15.45:0/3246478746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1840b2f60 0x7fe18406b350 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009840 tx=0x7fe16c009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.035+0000 7fe189030700 1 -- 172.21.15.45:0/3246478746 shutdown_connections 2023-08-27T11:13:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.035+0000 7fe189030700 1 --2- 172.21.15.45:0/3246478746 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe18415a2c0 0x7fe184073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.035+0000 7fe189030700 1 --2- 172.21.15.45:0/3246478746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1840b2f60 0x7fe18406b350 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.035+0000 7fe189030700 1 -- 172.21.15.45:0/3246478746 >> 172.21.15.45:0/3246478746 conn(0x7fe1840b2440 msgr2=0x7fe1840b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.035+0000 7fe189030700 1 -- 172.21.15.45:0/3246478746 shutdown_connections 2023-08-27T11:13:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.035+0000 7fe189030700 1 -- 172.21.15.45:0/3246478746 wait complete. 2023-08-27T11:13:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.037+0000 7fe189030700 1 Processor -- start 2023-08-27T11:13:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.037+0000 7fe189030700 1 -- start start 2023-08-27T11:13:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.037+0000 7fe189030700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18406b890 0x7fe18415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe189030700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1840b2f60 0x7fe1840ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe189030700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe18415a2c0 0x7fe1841f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe189030700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18415d800 con 0x7fe18406b890 2023-08-27T11:13:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe189030700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18415ad30 con 0x7fe18415a2c0 2023-08-27T11:13:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe189030700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1840c96a0 con 0x7fe1840b2f60 2023-08-27T11:13:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe182d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18406b890 0x7fe18415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe182d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18406b890 0x7fe18415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53518/0 (socket says 172.21.15.45:53518) 2023-08-27T11:13:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe182d9d700 1 -- 172.21.15.45:0/3353522314 learned_addr learned my addr 172.21.15.45:0/3353522314 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe18259c700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1840b2f60 0x7fe1840ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe18359e700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe18415a2c0 0x7fe1841f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe182d9d700 1 -- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1840b2f60 msgr2=0x7fe1840ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe182d9d700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1840b2f60 0x7fe1840ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe182d9d700 1 -- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe18415a2c0 msgr2=0x7fe1841f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe182d9d700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe18415a2c0 0x7fe1841f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe182d9d700 1 -- 172.21.15.45:0/3353522314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0094a0 con 0x7fe18406b890 2023-08-27T11:13:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe18259c700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1840b2f60 0x7fe1840ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe182d9d700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18406b890 0x7fe18415a7f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe17400d7d0 tx=0x7fe17400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe17b7fe700 1 -- 172.21.15.45:0/3353522314 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740125f0 con 0x7fe18406b890 2023-08-27T11:13:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe17b7fe700 1 -- 172.21.15.45:0/3353522314 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe174012750 con 0x7fe18406b890 2023-08-27T11:13:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18406cb00 con 0x7fe18406b890 2023-08-27T11:13:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe17b7fe700 1 -- 172.21.15.45:0/3353522314 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174012b00 con 0x7fe18406b890 2023-08-27T11:13:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.038+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18406d050 con 0x7fe18406b890 2023-08-27T11:13:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.039+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18409d4b0 con 0x7fe18406b890 2023-08-27T11:13:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.040+0000 7fe17b7fe700 1 -- 172.21.15.45:0/3353522314 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe174012c60 con 0x7fe18406b890 2023-08-27T11:13:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.040+0000 7fe17b7fe700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe16406a0e0 0x7fe16406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.040+0000 7fe17b7fe700 1 -- 172.21.15.45:0/3353522314 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe174088970 con 0x7fe18406b890 2023-08-27T11:13:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.040+0000 7fe18259c700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe16406a0e0 0x7fe16406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.041+0000 7fe18259c700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe16406a0e0 0x7fe16406c5a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0090b0 tx=0x7fe16c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.045+0000 7fe17b7fe700 1 -- 172.21.15.45:0/3353522314 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe174053d30 con 0x7fe18406b890 2023-08-27T11:13:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.244+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe184073b80 con 0x7fe18406b890 2023-08-27T11:13:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.244+0000 7fe17b7fe700 1 -- 172.21.15.45:0/3353522314 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe174057350 con 0x7fe18406b890 2023-08-27T11:13:18.246 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:13:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe16406a0e0 msgr2=0x7fe16406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe16406a0e0 0x7fe16406c5a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0090b0 tx=0x7fe16c008030 comp rx=0 tx=0).stop 2023-08-27T11:13:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18406b890 msgr2=0x7fe18415a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18406b890 0x7fe18415a7f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe17400d7d0 tx=0x7fe17400db90 comp rx=0 tx=0).stop 2023-08-27T11:13:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 shutdown_connections 2023-08-27T11:13:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18406b890 0x7fe18415a7f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe16406a0e0 0x7fe16406c5a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe18415a2c0 0x7fe1841f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 --2- 172.21.15.45:0/3353522314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1840b2f60 0x7fe1840ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 >> 172.21.15.45:0/3353522314 conn(0x7fe1840b2440 msgr2=0x7fe1840cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 shutdown_connections 2023-08-27T11:13:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.246+0000 7fe189030700 1 -- 172.21.15.45:0/3353522314 wait complete. 2023-08-27T11:13:18.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:13:18.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.388+0000 7fab2c9d7700 1 Processor -- start 2023-08-27T11:13:18.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.388+0000 7fab2c9d7700 1 -- start start 2023-08-27T11:13:18.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab2c9d7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab280ce4b0 0x7fab28167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab2c9d7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2815d6a0 con 0x7fab2815d2c0 2023-08-27T11:13:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab2c9d7700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab280d2c90 con 0x7fab280b8c20 2023-08-27T11:13:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab2c9d7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab280d1bf0 con 0x7fab280ce4b0 2023-08-27T11:13:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab26d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab280ce4b0 0x7fab28167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab2659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fab2815d2c0 0x7fab280d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40302/0 (socket says 172.21.15.45:40302) 2023-08-27T11:13:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab2659c700 1 -- 172.21.15.45:0/2271539803 learned_addr learned my addr 172.21.15.45:0/2271539803 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab26d9d700 1 -- 172.21.15.45:0/2271539803 >> v1:172.21.15.135:6789/0 conn(0x7fab280b8c20 legacy=0x7fab280cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab26d9d700 1 -- 172.21.15.45:0/2271539803 >> v1:172.21.15.45:6789/0 conn(0x7fab2815d2c0 legacy=0x7fab280d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab2559a700 1 -- 172.21.15.45:0/2271539803 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555970361 0 0) 0x7fab1c001110 con 0x7fab2815d2c0 2023-08-27T11:13:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.389+0000 7fab26d9d700 1 -- 172.21.15.45:0/2271539803 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab280d1ed0 con 0x7fab280ce4b0 2023-08-27T11:13:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab26d9d700 1 --2- 172.21.15.45:0/2271539803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280ce4b0 0x7fab28167d80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fab20009dc0 tx=0x7fab2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e38950ef0adcf0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2559a700 1 -- 172.21.15.45:0/2271539803 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2000e070 con 0x7fab280ce4b0 2023-08-27T11:13:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2559a700 1 -- 172.21.15.45:0/2271539803 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2000bb90 con 0x7fab280ce4b0 2023-08-27T11:13:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2559a700 1 -- 172.21.15.45:0/2271539803 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20012430 con 0x7fab280ce4b0 2023-08-27T11:13:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2c9d7700 1 -- 172.21.15.45:0/2271539803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280ce4b0 msgr2=0x7fab28167d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2c9d7700 1 --2- 172.21.15.45:0/2271539803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280ce4b0 0x7fab28167d80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fab20009dc0 tx=0x7fab2000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2c9d7700 1 -- 172.21.15.45:0/2271539803 shutdown_connections 2023-08-27T11:13:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2c9d7700 1 --2- 172.21.15.45:0/2271539803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280ce4b0 0x7fab28167d80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2c9d7700 1 -- 172.21.15.45:0/2271539803 >> 172.21.15.45:0/2271539803 conn(0x7fab280d2fd0 msgr2=0x7fab280d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2c9d7700 1 -- 172.21.15.45:0/2271539803 shutdown_connections 2023-08-27T11:13:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.390+0000 7fab2c9d7700 1 -- 172.21.15.45:0/2271539803 wait complete. 2023-08-27T11:13:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.392+0000 7fab2c9d7700 1 Processor -- start 2023-08-27T11:13:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.392+0000 7fab2c9d7700 1 -- start start 2023-08-27T11:13:18.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.392+0000 7fab2c9d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280b8c20 0x7fab2815fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.392+0000 7fab2c9d7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab280ce4b0 0x7fab2814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.392+0000 7fab2c9d7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab2815d2c0 0x7fab280b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.392+0000 7fab2c9d7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2815ee40 con 0x7fab280b8c20 2023-08-27T11:13:18.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.392+0000 7fab2c9d7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2814e1c0 con 0x7fab280ce4b0 2023-08-27T11:13:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.392+0000 7fab2c9d7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab280b7050 con 0x7fab2815d2c0 2023-08-27T11:13:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280b8c20 0x7fab2815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280b8c20 0x7fab2815fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53532/0 (socket says 172.21.15.45:53532) 2023-08-27T11:13:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2659c700 1 -- 172.21.15.45:0/3115050932 learned_addr learned my addr 172.21.15.45:0/3115050932 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:18.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2659c700 1 -- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab2815d2c0 msgr2=0x7fab280b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2659c700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab2815d2c0 0x7fab280b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2659c700 1 -- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab280ce4b0 msgr2=0x7fab2814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2659c700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab280ce4b0 0x7fab2814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2659c700 1 -- 172.21.15.45:0/3115050932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab20009a70 con 0x7fab280b8c20 2023-08-27T11:13:18.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2659c700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280b8c20 0x7fab2815fb70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c550 tx=0x7fab1c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab16ffd700 1 -- 172.21.15.45:0/3115050932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c004c80 con 0x7fab280b8c20 2023-08-27T11:13:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab16ffd700 1 -- 172.21.15.45:0/3115050932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c004de0 con 0x7fab280b8c20 2023-08-27T11:13:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab280b7c50 con 0x7fab280b8c20 2023-08-27T11:13:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab16ffd700 1 -- 172.21.15.45:0/3115050932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c007310 con 0x7fab280b8c20 2023-08-27T11:13:18.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.393+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2815f370 con 0x7fab280b8c20 2023-08-27T11:13:18.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.394+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2804e620 con 0x7fab280b8c20 2023-08-27T11:13:18.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.395+0000 7fab16ffd700 1 -- 172.21.15.45:0/3115050932 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fab1c007470 con 0x7fab280b8c20 2023-08-27T11:13:18.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.395+0000 7fab16ffd700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fab0806a110 0x7fab0806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.395+0000 7fab16ffd700 1 -- 172.21.15.45:0/3115050932 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fab1c088380 con 0x7fab280b8c20 2023-08-27T11:13:18.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.395+0000 7fab25d9b700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fab0806a110 0x7fab0806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.396+0000 7fab25d9b700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fab0806a110 0x7fab0806c5d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab18000f60 tx=0x7fab18007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:18.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.399+0000 7fab16ffd700 1 -- 172.21.15.45:0/3115050932 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fab1c053740 con 0x7fab280b8c20 2023-08-27T11:13:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.599+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab2815a060 con 0x7fab280b8c20 2023-08-27T11:13:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.599+0000 7fab16ffd700 1 -- 172.21.15.45:0/3115050932 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab1c056d60 con 0x7fab280b8c20 2023-08-27T11:13:18.601 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fab0806a110 msgr2=0x7fab0806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fab0806a110 0x7fab0806c5d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab18000f60 tx=0x7fab18007a40 comp rx=0 tx=0).stop 2023-08-27T11:13:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280b8c20 msgr2=0x7fab2815fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280b8c20 0x7fab2815fb70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c550 tx=0x7fab1c00c910 comp rx=0 tx=0).stop 2023-08-27T11:13:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 shutdown_connections 2023-08-27T11:13:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab280b8c20 0x7fab2815fb70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fab0806a110 0x7fab0806c5d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab2815d2c0 0x7fab280b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 --2- 172.21.15.45:0/3115050932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab280ce4b0 0x7fab2814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 >> 172.21.15.45:0/3115050932 conn(0x7fab280d2fd0 msgr2=0x7fab2815b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 shutdown_connections 2023-08-27T11:13:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.601+0000 7fab2c9d7700 1 -- 172.21.15.45:0/3115050932 wait complete. 2023-08-27T11:13:18.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:13:18.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.741+0000 7f263f858700 1 Processor -- start 2023-08-27T11:13:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.742+0000 7f263f858700 1 -- start start 2023-08-27T11:13:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.742+0000 7f263f858700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 0x7f26380cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.742+0000 7f263f858700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380cdf40 con 0x7f2638160890 2023-08-27T11:13:18.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.742+0000 7f263f858700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380d0fd0 con 0x7f26380b5220 2023-08-27T11:13:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.742+0000 7f263f858700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380ce190 con 0x7f26380b62c0 2023-08-27T11:13:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.742+0000 7f263d5f4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2638160890 0x7f26380b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40316/0 (socket says 172.21.15.45:40316) 2023-08-27T11:13:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.742+0000 7f263d5f4700 1 -- 172.21.15.45:0/1953057812 learned_addr learned my addr 172.21.15.45:0/1953057812 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f263ddf5700 1 --2- 172.21.15.45:0/1953057812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 0x7f26380cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f262ffff700 1 -- 172.21.15.45:0/1953057812 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132549962 0 0) 0x7f26380cdf40 con 0x7f2638160890 2023-08-27T11:13:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f262ffff700 1 -- 172.21.15.45:0/1953057812 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26240035b0 con 0x7f2638160890 2023-08-27T11:13:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f262ffff700 1 -- 172.21.15.45:0/1953057812 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426546313 0 0) 0x7f26380d0fd0 con 0x7f26380b5220 2023-08-27T11:13:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f262ffff700 1 -- 172.21.15.45:0/1953057812 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26380cdf40 con 0x7f26380b5220 2023-08-27T11:13:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f263ddf5700 1 -- 172.21.15.45:0/1953057812 >> v1:172.21.15.135:6789/0 conn(0x7f26380b5220 legacy=0x7f26380d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f263ddf5700 1 -- 172.21.15.45:0/1953057812 >> v1:172.21.15.45:6789/0 conn(0x7f2638160890 legacy=0x7f26380b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f262ffff700 1 -- 172.21.15.45:0/1953057812 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89227904 0 0) 0x7f26240035b0 con 0x7f2638160890 2023-08-27T11:13:18.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f263ddf5700 1 -- 172.21.15.45:0/1953057812 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380ce470 con 0x7f26380b62c0 2023-08-27T11:13:18.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f263ddf5700 1 --2- 172.21.15.45:0/1953057812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 0x7f26380cd8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2630009dc0 tx=0x7f263000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b522a5c516d9fdb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f262ffff700 1 -- 172.21.15.45:0/1953057812 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263000e070 con 0x7f26380b62c0 2023-08-27T11:13:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f262ffff700 1 -- 172.21.15.45:0/1953057812 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263000bb20 con 0x7f26380b62c0 2023-08-27T11:13:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.743+0000 7f262ffff700 1 -- 172.21.15.45:0/1953057812 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263000be90 con 0x7f26380b62c0 2023-08-27T11:13:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.744+0000 7f263f858700 1 -- 172.21.15.45:0/1953057812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 msgr2=0x7f26380cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.744+0000 7f263f858700 1 --2- 172.21.15.45:0/1953057812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 0x7f26380cd8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2630009dc0 tx=0x7f263000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.744+0000 7f263f858700 1 -- 172.21.15.45:0/1953057812 shutdown_connections 2023-08-27T11:13:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.744+0000 7f263f858700 1 --2- 172.21.15.45:0/1953057812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 0x7f26380cd8f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.744+0000 7f263f858700 1 -- 172.21.15.45:0/1953057812 >> 172.21.15.45:0/1953057812 conn(0x7f26380cfb10 msgr2=0x7f26380cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.744+0000 7f263f858700 1 -- 172.21.15.45:0/1953057812 shutdown_connections 2023-08-27T11:13:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.744+0000 7f263f858700 1 -- 172.21.15.45:0/1953057812 wait complete. 2023-08-27T11:13:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.745+0000 7f263f858700 1 Processor -- start 2023-08-27T11:13:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.745+0000 7f263f858700 1 -- start start 2023-08-27T11:13:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263f858700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26380b5220 0x7f26380cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263f858700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 0x7f26381f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263f858700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2638160890 0x7f26381f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263f858700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2638149c70 con 0x7f2638160890 2023-08-27T11:13:18.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263f858700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26381498b0 con 0x7f26380b5220 2023-08-27T11:13:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263f858700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380cc480 con 0x7f26380b62c0 2023-08-27T11:13:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263ddf5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2638160890 0x7f26381f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263ddf5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2638160890 0x7f26381f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53550/0 (socket says 172.21.15.45:53550) 2023-08-27T11:13:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263ddf5700 1 -- 172.21.15.45:0/311505716 learned_addr learned my addr 172.21.15.45:0/311505716 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.746+0000 7f263cdf3700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 0x7f26381f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f263d5f4700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26380b5220 0x7f26380cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f263ddf5700 1 -- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 msgr2=0x7f26381f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f263ddf5700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 0x7f26381f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f263ddf5700 1 -- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26380b5220 msgr2=0x7f26380cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f263ddf5700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26380b5220 0x7f26380cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f263ddf5700 1 -- 172.21.15.45:0/311505716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2630009a70 con 0x7f2638160890 2023-08-27T11:13:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f263ddf5700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2638160890 0x7f26381f6530 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2630013000 tx=0x7f2630004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f262dffb700 1 -- 172.21.15.45:0/311505716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630004810 con 0x7f2638160890 2023-08-27T11:13:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f262dffb700 1 -- 172.21.15.45:0/311505716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2630021070 con 0x7f2638160890 2023-08-27T11:13:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26380cc760 con 0x7f2638160890 2023-08-27T11:13:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f262dffb700 1 -- 172.21.15.45:0/311505716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630026670 con 0x7f2638160890 2023-08-27T11:13:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.747+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26380cc0e0 con 0x7f2638160890 2023-08-27T11:13:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.748+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2638149db0 con 0x7f2638160890 2023-08-27T11:13:18.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.748+0000 7f262dffb700 1 -- 172.21.15.45:0/311505716 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2630023030 con 0x7f2638160890 2023-08-27T11:13:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.749+0000 7f262dffb700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f261c06a0e0 0x7f261c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.749+0000 7f262dffb700 1 -- 172.21.15.45:0/311505716 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f263008a6a0 con 0x7f2638160890 2023-08-27T11:13:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.749+0000 7f263d5f4700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f261c06a0e0 0x7f261c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:18.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.750+0000 7f263d5f4700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f261c06a0e0 0x7f261c06c5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2634007fd0 tx=0x7f2634009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:18.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.753+0000 7f262dffb700 1 -- 172.21.15.45:0/311505716 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2638149db0 con 0x7f2638160890 2023-08-27T11:13:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.952+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2638149db0 con 0x7f2638160890 2023-08-27T11:13:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.952+0000 7f262dffb700 1 -- 172.21.15.45:0/311505716 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2638149db0 con 0x7f2638160890 2023-08-27T11:13:18.953 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:13:18.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f261c06a0e0 msgr2=0x7f261c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f261c06a0e0 0x7f261c06c5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2634007fd0 tx=0x7f2634009040 comp rx=0 tx=0).stop 2023-08-27T11:13:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2638160890 msgr2=0x7f26381f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2638160890 0x7f26381f6530 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2630013000 tx=0x7f2630004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 shutdown_connections 2023-08-27T11:13:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2638160890 0x7f26381f6530 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f261c06a0e0 0x7f261c06c5a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26380b5220 0x7f26380cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 --2- 172.21.15.45:0/311505716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f26380b62c0 0x7f26381f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 >> 172.21.15.45:0/311505716 conn(0x7f26380cfb10 msgr2=0x7f26380c6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 shutdown_connections 2023-08-27T11:13:18.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:18.954+0000 7f263f858700 1 -- 172.21.15.45:0/311505716 wait complete. 2023-08-27T11:13:18.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:13:19.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.095+0000 7fdc6586e700 1 Processor -- start 2023-08-27T11:13:19.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.095+0000 7fdc6586e700 1 -- start start 2023-08-27T11:13:19.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc6586e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdc6015af30 0x7fdc600cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc6586e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc600cde80 0x7fdc60164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc6586e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc600cb8b0 con 0x7fdc600cf960 2023-08-27T11:13:19.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc6586e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6006f9c0 con 0x7fdc600cde80 2023-08-27T11:13:19.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc6586e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc600cecd0 con 0x7fdc6015af30 2023-08-27T11:13:19.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc5f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc600cde80 0x7fdc60164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc5f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc600cde80 0x7fdc60164fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53558/0 (socket says 172.21.15.45:53558) 2023-08-27T11:13:19.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc5f7fe700 1 -- 172.21.15.45:0/1320917984 learned_addr learned my addr 172.21.15.45:0/1320917984 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:19.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc5effd700 1 --2- 172.21.15.45:0/1320917984 >> v2:172.21.15.135:3300/0 conn(0x7fdc6015af30 0x7fdc600cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc5f7fe700 1 -- 172.21.15.45:0/1320917984 >> v2:172.21.15.135:3300/0 conn(0x7fdc6015af30 msgr2=0x7fdc600cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:19.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc5f7fe700 1 --2- 172.21.15.45:0/1320917984 >> v2:172.21.15.135:3300/0 conn(0x7fdc6015af30 0x7fdc600cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc5f7fe700 1 -- 172.21.15.45:0/1320917984 >> v1:172.21.15.135:6789/0 conn(0x7fdc600cf960 legacy=0x7fdc600cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.096+0000 7fdc5f7fe700 1 -- 172.21.15.45:0/1320917984 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc600cbae0 con 0x7fdc600cde80 2023-08-27T11:13:19.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc5f7fe700 1 --2- 172.21.15.45:0/1320917984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600cde80 0x7fdc60164fe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdc6009a8d0 tx=0x7fdc5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d114510f555dc3ef server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:19.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc5dffb700 1 -- 172.21.15.45:0/1320917984 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5800e070 con 0x7fdc600cde80 2023-08-27T11:13:19.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc5dffb700 1 -- 172.21.15.45:0/1320917984 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5800b910 con 0x7fdc600cde80 2023-08-27T11:13:19.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc5dffb700 1 -- 172.21.15.45:0/1320917984 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5800bcc0 con 0x7fdc600cde80 2023-08-27T11:13:19.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc6586e700 1 -- 172.21.15.45:0/1320917984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600cde80 msgr2=0x7fdc60164fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc6586e700 1 --2- 172.21.15.45:0/1320917984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600cde80 0x7fdc60164fe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdc6009a8d0 tx=0x7fdc5800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:19.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc6586e700 1 -- 172.21.15.45:0/1320917984 shutdown_connections 2023-08-27T11:13:19.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc6586e700 1 --2- 172.21.15.45:0/1320917984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc600cde80 0x7fdc60164fe0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc6586e700 1 --2- 172.21.15.45:0/1320917984 >> v2:172.21.15.135:3300/0 conn(0x7fdc6015af30 0x7fdc600cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc6586e700 1 -- 172.21.15.45:0/1320917984 >> 172.21.15.45:0/1320917984 conn(0x7fdc6009a1a0 msgr2=0x7fdc600d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:19.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc6586e700 1 -- 172.21.15.45:0/1320917984 shutdown_connections 2023-08-27T11:13:19.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.097+0000 7fdc6586e700 1 -- 172.21.15.45:0/1320917984 wait complete. 2023-08-27T11:13:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.099+0000 7fdc6586e700 1 Processor -- start 2023-08-27T11:13:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.099+0000 7fdc6586e700 1 -- start start 2023-08-27T11:13:19.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc6586e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc600cde80 0x7fdc600b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc6586e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc600cf960 0x7fdc60155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc6586e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6015af30 0x7fdc601f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc6586e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601f1a40 con 0x7fdc6015af30 2023-08-27T11:13:19.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc6586e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601f17f0 con 0x7fdc600cde80 2023-08-27T11:13:19.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc6586e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600b6070 con 0x7fdc600cf960 2023-08-27T11:13:19.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6015af30 0x7fdc601f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6015af30 0x7fdc601f12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53564/0 (socket says 172.21.15.45:53564) 2023-08-27T11:13:19.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5f7fe700 1 -- 172.21.15.45:0/2824068213 learned_addr learned my addr 172.21.15.45:0/2824068213 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:19.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5e7fc700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc600cf960 0x7fdc60155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5effd700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc600cde80 0x7fdc600b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5f7fe700 1 -- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc600cf960 msgr2=0x7fdc60155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5f7fe700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc600cf960 0x7fdc60155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5f7fe700 1 -- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc600cde80 msgr2=0x7fdc600b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5f7fe700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc600cde80 0x7fdc600b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5f7fe700 1 -- 172.21.15.45:0/2824068213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc58009b30 con 0x7fdc6015af30 2023-08-27T11:13:19.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.100+0000 7fdc5e7fc700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc600cf960 0x7fdc60155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:19.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.101+0000 7fdc5f7fe700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6015af30 0x7fdc601f12b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdc58012000 tx=0x7fdc58004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:19.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.101+0000 7fdc47fff700 1 -- 172.21.15.45:0/2824068213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc580099b0 con 0x7fdc6015af30 2023-08-27T11:13:19.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.101+0000 7fdc47fff700 1 -- 172.21.15.45:0/2824068213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc580092e0 con 0x7fdc6015af30 2023-08-27T11:13:19.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.101+0000 7fdc47fff700 1 -- 172.21.15.45:0/2824068213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5801c640 con 0x7fdc6015af30 2023-08-27T11:13:19.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.101+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc600b6350 con 0x7fdc6015af30 2023-08-27T11:13:19.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.101+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6015cb60 con 0x7fdc6015af30 2023-08-27T11:13:19.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.102+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc600734a0 con 0x7fdc6015af30 2023-08-27T11:13:19.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.102+0000 7fdc47fff700 1 -- 172.21.15.45:0/2824068213 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdc5801c7a0 con 0x7fdc6015af30 2023-08-27T11:13:19.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.103+0000 7fdc47fff700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc4c06a0e0 0x7fdc4c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.103+0000 7fdc47fff700 1 -- 172.21.15.45:0/2824068213 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdc58089f20 con 0x7fdc6015af30 2023-08-27T11:13:19.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.103+0000 7fdc5effd700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc4c06a0e0 0x7fdc4c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.103+0000 7fdc5effd700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc4c06a0e0 0x7fdc4c06c5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009a10 tx=0x7fdc50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:19.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.107+0000 7fdc47fff700 1 -- 172.21.15.45:0/2824068213 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdc58055240 con 0x7fdc6015af30 2023-08-27T11:13:19.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.307+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdc600ce4c0 con 0x7fdc6015af30 2023-08-27T11:13:19.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.308+0000 7fdc47fff700 1 -- 172.21.15.45:0/2824068213 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdc58058860 con 0x7fdc6015af30 2023-08-27T11:13:19.309 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:19.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc4c06a0e0 msgr2=0x7fdc4c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc4c06a0e0 0x7fdc4c06c5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009a10 tx=0x7fdc50008040 comp rx=0 tx=0).stop 2023-08-27T11:13:19.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6015af30 msgr2=0x7fdc601f12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6015af30 0x7fdc601f12b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdc58012000 tx=0x7fdc58004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 shutdown_connections 2023-08-27T11:13:19.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6015af30 0x7fdc601f12b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc4c06a0e0 0x7fdc4c06c5a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc600cde80 0x7fdc600b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 --2- 172.21.15.45:0/2824068213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc600cf960 0x7fdc60155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 >> 172.21.15.45:0/2824068213 conn(0x7fdc6009a1a0 msgr2=0x7fdc600d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:19.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 shutdown_connections 2023-08-27T11:13:19.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.310+0000 7fdc6586e700 1 -- 172.21.15.45:0/2824068213 wait complete. 2023-08-27T11:13:19.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:13:19.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.448+0000 7feb2fcf3700 1 Processor -- start 2023-08-27T11:13:19.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.448+0000 7feb2fcf3700 1 -- start start 2023-08-27T11:13:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.449+0000 7feb2fcf3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 0x7feb280cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.449+0000 7feb2fcf3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb280ce2e0 0x7feb280b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.449+0000 7feb2fcf3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2815d6a0 con 0x7feb2815d2c0 2023-08-27T11:13:19.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.449+0000 7feb2fcf3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb280ce770 con 0x7feb280ce2e0 2023-08-27T11:13:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.449+0000 7feb2fcf3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb280d2c90 con 0x7feb280b8c20 2023-08-27T11:13:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.449+0000 7feb2e290700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb280ce2e0 0x7feb280b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.449+0000 7feb2e290700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb280ce2e0 0x7feb280b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53572/0 (socket says 172.21.15.45:53572) 2023-08-27T11:13:19.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.449+0000 7feb2e290700 1 -- 172.21.15.45:0/1247042452 learned_addr learned my addr 172.21.15.45:0/1247042452 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2d28e700 1 --2- 172.21.15.45:0/1247042452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 0x7feb280cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2e290700 1 -- 172.21.15.45:0/1247042452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 msgr2=0x7feb280cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2e290700 1 --2- 172.21.15.45:0/1247042452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 0x7feb280cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2e290700 1 -- 172.21.15.45:0/1247042452 >> v1:172.21.15.135:6789/0 conn(0x7feb2815d2c0 legacy=0x7feb280d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2e290700 1 -- 172.21.15.45:0/1247042452 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb280d1bf0 con 0x7feb280ce2e0 2023-08-27T11:13:19.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2d28e700 1 --2- 172.21.15.45:0/1247042452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 0x7feb280cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:19.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2e290700 1 --2- 172.21.15.45:0/1247042452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb280ce2e0 0x7feb280b6100 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb20009e50 tx=0x7feb2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d1581ef89d1815 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:19.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2ca8d700 1 -- 172.21.15.45:0/1247042452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2000e070 con 0x7feb280ce2e0 2023-08-27T11:13:19.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2ca8d700 1 -- 172.21.15.45:0/1247042452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb20003a20 con 0x7feb280ce2e0 2023-08-27T11:13:19.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2ca8d700 1 -- 172.21.15.45:0/1247042452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20003dd0 con 0x7feb280ce2e0 2023-08-27T11:13:19.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2fcf3700 1 -- 172.21.15.45:0/1247042452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb280ce2e0 msgr2=0x7feb280b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.450+0000 7feb2fcf3700 1 --2- 172.21.15.45:0/1247042452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb280ce2e0 0x7feb280b6100 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb20009e50 tx=0x7feb2000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.451+0000 7feb2fcf3700 1 -- 172.21.15.45:0/1247042452 shutdown_connections 2023-08-27T11:13:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.451+0000 7feb2fcf3700 1 --2- 172.21.15.45:0/1247042452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb280ce2e0 0x7feb280b6100 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.451+0000 7feb2fcf3700 1 --2- 172.21.15.45:0/1247042452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 0x7feb280cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.451+0000 7feb2fcf3700 1 -- 172.21.15.45:0/1247042452 >> 172.21.15.45:0/1247042452 conn(0x7feb280d2fd0 msgr2=0x7feb280d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.451+0000 7feb2fcf3700 1 -- 172.21.15.45:0/1247042452 shutdown_connections 2023-08-27T11:13:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.451+0000 7feb2fcf3700 1 -- 172.21.15.45:0/1247042452 wait complete. 2023-08-27T11:13:19.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.452+0000 7feb2fcf3700 1 Processor -- start 2023-08-27T11:13:19.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.452+0000 7feb2fcf3700 1 -- start start 2023-08-27T11:13:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2fcf3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 0x7feb280c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2fcf3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb280ce2e0 0x7feb2815fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2fcf3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb2815d2c0 0x7feb2815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2fcf3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280c7ab0 con 0x7feb2815d2c0 2023-08-27T11:13:19.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2fcf3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb28151460 con 0x7feb280ce2e0 2023-08-27T11:13:19.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2fcf3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb280c7000 con 0x7feb280b8c20 2023-08-27T11:13:19.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2e290700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb2815d2c0 0x7feb2815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2e290700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb2815d2c0 0x7feb2815ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53578/0 (socket says 172.21.15.45:53578) 2023-08-27T11:13:19.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2e290700 1 -- 172.21.15.45:0/3498242956 learned_addr learned my addr 172.21.15.45:0/3498242956 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:19.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2da8f700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 0x7feb280c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2d28e700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb280ce2e0 0x7feb2815fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2e290700 1 -- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 msgr2=0x7feb280c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2e290700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 0x7feb280c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2e290700 1 -- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb280ce2e0 msgr2=0x7feb2815fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2e290700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb280ce2e0 0x7feb2815fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2e290700 1 -- 172.21.15.45:0/3498242956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20009b30 con 0x7feb2815d2c0 2023-08-27T11:13:19.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2d28e700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb280ce2e0 0x7feb2815fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:19.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2e290700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb2815d2c0 0x7feb2815ff50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb2000b730 tx=0x7feb20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:19.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb1e7fc700 1 -- 172.21.15.45:0/3498242956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20004440 con 0x7feb2815d2c0 2023-08-27T11:13:19.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb1e7fc700 1 -- 172.21.15.45:0/3498242956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb200045a0 con 0x7feb2815d2c0 2023-08-27T11:13:19.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb28151cc0 con 0x7feb2815d2c0 2023-08-27T11:13:19.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.453+0000 7feb1e7fc700 1 -- 172.21.15.45:0/3498242956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20004910 con 0x7feb2815d2c0 2023-08-27T11:13:19.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.454+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb28151670 con 0x7feb2815d2c0 2023-08-27T11:13:19.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.454+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb28151e50 con 0x7feb2815d2c0 2023-08-27T11:13:19.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.455+0000 7feb1e7fc700 1 -- 172.21.15.45:0/3498242956 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feb20012430 con 0x7feb2815d2c0 2023-08-27T11:13:19.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.455+0000 7feb1e7fc700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7feb0c06a0e0 0x7feb0c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.455+0000 7feb1e7fc700 1 -- 172.21.15.45:0/3498242956 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7feb20089fc0 con 0x7feb2815d2c0 2023-08-27T11:13:19.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.456+0000 7feb2da8f700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7feb0c06a0e0 0x7feb0c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.456+0000 7feb2da8f700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7feb0c06a0e0 0x7feb0c06c5a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb18004f30 tx=0x7feb18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:19.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.460+0000 7feb1e7fc700 1 -- 172.21.15.45:0/3498242956 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feb20055370 con 0x7feb2815d2c0 2023-08-27T11:13:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.675+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb2815e820 con 0x7feb2815d2c0 2023-08-27T11:13:19.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.675+0000 7feb1e7fc700 1 -- 172.21.15.45:0/3498242956 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb2815e820 con 0x7feb2815d2c0 2023-08-27T11:13:19.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7feb0c06a0e0 msgr2=0x7feb0c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7feb0c06a0e0 0x7feb0c06c5a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb18004f30 tx=0x7feb18009040 comp rx=0 tx=0).stop 2023-08-27T11:13:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb2815d2c0 msgr2=0x7feb2815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb2815d2c0 0x7feb2815ff50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb2000b730 tx=0x7feb20004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 shutdown_connections 2023-08-27T11:13:19.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb2815d2c0 0x7feb2815ff50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7feb0c06a0e0 0x7feb0c06c5a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feb280b8c20 0x7feb280c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 --2- 172.21.15.45:0/3498242956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb280ce2e0 0x7feb2815fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.677+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 >> 172.21.15.45:0/3498242956 conn(0x7feb280d2fd0 msgr2=0x7feb280b8110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.678+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 shutdown_connections 2023-08-27T11:13:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.678+0000 7feb2fcf3700 1 -- 172.21.15.45:0/3498242956 wait complete. 2023-08-27T11:13:19.681 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2023-08-27T11:13:19.691 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-08-27T11:13:19.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:19.692 DEBUG:teuthology.orchestra.run.smithi135:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-08-27T11:13:19.740 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2023-08-27T11:13:19.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:19.741 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-27T11:13:19.760 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:13:19.760 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:13:19.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.886+0000 7ff694b81700 1 Processor -- start 2023-08-27T11:13:19.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.886+0000 7ff694b81700 1 -- start start 2023-08-27T11:13:19.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff694b81700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6900cd300 0x7ff6900cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff694b81700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6900cfae0 con 0x7ff69015a0f0 2023-08-27T11:13:19.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff694b81700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69015e900 con 0x7ff69006ed90 2023-08-27T11:13:19.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff694b81700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900cae60 con 0x7ff6900cd300 2023-08-27T11:13:19.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68dd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6900cd300 0x7ff6900cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68e59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff69015a0f0 0x7ff69015daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40358/0 (socket says 172.21.15.45:40358) 2023-08-27T11:13:19.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68e59c700 1 -- 172.21.15.45:0/957715974 learned_addr learned my addr 172.21.15.45:0/957715974 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:19.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68d59a700 1 -- 172.21.15.45:0/957715974 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881466565 0 0) 0x7ff6900cfae0 con 0x7ff69015a0f0 2023-08-27T11:13:19.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68d59a700 1 -- 172.21.15.45:0/957715974 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff67c0035b0 con 0x7ff69015a0f0 2023-08-27T11:13:19.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68dd9b700 1 -- 172.21.15.45:0/957715974 >> v1:172.21.15.135:6789/0 conn(0x7ff69006ed90 legacy=0x7ff6900cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:19.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68dd9b700 1 -- 172.21.15.45:0/957715974 >> v1:172.21.15.45:6789/0 conn(0x7ff69015a0f0 legacy=0x7ff69015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68dd9b700 1 -- 172.21.15.45:0/957715974 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900cb140 con 0x7ff6900cd300 2023-08-27T11:13:19.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68dd9b700 1 --2- 172.21.15.45:0/957715974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 0x7ff6900cbc20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff678009840 tx=0x7ff678009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7557a2d543aa12d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:19.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68d59a700 1 -- 172.21.15.45:0/957715974 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678004230 con 0x7ff6900cd300 2023-08-27T11:13:19.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.888+0000 7ff68d59a700 1 -- 172.21.15.45:0/957715974 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678004390 con 0x7ff6900cd300 2023-08-27T11:13:19.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.889+0000 7ff68d59a700 1 -- 172.21.15.45:0/957715974 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780036c0 con 0x7ff6900cd300 2023-08-27T11:13:19.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.889+0000 7ff694b81700 1 -- 172.21.15.45:0/957715974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 msgr2=0x7ff6900cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.889+0000 7ff694b81700 1 --2- 172.21.15.45:0/957715974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 0x7ff6900cbc20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff678009840 tx=0x7ff678009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:19.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.889+0000 7ff694b81700 1 -- 172.21.15.45:0/957715974 shutdown_connections 2023-08-27T11:13:19.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.889+0000 7ff694b81700 1 --2- 172.21.15.45:0/957715974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 0x7ff6900cbc20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.889+0000 7ff694b81700 1 -- 172.21.15.45:0/957715974 >> 172.21.15.45:0/957715974 conn(0x7ff6900cc7e0 msgr2=0x7ff6900ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:19.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.889+0000 7ff694b81700 1 -- 172.21.15.45:0/957715974 shutdown_connections 2023-08-27T11:13:19.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.889+0000 7ff694b81700 1 -- 172.21.15.45:0/957715974 wait complete. 2023-08-27T11:13:19.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.891+0000 7ff694b81700 1 Processor -- start 2023-08-27T11:13:19.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.891+0000 7ff694b81700 1 -- start start 2023-08-27T11:13:19.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff694b81700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff69006ed90 0x7ff6901f0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff694b81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 0x7ff69006a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff694b81700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff69015a0f0 0x7ff69006a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff694b81700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69015d800 con 0x7ff6900cd300 2023-08-27T11:13:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff694b81700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69013e680 con 0x7ff69015a0f0 2023-08-27T11:13:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff694b81700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69013e960 con 0x7ff69006ed90 2023-08-27T11:13:19.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 0x7ff69006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 0x7ff69006a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53590/0 (socket says 172.21.15.45:53590) 2023-08-27T11:13:19.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68dd9b700 1 -- 172.21.15.45:0/504107965 learned_addr learned my addr 172.21.15.45:0/504107965 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:19.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68ed9d700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff69015a0f0 0x7ff69006a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68e59c700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff69006ed90 0x7ff6901f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68dd9b700 1 -- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff69006ed90 msgr2=0x7ff6901f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68dd9b700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff69006ed90 0x7ff6901f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68dd9b700 1 -- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff69015a0f0 msgr2=0x7ff69006a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:19.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68dd9b700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff69015a0f0 0x7ff69006a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:19.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68dd9b700 1 -- 172.21.15.45:0/504107965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6780094a0 con 0x7ff6900cd300 2023-08-27T11:13:19.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68e59c700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff69006ed90 0x7ff6901f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:19.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68ed9d700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff69015a0f0 0x7ff69006a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:19.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.892+0000 7ff68dd9b700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 0x7ff69006a280 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff6780090b0 tx=0x7ff678004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:19.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.893+0000 7ff686ffd700 1 -- 172.21.15.45:0/504107965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678011630 con 0x7ff6900cd300 2023-08-27T11:13:19.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.893+0000 7ff686ffd700 1 -- 172.21.15.45:0/504107965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678011790 con 0x7ff6900cd300 2023-08-27T11:13:19.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.893+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69013ec40 con 0x7ff6900cd300 2023-08-27T11:13:19.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.893+0000 7ff686ffd700 1 -- 172.21.15.45:0/504107965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678011b00 con 0x7ff6900cd300 2023-08-27T11:13:19.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.893+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6901f0660 con 0x7ff6900cd300 2023-08-27T11:13:19.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.894+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69013f160 con 0x7ff6900cd300 2023-08-27T11:13:19.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.894+0000 7ff686ffd700 1 -- 172.21.15.45:0/504107965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff678018040 con 0x7ff6900cd300 2023-08-27T11:13:19.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.895+0000 7ff686ffd700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff67006a0e0 0x7ff67006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:19.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.895+0000 7ff686ffd700 1 -- 172.21.15.45:0/504107965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff6780891b0 con 0x7ff6900cd300 2023-08-27T11:13:19.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.895+0000 7ff68e59c700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff67006a0e0 0x7ff67006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:19.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.895+0000 7ff68e59c700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff67006a0e0 0x7ff67006c5a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff680005910 tx=0x7ff680009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:19.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:19.899+0000 7ff686ffd700 1 -- 172.21.15.45:0/504107965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff69013f160 con 0x7ff6900cd300 2023-08-27T11:13:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.142+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff69013f160 con 0x7ff6900cd300 2023-08-27T11:13:20.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.143+0000 7ff686ffd700 1 -- 172.21.15.45:0/504107965 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7ff69013f160 con 0x7ff6900cd300 2023-08-27T11:13:20.145 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:20.145 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:33.983169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:13:20.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff67006a0e0 msgr2=0x7ff67006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff67006a0e0 0x7ff67006c5a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff680005910 tx=0x7ff680009040 comp rx=0 tx=0).stop 2023-08-27T11:13:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 msgr2=0x7ff69006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 0x7ff69006a280 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff6780090b0 tx=0x7ff678004750 comp rx=0 tx=0).stop 2023-08-27T11:13:20.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 shutdown_connections 2023-08-27T11:13:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff67006a0e0 0x7ff67006c5a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff69015a0f0 0x7ff69006a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff69006ed90 0x7ff6901f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 --2- 172.21.15.45:0/504107965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900cd300 0x7ff69006a280 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 >> 172.21.15.45:0/504107965 conn(0x7ff6900cc7e0 msgr2=0x7ff6900b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 shutdown_connections 2023-08-27T11:13:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.145+0000 7ff694b81700 1 -- 172.21.15.45:0/504107965 wait complete. 2023-08-27T11:13:20.149 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-08-27T11:13:20.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.282+0000 7ff8240bb700 1 Processor -- start 2023-08-27T11:13:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.283+0000 7ff8240bb700 1 -- start start 2023-08-27T11:13:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.283+0000 7ff8240bb700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff81c15b070 0x7ff81c0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.283+0000 7ff8240bb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c15dad0 con 0x7ff81c06e980 2023-08-27T11:13:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.283+0000 7ff8240bb700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c15ddb0 con 0x7ff81c0cdf60 2023-08-27T11:13:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.283+0000 7ff8240bb700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c15d8e0 con 0x7ff81c15b070 2023-08-27T11:13:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.283+0000 7ff822658700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff81c06e980 0x7ff81c0ae8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40376/0 (socket says 172.21.15.45:40376) 2023-08-27T11:13:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff822658700 1 -- 172.21.15.45:0/21109053 learned_addr learned my addr 172.21.15.45:0/21109053 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff821e57700 1 --2- 172.21.15.45:0/21109053 >> v2:172.21.15.135:3300/0 conn(0x7ff81c15b070 0x7ff81c0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110051466 0 0) 0x7ff81c15dad0 con 0x7ff81c06e980 2023-08-27T11:13:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8080035b0 con 0x7ff81c06e980 2023-08-27T11:13:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 843593126 0 0) 0x7ff8080035b0 con 0x7ff81c06e980 2023-08-27T11:13:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 >> v2:172.21.15.135:3300/0 conn(0x7ff81c15b070 msgr2=0x7ff81c0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 --2- 172.21.15.45:0/21109053 >> v2:172.21.15.135:3300/0 conn(0x7ff81c15b070 0x7ff81c0cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 >> v1:172.21.15.135:6789/0 conn(0x7ff81c0cdf60 legacy=0x7ff81c06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c15e090 con 0x7ff81c06e980 2023-08-27T11:13:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7ff8140019f0 con 0x7ff81c06e980 2023-08-27T11:13:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 >> v1:172.21.15.45:6789/0 conn(0x7ff81c06e980 legacy=0x7ff81c0ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff808003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff808004850 0x7ff808004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808005210 0x7ff80800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800c080 con 0x7ff808003a00 2023-08-27T11:13:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800c1f0 con 0x7ff808005210 2023-08-27T11:13:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800c3b0 con 0x7ff808004850 2023-08-27T11:13:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff821e57700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff808003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff822658700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808005210 0x7ff80800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff821656700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff808004850 0x7ff808004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff821e57700 1 -- 172.21.15.45:0/21109053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff808004850 msgr2=0x7ff808004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.284+0000 7ff821e57700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff808004850 0x7ff808004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff821e57700 1 -- 172.21.15.45:0/21109053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808005210 msgr2=0x7ff80800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff821e57700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff808005210 0x7ff80800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff821e57700 1 -- 172.21.15.45:0/21109053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080035b0 con 0x7ff808003a00 2023-08-27T11:13:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff822658700 1 -- 172.21.15.45:0/21109053 reap_dead start 2023-08-27T11:13:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff821e57700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff808003e20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff818009b60 tx=0x7ff808005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6717ecad0b35003 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800e6c0 con 0x7ff808003a00 2023-08-27T11:13:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81800e820 con 0x7ff808003a00 2023-08-27T11:13:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff820e55700 1 -- 172.21.15.45:0/21109053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800eb90 con 0x7ff808003a00 2023-08-27T11:13:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff8240bb700 1 -- 172.21.15.45:0/21109053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 msgr2=0x7ff808003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff8240bb700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff808003e20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff818009b60 tx=0x7ff808005880 comp rx=0 tx=0).stop 2023-08-27T11:13:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff8240bb700 1 -- 172.21.15.45:0/21109053 shutdown_connections 2023-08-27T11:13:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff8240bb700 1 --2- 172.21.15.45:0/21109053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff808003e20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff8240bb700 1 -- 172.21.15.45:0/21109053 >> 172.21.15.45:0/21109053 conn(0x7ff81c0cfe50 msgr2=0x7ff81c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff8240bb700 1 -- 172.21.15.45:0/21109053 shutdown_connections 2023-08-27T11:13:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.285+0000 7ff8240bb700 1 -- 172.21.15.45:0/21109053 wait complete. 2023-08-27T11:13:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.287+0000 7ff8240bb700 1 Processor -- start 2023-08-27T11:13:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.287+0000 7ff8240bb700 1 -- start start 2023-08-27T11:13:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.287+0000 7ff8240bb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff81c14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.287+0000 7ff8240bb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff81c1f61a0 0x7ff81c1f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.287+0000 7ff8240bb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff81c0c3fc0 0x7ff81c0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.287+0000 7ff8240bb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c1f6b60 con 0x7ff808003a00 2023-08-27T11:13:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.287+0000 7ff8240bb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c1f6ff0 con 0x7ff81c0c3fc0 2023-08-27T11:13:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.287+0000 7ff8240bb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c158550 con 0x7ff81c1f61a0 2023-08-27T11:13:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff821e57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff81c14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff821e57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff81c14c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53608/0 (socket says 172.21.15.45:53608) 2023-08-27T11:13:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff821e57700 1 -- 172.21.15.45:0/842792292 learned_addr learned my addr 172.21.15.45:0/842792292 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff821e57700 1 -- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff81c1f61a0 msgr2=0x7ff81c1f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff821e57700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff81c1f61a0 0x7ff81c1f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff822658700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff81c0c3fc0 0x7ff81c0c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff821e57700 1 -- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff81c0c3fc0 msgr2=0x7ff81c0c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff821e57700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff81c0c3fc0 0x7ff81c0c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff821e57700 1 -- 172.21.15.45:0/842792292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0c4980 con 0x7ff808003a00 2023-08-27T11:13:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff821e57700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff81c14c9d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c1c0 tx=0x7ff8180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff80e7fc700 1 -- 172.21.15.45:0/842792292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800e6c0 con 0x7ff808003a00 2023-08-27T11:13:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff80e7fc700 1 -- 172.21.15.45:0/842792292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff818003ad0 con 0x7ff808003a00 2023-08-27T11:13:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c154270 con 0x7ff808003a00 2023-08-27T11:13:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff80e7fc700 1 -- 172.21.15.45:0/842792292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818003e80 con 0x7ff808003a00 2023-08-27T11:13:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.288+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c157f90 con 0x7ff808003a00 2023-08-27T11:13:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.289+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c0cf8d0 con 0x7ff808003a00 2023-08-27T11:13:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.290+0000 7ff80e7fc700 1 -- 172.21.15.45:0/842792292 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff818013470 con 0x7ff808003a00 2023-08-27T11:13:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.290+0000 7ff80e7fc700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff80006a110 0x7ff80006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.290+0000 7ff80e7fc700 1 -- 172.21.15.45:0/842792292 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff81808a170 con 0x7ff808003a00 2023-08-27T11:13:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.290+0000 7ff821656700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff80006a110 0x7ff80006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.291+0000 7ff821656700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff80006a110 0x7ff80006c5d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff81c1f7260 tx=0x7ff810009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.294+0000 7ff80e7fc700 1 -- 172.21.15.45:0/842792292 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff8180556e0 con 0x7ff808003a00 2023-08-27T11:13:20.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.495+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff81c072810 con 0x7ff808003a00 2023-08-27T11:13:20.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.496+0000 7ff80e7fc700 1 -- 172.21.15.45:0/842792292 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7ff818058d00 con 0x7ff808003a00 2023-08-27T11:13:20.498 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:20.498 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff80006a110 msgr2=0x7ff80006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff80006a110 0x7ff80006c5d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff81c1f7260 tx=0x7ff810009040 comp rx=0 tx=0).stop 2023-08-27T11:13:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 msgr2=0x7ff81c14c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff81c14c9d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c1c0 tx=0x7ff8180047c0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 shutdown_connections 2023-08-27T11:13:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff808003a00 0x7ff81c14c9d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff80006a110 0x7ff80006c5d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff81c0c3fc0 0x7ff81c0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 --2- 172.21.15.45:0/842792292 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff81c1f61a0 0x7ff81c1f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 >> 172.21.15.45:0/842792292 conn(0x7ff81c0cfe50 msgr2=0x7ff81c160e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 shutdown_connections 2023-08-27T11:13:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.499+0000 7ff8240bb700 1 -- 172.21.15.45:0/842792292 wait complete. 2023-08-27T11:13:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:12:06.215374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:12:07.287854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-27T11:12:10.957364+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-27T11:13:20.513 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:13:20.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.641+0000 7f497df83700 1 Processor -- start 2023-08-27T11:13:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.641+0000 7f497df83700 1 -- start start 2023-08-27T11:13:20.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.641+0000 7f497df83700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f497815a700 0x7f497815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.641+0000 7f497df83700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f497815af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.641+0000 7f497df83700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49780cb060 con 0x7f497815aae0 2023-08-27T11:13:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.641+0000 7f497df83700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497806e320 con 0x7f497815a700 2023-08-27T11:13:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.641+0000 7f497df83700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497815d7d0 con 0x7f497806f620 2023-08-27T11:13:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f49777fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f497815a700 0x7f497815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f4977fff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f497815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f49777fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f497815a700 0x7f497815b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:42472/0 (socket says 172.21.15.45:42472) 2023-08-27T11:13:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f49777fe700 1 -- 172.21.15.45:0/264442928 learned_addr learned my addr 172.21.15.45:0/264442928 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f4977fff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f497815af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:51530/0 (socket says 172.21.15.45:51530) 2023-08-27T11:13:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f4977fff700 1 -- 172.21.15.45:0/264442928 >> v2:172.21.15.135:3300/0 conn(0x7f497815a700 msgr2=0x7f497815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f4977fff700 1 --2- 172.21.15.45:0/264442928 >> v2:172.21.15.135:3300/0 conn(0x7f497815a700 0x7f497815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f4977fff700 1 -- 172.21.15.45:0/264442928 >> v1:172.21.15.135:6789/0 conn(0x7f497815aae0 legacy=0x7f497806ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f4977fff700 1 -- 172.21.15.45:0/264442928 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49780ca450 con 0x7f497806f620 2023-08-27T11:13:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f49777fe700 1 --2- 172.21.15.45:0/264442928 >> v2:172.21.15.135:3300/0 conn(0x7f497815a700 0x7f497815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.642+0000 7f4977fff700 1 --2- 172.21.15.45:0/264442928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f497815af40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4970009d90 tx=0x7f497000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a04da3c013cebd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f49767fc700 1 -- 172.21.15.45:0/264442928 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497000e070 con 0x7f497806f620 2023-08-27T11:13:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f49767fc700 1 -- 172.21.15.45:0/264442928 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497000bb20 con 0x7f497806f620 2023-08-27T11:13:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f49767fc700 1 -- 172.21.15.45:0/264442928 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4970012430 con 0x7f497806f620 2023-08-27T11:13:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f497df83700 1 -- 172.21.15.45:0/264442928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 msgr2=0x7f497815af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f497df83700 1 --2- 172.21.15.45:0/264442928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f497815af40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4970009d90 tx=0x7f497000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f497df83700 1 -- 172.21.15.45:0/264442928 shutdown_connections 2023-08-27T11:13:20.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f497df83700 1 --2- 172.21.15.45:0/264442928 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f497815af40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f497df83700 1 --2- 172.21.15.45:0/264442928 >> v2:172.21.15.135:3300/0 conn(0x7f497815a700 0x7f497815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f497df83700 1 -- 172.21.15.45:0/264442928 >> 172.21.15.45:0/264442928 conn(0x7f49780cc7e0 msgr2=0x7f49780ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:20.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f497df83700 1 -- 172.21.15.45:0/264442928 shutdown_connections 2023-08-27T11:13:20.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.643+0000 7f497df83700 1 -- 172.21.15.45:0/264442928 wait complete. 2023-08-27T11:13:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.644+0000 7f497df83700 1 Processor -- start 2023-08-27T11:13:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.645+0000 7f497df83700 1 -- start start 2023-08-27T11:13:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.645+0000 7f497df83700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f49781f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.645+0000 7f497df83700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f497815a700 0x7f49781f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.645+0000 7f497df83700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f497815aae0 0x7f49780afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.645+0000 7f497df83700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781f6220 con 0x7f497815aae0 2023-08-27T11:13:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.645+0000 7f497df83700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4978148c40 con 0x7f497815a700 2023-08-27T11:13:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.645+0000 7f497df83700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49781f6410 con 0x7f497806f620 2023-08-27T11:13:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.645+0000 7f4977fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f497815aae0 0x7f49780afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f4977fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f497815aae0 0x7f49780afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53626/0 (socket says 172.21.15.45:53626) 2023-08-27T11:13:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f4977fff700 1 -- 172.21.15.45:0/2175809118 learned_addr learned my addr 172.21.15.45:0/2175809118 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f49777fe700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f49781f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f4976ffd700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f497815a700 0x7f49781f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f4977fff700 1 -- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 msgr2=0x7f49781f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f4977fff700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f49781f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f4977fff700 1 -- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f497815a700 msgr2=0x7f49781f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f4977fff700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f497815a700 0x7f49781f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f4977fff700 1 -- 172.21.15.45:0/2175809118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c009550 con 0x7f497815aae0 2023-08-27T11:13:20.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f4977fff700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f497815aae0 0x7f49780afcc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4970014040 tx=0x7f4970004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:20.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f495ffff700 1 -- 172.21.15.45:0/2175809118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4970012430 con 0x7f497815aae0 2023-08-27T11:13:20.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f495ffff700 1 -- 172.21.15.45:0/2175809118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4970020070 con 0x7f497815aae0 2023-08-27T11:13:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4970009a70 con 0x7f497815aae0 2023-08-27T11:13:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f495ffff700 1 -- 172.21.15.45:0/2175809118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4970003ea0 con 0x7f497815aae0 2023-08-27T11:13:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.646+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49781f69d0 con 0x7f497815aae0 2023-08-27T11:13:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.647+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49780726d0 con 0x7f497815aae0 2023-08-27T11:13:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.648+0000 7f495ffff700 1 -- 172.21.15.45:0/2175809118 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4970004040 con 0x7f497815aae0 2023-08-27T11:13:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.648+0000 7f495ffff700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f496406a0e0 0x7f496406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.648+0000 7f495ffff700 1 -- 172.21.15.45:0/2175809118 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4970022070 con 0x7f497815aae0 2023-08-27T11:13:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.649+0000 7f49777fe700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f496406a0e0 0x7f496406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.649+0000 7f49777fe700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f496406a0e0 0x7f496406c5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49781f5c40 tx=0x7f496c0092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.653+0000 7f495ffff700 1 -- 172.21.15.45:0/2175809118 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4970055d10 con 0x7f497815aae0 2023-08-27T11:13:20.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.853+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49780734a0 con 0x7f497815aae0 2023-08-27T11:13:20.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.853+0000 7f495ffff700 1 -- 172.21.15.45:0/2175809118 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4970059330 con 0x7f497815aae0 2023-08-27T11:13:20.857 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:13:20.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f496406a0e0 msgr2=0x7f496406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f496406a0e0 0x7f496406c5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49781f5c40 tx=0x7f496c0092c0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f497815aae0 msgr2=0x7f49780afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:20.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f497815aae0 0x7f49780afcc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4970014040 tx=0x7f4970004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 shutdown_connections 2023-08-27T11:13:20.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f497815aae0 0x7f49780afcc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f496406a0e0 0x7f496406c5a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f497806f620 0x7f49781f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 --2- 172.21.15.45:0/2175809118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f497815a700 0x7f49781f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:20.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.855+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 >> 172.21.15.45:0/2175809118 conn(0x7f49780cc7e0 msgr2=0x7f4978159570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:20.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.856+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 shutdown_connections 2023-08-27T11:13:20.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.856+0000 7f497df83700 1 -- 172.21.15.45:0/2175809118 wait complete. 2023-08-27T11:13:20.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:13:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.999+0000 7f73ae38f700 1 Processor -- start 2023-08-27T11:13:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:20.999+0000 7f73ae38f700 1 -- start start 2023-08-27T11:13:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73ae38f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f73a8160d00 0x7f73a806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73ae38f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a806b160 con 0x7f73a806a7c0 2023-08-27T11:13:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73ae38f700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a80cdf90 con 0x7f73a815d340 2023-08-27T11:13:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73ae38f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a8072010 con 0x7f73a8160d00 2023-08-27T11:13:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73ac92c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f73a806a7c0 0x7f73a80b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40410/0 (socket says 172.21.15.45:40410) 2023-08-27T11:13:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73ac92c700 1 -- 172.21.15.45:0/1114469137 learned_addr learned my addr 172.21.15.45:0/1114469137 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a77fe700 1 --2- 172.21.15.45:0/1114469137 >> v2:172.21.15.135:3300/0 conn(0x7f73a8160d00 0x7f73a806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210977396 0 0) 0x7f73a806b160 con 0x7f73a806a7c0 2023-08-27T11:13:21.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73900035b0 con 0x7f73a806a7c0 2023-08-27T11:13:21.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4074782339 0 0) 0x7f73900035b0 con 0x7f73a806a7c0 2023-08-27T11:13:21.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 >> v2:172.21.15.135:3300/0 conn(0x7f73a8160d00 msgr2=0x7f73a806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:21.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 --2- 172.21.15.45:0/1114469137 >> v2:172.21.15.135:3300/0 conn(0x7f73a8160d00 0x7f73a806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 >> v1:172.21.15.135:6789/0 conn(0x7f73a815d340 legacy=0x7f73a80cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80af590 con 0x7f73a806a7c0 2023-08-27T11:13:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f73a00019f0 con 0x7f73a806a7c0 2023-08-27T11:13:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 >> v1:172.21.15.45:6789/0 conn(0x7f73a806a7c0 legacy=0x7f73a80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.000+0000 7f73a6ffd700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7390003a00 0x7f7390003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a6ffd700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390004850 0x7f7390004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a6ffd700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7390005210 0x7f739000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a8072010 con 0x7f7390005210 2023-08-27T11:13:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739000c1b0 con 0x7f7390004850 2023-08-27T11:13:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739000c370 con 0x7f7390003a00 2023-08-27T11:13:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73ac92c700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7390005210 0x7f739000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a77fe700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390004850 0x7f7390004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a7fff700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7390003a00 0x7f7390003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73ac92c700 1 -- 172.21.15.45:0/1114469137 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7390003a00 msgr2=0x7f7390003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73ac92c700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7390003a00 0x7f7390003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73ac92c700 1 -- 172.21.15.45:0/1114469137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390004850 msgr2=0x7f7390004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73ac92c700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390004850 0x7f7390004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73ac92c700 1 -- 172.21.15.45:0/1114469137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73900035b0 con 0x7f7390005210 2023-08-27T11:13:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a77fe700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390004850 0x7f7390004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73ac92c700 1 -- 172.21.15.45:0/1114469137 reap_dead start 2023-08-27T11:13:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73ac92c700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7390005210 0x7f739000ba10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f73a000a820 tx=0x7f7390005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9f5d239ebd7717 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000a2f0 con 0x7f7390005210 2023-08-27T11:13:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a0012600 con 0x7f7390005210 2023-08-27T11:13:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.001+0000 7f73a6ffd700 1 -- 172.21.15.45:0/1114469137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00129b0 con 0x7f7390005210 2023-08-27T11:13:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.002+0000 7f73ae38f700 1 -- 172.21.15.45:0/1114469137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7390005210 msgr2=0x7f739000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.002+0000 7f73ae38f700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7390005210 0x7f739000ba10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f73a000a820 tx=0x7f7390005880 comp rx=0 tx=0).stop 2023-08-27T11:13:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.002+0000 7f73ae38f700 1 -- 172.21.15.45:0/1114469137 shutdown_connections 2023-08-27T11:13:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.002+0000 7f73ae38f700 1 --2- 172.21.15.45:0/1114469137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7390005210 0x7f739000ba10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.002+0000 7f73ae38f700 1 -- 172.21.15.45:0/1114469137 >> 172.21.15.45:0/1114469137 conn(0x7f73a80cc7c0 msgr2=0x7f73a80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.002+0000 7f73ae38f700 1 -- 172.21.15.45:0/1114469137 shutdown_connections 2023-08-27T11:13:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.002+0000 7f73ae38f700 1 -- 172.21.15.45:0/1114469137 wait complete. 2023-08-27T11:13:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.003+0000 7f73ae38f700 1 Processor -- start 2023-08-27T11:13:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.003+0000 7f73ae38f700 1 -- start start 2023-08-27T11:13:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73ae38f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390005210 0x7f73a8149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73ae38f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a80ad900 0x7f73a80add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73ae38f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73a80c7890 0x7f73a80c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73ae38f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80ae2c0 con 0x7f73a80ad900 2023-08-27T11:13:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73ae38f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a814a1d0 con 0x7f7390005210 2023-08-27T11:13:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73ae38f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80c8650 con 0x7f73a80c7890 2023-08-27T11:13:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73a77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a80ad900 0x7f73a80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73a77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a80ad900 0x7f73a80add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53644/0 (socket says 172.21.15.45:53644) 2023-08-27T11:13:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73a77fe700 1 -- 172.21.15.45:0/155670360 learned_addr learned my addr 172.21.15.45:0/155670360 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73a7fff700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390005210 0x7f73a8149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73ac92c700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73a80c7890 0x7f73a80c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73a77fe700 1 -- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73a80c7890 msgr2=0x7f73a80c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73a77fe700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73a80c7890 0x7f73a80c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73a77fe700 1 -- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390005210 msgr2=0x7f73a8149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73a77fe700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390005210 0x7f73a8149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.004+0000 7f73a77fe700 1 -- 172.21.15.45:0/155670360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a814a470 con 0x7f73a80ad900 2023-08-27T11:13:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.005+0000 7f73a77fe700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a80ad900 0x7f73a80add80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f739c010850 tx=0x7f739c010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.005+0000 7f73a4ff9700 1 -- 172.21.15.45:0/155670360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c004020 con 0x7f73a80ad900 2023-08-27T11:13:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.005+0000 7f73a4ff9700 1 -- 172.21.15.45:0/155670360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c004180 con 0x7f73a80ad900 2023-08-27T11:13:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.005+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a814a750 con 0x7f73a80ad900 2023-08-27T11:13:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.005+0000 7f73a4ff9700 1 -- 172.21.15.45:0/155670360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c004510 con 0x7f73a80ad900 2023-08-27T11:13:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.005+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a80c8930 con 0x7f73a80ad900 2023-08-27T11:13:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.006+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a8072690 con 0x7f73a80ad900 2023-08-27T11:13:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.006+0000 7f73a4ff9700 1 -- 172.21.15.45:0/155670360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f739c0036a0 con 0x7f73a80ad900 2023-08-27T11:13:21.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.007+0000 7f73a4ff9700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f739406a0e0 0x7f739406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.007+0000 7f73a4ff9700 1 -- 172.21.15.45:0/155670360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f739c088c00 con 0x7f73a80ad900 2023-08-27T11:13:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.007+0000 7f73a7fff700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f739406a0e0 0x7f739406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.007+0000 7f73a7fff700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f739406a0e0 0x7f739406c5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7398000c00 tx=0x7f7398009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.011+0000 7f73a4ff9700 1 -- 172.21.15.45:0/155670360 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f739c057920 con 0x7f73a80ad900 2023-08-27T11:13:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.213+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f73a80734a0 con 0x7f73a80ad900 2023-08-27T11:13:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.213+0000 7f73a4ff9700 1 -- 172.21.15.45:0/155670360 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f739c0230a0 con 0x7f73a80ad900 2023-08-27T11:13:21.214 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:13:21.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f739406a0e0 msgr2=0x7f739406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f739406a0e0 0x7f739406c5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7398000c00 tx=0x7f7398009040 comp rx=0 tx=0).stop 2023-08-27T11:13:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a80ad900 msgr2=0x7f73a80add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a80ad900 0x7f73a80add80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f739c010850 tx=0x7f739c010b60 comp rx=0 tx=0).stop 2023-08-27T11:13:21.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 shutdown_connections 2023-08-27T11:13:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f739406a0e0 0x7f739406c5a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73a80c7890 0x7f73a80c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7390005210 0x7f73a8149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 --2- 172.21.15.45:0/155670360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a80ad900 0x7f73a80add80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 >> 172.21.15.45:0/155670360 conn(0x7f73a80cc7c0 msgr2=0x7f73a815b300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:21.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 shutdown_connections 2023-08-27T11:13:21.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.215+0000 7f73ae38f700 1 -- 172.21.15.45:0/155670360 wait complete. 2023-08-27T11:13:21.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:13:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.355+0000 7fa4fc3a9700 1 Processor -- start 2023-08-27T11:13:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.356+0000 7fa4fc3a9700 1 -- start start 2023-08-27T11:13:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.356+0000 7fa4fc3a9700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa4f415d4f0 0x7fa4f40ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.356+0000 7fa4fc3a9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4f40aefb0 0x7fa4f40ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4fc3a9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 0x7fa4f40cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4fc3a9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f415d8d0 con 0x7fa4f40aefb0 2023-08-27T11:13:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4fc3a9700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f406b010 con 0x7fa4f415d4f0 2023-08-27T11:13:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4fc3a9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40ad9f0 con 0x7fa4f40aea60 2023-08-27T11:13:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9944700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4f40aefb0 0x7fa4f40ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9944700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4f40aefb0 0x7fa4f40ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53650/0 (socket says 172.21.15.45:53650) 2023-08-27T11:13:21.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9944700 1 -- 172.21.15.45:0/689622423 learned_addr learned my addr 172.21.15.45:0/689622423 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:21.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4fa145700 1 --2- 172.21.15.45:0/689622423 >> v2:172.21.15.135:3300/0 conn(0x7fa4f415d4f0 0x7fa4f40ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4fa946700 1 --2- 172.21.15.45:0/689622423 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 0x7fa4f40cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9944700 1 -- 172.21.15.45:0/689622423 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 msgr2=0x7fa4f40cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:21.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9944700 1 --2- 172.21.15.45:0/689622423 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 0x7fa4f40cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9944700 1 -- 172.21.15.45:0/689622423 >> v2:172.21.15.135:3300/0 conn(0x7fa4f415d4f0 msgr2=0x7fa4f40ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:21.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9944700 1 --2- 172.21.15.45:0/689622423 >> v2:172.21.15.135:3300/0 conn(0x7fa4f415d4f0 0x7fa4f40ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9944700 1 -- 172.21.15.45:0/689622423 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f406ad30 con 0x7fa4f40aefb0 2023-08-27T11:13:21.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9944700 1 --2- 172.21.15.45:0/689622423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f40aefb0 0x7fa4f40ca5c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4079e70 tx=0x7fa4f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf3b982b66425d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:21.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9143700 1 -- 172.21.15.45:0/689622423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000b870 con 0x7fa4f40aefb0 2023-08-27T11:13:21.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9143700 1 -- 172.21.15.45:0/689622423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f000b9d0 con 0x7fa4f40aefb0 2023-08-27T11:13:21.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.357+0000 7fa4f9143700 1 -- 172.21.15.45:0/689622423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000bd60 con 0x7fa4f40aefb0 2023-08-27T11:13:21.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.358+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/689622423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f40aefb0 msgr2=0x7fa4f40ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.358+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/689622423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f40aefb0 0x7fa4f40ca5c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4079e70 tx=0x7fa4f0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.358+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/689622423 shutdown_connections 2023-08-27T11:13:21.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.358+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/689622423 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 0x7fa4f40cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.358+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/689622423 >> v2:172.21.15.135:3300/0 conn(0x7fa4f415d4f0 0x7fa4f40ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.358+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/689622423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f40aefb0 0x7fa4f40ca5c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.358+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/689622423 >> 172.21.15.45:0/689622423 conn(0x7fa4f40cc7e0 msgr2=0x7fa4f40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:21.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.358+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/689622423 shutdown_connections 2023-08-27T11:13:21.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.358+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/689622423 wait complete. 2023-08-27T11:13:21.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.359+0000 7fa4fc3a9700 1 Processor -- start 2023-08-27T11:13:21.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.359+0000 7fa4fc3a9700 1 -- start start 2023-08-27T11:13:21.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fc3a9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 0x7fa4f40af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fc3a9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f40aefb0 0x7fa4f415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fc3a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41e7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fc3a9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f41f2e70 con 0x7fa4f415d4f0 2023-08-27T11:13:21.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fc3a9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40afe20 con 0x7fa4f40aefb0 2023-08-27T11:13:21.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fc3a9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f40b0100 con 0x7fa4f40aea60 2023-08-27T11:13:21.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa946700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa946700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41e7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53658/0 (socket says 172.21.15.45:53658) 2023-08-27T11:13:21.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa946700 1 -- 172.21.15.45:0/3366784656 learned_addr learned my addr 172.21.15.45:0/3366784656 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4f9944700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f40aefb0 0x7fa4f415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa145700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 0x7fa4f40af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa946700 1 -- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 msgr2=0x7fa4f40af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa946700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 0x7fa4f40af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa946700 1 -- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f40aefb0 msgr2=0x7fa4f415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa946700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f40aefb0 0x7fa4f415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa946700 1 -- 172.21.15.45:0/3366784656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f0009550 con 0x7fa4f415d4f0 2023-08-27T11:13:21.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.360+0000 7fa4fa145700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 0x7fa4f40af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:21.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.361+0000 7fa4fa946700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41e7660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00cc30 tx=0x7fa4ec00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:21.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.361+0000 7fa4e6ffd700 1 -- 172.21.15.45:0/3366784656 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00ab00 con 0x7fa4f415d4f0 2023-08-27T11:13:21.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.361+0000 7fa4e6ffd700 1 -- 172.21.15.45:0/3366784656 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec00ac60 con 0x7fa4f415d4f0 2023-08-27T11:13:21.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.361+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f40b03e0 con 0x7fa4f415d4f0 2023-08-27T11:13:21.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.361+0000 7fa4e6ffd700 1 -- 172.21.15.45:0/3366784656 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec003a20 con 0x7fa4f415d4f0 2023-08-27T11:13:21.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.361+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f40c88b0 con 0x7fa4f415d4f0 2023-08-27T11:13:21.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.362+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f40c8aa0 con 0x7fa4f415d4f0 2023-08-27T11:13:21.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.362+0000 7fa4e6ffd700 1 -- 172.21.15.45:0/3366784656 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa4ec00f3f0 con 0x7fa4f415d4f0 2023-08-27T11:13:21.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.363+0000 7fa4e6ffd700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4d806a0e0 0x7fa4d806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.363+0000 7fa4e6ffd700 1 -- 172.21.15.45:0/3366784656 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa4ec088990 con 0x7fa4f415d4f0 2023-08-27T11:13:21.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.363+0000 7fa4fa145700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4d806a0e0 0x7fa4d806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.363+0000 7fa4fa145700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4d806a0e0 0x7fa4d806c5a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40ce0d0 tx=0x7fa4e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:21.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.367+0000 7fa4e6ffd700 1 -- 172.21.15.45:0/3366784656 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa4ec053d50 con 0x7fa4f415d4f0 2023-08-27T11:13:21.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.567+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4f40726d0 con 0x7fa4f415d4f0 2023-08-27T11:13:21.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.568+0000 7fa4e6ffd700 1 -- 172.21.15.45:0/3366784656 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa4ec01d020 con 0x7fa4f415d4f0 2023-08-27T11:13:21.569 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:21.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4d806a0e0 msgr2=0x7fa4d806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4d806a0e0 0x7fa4d806c5a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40ce0d0 tx=0x7fa4e8008040 comp rx=0 tx=0).stop 2023-08-27T11:13:21.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f415d4f0 msgr2=0x7fa4f41e7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41e7660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00cc30 tx=0x7fa4ec00cf40 comp rx=0 tx=0).stop 2023-08-27T11:13:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 shutdown_connections 2023-08-27T11:13:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f415d4f0 0x7fa4f41e7660 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4d806a0e0 0x7fa4d806c5a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa4f40aea60 0x7fa4f40af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 --2- 172.21.15.45:0/3366784656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f40aefb0 0x7fa4f415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 >> 172.21.15.45:0/3366784656 conn(0x7fa4f40cc7e0 msgr2=0x7fa4f41e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 shutdown_connections 2023-08-27T11:13:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.570+0000 7fa4fc3a9700 1 -- 172.21.15.45:0/3366784656 wait complete. 2023-08-27T11:13:21.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:13:21.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.712+0000 7f74fbd12700 1 Processor -- start 2023-08-27T11:13:21.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.712+0000 7f74fbd12700 1 -- start start 2023-08-27T11:13:21.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74fbd12700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f74f4157be0 0x7f74f40716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74fbd12700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f40b17c0 0x7f74f40b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74fbd12700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f415b460 con 0x7f74f406ced0 2023-08-27T11:13:21.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74fbd12700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f4157fc0 con 0x7f74f4157be0 2023-08-27T11:13:21.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74fbd12700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40b2280 con 0x7f74f40b17c0 2023-08-27T11:13:21.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74fa2af700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f74f406ced0 0x7f74f40cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40444/0 (socket says 172.21.15.45:40444) 2023-08-27T11:13:21.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74fa2af700 1 -- 172.21.15.45:0/2932246492 learned_addr learned my addr 172.21.15.45:0/2932246492 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:21.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f92ad700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f40b17c0 0x7f74f40b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f9aae700 1 --2- 172.21.15.45:0/2932246492 >> v2:172.21.15.135:3300/0 conn(0x7f74f4157be0 0x7f74f40716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404003702 0 0) 0x7f74f415b460 con 0x7f74f406ced0 2023-08-27T11:13:21.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e00035b0 con 0x7f74f406ced0 2023-08-27T11:13:21.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3709504183 0 0) 0x7f74e00035b0 con 0x7f74f406ced0 2023-08-27T11:13:21.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f40b17c0 msgr2=0x7f74f40b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:21.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f40b17c0 0x7f74f40b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 >> v2:172.21.15.135:3300/0 conn(0x7f74f4157be0 msgr2=0x7f74f40716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:21.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 --2- 172.21.15.45:0/2932246492 >> v2:172.21.15.135:3300/0 conn(0x7f74f4157be0 0x7f74f40716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f40b2560 con 0x7f74f406ced0 2023-08-27T11:13:21.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f74ec0019f0 con 0x7f74f406ced0 2023-08-27T11:13:21.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 >> v1:172.21.15.45:6789/0 conn(0x7f74f406ced0 legacy=0x7f74f40cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f40b17c0 0x7f74e0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0003f80 0x7f74e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e0004d90 0x7f74e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f4157fc0 con 0x7f74e0004d90 2023-08-27T11:13:21.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40b2280 con 0x7f74e0003f80 2023-08-27T11:13:21.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.713+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e000ba10 con 0x7f74f40b17c0 2023-08-27T11:13:21.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74fa2af700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e0004d90 0x7f74e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74f9aae700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f40b17c0 0x7f74e0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74f92ad700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0003f80 0x7f74e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74fa2af700 1 -- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f40b17c0 msgr2=0x7f74e0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:21.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74fa2af700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f40b17c0 0x7f74e0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74fa2af700 1 -- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0003f80 msgr2=0x7f74e0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:21.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74fa2af700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0003f80 0x7f74e0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74fa2af700 1 -- 172.21.15.45:0/2932246492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00035b0 con 0x7f74e0004d90 2023-08-27T11:13:21.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74f92ad700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0003f80 0x7f74e0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:21.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74fa2af700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e0004d90 0x7f74e0005210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00a8d0 tx=0x7f74e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99117c6c31e2e00 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:21.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00a340 con 0x7f74e0004d90 2023-08-27T11:13:21.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec012600 con 0x7f74e0004d90 2023-08-27T11:13:21.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.714+0000 7f74f8aac700 1 -- 172.21.15.45:0/2932246492 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec0129b0 con 0x7f74e0004d90 2023-08-27T11:13:21.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.715+0000 7f74fbd12700 1 -- 172.21.15.45:0/2932246492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e0004d90 msgr2=0x7f74e0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.715+0000 7f74fbd12700 1 --2- 172.21.15.45:0/2932246492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74e0004d90 0x7f74e0005210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00a8d0 tx=0x7f74e0005880 comp rx=0 tx=0).stop 2023-08-27T11:13:21.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.715+0000 7f74fa2af700 1 -- 172.21.15.45:0/2932246492 reap_dead start 2023-08-27T11:13:21.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.715+0000 7f74fbd12700 1 -- 172.21.15.45:0/2932246492 shutdown_connections 2023-08-27T11:13:21.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.715+0000 7f74fbd12700 1 -- 172.21.15.45:0/2932246492 >> 172.21.15.45:0/2932246492 conn(0x7f74f40cc7c0 msgr2=0x7f74f40ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:21.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.715+0000 7f74fbd12700 1 -- 172.21.15.45:0/2932246492 shutdown_connections 2023-08-27T11:13:21.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.715+0000 7f74fbd12700 1 -- 172.21.15.45:0/2932246492 wait complete. 2023-08-27T11:13:21.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.716+0000 7f74fbd12700 1 Processor -- start 2023-08-27T11:13:21.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.716+0000 7f74fbd12700 1 -- start start 2023-08-27T11:13:21.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fbd12700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0004d90 0x7f74f414c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fbd12700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f4154bc0 0x7f74f4155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fbd12700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f40cb520 0x7f74f40cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fbd12700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f414c7e0 con 0x7f74f40cb520 2023-08-27T11:13:21.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fbd12700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40cbef0 con 0x7f74e0004d90 2023-08-27T11:13:21.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fbd12700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40c8630 con 0x7f74f4154bc0 2023-08-27T11:13:21.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fa2af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f40cb520 0x7f74f40cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fa2af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f40cb520 0x7f74f40cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53680/0 (socket says 172.21.15.45:53680) 2023-08-27T11:13:21.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fa2af700 1 -- 172.21.15.45:0/2214608361 learned_addr learned my addr 172.21.15.45:0/2214608361 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74f92ad700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f4154bc0 0x7f74f4155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74f9aae700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0004d90 0x7f74f414c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fa2af700 1 -- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f4154bc0 msgr2=0x7f74f4155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fa2af700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f4154bc0 0x7f74f4155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fa2af700 1 -- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0004d90 msgr2=0x7f74f414c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fa2af700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0004d90 0x7f74f414c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fa2af700 1 -- 172.21.15.45:0/2214608361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f4159b70 con 0x7f74f40cb520 2023-08-27T11:13:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74f9aae700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0004d90 0x7f74f414c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fa2af700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f40cb520 0x7f74f40cb9b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f74ec015040 tx=0x7f74ec001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74ea7fc700 1 -- 172.21.15.45:0/2214608361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec00a340 con 0x7f74f40cb520 2023-08-27T11:13:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74ea7fc700 1 -- 172.21.15.45:0/2214608361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec0019c0 con 0x7f74f40cb520 2023-08-27T11:13:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74ea7fc700 1 -- 172.21.15.45:0/2214608361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec026640 con 0x7f74f40cb520 2023-08-27T11:13:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.717+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f4159e50 con 0x7f74f40cb520 2023-08-27T11:13:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.718+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f40c7e30 con 0x7f74f40cb520 2023-08-27T11:13:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.718+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f4072690 con 0x7f74f40cb520 2023-08-27T11:13:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.719+0000 7f74ea7fc700 1 -- 172.21.15.45:0/2214608361 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f74ec019030 con 0x7f74f40cb520 2023-08-27T11:13:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.719+0000 7f74ea7fc700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f74d806a0e0 0x7f74d806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.719+0000 7f74ea7fc700 1 -- 172.21.15.45:0/2214608361 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f74ec08a250 con 0x7f74f40cb520 2023-08-27T11:13:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.720+0000 7f74f9aae700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f74d806a0e0 0x7f74d806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.720+0000 7f74f9aae700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f74d806a0e0 0x7f74d806c5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f74f40cad10 tx=0x7f74f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.724+0000 7f74ea7fc700 1 -- 172.21.15.45:0/2214608361 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f74ec055720 con 0x7f74f40cb520 2023-08-27T11:13:21.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.924+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74f4070e80 con 0x7f74f40cb520 2023-08-27T11:13:21.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.924+0000 7f74ea7fc700 1 -- 172.21.15.45:0/2214608361 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74ec058d40 con 0x7f74f40cb520 2023-08-27T11:13:21.925 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:13:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f74d806a0e0 msgr2=0x7f74d806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f74d806a0e0 0x7f74d806c5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f74f40cad10 tx=0x7f74f0008040 comp rx=0 tx=0).stop 2023-08-27T11:13:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f40cb520 msgr2=0x7f74f40cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f40cb520 0x7f74f40cb9b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f74ec015040 tx=0x7f74ec001360 comp rx=0 tx=0).stop 2023-08-27T11:13:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 shutdown_connections 2023-08-27T11:13:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f40cb520 0x7f74f40cb9b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f74d806a0e0 0x7f74d806c5a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74e0004d90 0x7f74f414c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 --2- 172.21.15.45:0/2214608361 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74f4154bc0 0x7f74f4155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 >> 172.21.15.45:0/2214608361 conn(0x7f74f40cc7c0 msgr2=0x7f74f40cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 shutdown_connections 2023-08-27T11:13:21.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:21.926+0000 7f74fbd12700 1 -- 172.21.15.45:0/2214608361 wait complete. 2023-08-27T11:13:21.939 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-27T11:13:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.066+0000 7f0854fa6700 1 Processor -- start 2023-08-27T11:13:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.066+0000 7f0854fa6700 1 -- start start 2023-08-27T11:13:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f0854fa6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085015b2e0 0x7f08500cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f0854fa6700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f08500cf950 0x7f08500cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f0854fa6700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08500cb9b0 con 0x7f08500cd540 2023-08-27T11:13:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f0854fa6700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08500ce2e0 con 0x7f08500cf950 2023-08-27T11:13:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f0854fa6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08500cc230 con 0x7f085015b2e0 2023-08-27T11:13:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f084ed9d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f08500cf950 0x7f08500cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f084e59c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085015b2e0 0x7f08500cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f084e59c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085015b2e0 0x7f08500cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:51594/0 (socket says 172.21.15.45:51594) 2023-08-27T11:13:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f084ed9d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f08500cf950 0x7f08500cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:42544/0 (socket says 172.21.15.45:42544) 2023-08-27T11:13:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.067+0000 7f084e59c700 1 -- 172.21.15.45:0/3163340018 learned_addr learned my addr 172.21.15.45:0/3163340018 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.068+0000 7f084e59c700 1 -- 172.21.15.45:0/3163340018 >> v2:172.21.15.135:3300/0 conn(0x7f08500cf950 msgr2=0x7f08500cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:22.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.068+0000 7f084e59c700 1 --2- 172.21.15.45:0/3163340018 >> v2:172.21.15.135:3300/0 conn(0x7f08500cf950 0x7f08500cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.068+0000 7f084e59c700 1 -- 172.21.15.45:0/3163340018 >> v1:172.21.15.135:6789/0 conn(0x7f08500cd540 legacy=0x7f08500cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.068+0000 7f084e59c700 1 -- 172.21.15.45:0/3163340018 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08500ce000 con 0x7f085015b2e0 2023-08-27T11:13:22.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.068+0000 7f084e59c700 1 --2- 172.21.15.45:0/3163340018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085015b2e0 0x7f08500cc6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08440098f0 tx=0x7f0844009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce2d33e4bac25fa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:22.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.068+0000 7f084d59a700 1 -- 172.21.15.45:0/3163340018 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844011830 con 0x7f085015b2e0 2023-08-27T11:13:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.068+0000 7f084d59a700 1 -- 172.21.15.45:0/3163340018 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0844011990 con 0x7f085015b2e0 2023-08-27T11:13:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.068+0000 7f084d59a700 1 -- 172.21.15.45:0/3163340018 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844011d20 con 0x7f085015b2e0 2023-08-27T11:13:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.069+0000 7f0854fa6700 1 -- 172.21.15.45:0/3163340018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085015b2e0 msgr2=0x7f08500cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.069+0000 7f0854fa6700 1 --2- 172.21.15.45:0/3163340018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085015b2e0 0x7f08500cc6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08440098f0 tx=0x7f0844009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:22.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.069+0000 7f0854fa6700 1 -- 172.21.15.45:0/3163340018 shutdown_connections 2023-08-27T11:13:22.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.069+0000 7f0854fa6700 1 --2- 172.21.15.45:0/3163340018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f085015b2e0 0x7f08500cc6c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.069+0000 7f0854fa6700 1 --2- 172.21.15.45:0/3163340018 >> v2:172.21.15.135:3300/0 conn(0x7f08500cf950 0x7f08500cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.069+0000 7f0854fa6700 1 -- 172.21.15.45:0/3163340018 >> 172.21.15.45:0/3163340018 conn(0x7f085009a1a0 msgr2=0x7f08500d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:22.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.069+0000 7f0854fa6700 1 -- 172.21.15.45:0/3163340018 shutdown_connections 2023-08-27T11:13:22.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.069+0000 7f0854fa6700 1 -- 172.21.15.45:0/3163340018 wait complete. 2023-08-27T11:13:22.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.070+0000 7f0854fa6700 1 Processor -- start 2023-08-27T11:13:22.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.070+0000 7f0854fa6700 1 -- start start 2023-08-27T11:13:22.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f0854fa6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08500cd540 0x7f085015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f0854fa6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08500cf950 0x7f0850155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f0854fa6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085015b2e0 0x7f08500b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f0854fa6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08500c49e0 con 0x7f08500cf950 2023-08-27T11:13:22.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f0854fa6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085015c1d0 con 0x7f085015b2e0 2023-08-27T11:13:22.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f0854fa6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085015c4b0 con 0x7f08500cd540 2023-08-27T11:13:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f084dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08500cf950 0x7f0850155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f084dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08500cf950 0x7f0850155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53692/0 (socket says 172.21.15.45:53692) 2023-08-27T11:13:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f084dd9b700 1 -- 172.21.15.45:0/2532425194 learned_addr learned my addr 172.21.15.45:0/2532425194 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f084e59c700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08500cd540 0x7f085015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.071+0000 7f084ed9d700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085015b2e0 0x7f08500b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f084dd9b700 1 -- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08500cd540 msgr2=0x7f085015b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f084dd9b700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08500cd540 0x7f085015b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f084dd9b700 1 -- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085015b2e0 msgr2=0x7f08500b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f084dd9b700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085015b2e0 0x7f08500b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f084dd9b700 1 -- 172.21.15.45:0/2532425194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0844009550 con 0x7f08500cf950 2023-08-27T11:13:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f084dd9b700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08500cf950 0x7f0850155260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f084000c450 tx=0x7f084000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f083effd700 1 -- 172.21.15.45:0/2532425194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08400052d0 con 0x7f08500cf950 2023-08-27T11:13:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f083effd700 1 -- 172.21.15.45:0/2532425194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0840005430 con 0x7f08500cf950 2023-08-27T11:13:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08500c4c50 con 0x7f08500cf950 2023-08-27T11:13:22.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f083effd700 1 -- 172.21.15.45:0/2532425194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08400038c0 con 0x7f08500cf950 2023-08-27T11:13:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.072+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085015be10 con 0x7f08500cf950 2023-08-27T11:13:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.073+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08500cbea0 con 0x7f08500cf950 2023-08-27T11:13:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.073+0000 7f083effd700 1 -- 172.21.15.45:0/2532425194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0840003a20 con 0x7f08500cf950 2023-08-27T11:13:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.074+0000 7f083effd700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f083006a0e0 0x7f083006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.074+0000 7f083effd700 1 -- 172.21.15.45:0/2532425194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0840087b50 con 0x7f08500cf950 2023-08-27T11:13:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.074+0000 7f084e59c700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f083006a0e0 0x7f083006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.074+0000 7f084e59c700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f083006a0e0 0x7f083006c5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0844009c40 tx=0x7f084401d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:22.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.078+0000 7f083effd700 1 -- 172.21.15.45:0/2532425194 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08500cbea0 con 0x7f08500cf950 2023-08-27T11:13:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.280+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f08500cbe70 con 0x7f08500cf950 2023-08-27T11:13:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.281+0000 7f083effd700 1 -- 172.21.15.45:0/2532425194 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0840017020 con 0x7f08500cf950 2023-08-27T11:13:22.282 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:13:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f083006a0e0 msgr2=0x7f083006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f083006a0e0 0x7f083006c5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0844009c40 tx=0x7f084401d040 comp rx=0 tx=0).stop 2023-08-27T11:13:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08500cf950 msgr2=0x7f0850155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08500cf950 0x7f0850155260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f084000c450 tx=0x7f084000c760 comp rx=0 tx=0).stop 2023-08-27T11:13:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 shutdown_connections 2023-08-27T11:13:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f083006a0e0 0x7f083006c5a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f085015b2e0 0x7f08500b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f08500cd540 0x7f085015b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 --2- 172.21.15.45:0/2532425194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08500cf950 0x7f0850155260 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 >> 172.21.15.45:0/2532425194 conn(0x7f085009a1a0 msgr2=0x7f08500b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 shutdown_connections 2023-08-27T11:13:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.283+0000 7f0854fa6700 1 -- 172.21.15.45:0/2532425194 wait complete. 2023-08-27T11:13:22.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:13:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.419+0000 7ff0b7b17700 1 Processor -- start 2023-08-27T11:13:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.420+0000 7ff0b7b17700 1 -- start start 2023-08-27T11:13:22.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.420+0000 7ff0b7b17700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff0b015dbc0 0x7ff0b015a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.420+0000 7ff0b7b17700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0b015a750 0x7ff0b00cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.420+0000 7ff0b7b17700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b00caef0 con 0x7ff0b0070e40 2023-08-27T11:13:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.420+0000 7ff0b7b17700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b00cac50 con 0x7ff0b015a750 2023-08-27T11:13:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.420+0000 7ff0b7b17700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b00ce000 con 0x7ff0b015dbc0 2023-08-27T11:13:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b60b4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0b015a750 0x7ff0b00cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b58b3700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff0b0070e40 0x7ff0b0071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40474/0 (socket says 172.21.15.45:40474) 2023-08-27T11:13:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b58b3700 1 -- 172.21.15.45:0/10875968 learned_addr learned my addr 172.21.15.45:0/10875968 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b60b4700 1 -- 172.21.15.45:0/10875968 >> v2:172.21.15.135:3300/0 conn(0x7ff0b015dbc0 msgr2=0x7ff0b015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b60b4700 1 --2- 172.21.15.45:0/10875968 >> v2:172.21.15.135:3300/0 conn(0x7ff0b015dbc0 0x7ff0b015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b60b4700 1 -- 172.21.15.45:0/10875968 >> v1:172.21.15.45:6789/0 conn(0x7ff0b0070e40 legacy=0x7ff0b0071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b60b4700 1 -- 172.21.15.45:0/10875968 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b015b5c0 con 0x7ff0b015a750 2023-08-27T11:13:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b48b1700 1 -- 172.21.15.45:0/10875968 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607158280 0 0) 0x7ff0b00caef0 con 0x7ff0b0070e40 2023-08-27T11:13:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b60b4700 1 --2- 172.21.15.45:0/10875968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015a750 0x7ff0b00cb380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009d90 tx=0x7ff0a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb68738c7f6a93a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b48b1700 1 -- 172.21.15.45:0/10875968 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a800e070 con 0x7ff0b015a750 2023-08-27T11:13:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b48b1700 1 -- 172.21.15.45:0/10875968 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a800bb90 con 0x7ff0b015a750 2023-08-27T11:13:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.421+0000 7ff0b48b1700 1 -- 172.21.15.45:0/10875968 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8012430 con 0x7ff0b015a750 2023-08-27T11:13:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.422+0000 7ff0b7b17700 1 -- 172.21.15.45:0/10875968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015a750 msgr2=0x7ff0b00cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.422+0000 7ff0b7b17700 1 --2- 172.21.15.45:0/10875968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015a750 0x7ff0b00cb380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009d90 tx=0x7ff0a800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:22.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.422+0000 7ff0b7b17700 1 -- 172.21.15.45:0/10875968 shutdown_connections 2023-08-27T11:13:22.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.422+0000 7ff0b7b17700 1 --2- 172.21.15.45:0/10875968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015a750 0x7ff0b00cb380 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.422+0000 7ff0b7b17700 1 --2- 172.21.15.45:0/10875968 >> v2:172.21.15.135:3300/0 conn(0x7ff0b015dbc0 0x7ff0b015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.422+0000 7ff0b7b17700 1 -- 172.21.15.45:0/10875968 >> 172.21.15.45:0/10875968 conn(0x7ff0b00cfd60 msgr2=0x7ff0b00d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:22.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.422+0000 7ff0b7b17700 1 -- 172.21.15.45:0/10875968 shutdown_connections 2023-08-27T11:13:22.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.422+0000 7ff0b7b17700 1 -- 172.21.15.45:0/10875968 wait complete. 2023-08-27T11:13:22.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.423+0000 7ff0b7b17700 1 Processor -- start 2023-08-27T11:13:22.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.424+0000 7ff0b7b17700 1 -- start start 2023-08-27T11:13:22.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.424+0000 7ff0b7b17700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0b0070e40 0x7ff0b0158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.424+0000 7ff0b7b17700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b015a750 0x7ff0b00a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.424+0000 7ff0b7b17700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015dbc0 0x7ff0b01f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b7b17700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01f5040 con 0x7ff0b015dbc0 2023-08-27T11:13:22.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b7b17700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01f4a80 con 0x7ff0b015a750 2023-08-27T11:13:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b7b17700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01f4d60 con 0x7ff0b0070e40 2023-08-27T11:13:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b60b4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015dbc0 0x7ff0b01f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b60b4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015dbc0 0x7ff0b01f40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53708/0 (socket says 172.21.15.45:53708) 2023-08-27T11:13:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b60b4700 1 -- 172.21.15.45:0/1559821240 learned_addr learned my addr 172.21.15.45:0/1559821240 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:22.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b58b3700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0b0070e40 0x7ff0b0158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b50b2700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b015a750 0x7ff0b00a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b60b4700 1 -- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0b0070e40 msgr2=0x7ff0b0158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b60b4700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0b0070e40 0x7ff0b0158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b60b4700 1 -- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b015a750 msgr2=0x7ff0b00a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b60b4700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b015a750 0x7ff0b00a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b60b4700 1 -- 172.21.15.45:0/1559821240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8009a70 con 0x7ff0b015dbc0 2023-08-27T11:13:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b50b2700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b015a750 0x7ff0b00a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b60b4700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015dbc0 0x7ff0b01f40a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0072b20 tx=0x7ff0a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0a27fc700 1 -- 172.21.15.45:0/1559821240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8012430 con 0x7ff0b015dbc0 2023-08-27T11:13:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0a27fc700 1 -- 172.21.15.45:0/1559821240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8020070 con 0x7ff0b015dbc0 2023-08-27T11:13:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0a27fc700 1 -- 172.21.15.45:0/1559821240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8004280 con 0x7ff0b015dbc0 2023-08-27T11:13:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b00a8640 con 0x7ff0b015dbc0 2023-08-27T11:13:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.425+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b0158bc0 con 0x7ff0b015dbc0 2023-08-27T11:13:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.426+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b0072180 con 0x7ff0b015dbc0 2023-08-27T11:13:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.427+0000 7ff0a27fc700 1 -- 172.21.15.45:0/1559821240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff0a80043e0 con 0x7ff0b015dbc0 2023-08-27T11:13:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.427+0000 7ff0a27fc700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff09406a0e0 0x7ff09406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.427+0000 7ff0a27fc700 1 -- 172.21.15.45:0/1559821240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff0a8022070 con 0x7ff0b015dbc0 2023-08-27T11:13:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.427+0000 7ff0b58b3700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff09406a0e0 0x7ff09406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.428+0000 7ff0b58b3700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff09406a0e0 0x7ff09406c5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac000f60 tx=0x7ff0ac007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.431+0000 7ff0a27fc700 1 -- 172.21.15.45:0/1559821240 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff0a8055b90 con 0x7ff0b015dbc0 2023-08-27T11:13:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.681+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0b00a84a0 con 0x7ff0b015dbc0 2023-08-27T11:13:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.682+0000 7ff0a27fc700 1 -- 172.21.15.45:0/1559821240 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7ff0b00a84a0 con 0x7ff0b015dbc0 2023-08-27T11:13:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:22.683 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:33.983169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:13:22.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff09406a0e0 msgr2=0x7ff09406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff09406a0e0 0x7ff09406c5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac000f60 tx=0x7ff0ac007c90 comp rx=0 tx=0).stop 2023-08-27T11:13:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015dbc0 msgr2=0x7ff0b01f40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015dbc0 0x7ff0b01f40a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0072b20 tx=0x7ff0a8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 shutdown_connections 2023-08-27T11:13:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0b015dbc0 0x7ff0b01f40a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7ff09406a0e0 0x7ff09406c5a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0b0070e40 0x7ff0b0158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 --2- 172.21.15.45:0/1559821240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0b015a750 0x7ff0b00a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 >> 172.21.15.45:0/1559821240 conn(0x7ff0b00cfd60 msgr2=0x7ff0b00b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 shutdown_connections 2023-08-27T11:13:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.684+0000 7ff0b7b17700 1 -- 172.21.15.45:0/1559821240 wait complete. 2023-08-27T11:13:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-08-27T11:13:22.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.821+0000 7f1c9a83d700 1 Processor -- start 2023-08-27T11:13:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.822+0000 7f1c9a83d700 1 -- start start 2023-08-27T11:13:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.822+0000 7f1c9a83d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1c9415d3a0 0x7f1c940d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.822+0000 7f1c9a83d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9415d780 con 0x7f1c940d02c0 2023-08-27T11:13:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.822+0000 7f1c9a83d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c940cfe30 con 0x7f1c9406c4b0 2023-08-27T11:13:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.822+0000 7f1c9a83d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c940cf220 con 0x7f1c9415d3a0 2023-08-27T11:13:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.822+0000 7f1c937fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1c940d02c0 0x7f1c9406bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40490/0 (socket says 172.21.15.45:40490) 2023-08-27T11:13:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.822+0000 7f1c937fe700 1 -- 172.21.15.45:0/2758941555 learned_addr learned my addr 172.21.15.45:0/2758941555 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c93fff700 1 --2- 172.21.15.45:0/2758941555 >> v2:172.21.15.135:3300/0 conn(0x7f1c9415d3a0 0x7f1c940d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446760698 0 0) 0x7f1c88001110 con 0x7f1c940d02c0 2023-08-27T11:13:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c7c0035b0 con 0x7f1c940d02c0 2023-08-27T11:13:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1405496124 0 0) 0x7f1c7c0035b0 con 0x7f1c940d02c0 2023-08-27T11:13:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 >> v2:172.21.15.135:3300/0 conn(0x7f1c9415d3a0 msgr2=0x7f1c940d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 --2- 172.21.15.45:0/2758941555 >> v2:172.21.15.135:3300/0 conn(0x7f1c9415d3a0 0x7f1c940d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 >> v1:172.21.15.135:6789/0 conn(0x7f1c9406c4b0 legacy=0x7f1c940cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940cf500 con 0x7f1c940d02c0 2023-08-27T11:13:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f1c88001800 con 0x7f1c940d02c0 2023-08-27T11:13:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 >> v1:172.21.15.45:6789/0 conn(0x7f1c940d02c0 legacy=0x7f1c9406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c7c003a00 0x7f1c7c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c7c004850 0x7f1c7c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c7c005210 0x7f1c7c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940cf220 con 0x7f1c7c004850 2023-08-27T11:13:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c00c0b0 con 0x7f1c7c003a00 2023-08-27T11:13:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c00c270 con 0x7f1c7c005210 2023-08-27T11:13:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c937fe700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c7c004850 0x7f1c7c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c93fff700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c7c003a00 0x7f1c7c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c98dda700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c7c005210 0x7f1c7c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c937fe700 1 -- 172.21.15.45:0/2758941555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c7c005210 msgr2=0x7f1c7c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c937fe700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c7c005210 0x7f1c7c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c937fe700 1 -- 172.21.15.45:0/2758941555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c7c003a00 msgr2=0x7f1c7c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c937fe700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c7c003a00 0x7f1c7c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c937fe700 1 -- 172.21.15.45:0/2758941555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0035b0 con 0x7f1c7c004850 2023-08-27T11:13:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.823+0000 7f1c98dda700 1 -- 172.21.15.45:0/2758941555 reap_dead start 2023-08-27T11:13:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c937fe700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c7c004850 0x7f1c7c004cd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800c240 tx=0x7f1c7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d4570819eb9bdf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88007db0 con 0x7f1c7c004850 2023-08-27T11:13:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8800ce00 con 0x7f1c7c004850 2023-08-27T11:13:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c92ffd700 1 -- 172.21.15.45:0/2758941555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88005ba0 con 0x7f1c7c004850 2023-08-27T11:13:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2758941555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c7c004850 msgr2=0x7f1c7c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c9a83d700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c7c004850 0x7f1c7c004cd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800c240 tx=0x7f1c7c005880 comp rx=0 tx=0).stop 2023-08-27T11:13:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2758941555 shutdown_connections 2023-08-27T11:13:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c9a83d700 1 --2- 172.21.15.45:0/2758941555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c7c004850 0x7f1c7c004cd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2758941555 >> 172.21.15.45:0/2758941555 conn(0x7f1c9409a1a0 msgr2=0x7f1c940d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2758941555 shutdown_connections 2023-08-27T11:13:22.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.824+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2758941555 wait complete. 2023-08-27T11:13:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.826+0000 7f1c9a83d700 1 Processor -- start 2023-08-27T11:13:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.826+0000 7f1c9a83d700 1 -- start start 2023-08-27T11:13:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c9a83d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c7c004850 0x7f1c9414a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c9a83d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c940b4cf0 0x7f1c940b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c9a83d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c940b6270 0x7f1c9415e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c9a83d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940b5de0 con 0x7f1c940b4cf0 2023-08-27T11:13:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c9a83d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940b3740 con 0x7f1c7c004850 2023-08-27T11:13:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c9a83d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940b51d0 con 0x7f1c940b6270 2023-08-27T11:13:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c940b4cf0 0x7f1c940b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c940b4cf0 0x7f1c940b3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53726/0 (socket says 172.21.15.45:53726) 2023-08-27T11:13:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c937fe700 1 -- 172.21.15.45:0/2826571706 learned_addr learned my addr 172.21.15.45:0/2826571706 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c98dda700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c940b6270 0x7f1c9415e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c93fff700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c7c004850 0x7f1c9414a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c937fe700 1 -- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c940b6270 msgr2=0x7f1c9415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c937fe700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c940b6270 0x7f1c9415e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c937fe700 1 -- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c7c004850 msgr2=0x7f1c9414a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c937fe700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c7c004850 0x7f1c9414a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c937fe700 1 -- 172.21.15.45:0/2826571706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940b54b0 con 0x7f1c940b4cf0 2023-08-27T11:13:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c937fe700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c940b4cf0 0x7f1c940b3200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00c080 tx=0x7f1c88001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c90ff9700 1 -- 172.21.15.45:0/2826571706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c880017c0 con 0x7f1c940b4cf0 2023-08-27T11:13:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c90ff9700 1 -- 172.21.15.45:0/2826571706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c88007d50 con 0x7f1c940b4cf0 2023-08-27T11:13:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c940b5790 con 0x7f1c940b4cf0 2023-08-27T11:13:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c90ff9700 1 -- 172.21.15.45:0/2826571706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88007310 con 0x7f1c940b4cf0 2023-08-27T11:13:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.827+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c94158b40 con 0x7f1c940b4cf0 2023-08-27T11:13:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.828+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c940d0c40 con 0x7f1c940b4cf0 2023-08-27T11:13:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.829+0000 7f1c90ff9700 1 -- 172.21.15.45:0/2826571706 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1c88007470 con 0x7f1c940b4cf0 2023-08-27T11:13:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.829+0000 7f1c90ff9700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8006a110 0x7f1c8006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.829+0000 7f1c90ff9700 1 -- 172.21.15.45:0/2826571706 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1c88089a90 con 0x7f1c940b4cf0 2023-08-27T11:13:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.830+0000 7f1c93fff700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8006a110 0x7f1c8006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.830+0000 7f1c93fff700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8006a110 0x7f1c8006c5d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009c20 tx=0x7f1c84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:22.834+0000 7f1c90ff9700 1 -- 172.21.15.45:0/2826571706 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1c88055000 con 0x7f1c940b4cf0 2023-08-27T11:13:23.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.037+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c9415d780 con 0x7f1c940b4cf0 2023-08-27T11:13:23.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.038+0000 7f1c90ff9700 1 -- 172.21.15.45:0/2826571706 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7f1c9415d780 con 0x7f1c940b4cf0 2023-08-27T11:13:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:23.040 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:23.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8006a110 msgr2=0x7f1c8006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8006a110 0x7f1c8006c5d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009c20 tx=0x7f1c84008040 comp rx=0 tx=0).stop 2023-08-27T11:13:23.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c940b4cf0 msgr2=0x7f1c940b3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c940b4cf0 0x7f1c940b3200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00c080 tx=0x7f1c88001b90 comp rx=0 tx=0).stop 2023-08-27T11:13:23.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 shutdown_connections 2023-08-27T11:13:23.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8006a110 0x7f1c8006c5d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c940b6270 0x7f1c9415e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c7c004850 0x7f1c9414a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 --2- 172.21.15.45:0/2826571706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c940b4cf0 0x7f1c940b3200 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.040+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 >> 172.21.15.45:0/2826571706 conn(0x7f1c9409a1a0 msgr2=0x7f1c940af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:23.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.041+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 shutdown_connections 2023-08-27T11:13:23.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.041+0000 7f1c9a83d700 1 -- 172.21.15.45:0/2826571706 wait complete. 2023-08-27T11:13:23.055 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:13:23.055 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:13:23.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.180+0000 7fcba4fae700 1 Processor -- start 2023-08-27T11:13:23.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.181+0000 7fcba4fae700 1 -- start start 2023-08-27T11:13:23.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.181+0000 7fcba4fae700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006cec0 0x7fcba006e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.181+0000 7fcba4fae700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcba006e7b0 0x7fcba00ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcba4fae700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba006ec00 con 0x7fcba015b070 2023-08-27T11:13:23.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcba4fae700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba015d8e0 con 0x7fcba006e7b0 2023-08-27T11:13:23.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcba4fae700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba015dad0 con 0x7fcba006cec0 2023-08-27T11:13:23.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcba006e7b0 0x7fcba00ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcba006e7b0 0x7fcba00ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53736/0 (socket says 172.21.15.45:53736) 2023-08-27T11:13:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9ed9d700 1 -- 172.21.15.45:0/1463103233 learned_addr learned my addr 172.21.15.45:0/1463103233 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:23.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9dd9b700 1 --2- 172.21.15.45:0/1463103233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006cec0 0x7fcba006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9ed9d700 1 -- 172.21.15.45:0/1463103233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006cec0 msgr2=0x7fcba006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:23.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9ed9d700 1 --2- 172.21.15.45:0/1463103233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006cec0 0x7fcba006e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9ed9d700 1 -- 172.21.15.45:0/1463103233 >> v1:172.21.15.135:6789/0 conn(0x7fcba015b070 legacy=0x7fcba00cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9ed9d700 1 -- 172.21.15.45:0/1463103233 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba015ddb0 con 0x7fcba006e7b0 2023-08-27T11:13:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9ed9d700 1 --2- 172.21.15.45:0/1463103233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba006e7b0 0x7fcba00ae8d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009e50 tx=0x7fcb9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405e9ba60500389c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:23.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9d59a700 1 -- 172.21.15.45:0/1463103233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800e070 con 0x7fcba006e7b0 2023-08-27T11:13:23.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9d59a700 1 -- 172.21.15.45:0/1463103233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb98003a20 con 0x7fcba006e7b0 2023-08-27T11:13:23.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.182+0000 7fcb9d59a700 1 -- 172.21.15.45:0/1463103233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98003dd0 con 0x7fcba006e7b0 2023-08-27T11:13:23.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.183+0000 7fcba4fae700 1 -- 172.21.15.45:0/1463103233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba006e7b0 msgr2=0x7fcba00ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.183+0000 7fcba4fae700 1 --2- 172.21.15.45:0/1463103233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba006e7b0 0x7fcba00ae8d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009e50 tx=0x7fcb9800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:23.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.183+0000 7fcba4fae700 1 -- 172.21.15.45:0/1463103233 shutdown_connections 2023-08-27T11:13:23.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.183+0000 7fcba4fae700 1 --2- 172.21.15.45:0/1463103233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba006e7b0 0x7fcba00ae8d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.183+0000 7fcba4fae700 1 --2- 172.21.15.45:0/1463103233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006cec0 0x7fcba006e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.183+0000 7fcba4fae700 1 -- 172.21.15.45:0/1463103233 >> 172.21.15.45:0/1463103233 conn(0x7fcba00cfe50 msgr2=0x7fcba00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:23.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.183+0000 7fcba4fae700 1 -- 172.21.15.45:0/1463103233 shutdown_connections 2023-08-27T11:13:23.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.183+0000 7fcba4fae700 1 -- 172.21.15.45:0/1463103233 wait complete. 2023-08-27T11:13:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.184+0000 7fcba4fae700 1 Processor -- start 2023-08-27T11:13:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.184+0000 7fcba4fae700 1 -- start start 2023-08-27T11:13:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcba4fae700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcba006cec0 0x7fcba01482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcba4fae700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006e7b0 0x7fcba00c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcba4fae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba015b070 0x7fcba00b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcba4fae700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c4fd0 con 0x7fcba015b070 2023-08-27T11:13:23.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcba4fae700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0148820 con 0x7fcba006cec0 2023-08-27T11:13:23.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcba4fae700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c4400 con 0x7fcba006e7b0 2023-08-27T11:13:23.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba015b070 0x7fcba00b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba015b070 0x7fcba00b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53742/0 (socket says 172.21.15.45:53742) 2023-08-27T11:13:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9ed9d700 1 -- 172.21.15.45:0/2570721036 learned_addr learned my addr 172.21.15.45:0/2570721036 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9dd9b700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006e7b0 0x7fcba00c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9e59c700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcba006cec0 0x7fcba01482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9ed9d700 1 -- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006e7b0 msgr2=0x7fcba00c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9ed9d700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006e7b0 0x7fcba00c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9ed9d700 1 -- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcba006cec0 msgr2=0x7fcba01482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9ed9d700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcba006cec0 0x7fcba01482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9ed9d700 1 -- 172.21.15.45:0/2570721036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb98009b30 con 0x7fcba015b070 2023-08-27T11:13:23.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.185+0000 7fcb9e59c700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcba006cec0 0x7fcba01482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.186+0000 7fcb9ed9d700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba015b070 0x7fcba00b4510 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcba006c730 tx=0x7fcb98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.186+0000 7fcb96ffd700 1 -- 172.21.15.45:0/2570721036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98004440 con 0x7fcba015b070 2023-08-27T11:13:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.186+0000 7fcb96ffd700 1 -- 172.21.15.45:0/2570721036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb980045c0 con 0x7fcba015b070 2023-08-27T11:13:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.186+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba00c46a0 con 0x7fcba015b070 2023-08-27T11:13:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.186+0000 7fcb96ffd700 1 -- 172.21.15.45:0/2570721036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98004970 con 0x7fcba015b070 2023-08-27T11:13:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.186+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba00c4c60 con 0x7fcba015b070 2023-08-27T11:13:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.187+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba00c5180 con 0x7fcba015b070 2023-08-27T11:13:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.187+0000 7fcb96ffd700 1 -- 172.21.15.45:0/2570721036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcb98025450 con 0x7fcba015b070 2023-08-27T11:13:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.188+0000 7fcb96ffd700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcb8006a0e0 0x7fcb8006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.188+0000 7fcb96ffd700 1 -- 172.21.15.45:0/2570721036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcb98089e70 con 0x7fcba015b070 2023-08-27T11:13:23.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.188+0000 7fcb9e59c700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcb8006a0e0 0x7fcb8006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.188+0000 7fcb9e59c700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcb8006a0e0 0x7fcb8006c5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009930 tx=0x7fcb90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:23.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.192+0000 7fcb96ffd700 1 -- 172.21.15.45:0/2570721036 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcba00c5180 con 0x7fcba015b070 2023-08-27T11:13:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.434+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcba00c5360 con 0x7fcba015b070 2023-08-27T11:13:23.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.434+0000 7fcb96ffd700 1 -- 172.21.15.45:0/2570721036 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fcba00c5360 con 0x7fcba015b070 2023-08-27T11:13:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:23.436 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:33.983169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:13:23.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcb8006a0e0 msgr2=0x7fcb8006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcb8006a0e0 0x7fcb8006c5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009930 tx=0x7fcb90008040 comp rx=0 tx=0).stop 2023-08-27T11:13:23.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba015b070 msgr2=0x7fcba00b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba015b070 0x7fcba00b4510 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcba006c730 tx=0x7fcb98004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 shutdown_connections 2023-08-27T11:13:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba015b070 0x7fcba00b4510 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcb8006a0e0 0x7fcb8006c5a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcba006cec0 0x7fcba01482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 --2- 172.21.15.45:0/2570721036 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcba006e7b0 0x7fcba00c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 >> 172.21.15.45:0/2570721036 conn(0x7fcba00cfe50 msgr2=0x7fcba015f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:23.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 shutdown_connections 2023-08-27T11:13:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.436+0000 7fcba4fae700 1 -- 172.21.15.45:0/2570721036 wait complete. 2023-08-27T11:13:23.440 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-08-27T11:13:23.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:13:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.574+0000 7f1099388700 1 Processor -- start 2023-08-27T11:13:23.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.574+0000 7f1099388700 1 -- start start 2023-08-27T11:13:23.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.575+0000 7f1099388700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f10940ce4b0 0x7f1094167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.575+0000 7f1099388700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109415d6a0 con 0x7f10940b8c20 2023-08-27T11:13:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.575+0000 7f1099388700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10940d2c90 con 0x7f109415d2c0 2023-08-27T11:13:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.575+0000 7f1099388700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10940d1bf0 con 0x7f10940ce4b0 2023-08-27T11:13:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.575+0000 7f10927fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f10940b8c20 0x7f10940cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40524/0 (socket says 172.21.15.45:40524) 2023-08-27T11:13:23.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.575+0000 7f10927fc700 1 -- 172.21.15.45:0/136783831 learned_addr learned my addr 172.21.15.45:0/136783831 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:23.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.575+0000 7f10937fe700 1 --2- 172.21.15.45:0/136783831 >> v2:172.21.15.135:3300/0 conn(0x7f10940ce4b0 0x7f1094167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.575+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619254891 0 0) 0x7f1084001110 con 0x7f10940b8c20 2023-08-27T11:13:23.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.575+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f107c0035b0 con 0x7f10940b8c20 2023-08-27T11:13:23.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2054973893 0 0) 0x7f107c0035b0 con 0x7f10940b8c20 2023-08-27T11:13:23.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 >> v2:172.21.15.135:3300/0 conn(0x7f10940ce4b0 msgr2=0x7f1094167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:23.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 --2- 172.21.15.45:0/136783831 >> v2:172.21.15.135:3300/0 conn(0x7f10940ce4b0 0x7f1094167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 >> v1:172.21.15.135:6789/0 conn(0x7f109415d2c0 legacy=0x7f10940d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940d1ed0 con 0x7f10940b8c20 2023-08-27T11:13:23.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f1084001800 con 0x7f10940b8c20 2023-08-27T11:13:23.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 >> v1:172.21.15.45:6789/0 conn(0x7f10940b8c20 legacy=0x7f10940cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f107c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f107c004850 0x7f107c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f107c005210 0x7f107c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940d1bf0 con 0x7f107c003a00 2023-08-27T11:13:23.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00c0b0 con 0x7f107c005210 2023-08-27T11:13:23.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00c270 con 0x7f107c004850 2023-08-27T11:13:23.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1092ffd700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f107c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f10937fe700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f107c005210 0x7f107c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f10927fc700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f107c004850 0x7f107c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1092ffd700 1 -- 172.21.15.45:0/136783831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f107c004850 msgr2=0x7f107c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:23.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1092ffd700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f107c004850 0x7f107c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1092ffd700 1 -- 172.21.15.45:0/136783831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f107c005210 msgr2=0x7f107c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:23.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1092ffd700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f107c005210 0x7f107c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f10937fe700 1 -- 172.21.15.45:0/136783831 reap_dead start 2023-08-27T11:13:23.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1092ffd700 1 -- 172.21.15.45:0/136783831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0035b0 con 0x7f107c003a00 2023-08-27T11:13:23.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.576+0000 7f1092ffd700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f107c003e20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f10880017c0 tx=0x7f107c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fe82985666d968 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:23.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10880052d0 con 0x7f107c003a00 2023-08-27T11:13:23.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1088005450 con 0x7f107c003a00 2023-08-27T11:13:23.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1091ffb700 1 -- 172.21.15.45:0/136783831 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10880038c0 con 0x7f107c003a00 2023-08-27T11:13:23.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1099388700 1 -- 172.21.15.45:0/136783831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 msgr2=0x7f107c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1099388700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f107c003e20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f10880017c0 tx=0x7f107c005880 comp rx=0 tx=0).stop 2023-08-27T11:13:23.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1099388700 1 -- 172.21.15.45:0/136783831 shutdown_connections 2023-08-27T11:13:23.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1099388700 1 --2- 172.21.15.45:0/136783831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f107c003e20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1099388700 1 -- 172.21.15.45:0/136783831 >> 172.21.15.45:0/136783831 conn(0x7f10940d2fd0 msgr2=0x7f10940d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:23.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1099388700 1 -- 172.21.15.45:0/136783831 shutdown_connections 2023-08-27T11:13:23.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.577+0000 7f1099388700 1 -- 172.21.15.45:0/136783831 wait complete. 2023-08-27T11:13:23.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1099388700 1 Processor -- start 2023-08-27T11:13:23.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1099388700 1 -- start start 2023-08-27T11:13:23.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1099388700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f1094150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1099388700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1094151780 0x7f10940b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1099388700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10941511c0 0x7f10940b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1099388700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940c74c0 con 0x7f107c003a00 2023-08-27T11:13:23.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1099388700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940b55e0 con 0x7f1094151780 2023-08-27T11:13:23.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1099388700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10940c6e70 con 0x7f10941511c0 2023-08-27T11:13:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1092ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f1094150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1092ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f1094150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53758/0 (socket says 172.21.15.45:53758) 2023-08-27T11:13:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.579+0000 7f1092ffd700 1 -- 172.21.15.45:0/247715391 learned_addr learned my addr 172.21.15.45:0/247715391 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f10927fc700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1094151780 0x7f10940b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f10937fe700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10941511c0 0x7f10940b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f1092ffd700 1 -- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10941511c0 msgr2=0x7f10940b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f1092ffd700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10941511c0 0x7f10940b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f1092ffd700 1 -- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1094151780 msgr2=0x7f10940b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f1092ffd700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1094151780 0x7f10940b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f1092ffd700 1 -- 172.21.15.45:0/247715391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940c7150 con 0x7f107c003a00 2023-08-27T11:13:23.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f1092ffd700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f1094150bb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1088000c00 tx=0x7f1088001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:23.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f107b7fe700 1 -- 172.21.15.45:0/247715391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088001290 con 0x7f107c003a00 2023-08-27T11:13:23.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f107b7fe700 1 -- 172.21.15.45:0/247715391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1088007040 con 0x7f107c003a00 2023-08-27T11:13:23.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f107b7fe700 1 -- 172.21.15.45:0/247715391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10880073f0 con 0x7f107c003a00 2023-08-27T11:13:23.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10940b58c0 con 0x7f107c003a00 2023-08-27T11:13:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.580+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10940b4da0 con 0x7f107c003a00 2023-08-27T11:13:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.581+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10940b2b40 con 0x7f107c003a00 2023-08-27T11:13:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.582+0000 7f107b7fe700 1 -- 172.21.15.45:0/247715391 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1088007550 con 0x7f107c003a00 2023-08-27T11:13:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.582+0000 7f107b7fe700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f108006a0e0 0x7f108006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.582+0000 7f107b7fe700 1 -- 172.21.15.45:0/247715391 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f10880891d0 con 0x7f107c003a00 2023-08-27T11:13:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.582+0000 7f10927fc700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f108006a0e0 0x7f108006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.583+0000 7f10927fc700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f108006a0e0 0x7f108006c5a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1084001390 tx=0x7f1084000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:23.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.586+0000 7f107b7fe700 1 -- 172.21.15.45:0/247715391 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1088054620 con 0x7f107c003a00 2023-08-27T11:13:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.833+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10940c72e0 con 0x7f107c003a00 2023-08-27T11:13:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.834+0000 7f107b7fe700 1 -- 172.21.15.45:0/247715391 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f1088057c40 con 0x7f107c003a00 2023-08-27T11:13:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:23.835 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2489498728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2489498728},{"type":"v1","addr":"172.21.15.135:6837","nonce":2489498728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3941277612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3941277612},{"type":"v1","addr":"172.21.15.45:6834","nonce":3941277612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3809585329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3809585329},{"type":"v1","addr":"172.21.15.45:6835","nonce":3809585329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:12:08.309320+0000","modified":"2023-08-27T11:12:33.983169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6836/739639682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":739639682},{"type":"v1","addr":"172.21.15.135:6836","nonce":739639682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-27T11:13:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f108006a0e0 msgr2=0x7f108006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f108006a0e0 0x7f108006c5a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1084001390 tx=0x7f1084000ed0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 msgr2=0x7f1094150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f1094150bb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1088000c00 tx=0x7f1088001120 comp rx=0 tx=0).stop 2023-08-27T11:13:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 shutdown_connections 2023-08-27T11:13:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f107c003a00 0x7f1094150bb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f108006a0e0 0x7f108006c5a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10941511c0 0x7f10940b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 --2- 172.21.15.45:0/247715391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1094151780 0x7f10940b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 >> 172.21.15.45:0/247715391 conn(0x7f10940d2fd0 msgr2=0x7f10940cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 shutdown_connections 2023-08-27T11:13:23.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.836+0000 7f1099388700 1 -- 172.21.15.45:0/247715391 wait complete. 2023-08-27T11:13:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2023-08-27T11:13:23.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.975+0000 7f3618e7d700 1 Processor -- start 2023-08-27T11:13:23.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.975+0000 7f3618e7d700 1 -- start start 2023-08-27T11:13:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3618e7d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f361415b070 0x7f36140cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3618e7d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36140cdf60 0x7f361406e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3618e7d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361415dad0 con 0x7f361406cec0 2023-08-27T11:13:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3618e7d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361415d8e0 con 0x7f36140cdf60 2023-08-27T11:13:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3618e7d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361415ddb0 con 0x7f361415b070 2023-08-27T11:13:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3611d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36140cdf60 0x7f361406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3612d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f361406cec0 0x7f36140ae8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40544/0 (socket says 172.21.15.45:40544) 2023-08-27T11:13:23.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3612d9d700 1 -- 172.21.15.45:0/4105185307 learned_addr learned my addr 172.21.15.45:0/4105185307 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f361259c700 1 --2- 172.21.15.45:0/4105185307 >> v2:172.21.15.135:3300/0 conn(0x7f361415b070 0x7f36140cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3611d9b700 1 -- 172.21.15.45:0/4105185307 >> v2:172.21.15.135:3300/0 conn(0x7f361415b070 msgr2=0x7f36140cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3611d9b700 1 --2- 172.21.15.45:0/4105185307 >> v2:172.21.15.135:3300/0 conn(0x7f361415b070 0x7f36140cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3611d9b700 1 -- 172.21.15.45:0/4105185307 >> v1:172.21.15.45:6789/0 conn(0x7f361406cec0 legacy=0x7f36140ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3611d9b700 1 -- 172.21.15.45:0/4105185307 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361415e090 con 0x7f36140cdf60 2023-08-27T11:13:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f361159a700 1 -- 172.21.15.45:0/4105185307 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833288865 0 0) 0x7f361415dad0 con 0x7f361406cec0 2023-08-27T11:13:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.976+0000 7f3611d9b700 1 --2- 172.21.15.45:0/4105185307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36140cdf60 0x7f361406e270 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f361406ef10 tx=0x7f3608009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d14845a67d3c47 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f361159a700 1 -- 172.21.15.45:0/4105185307 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608011830 con 0x7f36140cdf60 2023-08-27T11:13:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f361159a700 1 -- 172.21.15.45:0/4105185307 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3608011990 con 0x7f36140cdf60 2023-08-27T11:13:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f361159a700 1 -- 172.21.15.45:0/4105185307 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608011d20 con 0x7f36140cdf60 2023-08-27T11:13:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f3618e7d700 1 -- 172.21.15.45:0/4105185307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36140cdf60 msgr2=0x7f361406e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f3618e7d700 1 --2- 172.21.15.45:0/4105185307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36140cdf60 0x7f361406e270 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f361406ef10 tx=0x7f3608009b20 comp rx=0 tx=0).stop 2023-08-27T11:13:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f3618e7d700 1 -- 172.21.15.45:0/4105185307 shutdown_connections 2023-08-27T11:13:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f3618e7d700 1 --2- 172.21.15.45:0/4105185307 >> v2:172.21.15.135:3300/0 conn(0x7f361415b070 0x7f36140cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f3618e7d700 1 --2- 172.21.15.45:0/4105185307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36140cdf60 0x7f361406e270 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f3618e7d700 1 -- 172.21.15.45:0/4105185307 >> 172.21.15.45:0/4105185307 conn(0x7f36140cfe50 msgr2=0x7f36140d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f3618e7d700 1 -- 172.21.15.45:0/4105185307 shutdown_connections 2023-08-27T11:13:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.977+0000 7f3618e7d700 1 -- 172.21.15.45:0/4105185307 wait complete. 2023-08-27T11:13:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.979+0000 7f3618e7d700 1 Processor -- start 2023-08-27T11:13:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.979+0000 7f3618e7d700 1 -- start start 2023-08-27T11:13:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.979+0000 7f3618e7d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361406cec0 0x7f361414c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.979+0000 7f3618e7d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36140cdf60 0x7f361414cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.979+0000 7f3618e7d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f361415b070 0x7f3614160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.979+0000 7f3618e7d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140c6800 con 0x7f361406cec0 2023-08-27T11:13:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.979+0000 7f3618e7d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3614154df0 con 0x7f361415b070 2023-08-27T11:13:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.979+0000 7f3618e7d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361415c350 con 0x7f36140cdf60 2023-08-27T11:13:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.979+0000 7f361259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361406cec0 0x7f361414c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f361259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361406cec0 0x7f361414c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53772/0 (socket says 172.21.15.45:53772) 2023-08-27T11:13:23.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f361259c700 1 -- 172.21.15.45:0/1071138061 learned_addr learned my addr 172.21.15.45:0/1071138061 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:23.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f3611d9b700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36140cdf60 0x7f361414cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f3612d9d700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f361415b070 0x7f3614160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f361259c700 1 -- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36140cdf60 msgr2=0x7f361414cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f361259c700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36140cdf60 0x7f361414cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f361259c700 1 -- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f361415b070 msgr2=0x7f3614160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:23.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f361259c700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f361415b070 0x7f3614160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:23.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f361259c700 1 -- 172.21.15.45:0/1071138061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36080094a0 con 0x7f361406cec0 2023-08-27T11:13:23.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f3612d9d700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f361415b070 0x7f3614160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:23.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f3611d9b700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36140cdf60 0x7f361414cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f361259c700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361406cec0 0x7f361414c3d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f360400e810 tx=0x7f360400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f3602ffd700 1 -- 172.21.15.45:0/1071138061 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604013610 con 0x7f361406cec0 2023-08-27T11:13:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f3602ffd700 1 -- 172.21.15.45:0/1071138061 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3604013770 con 0x7f361406cec0 2023-08-27T11:13:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361415c070 con 0x7f361406cec0 2023-08-27T11:13:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f3602ffd700 1 -- 172.21.15.45:0/1071138061 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604013b20 con 0x7f361406cec0 2023-08-27T11:13:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.980+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3614154a80 con 0x7f361406cec0 2023-08-27T11:13:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.981+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3614154600 con 0x7f361406cec0 2023-08-27T11:13:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.982+0000 7f3602ffd700 1 -- 172.21.15.45:0/1071138061 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3604013c80 con 0x7f361406cec0 2023-08-27T11:13:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.982+0000 7f3602ffd700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f35f406a0e0 0x7f35f406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.982+0000 7f3602ffd700 1 -- 172.21.15.45:0/1071138061 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3604088600 con 0x7f361406cec0 2023-08-27T11:13:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.983+0000 7f3611d9b700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f35f406a0e0 0x7f35f406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.983+0000 7f3611d9b700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f35f406a0e0 0x7f35f406c5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f361406ef10 tx=0x7f3608008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:23.987+0000 7f3602ffd700 1 -- 172.21.15.45:0/1071138061 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3614154600 con 0x7f361406cec0 2023-08-27T11:13:24.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.192+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3614154600 con 0x7f361406cec0 2023-08-27T11:13:24.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.194+0000 7f3602ffd700 1 -- 172.21.15.45:0/1071138061 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+20227 (secure 0 0 0) 0x7f3614154600 con 0x7f361406cec0 2023-08-27T11:13:24.195 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:12:50.061834+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:12:06.215374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-27T11:12:07.287854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-27T11:12:10.957364+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:24.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f35f406a0e0 msgr2=0x7f35f406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f35f406a0e0 0x7f35f406c5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f361406ef10 tx=0x7f3608008030 comp rx=0 tx=0).stop 2023-08-27T11:13:24.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361406cec0 msgr2=0x7f361414c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361406cec0 0x7f361414c3d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f360400e810 tx=0x7f360400ebd0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 shutdown_connections 2023-08-27T11:13:24.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f361406cec0 0x7f361414c3d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f35f406a0e0 0x7f35f406c5a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f361415b070 0x7f3614160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 --2- 172.21.15.45:0/1071138061 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36140cdf60 0x7f361414cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 >> 172.21.15.45:0/1071138061 conn(0x7f36140cfe50 msgr2=0x7f361415ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 shutdown_connections 2023-08-27T11:13:24.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.196+0000 7f3618e7d700 1 -- 172.21.15.45:0/1071138061 wait complete. 2023-08-27T11:13:24.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:13:24.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.335+0000 7fc19d4c6700 1 Processor -- start 2023-08-27T11:13:24.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.335+0000 7fc19d4c6700 1 -- start start 2023-08-27T11:13:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.336+0000 7fc19d4c6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1980b8c20 0x7fc1980cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.336+0000 7fc19d4c6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19815d6a0 con 0x7fc19815d2c0 2023-08-27T11:13:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.336+0000 7fc19d4c6700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1980d2c90 con 0x7fc1980ce2e0 2023-08-27T11:13:24.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.336+0000 7fc19d4c6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980ce770 con 0x7fc1980b8c20 2023-08-27T11:13:24.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.336+0000 7fc196ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc19815d2c0 0x7fc1980d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40556/0 (socket says 172.21.15.45:40556) 2023-08-27T11:13:24.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.336+0000 7fc196ffd700 1 -- 172.21.15.45:0/49019910 learned_addr learned my addr 172.21.15.45:0/49019910 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:24.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.336+0000 7fc1967fc700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1980b8c20 0x7fc1980cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.336+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000492182 0 0) 0x7fc18c001110 con 0x7fc19815d2c0 2023-08-27T11:13:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.336+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1800035b0 con 0x7fc19815d2c0 2023-08-27T11:13:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 608479806 0 0) 0x7fc1800035b0 con 0x7fc19815d2c0 2023-08-27T11:13:24.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1980b8c20 msgr2=0x7fc1980cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:24.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1980b8c20 0x7fc1980cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 >> v1:172.21.15.135:6789/0 conn(0x7fc1980ce2e0 legacy=0x7fc1980b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980d1bf0 con 0x7fc19815d2c0 2023-08-27T11:13:24.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fc18c001800 con 0x7fc19815d2c0 2023-08-27T11:13:24.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 >> v1:172.21.15.45:6789/0 conn(0x7fc19815d2c0 legacy=0x7fc1980d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc180003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1980b8c20 0x7fc180004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc180004d90 0x7fc180005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980ce770 con 0x7fc180003a30 2023-08-27T11:13:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18000baf0 con 0x7fc180004d90 2023-08-27T11:13:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18000bcb0 con 0x7fc1980b8c20 2023-08-27T11:13:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc196ffd700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc180003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc1967fc700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1980b8c20 0x7fc180004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc1977fe700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc180004d90 0x7fc180005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc196ffd700 1 -- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1980b8c20 msgr2=0x7fc180004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:24.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc196ffd700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1980b8c20 0x7fc180004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc196ffd700 1 -- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc180004d90 msgr2=0x7fc180005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:24.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc196ffd700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc180004d90 0x7fc180005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc196ffd700 1 -- 172.21.15.45:0/49019910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800035b0 con 0x7fc180003a30 2023-08-27T11:13:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc196ffd700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc180003e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c210 tx=0x7fc180005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7776a1fc40f585 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c007db0 con 0x7fc180003a30 2023-08-27T11:13:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.337+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c00cdd0 con 0x7fc180003a30 2023-08-27T11:13:24.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.338+0000 7fc195ffb700 1 -- 172.21.15.45:0/49019910 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c005ba0 con 0x7fc180003a30 2023-08-27T11:13:24.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.338+0000 7fc19d4c6700 1 -- 172.21.15.45:0/49019910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 msgr2=0x7fc180003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.338+0000 7fc19d4c6700 1 --2- 172.21.15.45:0/49019910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc180003e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c210 tx=0x7fc180005880 comp rx=0 tx=0).stop 2023-08-27T11:13:24.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.338+0000 7fc1977fe700 1 -- 172.21.15.45:0/49019910 reap_dead start 2023-08-27T11:13:24.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.338+0000 7fc19d4c6700 1 -- 172.21.15.45:0/49019910 shutdown_connections 2023-08-27T11:13:24.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.338+0000 7fc19d4c6700 1 -- 172.21.15.45:0/49019910 >> 172.21.15.45:0/49019910 conn(0x7fc1980d2fd0 msgr2=0x7fc1980d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:24.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.338+0000 7fc19d4c6700 1 -- 172.21.15.45:0/49019910 shutdown_connections 2023-08-27T11:13:24.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.338+0000 7fc19d4c6700 1 -- 172.21.15.45:0/49019910 wait complete. 2023-08-27T11:13:24.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.340+0000 7fc19d4c6700 1 Processor -- start 2023-08-27T11:13:24.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.340+0000 7fc19d4c6700 1 -- start start 2023-08-27T11:13:24.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc19d4c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc1981598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc19d4c6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1981f2a40 0x7fc1980cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc19d4c6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1981f2ec0 0x7fc1980cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc19d4c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc198159e10 con 0x7fc180003a30 2023-08-27T11:13:24.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc19d4c6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980c7f30 con 0x7fc1981f2ec0 2023-08-27T11:13:24.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc19d4c6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980c81d0 con 0x7fc1981f2a40 2023-08-27T11:13:24.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc196ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc1981598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc196ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc1981598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53792/0 (socket says 172.21.15.45:53792) 2023-08-27T11:13:24.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc196ffd700 1 -- 172.21.15.45:0/143134826 learned_addr learned my addr 172.21.15.45:0/143134826 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:24.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc1967fc700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1981f2a40 0x7fc1980cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc1977fe700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1981f2ec0 0x7fc1980cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc196ffd700 1 -- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1981f2a40 msgr2=0x7fc1980cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc196ffd700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1981f2a40 0x7fc1980cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc196ffd700 1 -- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1981f2ec0 msgr2=0x7fc1980cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc196ffd700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1981f2ec0 0x7fc1980cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc196ffd700 1 -- 172.21.15.45:0/143134826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980c84b0 con 0x7fc180003a30 2023-08-27T11:13:24.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc1977fe700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1981f2ec0 0x7fc1980cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:24.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc196ffd700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc1981598d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc18000bac0 tx=0x7fc18c001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:24.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc17f7fe700 1 -- 172.21.15.45:0/143134826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004870 con 0x7fc180003a30 2023-08-27T11:13:24.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc17f7fe700 1 -- 172.21.15.45:0/143134826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c0049d0 con 0x7fc180003a30 2023-08-27T11:13:24.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1980c8790 con 0x7fc180003a30 2023-08-27T11:13:24.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc17f7fe700 1 -- 172.21.15.45:0/143134826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004d80 con 0x7fc180003a30 2023-08-27T11:13:24.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.341+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1980c8f90 con 0x7fc180003a30 2023-08-27T11:13:24.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.342+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19804e620 con 0x7fc180003a30 2023-08-27T11:13:24.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.343+0000 7fc17f7fe700 1 -- 172.21.15.45:0/143134826 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc18c019070 con 0x7fc180003a30 2023-08-27T11:13:24.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.343+0000 7fc17f7fe700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc18406a0e0 0x7fc18406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.343+0000 7fc17f7fe700 1 -- 172.21.15.45:0/143134826 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc18c089730 con 0x7fc180003a30 2023-08-27T11:13:24.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.343+0000 7fc1967fc700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc18406a0e0 0x7fc18406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.344+0000 7fc1967fc700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc18406a0e0 0x7fc18406c5a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc1980ccdc0 tx=0x7fc188008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:24.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.347+0000 7fc17f7fe700 1 -- 172.21.15.45:0/143134826 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc18c054ca0 con 0x7fc180003a30 2023-08-27T11:13:24.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.590+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc19815d6a0 con 0x7fc180003a30 2023-08-27T11:13:24.591 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-08-27T11:13:24.590+0000 7f87a1c48700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:13:24.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.594+0000 7fc17f7fe700 1 -- 172.21.15.45:0/143134826 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc19815d6a0 con 0x7fc180003a30 2023-08-27T11:13:24.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc18406a0e0 msgr2=0x7fc18406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc18406a0e0 0x7fc18406c5a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc1980ccdc0 tx=0x7fc188008040 comp rx=0 tx=0).stop 2023-08-27T11:13:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 msgr2=0x7fc1981598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc1981598d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc18000bac0 tx=0x7fc18c001b90 comp rx=0 tx=0).stop 2023-08-27T11:13:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 shutdown_connections 2023-08-27T11:13:24.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003a30 0x7fc1981598d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc18406a0e0 0x7fc18406c5a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1981f2ec0 0x7fc1980cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 --2- 172.21.15.45:0/143134826 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc1981f2a40 0x7fc1980cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.596+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 >> 172.21.15.45:0/143134826 conn(0x7fc1980d2fd0 msgr2=0x7fc19815b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:24.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.597+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 shutdown_connections 2023-08-27T11:13:24.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.597+0000 7fc19d4c6700 1 -- 172.21.15.45:0/143134826 wait complete. 2023-08-27T11:13:24.600 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:13:24.600 INFO:tasks.ceph.mds.d.smithi135.stderr: -6217> 2023-08-27T11:12:12.991+0000 7fdc14a59700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:13:24.610 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:13:24.633 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2023-08-27T11:13:24.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.736+0000 7fa510056700 1 Processor -- start 2023-08-27T11:13:24.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.737+0000 7fa510056700 1 -- start start 2023-08-27T11:13:24.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.737+0000 7fa510056700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5080b5220 0x7fa5080cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.737+0000 7fa510056700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5080cdf40 con 0x7fa5080b62c0 2023-08-27T11:13:24.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.737+0000 7fa510056700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5080d0fd0 con 0x7fa508160890 2023-08-27T11:13:24.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.737+0000 7fa510056700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5080ce190 con 0x7fa5080b5220 2023-08-27T11:13:24.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.737+0000 7fa50d5f1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5080b5220 0x7fa5080cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50e5f3700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa5080b62c0 0x7fa5080d1460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40580/0 (socket says 172.21.15.45:40580) 2023-08-27T11:13:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50e5f3700 1 -- 172.21.15.45:0/1581914736 learned_addr learned my addr 172.21.15.45:0/1581914736 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50d5f1700 1 -- 172.21.15.45:0/1581914736 >> v1:172.21.15.135:6789/0 conn(0x7fa508160890 legacy=0x7fa5080b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50d5f1700 1 -- 172.21.15.45:0/1581914736 >> v1:172.21.15.45:6789/0 conn(0x7fa5080b62c0 legacy=0x7fa5080d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50d5f1700 1 -- 172.21.15.45:0/1581914736 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080ce470 con 0x7fa5080b5220 2023-08-27T11:13:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50cdf0700 1 -- 172.21.15.45:0/1581914736 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571771910 0 0) 0x7fa5080cdf40 con 0x7fa5080b62c0 2023-08-27T11:13:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50d5f1700 1 --2- 172.21.15.45:0/1581914736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b5220 0x7fa5080cd8f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009840 tx=0x7fa4fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c7be1c8406d4d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50cdf0700 1 -- 172.21.15.45:0/1581914736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc011830 con 0x7fa5080b5220 2023-08-27T11:13:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50cdf0700 1 -- 172.21.15.45:0/1581914736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc011990 con 0x7fa5080b5220 2023-08-27T11:13:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.738+0000 7fa50cdf0700 1 -- 172.21.15.45:0/1581914736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc011d20 con 0x7fa5080b5220 2023-08-27T11:13:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.739+0000 7fa510056700 1 -- 172.21.15.45:0/1581914736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b5220 msgr2=0x7fa5080cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.739+0000 7fa510056700 1 --2- 172.21.15.45:0/1581914736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b5220 0x7fa5080cd8f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009840 tx=0x7fa4fc009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.739+0000 7fa510056700 1 -- 172.21.15.45:0/1581914736 shutdown_connections 2023-08-27T11:13:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.739+0000 7fa510056700 1 --2- 172.21.15.45:0/1581914736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b5220 0x7fa5080cd8f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.739+0000 7fa510056700 1 -- 172.21.15.45:0/1581914736 >> 172.21.15.45:0/1581914736 conn(0x7fa5080cfb10 msgr2=0x7fa5080cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:24.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.739+0000 7fa510056700 1 -- 172.21.15.45:0/1581914736 shutdown_connections 2023-08-27T11:13:24.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.739+0000 7fa510056700 1 -- 172.21.15.45:0/1581914736 wait complete. 2023-08-27T11:13:24.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.740+0000 7fa510056700 1 Processor -- start 2023-08-27T11:13:24.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.740+0000 7fa510056700 1 -- start start 2023-08-27T11:13:24.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa510056700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5080b5220 0x7fa5080b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa510056700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b62c0 0x7fa5080a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa510056700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa508160890 0x7fa5080a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa510056700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080b4940 con 0x7fa5080b62c0 2023-08-27T11:13:24.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa510056700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50814a220 con 0x7fa5080b5220 2023-08-27T11:13:24.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa510056700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50815cdf0 con 0x7fa508160890 2023-08-27T11:13:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50d5f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b62c0 0x7fa5080a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50d5f1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b62c0 0x7fa5080a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53808/0 (socket says 172.21.15.45:53808) 2023-08-27T11:13:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50d5f1700 1 -- 172.21.15.45:0/2581009955 learned_addr learned my addr 172.21.15.45:0/2581009955 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50e5f3700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa508160890 0x7fa5080a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50ddf2700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5080b5220 0x7fa5080b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50d5f1700 1 -- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa508160890 msgr2=0x7fa5080a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50d5f1700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa508160890 0x7fa5080a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50d5f1700 1 -- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5080b5220 msgr2=0x7fa5080b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50d5f1700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5080b5220 0x7fa5080b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50d5f1700 1 -- 172.21.15.45:0/2581009955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc0094a0 con 0x7fa5080b62c0 2023-08-27T11:13:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.741+0000 7fa50d5f1700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b62c0 0x7fa5080a9a10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0090b0 tx=0x7fa4fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.742+0000 7fa4fa7fc700 1 -- 172.21.15.45:0/2581009955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0045d0 con 0x7fa5080b62c0 2023-08-27T11:13:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.742+0000 7fa4fa7fc700 1 -- 172.21.15.45:0/2581009955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc01f070 con 0x7fa5080b62c0 2023-08-27T11:13:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.742+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5080d2be0 con 0x7fa5080b62c0 2023-08-27T11:13:24.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.742+0000 7fa4fa7fc700 1 -- 172.21.15.45:0/2581009955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0038a0 con 0x7fa5080b62c0 2023-08-27T11:13:24.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.742+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50814a750 con 0x7fa5080b62c0 2023-08-27T11:13:24.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.743+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50806c120 con 0x7fa5080b62c0 2023-08-27T11:13:24.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.743+0000 7fa4fa7fc700 1 -- 172.21.15.45:0/2581009955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa4fc003a00 con 0x7fa5080b62c0 2023-08-27T11:13:24.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.743+0000 7fa4fa7fc700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4ec06a0e0 0x7fa4ec06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:24.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.744+0000 7fa4fa7fc700 1 -- 172.21.15.45:0/2581009955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fa4fc021070 con 0x7fa5080b62c0 2023-08-27T11:13:24.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.744+0000 7fa50ddf2700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4ec06a0e0 0x7fa4ec06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:24.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.744+0000 7fa50ddf2700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4ec06a0e0 0x7fa4ec06c5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa50815ac90 tx=0x7fa504009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:24.748+0000 7fa4fa7fc700 1 -- 172.21.15.45:0/2581009955 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa4fc055610 con 0x7fa5080b62c0 2023-08-27T11:13:25.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.000+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa5080734a0 con 0x7fa5080b62c0 2023-08-27T11:13:25.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.597+0000 7fa4fa7fc700 1 -- 172.21.15.45:0/2581009955 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa4fc003cb0 con 0x7fa5080b62c0 2023-08-27T11:13:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4ec06a0e0 msgr2=0x7fa4ec06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4ec06a0e0 0x7fa4ec06c5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa50815ac90 tx=0x7fa504009040 comp rx=0 tx=0).stop 2023-08-27T11:13:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b62c0 msgr2=0x7fa5080a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:25.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b62c0 0x7fa5080a9a10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0090b0 tx=0x7fa4fc004750 comp rx=0 tx=0).stop 2023-08-27T11:13:25.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 shutdown_connections 2023-08-27T11:13:25.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fa4ec06a0e0 0x7fa4ec06c5a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:25.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa508160890 0x7fa5080a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:25.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5080b5220 0x7fa5080b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 --2- 172.21.15.45:0/2581009955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5080b62c0 0x7fa5080a9a10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 >> 172.21.15.45:0/2581009955 conn(0x7fa5080cfb10 msgr2=0x7fa5080c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 shutdown_connections 2023-08-27T11:13:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.600+0000 7fa510056700 1 -- 172.21.15.45:0/2581009955 wait complete. 2023-08-27T11:13:25.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:13:25.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.751+0000 7efe9d63b700 1 Processor -- start 2023-08-27T11:13:25.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.751+0000 7efe9d63b700 1 -- start start 2023-08-27T11:13:25.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.752+0000 7efe9d63b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efe980b8c20 0x7efe980cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:25.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.752+0000 7efe9d63b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:25.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.752+0000 7efe9d63b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9815d6a0 con 0x7efe9815d2c0 2023-08-27T11:13:25.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.752+0000 7efe9d63b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe980ce770 con 0x7efe980b8c20 2023-08-27T11:13:25.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.752+0000 7efe9d63b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe980d2c90 con 0x7efe980ce2e0 2023-08-27T11:13:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.752+0000 7efe977fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.752+0000 7efe977fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:51728/0 (socket says 172.21.15.45:51728) 2023-08-27T11:13:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.752+0000 7efe967fc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7efe980b8c20 0x7efe980cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.752+0000 7efe977fe700 1 -- 172.21.15.45:0/3856866512 learned_addr learned my addr 172.21.15.45:0/3856866512 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe967fc700 1 -- 172.21.15.45:0/3856866512 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 msgr2=0x7efe980b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe967fc700 1 --2- 172.21.15.45:0/3856866512 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe967fc700 1 -- 172.21.15.45:0/3856866512 >> v1:172.21.15.135:6789/0 conn(0x7efe9815d2c0 legacy=0x7efe980d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe967fc700 1 -- 172.21.15.45:0/3856866512 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe980d1bf0 con 0x7efe980b8c20 2023-08-27T11:13:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe977fe700 1 --2- 172.21.15.45:0/3856866512 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe967fc700 1 --2- 172.21.15.45:0/3856866512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe980b8c20 0x7efe980cdda0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe88009870 tx=0x7efe88009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49db309e033e3a07 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe95ffb700 1 -- 172.21.15.45:0/3856866512 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88011830 con 0x7efe980b8c20 2023-08-27T11:13:25.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe95ffb700 1 -- 172.21.15.45:0/3856866512 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe88011990 con 0x7efe980b8c20 2023-08-27T11:13:25.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe95ffb700 1 -- 172.21.15.45:0/3856866512 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88011d20 con 0x7efe980b8c20 2023-08-27T11:13:25.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.753+0000 7efe9d63b700 1 -- 172.21.15.45:0/3856866512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe980b8c20 msgr2=0x7efe980cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:25.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.754+0000 7efe9d63b700 1 --2- 172.21.15.45:0/3856866512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe980b8c20 0x7efe980cdda0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe88009870 tx=0x7efe88009b80 comp rx=0 tx=0).stop 2023-08-27T11:13:25.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.754+0000 7efe9d63b700 1 -- 172.21.15.45:0/3856866512 shutdown_connections 2023-08-27T11:13:25.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.754+0000 7efe9d63b700 1 --2- 172.21.15.45:0/3856866512 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:25.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.754+0000 7efe9d63b700 1 --2- 172.21.15.45:0/3856866512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe980b8c20 0x7efe980cdda0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:25.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.754+0000 7efe9d63b700 1 -- 172.21.15.45:0/3856866512 >> 172.21.15.45:0/3856866512 conn(0x7efe980d2fd0 msgr2=0x7efe980d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:25.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.754+0000 7efe9d63b700 1 -- 172.21.15.45:0/3856866512 shutdown_connections 2023-08-27T11:13:25.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.754+0000 7efe9d63b700 1 -- 172.21.15.45:0/3856866512 wait complete. 2023-08-27T11:13:25.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.755+0000 7efe9d63b700 1 Processor -- start 2023-08-27T11:13:25.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.755+0000 7efe9d63b700 1 -- start start 2023-08-27T11:13:25.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe9d63b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe980b8c20 0x7efe981f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:25.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe9d63b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:25.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe9d63b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe9815d2c0 0x7efe9815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:25.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe9d63b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe980c9060 con 0x7efe980b8c20 2023-08-27T11:13:25.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe9d63b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9815f7e0 con 0x7efe9815d2c0 2023-08-27T11:13:25.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe9d63b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe981602b0 con 0x7efe980ce2e0 2023-08-27T11:13:25.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe96ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe980b8c20 0x7efe981f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:25.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe96ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe980b8c20 0x7efe981f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53820/0 (socket says 172.21.15.45:53820) 2023-08-27T11:13:25.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe96ffd700 1 -- 172.21.15.45:0/1729388770 learned_addr learned my addr 172.21.15.45:0/1729388770 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe967fc700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe977fe700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe9815d2c0 0x7efe9815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe96ffd700 1 -- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 msgr2=0x7efe980c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe96ffd700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe96ffd700 1 -- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe9815d2c0 msgr2=0x7efe9815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:25.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe96ffd700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe9815d2c0 0x7efe9815f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:25.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe96ffd700 1 -- 172.21.15.45:0/1729388770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe90009b30 con 0x7efe980b8c20 2023-08-27T11:13:25.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.756+0000 7efe977fe700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe9815d2c0 0x7efe9815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:25.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.757+0000 7efe96ffd700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe980b8c20 0x7efe981f58e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c4e0 tx=0x7efe8c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:25.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.757+0000 7efe7f7fe700 1 -- 172.21.15.45:0/1729388770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0052d0 con 0x7efe980b8c20 2023-08-27T11:13:25.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.757+0000 7efe7f7fe700 1 -- 172.21.15.45:0/1729388770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c005430 con 0x7efe980b8c20 2023-08-27T11:13:25.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.757+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe880094a0 con 0x7efe980b8c20 2023-08-27T11:13:25.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.757+0000 7efe7f7fe700 1 -- 172.21.15.45:0/1729388770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0038c0 con 0x7efe980b8c20 2023-08-27T11:13:25.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.757+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9815fcf0 con 0x7efe980b8c20 2023-08-27T11:13:25.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.758+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9804e620 con 0x7efe980b8c20 2023-08-27T11:13:25.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.758+0000 7efe7f7fe700 1 -- 172.21.15.45:0/1729388770 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efe8c003a20 con 0x7efe980b8c20 2023-08-27T11:13:25.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.759+0000 7efe7f7fe700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7efe8406a0e0 0x7efe8406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:25.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.759+0000 7efe7f7fe700 1 -- 172.21.15.45:0/1729388770 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6957+0+0 (secure 0 0 0) 0x7efe8c087b60 con 0x7efe980b8c20 2023-08-27T11:13:25.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.759+0000 7efe967fc700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7efe8406a0e0 0x7efe8406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:25.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.759+0000 7efe967fc700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7efe8406a0e0 0x7efe8406c5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efe880090b0 tx=0x7efe8801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:25.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.763+0000 7efe7f7fe700 1 -- 172.21.15.45:0/1729388770 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efe8c052ec0 con 0x7efe980b8c20 2023-08-27T11:13:25.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:25.964+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efe9815d6a0 con 0x7efe980b8c20 2023-08-27T11:13:26.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.600+0000 7efe7f7fe700 1 -- 172.21.15.45:0/1729388770 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) v1 ==== 181+0+0 (secure 0 0 0) 0x7efe9815d6a0 con 0x7efe980b8c20 2023-08-27T11:13:26.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7efe8406a0e0 msgr2=0x7efe8406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:26.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7efe8406a0e0 0x7efe8406c5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efe880090b0 tx=0x7efe8801c040 comp rx=0 tx=0).stop 2023-08-27T11:13:26.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe980b8c20 msgr2=0x7efe981f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:26.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe980b8c20 0x7efe981f58e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c4e0 tx=0x7efe8c00c7f0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 shutdown_connections 2023-08-27T11:13:26.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe980b8c20 0x7efe981f58e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7efe8406a0e0 0x7efe8406c5a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe9815d2c0 0x7efe9815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 --2- 172.21.15.45:0/1729388770 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efe980ce2e0 0x7efe980c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 >> 172.21.15.45:0/1729388770 conn(0x7efe980d2fd0 msgr2=0x7efe9815c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:26.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 shutdown_connections 2023-08-27T11:13:26.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.604+0000 7efe9d63b700 1 -- 172.21.15.45:0/1729388770 wait complete. 2023-08-27T11:13:26.608 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:13:26.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:13:26.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.754+0000 7fbb40881700 1 Processor -- start 2023-08-27T11:13:26.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.754+0000 7fbb40881700 1 -- start start 2023-08-27T11:13:26.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.755+0000 7fbb40881700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb3c06a280 0x7fbb3c06a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:26.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.755+0000 7fbb40881700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 0x7fbb3c15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:26.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.755+0000 7fbb40881700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c0b2290 con 0x7fbb3c071400 2023-08-27T11:13:26.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.755+0000 7fbb40881700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c071fa0 con 0x7fbb3c06a280 2023-08-27T11:13:26.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.755+0000 7fbb40881700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c06aba0 con 0x7fbb3c15b5b0 2023-08-27T11:13:26.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb3c06a280 0x7fbb3c06a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:26.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb3c06a280 0x7fbb3c06a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53828/0 (socket says 172.21.15.45:53828) 2023-08-27T11:13:26.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3a59c700 1 -- 172.21.15.45:0/735140349 learned_addr learned my addr 172.21.15.45:0/735140349 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:26.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3ad9d700 1 --2- 172.21.15.45:0/735140349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 0x7fbb3c15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:26.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3a59c700 1 -- 172.21.15.45:0/735140349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 msgr2=0x7fbb3c15ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:26.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3a59c700 1 --2- 172.21.15.45:0/735140349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 0x7fbb3c15ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3a59c700 1 -- 172.21.15.45:0/735140349 >> v1:172.21.15.135:6789/0 conn(0x7fbb3c071400 legacy=0x7fbb3c071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:26.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3a59c700 1 -- 172.21.15.45:0/735140349 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0b1960 con 0x7fbb3c06a280 2023-08-27T11:13:26.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3a59c700 1 --2- 172.21.15.45:0/735140349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c06a280 0x7fbb3c06a660 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098f0 tx=0x7fbb2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bd8d3a32133f95 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:26.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3959a700 1 -- 172.21.15.45:0/735140349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00b8a0 con 0x7fbb3c06a280 2023-08-27T11:13:26.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3959a700 1 -- 172.21.15.45:0/735140349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00ba00 con 0x7fbb3c06a280 2023-08-27T11:13:26.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.756+0000 7fbb3959a700 1 -- 172.21.15.45:0/735140349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00bd90 con 0x7fbb3c06a280 2023-08-27T11:13:26.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.757+0000 7fbb40881700 1 -- 172.21.15.45:0/735140349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c06a280 msgr2=0x7fbb3c06a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:26.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.757+0000 7fbb40881700 1 --2- 172.21.15.45:0/735140349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c06a280 0x7fbb3c06a660 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098f0 tx=0x7fbb2c009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:26.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.757+0000 7fbb40881700 1 -- 172.21.15.45:0/735140349 shutdown_connections 2023-08-27T11:13:26.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.757+0000 7fbb40881700 1 --2- 172.21.15.45:0/735140349 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 0x7fbb3c15ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.757+0000 7fbb40881700 1 --2- 172.21.15.45:0/735140349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c06a280 0x7fbb3c06a660 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.757+0000 7fbb40881700 1 -- 172.21.15.45:0/735140349 >> 172.21.15.45:0/735140349 conn(0x7fbb3c0cc7c0 msgr2=0x7fbb3c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:26.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.757+0000 7fbb40881700 1 -- 172.21.15.45:0/735140349 shutdown_connections 2023-08-27T11:13:26.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.757+0000 7fbb40881700 1 -- 172.21.15.45:0/735140349 wait complete. 2023-08-27T11:13:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.758+0000 7fbb40881700 1 Processor -- start 2023-08-27T11:13:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.758+0000 7fbb40881700 1 -- start start 2023-08-27T11:13:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb40881700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb3c06a280 0x7fbb3c1520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb40881700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c071400 0x7fbb3c15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:26.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb40881700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 0x7fbb3c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:26.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb40881700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0b4ca0 con 0x7fbb3c071400 2023-08-27T11:13:26.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb40881700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0cda20 con 0x7fbb3c06a280 2023-08-27T11:13:26.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb40881700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0b49c0 con 0x7fbb3c15b5b0 2023-08-27T11:13:26.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb39d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c071400 0x7fbb3c15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:26.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb39d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c071400 0x7fbb3c15dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53834/0 (socket says 172.21.15.45:53834) 2023-08-27T11:13:26.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb39d9b700 1 -- 172.21.15.45:0/3240786584 learned_addr learned my addr 172.21.15.45:0/3240786584 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:26.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb3ad9d700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 0x7fbb3c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:26.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.759+0000 7fbb3a59c700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb3c06a280 0x7fbb3c1520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:26.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb39d9b700 1 -- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 msgr2=0x7fbb3c15e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:26.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb39d9b700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 0x7fbb3c15e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb39d9b700 1 -- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb3c06a280 msgr2=0x7fbb3c1520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:26.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb39d9b700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb3c06a280 0x7fbb3c1520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:26.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb39d9b700 1 -- 172.21.15.45:0/3240786584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c009550 con 0x7fbb3c071400 2023-08-27T11:13:26.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb39d9b700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c071400 0x7fbb3c15dd30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c450 tx=0x7fbb3000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb2affd700 1 -- 172.21.15.45:0/3240786584 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb300052d0 con 0x7fbb3c071400 2023-08-27T11:13:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb2affd700 1 -- 172.21.15.45:0/3240786584 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb30005430 con 0x7fbb3c071400 2023-08-27T11:13:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c0cdcc0 con 0x7fbb3c071400 2023-08-27T11:13:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb2affd700 1 -- 172.21.15.45:0/3240786584 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb300038c0 con 0x7fbb3c071400 2023-08-27T11:13:26.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.760+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c1525f0 con 0x7fbb3c071400 2023-08-27T11:13:26.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.761+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c0ce1e0 con 0x7fbb3c071400 2023-08-27T11:13:26.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.761+0000 7fbb2affd700 1 -- 172.21.15.45:0/3240786584 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbb30003a20 con 0x7fbb3c071400 2023-08-27T11:13:26.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.762+0000 7fbb2affd700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbb1c06a0e0 0x7fbb1c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:26.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.762+0000 7fbb2affd700 1 -- 172.21.15.45:0/3240786584 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fbb300879f0 con 0x7fbb3c071400 2023-08-27T11:13:26.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.762+0000 7fbb3a59c700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbb1c06a0e0 0x7fbb1c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:26.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.763+0000 7fbb3a59c700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbb1c06a0e0 0x7fbb1c06c5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c014040 tx=0x7fbb2c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:26.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.766+0000 7fbb2affd700 1 -- 172.21.15.45:0/3240786584 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbb3c0ce1e0 con 0x7fbb3c071400 2023-08-27T11:13:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:26.970+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbb3c0ce1e0 con 0x7fbb3c071400 2023-08-27T11:13:27.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.606+0000 7fbb2affd700 1 -- 172.21.15.45:0/3240786584 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbb3c0ce1e0 con 0x7fbb3c071400 2023-08-27T11:13:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbb1c06a0e0 msgr2=0x7fbb1c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:27.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbb1c06a0e0 0x7fbb1c06c5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c014040 tx=0x7fbb2c008030 comp rx=0 tx=0).stop 2023-08-27T11:13:27.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c071400 msgr2=0x7fbb3c15dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:27.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c071400 0x7fbb3c15dd30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c450 tx=0x7fbb3000c760 comp rx=0 tx=0).stop 2023-08-27T11:13:27.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 shutdown_connections 2023-08-27T11:13:27.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fbb1c06a0e0 0x7fbb1c06c5a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:27.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb3c15b5b0 0x7fbb3c15e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:27.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb3c06a280 0x7fbb3c1520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:27.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 --2- 172.21.15.45:0/3240786584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb3c071400 0x7fbb3c15dd30 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c450 tx=0x7fbb3000c760 comp rx=0 tx=0).stop 2023-08-27T11:13:27.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.611+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 >> 172.21.15.45:0/3240786584 conn(0x7fbb3c0cc7c0 msgr2=0x7fbb3c15a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:27.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.613+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 shutdown_connections 2023-08-27T11:13:27.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.613+0000 7fbb40881700 1 -- 172.21.15.45:0/3240786584 wait complete. 2023-08-27T11:13:27.615 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-08-27T11:13:27.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-08-27T11:13:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.765+0000 7f88b74e9700 1 Processor -- start 2023-08-27T11:13:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.765+0000 7f88b74e9700 1 -- start start 2023-08-27T11:13:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.766+0000 7f88b74e9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015a0f0 0x7f88b00cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.766+0000 7f88b74e9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88b00cd300 0x7f88b015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.766+0000 7f88b74e9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b00cfae0 con 0x7f88b015e030 2023-08-27T11:13:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.766+0000 7f88b74e9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b015e900 con 0x7f88b00cd300 2023-08-27T11:13:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.766+0000 7f88b74e9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00cae60 con 0x7f88b015a0f0 2023-08-27T11:13:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.766+0000 7f88b4a84700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88b00cd300 0x7f88b015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.766+0000 7f88b5a86700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f88b015e030 0x7f88b00cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40620/0 (socket says 172.21.15.45:40620) 2023-08-27T11:13:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.766+0000 7f88b5a86700 1 -- 172.21.15.45:0/508449242 learned_addr learned my addr 172.21.15.45:0/508449242 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88b5285700 1 --2- 172.21.15.45:0/508449242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015a0f0 0x7f88b00cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88b4a84700 1 -- 172.21.15.45:0/508449242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015a0f0 msgr2=0x7f88b00cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88b4a84700 1 --2- 172.21.15.45:0/508449242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015a0f0 0x7f88b00cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88b4a84700 1 -- 172.21.15.45:0/508449242 >> v1:172.21.15.45:6789/0 conn(0x7f88b015e030 legacy=0x7f88b00cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88a7fff700 1 -- 172.21.15.45:0/508449242 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595959551 0 0) 0x7f88b00cfae0 con 0x7f88b015e030 2023-08-27T11:13:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88b4a84700 1 -- 172.21.15.45:0/508449242 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00cb140 con 0x7f88b00cd300 2023-08-27T11:13:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88b4a84700 1 --2- 172.21.15.45:0/508449242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b00cd300 0x7f88b015daf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88a000bae0 tx=0x7f88a000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b53c80b85bf7dc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88a7fff700 1 -- 172.21.15.45:0/508449242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a000dd30 con 0x7f88b00cd300 2023-08-27T11:13:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88a7fff700 1 -- 172.21.15.45:0/508449242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a000de90 con 0x7f88b00cd300 2023-08-27T11:13:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.767+0000 7f88a7fff700 1 -- 172.21.15.45:0/508449242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0010690 con 0x7f88b00cd300 2023-08-27T11:13:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.768+0000 7f88b74e9700 1 -- 172.21.15.45:0/508449242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b00cd300 msgr2=0x7f88b015daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.768+0000 7f88b74e9700 1 --2- 172.21.15.45:0/508449242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b00cd300 0x7f88b015daf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88a000bae0 tx=0x7f88a000bdf0 comp rx=0 tx=0).stop 2023-08-27T11:13:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.768+0000 7f88b74e9700 1 -- 172.21.15.45:0/508449242 shutdown_connections 2023-08-27T11:13:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.768+0000 7f88b74e9700 1 --2- 172.21.15.45:0/508449242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015a0f0 0x7f88b00cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.768+0000 7f88b74e9700 1 --2- 172.21.15.45:0/508449242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b00cd300 0x7f88b015daf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.768+0000 7f88b74e9700 1 -- 172.21.15.45:0/508449242 >> 172.21.15.45:0/508449242 conn(0x7f88b00cc7e0 msgr2=0x7f88b00ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:27.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.768+0000 7f88b74e9700 1 -- 172.21.15.45:0/508449242 shutdown_connections 2023-08-27T11:13:27.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.768+0000 7f88b74e9700 1 -- 172.21.15.45:0/508449242 wait complete. 2023-08-27T11:13:27.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.769+0000 7f88b74e9700 1 Processor -- start 2023-08-27T11:13:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.769+0000 7f88b74e9700 1 -- start start 2023-08-27T11:13:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b74e9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88b00cd300 0x7f88b015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b74e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b015a0f0 0x7f88b015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b74e9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015e030 0x7f88b01f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b74e9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b015afa0 con 0x7f88b015a0f0 2023-08-27T11:13:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b74e9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b015b1e0 con 0x7f88b00cd300 2023-08-27T11:13:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b74e9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b015b4c0 con 0x7f88b015e030 2023-08-27T11:13:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b4a84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b015a0f0 0x7f88b015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b4a84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b015a0f0 0x7f88b015aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53848/0 (socket says 172.21.15.45:53848) 2023-08-27T11:13:27.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b4a84700 1 -- 172.21.15.45:0/3886987878 learned_addr learned my addr 172.21.15.45:0/3886987878 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b5a86700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015e030 0x7f88b01f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b5285700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88b00cd300 0x7f88b015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b4a84700 1 -- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015e030 msgr2=0x7f88b01f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b4a84700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015e030 0x7f88b01f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b4a84700 1 -- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88b00cd300 msgr2=0x7f88b015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b4a84700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88b00cd300 0x7f88b015a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b4a84700 1 -- 172.21.15.45:0/3886987878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a000b740 con 0x7f88b015a0f0 2023-08-27T11:13:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.770+0000 7f88b4a84700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b015a0f0 0x7f88b015aa60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f88a000b2c0 tx=0x7f88a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.771+0000 7f88a5ffb700 1 -- 172.21.15.45:0/3886987878 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0003da0 con 0x7f88b015a0f0 2023-08-27T11:13:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.771+0000 7f88a5ffb700 1 -- 172.21.15.45:0/3886987878 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0010470 con 0x7f88b015a0f0 2023-08-27T11:13:27.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.771+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b015b7a0 con 0x7f88b015a0f0 2023-08-27T11:13:27.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.771+0000 7f88a5ffb700 1 -- 172.21.15.45:0/3886987878 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00107e0 con 0x7f88b015a0f0 2023-08-27T11:13:27.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.771+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b00c4c40 con 0x7f88b015a0f0 2023-08-27T11:13:27.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.772+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b00726d0 con 0x7f88b015a0f0 2023-08-27T11:13:27.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.772+0000 7f88a5ffb700 1 -- 172.21.15.45:0/3886987878 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f88a0019400 con 0x7f88b015a0f0 2023-08-27T11:13:27.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.773+0000 7f88a5ffb700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f889406a0e0 0x7f889406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:27.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.773+0000 7f88a5ffb700 1 -- 172.21.15.45:0/3886987878 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f88a008a120 con 0x7f88b015a0f0 2023-08-27T11:13:27.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.773+0000 7f88b5285700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f889406a0e0 0x7f889406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:27.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.774+0000 7f88b5285700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f889406a0e0 0x7f889406c5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88b00b1390 tx=0x7f88ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:27.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.777+0000 7f88a5ffb700 1 -- 172.21.15.45:0/3886987878 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f88a00558b0 con 0x7f88b015a0f0 2023-08-27T11:13:27.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:27.989+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f88b00734a0 con 0x7f88b015a0f0 2023-08-27T11:13:28.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.864+0000 7f88a5ffb700 1 -- 172.21.15.45:0/3886987878 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) v1 ==== 178+0+0 (secure 0 0 0) 0x7f88a0058ed0 con 0x7f88b015a0f0 2023-08-27T11:13:28.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f889406a0e0 msgr2=0x7f889406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:28.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f889406a0e0 0x7f889406c5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88b00b1390 tx=0x7f88ac008040 comp rx=0 tx=0).stop 2023-08-27T11:13:28.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b015a0f0 msgr2=0x7f88b015aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:28.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b015a0f0 0x7f88b015aa60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f88a000b2c0 tx=0x7f88a0004750 comp rx=0 tx=0).stop 2023-08-27T11:13:28.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 shutdown_connections 2023-08-27T11:13:28.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f889406a0e0 0x7f889406c5a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:28.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f88b015e030 0x7f88b01f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:28.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88b00cd300 0x7f88b015a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:28.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 --2- 172.21.15.45:0/3886987878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88b015a0f0 0x7f88b015aa60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:28.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 >> 172.21.15.45:0/3886987878 conn(0x7f88b00cc7e0 msgr2=0x7f88b0157580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:28.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.869+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 shutdown_connections 2023-08-27T11:13:28.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:28.870+0000 7f88b74e9700 1 -- 172.21.15.45:0/3886987878 wait complete. 2023-08-27T11:13:28.873 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' removed 2023-08-27T11:13:28.886 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:13:28.886 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:13:28.886 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:13:28.887 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:13:28.887 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:13:28.887 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:13:28.887 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:13:28.888 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:13:28.888 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:13:28.888 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:13:28.889 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:13:28.889 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:13:28.889 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:13:28.889 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:13:28.890 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:13:28.890 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:13:28.898 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:13:28.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:13:28.901 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:13:28.902 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:13:28.903 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:13:28.903 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:13:28.904 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:13:28.904 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:13:28.909 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:13:28.910 DEBUG:teuthology.parallel:result is None 2023-08-27T11:13:28.911 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:13:28.912 DEBUG:teuthology.parallel:result is None 2023-08-27T11:13:28.950 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:13:28.950 DEBUG:teuthology.parallel:result is None 2023-08-27T11:13:28.953 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:13:28.954 DEBUG:teuthology.parallel:result is None 2023-08-27T11:13:28.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:13:28.997 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-08-27T11:13:28.998 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-08-27T11:13:29.039 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2023-08-27T11:13:29.040 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2023-08-27T11:13:29.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.090+0000 7f3bc42d1700 1 Processor -- start 2023-08-27T11:13:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.090+0000 7f3bc42d1700 1 -- start start 2023-08-27T11:13:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.091+0000 7f3bc42d1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0ca620 0x7f3bbc0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.091+0000 7f3bc42d1700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3bbc0af200 0x7f3bbc0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.091+0000 7f3bc42d1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bbc0ce340 con 0x7f3bbc0caa00 2023-08-27T11:13:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.091+0000 7f3bc42d1700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bbc0ceec0 con 0x7f3bbc0af200 2023-08-27T11:13:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.091+0000 7f3bc42d1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc15d7c0 con 0x7f3bbc0ca620 2023-08-27T11:13:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.091+0000 7f3bc286e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3bbc0caa00 0x7f3bbc0acdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40658/0 (socket says 172.21.15.45:40658) 2023-08-27T11:13:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.091+0000 7f3bc286e700 1 -- 172.21.15.45:0/520368253 learned_addr learned my addr 172.21.15.45:0/520368253 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc206d700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0ca620 0x7f3bbc0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc186c700 1 --2- 172.21.15.45:0/520368253 >> v2:172.21.15.135:3300/0 conn(0x7f3bbc0af200 0x7f3bbc0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295409320 0 0) 0x7f3bbc0ce340 con 0x7f3bbc0caa00 2023-08-27T11:13:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ba80035b0 con 0x7f3bbc0caa00 2023-08-27T11:13:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 810607579 0 0) 0x7f3ba80035b0 con 0x7f3bbc0caa00 2023-08-27T11:13:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0ca620 msgr2=0x7f3bbc0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0ca620 0x7f3bbc0cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 >> v2:172.21.15.135:3300/0 conn(0x7f3bbc0af200 msgr2=0x7f3bbc0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 --2- 172.21.15.45:0/520368253 >> v2:172.21.15.135:3300/0 conn(0x7f3bbc0af200 0x7f3bbc0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0cc570 con 0x7f3bbc0caa00 2023-08-27T11:13:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f3bb40019f0 con 0x7f3bbc0caa00 2023-08-27T11:13:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc186c700 1 --2- 172.21.15.45:0/520368253 >> v2:172.21.15.135:3300/0 conn(0x7f3bbc0af200 0x7f3bbc0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 >> v1:172.21.15.45:6789/0 conn(0x7f3bbc0caa00 legacy=0x7f3bbc0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0ca620 0x7f3ba8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ba8003f80 0x7f3ba8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3ba8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0ceec0 con 0x7f3ba8004d90 2023-08-27T11:13:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc15d7c0 con 0x7f3ba8003f80 2023-08-27T11:13:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba800ba60 con 0x7f3bbc0ca620 2023-08-27T11:13:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc286e700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3ba8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc206d700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0ca620 0x7f3ba8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.092+0000 7f3bc186c700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ba8003f80 0x7f3ba8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc286e700 1 -- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0ca620 msgr2=0x7f3ba8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc286e700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0ca620 0x7f3ba8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc286e700 1 -- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ba8003f80 msgr2=0x7f3ba8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc286e700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ba8003f80 0x7f3ba8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc286e700 1 -- 172.21.15.45:0/520368253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80035b0 con 0x7f3ba8004d90 2023-08-27T11:13:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc286e700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3ba8005210 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400a8d0 tx=0x7f3ba8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bf572e9aef0c4f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400a340 con 0x7f3ba8004d90 2023-08-27T11:13:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb4012600 con 0x7f3ba8004d90 2023-08-27T11:13:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc106b700 1 -- 172.21.15.45:0/520368253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40129b0 con 0x7f3ba8004d90 2023-08-27T11:13:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc42d1700 1 -- 172.21.15.45:0/520368253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 msgr2=0x7f3ba8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc42d1700 1 --2- 172.21.15.45:0/520368253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3ba8005210 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400a8d0 tx=0x7f3ba8005880 comp rx=0 tx=0).stop 2023-08-27T11:13:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc286e700 1 -- 172.21.15.45:0/520368253 reap_dead start 2023-08-27T11:13:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc42d1700 1 -- 172.21.15.45:0/520368253 shutdown_connections 2023-08-27T11:13:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.093+0000 7f3bc42d1700 1 -- 172.21.15.45:0/520368253 >> 172.21.15.45:0/520368253 conn(0x7f3bbc0cc7e0 msgr2=0x7f3bbc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.094+0000 7f3bc42d1700 1 -- 172.21.15.45:0/520368253 shutdown_connections 2023-08-27T11:13:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.094+0000 7f3bc42d1700 1 -- 172.21.15.45:0/520368253 wait complete. 2023-08-27T11:13:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.095+0000 7f3bc42d1700 1 Processor -- start 2023-08-27T11:13:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.095+0000 7f3bc42d1700 1 -- start start 2023-08-27T11:13:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc42d1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3bbc1e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc42d1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bbc0c48e0 0x7f3bbc1f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc42d1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0c4d60 0x7f3bbc1f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc42d1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0c5f20 con 0x7f3ba8004d90 2023-08-27T11:13:29.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc42d1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0c5310 con 0x7f3bbc0c48e0 2023-08-27T11:13:29.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc42d1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0c55f0 con 0x7f3bbc0c4d60 2023-08-27T11:13:29.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc206d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3bbc1e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:29.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc206d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3bbc1e8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53890/0 (socket says 172.21.15.45:53890) 2023-08-27T11:13:29.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc206d700 1 -- 172.21.15.45:0/2706117956 learned_addr learned my addr 172.21.15.45:0/2706117956 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:29.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc186c700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bbc0c48e0 0x7f3bbc1f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc286e700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0c4d60 0x7f3bbc1f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc206d700 1 -- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0c4d60 msgr2=0x7f3bbc1f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc206d700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0c4d60 0x7f3bbc1f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc206d700 1 -- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bbc0c48e0 msgr2=0x7f3bbc1f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc206d700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bbc0c48e0 0x7f3bbc1f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc206d700 1 -- 172.21.15.45:0/2706117956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0c58d0 con 0x7f3ba8004d90 2023-08-27T11:13:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc206d700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3bbc1e8b10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8011aa0 tx=0x7f3bb8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bb2ffd700 1 -- 172.21.15.45:0/2706117956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8011ee0 con 0x7f3ba8004d90 2023-08-27T11:13:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bb2ffd700 1 -- 172.21.15.45:0/2706117956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb800f940 con 0x7f3ba8004d90 2023-08-27T11:13:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bb2ffd700 1 -- 172.21.15.45:0/2706117956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800fcf0 con 0x7f3ba8004d90 2023-08-27T11:13:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc0c5bb0 con 0x7f3ba8004d90 2023-08-27T11:13:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.096+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc15bde0 con 0x7f3ba8004d90 2023-08-27T11:13:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.097+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc0c6200 con 0x7f3ba8004d90 2023-08-27T11:13:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.098+0000 7f3bb2ffd700 1 -- 172.21.15.45:0/2706117956 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3bb8014440 con 0x7f3ba8004d90 2023-08-27T11:13:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.098+0000 7f3bb2ffd700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ba006a0e0 0x7f3ba006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.098+0000 7f3bb2ffd700 1 -- 172.21.15.45:0/2706117956 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f3bb8088980 con 0x7f3ba8004d90 2023-08-27T11:13:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.098+0000 7f3bc186c700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ba006a0e0 0x7f3ba006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.099+0000 7f3bc186c700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ba006a0e0 0x7f3ba006c5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0c35e0 tx=0x7f3bac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.103+0000 7f3bb2ffd700 1 -- 172.21.15.45:0/2706117956 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3bb8057e10 con 0x7f3ba8004d90 2023-08-27T11:13:29.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:29.316+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3bbc15c0c0 con 0x7f3ba8004d90 2023-08-27T11:13:30.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.631+0000 7f3bb2ffd700 1 -- 172.21.15.45:0/2706117956 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3bb8022020 con 0x7f3ba8004d90 2023-08-27T11:13:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ba006a0e0 msgr2=0x7f3ba006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ba006a0e0 0x7f3ba006c5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0c35e0 tx=0x7f3bac008040 comp rx=0 tx=0).stop 2023-08-27T11:13:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 msgr2=0x7f3bbc1e8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3bbc1e8b10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8011aa0 tx=0x7f3bb8011db0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 shutdown_connections 2023-08-27T11:13:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ba8004d90 0x7f3bbc1e8b10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3ba006a0e0 0x7f3ba006c5a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3bbc0c4d60 0x7f3bbc1f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 --2- 172.21.15.45:0/2706117956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3bbc0c48e0 0x7f3bbc1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.634+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 >> 172.21.15.45:0/2706117956 conn(0x7f3bbc0cc7e0 msgr2=0x7f3bbc0ad690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.635+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 shutdown_connections 2023-08-27T11:13:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.635+0000 7f3bc42d1700 1 -- 172.21.15.45:0/2706117956 wait complete. 2023-08-27T11:13:30.638 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-08-27T11:13:30.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:13:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.822+0000 7f43bc0ef700 1 Processor -- start 2023-08-27T11:13:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.823+0000 7f43bc0ef700 1 -- start start 2023-08-27T11:13:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.823+0000 7f43bc0ef700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43b4070e40 0x7f43b4071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.823+0000 7f43bc0ef700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f43b415dbc0 0x7f43b415a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.823+0000 7f43bc0ef700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b415a750 0x7f43b40ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.823+0000 7f43bc0ef700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b40cac50 con 0x7f43b4070e40 2023-08-27T11:13:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.823+0000 7f43bc0ef700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b40caef0 con 0x7f43b415dbc0 2023-08-27T11:13:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.823+0000 7f43bc0ef700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b40ce000 con 0x7f43b415a750 2023-08-27T11:13:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b9e8b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43b4070e40 0x7f43b4071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b9e8b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43b4070e40 0x7f43b4071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53902/0 (socket says 172.21.15.45:53902) 2023-08-27T11:13:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b9e8b700 1 -- 172.21.15.45:0/3860830011 learned_addr learned my addr 172.21.15.45:0/3860830011 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b968a700 1 --2- 172.21.15.45:0/3860830011 >> v2:172.21.15.135:3300/0 conn(0x7f43b415dbc0 0x7f43b415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43ba68c700 1 --2- 172.21.15.45:0/3860830011 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b415a750 0x7f43b40ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b9e8b700 1 -- 172.21.15.45:0/3860830011 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b415a750 msgr2=0x7f43b40ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b9e8b700 1 --2- 172.21.15.45:0/3860830011 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b415a750 0x7f43b40ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b9e8b700 1 -- 172.21.15.45:0/3860830011 >> v2:172.21.15.135:3300/0 conn(0x7f43b415dbc0 msgr2=0x7f43b415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b9e8b700 1 --2- 172.21.15.45:0/3860830011 >> v2:172.21.15.135:3300/0 conn(0x7f43b415dbc0 0x7f43b415a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b9e8b700 1 -- 172.21.15.45:0/3860830011 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b415b5c0 con 0x7f43b4070e40 2023-08-27T11:13:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b968a700 1 --2- 172.21.15.45:0/3860830011 >> v2:172.21.15.135:3300/0 conn(0x7f43b415dbc0 0x7f43b415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b9e8b700 1 --2- 172.21.15.45:0/3860830011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b4070e40 0x7f43b4071220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f43b4070a60 tx=0x7f43b000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf55849151c43da server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b8e89700 1 -- 172.21.15.45:0/3860830011 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000dd30 con 0x7f43b4070e40 2023-08-27T11:13:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b8e89700 1 -- 172.21.15.45:0/3860830011 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b000de90 con 0x7f43b4070e40 2023-08-27T11:13:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.824+0000 7f43b8e89700 1 -- 172.21.15.45:0/3860830011 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0010690 con 0x7f43b4070e40 2023-08-27T11:13:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.825+0000 7f43bc0ef700 1 -- 172.21.15.45:0/3860830011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b4070e40 msgr2=0x7f43b4071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:30.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.825+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/3860830011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b4070e40 0x7f43b4071220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f43b4070a60 tx=0x7f43b000be70 comp rx=0 tx=0).stop 2023-08-27T11:13:30.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.825+0000 7f43bc0ef700 1 -- 172.21.15.45:0/3860830011 shutdown_connections 2023-08-27T11:13:30.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.825+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/3860830011 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b415a750 0x7f43b40ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.825+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/3860830011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b4070e40 0x7f43b4071220 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.825+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/3860830011 >> v2:172.21.15.135:3300/0 conn(0x7f43b415dbc0 0x7f43b415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.825+0000 7f43bc0ef700 1 -- 172.21.15.45:0/3860830011 >> 172.21.15.45:0/3860830011 conn(0x7f43b40cfd60 msgr2=0x7f43b40d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:30.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.825+0000 7f43bc0ef700 1 -- 172.21.15.45:0/3860830011 shutdown_connections 2023-08-27T11:13:30.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.825+0000 7f43bc0ef700 1 -- 172.21.15.45:0/3860830011 wait complete. 2023-08-27T11:13:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.826+0000 7f43bc0ef700 1 Processor -- start 2023-08-27T11:13:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.826+0000 7f43bc0ef700 1 -- start start 2023-08-27T11:13:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43bc0ef700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b4070e40 0x7f43b40b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43bc0ef700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b415a750 0x7f43b40c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43bc0ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b415dbc0 0x7f43b40c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43bc0ef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b41450e0 con 0x7f43b415dbc0 2023-08-27T11:13:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43bc0ef700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4144460 con 0x7f43b415a750 2023-08-27T11:13:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43bc0ef700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4144740 con 0x7f43b4070e40 2023-08-27T11:13:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43ba68c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b415dbc0 0x7f43b40c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43ba68c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b415dbc0 0x7f43b40c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53912/0 (socket says 172.21.15.45:53912) 2023-08-27T11:13:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43ba68c700 1 -- 172.21.15.45:0/670494925 learned_addr learned my addr 172.21.15.45:0/670494925 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43b968a700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b415a750 0x7f43b40c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43b9e8b700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b4070e40 0x7f43b40b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43ba68c700 1 -- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b4070e40 msgr2=0x7f43b40b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43ba68c700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b4070e40 0x7f43b40b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43ba68c700 1 -- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b415a750 msgr2=0x7f43b40c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:30.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43ba68c700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b415a750 0x7f43b40c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:30.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.827+0000 7f43ba68c700 1 -- 172.21.15.45:0/670494925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b000b7f0 con 0x7f43b415dbc0 2023-08-27T11:13:30.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.828+0000 7f43ba68c700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b415dbc0 0x7f43b40c9610 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f43ac007e60 tx=0x7f43ac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:30.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.828+0000 7f43a67fc700 1 -- 172.21.15.45:0/670494925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac00ed30 con 0x7f43b415dbc0 2023-08-27T11:13:30.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.828+0000 7f43a67fc700 1 -- 172.21.15.45:0/670494925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac014420 con 0x7f43b415dbc0 2023-08-27T11:13:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.828+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b4144a20 con 0x7f43b415dbc0 2023-08-27T11:13:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.828+0000 7f43a67fc700 1 -- 172.21.15.45:0/670494925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac0147d0 con 0x7f43b415dbc0 2023-08-27T11:13:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.828+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b41440f0 con 0x7f43b415dbc0 2023-08-27T11:13:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.829+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b4072180 con 0x7f43b415dbc0 2023-08-27T11:13:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.829+0000 7f43a67fc700 1 -- 172.21.15.45:0/670494925 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f43ac014930 con 0x7f43b415dbc0 2023-08-27T11:13:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.830+0000 7f43a67fc700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f439806a0e0 0x7f439806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.830+0000 7f43a67fc700 1 -- 172.21.15.45:0/670494925 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f43ac088a50 con 0x7f43b415dbc0 2023-08-27T11:13:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.830+0000 7f43b9e8b700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f439806a0e0 0x7f439806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.831+0000 7f43b9e8b700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f439806a0e0 0x7f439806c5a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f43b000beb0 tx=0x7f43b0012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:30.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:30.834+0000 7f43a67fc700 1 -- 172.21.15.45:0/670494925 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f43ac054360 con 0x7f43b415dbc0 2023-08-27T11:13:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.117+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f43b4072780 con 0x7f43b415dbc0 2023-08-27T11:13:31.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.117+0000 7f43a67fc700 1 -- 172.21.15.45:0/670494925 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f43ac057980 con 0x7f43b415dbc0 2023-08-27T11:13:31.118 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:13:31.119 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-08-27T11:13:31.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.119 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-08-27T11:13:31.119 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tknzGlBhAABVGsUvHGyOdTgH9miKO0Cg==", 2023-08-27T11:13:31.120 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.120 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:13:31.120 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:13:31.120 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:13:31.120 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.121 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.121 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-08-27T11:13:31.121 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkEWbPExAAgt/9BpPD1vK9QoG8l3HeFw==", 2023-08-27T11:13:31.122 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.122 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:13:31.122 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:13:31.122 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:13:31.122 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.122 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.123 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-08-27T11:13:31.123 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk6qRCDhAAGaj+NY32IDjdRcuSNbASpw==", 2023-08-27T11:13:31.123 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:13:31.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:13:31.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:13:31.124 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.124 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.125 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-08-27T11:13:31.125 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkE9NxGxAAnWum05rSbYQSLZ9IjjqDpQ==", 2023-08-27T11:13:31.125 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:13:31.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:13:31.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:13:31.126 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.127 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-08-27T11:13:31.127 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBML+tkOzvBBBAA196QdZiLzcsm5hLBGDl0Qw==", 2023-08-27T11:13:31.127 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.128 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:13:31.128 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:13:31.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.128 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.129 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-08-27T11:13:31.129 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBPL+tkIXaxBBAAfHwZ2SsmOENhnNxZsG230Q==", 2023-08-27T11:13:31.129 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.129 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:13:31.130 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:13:31.130 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.130 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-08-27T11:13:31.131 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBSL+tkpzphBBAA37UzwlmuLdSWDIrkBu/8Rw==", 2023-08-27T11:13:31.131 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.131 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:13:31.131 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:13:31.131 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.132 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-08-27T11:13:31.132 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVL+tkt63hAxAAJReGReJHqyggSbdFEhm3+A==", 2023-08-27T11:13:31.133 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.133 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:13:31.133 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:13:31.133 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.133 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.134 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-08-27T11:13:31.134 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBbL+tkTOFXBxAAZ1I44e50IrNzgj7mjKyuGA==", 2023-08-27T11:13:31.134 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.134 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:13:31.135 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:13:31.135 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.135 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.135 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-08-27T11:13:31.136 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBeL+tkdezFBhAANof2C80WiBZ6hWm3oB3kfg==", 2023-08-27T11:13:31.136 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.136 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:13:31.136 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:13:31.136 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.137 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.137 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-08-27T11:13:31.137 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBhL+tk7auXBhAAJPwKwTVIUzao5D3t4iXYcw==", 2023-08-27T11:13:31.137 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.138 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:13:31.138 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:13:31.138 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.139 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-08-27T11:13:31.139 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBkL+tkY+ztBRAAvM2oxELLN8WV0i0spqfY6Q==", 2023-08-27T11:13:31.139 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.139 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:13:31.139 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:13:31.140 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.140 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.140 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-08-27T11:13:31.141 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ==", 2023-08-27T11:13:31.141 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.141 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:13:31.141 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:13:31.141 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:13:31.142 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:13:31.142 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.142 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-08-27T11:13:31.142 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkMCDoJhAAevee0PlTshlWJcPzX8fpYg==", 2023-08-27T11:13:31.143 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.143 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:13:31.143 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:13:31.143 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:13:31.143 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:13:31.144 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.144 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-08-27T11:13:31.144 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk1xh+IxAAswoUpd+D70hUwfeKI2gc0w==", 2023-08-27T11:13:31.144 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:13:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:13:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:13:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:13:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-08-27T11:13:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkGWuaLBAAYM2t+TMEs59u635jL9f9kg==", 2023-08-27T11:13:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:13:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:13:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:13:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:13:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.148 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-08-27T11:13:31.148 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkkw7pGRAAVim28nKfumQg/LcBfJpivA==", 2023-08-27T11:13:31.148 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.148 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:13:31.149 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:13:31.149 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:13:31.149 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.149 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.150 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-mds", 2023-08-27T11:13:31.150 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBvL+tklU5vBRAALjSUTuAw8F7YmGft/KrEVA==", 2023-08-27T11:13:31.150 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.150 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-mds" 2023-08-27T11:13:31.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.151 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-mgr", 2023-08-27T11:13:31.151 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBvL+tk+FpvBRAADvZU9ihmWNP1jGwzYm92PQ==", 2023-08-27T11:13:31.151 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.151 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-mgr" 2023-08-27T11:13:31.152 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.152 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-osd", 2023-08-27T11:13:31.152 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBvL+tkIWhvBRAAKRaj9aYp/1NOp4C+eZYmAA==", 2023-08-27T11:13:31.153 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.153 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-osd" 2023-08-27T11:13:31.153 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.154 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rbd", 2023-08-27T11:13:31.154 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBvL+tkn3FvBRAAEhLJ4DAk0++J8yfW/Dzk2Q==", 2023-08-27T11:13:31.154 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.154 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rbd" 2023-08-27T11:13:31.154 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.155 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-08-27T11:13:31.155 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBvL+tkIXtvBRAAzeaegWf/AxV/WFWWY/+sDw==", 2023-08-27T11:13:31.155 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.155 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-08-27T11:13:31.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.156 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rgw", 2023-08-27T11:13:31.156 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBvL+tkWYRvBRAAfnoL3ZF3jSDbB34KIYmCRw==", 2023-08-27T11:13:31.156 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.157 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rgw" 2023-08-27T11:13:31.157 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.157 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-08-27T11:13:31.158 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkNxfXAhAA/K5r8Mx/C7nRitNuzXXvKg==", 2023-08-27T11:13:31.158 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.158 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:13:31.158 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:13:31.158 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:31.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:31.159 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-08-27T11:13:31.159 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkW6B8NRAAF2hsNVRi3Ji4M8anuYwpDg==", 2023-08-27T11:13:31.159 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:13:31.160 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:13:31.160 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:13:31.160 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:13:31.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:31.160 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-08-27T11:13:31.161 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:13:31.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f439806a0e0 msgr2=0x7f439806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f439806a0e0 0x7f439806c5a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f43b000beb0 tx=0x7f43b0012000 comp rx=0 tx=0).stop 2023-08-27T11:13:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b415dbc0 msgr2=0x7f43b40c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b415dbc0 0x7f43b40c9610 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f43ac007e60 tx=0x7f43ac00e3c0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 shutdown_connections 2023-08-27T11:13:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43b415dbc0 0x7f43b40c9610 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f439806a0e0 0x7f439806c5a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f43b4070e40 0x7f43b40b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 --2- 172.21.15.45:0/670494925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b415a750 0x7f43b40c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 >> 172.21.15.45:0/670494925 conn(0x7f43b40cfd60 msgr2=0x7f43b406d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.119+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 shutdown_connections 2023-08-27T11:13:31.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.120+0000 7f43bc0ef700 1 -- 172.21.15.45:0/670494925 wait complete. 2023-08-27T11:13:31.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-08-27T11:13:31.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.291+0000 7f1ca1271700 1 Processor -- start 2023-08-27T11:13:31.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.291+0000 7f1ca1271700 1 -- start start 2023-08-27T11:13:31.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1ca1271700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c9c0b8c20 0x7f1c9c0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1ca1271700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c0ce2e0 0x7f1c9c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1ca1271700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9c15d6a0 con 0x7f1c9c15d2c0 2023-08-27T11:13:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1ca1271700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9c0ce770 con 0x7f1c9c0b8c20 2023-08-27T11:13:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1ca1271700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c0d2c90 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c9a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c9c0b8c20 0x7f1c9c0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c9affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1c9c15d2c0 0x7f1c9c0d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40692/0 (socket says 172.21.15.45:40692) 2023-08-27T11:13:31.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c9affd700 1 -- 172.21.15.45:0/1707469235 learned_addr learned my addr 172.21.15.45:0/1707469235 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:31.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c9b7fe700 1 --2- 172.21.15.45:0/1707469235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c0ce2e0 0x7f1c9c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c9a7fc700 1 -- 172.21.15.45:0/1707469235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c0ce2e0 msgr2=0x7f1c9c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:31.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c9a7fc700 1 --2- 172.21.15.45:0/1707469235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c0ce2e0 0x7f1c9c0b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c9a7fc700 1 -- 172.21.15.45:0/1707469235 >> v1:172.21.15.45:6789/0 conn(0x7f1c9c15d2c0 legacy=0x7f1c9c0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c9a7fc700 1 -- 172.21.15.45:0/1707469235 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c0d1bf0 con 0x7f1c9c0b8c20 2023-08-27T11:13:31.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c99ffb700 1 -- 172.21.15.45:0/1707469235 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124002714 0 0) 0x7f1c8c001110 con 0x7f1c9c15d2c0 2023-08-27T11:13:31.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.292+0000 7f1c9a7fc700 1 --2- 172.21.15.45:0/1707469235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0b8c20 0x7f1c9c0cdda0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009870 tx=0x7f1c90009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9451aa26a9af464f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:31.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1c99ffb700 1 -- 172.21.15.45:0/1707469235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90011830 con 0x7f1c9c0b8c20 2023-08-27T11:13:31.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1c99ffb700 1 -- 172.21.15.45:0/1707469235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c90011990 con 0x7f1c9c0b8c20 2023-08-27T11:13:31.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1c99ffb700 1 -- 172.21.15.45:0/1707469235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90011d20 con 0x7f1c9c0b8c20 2023-08-27T11:13:31.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1ca1271700 1 -- 172.21.15.45:0/1707469235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0b8c20 msgr2=0x7f1c9c0cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1ca1271700 1 --2- 172.21.15.45:0/1707469235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0b8c20 0x7f1c9c0cdda0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009870 tx=0x7f1c90009b80 comp rx=0 tx=0).stop 2023-08-27T11:13:31.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1ca1271700 1 -- 172.21.15.45:0/1707469235 shutdown_connections 2023-08-27T11:13:31.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1ca1271700 1 --2- 172.21.15.45:0/1707469235 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c0ce2e0 0x7f1c9c0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1ca1271700 1 --2- 172.21.15.45:0/1707469235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0b8c20 0x7f1c9c0cdda0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1ca1271700 1 -- 172.21.15.45:0/1707469235 >> 172.21.15.45:0/1707469235 conn(0x7f1c9c0d2fd0 msgr2=0x7f1c9c0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:31.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1ca1271700 1 -- 172.21.15.45:0/1707469235 shutdown_connections 2023-08-27T11:13:31.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.293+0000 7f1ca1271700 1 -- 172.21.15.45:0/1707469235 wait complete. 2023-08-27T11:13:31.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.295+0000 7f1ca1271700 1 Processor -- start 2023-08-27T11:13:31.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.295+0000 7f1ca1271700 1 -- start start 2023-08-27T11:13:31.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1ca1271700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c9c0b8c20 0x7f1c9c1f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1ca1271700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0ce2e0 0x7f1c9c15d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1ca1271700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c15d2c0 0x7f1c9c0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1ca1271700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c1447f0 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1ca1271700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c15deb0 con 0x7f1c9c0b8c20 2023-08-27T11:13:31.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1ca1271700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c15e190 con 0x7f1c9c15d2c0 2023-08-27T11:13:31.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0ce2e0 0x7f1c9c15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0ce2e0 0x7f1c9c15d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53922/0 (socket says 172.21.15.45:53922) 2023-08-27T11:13:31.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9a7fc700 1 -- 172.21.15.45:0/1219447090 learned_addr learned my addr 172.21.15.45:0/1219447090 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:31.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9a7fc700 1 -- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c15d2c0 msgr2=0x7f1c9c0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9b7fe700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c15d2c0 0x7f1c9c0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9affd700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c9c0b8c20 0x7f1c9c1f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9a7fc700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c15d2c0 0x7f1c9c0b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9a7fc700 1 -- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c9c0b8c20 msgr2=0x7f1c9c1f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9a7fc700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c9c0b8c20 0x7f1c9c1f2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9a7fc700 1 -- 172.21.15.45:0/1219447090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c900094a0 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9b7fe700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c15d2c0 0x7f1c9c0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:31.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9affd700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c9c0b8c20 0x7f1c9c1f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:31.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.296+0000 7f1c9a7fc700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0ce2e0 0x7f1c9c15d840 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009840 tx=0x7f1c90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:31.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.297+0000 7f1c837fe700 1 -- 172.21.15.45:0/1219447090 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c900036a0 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.297+0000 7f1c837fe700 1 -- 172.21.15.45:0/1219447090 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9001e070 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.297+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c15e470 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.297+0000 7f1c837fe700 1 -- 172.21.15.45:0/1219447090 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90003a60 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.297+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c15ea30 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.298+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9c15e2d0 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.298+0000 7f1c837fe700 1 -- 172.21.15.45:0/1219447090 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1c90003bc0 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.299+0000 7f1c837fe700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8806a0e0 0x7f1c8806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.299+0000 7f1c837fe700 1 -- 172.21.15.45:0/1219447090 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1c900892c0 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.299+0000 7f1c9affd700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8806a0e0 0x7f1c8806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.299+0000 7f1c9affd700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8806a0e0 0x7f1c8806c5a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c007fd0 tx=0x7f1c8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:31.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.303+0000 7f1c837fe700 1 -- 172.21.15.45:0/1219447090 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1c9c15e2d0 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.586+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f1c9c15e2d0 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.590+0000 7f1c837fe700 1 -- 172.21.15.45:0/1219447090 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1c9c15e2d0 con 0x7f1c9c0ce2e0 2023-08-27T11:13:31.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8806a0e0 msgr2=0x7f1c8806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8806a0e0 0x7f1c8806c5a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c007fd0 tx=0x7f1c8c009040 comp rx=0 tx=0).stop 2023-08-27T11:13:31.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0ce2e0 msgr2=0x7f1c9c15d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0ce2e0 0x7f1c9c15d840 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009840 tx=0x7f1c90004750 comp rx=0 tx=0).stop 2023-08-27T11:13:31.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 shutdown_connections 2023-08-27T11:13:31.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f1c8806a0e0 0x7f1c8806c5a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1c9c15d2c0 0x7f1c9c0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c9c0b8c20 0x7f1c9c1f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 --2- 172.21.15.45:0/1219447090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c9c0ce2e0 0x7f1c9c15d840 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 >> 172.21.15.45:0/1219447090 conn(0x7f1c9c0d2fd0 msgr2=0x7f1c9c15b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:31.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 shutdown_connections 2023-08-27T11:13:31.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.592+0000 7f1ca1271700 1 -- 172.21.15.45:0/1219447090 wait complete. 2023-08-27T11:13:31.606 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-08-27T11:13:31.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.731+0000 7f89ea24a700 1 Processor -- start 2023-08-27T11:13:31.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.731+0000 7f89ea24a700 1 -- start start 2023-08-27T11:13:31.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.732+0000 7f89ea24a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f89e4071e10 0x7f89e4071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.732+0000 7f89ea24a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89e40c5740 0x7f89e40c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.732+0000 7f89ea24a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e415ea60 con 0x7f89e415a000 2023-08-27T11:13:31.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.732+0000 7f89ea24a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e40c6130 con 0x7f89e4071e10 2023-08-27T11:13:31.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.732+0000 7f89ea24a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e415b3f0 con 0x7f89e40c5740 2023-08-27T11:13:31.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.732+0000 7f89e37fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f89e415a000 0x7f89e415e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40706/0 (socket says 172.21.15.45:40706) 2023-08-27T11:13:31.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.732+0000 7f89e37fe700 1 -- 172.21.15.45:0/2347718700 learned_addr learned my addr 172.21.15.45:0/2347718700 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:31.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e2ffd700 1 --2- 172.21.15.45:0/2347718700 >> v2:172.21.15.135:3300/0 conn(0x7f89e4071e10 0x7f89e4071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e3fff700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89e40c5740 0x7f89e40c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696796774 0 0) 0x7f89d4001110 con 0x7f89e415a000 2023-08-27T11:13:31.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d00035b0 con 0x7f89e415a000 2023-08-27T11:13:31.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1659844112 0 0) 0x7f89d00035b0 con 0x7f89e415a000 2023-08-27T11:13:31.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89e40c5740 msgr2=0x7f89e40c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:31.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89e40c5740 0x7f89e40c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 >> v2:172.21.15.135:3300/0 conn(0x7f89e4071e10 msgr2=0x7f89e4071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:31.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 --2- 172.21.15.45:0/2347718700 >> v2:172.21.15.135:3300/0 conn(0x7f89e4071e10 0x7f89e4071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e415dec0 con 0x7f89e415a000 2023-08-27T11:13:31.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f89d4001800 con 0x7f89e415a000 2023-08-27T11:13:31.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e3fff700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89e40c5740 0x7f89e40c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:31.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 >> v1:172.21.15.45:6789/0 conn(0x7f89e415a000 legacy=0x7f89e415e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89d0003a00 0x7f89d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89e40c5740 0x7f89d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d0004d90 0x7f89d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40c6130 con 0x7f89d0004d90 2023-08-27T11:13:31.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e415b3f0 con 0x7f89d0003a00 2023-08-27T11:13:31.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d000ba10 con 0x7f89e40c5740 2023-08-27T11:13:31.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e3fff700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d0004d90 0x7f89d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e37fe700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89d0003a00 0x7f89d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.733+0000 7f89e2ffd700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89e40c5740 0x7f89d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e3fff700 1 -- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89e40c5740 msgr2=0x7f89d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:31.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e3fff700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89e40c5740 0x7f89d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e3fff700 1 -- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89d0003a00 msgr2=0x7f89d0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:31.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e3fff700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89d0003a00 0x7f89d0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e3fff700 1 -- 172.21.15.45:0/2347718700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc009a70 con 0x7f89d0004d90 2023-08-27T11:13:31.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e3fff700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d0004d90 0x7f89d0005210 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00e350 tx=0x7f89d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a506ba4da54a395e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:31.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00edf0 con 0x7f89d0004d90 2023-08-27T11:13:31.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00f040 con 0x7f89d0004d90 2023-08-27T11:13:31.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e27fc700 1 -- 172.21.15.45:0/2347718700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0146a0 con 0x7f89d0004d90 2023-08-27T11:13:31.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89ea24a700 1 -- 172.21.15.45:0/2347718700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d0004d90 msgr2=0x7f89d0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89ea24a700 1 --2- 172.21.15.45:0/2347718700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89d0004d90 0x7f89d0005210 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00e350 tx=0x7f89d0005880 comp rx=0 tx=0).stop 2023-08-27T11:13:31.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89e3fff700 1 -- 172.21.15.45:0/2347718700 reap_dead start 2023-08-27T11:13:31.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89ea24a700 1 -- 172.21.15.45:0/2347718700 shutdown_connections 2023-08-27T11:13:31.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.734+0000 7f89ea24a700 1 -- 172.21.15.45:0/2347718700 >> 172.21.15.45:0/2347718700 conn(0x7f89e40cfdd0 msgr2=0x7f89e40d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:31.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.735+0000 7f89ea24a700 1 -- 172.21.15.45:0/2347718700 shutdown_connections 2023-08-27T11:13:31.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.735+0000 7f89ea24a700 1 -- 172.21.15.45:0/2347718700 wait complete. 2023-08-27T11:13:31.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.736+0000 7f89ea24a700 1 Processor -- start 2023-08-27T11:13:31.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.736+0000 7f89ea24a700 1 -- start start 2023-08-27T11:13:31.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89ea24a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89d0004d90 0x7f89e414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89ea24a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89e41516b0 0x7f89e4159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89ea24a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89e414f2e0 0x7f89e414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89ea24a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e414fcd0 con 0x7f89e414f2e0 2023-08-27T11:13:31.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89ea24a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e414b170 con 0x7f89e41516b0 2023-08-27T11:13:31.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89ea24a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41f12f0 con 0x7f89d0004d90 2023-08-27T11:13:31.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89e414f2e0 0x7f89e414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89e414f2e0 0x7f89e414f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53946/0 (socket says 172.21.15.45:53946) 2023-08-27T11:13:31.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e3fff700 1 -- 172.21.15.45:0/3060477766 learned_addr learned my addr 172.21.15.45:0/3060477766 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:31.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e2ffd700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89e41516b0 0x7f89e4159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e37fe700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89d0004d90 0x7f89e414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e3fff700 1 -- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89d0004d90 msgr2=0x7f89e414d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e3fff700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89d0004d90 0x7f89e414d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e3fff700 1 -- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89e41516b0 msgr2=0x7f89e4159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:31.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e3fff700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89e41516b0 0x7f89e4159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:31.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e3fff700 1 -- 172.21.15.45:0/3060477766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e41f1010 con 0x7f89e414f2e0 2023-08-27T11:13:31.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89e3fff700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89e414f2e0 0x7f89e414f790 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f89dc002230 tx=0x7f89dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:31.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.737+0000 7f89dbfff700 1 -- 172.21.15.45:0/3060477766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc014890 con 0x7f89e414f2e0 2023-08-27T11:13:31.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.738+0000 7f89dbfff700 1 -- 172.21.15.45:0/3060477766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc0149f0 con 0x7f89e414f2e0 2023-08-27T11:13:31.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.738+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e414e990 con 0x7f89e414f2e0 2023-08-27T11:13:31.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.738+0000 7f89dbfff700 1 -- 172.21.15.45:0/3060477766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc014de0 con 0x7f89e414f2e0 2023-08-27T11:13:31.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.738+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e4151b30 con 0x7f89e414f2e0 2023-08-27T11:13:31.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.739+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e40727c0 con 0x7f89e414f2e0 2023-08-27T11:13:31.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.739+0000 7f89dbfff700 1 -- 172.21.15.45:0/3060477766 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f89dc0133e0 con 0x7f89e414f2e0 2023-08-27T11:13:31.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.739+0000 7f89dbfff700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f89c406a0e0 0x7f89c406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:31.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.739+0000 7f89dbfff700 1 -- 172.21.15.45:0/3060477766 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f89dc08c0f0 con 0x7f89e414f2e0 2023-08-27T11:13:31.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.740+0000 7f89e37fe700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f89c406a0e0 0x7f89c406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:31.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.740+0000 7f89e37fe700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f89c406a0e0 0x7f89c406c5a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89d40012b0 tx=0x7f89d4007b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:31.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:31.744+0000 7f89dbfff700 1 -- 172.21.15.45:0/3060477766 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f89dc057ba0 con 0x7f89e414f2e0 2023-08-27T11:13:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.030+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f89e406e5e0 con 0x7f89e414f2e0 2023-08-27T11:13:32.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.034+0000 7f89dbfff700 1 -- 172.21.15.45:0/3060477766 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f89dc05b1c0 con 0x7f89e414f2e0 2023-08-27T11:13:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f89c406a0e0 msgr2=0x7f89c406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f89c406a0e0 0x7f89c406c5a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89d40012b0 tx=0x7f89d4007b00 comp rx=0 tx=0).stop 2023-08-27T11:13:32.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89e414f2e0 msgr2=0x7f89e414f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89e414f2e0 0x7f89e414f790 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f89dc002230 tx=0x7f89dc009750 comp rx=0 tx=0).stop 2023-08-27T11:13:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 shutdown_connections 2023-08-27T11:13:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89e414f2e0 0x7f89e414f790 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f89c406a0e0 0x7f89c406c5a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f89d0004d90 0x7f89e414d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 --2- 172.21.15.45:0/3060477766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89e41516b0 0x7f89e4159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 >> 172.21.15.45:0/3060477766 conn(0x7f89e40cfdd0 msgr2=0x7f89e40af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:32.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 shutdown_connections 2023-08-27T11:13:32.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.036+0000 7f89ea24a700 1 -- 172.21.15.45:0/3060477766 wait complete. 2023-08-27T11:13:32.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-08-27T11:13:32.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.174+0000 7fefe5bd5700 1 Processor -- start 2023-08-27T11:13:32.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.174+0000 7fefe5bd5700 1 -- start start 2023-08-27T11:13:32.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.175+0000 7fefe5bd5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe006fa40 0x7fefe00cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.175+0000 7fefe5bd5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fefe00cd2e0 0x7fefe015b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.175+0000 7fefe5bd5700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe015eef0 con 0x7fefe015b2e0 2023-08-27T11:13:32.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.175+0000 7fefe5bd5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe015b6c0 con 0x7fefe00cd2e0 2023-08-27T11:13:32.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.175+0000 7fefe5bd5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe015f190 con 0x7fefe006fa40 2023-08-27T11:13:32.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.175+0000 7fefdffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fefe00cd2e0 0x7fefe015b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.175+0000 7fefdffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fefe00cd2e0 0x7fefe015b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53954/0 (socket says 172.21.15.45:53954) 2023-08-27T11:13:32.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.175+0000 7fefdffff700 1 -- 172.21.15.45:0/2062568083 learned_addr learned my addr 172.21.15.45:0/2062568083 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.175+0000 7fefdeffd700 1 --2- 172.21.15.45:0/2062568083 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe006fa40 0x7fefe00cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefdffff700 1 -- 172.21.15.45:0/2062568083 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe006fa40 msgr2=0x7fefe00cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:32.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefdffff700 1 --2- 172.21.15.45:0/2062568083 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe006fa40 0x7fefe00cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefdffff700 1 -- 172.21.15.45:0/2062568083 >> v1:172.21.15.135:6789/0 conn(0x7fefe015b2e0 legacy=0x7fefe00cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefdffff700 1 -- 172.21.15.45:0/2062568083 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe015f470 con 0x7fefe00cd2e0 2023-08-27T11:13:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefdeffd700 1 --2- 172.21.15.45:0/2062568083 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe006fa40 0x7fefe00cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:32.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefdffff700 1 --2- 172.21.15.45:0/2062568083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe00cd2e0 0x7fefe015b960 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009e50 tx=0x7fefd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa59a40e3cf427f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:32.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefde7fc700 1 -- 172.21.15.45:0/2062568083 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800e070 con 0x7fefe00cd2e0 2023-08-27T11:13:32.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefde7fc700 1 -- 172.21.15.45:0/2062568083 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd800b9c0 con 0x7fefe00cd2e0 2023-08-27T11:13:32.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefde7fc700 1 -- 172.21.15.45:0/2062568083 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800bd70 con 0x7fefe00cd2e0 2023-08-27T11:13:32.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.176+0000 7fefe5bd5700 1 -- 172.21.15.45:0/2062568083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe00cd2e0 msgr2=0x7fefe015b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.177+0000 7fefe5bd5700 1 --2- 172.21.15.45:0/2062568083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe00cd2e0 0x7fefe015b960 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009e50 tx=0x7fefd800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:32.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.177+0000 7fefe5bd5700 1 -- 172.21.15.45:0/2062568083 shutdown_connections 2023-08-27T11:13:32.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.177+0000 7fefe5bd5700 1 --2- 172.21.15.45:0/2062568083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe00cd2e0 0x7fefe015b960 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.177+0000 7fefe5bd5700 1 --2- 172.21.15.45:0/2062568083 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe006fa40 0x7fefe00cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.177+0000 7fefe5bd5700 1 -- 172.21.15.45:0/2062568083 >> 172.21.15.45:0/2062568083 conn(0x7fefe00d0ed0 msgr2=0x7fefe00d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:32.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.177+0000 7fefe5bd5700 1 -- 172.21.15.45:0/2062568083 shutdown_connections 2023-08-27T11:13:32.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.177+0000 7fefe5bd5700 1 -- 172.21.15.45:0/2062568083 wait complete. 2023-08-27T11:13:32.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.178+0000 7fefe5bd5700 1 Processor -- start 2023-08-27T11:13:32.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.178+0000 7fefe5bd5700 1 -- start start 2023-08-27T11:13:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefe5bd5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefe006fa40 0x7fefe014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefe5bd5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe00cd2e0 0x7fefe015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefe5bd5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe015b2e0 0x7fefe0158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefe5bd5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe015df90 con 0x7fefe015b2e0 2023-08-27T11:13:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefe5bd5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe015c050 con 0x7fefe006fa40 2023-08-27T11:13:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefe5bd5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe014ff70 con 0x7fefe00cd2e0 2023-08-27T11:13:32.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe015b2e0 0x7fefe0158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe015b2e0 0x7fefe0158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53960/0 (socket says 172.21.15.45:53960) 2023-08-27T11:13:32.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdffff700 1 -- 172.21.15.45:0/796233479 learned_addr learned my addr 172.21.15.45:0/796233479 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:32.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdf7fe700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefe006fa40 0x7fefe014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdeffd700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe00cd2e0 0x7fefe015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdffff700 1 -- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe00cd2e0 msgr2=0x7fefe015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdffff700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe00cd2e0 0x7fefe015d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdffff700 1 -- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefe006fa40 msgr2=0x7fefe014fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdffff700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefe006fa40 0x7fefe014fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdffff700 1 -- 172.21.15.45:0/796233479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8009b30 con 0x7fefe015b2e0 2023-08-27T11:13:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefdffff700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe015b2e0 0x7fefe0158410 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fefd800b730 tx=0x7fefd8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.179+0000 7fefd7fff700 1 -- 172.21.15.45:0/796233479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd80099b0 con 0x7fefe015b2e0 2023-08-27T11:13:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.180+0000 7fefd7fff700 1 -- 172.21.15.45:0/796233479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd80092e0 con 0x7fefe015b2e0 2023-08-27T11:13:32.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.180+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe015dcb0 con 0x7fefe015b2e0 2023-08-27T11:13:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.180+0000 7fefd7fff700 1 -- 172.21.15.45:0/796233479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd801c640 con 0x7fefe015b2e0 2023-08-27T11:13:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.180+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe015bce0 con 0x7fefe015b2e0 2023-08-27T11:13:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.181+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe015c1c0 con 0x7fefe015b2e0 2023-08-27T11:13:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.181+0000 7fefd7fff700 1 -- 172.21.15.45:0/796233479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fefd801c7a0 con 0x7fefe015b2e0 2023-08-27T11:13:32.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.181+0000 7fefd7fff700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fefc006a0e0 0x7fefc006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.181+0000 7fefd7fff700 1 -- 172.21.15.45:0/796233479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fefd8089fe0 con 0x7fefe015b2e0 2023-08-27T11:13:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.182+0000 7fefdf7fe700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fefc006a0e0 0x7fefc006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.182+0000 7fefdf7fe700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fefc006a0e0 0x7fefc006c5a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fefd0000f60 tx=0x7fefd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:32.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.186+0000 7fefd7fff700 1 -- 172.21.15.45:0/796233479 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fefd8055970 con 0x7fefe015b2e0 2023-08-27T11:13:32.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.472+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fefe00cfaf0 con 0x7fefe015b2e0 2023-08-27T11:13:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.476+0000 7fefd7fff700 1 -- 172.21.15.45:0/796233479 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fefd8058f90 con 0x7fefe015b2e0 2023-08-27T11:13:32.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fefc006a0e0 msgr2=0x7fefc006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fefc006a0e0 0x7fefc006c5a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fefd0000f60 tx=0x7fefd0009040 comp rx=0 tx=0).stop 2023-08-27T11:13:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe015b2e0 msgr2=0x7fefe0158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe015b2e0 0x7fefe0158410 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fefd800b730 tx=0x7fefd8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 shutdown_connections 2023-08-27T11:13:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefe015b2e0 0x7fefe0158410 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fefc006a0e0 0x7fefc006c5a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefe006fa40 0x7fefe014fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 --2- 172.21.15.45:0/796233479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fefe00cd2e0 0x7fefe015d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 >> 172.21.15.45:0/796233479 conn(0x7fefe00d0ed0 msgr2=0x7fefe015e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 shutdown_connections 2023-08-27T11:13:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.478+0000 7fefe5bd5700 1 -- 172.21.15.45:0/796233479 wait complete. 2023-08-27T11:13:32.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-08-27T11:13:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.618+0000 7fde6164d700 1 Processor -- start 2023-08-27T11:13:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.618+0000 7fde6164d700 1 -- start start 2023-08-27T11:13:32.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde6164d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fde5c15d2c0 0x7fde5c0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde6164d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c0b8c20 0x7fde5c0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde6164d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c15d6a0 con 0x7fde5c0ce2e0 2023-08-27T11:13:32.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde6164d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5c0ce770 con 0x7fde5c15d2c0 2023-08-27T11:13:32.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde6164d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0d2c90 con 0x7fde5c0b8c20 2023-08-27T11:13:32.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde5b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fde5c0ce2e0 0x7fde5c06c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40744/0 (socket says 172.21.15.45:40744) 2023-08-27T11:13:32.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde5b7fe700 1 -- 172.21.15.45:0/1036060144 learned_addr learned my addr 172.21.15.45:0/1036060144 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:32.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde5affd700 1 --2- 172.21.15.45:0/1036060144 >> v2:172.21.15.135:3300/0 conn(0x7fde5c15d2c0 0x7fde5c0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde5a7fc700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c0b8c20 0x7fde5c0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981926812 0 0) 0x7fde54001490 con 0x7fde5c0ce2e0 2023-08-27T11:13:32.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde440035b0 con 0x7fde5c0ce2e0 2023-08-27T11:13:32.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2166505455 0 0) 0x7fde440035b0 con 0x7fde5c0ce2e0 2023-08-27T11:13:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c0b8c20 msgr2=0x7fde5c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde59ffb700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c0b8c20 0x7fde5c0cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 >> v2:172.21.15.135:3300/0 conn(0x7fde5c15d2c0 msgr2=0x7fde5c0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde59ffb700 1 --2- 172.21.15.45:0/1036060144 >> v2:172.21.15.135:3300/0 conn(0x7fde5c15d2c0 0x7fde5c0d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0d1bf0 con 0x7fde5c0ce2e0 2023-08-27T11:13:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.619+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fde54001b80 con 0x7fde5c0ce2e0 2023-08-27T11:13:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 >> v1:172.21.15.45:6789/0 conn(0x7fde5c0ce2e0 legacy=0x7fde5c06c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde44003a00 0x7fde44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde44004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde44005210 0x7fde4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0ce770 con 0x7fde44004850 2023-08-27T11:13:32.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0d2c90 con 0x7fde44003a00 2023-08-27T11:13:32.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4400bf50 con 0x7fde44005210 2023-08-27T11:13:32.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5a7fc700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c0b8c20 0x7fde5c0cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:32.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5affd700 1 --2- 172.21.15.45:0/1036060144 >> v2:172.21.15.135:3300/0 conn(0x7fde5c15d2c0 0x7fde5c0d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:32.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5a7fc700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde44004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5b7fe700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde44005210 0x7fde4400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5affd700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde44003a00 0x7fde44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5a7fc700 1 -- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde44005210 msgr2=0x7fde4400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:32.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5a7fc700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde44005210 0x7fde4400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5a7fc700 1 -- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde44003a00 msgr2=0x7fde44003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:32.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5a7fc700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde44003a00 0x7fde44003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5a7fc700 1 -- 172.21.15.45:0/1036060144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde440035b0 con 0x7fde44004850 2023-08-27T11:13:32.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde5a7fc700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde44004cd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009bd0 tx=0x7fde44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8999d667ff318ac0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:32.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c004030 con 0x7fde44004850 2023-08-27T11:13:32.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c004190 con 0x7fde44004850 2023-08-27T11:13:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.620+0000 7fde59ffb700 1 -- 172.21.15.45:0/1036060144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c004540 con 0x7fde44004850 2023-08-27T11:13:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.621+0000 7fde6164d700 1 -- 172.21.15.45:0/1036060144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 msgr2=0x7fde44004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.621+0000 7fde6164d700 1 --2- 172.21.15.45:0/1036060144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde44004cd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009bd0 tx=0x7fde44005880 comp rx=0 tx=0).stop 2023-08-27T11:13:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.621+0000 7fde5b7fe700 1 -- 172.21.15.45:0/1036060144 reap_dead start 2023-08-27T11:13:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.621+0000 7fde6164d700 1 -- 172.21.15.45:0/1036060144 shutdown_connections 2023-08-27T11:13:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.621+0000 7fde6164d700 1 -- 172.21.15.45:0/1036060144 >> 172.21.15.45:0/1036060144 conn(0x7fde5c0d2fd0 msgr2=0x7fde5c0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.621+0000 7fde6164d700 1 -- 172.21.15.45:0/1036060144 shutdown_connections 2023-08-27T11:13:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.621+0000 7fde6164d700 1 -- 172.21.15.45:0/1036060144 wait complete. 2023-08-27T11:13:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.622+0000 7fde6164d700 1 Processor -- start 2023-08-27T11:13:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.622+0000 7fde6164d700 1 -- start start 2023-08-27T11:13:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde6164d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde5c0c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde6164d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c150140 0x7fde5c0c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde6164d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde5c0c9360 0x7fde5c15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde6164d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0c97e0 con 0x7fde44004850 2023-08-27T11:13:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde6164d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0c7640 con 0x7fde5c0c9360 2023-08-27T11:13:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde6164d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c160610 con 0x7fde5c150140 2023-08-27T11:13:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde5affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde5c0c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde5affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde5c0c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53976/0 (socket says 172.21.15.45:53976) 2023-08-27T11:13:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde5affd700 1 -- 172.21.15.45:0/879055001 learned_addr learned my addr 172.21.15.45:0/879055001 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde5a7fc700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c150140 0x7fde5c0c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.623+0000 7fde5b7fe700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde5c0c9360 0x7fde5c15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde5affd700 1 -- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c150140 msgr2=0x7fde5c0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde5affd700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c150140 0x7fde5c0c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde5affd700 1 -- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde5c0c9360 msgr2=0x7fde5c15f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde5affd700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde5c0c9360 0x7fde5c15f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde5affd700 1 -- 172.21.15.45:0/879055001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0c7870 con 0x7fde44004850 2023-08-27T11:13:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde5affd700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde5c0c85f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fde50011d00 tx=0x7fde5000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde437fe700 1 -- 172.21.15.45:0/879055001 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5000fb30 con 0x7fde44004850 2023-08-27T11:13:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde437fe700 1 -- 172.21.15.45:0/879055001 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5000fc90 con 0x7fde44004850 2023-08-27T11:13:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c0c7b10 con 0x7fde44004850 2023-08-27T11:13:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde437fe700 1 -- 172.21.15.45:0/879055001 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50004030 con 0x7fde44004850 2023-08-27T11:13:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.624+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c0c80d0 con 0x7fde44004850 2023-08-27T11:13:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.625+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5c0c82c0 con 0x7fde44004850 2023-08-27T11:13:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.625+0000 7fde437fe700 1 -- 172.21.15.45:0/879055001 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fde50004190 con 0x7fde44004850 2023-08-27T11:13:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.626+0000 7fde437fe700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fde4806a110 0x7fde4806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.626+0000 7fde437fe700 1 -- 172.21.15.45:0/879055001 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fde50088b20 con 0x7fde44004850 2023-08-27T11:13:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.626+0000 7fde5a7fc700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fde4806a110 0x7fde4806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:32.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.626+0000 7fde5a7fc700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fde4806a110 0x7fde4806c5d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fde4c016040 tx=0x7fde4c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.630+0000 7fde437fe700 1 -- 172.21.15.45:0/879055001 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fde50057e20 con 0x7fde44004850 2023-08-27T11:13:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.920+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fde5c04e620 con 0x7fde44004850 2023-08-27T11:13:32.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.923+0000 7fde437fe700 1 -- 172.21.15.45:0/879055001 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fde50022030 con 0x7fde44004850 2023-08-27T11:13:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fde4806a110 msgr2=0x7fde4806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fde4806a110 0x7fde4806c5d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fde4c016040 tx=0x7fde4c008070 comp rx=0 tx=0).stop 2023-08-27T11:13:32.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 msgr2=0x7fde5c0c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde5c0c85f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fde50011d00 tx=0x7fde5000f520 comp rx=0 tx=0).stop 2023-08-27T11:13:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 shutdown_connections 2023-08-27T11:13:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde44004850 0x7fde5c0c85f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fde4806a110 0x7fde4806c5d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fde5c0c9360 0x7fde5c15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 --2- 172.21.15.45:0/879055001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fde5c150140 0x7fde5c0c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 >> 172.21.15.45:0/879055001 conn(0x7fde5c0d2fd0 msgr2=0x7fde5c0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 shutdown_connections 2023-08-27T11:13:32.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:32.925+0000 7fde6164d700 1 -- 172.21.15.45:0/879055001 wait complete. 2023-08-27T11:13:32.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-08-27T11:13:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.063+0000 7f377c3cb700 1 Processor -- start 2023-08-27T11:13:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.063+0000 7f377c3cb700 1 -- start start 2023-08-27T11:13:33.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.064+0000 7f377c3cb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f377406d5e0 0x7f37740b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.064+0000 7f377c3cb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377415b460 con 0x7f37740b17c0 2023-08-27T11:13:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.064+0000 7f377c3cb700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37740b2280 con 0x7f3774157be0 2023-08-27T11:13:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.064+0000 7f377c3cb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3774157fc0 con 0x7f377406d5e0 2023-08-27T11:13:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.064+0000 7f3779966700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f37740b17c0 0x7f377406ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40760/0 (socket says 172.21.15.45:40760) 2023-08-27T11:13:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.064+0000 7f3779966700 1 -- 172.21.15.45:0/2808478475 learned_addr learned my addr 172.21.15.45:0/2808478475 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:33.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.064+0000 7f377a968700 1 --2- 172.21.15.45:0/2808478475 >> v2:172.21.15.45:3300/0 conn(0x7f377406d5e0 0x7f37740b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.064+0000 7f3779165700 1 -- 172.21.15.45:0/2808478475 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735291918 0 0) 0x7f377415b460 con 0x7f37740b17c0 2023-08-27T11:13:33.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f3779165700 1 -- 172.21.15.45:0/2808478475 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37600035b0 con 0x7f37740b17c0 2023-08-27T11:13:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f377a968700 1 -- 172.21.15.45:0/2808478475 >> v1:172.21.15.135:6789/0 conn(0x7f3774157be0 legacy=0x7f37740716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f377a968700 1 -- 172.21.15.45:0/2808478475 >> v1:172.21.15.45:6789/0 conn(0x7f37740b17c0 legacy=0x7f377406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f377a968700 1 -- 172.21.15.45:0/2808478475 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37740b2560 con 0x7f377406d5e0 2023-08-27T11:13:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f377a968700 1 --2- 172.21.15.45:0/2808478475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 0x7f37740b1c70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f376c009d90 tx=0x7f376c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0af7bf0bf289708 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f3779165700 1 -- 172.21.15.45:0/2808478475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c00e070 con 0x7f377406d5e0 2023-08-27T11:13:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f3779165700 1 -- 172.21.15.45:0/2808478475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c00bae0 con 0x7f377406d5e0 2023-08-27T11:13:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f3779165700 1 -- 172.21.15.45:0/2808478475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c00be50 con 0x7f377406d5e0 2023-08-27T11:13:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f377c3cb700 1 -- 172.21.15.45:0/2808478475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 msgr2=0x7f37740b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.065+0000 7f377c3cb700 1 --2- 172.21.15.45:0/2808478475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 0x7f37740b1c70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f376c009d90 tx=0x7f376c00b330 comp rx=0 tx=0).stop 2023-08-27T11:13:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.066+0000 7f377c3cb700 1 -- 172.21.15.45:0/2808478475 shutdown_connections 2023-08-27T11:13:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.066+0000 7f377c3cb700 1 --2- 172.21.15.45:0/2808478475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 0x7f37740b1c70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.066+0000 7f377c3cb700 1 -- 172.21.15.45:0/2808478475 >> 172.21.15.45:0/2808478475 conn(0x7f37740cc7c0 msgr2=0x7f37740ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.066+0000 7f377c3cb700 1 -- 172.21.15.45:0/2808478475 shutdown_connections 2023-08-27T11:13:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.066+0000 7f377c3cb700 1 -- 172.21.15.45:0/2808478475 wait complete. 2023-08-27T11:13:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.067+0000 7f377c3cb700 1 Processor -- start 2023-08-27T11:13:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.067+0000 7f377c3cb700 1 -- start start 2023-08-27T11:13:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377c3cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 0x7f37740c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377c3cb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37740b17c0 0x7f37740b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377c3cb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3774157be0 0x7f37740b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377c3cb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37740b44e0 con 0x7f377406d5e0 2023-08-27T11:13:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377c3cb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37740c32e0 con 0x7f3774157be0 2023-08-27T11:13:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377c3cb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37740c8130 con 0x7f37740b17c0 2023-08-27T11:13:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a167700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 0x7f37740c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a167700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 0x7f37740c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53990/0 (socket says 172.21.15.45:53990) 2023-08-27T11:13:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a167700 1 -- 172.21.15.45:0/3583145257 learned_addr learned my addr 172.21.15.45:0/3583145257 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a968700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3774157be0 0x7f37740b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f3779966700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37740b17c0 0x7f37740b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a167700 1 -- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37740b17c0 msgr2=0x7f37740b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a167700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37740b17c0 0x7f37740b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a167700 1 -- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3774157be0 msgr2=0x7f37740b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a167700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3774157be0 0x7f37740b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a167700 1 -- 172.21.15.45:0/3583145257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c009a70 con 0x7f377406d5e0 2023-08-27T11:13:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a968700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3774157be0 0x7f37740b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f3779966700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37740b17c0 0x7f37740b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377a167700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 0x7f37740c6310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f376400c500 tx=0x7f376400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f376affd700 1 -- 172.21.15.45:0/3583145257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764011790 con 0x7f377406d5e0 2023-08-27T11:13:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f376affd700 1 -- 172.21.15.45:0/3583145257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37640118f0 con 0x7f377406d5e0 2023-08-27T11:13:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f376affd700 1 -- 172.21.15.45:0/3583145257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764011ca0 con 0x7f377406d5e0 2023-08-27T11:13:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37740c7890 con 0x7f377406d5e0 2023-08-27T11:13:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.068+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37740c7e50 con 0x7f377406d5e0 2023-08-27T11:13:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.069+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3774072690 con 0x7f377406d5e0 2023-08-27T11:13:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.070+0000 7f376affd700 1 -- 172.21.15.45:0/3583145257 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f376401c460 con 0x7f377406d5e0 2023-08-27T11:13:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.070+0000 7f376affd700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f375806a0e0 0x7f375806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.070+0000 7f376affd700 1 -- 172.21.15.45:0/3583145257 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3764088190 con 0x7f377406d5e0 2023-08-27T11:13:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.070+0000 7f3779966700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f375806a0e0 0x7f375806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.071+0000 7f3779966700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f375806a0e0 0x7f375806c5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f37740c5fe0 tx=0x7f3770009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.075+0000 7f376affd700 1 -- 172.21.15.45:0/3583145257 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3764053a90 con 0x7f377406d5e0 2023-08-27T11:13:33.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.356+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f37740734a0 con 0x7f377406d5e0 2023-08-27T11:13:33.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.359+0000 7f376affd700 1 -- 172.21.15.45:0/3583145257 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f37640570b0 con 0x7f377406d5e0 2023-08-27T11:13:33.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f375806a0e0 msgr2=0x7f375806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f375806a0e0 0x7f375806c5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f37740c5fe0 tx=0x7f3770009040 comp rx=0 tx=0).stop 2023-08-27T11:13:33.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 msgr2=0x7f37740c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 0x7f37740c6310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f376400c500 tx=0x7f376400c8c0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 shutdown_connections 2023-08-27T11:13:33.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377406d5e0 0x7f37740c6310 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f375806a0e0 0x7f375806c5a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3774157be0 0x7f37740b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 --2- 172.21.15.45:0/3583145257 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37740b17c0 0x7f37740b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.361+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 >> 172.21.15.45:0/3583145257 conn(0x7f37740cc7c0 msgr2=0x7f37740ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:33.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.362+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 shutdown_connections 2023-08-27T11:13:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.362+0000 7f377c3cb700 1 -- 172.21.15.45:0/3583145257 wait complete. 2023-08-27T11:13:33.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-08-27T11:13:33.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.500+0000 7fb9e5a0b700 1 Processor -- start 2023-08-27T11:13:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.500+0000 7fb9e5a0b700 1 -- start start 2023-08-27T11:13:33.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9e5a0b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb9e00ca420 0x7fb9e0071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9e5a0b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00cd9a0 con 0x7fb9e015ed10 2023-08-27T11:13:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9e5a0b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00cdc60 con 0x7fb9e0144a60 2023-08-27T11:13:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9e5a0b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e00cdf40 con 0x7fb9e00ca420 2023-08-27T11:13:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9df7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb9e015ed10 0x7fb9e015f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40778/0 (socket says 172.21.15.45:40778) 2023-08-27T11:13:33.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9de7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb9e00ca420 0x7fb9e0071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9df7fe700 1 -- 172.21.15.45:0/4090992384 learned_addr learned my addr 172.21.15.45:0/4090992384 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9de7fc700 1 -- 172.21.15.45:0/4090992384 >> v1:172.21.15.135:6789/0 conn(0x7fb9e0144a60 legacy=0x7fb9e00cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9de7fc700 1 -- 172.21.15.45:0/4090992384 >> v1:172.21.15.45:6789/0 conn(0x7fb9e015ed10 legacy=0x7fb9e015f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9ddffb700 1 -- 172.21.15.45:0/4090992384 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828473060 0 0) 0x7fb9e00cd9a0 con 0x7fb9e015ed10 2023-08-27T11:13:33.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9de7fc700 1 -- 172.21.15.45:0/4090992384 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e006ce80 con 0x7fb9e00ca420 2023-08-27T11:13:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9de7fc700 1 --2- 172.21.15.45:0/4090992384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e00ca420 0x7fb9e0071920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009840 tx=0x7fb9d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b91527355334dd1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.501+0000 7fb9ddffb700 1 -- 172.21.15.45:0/4090992384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4011830 con 0x7fb9e00ca420 2023-08-27T11:13:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.502+0000 7fb9ddffb700 1 -- 172.21.15.45:0/4090992384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4011990 con 0x7fb9e00ca420 2023-08-27T11:13:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.502+0000 7fb9ddffb700 1 -- 172.21.15.45:0/4090992384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4011d20 con 0x7fb9e00ca420 2023-08-27T11:13:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.502+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/4090992384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e00ca420 msgr2=0x7fb9e0071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.502+0000 7fb9e5a0b700 1 --2- 172.21.15.45:0/4090992384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e00ca420 0x7fb9e0071920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009840 tx=0x7fb9d4009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.502+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/4090992384 shutdown_connections 2023-08-27T11:13:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.502+0000 7fb9e5a0b700 1 --2- 172.21.15.45:0/4090992384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e00ca420 0x7fb9e0071920 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.502+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/4090992384 >> 172.21.15.45:0/4090992384 conn(0x7fb9e00cfdd0 msgr2=0x7fb9e00d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:33.566 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.502+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/4090992384 shutdown_connections 2023-08-27T11:13:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.502+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/4090992384 wait complete. 2023-08-27T11:13:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.504+0000 7fb9e5a0b700 1 Processor -- start 2023-08-27T11:13:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.504+0000 7fb9e5a0b700 1 -- start start 2023-08-27T11:13:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9e5a0b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9e00ca420 0x7fb9e00c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9e5a0b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9e0144a60 0x7fb9e015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9e5a0b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e015ed10 0x7fb9e015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9e5a0b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e015c4d0 con 0x7fb9e015ed10 2023-08-27T11:13:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9e5a0b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e01f4970 con 0x7fb9e00ca420 2023-08-27T11:13:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9e5a0b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e015e410 con 0x7fb9e0144a60 2023-08-27T11:13:33.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9df7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e015ed10 0x7fb9e015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9df7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e015ed10 0x7fb9e015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54008/0 (socket says 172.21.15.45:54008) 2023-08-27T11:13:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9df7fe700 1 -- 172.21.15.45:0/2454425910 learned_addr learned my addr 172.21.15.45:0/2454425910 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9deffd700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9e00ca420 0x7fb9e00c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9de7fc700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9e0144a60 0x7fb9e015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9df7fe700 1 -- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9e0144a60 msgr2=0x7fb9e015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9df7fe700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9e0144a60 0x7fb9e015d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9df7fe700 1 -- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9e00ca420 msgr2=0x7fb9e00c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9df7fe700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9e00ca420 0x7fb9e00c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9df7fe700 1 -- 172.21.15.45:0/2454425910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d40094a0 con 0x7fb9e015ed10 2023-08-27T11:13:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.505+0000 7fb9de7fc700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9e0144a60 0x7fb9e015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:33.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.506+0000 7fb9df7fe700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e015ed10 0x7fb9e015de00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800cba0 tx=0x7fb9d800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:33.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.506+0000 7fb9c7fff700 1 -- 172.21.15.45:0/2454425910 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800aad0 con 0x7fb9e015ed10 2023-08-27T11:13:33.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.506+0000 7fb9c7fff700 1 -- 172.21.15.45:0/2454425910 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d800ac50 con 0x7fb9e015ed10 2023-08-27T11:13:33.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.506+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e01f4320 con 0x7fb9e015ed10 2023-08-27T11:13:33.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.506+0000 7fb9c7fff700 1 -- 172.21.15.45:0/2454425910 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80070d0 con 0x7fb9e015ed10 2023-08-27T11:13:33.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.506+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e015b5c0 con 0x7fb9e015ed10 2023-08-27T11:13:33.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.507+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e01f4180 con 0x7fb9e015ed10 2023-08-27T11:13:33.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.507+0000 7fb9c7fff700 1 -- 172.21.15.45:0/2454425910 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb9d8007230 con 0x7fb9e015ed10 2023-08-27T11:13:33.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.507+0000 7fb9c7fff700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb9cc06a0e0 0x7fb9cc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.508+0000 7fb9c7fff700 1 -- 172.21.15.45:0/2454425910 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb9d8088100 con 0x7fb9e015ed10 2023-08-27T11:13:33.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.508+0000 7fb9deffd700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb9cc06a0e0 0x7fb9cc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.508+0000 7fb9deffd700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb9cc06a0e0 0x7fb9cc06c5a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0007fd0 tx=0x7fb9d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:33.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.512+0000 7fb9c7fff700 1 -- 172.21.15.45:0/2454425910 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb9e01f4180 con 0x7fb9e015ed10 2023-08-27T11:13:33.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:13:33.583 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:13:33.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.795+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fb9e01f4180 con 0x7fb9e015ed10 2023-08-27T11:13:33.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.798+0000 7fb9c7fff700 1 -- 172.21.15.45:0/2454425910 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb9e01f4180 con 0x7fb9e015ed10 2023-08-27T11:13:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb9cc06a0e0 msgr2=0x7fb9cc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb9cc06a0e0 0x7fb9cc06c5a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0007fd0 tx=0x7fb9d0009040 comp rx=0 tx=0).stop 2023-08-27T11:13:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e015ed10 msgr2=0x7fb9e015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e015ed10 0x7fb9e015de00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800cba0 tx=0x7fb9d800ceb0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 shutdown_connections 2023-08-27T11:13:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9e015ed10 0x7fb9e015de00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb9cc06a0e0 0x7fb9cc06c5a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9e00ca420 0x7fb9e00c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 --2- 172.21.15.45:0/2454425910 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9e0144a60 0x7fb9e015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 >> 172.21.15.45:0/2454425910 conn(0x7fb9e00cfdd0 msgr2=0x7fb9e00af830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 shutdown_connections 2023-08-27T11:13:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.801+0000 7fb9e5a0b700 1 -- 172.21.15.45:0/2454425910 wait complete. 2023-08-27T11:13:33.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.938+0000 7fb59795c700 1 Processor -- start 2023-08-27T11:13:33.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.938+0000 7fb59795c700 1 -- start start 2023-08-27T11:13:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb59795c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb59015b070 0x7fb5900cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb59795c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb5900cdf60 0x7fb59006e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb59795c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb59006cec0 0x7fb59006e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb59795c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59015d8e0 con 0x7fb59006cec0 2023-08-27T11:13:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb59795c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59015dad0 con 0x7fb59015b070 2023-08-27T11:13:33.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb59795c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59015ddb0 con 0x7fb5900cdf60 2023-08-27T11:13:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb595ef9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb59006cec0 0x7fb59006e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb595ef9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb59006cec0 0x7fb59006e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54016/0 (socket says 172.21.15.45:54016) 2023-08-27T11:13:33.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb595ef9700 1 -- 172.21.15.45:0/731090007 learned_addr learned my addr 172.21.15.45:0/731090007 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:33.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb5956f8700 1 --2- 172.21.15.45:0/731090007 >> v2:172.21.15.135:3300/0 conn(0x7fb59015b070 0x7fb5900cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb594ef7700 1 --2- 172.21.15.45:0/731090007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb5900cdf60 0x7fb59006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb595ef9700 1 -- 172.21.15.45:0/731090007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb5900cdf60 msgr2=0x7fb59006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb595ef9700 1 --2- 172.21.15.45:0/731090007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb5900cdf60 0x7fb59006e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb595ef9700 1 -- 172.21.15.45:0/731090007 >> v2:172.21.15.135:3300/0 conn(0x7fb59015b070 msgr2=0x7fb5900cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:33.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb595ef9700 1 --2- 172.21.15.45:0/731090007 >> v2:172.21.15.135:3300/0 conn(0x7fb59015b070 0x7fb5900cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb595ef9700 1 -- 172.21.15.45:0/731090007 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59015e090 con 0x7fb59006cec0 2023-08-27T11:13:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.939+0000 7fb594ef7700 1 --2- 172.21.15.45:0/731090007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb5900cdf60 0x7fb59006e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:33.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb595ef9700 1 --2- 172.21.15.45:0/731090007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59006cec0 0x7fb59006e7b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb59006ef10 tx=0x7fb58800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e1dd9f11b5c974 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb587fff700 1 -- 172.21.15.45:0/731090007 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800e070 con 0x7fb59006cec0 2023-08-27T11:13:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb587fff700 1 -- 172.21.15.45:0/731090007 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58800b9c0 con 0x7fb59006cec0 2023-08-27T11:13:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb587fff700 1 -- 172.21.15.45:0/731090007 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58800bd70 con 0x7fb59006cec0 2023-08-27T11:13:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb59795c700 1 -- 172.21.15.45:0/731090007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59006cec0 msgr2=0x7fb59006e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb59795c700 1 --2- 172.21.15.45:0/731090007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59006cec0 0x7fb59006e7b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb59006ef10 tx=0x7fb58800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb59795c700 1 -- 172.21.15.45:0/731090007 shutdown_connections 2023-08-27T11:13:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb59795c700 1 --2- 172.21.15.45:0/731090007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59006cec0 0x7fb59006e7b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb59795c700 1 --2- 172.21.15.45:0/731090007 >> v2:172.21.15.135:3300/0 conn(0x7fb59015b070 0x7fb5900cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb59795c700 1 --2- 172.21.15.45:0/731090007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb5900cdf60 0x7fb59006e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb59795c700 1 -- 172.21.15.45:0/731090007 >> 172.21.15.45:0/731090007 conn(0x7fb5900cfe50 msgr2=0x7fb5900d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb59795c700 1 -- 172.21.15.45:0/731090007 shutdown_connections 2023-08-27T11:13:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.940+0000 7fb59795c700 1 -- 172.21.15.45:0/731090007 wait complete. 2023-08-27T11:13:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.942+0000 7fb59795c700 1 Processor -- start 2023-08-27T11:13:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.942+0000 7fb59795c700 1 -- start start 2023-08-27T11:13:33.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.942+0000 7fb59795c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb59006cec0 0x7fb59014c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.942+0000 7fb59795c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5900cdf60 0x7fb5901f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.942+0000 7fb59795c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59015b070 0x7fb5900b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.942+0000 7fb59795c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5901608b0 con 0x7fb59015b070 2023-08-27T11:13:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.942+0000 7fb59795c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59015fca0 con 0x7fb5900cdf60 2023-08-27T11:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.942+0000 7fb59795c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59015ff80 con 0x7fb59006cec0 2023-08-27T11:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb595ef9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59015b070 0x7fb5900b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb595ef9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59015b070 0x7fb5900b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54022/0 (socket says 172.21.15.45:54022) 2023-08-27T11:13:33.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb595ef9700 1 -- 172.21.15.45:0/3252959893 learned_addr learned my addr 172.21.15.45:0/3252959893 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:33.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb5956f8700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb59006cec0 0x7fb59014c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb594ef7700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5900cdf60 0x7fb5901f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb595ef9700 1 -- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb59006cec0 msgr2=0x7fb59014c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb595ef9700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb59006cec0 0x7fb59014c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb595ef9700 1 -- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5900cdf60 msgr2=0x7fb5901f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb595ef9700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5900cdf60 0x7fb5901f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb595ef9700 1 -- 172.21.15.45:0/3252959893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb588009b30 con 0x7fb59015b070 2023-08-27T11:13:33.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb5956f8700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb59006cec0 0x7fb59014c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:33.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb594ef7700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5900cdf60 0x7fb5901f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:33.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb595ef9700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59015b070 0x7fb5900b54f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb59006c730 tx=0x7fb588004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:33.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb585ffb700 1 -- 172.21.15.45:0/3252959893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588004250 con 0x7fb59015b070 2023-08-27T11:13:33.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb585ffb700 1 -- 172.21.15.45:0/3252959893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5880043d0 con 0x7fb59015b070 2023-08-27T11:13:33.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb590160260 con 0x7fb59015b070 2023-08-27T11:13:33.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb585ffb700 1 -- 172.21.15.45:0/3252959893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588004780 con 0x7fb59015b070 2023-08-27T11:13:33.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.943+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5901f35c0 con 0x7fb59015b070 2023-08-27T11:13:33.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.944+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb590072810 con 0x7fb59015b070 2023-08-27T11:13:33.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.945+0000 7fb585ffb700 1 -- 172.21.15.45:0/3252959893 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb58801c440 con 0x7fb59015b070 2023-08-27T11:13:33.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.945+0000 7fb585ffb700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb57406a0e0 0x7fb57406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:33.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.945+0000 7fb585ffb700 1 -- 172.21.15.45:0/3252959893 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb588089890 con 0x7fb59015b070 2023-08-27T11:13:33.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.945+0000 7fb5956f8700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb57406a0e0 0x7fb57406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:33.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.946+0000 7fb5956f8700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb57406a0e0 0x7fb57406c5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb5800079d0 tx=0x7fb580008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:33.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:33.949+0000 7fb585ffb700 1 -- 172.21.15.45:0/3252959893 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb588055220 con 0x7fb59015b070 2023-08-27T11:13:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.150+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5901609f0 con 0x7fb59015b070 2023-08-27T11:13:34.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.151+0000 7fb585ffb700 1 -- 172.21.15.45:0/3252959893 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+14856 (secure 0 0 0) 0x7fb588020090 con 0x7fb59015b070 2023-08-27T11:13:34.152 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:34.152 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":38,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:13:30.628495+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb57406a0e0 msgr2=0x7fb57406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb57406a0e0 0x7fb57406c5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb5800079d0 tx=0x7fb580008040 comp rx=0 tx=0).stop 2023-08-27T11:13:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59015b070 msgr2=0x7fb5900b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59015b070 0x7fb5900b54f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb59006c730 tx=0x7fb588004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 shutdown_connections 2023-08-27T11:13:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59015b070 0x7fb5900b54f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb57406a0e0 0x7fb57406c5a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb59006cec0 0x7fb59014c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 --2- 172.21.15.45:0/3252959893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5900cdf60 0x7fb5901f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 >> 172.21.15.45:0/3252959893 conn(0x7fb5900cfe50 msgr2=0x7fb59015a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 shutdown_connections 2023-08-27T11:13:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.153+0000 7fb59795c700 1 -- 172.21.15.45:0/3252959893 wait complete. 2023-08-27T11:13:34.166 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:13:34.167 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:13:34.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.294+0000 7fd2389e2700 1 Processor -- start 2023-08-27T11:13:34.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.294+0000 7fd2389e2700 1 -- start start 2023-08-27T11:13:34.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.295+0000 7fd2389e2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd23415d340 0x7fd234160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.295+0000 7fd2389e2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2340cd880 0x7fd23406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.295+0000 7fd2389e2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23406b160 con 0x7fd2340cdd00 2023-08-27T11:13:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.295+0000 7fd2389e2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd234072010 con 0x7fd2340cd880 2023-08-27T11:13:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.295+0000 7fd2389e2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2340af590 con 0x7fd23415d340 2023-08-27T11:13:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.295+0000 7fd231d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2340cd880 0x7fd23406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.295+0000 7fd232d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd2340cdd00 0x7fd2340b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40804/0 (socket says 172.21.15.45:40804) 2023-08-27T11:13:34.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.295+0000 7fd232d9d700 1 -- 172.21.15.45:0/1061007328 learned_addr learned my addr 172.21.15.45:0/1061007328 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:34.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd23259c700 1 --2- 172.21.15.45:0/1061007328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd23415d340 0x7fd234160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd231d9b700 1 -- 172.21.15.45:0/1061007328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd23415d340 msgr2=0x7fd234160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd231d9b700 1 --2- 172.21.15.45:0/1061007328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd23415d340 0x7fd234160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd231d9b700 1 -- 172.21.15.45:0/1061007328 >> v1:172.21.15.45:6789/0 conn(0x7fd2340cdd00 legacy=0x7fd2340b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd231d9b700 1 -- 172.21.15.45:0/1061007328 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2340af2b0 con 0x7fd2340cd880 2023-08-27T11:13:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd23159a700 1 -- 172.21.15.45:0/1061007328 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205366004 0 0) 0x7fd23406b160 con 0x7fd2340cdd00 2023-08-27T11:13:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd231d9b700 1 --2- 172.21.15.45:0/1061007328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340cd880 0x7fd23406a280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd224009840 tx=0x7fd224009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40d86a437d2830e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd23159a700 1 -- 172.21.15.45:0/1061007328 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224011830 con 0x7fd2340cd880 2023-08-27T11:13:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd23159a700 1 -- 172.21.15.45:0/1061007328 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd224011990 con 0x7fd2340cd880 2023-08-27T11:13:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.296+0000 7fd23159a700 1 -- 172.21.15.45:0/1061007328 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224011d20 con 0x7fd2340cd880 2023-08-27T11:13:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.297+0000 7fd2389e2700 1 -- 172.21.15.45:0/1061007328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340cd880 msgr2=0x7fd23406a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.297+0000 7fd2389e2700 1 --2- 172.21.15.45:0/1061007328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340cd880 0x7fd23406a280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd224009840 tx=0x7fd224009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.297+0000 7fd2389e2700 1 -- 172.21.15.45:0/1061007328 shutdown_connections 2023-08-27T11:13:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.297+0000 7fd2389e2700 1 --2- 172.21.15.45:0/1061007328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd23415d340 0x7fd234160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.297+0000 7fd2389e2700 1 --2- 172.21.15.45:0/1061007328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2340cd880 0x7fd23406a280 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.297+0000 7fd2389e2700 1 -- 172.21.15.45:0/1061007328 >> 172.21.15.45:0/1061007328 conn(0x7fd2340cc7c0 msgr2=0x7fd2340ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.297+0000 7fd2389e2700 1 -- 172.21.15.45:0/1061007328 shutdown_connections 2023-08-27T11:13:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.297+0000 7fd2389e2700 1 -- 172.21.15.45:0/1061007328 wait complete. 2023-08-27T11:13:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.298+0000 7fd2389e2700 1 Processor -- start 2023-08-27T11:13:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.298+0000 7fd2389e2700 1 -- start start 2023-08-27T11:13:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd2389e2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2340cd880 0x7fd234147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd2389e2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2340cdd00 0x7fd23406a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd2389e2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd23415d340 0x7fd23406a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd2389e2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23415f890 con 0x7fd23415d340 2023-08-27T11:13:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd2389e2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23415ec80 con 0x7fd2340cdd00 2023-08-27T11:13:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd2389e2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23415ef60 con 0x7fd2340cd880 2023-08-27T11:13:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd232d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd23415d340 0x7fd23406a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd232d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd23415d340 0x7fd23406a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54036/0 (socket says 172.21.15.45:54036) 2023-08-27T11:13:34.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd232d9d700 1 -- 172.21.15.45:0/1412862421 learned_addr learned my addr 172.21.15.45:0/1412862421 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:34.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd23259c700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2340cd880 0x7fd234147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd231d9b700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2340cdd00 0x7fd23406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd232d9d700 1 -- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2340cd880 msgr2=0x7fd234147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd232d9d700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2340cd880 0x7fd234147270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd232d9d700 1 -- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2340cdd00 msgr2=0x7fd23406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd232d9d700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2340cdd00 0x7fd23406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd232d9d700 1 -- 172.21.15.45:0/1412862421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240094a0 con 0x7fd23415d340 2023-08-27T11:13:34.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.299+0000 7fd232d9d700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd23415d340 0x7fd23406a7c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00cc10 tx=0x7fd22c0021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:34.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.300+0000 7fd222ffd700 1 -- 172.21.15.45:0/1412862421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c00ab80 con 0x7fd23415d340 2023-08-27T11:13:34.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.300+0000 7fd222ffd700 1 -- 172.21.15.45:0/1412862421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c00ad00 con 0x7fd23415d340 2023-08-27T11:13:34.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.300+0000 7fd222ffd700 1 -- 172.21.15.45:0/1412862421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c007310 con 0x7fd23415d340 2023-08-27T11:13:34.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.300+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23415f240 con 0x7fd23415d340 2023-08-27T11:13:34.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.300+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23415fd20 con 0x7fd23415d340 2023-08-27T11:13:34.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.301+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd234072690 con 0x7fd23415d340 2023-08-27T11:13:34.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.301+0000 7fd222ffd700 1 -- 172.21.15.45:0/1412862421 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd22c007470 con 0x7fd23415d340 2023-08-27T11:13:34.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.301+0000 7fd222ffd700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd21406a0e0 0x7fd21406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.302+0000 7fd222ffd700 1 -- 172.21.15.45:0/1412862421 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd22c087e50 con 0x7fd23415d340 2023-08-27T11:13:34.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.302+0000 7fd23259c700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd21406a0e0 0x7fd21406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.302+0000 7fd23259c700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd21406a0e0 0x7fd21406c5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd228009930 tx=0x7fd228008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:34.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.306+0000 7fd222ffd700 1 -- 172.21.15.45:0/1412862421 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd22c053750 con 0x7fd23415d340 2023-08-27T11:13:34.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.506+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2340734a0 con 0x7fd23415d340 2023-08-27T11:13:34.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.507+0000 7fd222ffd700 1 -- 172.21.15.45:0/1412862421 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd22c0180a0 con 0x7fd23415d340 2023-08-27T11:13:34.508 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:13:34.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd21406a0e0 msgr2=0x7fd21406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd21406a0e0 0x7fd21406c5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd228009930 tx=0x7fd228008040 comp rx=0 tx=0).stop 2023-08-27T11:13:34.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd23415d340 msgr2=0x7fd23406a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd23415d340 0x7fd23406a7c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00cc10 tx=0x7fd22c0021c0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 shutdown_connections 2023-08-27T11:13:34.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd23415d340 0x7fd23406a7c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fd21406a0e0 0x7fd21406c5a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd2340cd880 0x7fd234147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 --2- 172.21.15.45:0/1412862421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2340cdd00 0x7fd23406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 >> 172.21.15.45:0/1412862421 conn(0x7fd2340cc7c0 msgr2=0x7fd23406f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:34.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 shutdown_connections 2023-08-27T11:13:34.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.509+0000 7fd2389e2700 1 -- 172.21.15.45:0/1412862421 wait complete. 2023-08-27T11:13:34.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:13:34.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.649+0000 7fcfc0bbf700 1 Processor -- start 2023-08-27T11:13:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.649+0000 7fcfc0bbf700 1 -- start start 2023-08-27T11:13:34.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.650+0000 7fcfc0bbf700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcfbc0b17c0 0x7fcfbc0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.650+0000 7fcfc0bbf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc15b460 con 0x7fcfbc157be0 2023-08-27T11:13:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.650+0000 7fcfc0bbf700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc0b2280 con 0x7fcfbc06ced0 2023-08-27T11:13:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.650+0000 7fcfc0bbf700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc157fc0 con 0x7fcfbc0b17c0 2023-08-27T11:13:34.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.650+0000 7fcfba59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcfbc157be0 0x7fcfbc0716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40816/0 (socket says 172.21.15.45:40816) 2023-08-27T11:13:34.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.650+0000 7fcfba59c700 1 -- 172.21.15.45:0/3883165148 learned_addr learned my addr 172.21.15.45:0/3883165148 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:34.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.650+0000 7fcfb9d9b700 1 --2- 172.21.15.45:0/3883165148 >> v2:172.21.15.135:3300/0 conn(0x7fcfbc0b17c0 0x7fcfbc0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.650+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899925652 0 0) 0x7fcfbc15b460 con 0x7fcfbc157be0 2023-08-27T11:13:34.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.650+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa40035b0 con 0x7fcfbc157be0 2023-08-27T11:13:34.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101715698 0 0) 0x7fcfa40035b0 con 0x7fcfbc157be0 2023-08-27T11:13:34.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 >> v2:172.21.15.135:3300/0 conn(0x7fcfbc0b17c0 msgr2=0x7fcfbc0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:34.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 --2- 172.21.15.45:0/3883165148 >> v2:172.21.15.135:3300/0 conn(0x7fcfbc0b17c0 0x7fcfbc0b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 >> v1:172.21.15.135:6789/0 conn(0x7fcfbc06ced0 legacy=0x7fcfbc0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0b2560 con 0x7fcfbc157be0 2023-08-27T11:13:34.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fcfb0001670 con 0x7fcfbc157be0 2023-08-27T11:13:34.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb9d9b700 1 --2- 172.21.15.45:0/3883165148 >> v2:172.21.15.135:3300/0 conn(0x7fcfbc0b17c0 0x7fcfbc0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:34.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 >> v1:172.21.15.45:6789/0 conn(0x7fcfbc157be0 legacy=0x7fcfbc0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfa4003a00 0x7fcfa4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa4004400 0x7fcfa40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfa400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc157fc0 con 0x7fcfa40050d0 2023-08-27T11:13:34.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa400c030 con 0x7fcfa4004400 2023-08-27T11:13:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa400c1f0 con 0x7fcfa4003a00 2023-08-27T11:13:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfbad9d700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfa400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb9d9b700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa4004400 0x7fcfa40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfba59c700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfa4003a00 0x7fcfa4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfbad9d700 1 -- 172.21.15.45:0/3883165148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfa4003a00 msgr2=0x7fcfa4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:34.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfbad9d700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfa4003a00 0x7fcfa4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfbad9d700 1 -- 172.21.15.45:0/3883165148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa4004400 msgr2=0x7fcfa40048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:34.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfbad9d700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfa4004400 0x7fcfa40048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfbad9d700 1 -- 172.21.15.45:0/3883165148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40035b0 con 0x7fcfa40050d0 2023-08-27T11:13:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfbad9d700 1 -- 172.21.15.45:0/3883165148 reap_dead start 2023-08-27T11:13:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfbad9d700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfa400ba10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c290 tx=0x7fcfa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24e2eef6ca1985a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.651+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4001e70 con 0x7fcfa40050d0 2023-08-27T11:13:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.652+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb400ce70 con 0x7fcfa40050d0 2023-08-27T11:13:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.652+0000 7fcfb959a700 1 -- 172.21.15.45:0/3883165148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4003c20 con 0x7fcfa40050d0 2023-08-27T11:13:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.652+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/3883165148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 msgr2=0x7fcfa400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.652+0000 7fcfc0bbf700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfa400ba10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c290 tx=0x7fcfa4005880 comp rx=0 tx=0).stop 2023-08-27T11:13:34.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.652+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/3883165148 shutdown_connections 2023-08-27T11:13:34.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.652+0000 7fcfc0bbf700 1 --2- 172.21.15.45:0/3883165148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfa400ba10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.652+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/3883165148 >> 172.21.15.45:0/3883165148 conn(0x7fcfbc0cc7c0 msgr2=0x7fcfbc0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:34.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.652+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/3883165148 shutdown_connections 2023-08-27T11:13:34.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.652+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/3883165148 wait complete. 2023-08-27T11:13:34.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfc0bbf700 1 Processor -- start 2023-08-27T11:13:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfc0bbf700 1 -- start start 2023-08-27T11:13:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfc0bbf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfbc1f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfc0bbf700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfbc0b4730 0x7fcfbc15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfc0bbf700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfbc0b2b90 0x7fcfbc159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfc0bbf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0acc00 con 0x7fcfa40050d0 2023-08-27T11:13:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfc0bbf700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc1f2920 con 0x7fcfbc0b2b90 2023-08-27T11:13:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfc0bbf700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0b4bb0 con 0x7fcfbc0b4730 2023-08-27T11:13:34.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfba59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfbc1f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfba59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfbc1f5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54052/0 (socket says 172.21.15.45:54052) 2023-08-27T11:13:34.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.654+0000 7fcfba59c700 1 -- 172.21.15.45:0/976541598 learned_addr learned my addr 172.21.15.45:0/976541598 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:34.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfb9d9b700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfbc0b4730 0x7fcfbc15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfbad9d700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfbc0b2b90 0x7fcfbc159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfba59c700 1 -- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfbc0b4730 msgr2=0x7fcfbc15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfba59c700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfbc0b4730 0x7fcfbc15ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfba59c700 1 -- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfbc0b2b90 msgr2=0x7fcfbc159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfba59c700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfbc0b2b90 0x7fcfbc159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfba59c700 1 -- 172.21.15.45:0/976541598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc1f63e0 con 0x7fcfa40050d0 2023-08-27T11:13:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfba59c700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfbc1f5e00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0007290 tx=0x7fcfb00075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfaaffd700 1 -- 172.21.15.45:0/976541598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0007780 con 0x7fcfa40050d0 2023-08-27T11:13:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfaaffd700 1 -- 172.21.15.45:0/976541598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb000f8d0 con 0x7fcfa40050d0 2023-08-27T11:13:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfaaffd700 1 -- 172.21.15.45:0/976541598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000fc60 con 0x7fcfa40050d0 2023-08-27T11:13:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc1f66c0 con 0x7fcfa40050d0 2023-08-27T11:13:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.655+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc1f6c80 con 0x7fcfa40050d0 2023-08-27T11:13:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.656+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc1f2af0 con 0x7fcfa40050d0 2023-08-27T11:13:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.657+0000 7fcfaaffd700 1 -- 172.21.15.45:0/976541598 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcfb001d460 con 0x7fcfa40050d0 2023-08-27T11:13:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.657+0000 7fcfaaffd700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf9c06a0e0 0x7fcf9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.657+0000 7fcfaaffd700 1 -- 172.21.15.45:0/976541598 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fcfb0088300 con 0x7fcfa40050d0 2023-08-27T11:13:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.657+0000 7fcfb9d9b700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf9c06a0e0 0x7fcf9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.658+0000 7fcfb9d9b700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf9c06a0e0 0x7fcf9c06c5a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc157790 tx=0x7fcfac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.661+0000 7fcfaaffd700 1 -- 172.21.15.45:0/976541598 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcfbc1f2af0 con 0x7fcfa40050d0 2023-08-27T11:13:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.860+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcfbc1f2af0 con 0x7fcfa40050d0 2023-08-27T11:13:34.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.861+0000 7fcfaaffd700 1 -- 172.21.15.45:0/976541598 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcfbc1f2af0 con 0x7fcfa40050d0 2023-08-27T11:13:34.862 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:13:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf9c06a0e0 msgr2=0x7fcf9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf9c06a0e0 0x7fcf9c06c5a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc157790 tx=0x7fcfac008040 comp rx=0 tx=0).stop 2023-08-27T11:13:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 msgr2=0x7fcfbc1f5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfbc1f5e00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0007290 tx=0x7fcfb00075a0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 shutdown_connections 2023-08-27T11:13:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfa40050d0 0x7fcfbc1f5e00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fcf9c06a0e0 0x7fcf9c06c5a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfbc0b2b90 0x7fcfbc159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 --2- 172.21.15.45:0/976541598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcfbc0b4730 0x7fcfbc15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 >> 172.21.15.45:0/976541598 conn(0x7fcfbc0cc7c0 msgr2=0x7fcfbc0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 shutdown_connections 2023-08-27T11:13:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:34.863+0000 7fcfc0bbf700 1 -- 172.21.15.45:0/976541598 wait complete. 2023-08-27T11:13:34.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:13:35.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.002+0000 7f6451ecd700 1 Processor -- start 2023-08-27T11:13:35.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.002+0000 7f6451ecd700 1 -- start start 2023-08-27T11:13:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.003+0000 7f6451ecd700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f644c158cb0 0x7f644c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.003+0000 7f6451ecd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c0cc5c0 con 0x7f644c15be70 2023-08-27T11:13:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.003+0000 7f6451ecd700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c0cbd80 con 0x7f644c06ac60 2023-08-27T11:13:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.003+0000 7f6451ecd700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644c06b230 con 0x7f644c158cb0 2023-08-27T11:13:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.003+0000 7f644bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f644c15be70 0x7f644c06e210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40840/0 (socket says 172.21.15.45:40840) 2023-08-27T11:13:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.003+0000 7f644bfff700 1 -- 172.21.15.45:0/2496165113 learned_addr learned my addr 172.21.15.45:0/2496165113 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.003+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551783552 0 0) 0x7f644c0cc5c0 con 0x7f644c15be70 2023-08-27T11:13:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.003+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64340035b0 con 0x7f644c15be70 2023-08-27T11:13:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.003+0000 7f644b7fe700 1 --2- 172.21.15.45:0/2496165113 >> v2:172.21.15.135:3300/0 conn(0x7f644c158cb0 0x7f644c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1546114752 0 0) 0x7f64340035b0 con 0x7f644c15be70 2023-08-27T11:13:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 >> v2:172.21.15.135:3300/0 conn(0x7f644c158cb0 msgr2=0x7f644c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 --2- 172.21.15.45:0/2496165113 >> v2:172.21.15.135:3300/0 conn(0x7f644c158cb0 0x7f644c0ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 >> v1:172.21.15.135:6789/0 conn(0x7f644c06ac60 legacy=0x7f644c06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0ca8f0 con 0x7f644c15be70 2023-08-27T11:13:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f64440019f0 con 0x7f644c15be70 2023-08-27T11:13:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 >> v1:172.21.15.45:6789/0 conn(0x7f644c15be70 legacy=0x7f644c06e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6434003a00 0x7f6434003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f6434004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6434005210 0x7f643400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c06b230 con 0x7f6434004850 2023-08-27T11:13:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643400c0b0 con 0x7f6434003a00 2023-08-27T11:13:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643400c270 con 0x7f6434005210 2023-08-27T11:13:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644affd700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f6434004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644b7fe700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6434003a00 0x7f6434003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644bfff700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6434005210 0x7f643400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644affd700 1 -- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6434005210 msgr2=0x7f643400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644affd700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6434005210 0x7f643400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644affd700 1 -- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6434003a00 msgr2=0x7f6434003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644affd700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6434003a00 0x7f6434003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644affd700 1 -- 172.21.15.45:0/2496165113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340035b0 con 0x7f6434004850 2023-08-27T11:13:35.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644b7fe700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6434003a00 0x7f6434003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644bfff700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6434005210 0x7f643400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644bfff700 1 -- 172.21.15.45:0/2496165113 reap_dead start 2023-08-27T11:13:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644affd700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f6434004cd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6440001800 tx=0x7f6434005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9011bfa4ba6f6b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64400012f0 con 0x7f6434004850 2023-08-27T11:13:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6440005310 con 0x7f6434004850 2023-08-27T11:13:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.004+0000 7f644a7fc700 1 -- 172.21.15.45:0/2496165113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440006860 con 0x7f6434004850 2023-08-27T11:13:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.005+0000 7f6451ecd700 1 -- 172.21.15.45:0/2496165113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 msgr2=0x7f6434004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.005+0000 7f6451ecd700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f6434004cd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6440001800 tx=0x7f6434005880 comp rx=0 tx=0).stop 2023-08-27T11:13:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.005+0000 7f6451ecd700 1 -- 172.21.15.45:0/2496165113 shutdown_connections 2023-08-27T11:13:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.005+0000 7f6451ecd700 1 --2- 172.21.15.45:0/2496165113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f6434004cd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.005+0000 7f6451ecd700 1 -- 172.21.15.45:0/2496165113 >> 172.21.15.45:0/2496165113 conn(0x7f644c0cfd60 msgr2=0x7f644c0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.005+0000 7f6451ecd700 1 -- 172.21.15.45:0/2496165113 shutdown_connections 2023-08-27T11:13:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.005+0000 7f6451ecd700 1 -- 172.21.15.45:0/2496165113 wait complete. 2023-08-27T11:13:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.006+0000 7f6451ecd700 1 Processor -- start 2023-08-27T11:13:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f6451ecd700 1 -- start start 2023-08-27T11:13:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f6451ecd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f644c14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f6451ecd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f644c0c9610 0x7f644c0b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f6451ecd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f644c0c5950 0x7f644c0c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f6451ecd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0c6340 con 0x7f6434004850 2023-08-27T11:13:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f6451ecd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0c4280 con 0x7f644c0c9610 2023-08-27T11:13:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f6451ecd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0c4560 con 0x7f644c0c5950 2023-08-27T11:13:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f644b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f644c14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f644b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f644c14c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54072/0 (socket says 172.21.15.45:54072) 2023-08-27T11:13:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.007+0000 7f644b7fe700 1 -- 172.21.15.45:0/2514980963 learned_addr learned my addr 172.21.15.45:0/2514980963 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644bfff700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f644c0c5950 0x7f644c0c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644affd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f644c0c9610 0x7f644c0b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644b7fe700 1 -- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f644c0c5950 msgr2=0x7f644c0c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644b7fe700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f644c0c5950 0x7f644c0c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644b7fe700 1 -- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f644c0c9610 msgr2=0x7f644c0b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644b7fe700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f644c0c9610 0x7f644c0b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644b7fe700 1 -- 172.21.15.45:0/2514980963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0c4840 con 0x7f6434004850 2023-08-27T11:13:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644bfff700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f644c0c5950 0x7f644c0c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644affd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f644c0c9610 0x7f644c0b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f644b7fe700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f644c14c2a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f643c0118d0 tx=0x7f643c011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f6433fff700 1 -- 172.21.15.45:0/2514980963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c004220 con 0x7f6434004850 2023-08-27T11:13:35.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f6433fff700 1 -- 172.21.15.45:0/2514980963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c004380 con 0x7f6434004850 2023-08-27T11:13:35.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c0c4b20 con 0x7f6434004850 2023-08-27T11:13:35.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f6433fff700 1 -- 172.21.15.45:0/2514980963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c0036a0 con 0x7f6434004850 2023-08-27T11:13:35.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.008+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c0c5320 con 0x7f6434004850 2023-08-27T11:13:35.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.009+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c0c5170 con 0x7f6434004850 2023-08-27T11:13:35.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.010+0000 7f6433fff700 1 -- 172.21.15.45:0/2514980963 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f643c003800 con 0x7f6434004850 2023-08-27T11:13:35.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.010+0000 7f6433fff700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f643806a0e0 0x7f643806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.010+0000 7f6433fff700 1 -- 172.21.15.45:0/2514980963 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f643c088850 con 0x7f6434004850 2023-08-27T11:13:35.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.010+0000 7f644affd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f643806a0e0 0x7f643806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.011+0000 7f644affd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f643806a0e0 0x7f643806c5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6440001800 tx=0x7f6440009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:35.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.015+0000 7f6433fff700 1 -- 172.21.15.45:0/2514980963 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f643c057be0 con 0x7f6434004850 2023-08-27T11:13:35.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.221+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f644c072780 con 0x7f6434004850 2023-08-27T11:13:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.222+0000 7f6433fff700 1 -- 172.21.15.45:0/2514980963 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) v1 ==== 84+0+126 (secure 0 0 0) 0x7f643c022090 con 0x7f6434004850 2023-08-27T11:13:35.223 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:35.223 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:13:35.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:35.224 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-08-27T11:13:35.224 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-08-27T11:13:35.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:13:35.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:35.224 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-08-27T11:13:35.225 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-08-27T11:13:35.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:35.225 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:13:35.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f643806a0e0 msgr2=0x7f643806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f643806a0e0 0x7f643806c5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6440001800 tx=0x7f6440009030 comp rx=0 tx=0).stop 2023-08-27T11:13:35.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 msgr2=0x7f644c14c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f644c14c2a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f643c0118d0 tx=0x7f643c011c90 comp rx=0 tx=0).stop 2023-08-27T11:13:35.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 shutdown_connections 2023-08-27T11:13:35.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6434004850 0x7f644c14c2a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f643806a0e0 0x7f643806c5a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f644c0c5950 0x7f644c0c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 --2- 172.21.15.45:0/2514980963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f644c0c9610 0x7f644c0b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 >> 172.21.15.45:0/2514980963 conn(0x7f644c0cfd60 msgr2=0x7f644c06e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:35.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 shutdown_connections 2023-08-27T11:13:35.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.224+0000 7f6451ecd700 1 -- 172.21.15.45:0/2514980963 wait complete. 2023-08-27T11:13:35.237 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:13:35.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:13:35.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.366+0000 7fb36b6c6700 1 Processor -- start 2023-08-27T11:13:35.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.366+0000 7fb36b6c6700 1 -- start start 2023-08-27T11:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.367+0000 7fb36b6c6700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb3640aeb10 0x7fb364072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.367+0000 7fb36b6c6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb36406aa70 0x7fb36406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.367+0000 7fb36b6c6700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36406ddf0 con 0x7fb3640ae730 2023-08-27T11:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.367+0000 7fb36b6c6700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36406b440 con 0x7fb3640aeb10 2023-08-27T11:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.367+0000 7fb36b6c6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36406d830 con 0x7fb36406aa70 2023-08-27T11:13:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.367+0000 7fb369c63700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb36406aa70 0x7fb36406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.367+0000 7fb368c61700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb3640aeb10 0x7fb364072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.367+0000 7fb369c63700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb36406aa70 0x7fb36406af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:51994/0 (socket says 172.21.15.45:51994) 2023-08-27T11:13:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.367+0000 7fb369c63700 1 -- 172.21.15.45:0/3452786530 learned_addr learned my addr 172.21.15.45:0/3452786530 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.368+0000 7fb368c61700 1 -- 172.21.15.45:0/3452786530 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb36406aa70 msgr2=0x7fb36406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.368+0000 7fb368c61700 1 --2- 172.21.15.45:0/3452786530 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb36406aa70 0x7fb36406af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.368+0000 7fb368c61700 1 -- 172.21.15.45:0/3452786530 >> v1:172.21.15.135:6789/0 conn(0x7fb3640ae730 legacy=0x7fb3640ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.368+0000 7fb368c61700 1 -- 172.21.15.45:0/3452786530 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36406db10 con 0x7fb3640aeb10 2023-08-27T11:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.368+0000 7fb369c63700 1 --2- 172.21.15.45:0/3452786530 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb36406aa70 0x7fb36406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:35.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.368+0000 7fb368c61700 1 --2- 172.21.15.45:0/3452786530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640aeb10 0x7fb364072330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb358009840 tx=0x7fb358009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c52f7f2e4decd03 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:35.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.368+0000 7fb357fff700 1 -- 172.21.15.45:0/3452786530 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358011830 con 0x7fb3640aeb10 2023-08-27T11:13:35.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.368+0000 7fb357fff700 1 -- 172.21.15.45:0/3452786530 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358011990 con 0x7fb3640aeb10 2023-08-27T11:13:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.368+0000 7fb357fff700 1 -- 172.21.15.45:0/3452786530 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358011d20 con 0x7fb3640aeb10 2023-08-27T11:13:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.369+0000 7fb36b6c6700 1 -- 172.21.15.45:0/3452786530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640aeb10 msgr2=0x7fb364072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.369+0000 7fb36b6c6700 1 --2- 172.21.15.45:0/3452786530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640aeb10 0x7fb364072330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb358009840 tx=0x7fb358009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.369+0000 7fb36b6c6700 1 -- 172.21.15.45:0/3452786530 shutdown_connections 2023-08-27T11:13:35.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.369+0000 7fb36b6c6700 1 --2- 172.21.15.45:0/3452786530 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb36406aa70 0x7fb36406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.369+0000 7fb36b6c6700 1 --2- 172.21.15.45:0/3452786530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640aeb10 0x7fb364072330 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.369+0000 7fb36b6c6700 1 -- 172.21.15.45:0/3452786530 >> 172.21.15.45:0/3452786530 conn(0x7fb3640b2440 msgr2=0x7fb3640b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:35.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.369+0000 7fb36b6c6700 1 -- 172.21.15.45:0/3452786530 shutdown_connections 2023-08-27T11:13:35.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.369+0000 7fb36b6c6700 1 -- 172.21.15.45:0/3452786530 wait complete. 2023-08-27T11:13:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.370+0000 7fb36b6c6700 1 Processor -- start 2023-08-27T11:13:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.370+0000 7fb36b6c6700 1 -- start start 2023-08-27T11:13:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb36b6c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36406aa70 0x7fb3640cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb36b6c6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640ae730 0x7fb36414ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb36b6c6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3640aeb10 0x7fb3640984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb36b6c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640cffb0 con 0x7fb36406aa70 2023-08-27T11:13:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb36b6c6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415c980 con 0x7fb3640ae730 2023-08-27T11:13:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb36b6c6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415c0e0 con 0x7fb3640aeb10 2023-08-27T11:13:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369462700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36406aa70 0x7fb3640cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369462700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36406aa70 0x7fb3640cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54086/0 (socket says 172.21.15.45:54086) 2023-08-27T11:13:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369462700 1 -- 172.21.15.45:0/781143379 learned_addr learned my addr 172.21.15.45:0/781143379 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb368c61700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640ae730 0x7fb36414ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369c63700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3640aeb10 0x7fb3640984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369462700 1 -- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3640aeb10 msgr2=0x7fb3640984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369462700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3640aeb10 0x7fb3640984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369462700 1 -- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640ae730 msgr2=0x7fb36414ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369462700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640ae730 0x7fb36414ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369462700 1 -- 172.21.15.45:0/781143379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c009b30 con 0x7fb36406aa70 2023-08-27T11:13:35.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb368c61700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640ae730 0x7fb36414ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.371+0000 7fb369c63700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3640aeb10 0x7fb3640984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.372+0000 7fb369462700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36406aa70 0x7fb3640cfa70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c4d0 tx=0x7fb36000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.372+0000 7fb355ffb700 1 -- 172.21.15.45:0/781143379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600052d0 con 0x7fb36406aa70 2023-08-27T11:13:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.372+0000 7fb355ffb700 1 -- 172.21.15.45:0/781143379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb360005430 con 0x7fb36406aa70 2023-08-27T11:13:35.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.372+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3580094a0 con 0x7fb36406aa70 2023-08-27T11:13:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.372+0000 7fb355ffb700 1 -- 172.21.15.45:0/781143379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600038f0 con 0x7fb36406aa70 2023-08-27T11:13:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.372+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36415c6a0 con 0x7fb36406aa70 2023-08-27T11:13:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.373+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36409d4b0 con 0x7fb36406aa70 2023-08-27T11:13:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.373+0000 7fb355ffb700 1 -- 172.21.15.45:0/781143379 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb360003a50 con 0x7fb36406aa70 2023-08-27T11:13:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.374+0000 7fb355ffb700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb34806a0e0 0x7fb34806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:35.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.374+0000 7fb355ffb700 1 -- 172.21.15.45:0/781143379 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb3600877a0 con 0x7fb36406aa70 2023-08-27T11:13:35.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.374+0000 7fb368c61700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb34806a0e0 0x7fb34806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:35.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.375+0000 7fb368c61700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb34806a0e0 0x7fb34806c5a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb358000c00 tx=0x7fb358008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:35.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.378+0000 7fb355ffb700 1 -- 172.21.15.45:0/781143379 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb360053130 con 0x7fb36406aa70 2023-08-27T11:13:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.580+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb36406bf60 con 0x7fb36406aa70 2023-08-27T11:13:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.885+0000 7fb355ffb700 1 -- 172.21.15.45:0/781143379 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v39) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb360056750 con 0x7fb36406aa70 2023-08-27T11:13:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb34806a0e0 msgr2=0x7fb34806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb34806a0e0 0x7fb34806c5a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb358000c00 tx=0x7fb358008030 comp rx=0 tx=0).stop 2023-08-27T11:13:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36406aa70 msgr2=0x7fb3640cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36406aa70 0x7fb3640cfa70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c4d0 tx=0x7fb36000c890 comp rx=0 tx=0).stop 2023-08-27T11:13:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 shutdown_connections 2023-08-27T11:13:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36406aa70 0x7fb3640cfa70 secure :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c4d0 tx=0x7fb36000c890 comp rx=0 tx=0).stop 2023-08-27T11:13:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb34806a0e0 0x7fb34806c5a0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb358000c00 tx=0x7fb358008030 comp rx=0 tx=0).stop 2023-08-27T11:13:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3640aeb10 0x7fb3640984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 --2- 172.21.15.45:0/781143379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3640ae730 0x7fb36414ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 >> 172.21.15.45:0/781143379 conn(0x7fb3640b2440 msgr2=0x7fb3640adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.889+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 shutdown_connections 2023-08-27T11:13:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:35.890+0000 7fb36b6c6700 1 -- 172.21.15.45:0/781143379 wait complete. 2023-08-27T11:13:35.893 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-08-27T11:13:35.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:13:36.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.039+0000 7f5fdcea5700 1 Processor -- start 2023-08-27T11:13:36.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.039+0000 7f5fdcea5700 1 -- start start 2023-08-27T11:13:36.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.040+0000 7f5fdcea5700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5fd815b5e0 0x7f5fd806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:36.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.040+0000 7f5fdcea5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd806f450 0x7f5fd80cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:36.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.040+0000 7f5fdcea5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd80cb060 con 0x7f5fd815a700 2023-08-27T11:13:36.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.040+0000 7f5fdcea5700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd806e320 con 0x7f5fd815b5e0 2023-08-27T11:13:36.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.040+0000 7f5fdcea5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd806f8c0 con 0x7f5fd806f450 2023-08-27T11:13:36.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.040+0000 7f5fd659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5fd815a700 0x7f5fd815aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40870/0 (socket says 172.21.15.45:40870) 2023-08-27T11:13:36.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.040+0000 7f5fd659c700 1 -- 172.21.15.45:0/1713482351 learned_addr learned my addr 172.21.15.45:0/1713482351 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:36.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.040+0000 7f5fd5d9b700 1 --2- 172.21.15.45:0/1713482351 >> v2:172.21.15.135:3300/0 conn(0x7f5fd815b5e0 0x7f5fd806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:36.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.040+0000 7f5fd6d9d700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd806f450 0x7f5fd80cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:36.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660714714 0 0) 0x7f5fd80cb060 con 0x7f5fd815a700 2023-08-27T11:13:36.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc00035b0 con 0x7f5fd815a700 2023-08-27T11:13:36.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2690788758 0 0) 0x7f5fc00035b0 con 0x7f5fd815a700 2023-08-27T11:13:36.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd806f450 msgr2=0x7f5fd80cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:36.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd806f450 0x7f5fd80cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:36.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 >> v2:172.21.15.135:3300/0 conn(0x7f5fd815b5e0 msgr2=0x7f5fd806ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:36.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 --2- 172.21.15.45:0/1713482351 >> v2:172.21.15.135:3300/0 conn(0x7f5fd815b5e0 0x7f5fd806ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:36.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd815d7d0 con 0x7f5fd815a700 2023-08-27T11:13:36.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f5fc8001670 con 0x7f5fd815a700 2023-08-27T11:13:36.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 >> v1:172.21.15.45:6789/0 conn(0x7f5fd815a700 legacy=0x7f5fd815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:36.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fc0003a00 0x7f5fc0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:36.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fc00048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:36.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc0004f40 0x7f5fc000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:36.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd806e320 con 0x7f5fc00043f0 2023-08-27T11:13:36.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd806f8c0 con 0x7f5fc0004f40 2023-08-27T11:13:36.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc000e160 con 0x7f5fc0003a00 2023-08-27T11:13:36.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd6d9d700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd806f450 0x7f5fd80cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:36.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd5d9b700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fc00048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:36.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd659c700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fc0003a00 0x7f5fc0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:36.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd6d9d700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc0004f40 0x7f5fc000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:36.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd5d9b700 1 -- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fc0003a00 msgr2=0x7f5fc0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:36.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd5d9b700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fc0003a00 0x7f5fc0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:36.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd5d9b700 1 -- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc0004f40 msgr2=0x7f5fc000dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:36.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd5d9b700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc0004f40 0x7f5fc000dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:36.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd5d9b700 1 -- 172.21.15.45:0/1713482351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00035b0 con 0x7f5fc00043f0 2023-08-27T11:13:36.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd6d9d700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc0004f40 0x7f5fc000dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:36.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.041+0000 7f5fd5d9b700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fc00048d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009b60 tx=0x7f5fc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33952aa23df032af server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:36.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00e6c0 con 0x7f5fc00043f0 2023-08-27T11:13:36.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc00e820 con 0x7f5fc00043f0 2023-08-27T11:13:36.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fd559a700 1 -- 172.21.15.45:0/1713482351 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00eb90 con 0x7f5fc00043f0 2023-08-27T11:13:36.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fdcea5700 1 -- 172.21.15.45:0/1713482351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 msgr2=0x7f5fc00048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:36.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fdcea5700 1 --2- 172.21.15.45:0/1713482351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fc00048d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009b60 tx=0x7f5fc0005880 comp rx=0 tx=0).stop 2023-08-27T11:13:36.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fd6d9d700 1 -- 172.21.15.45:0/1713482351 reap_dead start 2023-08-27T11:13:36.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fdcea5700 1 -- 172.21.15.45:0/1713482351 shutdown_connections 2023-08-27T11:13:36.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fdcea5700 1 -- 172.21.15.45:0/1713482351 >> 172.21.15.45:0/1713482351 conn(0x7f5fd80cc7e0 msgr2=0x7f5fd80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:36.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fdcea5700 1 -- 172.21.15.45:0/1713482351 shutdown_connections 2023-08-27T11:13:36.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.042+0000 7f5fdcea5700 1 -- 172.21.15.45:0/1713482351 wait complete. 2023-08-27T11:13:36.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.044+0000 7f5fdcea5700 1 Processor -- start 2023-08-27T11:13:36.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.044+0000 7f5fdcea5700 1 -- start start 2023-08-27T11:13:36.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fdcea5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fd80b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:36.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fdcea5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd80b2cc0 0x7f5fd81ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:36.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fdcea5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd81eea10 0x7f5fd81eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:36.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fdcea5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80b3e70 con 0x7f5fc00043f0 2023-08-27T11:13:36.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fdcea5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81ef3e0 con 0x7f5fd80b2cc0 2023-08-27T11:13:36.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fdcea5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81577c0 con 0x7f5fd81eea10 2023-08-27T11:13:36.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fd80b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:36.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fd80b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54106/0 (socket says 172.21.15.45:54106) 2023-08-27T11:13:36.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd659c700 1 -- 172.21.15.45:0/2988539980 learned_addr learned my addr 172.21.15.45:0/2988539980 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:36.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd6d9d700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd81eea10 0x7f5fd81eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:36.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd5d9b700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd80b2cc0 0x7f5fd81ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:36.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd659c700 1 -- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd81eea10 msgr2=0x7f5fd81eeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:36.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd659c700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd81eea10 0x7f5fd81eeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:36.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd659c700 1 -- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd80b2cc0 msgr2=0x7f5fd81ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:36.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd659c700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd80b2cc0 0x7f5fd81ee4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:36.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.045+0000 7f5fd659c700 1 -- 172.21.15.45:0/2988539980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd80b32f0 con 0x7f5fc00043f0 2023-08-27T11:13:36.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.046+0000 7f5fd659c700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fd80b2780 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8007340 tx=0x7f5fc8007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:36.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.046+0000 7f5fc6ffd700 1 -- 172.21.15.45:0/2988539980 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800f660 con 0x7f5fc00043f0 2023-08-27T11:13:36.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.046+0000 7f5fc6ffd700 1 -- 172.21.15.45:0/2988539980 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc800f800 con 0x7f5fc00043f0 2023-08-27T11:13:36.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.046+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd80b35d0 con 0x7f5fc00043f0 2023-08-27T11:13:36.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.046+0000 7f5fc6ffd700 1 -- 172.21.15.45:0/2988539980 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800fbb0 con 0x7f5fc00043f0 2023-08-27T11:13:36.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.046+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd80b3b90 con 0x7f5fc00043f0 2023-08-27T11:13:36.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.047+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd80c7f10 con 0x7f5fc00043f0 2023-08-27T11:13:36.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.047+0000 7f5fc6ffd700 1 -- 172.21.15.45:0/2988539980 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5fc800fd10 con 0x7f5fc00043f0 2023-08-27T11:13:36.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.048+0000 7f5fc6ffd700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5fb806a0e0 0x7f5fb806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:36.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.048+0000 7f5fc6ffd700 1 -- 172.21.15.45:0/2988539980 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f5fc80883a0 con 0x7f5fc00043f0 2023-08-27T11:13:36.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.048+0000 7f5fd5d9b700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5fb806a0e0 0x7f5fb806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:36.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.048+0000 7f5fd5d9b700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5fb806a0e0 0x7f5fb806c5a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00047d0 tx=0x7f5fcc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:36.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.052+0000 7f5fc6ffd700 1 -- 172.21.15.45:0/2988539980 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5fd80c7f10 con 0x7f5fc00043f0 2023-08-27T11:13:36.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.251+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5fd80c7f10 con 0x7f5fc00043f0 2023-08-27T11:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.892+0000 7f5fc6ffd700 1 -- 172.21.15.45:0/2988539980 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v40) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5fd80c7f10 con 0x7f5fc00043f0 2023-08-27T11:13:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5fb806a0e0 msgr2=0x7f5fb806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5fb806a0e0 0x7f5fb806c5a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00047d0 tx=0x7f5fcc008030 comp rx=0 tx=0).stop 2023-08-27T11:13:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 msgr2=0x7f5fd80b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fd80b2780 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8007340 tx=0x7f5fc8007650 comp rx=0 tx=0).stop 2023-08-27T11:13:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 shutdown_connections 2023-08-27T11:13:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fc00043f0 0x7f5fd80b2780 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f5fb806a0e0 0x7f5fb806c5a0 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00047d0 tx=0x7f5fcc008030 comp rx=0 tx=0).stop 2023-08-27T11:13:36.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5fd81eea10 0x7f5fd81eeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:36.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 --2- 172.21.15.45:0/2988539980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fd80b2cc0 0x7f5fd81ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:36.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.901+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 >> 172.21.15.45:0/2988539980 conn(0x7f5fd80cc7e0 msgr2=0x7f5fd80cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:36.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.903+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 shutdown_connections 2023-08-27T11:13:36.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:36.904+0000 7f5fdcea5700 1 -- 172.21.15.45:0/2988539980 wait complete. 2023-08-27T11:13:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-08-27T11:13:36.926 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:13:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.059+0000 7fc1721cf700 1 Processor -- start 2023-08-27T11:13:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.059+0000 7fc1721cf700 1 -- start start 2023-08-27T11:13:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc1721cf700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0b65d0 0x7fc16c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc1721cf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc16c0d0520 0x7fc16c15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc1721cf700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c15ec50 con 0x7fc16c0cb5c0 2023-08-27T11:13:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc1721cf700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c15c620 con 0x7fc16c0d0520 2023-08-27T11:13:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc1721cf700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c15eea0 con 0x7fc16c0b65d0 2023-08-27T11:13:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc16bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc16c0d0520 0x7fc16c15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc16bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc16c0d0520 0x7fc16c15c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54118/0 (socket says 172.21.15.45:54118) 2023-08-27T11:13:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc16bfff700 1 -- 172.21.15.45:0/2253229498 learned_addr learned my addr 172.21.15.45:0/2253229498 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc16b7fe700 1 --2- 172.21.15.45:0/2253229498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0b65d0 0x7fc16c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc16bfff700 1 -- 172.21.15.45:0/2253229498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0b65d0 msgr2=0x7fc16c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc16bfff700 1 --2- 172.21.15.45:0/2253229498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0b65d0 0x7fc16c0ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc16bfff700 1 -- 172.21.15.45:0/2253229498 >> v1:172.21.15.135:6789/0 conn(0x7fc16c0cb5c0 legacy=0x7fc16c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc16bfff700 1 -- 172.21.15.45:0/2253229498 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c15f180 con 0x7fc16c0d0520 2023-08-27T11:13:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.060+0000 7fc16bfff700 1 --2- 172.21.15.45:0/2253229498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 0x7fc16c15c0e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc164009e50 tx=0x7fc16400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b797621388f51da5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc16a7fc700 1 -- 172.21.15.45:0/2253229498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16400e070 con 0x7fc16c0d0520 2023-08-27T11:13:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc16a7fc700 1 -- 172.21.15.45:0/2253229498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16400b910 con 0x7fc16c0d0520 2023-08-27T11:13:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc16a7fc700 1 -- 172.21.15.45:0/2253229498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16400bcc0 con 0x7fc16c0d0520 2023-08-27T11:13:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc1721cf700 1 -- 172.21.15.45:0/2253229498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 msgr2=0x7fc16c15c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc1721cf700 1 --2- 172.21.15.45:0/2253229498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 0x7fc16c15c0e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc164009e50 tx=0x7fc16400b330 comp rx=0 tx=0).stop 2023-08-27T11:13:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc1721cf700 1 -- 172.21.15.45:0/2253229498 shutdown_connections 2023-08-27T11:13:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc1721cf700 1 --2- 172.21.15.45:0/2253229498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0b65d0 0x7fc16c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc1721cf700 1 --2- 172.21.15.45:0/2253229498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 0x7fc16c15c0e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc1721cf700 1 -- 172.21.15.45:0/2253229498 >> 172.21.15.45:0/2253229498 conn(0x7fc16c06f960 msgr2=0x7fc16c06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc1721cf700 1 -- 172.21.15.45:0/2253229498 shutdown_connections 2023-08-27T11:13:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.061+0000 7fc1721cf700 1 -- 172.21.15.45:0/2253229498 wait complete. 2023-08-27T11:13:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.062+0000 7fc1721cf700 1 Processor -- start 2023-08-27T11:13:37.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.063+0000 7fc1721cf700 1 -- start start 2023-08-27T11:13:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.063+0000 7fc1721cf700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc16c0b65d0 0x7fc16c1f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.063+0000 7fc1721cf700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0cb5c0 0x7fc16c0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.063+0000 7fc1721cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 0x7fc16c0c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.063+0000 7fc1721cf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0c80e0 con 0x7fc16c0d0520 2023-08-27T11:13:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.063+0000 7fc1721cf700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0ca8a0 con 0x7fc16c0b65d0 2023-08-27T11:13:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.063+0000 7fc1721cf700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0c8380 con 0x7fc16c0cb5c0 2023-08-27T11:13:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 0x7fc16c0c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:37.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 0x7fc16c0c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54124/0 (socket says 172.21.15.45:54124) 2023-08-27T11:13:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16bfff700 1 -- 172.21.15.45:0/1130447693 learned_addr learned my addr 172.21.15.45:0/1130447693 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16b7fe700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc16c0b65d0 0x7fc16c1f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16affd700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0cb5c0 0x7fc16c0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16bfff700 1 -- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0cb5c0 msgr2=0x7fc16c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:37.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16bfff700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0cb5c0 0x7fc16c0c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16bfff700 1 -- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc16c0b65d0 msgr2=0x7fc16c1f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16bfff700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc16c0b65d0 0x7fc16c1f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16bfff700 1 -- 172.21.15.45:0/1130447693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164009b30 con 0x7fc16c0d0520 2023-08-27T11:13:37.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16affd700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0cb5c0 0x7fc16c0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc16bfff700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 0x7fc16c0c9d50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc164012000 tx=0x7fc164004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc153fff700 1 -- 172.21.15.45:0/1130447693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1640048b0 con 0x7fc16c0d0520 2023-08-27T11:13:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc153fff700 1 -- 172.21.15.45:0/1130447693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1640099b0 con 0x7fc16c0d0520 2023-08-27T11:13:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc153fff700 1 -- 172.21.15.45:0/1130447693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16401c680 con 0x7fc16c0d0520 2023-08-27T11:13:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c0c8660 con 0x7fc16c0d0520 2023-08-27T11:13:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.064+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c0c9a40 con 0x7fc16c0d0520 2023-08-27T11:13:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.065+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c0c8b20 con 0x7fc16c0d0520 2023-08-27T11:13:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.066+0000 7fc153fff700 1 -- 172.21.15.45:0/1130447693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc16401c7e0 con 0x7fc16c0d0520 2023-08-27T11:13:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.066+0000 7fc153fff700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc15806a0e0 0x7fc15806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.066+0000 7fc153fff700 1 -- 172.21.15.45:0/1130447693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fc16408a270 con 0x7fc16c0d0520 2023-08-27T11:13:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.066+0000 7fc16b7fe700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc15806a0e0 0x7fc15806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.067+0000 7fc16b7fe700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc15806a0e0 0x7fc15806c5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009be0 tx=0x7fc15c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.071+0000 7fc153fff700 1 -- 172.21.15.45:0/1130447693 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc164055890 con 0x7fc16c0d0520 2023-08-27T11:13:37.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.319+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc16c06f4a0 con 0x7fc16c0d0520 2023-08-27T11:13:37.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.893+0000 7fc153fff700 1 -- 172.21.15.45:0/1130447693 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc164058eb0 con 0x7fc16c0d0520 2023-08-27T11:13:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc15806a0e0 msgr2=0x7fc15806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc15806a0e0 0x7fc15806c5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009be0 tx=0x7fc15c009210 comp rx=0 tx=0).stop 2023-08-27T11:13:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 msgr2=0x7fc16c0c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 0x7fc16c0c9d50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc164012000 tx=0x7fc164004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 shutdown_connections 2023-08-27T11:13:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16c0d0520 0x7fc16c0c9d50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fc15806a0e0 0x7fc15806c5a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc16c0b65d0 0x7fc16c1f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 --2- 172.21.15.45:0/1130447693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc16c0cb5c0 0x7fc16c0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.898+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 >> 172.21.15.45:0/1130447693 conn(0x7fc16c06f960 msgr2=0x7fc16c0b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.900+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 shutdown_connections 2023-08-27T11:13:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:37.900+0000 7fc1721cf700 1 -- 172.21.15.45:0/1130447693 wait complete. 2023-08-27T11:13:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 6 and data pool 7 2023-08-27T11:13:37.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.045+0000 7f2f584c8700 1 Processor -- start 2023-08-27T11:13:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.045+0000 7f2f584c8700 1 -- start start 2023-08-27T11:13:38.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.046+0000 7f2f584c8700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2f5015b0d0 0x7f2f500d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.046+0000 7f2f584c8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500727f0 con 0x7f2f5015b4b0 2023-08-27T11:13:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.046+0000 7f2f584c8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500cf300 con 0x7f2f5006a990 2023-08-27T11:13:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.046+0000 7f2f584c8700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f500cf5e0 con 0x7f2f5015b0d0 2023-08-27T11:13:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.046+0000 7f2f55a63700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2f5015b4b0 0x7f2f5006a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40918/0 (socket says 172.21.15.45:40918) 2023-08-27T11:13:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.046+0000 7f2f55a63700 1 -- 172.21.15.45:0/2774523697 learned_addr learned my addr 172.21.15.45:0/2774523697 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f56264700 1 --2- 172.21.15.45:0/2774523697 >> v2:172.21.15.135:3300/0 conn(0x7f2f5015b0d0 0x7f2f500d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572609719 0 0) 0x7f2f500727f0 con 0x7f2f5015b4b0 2023-08-27T11:13:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f3c0035b0 con 0x7f2f5015b4b0 2023-08-27T11:13:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2857313682 0 0) 0x7f2f3c0035b0 con 0x7f2f5015b4b0 2023-08-27T11:13:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 >> v2:172.21.15.135:3300/0 conn(0x7f2f5015b0d0 msgr2=0x7f2f500d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 --2- 172.21.15.45:0/2774523697 >> v2:172.21.15.135:3300/0 conn(0x7f2f5015b0d0 0x7f2f500d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 >> v1:172.21.15.135:6789/0 conn(0x7f2f5006a990 legacy=0x7f2f500af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500cf8c0 con 0x7f2f5015b4b0 2023-08-27T11:13:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f2f4c001670 con 0x7f2f5015b4b0 2023-08-27T11:13:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 >> v1:172.21.15.45:6789/0 conn(0x7f2f5015b4b0 legacy=0x7f2f5006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f3c003a00 0x7f2f3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f3c004850 0x7f2f3c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f3c005210 0x7f2f3c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cf5e0 con 0x7f2f3c004850 2023-08-27T11:13:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c00c0b0 con 0x7f2f3c005210 2023-08-27T11:13:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c00c270 con 0x7f2f3c003a00 2023-08-27T11:13:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55a63700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f3c004850 0x7f2f3c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f56264700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f3c003a00 0x7f2f3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f56a65700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f3c005210 0x7f2f3c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55a63700 1 -- 172.21.15.45:0/2774523697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f3c003a00 msgr2=0x7f2f3c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55a63700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f3c003a00 0x7f2f3c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55a63700 1 -- 172.21.15.45:0/2774523697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f3c005210 msgr2=0x7f2f3c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55a63700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f3c005210 0x7f2f3c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f55a63700 1 -- 172.21.15.45:0/2774523697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0035b0 con 0x7f2f3c004850 2023-08-27T11:13:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f56a65700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f3c005210 0x7f2f3c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.047+0000 7f2f56a65700 1 -- 172.21.15.45:0/2774523697 reap_dead start 2023-08-27T11:13:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f55a63700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f3c004850 0x7f2f3c004cd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00c240 tx=0x7f2f3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb26ab5d830c95b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c007c10 con 0x7f2f3c004850 2023-08-27T11:13:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c00cde0 con 0x7f2f3c004850 2023-08-27T11:13:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f55262700 1 -- 172.21.15.45:0/2774523697 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c005ba0 con 0x7f2f3c004850 2023-08-27T11:13:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f584c8700 1 -- 172.21.15.45:0/2774523697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f3c004850 msgr2=0x7f2f3c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f584c8700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f3c004850 0x7f2f3c004cd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00c240 tx=0x7f2f3c005880 comp rx=0 tx=0).stop 2023-08-27T11:13:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f584c8700 1 -- 172.21.15.45:0/2774523697 shutdown_connections 2023-08-27T11:13:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f584c8700 1 --2- 172.21.15.45:0/2774523697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f3c004850 0x7f2f3c004cd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f584c8700 1 -- 172.21.15.45:0/2774523697 >> 172.21.15.45:0/2774523697 conn(0x7f2f500b2470 msgr2=0x7f2f500b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f584c8700 1 -- 172.21.15.45:0/2774523697 shutdown_connections 2023-08-27T11:13:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.048+0000 7f2f584c8700 1 -- 172.21.15.45:0/2774523697 wait complete. 2023-08-27T11:13:38.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.050+0000 7f2f584c8700 1 Processor -- start 2023-08-27T11:13:38.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.050+0000 7f2f584c8700 1 -- start start 2023-08-27T11:13:38.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.050+0000 7f2f584c8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f3c004850 0x7f2f50155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.050+0000 7f2f584c8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f50157b30 0x7f2f501567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.050+0000 7f2f584c8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f501f1e80 0x7f2f501f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.050+0000 7f2f584c8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50156cf0 con 0x7f2f501f1e80 2023-08-27T11:13:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.050+0000 7f2f584c8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50157fb0 con 0x7f2f50157b30 2023-08-27T11:13:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.050+0000 7f2f584c8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50155710 con 0x7f2f3c004850 2023-08-27T11:13:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56a65700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f501f1e80 0x7f2f501f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56a65700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f501f1e80 0x7f2f501f2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54154/0 (socket says 172.21.15.45:54154) 2023-08-27T11:13:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56a65700 1 -- 172.21.15.45:0/3121123792 learned_addr learned my addr 172.21.15.45:0/3121123792 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f55a63700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f50157b30 0x7f2f501567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56264700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f3c004850 0x7f2f50155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56a65700 1 -- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f3c004850 msgr2=0x7f2f50155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56a65700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f3c004850 0x7f2f50155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56a65700 1 -- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f50157b30 msgr2=0x7f2f501567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56a65700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f50157b30 0x7f2f501567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56a65700 1 -- 172.21.15.45:0/3121123792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f501559f0 con 0x7f2f501f1e80 2023-08-27T11:13:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f56a65700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f501f1e80 0x7f2f501f2300 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2f5015ff20 tx=0x7f2f4800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f46ffd700 1 -- 172.21.15.45:0/3121123792 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48003c10 con 0x7f2f501f1e80 2023-08-27T11:13:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f46ffd700 1 -- 172.21.15.45:0/3121123792 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48003d70 con 0x7f2f501f1e80 2023-08-27T11:13:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f46ffd700 1 -- 172.21.15.45:0/3121123792 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48005310 con 0x7f2f501f1e80 2023-08-27T11:13:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f50155cd0 con 0x7f2f501f1e80 2023-08-27T11:13:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.051+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f50156290 con 0x7f2f501f1e80 2023-08-27T11:13:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.052+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5009d5b0 con 0x7f2f501f1e80 2023-08-27T11:13:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.053+0000 7f2f46ffd700 1 -- 172.21.15.45:0/3121123792 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2f48005470 con 0x7f2f501f1e80 2023-08-27T11:13:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.053+0000 7f2f46ffd700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2f3406a110 0x7f2f3406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.053+0000 7f2f46ffd700 1 -- 172.21.15.45:0/3121123792 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2f480885a0 con 0x7f2f501f1e80 2023-08-27T11:13:38.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.054+0000 7f2f56264700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2f3406a110 0x7f2f3406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.054+0000 7f2f56264700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2f3406a110 0x7f2f3406c5d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009b20 tx=0x7f2f40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:38.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.058+0000 7f2f46ffd700 1 -- 172.21.15.45:0/3121123792 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2f48053b50 con 0x7f2f501f1e80 2023-08-27T11:13:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.258+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f5006aea0 con 0x7f2f501f1e80 2023-08-27T11:13:38.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.259+0000 7f2f46ffd700 1 -- 172.21.15.45:0/3121123792 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18525 (secure 0 0 0) 0x7f2f4801a310 con 0x7f2f501f1e80 2023-08-27T11:13:38.260 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:13:37.320452+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:13:35.610850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-27T11:13:36.282109+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:38.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2f3406a110 msgr2=0x7f2f3406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2f3406a110 0x7f2f3406c5d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009b20 tx=0x7f2f40008040 comp rx=0 tx=0).stop 2023-08-27T11:13:38.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f501f1e80 msgr2=0x7f2f501f2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f501f1e80 0x7f2f501f2300 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2f5015ff20 tx=0x7f2f4800ee00 comp rx=0 tx=0).stop 2023-08-27T11:13:38.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 shutdown_connections 2023-08-27T11:13:38.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f501f1e80 0x7f2f501f2300 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2f3406a110 0x7f2f3406c5d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2f3c004850 0x7f2f50155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 --2- 172.21.15.45:0/3121123792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f50157b30 0x7f2f501567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 >> 172.21.15.45:0/3121123792 conn(0x7f2f500b2470 msgr2=0x7f2f500ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:38.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 shutdown_connections 2023-08-27T11:13:38.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.261+0000 7f2f584c8700 1 -- 172.21.15.45:0/3121123792 wait complete. 2023-08-27T11:13:38.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:38.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.398+0000 7f917d873700 1 Processor -- start 2023-08-27T11:13:38.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.398+0000 7f917d873700 1 -- start start 2023-08-27T11:13:38.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.399+0000 7f917d873700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f917815d450 0x7f91780cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.399+0000 7f917d873700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f91780d0440 0x7f9178161250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.399+0000 7f917d873700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91780ce1c0 con 0x7f91780cbaf0 2023-08-27T11:13:38.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.399+0000 7f917d873700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91780cf290 con 0x7f917815d450 2023-08-27T11:13:38.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.399+0000 7f917d873700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91780ce440 con 0x7f91780d0440 2023-08-27T11:13:38.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.399+0000 7f9176ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f917815d450 0x7f91780cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.399+0000 7f9176ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f917815d450 0x7f91780cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54160/0 (socket says 172.21.15.45:54160) 2023-08-27T11:13:38.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.399+0000 7f9176ffd700 1 -- 172.21.15.45:0/3045428921 learned_addr learned my addr 172.21.15.45:0/3045428921 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:38.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f91777fe700 1 --2- 172.21.15.45:0/3045428921 >> v2:172.21.15.135:3300/0 conn(0x7f91780d0440 0x7f9178161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f9176ffd700 1 -- 172.21.15.45:0/3045428921 >> v2:172.21.15.135:3300/0 conn(0x7f91780d0440 msgr2=0x7f9178161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:38.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f9176ffd700 1 --2- 172.21.15.45:0/3045428921 >> v2:172.21.15.135:3300/0 conn(0x7f91780d0440 0x7f9178161250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f9176ffd700 1 -- 172.21.15.45:0/3045428921 >> v1:172.21.15.135:6789/0 conn(0x7f91780cbaf0 legacy=0x7f91780cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f9176ffd700 1 -- 172.21.15.45:0/3045428921 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91780ce720 con 0x7f917815d450 2023-08-27T11:13:38.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f9176ffd700 1 --2- 172.21.15.45:0/3045428921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f917815d450 0x7f91780cb5b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f91780d0f30 tx=0x7f9168009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104171e4b0332be0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f9175ffb700 1 -- 172.21.15.45:0/3045428921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168011830 con 0x7f917815d450 2023-08-27T11:13:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f9175ffb700 1 -- 172.21.15.45:0/3045428921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9168011990 con 0x7f917815d450 2023-08-27T11:13:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f9175ffb700 1 -- 172.21.15.45:0/3045428921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168011d20 con 0x7f917815d450 2023-08-27T11:13:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f917d873700 1 -- 172.21.15.45:0/3045428921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f917815d450 msgr2=0x7f91780cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.400+0000 7f917d873700 1 --2- 172.21.15.45:0/3045428921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f917815d450 0x7f91780cb5b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f91780d0f30 tx=0x7f9168009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.401+0000 7f917d873700 1 -- 172.21.15.45:0/3045428921 shutdown_connections 2023-08-27T11:13:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.401+0000 7f917d873700 1 --2- 172.21.15.45:0/3045428921 >> v2:172.21.15.135:3300/0 conn(0x7f91780d0440 0x7f9178161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.401+0000 7f917d873700 1 --2- 172.21.15.45:0/3045428921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f917815d450 0x7f91780cb5b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.401+0000 7f917d873700 1 -- 172.21.15.45:0/3045428921 >> 172.21.15.45:0/3045428921 conn(0x7f91780d0f60 msgr2=0x7f91780d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:38.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.401+0000 7f917d873700 1 -- 172.21.15.45:0/3045428921 shutdown_connections 2023-08-27T11:13:38.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.401+0000 7f917d873700 1 -- 172.21.15.45:0/3045428921 wait complete. 2023-08-27T11:13:38.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.402+0000 7f917d873700 1 Processor -- start 2023-08-27T11:13:38.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.402+0000 7f917d873700 1 -- start start 2023-08-27T11:13:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f917d873700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91780cbaf0 0x7f91781f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f917d873700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91780d0440 0x7f917815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f917d873700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f917815d450 0x7f9178160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f917d873700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9178160bd0 con 0x7f91780d0440 2023-08-27T11:13:38.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f917d873700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9178160eb0 con 0x7f91780cbaf0 2023-08-27T11:13:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f917d873700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91781f5fc0 con 0x7f917815d450 2023-08-27T11:13:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91767fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91780d0440 0x7f917815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91767fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91780d0440 0x7f917815b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54168/0 (socket says 172.21.15.45:54168) 2023-08-27T11:13:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91767fc700 1 -- 172.21.15.45:0/1685860864 learned_addr learned my addr 172.21.15.45:0/1685860864 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:38.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f9176ffd700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91780cbaf0 0x7f91781f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91777fe700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f917815d450 0x7f9178160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91767fc700 1 -- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f917815d450 msgr2=0x7f9178160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91767fc700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f917815d450 0x7f9178160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91767fc700 1 -- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91780cbaf0 msgr2=0x7f91781f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91767fc700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91780cbaf0 0x7f91781f6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91767fc700 1 -- 172.21.15.45:0/1685860864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9168009550 con 0x7f91780d0440 2023-08-27T11:13:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f91767fc700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91780d0440 0x7f917815b7c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f916c00c4b0 tx=0x7f916c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f915ffff700 1 -- 172.21.15.45:0/1685860864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0052d0 con 0x7f91780d0440 2023-08-27T11:13:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.403+0000 7f915ffff700 1 -- 172.21.15.45:0/1685860864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c005430 con 0x7f91780d0440 2023-08-27T11:13:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.404+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91781f62a0 con 0x7f91780d0440 2023-08-27T11:13:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.404+0000 7f915ffff700 1 -- 172.21.15.45:0/1685860864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0038c0 con 0x7f91780d0440 2023-08-27T11:13:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.404+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917815bd00 con 0x7f91780d0440 2023-08-27T11:13:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.404+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917815be90 con 0x7f91780d0440 2023-08-27T11:13:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.405+0000 7f915ffff700 1 -- 172.21.15.45:0/1685860864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f916c003a20 con 0x7f91780d0440 2023-08-27T11:13:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.405+0000 7f915ffff700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f916406a0e0 0x7f916406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.405+0000 7f915ffff700 1 -- 172.21.15.45:0/1685860864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f916c087a10 con 0x7f91780d0440 2023-08-27T11:13:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.406+0000 7f9176ffd700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f916406a0e0 0x7f916406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.406+0000 7f9176ffd700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f916406a0e0 0x7f916406c5a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9168014040 tx=0x7f9168008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:38.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.410+0000 7f915ffff700 1 -- 172.21.15.45:0/1685860864 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f917815be90 con 0x7f91780d0440 2023-08-27T11:13:38.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.610+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91781f67c0 con 0x7f91780d0440 2023-08-27T11:13:38.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.611+0000 7f915ffff700 1 -- 172.21.15.45:0/1685860864 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18525 (secure 0 0 0) 0x7f91781f67c0 con 0x7f91780d0440 2023-08-27T11:13:38.612 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:38.613 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:13:37.320452+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:13:35.610850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-27T11:13:36.282109+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f916406a0e0 msgr2=0x7f916406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f916406a0e0 0x7f916406c5a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9168014040 tx=0x7f9168008030 comp rx=0 tx=0).stop 2023-08-27T11:13:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91780d0440 msgr2=0x7f917815b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91780d0440 0x7f917815b7c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f916c00c4b0 tx=0x7f916c00c7c0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 shutdown_connections 2023-08-27T11:13:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f916406a0e0 0x7f916406c5a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f917815d450 0x7f9178160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91780cbaf0 0x7f91781f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 --2- 172.21.15.45:0/1685860864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91780d0440 0x7f917815b7c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 >> 172.21.15.45:0/1685860864 conn(0x7f91780d0f60 msgr2=0x7f91780b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 shutdown_connections 2023-08-27T11:13:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.613+0000 7f917d873700 1 -- 172.21.15.45:0/1685860864 wait complete. 2023-08-27T11:13:38.627 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:13:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.754+0000 7f3d7f2ad700 1 Processor -- start 2023-08-27T11:13:38.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.754+0000 7f3d7f2ad700 1 -- start start 2023-08-27T11:13:38.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7f2ad700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3d780cdf90 0x7f3d7806ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7f2ad700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d7806b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7f2ad700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d78161870 con 0x7f3d7815d0a0 2023-08-27T11:13:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7f2ad700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7806d3f0 con 0x7f3d780cdf90 2023-08-27T11:13:38.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7f2ad700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78161da0 con 0x7f3d780d0a50 2023-08-27T11:13:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7d84a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d7806b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7c848700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3d780cdf90 0x7f3d7806ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7d84a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d7806b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:52088/0 (socket says 172.21.15.45:52088) 2023-08-27T11:13:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7d049700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3d7815d0a0 0x7f3d780cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:36614/0 (socket says 172.21.15.45:36614) 2023-08-27T11:13:38.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7c848700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3d780cdf90 0x7f3d7806ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:43032/0 (socket says 172.21.15.45:43032) 2023-08-27T11:13:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.755+0000 7f3d7d84a700 1 -- 172.21.15.45:0/4102024059 learned_addr learned my addr 172.21.15.45:0/4102024059 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.756+0000 7f3d7d84a700 1 -- 172.21.15.45:0/4102024059 >> v2:172.21.15.135:3300/0 conn(0x7f3d780cdf90 msgr2=0x7f3d7806ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.756+0000 7f3d7d84a700 1 --2- 172.21.15.45:0/4102024059 >> v2:172.21.15.135:3300/0 conn(0x7f3d780cdf90 0x7f3d7806ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.756+0000 7f3d7d84a700 1 -- 172.21.15.45:0/4102024059 >> v1:172.21.15.135:6789/0 conn(0x7f3d7815d0a0 legacy=0x7f3d780cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.756+0000 7f3d7d84a700 1 -- 172.21.15.45:0/4102024059 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d78161ac0 con 0x7f3d780d0a50 2023-08-27T11:13:38.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.756+0000 7f3d7c848700 1 --2- 172.21.15.45:0/4102024059 >> v2:172.21.15.135:3300/0 conn(0x7f3d780cdf90 0x7f3d7806ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.756+0000 7f3d7d84a700 1 --2- 172.21.15.45:0/4102024059 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d7806b390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009e50 tx=0x7f3d7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6010183d8110250f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.756+0000 7f3d6ffff700 1 -- 172.21.15.45:0/4102024059 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000e070 con 0x7f3d780d0a50 2023-08-27T11:13:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.756+0000 7f3d6ffff700 1 -- 172.21.15.45:0/4102024059 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7000b910 con 0x7f3d780d0a50 2023-08-27T11:13:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.756+0000 7f3d6ffff700 1 -- 172.21.15.45:0/4102024059 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000bcc0 con 0x7f3d780d0a50 2023-08-27T11:13:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.757+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/4102024059 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 msgr2=0x7f3d7806b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.757+0000 7f3d7f2ad700 1 --2- 172.21.15.45:0/4102024059 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d7806b390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009e50 tx=0x7f3d7000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.757+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/4102024059 shutdown_connections 2023-08-27T11:13:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.757+0000 7f3d7f2ad700 1 --2- 172.21.15.45:0/4102024059 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d7806b390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.757+0000 7f3d7f2ad700 1 --2- 172.21.15.45:0/4102024059 >> v2:172.21.15.135:3300/0 conn(0x7f3d780cdf90 0x7f3d7806ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.757+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/4102024059 >> 172.21.15.45:0/4102024059 conn(0x7f3d7806f960 msgr2=0x7f3d7806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.757+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/4102024059 shutdown_connections 2023-08-27T11:13:38.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.757+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/4102024059 wait complete. 2023-08-27T11:13:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.758+0000 7f3d7f2ad700 1 Processor -- start 2023-08-27T11:13:38.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.758+0000 7f3d7f2ad700 1 -- start start 2023-08-27T11:13:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7f2ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780cdf90 0x7f3d7814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7f2ad700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d780b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7f2ad700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d7815d0a0 0x7f3d780ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7f2ad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7815c1a0 con 0x7f3d780cdf90 2023-08-27T11:13:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7f2ad700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780b3770 con 0x7f3d7815d0a0 2023-08-27T11:13:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7f2ad700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780aff60 con 0x7f3d780d0a50 2023-08-27T11:13:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d049700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780cdf90 0x7f3d7814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d049700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780cdf90 0x7f3d7814e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54180/0 (socket says 172.21.15.45:54180) 2023-08-27T11:13:38.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d049700 1 -- 172.21.15.45:0/946641819 learned_addr learned my addr 172.21.15.45:0/946641819 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7c848700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d780b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d84a700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d7815d0a0 0x7f3d780ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d049700 1 -- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 msgr2=0x7f3d780b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d049700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d780b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d049700 1 -- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d7815d0a0 msgr2=0x7f3d780ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d049700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d7815d0a0 0x7f3d780ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:38.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d049700 1 -- 172.21.15.45:0/946641819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740094a0 con 0x7f3d780cdf90 2023-08-27T11:13:38.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d84a700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d7815d0a0 0x7f3d780ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:38.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.759+0000 7f3d7d049700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780cdf90 0x7f3d7814e470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800c520 tx=0x7f3d6800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:38.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.760+0000 7f3d6dffb700 1 -- 172.21.15.45:0/946641819 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68005cb0 con 0x7f3d780cdf90 2023-08-27T11:13:38.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.760+0000 7f3d6dffb700 1 -- 172.21.15.45:0/946641819 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d68005e10 con 0x7f3d780cdf90 2023-08-27T11:13:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.760+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d70009b30 con 0x7f3d780cdf90 2023-08-27T11:13:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.760+0000 7f3d6dffb700 1 -- 172.21.15.45:0/946641819 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680072d0 con 0x7f3d780cdf90 2023-08-27T11:13:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.760+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d780b0520 con 0x7f3d780cdf90 2023-08-27T11:13:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.761+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7806f4a0 con 0x7f3d780cdf90 2023-08-27T11:13:38.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.761+0000 7f3d6dffb700 1 -- 172.21.15.45:0/946641819 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3d68007430 con 0x7f3d780cdf90 2023-08-27T11:13:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.761+0000 7f3d6dffb700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d5c06a0e0 0x7f3d5c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.761+0000 7f3d6dffb700 1 -- 172.21.15.45:0/946641819 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3d68087a00 con 0x7f3d780cdf90 2023-08-27T11:13:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.762+0000 7f3d7c848700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d5c06a0e0 0x7f3d5c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:38.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.762+0000 7f3d7c848700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d5c06a0e0 0x7f3d5c06c5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d781f65d0 tx=0x7f3d74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:38.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:38.766+0000 7f3d6dffb700 1 -- 172.21.15.45:0/946641819 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3d680530f0 con 0x7f3d780cdf90 2023-08-27T11:13:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.018+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3d780734a0 con 0x7f3d780cdf90 2023-08-27T11:13:39.871 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-08-27T11:13:39.870+0000 7f0cd6d00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:13:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.918+0000 7f3d6dffb700 1 -- 172.21.15.45:0/946641819 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3d680170b0 con 0x7f3d780cdf90 2023-08-27T11:13:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d5c06a0e0 msgr2=0x7f3d5c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d5c06a0e0 0x7f3d5c06c5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d781f65d0 tx=0x7f3d74008040 comp rx=0 tx=0).stop 2023-08-27T11:13:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780cdf90 msgr2=0x7f3d7814e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780cdf90 0x7f3d7814e470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800c520 tx=0x7f3d6800c8e0 comp rx=0 tx=0).stop 2023-08-27T11:13:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 shutdown_connections 2023-08-27T11:13:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d780cdf90 0x7f3d7814e470 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f3d5c06a0e0 0x7f3d5c06c5a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d7815d0a0 0x7f3d780ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 --2- 172.21.15.45:0/946641819 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3d780d0a50 0x7f3d780b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 >> 172.21.15.45:0/946641819 conn(0x7f3d7806f960 msgr2=0x7f3d780b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 shutdown_connections 2023-08-27T11:13:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:39.921+0000 7f3d7f2ad700 1 -- 172.21.15.45:0/946641819 wait complete. 2023-08-27T11:13:39.937 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:13:39.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:13:40.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.072+0000 7f09f63a2700 1 Processor -- start 2023-08-27T11:13:40.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.072+0000 7f09f63a2700 1 -- start start 2023-08-27T11:13:40.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f63a2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09f015b100 0x7f09f00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:40.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f63a2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09f00cb2d0 0x7f09f00cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:40.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f63a2700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f00cd0d0 con 0x7f09f00cdc60 2023-08-27T11:13:40.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f63a2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f00ce0d0 con 0x7f09f015b100 2023-08-27T11:13:40.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f63a2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00cd3b0 con 0x7f09f00cb2d0 2023-08-27T11:13:40.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f53a0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09f015b100 0x7f09f00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:40.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f53a0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09f015b100 0x7f09f00cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54188/0 (socket says 172.21.15.45:54188) 2023-08-27T11:13:40.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f53a0700 1 -- 172.21.15.45:0/935063650 learned_addr learned my addr 172.21.15.45:0/935063650 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:40.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f4b9f700 1 --2- 172.21.15.45:0/935063650 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09f00cb2d0 0x7f09f00cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:40.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f53a0700 1 -- 172.21.15.45:0/935063650 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09f00cb2d0 msgr2=0x7f09f00cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:40.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f53a0700 1 --2- 172.21.15.45:0/935063650 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09f00cb2d0 0x7f09f00cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:40.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f53a0700 1 -- 172.21.15.45:0/935063650 >> v1:172.21.15.135:6789/0 conn(0x7f09f00cdc60 legacy=0x7f09f015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:40.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f53a0700 1 -- 172.21.15.45:0/935063650 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f00ce850 con 0x7f09f015b100 2023-08-27T11:13:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.073+0000 7f09f53a0700 1 --2- 172.21.15.45:0/935063650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f015b100 0x7f09f00cf330 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098f0 tx=0x7f09ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fd36aee62ec6b2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09e7fff700 1 -- 172.21.15.45:0/935063650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00b8a0 con 0x7f09f015b100 2023-08-27T11:13:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09e7fff700 1 -- 172.21.15.45:0/935063650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec00ba00 con 0x7f09f015b100 2023-08-27T11:13:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09e7fff700 1 -- 172.21.15.45:0/935063650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec00bd90 con 0x7f09f015b100 2023-08-27T11:13:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09f63a2700 1 -- 172.21.15.45:0/935063650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f015b100 msgr2=0x7f09f00cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09f63a2700 1 --2- 172.21.15.45:0/935063650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f015b100 0x7f09f00cf330 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098f0 tx=0x7f09ec009c00 comp rx=0 tx=0).stop 2023-08-27T11:13:40.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09f63a2700 1 -- 172.21.15.45:0/935063650 shutdown_connections 2023-08-27T11:13:40.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09f63a2700 1 --2- 172.21.15.45:0/935063650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f015b100 0x7f09f00cf330 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:40.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09f63a2700 1 --2- 172.21.15.45:0/935063650 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09f00cb2d0 0x7f09f00cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:40.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09f63a2700 1 -- 172.21.15.45:0/935063650 >> 172.21.15.45:0/935063650 conn(0x7f09f006ddd0 msgr2=0x7f09f006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:40.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09f63a2700 1 -- 172.21.15.45:0/935063650 shutdown_connections 2023-08-27T11:13:40.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.074+0000 7f09f63a2700 1 -- 172.21.15.45:0/935063650 wait complete. 2023-08-27T11:13:40.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.075+0000 7f09f63a2700 1 Processor -- start 2023-08-27T11:13:40.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.075+0000 7f09f63a2700 1 -- start start 2023-08-27T11:13:40.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.076+0000 7f09f63a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f00cb2d0 0x7f09f00b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:40.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.076+0000 7f09f63a2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09f00cdc60 0x7f09f0157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:40.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.076+0000 7f09f63a2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f015b100 0x7f09f0157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:40.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.076+0000 7f09f63a2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f0158010 con 0x7f09f00cb2d0 2023-08-27T11:13:40.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.076+0000 7f09f63a2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00b60a0 con 0x7f09f015b100 2023-08-27T11:13:40.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.076+0000 7f09f63a2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00b5490 con 0x7f09f00cdc60 2023-08-27T11:13:40.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.076+0000 7f09f53a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f00cb2d0 0x7f09f00b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:40.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.076+0000 7f09f53a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f00cb2d0 0x7f09f00b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54194/0 (socket says 172.21.15.45:54194) 2023-08-27T11:13:40.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.076+0000 7f09f53a0700 1 -- 172.21.15.45:0/2093926616 learned_addr learned my addr 172.21.15.45:0/2093926616 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:40.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09f53a0700 1 -- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09f00cdc60 msgr2=0x7f09f0157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:40.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09f53a0700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09f00cdc60 0x7f09f0157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:40.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09f53a0700 1 -- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f015b100 msgr2=0x7f09f0157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:40.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09f53a0700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f015b100 0x7f09f0157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:40.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09f53a0700 1 -- 172.21.15.45:0/2093926616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec009550 con 0x7f09f00cb2d0 2023-08-27T11:13:40.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09f53a0700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f00cb2d0 0x7f09f00b4ab0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f09ec014040 tx=0x7f09ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09e5ffb700 1 -- 172.21.15.45:0/2093926616 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0036a0 con 0x7f09f00cb2d0 2023-08-27T11:13:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09e5ffb700 1 -- 172.21.15.45:0/2093926616 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec01e070 con 0x7f09f00cb2d0 2023-08-27T11:13:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f00b5770 con 0x7f09f00cb2d0 2023-08-27T11:13:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09e5ffb700 1 -- 172.21.15.45:0/2093926616 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec003a60 con 0x7f09f00cb2d0 2023-08-27T11:13:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.077+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f00b5d30 con 0x7f09f00cb2d0 2023-08-27T11:13:40.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.078+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f015f590 con 0x7f09f00cb2d0 2023-08-27T11:13:40.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.078+0000 7f09e5ffb700 1 -- 172.21.15.45:0/2093926616 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f09ec003bc0 con 0x7f09f00cb2d0 2023-08-27T11:13:40.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.079+0000 7f09e5ffb700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f09d406a0e0 0x7f09d406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:40.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.079+0000 7f09e5ffb700 1 -- 172.21.15.45:0/2093926616 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f09ec020070 con 0x7f09f00cb2d0 2023-08-27T11:13:40.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.079+0000 7f09f4b9f700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f09d406a0e0 0x7f09d406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:40.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.080+0000 7f09f4b9f700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f09d406a0e0 0x7f09d406c5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f09f01ee7a0 tx=0x7f09e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:40.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.084+0000 7f09e5ffb700 1 -- 172.21.15.45:0/2093926616 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f09ec0551e0 con 0x7f09f00cb2d0 2023-08-27T11:13:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.336+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f09f00b3690 con 0x7f09f00cb2d0 2023-08-27T11:13:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.926+0000 7f09e5ffb700 1 -- 172.21.15.45:0/2093926616 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f09ec058800 con 0x7f09f00cb2d0 2023-08-27T11:13:40.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f09d406a0e0 msgr2=0x7f09d406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:40.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f09d406a0e0 0x7f09d406c5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f09f01ee7a0 tx=0x7f09e0008040 comp rx=0 tx=0).stop 2023-08-27T11:13:40.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f00cb2d0 msgr2=0x7f09f00b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:40.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f00cb2d0 0x7f09f00b4ab0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f09ec014040 tx=0x7f09ec004750 comp rx=0 tx=0).stop 2023-08-27T11:13:40.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 shutdown_connections 2023-08-27T11:13:40.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09f00cb2d0 0x7f09f00b4ab0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:40.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f09d406a0e0 0x7f09d406c5a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:40.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09f015b100 0x7f09f0157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:40.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 --2- 172.21.15.45:0/2093926616 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f09f00cdc60 0x7f09f0157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:40.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.929+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 >> 172.21.15.45:0/2093926616 conn(0x7f09f006ddd0 msgr2=0x7f09f00b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:40.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.930+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 shutdown_connections 2023-08-27T11:13:40.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:40.930+0000 7f09f63a2700 1 -- 172.21.15.45:0/2093926616 wait complete. 2023-08-27T11:13:40.945 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:13:40.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:13:41.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.078+0000 7f0c4c118700 1 Processor -- start 2023-08-27T11:13:41.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.078+0000 7f0c4c118700 1 -- start start 2023-08-27T11:13:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4c118700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 0x7f0c440cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4c118700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c4406e980 0x7f0c440ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4c118700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4415dad0 con 0x7f0c440cdf60 2023-08-27T11:13:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4c118700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4415d8e0 con 0x7f0c4406e980 2023-08-27T11:13:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4c118700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4415ddb0 con 0x7f0c4415b070 2023-08-27T11:13:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4a6b5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c4406e980 0x7f0c440ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4a6b5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c4406e980 0x7f0c440ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54206/0 (socket says 172.21.15.45:54206) 2023-08-27T11:13:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4a6b5700 1 -- 172.21.15.45:0/3052793811 learned_addr learned my addr 172.21.15.45:0/3052793811 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c49eb4700 1 --2- 172.21.15.45:0/3052793811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 0x7f0c440cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4a6b5700 1 -- 172.21.15.45:0/3052793811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 msgr2=0x7f0c440cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4a6b5700 1 --2- 172.21.15.45:0/3052793811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 0x7f0c440cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4a6b5700 1 -- 172.21.15.45:0/3052793811 >> v1:172.21.15.135:6789/0 conn(0x7f0c440cdf60 legacy=0x7f0c4406e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.079+0000 7f0c4a6b5700 1 -- 172.21.15.45:0/3052793811 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4415e090 con 0x7f0c4406e980 2023-08-27T11:13:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c4a6b5700 1 --2- 172.21.15.45:0/3052793811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c4406e980 0x7f0c440ae8d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009e50 tx=0x7f0c3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a310e992ca6c304d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c48eb2700 1 -- 172.21.15.45:0/3052793811 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c00e070 con 0x7f0c4406e980 2023-08-27T11:13:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c48eb2700 1 -- 172.21.15.45:0/3052793811 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c003a20 con 0x7f0c4406e980 2023-08-27T11:13:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c48eb2700 1 -- 172.21.15.45:0/3052793811 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c003dd0 con 0x7f0c4406e980 2023-08-27T11:13:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c4c118700 1 -- 172.21.15.45:0/3052793811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c4406e980 msgr2=0x7f0c440ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c4c118700 1 --2- 172.21.15.45:0/3052793811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c4406e980 0x7f0c440ae8d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009e50 tx=0x7f0c3c00b330 comp rx=0 tx=0).stop 2023-08-27T11:13:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c4c118700 1 -- 172.21.15.45:0/3052793811 shutdown_connections 2023-08-27T11:13:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c4c118700 1 --2- 172.21.15.45:0/3052793811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 0x7f0c440cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c4c118700 1 --2- 172.21.15.45:0/3052793811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c4406e980 0x7f0c440ae8d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c4c118700 1 -- 172.21.15.45:0/3052793811 >> 172.21.15.45:0/3052793811 conn(0x7f0c440cfe50 msgr2=0x7f0c440d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c4c118700 1 -- 172.21.15.45:0/3052793811 shutdown_connections 2023-08-27T11:13:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.080+0000 7f0c4c118700 1 -- 172.21.15.45:0/3052793811 wait complete. 2023-08-27T11:13:41.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.082+0000 7f0c4c118700 1 Processor -- start 2023-08-27T11:13:41.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.082+0000 7f0c4c118700 1 -- start start 2023-08-27T11:13:41.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c4c118700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c4406e980 0x7f0c441f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c4c118700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c440cdf60 0x7f0c44142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c4c118700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 0x7f0c44155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c4c118700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440b5c50 con 0x7f0c440cdf60 2023-08-27T11:13:41.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c4c118700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c441f7180 con 0x7f0c4406e980 2023-08-27T11:13:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c4c118700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c441f6bc0 con 0x7f0c4415b070 2023-08-27T11:13:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c496b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c440cdf60 0x7f0c44142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c496b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c440cdf60 0x7f0c44142660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54210/0 (socket says 172.21.15.45:54210) 2023-08-27T11:13:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c496b3700 1 -- 172.21.15.45:0/3911810134 learned_addr learned my addr 172.21.15.45:0/3911810134 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c49eb4700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c4406e980 0x7f0c441f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.083+0000 7f0c4a6b5700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 0x7f0c44155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c496b3700 1 -- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 msgr2=0x7f0c44155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c496b3700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 0x7f0c44155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c496b3700 1 -- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c4406e980 msgr2=0x7f0c441f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c496b3700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c4406e980 0x7f0c441f6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c496b3700 1 -- 172.21.15.45:0/3911810134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c009b30 con 0x7f0c440cdf60 2023-08-27T11:13:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c496b3700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c440cdf60 0x7f0c44142660 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000b4a0 tx=0x7f0c4000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c3a7fc700 1 -- 172.21.15.45:0/3911810134 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c40011830 con 0x7f0c440cdf60 2023-08-27T11:13:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c3a7fc700 1 -- 172.21.15.45:0/3911810134 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c40011990 con 0x7f0c440cdf60 2023-08-27T11:13:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c441f6ea0 con 0x7f0c440cdf60 2023-08-27T11:13:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c3a7fc700 1 -- 172.21.15.45:0/3911810134 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c40011d40 con 0x7f0c440cdf60 2023-08-27T11:13:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.084+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c44142140 con 0x7f0c440cdf60 2023-08-27T11:13:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.085+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c44142330 con 0x7f0c440cdf60 2023-08-27T11:13:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.085+0000 7f0c3a7fc700 1 -- 172.21.15.45:0/3911810134 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0c4001a400 con 0x7f0c440cdf60 2023-08-27T11:13:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.086+0000 7f0c3a7fc700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0c2806a0e0 0x7f0c2806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.086+0000 7f0c3a7fc700 1 -- 172.21.15.45:0/3911810134 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0c400881f0 con 0x7f0c440cdf60 2023-08-27T11:13:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.086+0000 7f0c49eb4700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0c2806a0e0 0x7f0c2806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.086+0000 7f0c49eb4700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0c2806a0e0 0x7f0c2806c5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c440d0880 tx=0x7f0c34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.090+0000 7f0c3a7fc700 1 -- 172.21.15.45:0/3911810134 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0c40053850 con 0x7f0c440cdf60 2023-08-27T11:13:41.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.333+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c440cf8d0 con 0x7f0c440cdf60 2023-08-27T11:13:41.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.334+0000 7f0c3a7fc700 1 -- 172.21.15.45:0/3911810134 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f0c40056e70 con 0x7f0c440cdf60 2023-08-27T11:13:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1324661970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1324661970},{"type":"v1","addr":"172.21.15.135:6836","nonce":1324661970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2014390806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2014390806},{"type":"v1","addr":"172.21.15.135:6837","nonce":2014390806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2103056681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2103056681},{"type":"v1","addr":"172.21.15.45:6834","nonce":2103056681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:13:37.320852+0000","modified":"2023-08-27T11:13:40.917661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/2587151358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2587151358},{"type":"v1","addr":"172.21.15.45:6835","nonce":2587151358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:13:41.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0c2806a0e0 msgr2=0x7f0c2806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0c2806a0e0 0x7f0c2806c5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c440d0880 tx=0x7f0c34008040 comp rx=0 tx=0).stop 2023-08-27T11:13:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c440cdf60 msgr2=0x7f0c44142660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c440cdf60 0x7f0c44142660 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000b4a0 tx=0x7f0c4000b7b0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 shutdown_connections 2023-08-27T11:13:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f0c2806a0e0 0x7f0c2806c5a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c4415b070 0x7f0c44155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c4406e980 0x7f0c441f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 --2- 172.21.15.45:0/3911810134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c440cdf60 0x7f0c44142660 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 >> 172.21.15.45:0/3911810134 conn(0x7f0c440cfe50 msgr2=0x7f0c44159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:41.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 shutdown_connections 2023-08-27T11:13:41.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.336+0000 7f0c4c118700 1 -- 172.21.15.45:0/3911810134 wait complete. 2023-08-27T11:13:41.341 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2023-08-27T11:13:41.351 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:13:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.474+0000 7f597d1f7700 1 Processor -- start 2023-08-27T11:13:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.474+0000 7f597d1f7700 1 -- start start 2023-08-27T11:13:41.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f597d1f7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f597815a240 0x7f59780cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f597d1f7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597806d690 con 0x7f59780cbe70 2023-08-27T11:13:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f597d1f7700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59780cc440 con 0x7f59780ceba0 2023-08-27T11:13:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f597d1f7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597806d040 con 0x7f597815a240 2023-08-27T11:13:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f597659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f59780cbe70 0x7f59780ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40994/0 (socket says 172.21.15.45:40994) 2023-08-27T11:13:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f597659c700 1 -- 172.21.15.45:0/2232820720 learned_addr learned my addr 172.21.15.45:0/2232820720 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5976d9d700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f597815a240 0x7f59780cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531497666 0 0) 0x7f597806d690 con 0x7f59780cbe70 2023-08-27T11:13:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59600035b0 con 0x7f59780cbe70 2023-08-27T11:13:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4185720840 0 0) 0x7f59600035b0 con 0x7f59780cbe70 2023-08-27T11:13:41.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f597815a240 msgr2=0x7f59780cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:41.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f597815a240 0x7f59780cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 >> v1:172.21.15.135:6789/0 conn(0x7f59780ceba0 legacy=0x7f597806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597806d320 con 0x7f59780cbe70 2023-08-27T11:13:41.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f5968001670 con 0x7f59780cbe70 2023-08-27T11:13:41.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 >> v1:172.21.15.45:6789/0 conn(0x7f59780cbe70 legacy=0x7f59780ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5960003a30 0x7f5960003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5960004850 0x7f5960004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f597815a240 0x7f5960005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597806d040 con 0x7f5960003a30 2023-08-27T11:13:41.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596000bac0 con 0x7f5960004850 2023-08-27T11:13:41.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.475+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596000bc80 con 0x7f597815a240 2023-08-27T11:13:41.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5976d9d700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5960003a30 0x7f5960003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f597659c700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5960004850 0x7f5960004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f597759e700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f597815a240 0x7f5960005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5976d9d700 1 -- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f597815a240 msgr2=0x7f5960005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5976d9d700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f597815a240 0x7f5960005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5976d9d700 1 -- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5960004850 msgr2=0x7f5960004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:41.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5976d9d700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5960004850 0x7f5960004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5976d9d700 1 -- 172.21.15.45:0/2232820720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59600035b0 con 0x7f5960003a30 2023-08-27T11:13:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5976d9d700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5960003a30 0x7f5960003e50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f596c009b30 tx=0x7f5960005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450479493d5cb77e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c00e6c0 con 0x7f5960003a30 2023-08-27T11:13:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c00e820 con 0x7f5960003a30 2023-08-27T11:13:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.476+0000 7f5975d9b700 1 -- 172.21.15.45:0/2232820720 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c00eb90 con 0x7f5960003a30 2023-08-27T11:13:41.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.477+0000 7f597d1f7700 1 -- 172.21.15.45:0/2232820720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5960003a30 msgr2=0x7f5960003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.477+0000 7f597d1f7700 1 --2- 172.21.15.45:0/2232820720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5960003a30 0x7f5960003e50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f596c009b30 tx=0x7f5960005880 comp rx=0 tx=0).stop 2023-08-27T11:13:41.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.477+0000 7f597759e700 1 -- 172.21.15.45:0/2232820720 reap_dead start 2023-08-27T11:13:41.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.477+0000 7f597d1f7700 1 -- 172.21.15.45:0/2232820720 shutdown_connections 2023-08-27T11:13:41.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.477+0000 7f597d1f7700 1 -- 172.21.15.45:0/2232820720 >> 172.21.15.45:0/2232820720 conn(0x7f59780cfe50 msgr2=0x7f59780d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:41.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.477+0000 7f597d1f7700 1 -- 172.21.15.45:0/2232820720 shutdown_connections 2023-08-27T11:13:41.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.477+0000 7f597d1f7700 1 -- 172.21.15.45:0/2232820720 wait complete. 2023-08-27T11:13:41.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.478+0000 7f597d1f7700 1 Processor -- start 2023-08-27T11:13:41.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.478+0000 7f597d1f7700 1 -- start start 2023-08-27T11:13:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597d1f7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59780c2d70 0x7f59781f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597d1f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59780c8d20 0x7f59780c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597d1f7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59780c6090 0x7f59781580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597d1f7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780c6510 con 0x7f59780c8d20 2023-08-27T11:13:41.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597d1f7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780c9f30 con 0x7f59780c6090 2023-08-27T11:13:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597d1f7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59781f0b10 con 0x7f59780c2d70 2023-08-27T11:13:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59780c8d20 0x7f59780c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59780c8d20 0x7f59780c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54230/0 (socket says 172.21.15.45:54230) 2023-08-27T11:13:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597659c700 1 -- 172.21.15.45:0/2658249258 learned_addr learned my addr 172.21.15.45:0/2658249258 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f5976d9d700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59780c2d70 0x7f59781f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.479+0000 7f597759e700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59780c6090 0x7f59781580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f597659c700 1 -- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59780c2d70 msgr2=0x7f59781f05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f597659c700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59780c2d70 0x7f59781f05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f597659c700 1 -- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59780c6090 msgr2=0x7f59781580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f597659c700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59780c6090 0x7f59781580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f597659c700 1 -- 172.21.15.45:0/2658249258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780c71c0 con 0x7f59780c8d20 2023-08-27T11:13:41.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f597659c700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59780c8d20 0x7f59780c5b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f59680072f0 tx=0x7f5968007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f595f7fe700 1 -- 172.21.15.45:0/2658249258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800f600 con 0x7f59780c8d20 2023-08-27T11:13:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f595f7fe700 1 -- 172.21.15.45:0/2658249258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596800f760 con 0x7f59780c8d20 2023-08-27T11:13:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59780c92b0 con 0x7f59780c8d20 2023-08-27T11:13:41.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f595f7fe700 1 -- 172.21.15.45:0/2658249258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800fb10 con 0x7f59780c8d20 2023-08-27T11:13:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.480+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59780c9870 con 0x7f59780c8d20 2023-08-27T11:13:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.481+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59781f0cd0 con 0x7f59780c8d20 2023-08-27T11:13:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.482+0000 7f595f7fe700 1 -- 172.21.15.45:0/2658249258 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f596800fc70 con 0x7f59780c8d20 2023-08-27T11:13:41.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.482+0000 7f595f7fe700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f596406a0e0 0x7f596406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.482+0000 7f595f7fe700 1 -- 172.21.15.45:0/2658249258 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5968087ff0 con 0x7f59780c8d20 2023-08-27T11:13:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.482+0000 7f5976d9d700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f596406a0e0 0x7f596406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.483+0000 7f5976d9d700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f596406a0e0 0x7f596406c5a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59780748a0 tx=0x7f596c01f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:41.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.486+0000 7f595f7fe700 1 -- 172.21.15.45:0/2658249258 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59781f0cd0 con 0x7f59780c8d20 2023-08-27T11:13:41.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.686+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59781f0cd0 con 0x7f59780c8d20 2023-08-27T11:13:41.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.687+0000 7f595f7fe700 1 -- 172.21.15.45:0/2658249258 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18512 (secure 0 0 0) 0x7f59781f0cd0 con 0x7f59780c8d20 2023-08-27T11:13:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:41.690 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":42,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:13:38.862299+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:13:35.610850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-27T11:13:36.282109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:13:41.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f596406a0e0 msgr2=0x7f596406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f596406a0e0 0x7f596406c5a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59780748a0 tx=0x7f596c01f040 comp rx=0 tx=0).stop 2023-08-27T11:13:41.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59780c8d20 msgr2=0x7f59780c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59780c8d20 0x7f59780c5b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f59680072f0 tx=0x7f5968007600 comp rx=0 tx=0).stop 2023-08-27T11:13:41.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 shutdown_connections 2023-08-27T11:13:41.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f596406a0e0 0x7f596406c5a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59780c6090 0x7f59781580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59780c2d70 0x7f59781f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 --2- 172.21.15.45:0/2658249258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59780c8d20 0x7f59780c5b50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 >> 172.21.15.45:0/2658249258 conn(0x7f59780cfe50 msgr2=0x7f59780ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:41.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 shutdown_connections 2023-08-27T11:13:41.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.689+0000 7f597d1f7700 1 -- 172.21.15.45:0/2658249258 wait complete. 2023-08-27T11:13:41.703 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:13:41.704 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:13:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.826+0000 7fe5c7430700 1 Processor -- start 2023-08-27T11:13:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.827+0000 7fe5c7430700 1 -- start start 2023-08-27T11:13:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c7430700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe5c0070e80 0x7fe5c006e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c7430700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c006a990 0x7fe5c015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c7430700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c00cbf00 con 0x7fe5c0155660 2023-08-27T11:13:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c7430700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c015c760 con 0x7fe5c0070e80 2023-08-27T11:13:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c7430700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00cc180 con 0x7fe5c006a990 2023-08-27T11:13:41.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c51cc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe5c0155660 0x7fe5c006a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41012/0 (socket says 172.21.15.45:41012) 2023-08-27T11:13:41.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c51cc700 1 -- 172.21.15.45:0/4035423393 learned_addr learned my addr 172.21.15.45:0/4035423393 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:41.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c59cd700 1 --2- 172.21.15.45:0/4035423393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c006a990 0x7fe5c015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c49cb700 1 --2- 172.21.15.45:0/4035423393 >> v2:172.21.15.135:3300/0 conn(0x7fe5c0070e80 0x7fe5c006e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5b7fff700 1 -- 172.21.15.45:0/4035423393 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841452181 0 0) 0x7fe5c00cbf00 con 0x7fe5c0155660 2023-08-27T11:13:41.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5b7fff700 1 -- 172.21.15.45:0/4035423393 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5ac0035b0 con 0x7fe5c0155660 2023-08-27T11:13:41.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c49cb700 1 -- 172.21.15.45:0/4035423393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c006a990 msgr2=0x7fe5c015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:41.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c49cb700 1 --2- 172.21.15.45:0/4035423393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c006a990 0x7fe5c015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c49cb700 1 -- 172.21.15.45:0/4035423393 >> v1:172.21.15.45:6789/0 conn(0x7fe5c0155660 legacy=0x7fe5c006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c49cb700 1 -- 172.21.15.45:0/4035423393 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c00caa60 con 0x7fe5c0070e80 2023-08-27T11:13:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c59cd700 1 --2- 172.21.15.45:0/4035423393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c006a990 0x7fe5c015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.828+0000 7fe5c49cb700 1 --2- 172.21.15.45:0/4035423393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5c0070e80 0x7fe5c006e850 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009840 tx=0x7fe5b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d844a793a5a138d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5b7fff700 1 -- 172.21.15.45:0/4035423393 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0011830 con 0x7fe5c0070e80 2023-08-27T11:13:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5b7fff700 1 -- 172.21.15.45:0/4035423393 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b0011990 con 0x7fe5c0070e80 2023-08-27T11:13:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5b7fff700 1 -- 172.21.15.45:0/4035423393 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0011d20 con 0x7fe5c0070e80 2023-08-27T11:13:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5c7430700 1 -- 172.21.15.45:0/4035423393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5c0070e80 msgr2=0x7fe5c006e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5c7430700 1 --2- 172.21.15.45:0/4035423393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5c0070e80 0x7fe5c006e850 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009840 tx=0x7fe5b0009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5c7430700 1 -- 172.21.15.45:0/4035423393 shutdown_connections 2023-08-27T11:13:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5c7430700 1 --2- 172.21.15.45:0/4035423393 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c006a990 0x7fe5c015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5c7430700 1 --2- 172.21.15.45:0/4035423393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5c0070e80 0x7fe5c006e850 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5c7430700 1 -- 172.21.15.45:0/4035423393 >> 172.21.15.45:0/4035423393 conn(0x7fe5c00cfe50 msgr2=0x7fe5c00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:41.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5c7430700 1 -- 172.21.15.45:0/4035423393 shutdown_connections 2023-08-27T11:13:41.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.829+0000 7fe5c7430700 1 -- 172.21.15.45:0/4035423393 wait complete. 2023-08-27T11:13:41.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.830+0000 7fe5c7430700 1 Processor -- start 2023-08-27T11:13:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c7430700 1 -- start start 2023-08-27T11:13:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c7430700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c006a990 0x7fe5c00cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c7430700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c0070e80 0x7fe5c014b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c7430700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5c0155660 0x7fe5c00ced40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c7430700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00a8e40 con 0x7fe5c006a990 2023-08-27T11:13:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c7430700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00ce8b0 con 0x7fe5c0155660 2023-08-27T11:13:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c7430700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c014b850 con 0x7fe5c0070e80 2023-08-27T11:13:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c51cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c006a990 0x7fe5c00cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c51cc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c006a990 0x7fe5c00cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54242/0 (socket says 172.21.15.45:54242) 2023-08-27T11:13:41.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.831+0000 7fe5c51cc700 1 -- 172.21.15.45:0/3756981097 learned_addr learned my addr 172.21.15.45:0/3756981097 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:41.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c59cd700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5c0155660 0x7fe5c00ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c49cb700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c0070e80 0x7fe5c014b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c51cc700 1 -- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c0070e80 msgr2=0x7fe5c014b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c51cc700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c0070e80 0x7fe5c014b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c51cc700 1 -- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5c0155660 msgr2=0x7fe5c00ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c51cc700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5c0155660 0x7fe5c00ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c51cc700 1 -- 172.21.15.45:0/3756981097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b8009a70 con 0x7fe5c006a990 2023-08-27T11:13:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c51cc700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c006a990 0x7fe5c00cf5c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c4d0 tx=0x7fe5bc00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5b5ffb700 1 -- 172.21.15.45:0/3756981097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc005d70 con 0x7fe5c006a990 2023-08-27T11:13:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5b5ffb700 1 -- 172.21.15.45:0/3756981097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc00ce90 con 0x7fe5c006a990 2023-08-27T11:13:41.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b00094a0 con 0x7fe5c006a990 2023-08-27T11:13:41.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5b5ffb700 1 -- 172.21.15.45:0/3756981097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0072f0 con 0x7fe5c006a990 2023-08-27T11:13:41.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.832+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c00ce540 con 0x7fe5c006a990 2023-08-27T11:13:41.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.833+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c0072810 con 0x7fe5c006a990 2023-08-27T11:13:41.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.834+0000 7fe5b5ffb700 1 -- 172.21.15.45:0/3756981097 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe5bc007450 con 0x7fe5c006a990 2023-08-27T11:13:41.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.834+0000 7fe5b5ffb700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5a406a0e0 0x7fe5a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:41.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.834+0000 7fe5b5ffb700 1 -- 172.21.15.45:0/3756981097 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe5bc0879e0 con 0x7fe5c006a990 2023-08-27T11:13:41.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.834+0000 7fe5c49cb700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5a406a0e0 0x7fe5a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:41.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.835+0000 7fe5c49cb700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5a406a0e0 0x7fe5a406c5a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009840 tx=0x7fe5b0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:41.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:41.838+0000 7fe5b5ffb700 1 -- 172.21.15.45:0/3756981097 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe5bc053020 con 0x7fe5c006a990 2023-08-27T11:13:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.035+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 --> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5c00ca780 con 0x7fe5a406a0e0 2023-08-27T11:13:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.045+0000 7fe5b5ffb700 1 -- 172.21.15.45:0/3756981097 <== mgr.4100 v2:172.21.15.135:6832/4120550306 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311505 (secure 0 0 0) 0x7fe5b0011630 con 0x7fe5a406a0e0 2023-08-27T11:13:42.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:42.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5a406a0e0 msgr2=0x7fe5a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5a406a0e0 0x7fe5a406c5a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009840 tx=0x7fe5b0008030 comp rx=0 tx=0).stop 2023-08-27T11:13:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c006a990 msgr2=0x7fe5c00cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c006a990 0x7fe5c00cf5c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c4d0 tx=0x7fe5bc00c7e0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 shutdown_connections 2023-08-27T11:13:42.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c006a990 0x7fe5c00cf5c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fe5a406a0e0 0x7fe5a406c5a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5c0155660 0x7fe5c00ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 --2- 172.21.15.45:0/3756981097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe5c0070e80 0x7fe5c014b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.048+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 >> 172.21.15.45:0/3756981097 conn(0x7fe5c00cfe50 msgr2=0x7fe5c00b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:42.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.049+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 shutdown_connections 2023-08-27T11:13:42.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.049+0000 7fe5c7430700 1 -- 172.21.15.45:0/3756981097 wait complete. 2023-08-27T11:13:42.056 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:13:42.063 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2023-08-27T11:13:40.828703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221184,"kb_used_data":5020,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753536,"statfs":{"total":773094113280,"available":772867620864,"internally_reserved":0,"allocated":5140480,"data_stored":2928791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.943092"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931436+0000","last_change":"2023-08-27T11:13:37.882532+0000","last_active":"2023-08-27T11:13:38.931436+0000","last_peered":"2023-08-27T11:13:38.931436+0000","last_clean":"2023-08-27T11:13:38.931436+0000","last_became_active":"2023-08-27T11:13:37.882284+0000","last_became_peered":"2023-08-27T11:13:37.882284+0000","last_unstale":"2023-08-27T11:13:38.931436+0000","last_undegraded":"2023-08-27T11:13:38.931436+0000","last_fullsized":"2023-08-27T11:13:38.931436+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:38:47.084902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869485+0000","last_change":"2023-08-27T11:13:37.889572+0000","last_active":"2023-08-27T11:13:38.869485+0000","last_peered":"2023-08-27T11:13:38.869485+0000","last_clean":"2023-08-27T11:13:38.869485+0000","last_became_active":"2023-08-27T11:13:37.889256+0000","last_became_peered":"2023-08-27T11:13:37.889256+0000","last_unstale":"2023-08-27T11:13:38.869485+0000","last_undegraded":"2023-08-27T11:13:38.869485+0000","last_fullsized":"2023-08-27T11:13:38.869485+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:26:31.927729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869746+0000","last_change":"2023-08-27T11:13:37.880481+0000","last_active":"2023-08-27T11:13:38.869746+0000","last_peered":"2023-08-27T11:13:38.869746+0000","last_clean":"2023-08-27T11:13:38.869746+0000","last_became_active":"2023-08-27T11:13:37.880206+0000","last_became_peered":"2023-08-27T11:13:37.880206+0000","last_unstale":"2023-08-27T11:13:38.869746+0000","last_undegraded":"2023-08-27T11:13:38.869746+0000","last_fullsized":"2023-08-27T11:13:38.869746+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:37:04.899615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932238+0000","last_change":"2023-08-27T11:13:37.885826+0000","last_active":"2023-08-27T11:13:38.932238+0000","last_peered":"2023-08-27T11:13:38.932238+0000","last_clean":"2023-08-27T11:13:38.932238+0000","last_became_active":"2023-08-27T11:13:37.885537+0000","last_became_peered":"2023-08-27T11:13:37.885537+0000","last_unstale":"2023-08-27T11:13:38.932238+0000","last_undegraded":"2023-08-27T11:13:38.932238+0000","last_fullsized":"2023-08-27T11:13:38.932238+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:19:39.179917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869272+0000","last_change":"2023-08-27T11:13:37.883335+0000","last_active":"2023-08-27T11:13:38.869272+0000","last_peered":"2023-08-27T11:13:38.869272+0000","last_clean":"2023-08-27T11:13:38.869272+0000","last_became_active":"2023-08-27T11:13:37.883068+0000","last_became_peered":"2023-08-27T11:13:37.883068+0000","last_unstale":"2023-08-27T11:13:38.869272+0000","last_undegraded":"2023-08-27T11:13:38.869272+0000","last_fullsized":"2023-08-27T11:13:38.869272+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:55.062646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931276+0000","last_change":"2023-08-27T11:13:37.874972+0000","last_active":"2023-08-27T11:13:38.931276+0000","last_peered":"2023-08-27T11:13:38.931276+0000","last_clean":"2023-08-27T11:13:38.931276+0000","last_became_active":"2023-08-27T11:13:37.874482+0000","last_became_peered":"2023-08-27T11:13:37.874482+0000","last_unstale":"2023-08-27T11:13:38.931276+0000","last_undegraded":"2023-08-27T11:13:38.931276+0000","last_fullsized":"2023-08-27T11:13:38.931276+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:57.638778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931815+0000","last_change":"2023-08-27T11:13:37.879022+0000","last_active":"2023-08-27T11:13:38.931815+0000","last_peered":"2023-08-27T11:13:38.931815+0000","last_clean":"2023-08-27T11:13:38.931815+0000","last_became_active":"2023-08-27T11:13:37.878741+0000","last_became_peered":"2023-08-27T11:13:37.878741+0000","last_unstale":"2023-08-27T11:13:38.931815+0000","last_undegraded":"2023-08-27T11:13:38.931815+0000","last_fullsized":"2023-08-27T11:13:38.931815+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:40.235315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930715+0000","last_change":"2023-08-27T11:13:37.887341+0000","last_active":"2023-08-27T11:13:38.930715+0000","last_peered":"2023-08-27T11:13:38.930715+0000","last_clean":"2023-08-27T11:13:38.930715+0000","last_became_active":"2023-08-27T11:13:37.886843+0000","last_became_peered":"2023-08-27T11:13:37.886843+0000","last_unstale":"2023-08-27T11:13:38.930715+0000","last_undegraded":"2023-08-27T11:13:38.930715+0000","last_fullsized":"2023-08-27T11:13:38.930715+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:08:31.228633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932614+0000","last_change":"2023-08-27T11:13:37.881602+0000","last_active":"2023-08-27T11:13:38.932614+0000","last_peered":"2023-08-27T11:13:38.932614+0000","last_clean":"2023-08-27T11:13:38.932614+0000","last_became_active":"2023-08-27T11:13:37.881263+0000","last_became_peered":"2023-08-27T11:13:37.881263+0000","last_unstale":"2023-08-27T11:13:38.932614+0000","last_undegraded":"2023-08-27T11:13:38.932614+0000","last_fullsized":"2023-08-27T11:13:38.932614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:55:11.171065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932575+0000","last_change":"2023-08-27T11:13:37.881923+0000","last_active":"2023-08-27T11:13:38.932575+0000","last_peered":"2023-08-27T11:13:38.932575+0000","last_clean":"2023-08-27T11:13:38.932575+0000","last_became_active":"2023-08-27T11:13:37.880069+0000","last_became_peered":"2023-08-27T11:13:37.880069+0000","last_unstale":"2023-08-27T11:13:38.932575+0000","last_undegraded":"2023-08-27T11:13:38.932575+0000","last_fullsized":"2023-08-27T11:13:38.932575+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:26:38.075458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932287+0000","last_change":"2023-08-27T11:13:37.882289+0000","last_active":"2023-08-27T11:13:38.932287+0000","last_peered":"2023-08-27T11:13:38.932287+0000","last_clean":"2023-08-27T11:13:38.932287+0000","last_became_active":"2023-08-27T11:13:37.880993+0000","last_became_peered":"2023-08-27T11:13:37.880993+0000","last_unstale":"2023-08-27T11:13:38.932287+0000","last_undegraded":"2023-08-27T11:13:38.932287+0000","last_fullsized":"2023-08-27T11:13:38.932287+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:49:09.883091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869719+0000","last_change":"2023-08-27T11:13:37.889837+0000","last_active":"2023-08-27T11:13:38.869719+0000","last_peered":"2023-08-27T11:13:38.869719+0000","last_clean":"2023-08-27T11:13:38.869719+0000","last_became_active":"2023-08-27T11:13:37.889636+0000","last_became_peered":"2023-08-27T11:13:37.889636+0000","last_unstale":"2023-08-27T11:13:38.869719+0000","last_undegraded":"2023-08-27T11:13:38.869719+0000","last_fullsized":"2023-08-27T11:13:38.869719+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:59.902099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932305+0000","last_change":"2023-08-27T11:13:37.887145+0000","last_active":"2023-08-27T11:13:38.932305+0000","last_peered":"2023-08-27T11:13:38.932305+0000","last_clean":"2023-08-27T11:13:38.932305+0000","last_became_active":"2023-08-27T11:13:37.886823+0000","last_became_peered":"2023-08-27T11:13:37.886823+0000","last_unstale":"2023-08-27T11:13:38.932305+0000","last_undegraded":"2023-08-27T11:13:38.932305+0000","last_fullsized":"2023-08-27T11:13:38.932305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:37.476696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932242+0000","last_change":"2023-08-27T11:13:37.881949+0000","last_active":"2023-08-27T11:13:38.932242+0000","last_peered":"2023-08-27T11:13:38.932242+0000","last_clean":"2023-08-27T11:13:38.932242+0000","last_became_active":"2023-08-27T11:13:37.881657+0000","last_became_peered":"2023-08-27T11:13:37.881657+0000","last_unstale":"2023-08-27T11:13:38.932242+0000","last_undegraded":"2023-08-27T11:13:38.932242+0000","last_fullsized":"2023-08-27T11:13:38.932242+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:13.749860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931370+0000","last_change":"2023-08-27T11:13:37.876910+0000","last_active":"2023-08-27T11:13:38.931370+0000","last_peered":"2023-08-27T11:13:38.931370+0000","last_clean":"2023-08-27T11:13:38.931370+0000","last_became_active":"2023-08-27T11:13:37.876559+0000","last_became_peered":"2023-08-27T11:13:37.876559+0000","last_unstale":"2023-08-27T11:13:38.931370+0000","last_undegraded":"2023-08-27T11:13:38.931370+0000","last_fullsized":"2023-08-27T11:13:38.931370+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:37:35.119222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931631+0000","last_change":"2023-08-27T11:13:37.883263+0000","last_active":"2023-08-27T11:13:38.931631+0000","last_peered":"2023-08-27T11:13:38.931631+0000","last_clean":"2023-08-27T11:13:38.931631+0000","last_became_active":"2023-08-27T11:13:37.882564+0000","last_became_peered":"2023-08-27T11:13:37.882564+0000","last_unstale":"2023-08-27T11:13:38.931631+0000","last_undegraded":"2023-08-27T11:13:38.931631+0000","last_fullsized":"2023-08-27T11:13:38.931631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:32.609911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868802+0000","last_change":"2023-08-27T11:13:37.882840+0000","last_active":"2023-08-27T11:13:38.868802+0000","last_peered":"2023-08-27T11:13:38.868802+0000","last_clean":"2023-08-27T11:13:38.868802+0000","last_became_active":"2023-08-27T11:13:37.882137+0000","last_became_peered":"2023-08-27T11:13:37.882137+0000","last_unstale":"2023-08-27T11:13:38.868802+0000","last_undegraded":"2023-08-27T11:13:38.868802+0000","last_fullsized":"2023-08-27T11:13:38.868802+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:18:11.544766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868614+0000","last_change":"2023-08-27T11:13:37.881510+0000","last_active":"2023-08-27T11:13:38.868614+0000","last_peered":"2023-08-27T11:13:38.868614+0000","last_clean":"2023-08-27T11:13:38.868614+0000","last_became_active":"2023-08-27T11:13:37.881235+0000","last_became_peered":"2023-08-27T11:13:37.881235+0000","last_unstale":"2023-08-27T11:13:38.868614+0000","last_undegraded":"2023-08-27T11:13:38.868614+0000","last_fullsized":"2023-08-27T11:13:38.868614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:01.344642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931838+0000","last_change":"2023-08-27T11:13:37.882605+0000","last_active":"2023-08-27T11:13:38.931838+0000","last_peered":"2023-08-27T11:13:38.931838+0000","last_clean":"2023-08-27T11:13:38.931838+0000","last_became_active":"2023-08-27T11:13:37.882302+0000","last_became_peered":"2023-08-27T11:13:37.882302+0000","last_unstale":"2023-08-27T11:13:38.931838+0000","last_undegraded":"2023-08-27T11:13:38.931838+0000","last_fullsized":"2023-08-27T11:13:38.931838+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:52.295625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931728+0000","last_change":"2023-08-27T11:13:37.881684+0000","last_active":"2023-08-27T11:13:38.931728+0000","last_peered":"2023-08-27T11:13:38.931728+0000","last_clean":"2023-08-27T11:13:38.931728+0000","last_became_active":"2023-08-27T11:13:37.880614+0000","last_became_peered":"2023-08-27T11:13:37.880614+0000","last_unstale":"2023-08-27T11:13:38.931728+0000","last_undegraded":"2023-08-27T11:13:38.931728+0000","last_fullsized":"2023-08-27T11:13:38.931728+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:25:13.059199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931291+0000","last_change":"2023-08-27T11:13:37.884201+0000","last_active":"2023-08-27T11:13:38.931291+0000","last_peered":"2023-08-27T11:13:38.931291+0000","last_clean":"2023-08-27T11:13:38.931291+0000","last_became_active":"2023-08-27T11:13:37.883931+0000","last_became_peered":"2023-08-27T11:13:37.883931+0000","last_unstale":"2023-08-27T11:13:38.931291+0000","last_undegraded":"2023-08-27T11:13:38.931291+0000","last_fullsized":"2023-08-27T11:13:38.931291+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:42.147396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932276+0000","last_change":"2023-08-27T11:13:37.886522+0000","last_active":"2023-08-27T11:13:38.932276+0000","last_peered":"2023-08-27T11:13:38.932276+0000","last_clean":"2023-08-27T11:13:38.932276+0000","last_became_active":"2023-08-27T11:13:37.886075+0000","last_became_peered":"2023-08-27T11:13:37.886075+0000","last_unstale":"2023-08-27T11:13:38.932276+0000","last_undegraded":"2023-08-27T11:13:38.932276+0000","last_fullsized":"2023-08-27T11:13:38.932276+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:19:04.859147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931322+0000","last_change":"2023-08-27T11:13:37.886965+0000","last_active":"2023-08-27T11:13:38.931322+0000","last_peered":"2023-08-27T11:13:38.931322+0000","last_clean":"2023-08-27T11:13:38.931322+0000","last_became_active":"2023-08-27T11:13:37.885962+0000","last_became_peered":"2023-08-27T11:13:37.885962+0000","last_unstale":"2023-08-27T11:13:38.931322+0000","last_undegraded":"2023-08-27T11:13:38.931322+0000","last_fullsized":"2023-08-27T11:13:38.931322+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:25.530471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931721+0000","last_change":"2023-08-27T11:13:37.886432+0000","last_active":"2023-08-27T11:13:38.931721+0000","last_peered":"2023-08-27T11:13:38.931721+0000","last_clean":"2023-08-27T11:13:38.931721+0000","last_became_active":"2023-08-27T11:13:37.885541+0000","last_became_peered":"2023-08-27T11:13:37.885541+0000","last_unstale":"2023-08-27T11:13:38.931721+0000","last_undegraded":"2023-08-27T11:13:38.931721+0000","last_fullsized":"2023-08-27T11:13:38.931721+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:25:13.059199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932396+0000","last_change":"2023-08-27T11:13:37.883164+0000","last_active":"2023-08-27T11:13:38.932396+0000","last_peered":"2023-08-27T11:13:38.932396+0000","last_clean":"2023-08-27T11:13:38.932396+0000","last_became_active":"2023-08-27T11:13:37.882617+0000","last_became_peered":"2023-08-27T11:13:37.882617+0000","last_unstale":"2023-08-27T11:13:38.932396+0000","last_undegraded":"2023-08-27T11:13:38.932396+0000","last_fullsized":"2023-08-27T11:13:38.932396+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:46:30.188536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931437+0000","last_change":"2023-08-27T11:13:37.876059+0000","last_active":"2023-08-27T11:13:38.931437+0000","last_peered":"2023-08-27T11:13:38.931437+0000","last_clean":"2023-08-27T11:13:38.931437+0000","last_became_active":"2023-08-27T11:13:37.875735+0000","last_became_peered":"2023-08-27T11:13:37.875735+0000","last_unstale":"2023-08-27T11:13:38.931437+0000","last_undegraded":"2023-08-27T11:13:38.931437+0000","last_fullsized":"2023-08-27T11:13:38.931437+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:26.545550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869848+0000","last_change":"2023-08-27T11:13:37.882598+0000","last_active":"2023-08-27T11:13:38.869848+0000","last_peered":"2023-08-27T11:13:38.869848+0000","last_clean":"2023-08-27T11:13:38.869848+0000","last_became_active":"2023-08-27T11:13:37.881944+0000","last_became_peered":"2023-08-27T11:13:37.881944+0000","last_unstale":"2023-08-27T11:13:38.869848+0000","last_undegraded":"2023-08-27T11:13:38.869848+0000","last_fullsized":"2023-08-27T11:13:38.869848+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:58:06.147584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869787+0000","last_change":"2023-08-27T11:13:37.890892+0000","last_active":"2023-08-27T11:13:38.869787+0000","last_peered":"2023-08-27T11:13:38.869787+0000","last_clean":"2023-08-27T11:13:38.869787+0000","last_became_active":"2023-08-27T11:13:37.890604+0000","last_became_peered":"2023-08-27T11:13:37.890604+0000","last_unstale":"2023-08-27T11:13:38.869787+0000","last_undegraded":"2023-08-27T11:13:38.869787+0000","last_fullsized":"2023-08-27T11:13:38.869787+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:44:13.233649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931797+0000","last_change":"2023-08-27T11:13:37.879212+0000","last_active":"2023-08-27T11:13:38.931797+0000","last_peered":"2023-08-27T11:13:38.931797+0000","last_clean":"2023-08-27T11:13:38.931797+0000","last_became_active":"2023-08-27T11:13:37.878920+0000","last_became_peered":"2023-08-27T11:13:37.878920+0000","last_unstale":"2023-08-27T11:13:38.931797+0000","last_undegraded":"2023-08-27T11:13:38.931797+0000","last_fullsized":"2023-08-27T11:13:38.931797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:15.893609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868518+0000","last_change":"2023-08-27T11:13:37.875015+0000","last_active":"2023-08-27T11:13:38.868518+0000","last_peered":"2023-08-27T11:13:38.868518+0000","last_clean":"2023-08-27T11:13:38.868518+0000","last_became_active":"2023-08-27T11:13:37.874715+0000","last_became_peered":"2023-08-27T11:13:37.874715+0000","last_unstale":"2023-08-27T11:13:38.868518+0000","last_undegraded":"2023-08-27T11:13:38.868518+0000","last_fullsized":"2023-08-27T11:13:38.868518+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:57:24.646763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931798+0000","last_change":"2023-08-27T11:13:37.885957+0000","last_active":"2023-08-27T11:13:38.931798+0000","last_peered":"2023-08-27T11:13:38.931798+0000","last_clean":"2023-08-27T11:13:38.931798+0000","last_became_active":"2023-08-27T11:13:37.885305+0000","last_became_peered":"2023-08-27T11:13:37.885305+0000","last_unstale":"2023-08-27T11:13:38.931798+0000","last_undegraded":"2023-08-27T11:13:38.931798+0000","last_fullsized":"2023-08-27T11:13:38.931798+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:42:35.744996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931122+0000","last_change":"2023-08-27T11:13:37.880114+0000","last_active":"2023-08-27T11:13:38.931122+0000","last_peered":"2023-08-27T11:13:38.931122+0000","last_clean":"2023-08-27T11:13:38.931122+0000","last_became_active":"2023-08-27T11:13:37.879824+0000","last_became_peered":"2023-08-27T11:13:37.879824+0000","last_unstale":"2023-08-27T11:13:38.931122+0000","last_undegraded":"2023-08-27T11:13:38.931122+0000","last_fullsized":"2023-08-27T11:13:38.931122+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:25.530471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865467+0000","last_change":"2023-08-27T11:13:37.875343+0000","last_active":"2023-08-27T11:13:38.865467+0000","last_peered":"2023-08-27T11:13:38.865467+0000","last_clean":"2023-08-27T11:13:38.865467+0000","last_became_active":"2023-08-27T11:13:37.874779+0000","last_became_peered":"2023-08-27T11:13:37.874779+0000","last_unstale":"2023-08-27T11:13:38.865467+0000","last_undegraded":"2023-08-27T11:13:38.865467+0000","last_fullsized":"2023-08-27T11:13:38.865467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:47:28.478585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"41'4","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931533+0000","last_change":"2023-08-27T11:13:36.912028+0000","last_active":"2023-08-27T11:13:38.931533+0000","last_peered":"2023-08-27T11:13:38.931533+0000","last_clean":"2023-08-27T11:13:38.931533+0000","last_became_active":"2023-08-27T11:13:36.911755+0000","last_became_peered":"2023-08-27T11:13:36.911755+0000","last_unstale":"2023-08-27T11:13:38.931533+0000","last_undegraded":"2023-08-27T11:13:38.931533+0000","last_fullsized":"2023-08-27T11:13:38.931533+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:49.599460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932252+0000","last_change":"2023-08-27T11:13:37.886004+0000","last_active":"2023-08-27T11:13:38.932252+0000","last_peered":"2023-08-27T11:13:38.932252+0000","last_clean":"2023-08-27T11:13:38.932252+0000","last_became_active":"2023-08-27T11:13:37.885659+0000","last_became_peered":"2023-08-27T11:13:37.885659+0000","last_unstale":"2023-08-27T11:13:38.932252+0000","last_undegraded":"2023-08-27T11:13:38.932252+0000","last_fullsized":"2023-08-27T11:13:38.932252+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:38:37.634080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868807+0000","last_change":"2023-08-27T11:13:36.911917+0000","last_active":"2023-08-27T11:13:38.868807+0000","last_peered":"2023-08-27T11:13:38.868807+0000","last_clean":"2023-08-27T11:13:38.868807+0000","last_became_active":"2023-08-27T11:13:36.911571+0000","last_became_peered":"2023-08-27T11:13:36.911571+0000","last_unstale":"2023-08-27T11:13:38.868807+0000","last_undegraded":"2023-08-27T11:13:38.868807+0000","last_fullsized":"2023-08-27T11:13:38.868807+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:26:47.081246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869535+0000","last_change":"2023-08-27T11:13:37.887490+0000","last_active":"2023-08-27T11:13:38.869535+0000","last_peered":"2023-08-27T11:13:38.869535+0000","last_clean":"2023-08-27T11:13:38.869535+0000","last_became_active":"2023-08-27T11:13:37.887199+0000","last_became_peered":"2023-08-27T11:13:37.887199+0000","last_unstale":"2023-08-27T11:13:38.869535+0000","last_undegraded":"2023-08-27T11:13:38.869535+0000","last_fullsized":"2023-08-27T11:13:38.869535+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:40.235315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865883+0000","last_change":"2023-08-27T11:13:36.903935+0000","last_active":"2023-08-27T11:13:38.865883+0000","last_peered":"2023-08-27T11:13:38.865883+0000","last_clean":"2023-08-27T11:13:38.865883+0000","last_became_active":"2023-08-27T11:13:36.903638+0000","last_became_peered":"2023-08-27T11:13:36.903638+0000","last_unstale":"2023-08-27T11:13:38.865883+0000","last_undegraded":"2023-08-27T11:13:38.865883+0000","last_fullsized":"2023-08-27T11:13:38.865883+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:12.743856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865751+0000","last_change":"2023-08-27T11:13:37.884201+0000","last_active":"2023-08-27T11:13:38.865751+0000","last_peered":"2023-08-27T11:13:38.865751+0000","last_clean":"2023-08-27T11:13:38.865751+0000","last_became_active":"2023-08-27T11:13:37.883699+0000","last_became_peered":"2023-08-27T11:13:37.883699+0000","last_unstale":"2023-08-27T11:13:38.865751+0000","last_undegraded":"2023-08-27T11:13:38.865751+0000","last_fullsized":"2023-08-27T11:13:38.865751+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:54:52.214478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931765+0000","last_change":"2023-08-27T11:13:36.905588+0000","last_active":"2023-08-27T11:13:38.931765+0000","last_peered":"2023-08-27T11:13:38.931765+0000","last_clean":"2023-08-27T11:13:38.931765+0000","last_became_active":"2023-08-27T11:13:36.905314+0000","last_became_peered":"2023-08-27T11:13:36.905314+0000","last_unstale":"2023-08-27T11:13:38.931765+0000","last_undegraded":"2023-08-27T11:13:38.931765+0000","last_fullsized":"2023-08-27T11:13:38.931765+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:26:30.921725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868595+0000","last_change":"2023-08-27T11:13:37.884354+0000","last_active":"2023-08-27T11:13:38.868595+0000","last_peered":"2023-08-27T11:13:38.868595+0000","last_clean":"2023-08-27T11:13:38.868595+0000","last_became_active":"2023-08-27T11:13:37.883300+0000","last_became_peered":"2023-08-27T11:13:37.883300+0000","last_unstale":"2023-08-27T11:13:38.868595+0000","last_undegraded":"2023-08-27T11:13:38.868595+0000","last_fullsized":"2023-08-27T11:13:38.868595+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:54:09.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931601+0000","last_change":"2023-08-27T11:13:36.910275+0000","last_active":"2023-08-27T11:13:38.931601+0000","last_peered":"2023-08-27T11:13:38.931601+0000","last_clean":"2023-08-27T11:13:38.931601+0000","last_became_active":"2023-08-27T11:13:36.909749+0000","last_became_peered":"2023-08-27T11:13:36.909749+0000","last_unstale":"2023-08-27T11:13:38.931601+0000","last_undegraded":"2023-08-27T11:13:38.931601+0000","last_fullsized":"2023-08-27T11:13:38.931601+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:26:30.921725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932174+0000","last_change":"2023-08-27T11:13:37.881329+0000","last_active":"2023-08-27T11:13:38.932174+0000","last_peered":"2023-08-27T11:13:38.932174+0000","last_clean":"2023-08-27T11:13:38.932174+0000","last_became_active":"2023-08-27T11:13:37.880995+0000","last_became_peered":"2023-08-27T11:13:37.880995+0000","last_unstale":"2023-08-27T11:13:38.932174+0000","last_undegraded":"2023-08-27T11:13:38.932174+0000","last_fullsized":"2023-08-27T11:13:38.932174+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:09:26.514589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868710+0000","last_change":"2023-08-27T11:13:36.896944+0000","last_active":"2023-08-27T11:13:38.868710+0000","last_peered":"2023-08-27T11:13:38.868710+0000","last_clean":"2023-08-27T11:13:38.868710+0000","last_became_active":"2023-08-27T11:13:36.896738+0000","last_became_peered":"2023-08-27T11:13:36.896738+0000","last_unstale":"2023-08-27T11:13:38.868710+0000","last_undegraded":"2023-08-27T11:13:38.868710+0000","last_fullsized":"2023-08-27T11:13:38.868710+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:28.903378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931427+0000","last_change":"2023-08-27T11:13:37.880576+0000","last_active":"2023-08-27T11:13:38.931427+0000","last_peered":"2023-08-27T11:13:38.931427+0000","last_clean":"2023-08-27T11:13:38.931427+0000","last_became_active":"2023-08-27T11:13:37.880282+0000","last_became_peered":"2023-08-27T11:13:37.880282+0000","last_unstale":"2023-08-27T11:13:38.931427+0000","last_undegraded":"2023-08-27T11:13:38.931427+0000","last_fullsized":"2023-08-27T11:13:38.931427+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:52.370999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869015+0000","last_change":"2023-08-27T11:13:36.906448+0000","last_active":"2023-08-27T11:13:38.869015+0000","last_peered":"2023-08-27T11:13:38.869015+0000","last_clean":"2023-08-27T11:13:38.869015+0000","last_became_active":"2023-08-27T11:13:36.906160+0000","last_became_peered":"2023-08-27T11:13:36.906160+0000","last_unstale":"2023-08-27T11:13:38.869015+0000","last_undegraded":"2023-08-27T11:13:38.869015+0000","last_fullsized":"2023-08-27T11:13:38.869015+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:56.632773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931573+0000","last_change":"2023-08-27T11:13:37.886241+0000","last_active":"2023-08-27T11:13:38.931573+0000","last_peered":"2023-08-27T11:13:38.931573+0000","last_clean":"2023-08-27T11:13:38.931573+0000","last_became_active":"2023-08-27T11:13:37.885733+0000","last_became_peered":"2023-08-27T11:13:37.885733+0000","last_unstale":"2023-08-27T11:13:38.931573+0000","last_undegraded":"2023-08-27T11:13:38.931573+0000","last_fullsized":"2023-08-27T11:13:38.931573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:06.895181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930863+0000","last_change":"2023-08-27T11:13:36.904727+0000","last_active":"2023-08-27T11:13:38.930863+0000","last_peered":"2023-08-27T11:13:38.930863+0000","last_clean":"2023-08-27T11:13:38.930863+0000","last_became_active":"2023-08-27T11:13:36.904424+0000","last_became_peered":"2023-08-27T11:13:36.904424+0000","last_unstale":"2023-08-27T11:13:38.930863+0000","last_undegraded":"2023-08-27T11:13:38.930863+0000","last_fullsized":"2023-08-27T11:13:38.930863+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:47:27.472581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868769+0000","last_change":"2023-08-27T11:13:37.880618+0000","last_active":"2023-08-27T11:13:38.868769+0000","last_peered":"2023-08-27T11:13:38.868769+0000","last_clean":"2023-08-27T11:13:38.868769+0000","last_became_active":"2023-08-27T11:13:37.880308+0000","last_became_peered":"2023-08-27T11:13:37.880308+0000","last_unstale":"2023-08-27T11:13:38.868769+0000","last_undegraded":"2023-08-27T11:13:38.868769+0000","last_fullsized":"2023-08-27T11:13:38.868769+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:07.912152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869025+0000","last_change":"2023-08-27T11:13:36.908652+0000","last_active":"2023-08-27T11:13:38.869025+0000","last_peered":"2023-08-27T11:13:38.869025+0000","last_clean":"2023-08-27T11:13:38.869025+0000","last_became_active":"2023-08-27T11:13:36.908378+0000","last_became_peered":"2023-08-27T11:13:36.908378+0000","last_unstale":"2023-08-27T11:13:38.869025+0000","last_undegraded":"2023-08-27T11:13:38.869025+0000","last_fullsized":"2023-08-27T11:13:38.869025+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:24:36.068626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931576+0000","last_change":"2023-08-27T11:13:37.883817+0000","last_active":"2023-08-27T11:13:38.931576+0000","last_peered":"2023-08-27T11:13:38.931576+0000","last_clean":"2023-08-27T11:13:38.931576+0000","last_became_active":"2023-08-27T11:13:37.883538+0000","last_became_peered":"2023-08-27T11:13:37.883538+0000","last_unstale":"2023-08-27T11:13:38.931576+0000","last_undegraded":"2023-08-27T11:13:38.931576+0000","last_fullsized":"2023-08-27T11:13:38.931576+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:50.605464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932713+0000","last_change":"2023-08-27T11:13:36.905286+0000","last_active":"2023-08-27T11:13:38.932713+0000","last_peered":"2023-08-27T11:13:38.932713+0000","last_clean":"2023-08-27T11:13:38.932713+0000","last_became_active":"2023-08-27T11:13:36.905030+0000","last_became_peered":"2023-08-27T11:13:36.905030+0000","last_unstale":"2023-08-27T11:13:38.932713+0000","last_undegraded":"2023-08-27T11:13:38.932713+0000","last_fullsized":"2023-08-27T11:13:38.932713+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:05:08.549705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931523+0000","last_change":"2023-08-27T11:13:37.877830+0000","last_active":"2023-08-27T11:13:38.931523+0000","last_peered":"2023-08-27T11:13:38.931523+0000","last_clean":"2023-08-27T11:13:38.931523+0000","last_became_active":"2023-08-27T11:13:37.877519+0000","last_became_peered":"2023-08-27T11:13:37.877519+0000","last_unstale":"2023-08-27T11:13:38.931523+0000","last_undegraded":"2023-08-27T11:13:38.931523+0000","last_fullsized":"2023-08-27T11:13:38.931523+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:55.062646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"41'3","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932021+0000","last_change":"2023-08-27T11:13:36.912018+0000","last_active":"2023-08-27T11:13:38.932021+0000","last_peered":"2023-08-27T11:13:38.932021+0000","last_clean":"2023-08-27T11:13:38.932021+0000","last_became_active":"2023-08-27T11:13:36.911743+0000","last_became_peered":"2023-08-27T11:13:36.911743+0000","last_unstale":"2023-08-27T11:13:38.932021+0000","last_undegraded":"2023-08-27T11:13:38.932021+0000","last_fullsized":"2023-08-27T11:13:38.932021+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:44:12.227645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931733+0000","last_change":"2023-08-27T11:13:37.880332+0000","last_active":"2023-08-27T11:13:38.931733+0000","last_peered":"2023-08-27T11:13:38.931733+0000","last_clean":"2023-08-27T11:13:38.931733+0000","last_became_active":"2023-08-27T11:13:37.880046+0000","last_became_peered":"2023-08-27T11:13:37.880046+0000","last_unstale":"2023-08-27T11:13:38.931733+0000","last_undegraded":"2023-08-27T11:13:38.931733+0000","last_fullsized":"2023-08-27T11:13:38.931733+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:02.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"41'6","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931946+0000","last_change":"2023-08-27T11:13:36.911943+0000","last_active":"2023-08-27T11:13:38.931946+0000","last_peered":"2023-08-27T11:13:38.931946+0000","last_clean":"2023-08-27T11:13:38.931946+0000","last_became_active":"2023-08-27T11:13:36.911642+0000","last_became_peered":"2023-08-27T11:13:36.911642+0000","last_unstale":"2023-08-27T11:13:38.931946+0000","last_undegraded":"2023-08-27T11:13:38.931946+0000","last_fullsized":"2023-08-27T11:13:38.931946+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:51.289621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931644+0000","last_change":"2023-08-27T11:13:37.883119+0000","last_active":"2023-08-27T11:13:38.931644+0000","last_peered":"2023-08-27T11:13:38.931644+0000","last_clean":"2023-08-27T11:13:38.931644+0000","last_became_active":"2023-08-27T11:13:37.882868+0000","last_became_peered":"2023-08-27T11:13:37.882868+0000","last_unstale":"2023-08-27T11:13:38.931644+0000","last_undegraded":"2023-08-27T11:13:38.931644+0000","last_fullsized":"2023-08-27T11:13:38.931644+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:26.545550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931758+0000","last_change":"2023-08-27T11:13:36.910332+0000","last_active":"2023-08-27T11:13:38.931758+0000","last_peered":"2023-08-27T11:13:38.931758+0000","last_clean":"2023-08-27T11:13:38.931758+0000","last_became_active":"2023-08-27T11:13:36.910032+0000","last_became_peered":"2023-08-27T11:13:36.910032+0000","last_unstale":"2023-08-27T11:13:38.931758+0000","last_undegraded":"2023-08-27T11:13:38.931758+0000","last_fullsized":"2023-08-27T11:13:38.931758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:03.703758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932316+0000","last_change":"2023-08-27T11:13:37.888907+0000","last_active":"2023-08-27T11:13:38.932316+0000","last_peered":"2023-08-27T11:13:38.932316+0000","last_clean":"2023-08-27T11:13:38.932316+0000","last_became_active":"2023-08-27T11:13:37.888600+0000","last_became_peered":"2023-08-27T11:13:37.888600+0000","last_unstale":"2023-08-27T11:13:38.932316+0000","last_undegraded":"2023-08-27T11:13:38.932316+0000","last_fullsized":"2023-08-27T11:13:38.932316+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:46:01.691567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931360+0000","last_change":"2023-08-27T11:13:36.908298+0000","last_active":"2023-08-27T11:13:38.931360+0000","last_peered":"2023-08-27T11:13:38.931360+0000","last_clean":"2023-08-27T11:13:38.931360+0000","last_became_active":"2023-08-27T11:13:36.908063+0000","last_became_peered":"2023-08-27T11:13:36.908063+0000","last_unstale":"2023-08-27T11:13:38.931360+0000","last_undegraded":"2023-08-27T11:13:38.931360+0000","last_fullsized":"2023-08-27T11:13:38.931360+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:37:34.113218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930724+0000","last_change":"2023-08-27T11:13:37.878146+0000","last_active":"2023-08-27T11:13:38.930724+0000","last_peered":"2023-08-27T11:13:38.930724+0000","last_clean":"2023-08-27T11:13:38.930724+0000","last_became_active":"2023-08-27T11:13:37.877835+0000","last_became_peered":"2023-08-27T11:13:37.877835+0000","last_unstale":"2023-08-27T11:13:38.930724+0000","last_undegraded":"2023-08-27T11:13:38.930724+0000","last_fullsized":"2023-08-27T11:13:38.930724+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:03.589442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931227+0000","last_change":"2023-08-27T11:13:36.909688+0000","last_active":"2023-08-27T11:13:38.931227+0000","last_peered":"2023-08-27T11:13:38.931227+0000","last_clean":"2023-08-27T11:13:38.931227+0000","last_became_active":"2023-08-27T11:13:36.908823+0000","last_became_peered":"2023-08-27T11:13:36.908823+0000","last_unstale":"2023-08-27T11:13:38.931227+0000","last_undegraded":"2023-08-27T11:13:38.931227+0000","last_fullsized":"2023-08-27T11:13:38.931227+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:13.768824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931906+0000","last_change":"2023-08-27T11:13:37.880286+0000","last_active":"2023-08-27T11:13:38.931906+0000","last_peered":"2023-08-27T11:13:38.931906+0000","last_clean":"2023-08-27T11:13:38.931906+0000","last_became_active":"2023-08-27T11:13:37.879965+0000","last_became_peered":"2023-08-27T11:13:37.879965+0000","last_unstale":"2023-08-27T11:13:38.931906+0000","last_undegraded":"2023-08-27T11:13:38.931906+0000","last_fullsized":"2023-08-27T11:13:38.931906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:59.902099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"41'8","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930987+0000","last_change":"2023-08-27T11:13:36.904679+0000","last_active":"2023-08-27T11:13:38.930987+0000","last_peered":"2023-08-27T11:13:38.930987+0000","last_clean":"2023-08-27T11:13:38.930987+0000","last_became_active":"2023-08-27T11:13:36.904382+0000","last_became_peered":"2023-08-27T11:13:36.904382+0000","last_unstale":"2023-08-27T11:13:38.930987+0000","last_undegraded":"2023-08-27T11:13:38.930987+0000","last_fullsized":"2023-08-27T11:13:38.930987+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:23:25.324897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931575+0000","last_change":"2023-08-27T11:13:37.887091+0000","last_active":"2023-08-27T11:13:38.931575+0000","last_peered":"2023-08-27T11:13:38.931575+0000","last_clean":"2023-08-27T11:13:38.931575+0000","last_became_active":"2023-08-27T11:13:37.886773+0000","last_became_peered":"2023-08-27T11:13:37.886773+0000","last_unstale":"2023-08-27T11:13:38.931575+0000","last_undegraded":"2023-08-27T11:13:38.931575+0000","last_fullsized":"2023-08-27T11:13:38.931575+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:32.609911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931819+0000","last_change":"2023-08-27T11:13:36.910961+0000","last_active":"2023-08-27T11:13:38.931819+0000","last_peered":"2023-08-27T11:13:38.931819+0000","last_clean":"2023-08-27T11:13:38.931819+0000","last_became_active":"2023-08-27T11:13:36.910645+0000","last_became_peered":"2023-08-27T11:13:36.910645+0000","last_unstale":"2023-08-27T11:13:38.931819+0000","last_undegraded":"2023-08-27T11:13:38.931819+0000","last_fullsized":"2023-08-27T11:13:38.931819+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:14.887604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932758+0000","last_change":"2023-08-27T11:13:37.883074+0000","last_active":"2023-08-27T11:13:38.932758+0000","last_peered":"2023-08-27T11:13:38.932758+0000","last_clean":"2023-08-27T11:13:38.932758+0000","last_became_active":"2023-08-27T11:13:37.882584+0000","last_became_peered":"2023-08-27T11:13:37.882584+0000","last_unstale":"2023-08-27T11:13:38.932758+0000","last_undegraded":"2023-08-27T11:13:38.932758+0000","last_fullsized":"2023-08-27T11:13:38.932758+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:11:15.935661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869186+0000","last_change":"2023-08-27T11:13:36.899322+0000","last_active":"2023-08-27T11:13:38.869186+0000","last_peered":"2023-08-27T11:13:38.869186+0000","last_clean":"2023-08-27T11:13:38.869186+0000","last_became_active":"2023-08-27T11:13:36.899044+0000","last_became_peered":"2023-08-27T11:13:36.899044+0000","last_unstale":"2023-08-27T11:13:38.869186+0000","last_undegraded":"2023-08-27T11:13:38.869186+0000","last_fullsized":"2023-08-27T11:13:38.869186+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:42.124218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931330+0000","last_change":"2023-08-27T11:13:37.886414+0000","last_active":"2023-08-27T11:13:38.931330+0000","last_peered":"2023-08-27T11:13:38.931330+0000","last_clean":"2023-08-27T11:13:38.931330+0000","last_became_active":"2023-08-27T11:13:37.886127+0000","last_became_peered":"2023-08-27T11:13:37.886127+0000","last_unstale":"2023-08-27T11:13:38.931330+0000","last_undegraded":"2023-08-27T11:13:38.931330+0000","last_fullsized":"2023-08-27T11:13:38.931330+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:32.087877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869553+0000","last_change":"2023-08-27T11:13:36.900525+0000","last_active":"2023-08-27T11:13:38.869553+0000","last_peered":"2023-08-27T11:13:38.869553+0000","last_clean":"2023-08-27T11:13:38.869553+0000","last_became_active":"2023-08-27T11:13:36.900261+0000","last_became_peered":"2023-08-27T11:13:36.900261+0000","last_unstale":"2023-08-27T11:13:38.869553+0000","last_undegraded":"2023-08-27T11:13:38.869553+0000","last_fullsized":"2023-08-27T11:13:38.869553+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:03.703758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932025+0000","last_change":"2023-08-27T11:13:37.880950+0000","last_active":"2023-08-27T11:13:38.932025+0000","last_peered":"2023-08-27T11:13:38.932025+0000","last_clean":"2023-08-27T11:13:38.932025+0000","last_became_active":"2023-08-27T11:13:37.880680+0000","last_became_peered":"2023-08-27T11:13:37.880680+0000","last_unstale":"2023-08-27T11:13:38.932025+0000","last_undegraded":"2023-08-27T11:13:38.932025+0000","last_fullsized":"2023-08-27T11:13:38.932025+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:37.476696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931570+0000","last_change":"2023-08-27T11:13:36.913330+0000","last_active":"2023-08-27T11:13:38.931570+0000","last_peered":"2023-08-27T11:13:38.931570+0000","last_clean":"2023-08-27T11:13:38.931570+0000","last_became_active":"2023-08-27T11:13:36.913018+0000","last_became_peered":"2023-08-27T11:13:36.913018+0000","last_unstale":"2023-08-27T11:13:38.931570+0000","last_undegraded":"2023-08-27T11:13:38.931570+0000","last_fullsized":"2023-08-27T11:13:38.931570+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:01.408597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931072+0000","last_change":"2023-08-27T11:13:37.878052+0000","last_active":"2023-08-27T11:13:38.931072+0000","last_peered":"2023-08-27T11:13:38.931072+0000","last_clean":"2023-08-27T11:13:38.931072+0000","last_became_active":"2023-08-27T11:13:37.877780+0000","last_became_peered":"2023-08-27T11:13:37.877780+0000","last_unstale":"2023-08-27T11:13:38.931072+0000","last_undegraded":"2023-08-27T11:13:38.931072+0000","last_fullsized":"2023-08-27T11:13:38.931072+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:34:03.294902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931762+0000","last_change":"2023-08-27T11:13:36.910074+0000","last_active":"2023-08-27T11:13:38.931762+0000","last_peered":"2023-08-27T11:13:38.931762+0000","last_clean":"2023-08-27T11:13:38.931762+0000","last_became_active":"2023-08-27T11:13:36.908889+0000","last_became_peered":"2023-08-27T11:13:36.908889+0000","last_unstale":"2023-08-27T11:13:38.931762+0000","last_undegraded":"2023-08-27T11:13:38.931762+0000","last_fullsized":"2023-08-27T11:13:38.931762+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:58.896094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865528+0000","last_change":"2023-08-27T11:13:37.882625+0000","last_active":"2023-08-27T11:13:38.865528+0000","last_peered":"2023-08-27T11:13:38.865528+0000","last_clean":"2023-08-27T11:13:38.865528+0000","last_became_active":"2023-08-27T11:13:37.882339+0000","last_became_peered":"2023-08-27T11:13:37.882339+0000","last_unstale":"2023-08-27T11:13:38.865528+0000","last_undegraded":"2023-08-27T11:13:38.865528+0000","last_fullsized":"2023-08-27T11:13:38.865528+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:57.229171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869230+0000","last_change":"2023-08-27T11:13:36.911780+0000","last_active":"2023-08-27T11:13:38.869230+0000","last_peered":"2023-08-27T11:13:38.869230+0000","last_clean":"2023-08-27T11:13:38.869230+0000","last_became_active":"2023-08-27T11:13:36.911506+0000","last_became_peered":"2023-08-27T11:13:36.911506+0000","last_unstale":"2023-08-27T11:13:38.869230+0000","last_undegraded":"2023-08-27T11:13:38.869230+0000","last_fullsized":"2023-08-27T11:13:38.869230+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:37:34.113218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931248+0000","last_change":"2023-08-27T11:13:37.875570+0000","last_active":"2023-08-27T11:13:38.931248+0000","last_peered":"2023-08-27T11:13:38.931248+0000","last_clean":"2023-08-27T11:13:38.931248+0000","last_became_active":"2023-08-27T11:13:37.875255+0000","last_became_peered":"2023-08-27T11:13:37.875255+0000","last_unstale":"2023-08-27T11:13:38.931248+0000","last_undegraded":"2023-08-27T11:13:38.931248+0000","last_fullsized":"2023-08-27T11:13:38.931248+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:06.895181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930860+0000","last_change":"2023-08-27T11:13:36.907510+0000","last_active":"2023-08-27T11:13:38.930860+0000","last_peered":"2023-08-27T11:13:38.930860+0000","last_clean":"2023-08-27T11:13:38.930860+0000","last_became_active":"2023-08-27T11:13:36.907218+0000","last_became_peered":"2023-08-27T11:13:36.907218+0000","last_unstale":"2023-08-27T11:13:38.930860+0000","last_undegraded":"2023-08-27T11:13:38.930860+0000","last_fullsized":"2023-08-27T11:13:38.930860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:56.223167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931602+0000","last_change":"2023-08-27T11:13:37.879236+0000","last_active":"2023-08-27T11:13:38.931602+0000","last_peered":"2023-08-27T11:13:38.931602+0000","last_clean":"2023-08-27T11:13:38.931602+0000","last_became_active":"2023-08-27T11:13:37.878942+0000","last_became_peered":"2023-08-27T11:13:37.878942+0000","last_unstale":"2023-08-27T11:13:38.931602+0000","last_undegraded":"2023-08-27T11:13:38.931602+0000","last_fullsized":"2023-08-27T11:13:38.931602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:54:52.214478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932318+0000","last_change":"2023-08-27T11:13:36.908297+0000","last_active":"2023-08-27T11:13:38.932318+0000","last_peered":"2023-08-27T11:13:38.932318+0000","last_clean":"2023-08-27T11:13:38.932318+0000","last_became_active":"2023-08-27T11:13:36.908005+0000","last_became_peered":"2023-08-27T11:13:36.908005+0000","last_unstale":"2023-08-27T11:13:38.932318+0000","last_undegraded":"2023-08-27T11:13:38.932318+0000","last_fullsized":"2023-08-27T11:13:38.932318+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:57:13.418325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868886+0000","last_change":"2023-08-27T11:13:37.884316+0000","last_active":"2023-08-27T11:13:38.868886+0000","last_peered":"2023-08-27T11:13:38.868886+0000","last_clean":"2023-08-27T11:13:38.868886+0000","last_became_active":"2023-08-27T11:13:37.883967+0000","last_became_peered":"2023-08-27T11:13:37.883967+0000","last_unstale":"2023-08-27T11:13:38.868886+0000","last_undegraded":"2023-08-27T11:13:38.868886+0000","last_fullsized":"2023-08-27T11:13:38.868886+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:17:21.632858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931107+0000","last_change":"2023-08-27T11:13:36.909242+0000","last_active":"2023-08-27T11:13:38.931107+0000","last_peered":"2023-08-27T11:13:38.931107+0000","last_clean":"2023-08-27T11:13:38.931107+0000","last_became_active":"2023-08-27T11:13:36.908797+0000","last_became_peered":"2023-08-27T11:13:36.908797+0000","last_unstale":"2023-08-27T11:13:38.931107+0000","last_undegraded":"2023-08-27T11:13:38.931107+0000","last_fullsized":"2023-08-27T11:13:38.931107+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:50:48.793983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865671+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:13:38.865671+0000","last_peered":"2023-08-27T11:13:38.865671+0000","last_clean":"2023-08-27T11:13:38.865671+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:13:38.865671+0000","last_undegraded":"2023-08-27T11:13:38.865671+0000","last_fullsized":"2023-08-27T11:13:38.865671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:17.340508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932256+0000","last_change":"2023-08-27T11:13:37.883343+0000","last_active":"2023-08-27T11:13:38.932256+0000","last_peered":"2023-08-27T11:13:38.932256+0000","last_clean":"2023-08-27T11:13:38.932256+0000","last_became_active":"2023-08-27T11:13:37.882695+0000","last_became_peered":"2023-08-27T11:13:37.882695+0000","last_unstale":"2023-08-27T11:13:38.932256+0000","last_undegraded":"2023-08-27T11:13:38.932256+0000","last_fullsized":"2023-08-27T11:13:38.932256+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:03:31.199993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:39.003152+0000","last_change":"2023-08-27T11:13:36.906247+0000","last_active":"2023-08-27T11:13:39.003152+0000","last_peered":"2023-08-27T11:13:39.003152+0000","last_clean":"2023-08-27T11:13:39.003152+0000","last_became_active":"2023-08-27T11:13:36.905971+0000","last_became_peered":"2023-08-27T11:13:36.905971+0000","last_unstale":"2023-08-27T11:13:39.003152+0000","last_undegraded":"2023-08-27T11:13:39.003152+0000","last_fullsized":"2023-08-27T11:13:39.003152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:49.547584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869134+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:13:38.869134+0000","last_peered":"2023-08-27T11:13:38.869134+0000","last_clean":"2023-08-27T11:13:38.869134+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:13:38.869134+0000","last_undegraded":"2023-08-27T11:13:38.869134+0000","last_fullsized":"2023-08-27T11:13:38.869134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:37:12.054411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932390+0000","last_change":"2023-08-27T11:13:37.883076+0000","last_active":"2023-08-27T11:13:38.932390+0000","last_peered":"2023-08-27T11:13:38.932390+0000","last_clean":"2023-08-27T11:13:38.932390+0000","last_became_active":"2023-08-27T11:13:37.882793+0000","last_became_peered":"2023-08-27T11:13:37.882793+0000","last_unstale":"2023-08-27T11:13:38.932390+0000","last_undegraded":"2023-08-27T11:13:38.932390+0000","last_fullsized":"2023-08-27T11:13:38.932390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:01.061977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932139+0000","last_change":"2023-08-27T11:13:36.912325+0000","last_active":"2023-08-27T11:13:38.932139+0000","last_peered":"2023-08-27T11:13:38.932139+0000","last_clean":"2023-08-27T11:13:38.932139+0000","last_became_active":"2023-08-27T11:13:36.912054+0000","last_became_peered":"2023-08-27T11:13:36.912054+0000","last_unstale":"2023-08-27T11:13:38.932139+0000","last_undegraded":"2023-08-27T11:13:38.932139+0000","last_fullsized":"2023-08-27T11:13:38.932139+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:58:28.510547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931172+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:13:38.931172+0000","last_peered":"2023-08-27T11:13:38.931172+0000","last_clean":"2023-08-27T11:13:38.931172+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:13:38.931172+0000","last_undegraded":"2023-08-27T11:13:38.931172+0000","last_fullsized":"2023-08-27T11:13:38.931172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:57.057385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932653+0000","last_change":"2023-08-27T11:13:37.888938+0000","last_active":"2023-08-27T11:13:38.932653+0000","last_peered":"2023-08-27T11:13:38.932653+0000","last_clean":"2023-08-27T11:13:38.932653+0000","last_became_active":"2023-08-27T11:13:37.888625+0000","last_became_peered":"2023-08-27T11:13:37.888625+0000","last_unstale":"2023-08-27T11:13:38.932653+0000","last_undegraded":"2023-08-27T11:13:38.932653+0000","last_fullsized":"2023-08-27T11:13:38.932653+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:11:17.886182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869440+0000","last_change":"2023-08-27T11:13:36.898372+0000","last_active":"2023-08-27T11:13:38.869440+0000","last_peered":"2023-08-27T11:13:38.869440+0000","last_clean":"2023-08-27T11:13:38.869440+0000","last_became_active":"2023-08-27T11:13:36.898044+0000","last_became_peered":"2023-08-27T11:13:36.898044+0000","last_unstale":"2023-08-27T11:13:38.869440+0000","last_undegraded":"2023-08-27T11:13:38.869440+0000","last_fullsized":"2023-08-27T11:13:38.869440+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:49.599460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869367+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:13:38.869367+0000","last_peered":"2023-08-27T11:13:38.869367+0000","last_clean":"2023-08-27T11:13:38.869367+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:13:38.869367+0000","last_undegraded":"2023-08-27T11:13:38.869367+0000","last_fullsized":"2023-08-27T11:13:38.869367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:18:27.384111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931193+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:13:38.931193+0000","last_peered":"2023-08-27T11:13:38.931193+0000","last_clean":"2023-08-27T11:13:38.931193+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:13:38.931193+0000","last_undegraded":"2023-08-27T11:13:38.931193+0000","last_fullsized":"2023-08-27T11:13:38.931193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:08:00.473665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931874+0000","last_change":"2023-08-27T11:13:37.880908+0000","last_active":"2023-08-27T11:13:38.931874+0000","last_peered":"2023-08-27T11:13:38.931874+0000","last_clean":"2023-08-27T11:13:38.931874+0000","last_became_active":"2023-08-27T11:13:37.880615+0000","last_became_peered":"2023-08-27T11:13:37.880615+0000","last_unstale":"2023-08-27T11:13:38.931874+0000","last_undegraded":"2023-08-27T11:13:38.931874+0000","last_fullsized":"2023-08-27T11:13:38.931874+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:04.709762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865626+0000","last_change":"2023-08-27T11:13:36.905955+0000","last_active":"2023-08-27T11:13:38.865626+0000","last_peered":"2023-08-27T11:13:38.865626+0000","last_clean":"2023-08-27T11:13:38.865626+0000","last_became_active":"2023-08-27T11:13:36.905614+0000","last_became_peered":"2023-08-27T11:13:36.905614+0000","last_unstale":"2023-08-27T11:13:38.865626+0000","last_undegraded":"2023-08-27T11:13:38.865626+0000","last_fullsized":"2023-08-27T11:13:38.865626+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:23:25.324897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930880+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:13:38.930880+0000","last_peered":"2023-08-27T11:13:38.930880+0000","last_clean":"2023-08-27T11:13:38.930880+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:13:38.930880+0000","last_undegraded":"2023-08-27T11:13:38.930880+0000","last_fullsized":"2023-08-27T11:13:38.930880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:22.608286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868962+0000","last_change":"2023-08-27T11:13:37.884622+0000","last_active":"2023-08-27T11:13:38.868962+0000","last_peered":"2023-08-27T11:13:38.868962+0000","last_clean":"2023-08-27T11:13:38.868962+0000","last_became_active":"2023-08-27T11:13:37.884372+0000","last_became_peered":"2023-08-27T11:13:37.884372+0000","last_unstale":"2023-08-27T11:13:38.868962+0000","last_undegraded":"2023-08-27T11:13:38.868962+0000","last_fullsized":"2023-08-27T11:13:38.868962+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:42.147396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865921+0000","last_change":"2023-08-27T11:13:36.912357+0000","last_active":"2023-08-27T11:13:38.865921+0000","last_peered":"2023-08-27T11:13:38.865921+0000","last_clean":"2023-08-27T11:13:38.865921+0000","last_became_active":"2023-08-27T11:13:36.912075+0000","last_became_peered":"2023-08-27T11:13:36.912075+0000","last_unstale":"2023-08-27T11:13:38.865921+0000","last_undegraded":"2023-08-27T11:13:38.865921+0000","last_fullsized":"2023-08-27T11:13:38.865921+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:08:30.222629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932213+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:13:38.932213+0000","last_peered":"2023-08-27T11:13:38.932213+0000","last_clean":"2023-08-27T11:13:38.932213+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:13:38.932213+0000","last_undegraded":"2023-08-27T11:13:38.932213+0000","last_fullsized":"2023-08-27T11:13:38.932213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:27:49.048827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931619+0000","last_change":"2023-08-27T11:13:37.876107+0000","last_active":"2023-08-27T11:13:38.931619+0000","last_peered":"2023-08-27T11:13:38.931619+0000","last_clean":"2023-08-27T11:13:38.931619+0000","last_became_active":"2023-08-27T11:13:37.875821+0000","last_became_peered":"2023-08-27T11:13:37.875821+0000","last_unstale":"2023-08-27T11:13:38.931619+0000","last_undegraded":"2023-08-27T11:13:38.931619+0000","last_fullsized":"2023-08-27T11:13:38.931619+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:40.235315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869389+0000","last_change":"2023-08-27T11:13:36.903987+0000","last_active":"2023-08-27T11:13:38.869389+0000","last_peered":"2023-08-27T11:13:38.869389+0000","last_clean":"2023-08-27T11:13:38.869389+0000","last_became_active":"2023-08-27T11:13:36.903637+0000","last_became_peered":"2023-08-27T11:13:36.903637+0000","last_unstale":"2023-08-27T11:13:38.869389+0000","last_undegraded":"2023-08-27T11:13:38.869389+0000","last_fullsized":"2023-08-27T11:13:38.869389+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:23.977411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865608+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:13:38.865608+0000","last_peered":"2023-08-27T11:13:38.865608+0000","last_clean":"2023-08-27T11:13:38.865608+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:13:38.865608+0000","last_undegraded":"2023-08-27T11:13:38.865608+0000","last_fullsized":"2023-08-27T11:13:38.865608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:49:14.769496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931421+0000","last_change":"2023-08-27T11:13:37.882626+0000","last_active":"2023-08-27T11:13:38.931421+0000","last_peered":"2023-08-27T11:13:38.931421+0000","last_clean":"2023-08-27T11:13:38.931421+0000","last_became_active":"2023-08-27T11:13:37.882315+0000","last_became_peered":"2023-08-27T11:13:37.882315+0000","last_unstale":"2023-08-27T11:13:38.931421+0000","last_undegraded":"2023-08-27T11:13:38.931421+0000","last_fullsized":"2023-08-27T11:13:38.931421+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:38:47.084902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931471+0000","last_change":"2023-08-27T11:13:36.913330+0000","last_active":"2023-08-27T11:13:38.931471+0000","last_peered":"2023-08-27T11:13:38.931471+0000","last_clean":"2023-08-27T11:13:38.931471+0000","last_became_active":"2023-08-27T11:13:36.913062+0000","last_became_peered":"2023-08-27T11:13:36.913062+0000","last_unstale":"2023-08-27T11:13:38.931471+0000","last_undegraded":"2023-08-27T11:13:38.931471+0000","last_fullsized":"2023-08-27T11:13:38.931471+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:54.056642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932418+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:13:38.932418+0000","last_peered":"2023-08-27T11:13:38.932418+0000","last_clean":"2023-08-27T11:13:38.932418+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:13:38.932418+0000","last_undegraded":"2023-08-27T11:13:38.932418+0000","last_fullsized":"2023-08-27T11:13:38.932418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:45.131364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":809234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.495},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.59599999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65000000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.67100000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.627},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41499999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.38700000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.36699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":219448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.50900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.3780000000000001,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.3490000000000002,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.57999999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":9.3770000000000007,"5min":9.3770000000000007,"15min":9.3770000000000007},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.55200000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":8.3390000000000004,"5min":8.3390000000000004,"15min":8.3390000000000004},"last":0.748},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.72999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.77600000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.64400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":218846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41599999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.438}]},{"osd":4,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58999999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.622},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1073152,"data_stored":802205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.33100000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.252},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":8.1349999999999998,"5min":8.1349999999999998,"15min":8.1349999999999998},"last":0.30099999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.026999999999999,"5min":11.026999999999999,"15min":11.026999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":11.926,"5min":11.926,"15min":11.926},"last":0.35599999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":8.0779999999999994,"5min":8.0779999999999994,"15min":8.0779999999999994},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":10.922000000000001,"5min":10.922000000000001,"15min":10.922000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":8.0380000000000003,"5min":8.0380000000000003,"15min":8.0380000000000003},"last":0.55800000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":8.1069999999999993,"5min":8.1069999999999993,"15min":8.1069999999999993},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":10.996,"5min":10.996,"15min":10.996},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":220410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.28799999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.307},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.24099999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.37},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.626}]},{"osd":5,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.496}]},{"osd":7,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.51900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":499712,"data_stored":219414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":11.885999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.317}]},{"osd":1,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":6.0830000000000002,"5min":6.0830000000000002,"15min":6.0830000000000002},"last":0.255},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":12.050000000000001,"5min":12.050000000000001,"15min":12.050000000000001},"last":0.36199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":11.939,"5min":11.939,"15min":11.939},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":11.007,"5min":11.007,"15min":11.007},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.624}]},{"osd":5,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":11.981,"5min":11.981,"15min":11.981},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":11.09,"5min":11.09,"15min":11.09},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":12.789999999999999,"5min":12.789999999999999,"15min":12.789999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":12.016,"5min":12.016,"15min":12.016},"last":0.72199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":11.054,"5min":11.054,"15min":11.054},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.59799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":499712,"data_stored":220388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41599999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55000000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.83699999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.85099999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":218846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67100000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65500000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68600000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70799999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56200000000000006}]},{"osd":6,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46600000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.434}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:13:42.071 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:13:42.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.226+0000 7f30b34dd700 1 Processor -- start 2023-08-27T11:13:42.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b34dd700 1 -- start start 2023-08-27T11:13:42.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b34dd700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f30ac15e990 0x7f30ac0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b34dd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac0cc650 0x7f30ac0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b34dd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30ac0ce620 0x7f30ac0adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b34dd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac0ccb80 con 0x7f30ac0ce620 2023-08-27T11:13:42.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b34dd700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac0cce60 con 0x7f30ac15e990 2023-08-27T11:13:42.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b34dd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0cd600 con 0x7f30ac0cc650 2023-08-27T11:13:42.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b1a7a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30ac0ce620 0x7f30ac0adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b1a7a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30ac0ce620 0x7f30ac0adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54252/0 (socket says 172.21.15.45:54252) 2023-08-27T11:13:42.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.227+0000 7f30b1a7a700 1 -- 172.21.15.45:0/1451528657 learned_addr learned my addr 172.21.15.45:0/1451528657 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:42.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30b0a78700 1 --2- 172.21.15.45:0/1451528657 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac0cc650 0x7f30ac0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30b1279700 1 --2- 172.21.15.45:0/1451528657 >> v2:172.21.15.135:3300/0 conn(0x7f30ac15e990 0x7f30ac0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30b1a7a700 1 -- 172.21.15.45:0/1451528657 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac0cc650 msgr2=0x7f30ac0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:42.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30b1a7a700 1 --2- 172.21.15.45:0/1451528657 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac0cc650 0x7f30ac0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30b1a7a700 1 -- 172.21.15.45:0/1451528657 >> v2:172.21.15.135:3300/0 conn(0x7f30ac15e990 msgr2=0x7f30ac0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:42.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30b1a7a700 1 --2- 172.21.15.45:0/1451528657 >> v2:172.21.15.135:3300/0 conn(0x7f30ac15e990 0x7f30ac0ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30b1a7a700 1 -- 172.21.15.45:0/1451528657 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac0cd8e0 con 0x7f30ac0ce620 2023-08-27T11:13:42.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30b1a7a700 1 --2- 172.21.15.45:0/1451528657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 0x7f30ac0adad0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f30ac06d9d0 tx=0x7f30a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f091c0201f8763f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:42.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30a3fff700 1 -- 172.21.15.45:0/1451528657 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a400e070 con 0x7f30ac0ce620 2023-08-27T11:13:42.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30a3fff700 1 -- 172.21.15.45:0/1451528657 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a400b910 con 0x7f30ac0ce620 2023-08-27T11:13:42.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.228+0000 7f30a3fff700 1 -- 172.21.15.45:0/1451528657 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a400bcc0 con 0x7f30ac0ce620 2023-08-27T11:13:42.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.229+0000 7f30b34dd700 1 -- 172.21.15.45:0/1451528657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 msgr2=0x7f30ac0adad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.229+0000 7f30b34dd700 1 --2- 172.21.15.45:0/1451528657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 0x7f30ac0adad0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f30ac06d9d0 tx=0x7f30a400b330 comp rx=0 tx=0).stop 2023-08-27T11:13:42.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.229+0000 7f30b34dd700 1 -- 172.21.15.45:0/1451528657 shutdown_connections 2023-08-27T11:13:42.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.229+0000 7f30b34dd700 1 --2- 172.21.15.45:0/1451528657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 0x7f30ac0adad0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.229+0000 7f30b34dd700 1 --2- 172.21.15.45:0/1451528657 >> v2:172.21.15.135:3300/0 conn(0x7f30ac15e990 0x7f30ac0ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.229+0000 7f30b34dd700 1 --2- 172.21.15.45:0/1451528657 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac0cc650 0x7f30ac0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.229+0000 7f30b34dd700 1 -- 172.21.15.45:0/1451528657 >> 172.21.15.45:0/1451528657 conn(0x7f30ac0d0f60 msgr2=0x7f30ac0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:42.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.229+0000 7f30b34dd700 1 -- 172.21.15.45:0/1451528657 shutdown_connections 2023-08-27T11:13:42.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.229+0000 7f30b34dd700 1 -- 172.21.15.45:0/1451528657 wait complete. 2023-08-27T11:13:42.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.230+0000 7f30b34dd700 1 Processor -- start 2023-08-27T11:13:42.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.230+0000 7f30b34dd700 1 -- start start 2023-08-27T11:13:42.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b34dd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ac0cc650 0x7f30ac152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b34dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 0x7f30ac15dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b34dd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac15e990 0x7f30ac0b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b34dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac1532e0 con 0x7f30ac0ce620 2023-08-27T11:13:42.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b34dd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac1f2ad0 con 0x7f30ac0cc650 2023-08-27T11:13:42.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b34dd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0b6580 con 0x7f30ac15e990 2023-08-27T11:13:42.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b0a78700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 0x7f30ac15dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b0a78700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 0x7f30ac15dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54256/0 (socket says 172.21.15.45:54256) 2023-08-27T11:13:42.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b0a78700 1 -- 172.21.15.45:0/2177928399 learned_addr learned my addr 172.21.15.45:0/2177928399 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:42.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b1a7a700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac15e990 0x7f30ac0b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b1279700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ac0cc650 0x7f30ac152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b0a78700 1 -- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac15e990 msgr2=0x7f30ac0b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b0a78700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac15e990 0x7f30ac0b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b0a78700 1 -- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ac0cc650 msgr2=0x7f30ac152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b0a78700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ac0cc650 0x7f30ac152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b0a78700 1 -- 172.21.15.45:0/2177928399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4009b30 con 0x7f30ac0ce620 2023-08-27T11:13:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30b0a78700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 0x7f30ac15dad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f30a800e840 tx=0x7f30a800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.231+0000 7f30a1ffb700 1 -- 172.21.15.45:0/2177928399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8013680 con 0x7f30ac0ce620 2023-08-27T11:13:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.232+0000 7f30a1ffb700 1 -- 172.21.15.45:0/2177928399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a80137e0 con 0x7f30ac0ce620 2023-08-27T11:13:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.232+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ac0b5ce0 con 0x7f30ac0ce620 2023-08-27T11:13:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.232+0000 7f30a1ffb700 1 -- 172.21.15.45:0/2177928399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8013b70 con 0x7f30ac0ce620 2023-08-27T11:13:42.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.232+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac0b62a0 con 0x7f30ac0ce620 2023-08-27T11:13:42.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.233+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ac0c8440 con 0x7f30ac0ce620 2023-08-27T11:13:42.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.233+0000 7f30a1ffb700 1 -- 172.21.15.45:0/2177928399 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f30a8013cd0 con 0x7f30ac0ce620 2023-08-27T11:13:42.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.233+0000 7f30a1ffb700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f309006a0e0 0x7f309006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.234+0000 7f30a1ffb700 1 -- 172.21.15.45:0/2177928399 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f30a8088420 con 0x7f30ac0ce620 2023-08-27T11:13:42.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.234+0000 7f30b1279700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f309006a0e0 0x7f309006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.234+0000 7f30b1279700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f309006a0e0 0x7f309006c5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f309c009b40 tx=0x7f309c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:42.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.238+0000 7f30a1ffb700 1 -- 172.21.15.45:0/2177928399 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f30a8057460 con 0x7f30ac0ce620 2023-08-27T11:13:42.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.432+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 --> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30ac15ed70 con 0x7f309006a0e0 2023-08-27T11:13:42.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.441+0000 7f30a1ffb700 1 -- 172.21.15.45:0/2177928399 <== mgr.4100 v2:172.21.15.135:6832/4120550306 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311505 (secure 0 0 0) 0x7f309c011810 con 0x7f309006a0e0 2023-08-27T11:13:42.447 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f309006a0e0 msgr2=0x7f309006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f309006a0e0 0x7f309006c5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f309c009b40 tx=0x7f309c009210 comp rx=0 tx=0).stop 2023-08-27T11:13:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 msgr2=0x7f30ac15dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 0x7f30ac15dad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f30a800e840 tx=0x7f30a800eb50 comp rx=0 tx=0).stop 2023-08-27T11:13:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 shutdown_connections 2023-08-27T11:13:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f309006a0e0 0x7f309006c5a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30ac15e990 0x7f30ac0b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30ac0cc650 0x7f30ac152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 --2- 172.21.15.45:0/2177928399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30ac0ce620 0x7f30ac15dad0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 >> 172.21.15.45:0/2177928399 conn(0x7f30ac0d0f60 msgr2=0x7f30ac0cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 shutdown_connections 2023-08-27T11:13:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.445+0000 7f30b34dd700 1 -- 172.21.15.45:0/2177928399 wait complete. 2023-08-27T11:13:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:13:42.459 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2023-08-27T11:13:40.828703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221184,"kb_used_data":5020,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753536,"statfs":{"total":773094113280,"available":772867620864,"internally_reserved":0,"allocated":5140480,"data_stored":2928791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.943092"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931436+0000","last_change":"2023-08-27T11:13:37.882532+0000","last_active":"2023-08-27T11:13:38.931436+0000","last_peered":"2023-08-27T11:13:38.931436+0000","last_clean":"2023-08-27T11:13:38.931436+0000","last_became_active":"2023-08-27T11:13:37.882284+0000","last_became_peered":"2023-08-27T11:13:37.882284+0000","last_unstale":"2023-08-27T11:13:38.931436+0000","last_undegraded":"2023-08-27T11:13:38.931436+0000","last_fullsized":"2023-08-27T11:13:38.931436+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:38:47.084902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869485+0000","last_change":"2023-08-27T11:13:37.889572+0000","last_active":"2023-08-27T11:13:38.869485+0000","last_peered":"2023-08-27T11:13:38.869485+0000","last_clean":"2023-08-27T11:13:38.869485+0000","last_became_active":"2023-08-27T11:13:37.889256+0000","last_became_peered":"2023-08-27T11:13:37.889256+0000","last_unstale":"2023-08-27T11:13:38.869485+0000","last_undegraded":"2023-08-27T11:13:38.869485+0000","last_fullsized":"2023-08-27T11:13:38.869485+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:26:31.927729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869746+0000","last_change":"2023-08-27T11:13:37.880481+0000","last_active":"2023-08-27T11:13:38.869746+0000","last_peered":"2023-08-27T11:13:38.869746+0000","last_clean":"2023-08-27T11:13:38.869746+0000","last_became_active":"2023-08-27T11:13:37.880206+0000","last_became_peered":"2023-08-27T11:13:37.880206+0000","last_unstale":"2023-08-27T11:13:38.869746+0000","last_undegraded":"2023-08-27T11:13:38.869746+0000","last_fullsized":"2023-08-27T11:13:38.869746+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:37:04.899615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932238+0000","last_change":"2023-08-27T11:13:37.885826+0000","last_active":"2023-08-27T11:13:38.932238+0000","last_peered":"2023-08-27T11:13:38.932238+0000","last_clean":"2023-08-27T11:13:38.932238+0000","last_became_active":"2023-08-27T11:13:37.885537+0000","last_became_peered":"2023-08-27T11:13:37.885537+0000","last_unstale":"2023-08-27T11:13:38.932238+0000","last_undegraded":"2023-08-27T11:13:38.932238+0000","last_fullsized":"2023-08-27T11:13:38.932238+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:19:39.179917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869272+0000","last_change":"2023-08-27T11:13:37.883335+0000","last_active":"2023-08-27T11:13:38.869272+0000","last_peered":"2023-08-27T11:13:38.869272+0000","last_clean":"2023-08-27T11:13:38.869272+0000","last_became_active":"2023-08-27T11:13:37.883068+0000","last_became_peered":"2023-08-27T11:13:37.883068+0000","last_unstale":"2023-08-27T11:13:38.869272+0000","last_undegraded":"2023-08-27T11:13:38.869272+0000","last_fullsized":"2023-08-27T11:13:38.869272+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:55.062646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931276+0000","last_change":"2023-08-27T11:13:37.874972+0000","last_active":"2023-08-27T11:13:38.931276+0000","last_peered":"2023-08-27T11:13:38.931276+0000","last_clean":"2023-08-27T11:13:38.931276+0000","last_became_active":"2023-08-27T11:13:37.874482+0000","last_became_peered":"2023-08-27T11:13:37.874482+0000","last_unstale":"2023-08-27T11:13:38.931276+0000","last_undegraded":"2023-08-27T11:13:38.931276+0000","last_fullsized":"2023-08-27T11:13:38.931276+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:57.638778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931815+0000","last_change":"2023-08-27T11:13:37.879022+0000","last_active":"2023-08-27T11:13:38.931815+0000","last_peered":"2023-08-27T11:13:38.931815+0000","last_clean":"2023-08-27T11:13:38.931815+0000","last_became_active":"2023-08-27T11:13:37.878741+0000","last_became_peered":"2023-08-27T11:13:37.878741+0000","last_unstale":"2023-08-27T11:13:38.931815+0000","last_undegraded":"2023-08-27T11:13:38.931815+0000","last_fullsized":"2023-08-27T11:13:38.931815+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:40.235315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930715+0000","last_change":"2023-08-27T11:13:37.887341+0000","last_active":"2023-08-27T11:13:38.930715+0000","last_peered":"2023-08-27T11:13:38.930715+0000","last_clean":"2023-08-27T11:13:38.930715+0000","last_became_active":"2023-08-27T11:13:37.886843+0000","last_became_peered":"2023-08-27T11:13:37.886843+0000","last_unstale":"2023-08-27T11:13:38.930715+0000","last_undegraded":"2023-08-27T11:13:38.930715+0000","last_fullsized":"2023-08-27T11:13:38.930715+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:08:31.228633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932614+0000","last_change":"2023-08-27T11:13:37.881602+0000","last_active":"2023-08-27T11:13:38.932614+0000","last_peered":"2023-08-27T11:13:38.932614+0000","last_clean":"2023-08-27T11:13:38.932614+0000","last_became_active":"2023-08-27T11:13:37.881263+0000","last_became_peered":"2023-08-27T11:13:37.881263+0000","last_unstale":"2023-08-27T11:13:38.932614+0000","last_undegraded":"2023-08-27T11:13:38.932614+0000","last_fullsized":"2023-08-27T11:13:38.932614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:55:11.171065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932575+0000","last_change":"2023-08-27T11:13:37.881923+0000","last_active":"2023-08-27T11:13:38.932575+0000","last_peered":"2023-08-27T11:13:38.932575+0000","last_clean":"2023-08-27T11:13:38.932575+0000","last_became_active":"2023-08-27T11:13:37.880069+0000","last_became_peered":"2023-08-27T11:13:37.880069+0000","last_unstale":"2023-08-27T11:13:38.932575+0000","last_undegraded":"2023-08-27T11:13:38.932575+0000","last_fullsized":"2023-08-27T11:13:38.932575+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:26:38.075458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932287+0000","last_change":"2023-08-27T11:13:37.882289+0000","last_active":"2023-08-27T11:13:38.932287+0000","last_peered":"2023-08-27T11:13:38.932287+0000","last_clean":"2023-08-27T11:13:38.932287+0000","last_became_active":"2023-08-27T11:13:37.880993+0000","last_became_peered":"2023-08-27T11:13:37.880993+0000","last_unstale":"2023-08-27T11:13:38.932287+0000","last_undegraded":"2023-08-27T11:13:38.932287+0000","last_fullsized":"2023-08-27T11:13:38.932287+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:49:09.883091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869719+0000","last_change":"2023-08-27T11:13:37.889837+0000","last_active":"2023-08-27T11:13:38.869719+0000","last_peered":"2023-08-27T11:13:38.869719+0000","last_clean":"2023-08-27T11:13:38.869719+0000","last_became_active":"2023-08-27T11:13:37.889636+0000","last_became_peered":"2023-08-27T11:13:37.889636+0000","last_unstale":"2023-08-27T11:13:38.869719+0000","last_undegraded":"2023-08-27T11:13:38.869719+0000","last_fullsized":"2023-08-27T11:13:38.869719+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:59.902099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932305+0000","last_change":"2023-08-27T11:13:37.887145+0000","last_active":"2023-08-27T11:13:38.932305+0000","last_peered":"2023-08-27T11:13:38.932305+0000","last_clean":"2023-08-27T11:13:38.932305+0000","last_became_active":"2023-08-27T11:13:37.886823+0000","last_became_peered":"2023-08-27T11:13:37.886823+0000","last_unstale":"2023-08-27T11:13:38.932305+0000","last_undegraded":"2023-08-27T11:13:38.932305+0000","last_fullsized":"2023-08-27T11:13:38.932305+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:37.476696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932242+0000","last_change":"2023-08-27T11:13:37.881949+0000","last_active":"2023-08-27T11:13:38.932242+0000","last_peered":"2023-08-27T11:13:38.932242+0000","last_clean":"2023-08-27T11:13:38.932242+0000","last_became_active":"2023-08-27T11:13:37.881657+0000","last_became_peered":"2023-08-27T11:13:37.881657+0000","last_unstale":"2023-08-27T11:13:38.932242+0000","last_undegraded":"2023-08-27T11:13:38.932242+0000","last_fullsized":"2023-08-27T11:13:38.932242+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:13.749860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931370+0000","last_change":"2023-08-27T11:13:37.876910+0000","last_active":"2023-08-27T11:13:38.931370+0000","last_peered":"2023-08-27T11:13:38.931370+0000","last_clean":"2023-08-27T11:13:38.931370+0000","last_became_active":"2023-08-27T11:13:37.876559+0000","last_became_peered":"2023-08-27T11:13:37.876559+0000","last_unstale":"2023-08-27T11:13:38.931370+0000","last_undegraded":"2023-08-27T11:13:38.931370+0000","last_fullsized":"2023-08-27T11:13:38.931370+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:37:35.119222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931631+0000","last_change":"2023-08-27T11:13:37.883263+0000","last_active":"2023-08-27T11:13:38.931631+0000","last_peered":"2023-08-27T11:13:38.931631+0000","last_clean":"2023-08-27T11:13:38.931631+0000","last_became_active":"2023-08-27T11:13:37.882564+0000","last_became_peered":"2023-08-27T11:13:37.882564+0000","last_unstale":"2023-08-27T11:13:38.931631+0000","last_undegraded":"2023-08-27T11:13:38.931631+0000","last_fullsized":"2023-08-27T11:13:38.931631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:32.609911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868802+0000","last_change":"2023-08-27T11:13:37.882840+0000","last_active":"2023-08-27T11:13:38.868802+0000","last_peered":"2023-08-27T11:13:38.868802+0000","last_clean":"2023-08-27T11:13:38.868802+0000","last_became_active":"2023-08-27T11:13:37.882137+0000","last_became_peered":"2023-08-27T11:13:37.882137+0000","last_unstale":"2023-08-27T11:13:38.868802+0000","last_undegraded":"2023-08-27T11:13:38.868802+0000","last_fullsized":"2023-08-27T11:13:38.868802+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:18:11.544766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868614+0000","last_change":"2023-08-27T11:13:37.881510+0000","last_active":"2023-08-27T11:13:38.868614+0000","last_peered":"2023-08-27T11:13:38.868614+0000","last_clean":"2023-08-27T11:13:38.868614+0000","last_became_active":"2023-08-27T11:13:37.881235+0000","last_became_peered":"2023-08-27T11:13:37.881235+0000","last_unstale":"2023-08-27T11:13:38.868614+0000","last_undegraded":"2023-08-27T11:13:38.868614+0000","last_fullsized":"2023-08-27T11:13:38.868614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:53:01.344642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931838+0000","last_change":"2023-08-27T11:13:37.882605+0000","last_active":"2023-08-27T11:13:38.931838+0000","last_peered":"2023-08-27T11:13:38.931838+0000","last_clean":"2023-08-27T11:13:38.931838+0000","last_became_active":"2023-08-27T11:13:37.882302+0000","last_became_peered":"2023-08-27T11:13:37.882302+0000","last_unstale":"2023-08-27T11:13:38.931838+0000","last_undegraded":"2023-08-27T11:13:38.931838+0000","last_fullsized":"2023-08-27T11:13:38.931838+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:52.295625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931728+0000","last_change":"2023-08-27T11:13:37.881684+0000","last_active":"2023-08-27T11:13:38.931728+0000","last_peered":"2023-08-27T11:13:38.931728+0000","last_clean":"2023-08-27T11:13:38.931728+0000","last_became_active":"2023-08-27T11:13:37.880614+0000","last_became_peered":"2023-08-27T11:13:37.880614+0000","last_unstale":"2023-08-27T11:13:38.931728+0000","last_undegraded":"2023-08-27T11:13:38.931728+0000","last_fullsized":"2023-08-27T11:13:38.931728+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:25:13.059199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931291+0000","last_change":"2023-08-27T11:13:37.884201+0000","last_active":"2023-08-27T11:13:38.931291+0000","last_peered":"2023-08-27T11:13:38.931291+0000","last_clean":"2023-08-27T11:13:38.931291+0000","last_became_active":"2023-08-27T11:13:37.883931+0000","last_became_peered":"2023-08-27T11:13:37.883931+0000","last_unstale":"2023-08-27T11:13:38.931291+0000","last_undegraded":"2023-08-27T11:13:38.931291+0000","last_fullsized":"2023-08-27T11:13:38.931291+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:42.147396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932276+0000","last_change":"2023-08-27T11:13:37.886522+0000","last_active":"2023-08-27T11:13:38.932276+0000","last_peered":"2023-08-27T11:13:38.932276+0000","last_clean":"2023-08-27T11:13:38.932276+0000","last_became_active":"2023-08-27T11:13:37.886075+0000","last_became_peered":"2023-08-27T11:13:37.886075+0000","last_unstale":"2023-08-27T11:13:38.932276+0000","last_undegraded":"2023-08-27T11:13:38.932276+0000","last_fullsized":"2023-08-27T11:13:38.932276+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:19:04.859147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931322+0000","last_change":"2023-08-27T11:13:37.886965+0000","last_active":"2023-08-27T11:13:38.931322+0000","last_peered":"2023-08-27T11:13:38.931322+0000","last_clean":"2023-08-27T11:13:38.931322+0000","last_became_active":"2023-08-27T11:13:37.885962+0000","last_became_peered":"2023-08-27T11:13:37.885962+0000","last_unstale":"2023-08-27T11:13:38.931322+0000","last_undegraded":"2023-08-27T11:13:38.931322+0000","last_fullsized":"2023-08-27T11:13:38.931322+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:25.530471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931721+0000","last_change":"2023-08-27T11:13:37.886432+0000","last_active":"2023-08-27T11:13:38.931721+0000","last_peered":"2023-08-27T11:13:38.931721+0000","last_clean":"2023-08-27T11:13:38.931721+0000","last_became_active":"2023-08-27T11:13:37.885541+0000","last_became_peered":"2023-08-27T11:13:37.885541+0000","last_unstale":"2023-08-27T11:13:38.931721+0000","last_undegraded":"2023-08-27T11:13:38.931721+0000","last_fullsized":"2023-08-27T11:13:38.931721+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:25:13.059199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932396+0000","last_change":"2023-08-27T11:13:37.883164+0000","last_active":"2023-08-27T11:13:38.932396+0000","last_peered":"2023-08-27T11:13:38.932396+0000","last_clean":"2023-08-27T11:13:38.932396+0000","last_became_active":"2023-08-27T11:13:37.882617+0000","last_became_peered":"2023-08-27T11:13:37.882617+0000","last_unstale":"2023-08-27T11:13:38.932396+0000","last_undegraded":"2023-08-27T11:13:38.932396+0000","last_fullsized":"2023-08-27T11:13:38.932396+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:46:30.188536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931437+0000","last_change":"2023-08-27T11:13:37.876059+0000","last_active":"2023-08-27T11:13:38.931437+0000","last_peered":"2023-08-27T11:13:38.931437+0000","last_clean":"2023-08-27T11:13:38.931437+0000","last_became_active":"2023-08-27T11:13:37.875735+0000","last_became_peered":"2023-08-27T11:13:37.875735+0000","last_unstale":"2023-08-27T11:13:38.931437+0000","last_undegraded":"2023-08-27T11:13:38.931437+0000","last_fullsized":"2023-08-27T11:13:38.931437+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:26.545550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869848+0000","last_change":"2023-08-27T11:13:37.882598+0000","last_active":"2023-08-27T11:13:38.869848+0000","last_peered":"2023-08-27T11:13:38.869848+0000","last_clean":"2023-08-27T11:13:38.869848+0000","last_became_active":"2023-08-27T11:13:37.881944+0000","last_became_peered":"2023-08-27T11:13:37.881944+0000","last_unstale":"2023-08-27T11:13:38.869848+0000","last_undegraded":"2023-08-27T11:13:38.869848+0000","last_fullsized":"2023-08-27T11:13:38.869848+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:58:06.147584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869787+0000","last_change":"2023-08-27T11:13:37.890892+0000","last_active":"2023-08-27T11:13:38.869787+0000","last_peered":"2023-08-27T11:13:38.869787+0000","last_clean":"2023-08-27T11:13:38.869787+0000","last_became_active":"2023-08-27T11:13:37.890604+0000","last_became_peered":"2023-08-27T11:13:37.890604+0000","last_unstale":"2023-08-27T11:13:38.869787+0000","last_undegraded":"2023-08-27T11:13:38.869787+0000","last_fullsized":"2023-08-27T11:13:38.869787+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:44:13.233649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931797+0000","last_change":"2023-08-27T11:13:37.879212+0000","last_active":"2023-08-27T11:13:38.931797+0000","last_peered":"2023-08-27T11:13:38.931797+0000","last_clean":"2023-08-27T11:13:38.931797+0000","last_became_active":"2023-08-27T11:13:37.878920+0000","last_became_peered":"2023-08-27T11:13:37.878920+0000","last_unstale":"2023-08-27T11:13:38.931797+0000","last_undegraded":"2023-08-27T11:13:38.931797+0000","last_fullsized":"2023-08-27T11:13:38.931797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:15.893609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868518+0000","last_change":"2023-08-27T11:13:37.875015+0000","last_active":"2023-08-27T11:13:38.868518+0000","last_peered":"2023-08-27T11:13:38.868518+0000","last_clean":"2023-08-27T11:13:38.868518+0000","last_became_active":"2023-08-27T11:13:37.874715+0000","last_became_peered":"2023-08-27T11:13:37.874715+0000","last_unstale":"2023-08-27T11:13:38.868518+0000","last_undegraded":"2023-08-27T11:13:38.868518+0000","last_fullsized":"2023-08-27T11:13:38.868518+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:57:24.646763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931798+0000","last_change":"2023-08-27T11:13:37.885957+0000","last_active":"2023-08-27T11:13:38.931798+0000","last_peered":"2023-08-27T11:13:38.931798+0000","last_clean":"2023-08-27T11:13:38.931798+0000","last_became_active":"2023-08-27T11:13:37.885305+0000","last_became_peered":"2023-08-27T11:13:37.885305+0000","last_unstale":"2023-08-27T11:13:38.931798+0000","last_undegraded":"2023-08-27T11:13:38.931798+0000","last_fullsized":"2023-08-27T11:13:38.931798+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:42:35.744996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931122+0000","last_change":"2023-08-27T11:13:37.880114+0000","last_active":"2023-08-27T11:13:38.931122+0000","last_peered":"2023-08-27T11:13:38.931122+0000","last_clean":"2023-08-27T11:13:38.931122+0000","last_became_active":"2023-08-27T11:13:37.879824+0000","last_became_peered":"2023-08-27T11:13:37.879824+0000","last_unstale":"2023-08-27T11:13:38.931122+0000","last_undegraded":"2023-08-27T11:13:38.931122+0000","last_fullsized":"2023-08-27T11:13:38.931122+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:47:25.530471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865467+0000","last_change":"2023-08-27T11:13:37.875343+0000","last_active":"2023-08-27T11:13:38.865467+0000","last_peered":"2023-08-27T11:13:38.865467+0000","last_clean":"2023-08-27T11:13:38.865467+0000","last_became_active":"2023-08-27T11:13:37.874779+0000","last_became_peered":"2023-08-27T11:13:37.874779+0000","last_unstale":"2023-08-27T11:13:38.865467+0000","last_undegraded":"2023-08-27T11:13:38.865467+0000","last_fullsized":"2023-08-27T11:13:38.865467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:47:28.478585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"41'4","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931533+0000","last_change":"2023-08-27T11:13:36.912028+0000","last_active":"2023-08-27T11:13:38.931533+0000","last_peered":"2023-08-27T11:13:38.931533+0000","last_clean":"2023-08-27T11:13:38.931533+0000","last_became_active":"2023-08-27T11:13:36.911755+0000","last_became_peered":"2023-08-27T11:13:36.911755+0000","last_unstale":"2023-08-27T11:13:38.931533+0000","last_undegraded":"2023-08-27T11:13:38.931533+0000","last_fullsized":"2023-08-27T11:13:38.931533+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:49.599460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932252+0000","last_change":"2023-08-27T11:13:37.886004+0000","last_active":"2023-08-27T11:13:38.932252+0000","last_peered":"2023-08-27T11:13:38.932252+0000","last_clean":"2023-08-27T11:13:38.932252+0000","last_became_active":"2023-08-27T11:13:37.885659+0000","last_became_peered":"2023-08-27T11:13:37.885659+0000","last_unstale":"2023-08-27T11:13:38.932252+0000","last_undegraded":"2023-08-27T11:13:38.932252+0000","last_fullsized":"2023-08-27T11:13:38.932252+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:38:37.634080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868807+0000","last_change":"2023-08-27T11:13:36.911917+0000","last_active":"2023-08-27T11:13:38.868807+0000","last_peered":"2023-08-27T11:13:38.868807+0000","last_clean":"2023-08-27T11:13:38.868807+0000","last_became_active":"2023-08-27T11:13:36.911571+0000","last_became_peered":"2023-08-27T11:13:36.911571+0000","last_unstale":"2023-08-27T11:13:38.868807+0000","last_undegraded":"2023-08-27T11:13:38.868807+0000","last_fullsized":"2023-08-27T11:13:38.868807+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:26:47.081246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869535+0000","last_change":"2023-08-27T11:13:37.887490+0000","last_active":"2023-08-27T11:13:38.869535+0000","last_peered":"2023-08-27T11:13:38.869535+0000","last_clean":"2023-08-27T11:13:38.869535+0000","last_became_active":"2023-08-27T11:13:37.887199+0000","last_became_peered":"2023-08-27T11:13:37.887199+0000","last_unstale":"2023-08-27T11:13:38.869535+0000","last_undegraded":"2023-08-27T11:13:38.869535+0000","last_fullsized":"2023-08-27T11:13:38.869535+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:40.235315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865883+0000","last_change":"2023-08-27T11:13:36.903935+0000","last_active":"2023-08-27T11:13:38.865883+0000","last_peered":"2023-08-27T11:13:38.865883+0000","last_clean":"2023-08-27T11:13:38.865883+0000","last_became_active":"2023-08-27T11:13:36.903638+0000","last_became_peered":"2023-08-27T11:13:36.903638+0000","last_unstale":"2023-08-27T11:13:38.865883+0000","last_undegraded":"2023-08-27T11:13:38.865883+0000","last_fullsized":"2023-08-27T11:13:38.865883+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:53:12.743856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865751+0000","last_change":"2023-08-27T11:13:37.884201+0000","last_active":"2023-08-27T11:13:38.865751+0000","last_peered":"2023-08-27T11:13:38.865751+0000","last_clean":"2023-08-27T11:13:38.865751+0000","last_became_active":"2023-08-27T11:13:37.883699+0000","last_became_peered":"2023-08-27T11:13:37.883699+0000","last_unstale":"2023-08-27T11:13:38.865751+0000","last_undegraded":"2023-08-27T11:13:38.865751+0000","last_fullsized":"2023-08-27T11:13:38.865751+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:54:52.214478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931765+0000","last_change":"2023-08-27T11:13:36.905588+0000","last_active":"2023-08-27T11:13:38.931765+0000","last_peered":"2023-08-27T11:13:38.931765+0000","last_clean":"2023-08-27T11:13:38.931765+0000","last_became_active":"2023-08-27T11:13:36.905314+0000","last_became_peered":"2023-08-27T11:13:36.905314+0000","last_unstale":"2023-08-27T11:13:38.931765+0000","last_undegraded":"2023-08-27T11:13:38.931765+0000","last_fullsized":"2023-08-27T11:13:38.931765+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:26:30.921725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868595+0000","last_change":"2023-08-27T11:13:37.884354+0000","last_active":"2023-08-27T11:13:38.868595+0000","last_peered":"2023-08-27T11:13:38.868595+0000","last_clean":"2023-08-27T11:13:38.868595+0000","last_became_active":"2023-08-27T11:13:37.883300+0000","last_became_peered":"2023-08-27T11:13:37.883300+0000","last_unstale":"2023-08-27T11:13:38.868595+0000","last_undegraded":"2023-08-27T11:13:38.868595+0000","last_fullsized":"2023-08-27T11:13:38.868595+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:54:09.677667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931601+0000","last_change":"2023-08-27T11:13:36.910275+0000","last_active":"2023-08-27T11:13:38.931601+0000","last_peered":"2023-08-27T11:13:38.931601+0000","last_clean":"2023-08-27T11:13:38.931601+0000","last_became_active":"2023-08-27T11:13:36.909749+0000","last_became_peered":"2023-08-27T11:13:36.909749+0000","last_unstale":"2023-08-27T11:13:38.931601+0000","last_undegraded":"2023-08-27T11:13:38.931601+0000","last_fullsized":"2023-08-27T11:13:38.931601+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:26:30.921725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932174+0000","last_change":"2023-08-27T11:13:37.881329+0000","last_active":"2023-08-27T11:13:38.932174+0000","last_peered":"2023-08-27T11:13:38.932174+0000","last_clean":"2023-08-27T11:13:38.932174+0000","last_became_active":"2023-08-27T11:13:37.880995+0000","last_became_peered":"2023-08-27T11:13:37.880995+0000","last_unstale":"2023-08-27T11:13:38.932174+0000","last_undegraded":"2023-08-27T11:13:38.932174+0000","last_fullsized":"2023-08-27T11:13:38.932174+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:09:26.514589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868710+0000","last_change":"2023-08-27T11:13:36.896944+0000","last_active":"2023-08-27T11:13:38.868710+0000","last_peered":"2023-08-27T11:13:38.868710+0000","last_clean":"2023-08-27T11:13:38.868710+0000","last_became_active":"2023-08-27T11:13:36.896738+0000","last_became_peered":"2023-08-27T11:13:36.896738+0000","last_unstale":"2023-08-27T11:13:38.868710+0000","last_undegraded":"2023-08-27T11:13:38.868710+0000","last_fullsized":"2023-08-27T11:13:38.868710+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:53:28.903378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931427+0000","last_change":"2023-08-27T11:13:37.880576+0000","last_active":"2023-08-27T11:13:38.931427+0000","last_peered":"2023-08-27T11:13:38.931427+0000","last_clean":"2023-08-27T11:13:38.931427+0000","last_became_active":"2023-08-27T11:13:37.880282+0000","last_became_peered":"2023-08-27T11:13:37.880282+0000","last_unstale":"2023-08-27T11:13:38.931427+0000","last_undegraded":"2023-08-27T11:13:38.931427+0000","last_fullsized":"2023-08-27T11:13:38.931427+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:48:52.370999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869015+0000","last_change":"2023-08-27T11:13:36.906448+0000","last_active":"2023-08-27T11:13:38.869015+0000","last_peered":"2023-08-27T11:13:38.869015+0000","last_clean":"2023-08-27T11:13:38.869015+0000","last_became_active":"2023-08-27T11:13:36.906160+0000","last_became_peered":"2023-08-27T11:13:36.906160+0000","last_unstale":"2023-08-27T11:13:38.869015+0000","last_undegraded":"2023-08-27T11:13:38.869015+0000","last_fullsized":"2023-08-27T11:13:38.869015+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:56.632773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931573+0000","last_change":"2023-08-27T11:13:37.886241+0000","last_active":"2023-08-27T11:13:38.931573+0000","last_peered":"2023-08-27T11:13:38.931573+0000","last_clean":"2023-08-27T11:13:38.931573+0000","last_became_active":"2023-08-27T11:13:37.885733+0000","last_became_peered":"2023-08-27T11:13:37.885733+0000","last_unstale":"2023-08-27T11:13:38.931573+0000","last_undegraded":"2023-08-27T11:13:38.931573+0000","last_fullsized":"2023-08-27T11:13:38.931573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:06.895181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930863+0000","last_change":"2023-08-27T11:13:36.904727+0000","last_active":"2023-08-27T11:13:38.930863+0000","last_peered":"2023-08-27T11:13:38.930863+0000","last_clean":"2023-08-27T11:13:38.930863+0000","last_became_active":"2023-08-27T11:13:36.904424+0000","last_became_peered":"2023-08-27T11:13:36.904424+0000","last_unstale":"2023-08-27T11:13:38.930863+0000","last_undegraded":"2023-08-27T11:13:38.930863+0000","last_fullsized":"2023-08-27T11:13:38.930863+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:47:27.472581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868769+0000","last_change":"2023-08-27T11:13:37.880618+0000","last_active":"2023-08-27T11:13:38.868769+0000","last_peered":"2023-08-27T11:13:38.868769+0000","last_clean":"2023-08-27T11:13:38.868769+0000","last_became_active":"2023-08-27T11:13:37.880308+0000","last_became_peered":"2023-08-27T11:13:37.880308+0000","last_unstale":"2023-08-27T11:13:38.868769+0000","last_undegraded":"2023-08-27T11:13:38.868769+0000","last_fullsized":"2023-08-27T11:13:38.868769+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:07.912152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869025+0000","last_change":"2023-08-27T11:13:36.908652+0000","last_active":"2023-08-27T11:13:38.869025+0000","last_peered":"2023-08-27T11:13:38.869025+0000","last_clean":"2023-08-27T11:13:38.869025+0000","last_became_active":"2023-08-27T11:13:36.908378+0000","last_became_peered":"2023-08-27T11:13:36.908378+0000","last_unstale":"2023-08-27T11:13:38.869025+0000","last_undegraded":"2023-08-27T11:13:38.869025+0000","last_fullsized":"2023-08-27T11:13:38.869025+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:24:36.068626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931576+0000","last_change":"2023-08-27T11:13:37.883817+0000","last_active":"2023-08-27T11:13:38.931576+0000","last_peered":"2023-08-27T11:13:38.931576+0000","last_clean":"2023-08-27T11:13:38.931576+0000","last_became_active":"2023-08-27T11:13:37.883538+0000","last_became_peered":"2023-08-27T11:13:37.883538+0000","last_unstale":"2023-08-27T11:13:38.931576+0000","last_undegraded":"2023-08-27T11:13:38.931576+0000","last_fullsized":"2023-08-27T11:13:38.931576+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:50.605464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932713+0000","last_change":"2023-08-27T11:13:36.905286+0000","last_active":"2023-08-27T11:13:38.932713+0000","last_peered":"2023-08-27T11:13:38.932713+0000","last_clean":"2023-08-27T11:13:38.932713+0000","last_became_active":"2023-08-27T11:13:36.905030+0000","last_became_peered":"2023-08-27T11:13:36.905030+0000","last_unstale":"2023-08-27T11:13:38.932713+0000","last_undegraded":"2023-08-27T11:13:38.932713+0000","last_fullsized":"2023-08-27T11:13:38.932713+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:05:08.549705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931523+0000","last_change":"2023-08-27T11:13:37.877830+0000","last_active":"2023-08-27T11:13:38.931523+0000","last_peered":"2023-08-27T11:13:38.931523+0000","last_clean":"2023-08-27T11:13:38.931523+0000","last_became_active":"2023-08-27T11:13:37.877519+0000","last_became_peered":"2023-08-27T11:13:37.877519+0000","last_unstale":"2023-08-27T11:13:38.931523+0000","last_undegraded":"2023-08-27T11:13:38.931523+0000","last_fullsized":"2023-08-27T11:13:38.931523+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:55.062646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"41'3","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932021+0000","last_change":"2023-08-27T11:13:36.912018+0000","last_active":"2023-08-27T11:13:38.932021+0000","last_peered":"2023-08-27T11:13:38.932021+0000","last_clean":"2023-08-27T11:13:38.932021+0000","last_became_active":"2023-08-27T11:13:36.911743+0000","last_became_peered":"2023-08-27T11:13:36.911743+0000","last_unstale":"2023-08-27T11:13:38.932021+0000","last_undegraded":"2023-08-27T11:13:38.932021+0000","last_fullsized":"2023-08-27T11:13:38.932021+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:44:12.227645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931733+0000","last_change":"2023-08-27T11:13:37.880332+0000","last_active":"2023-08-27T11:13:38.931733+0000","last_peered":"2023-08-27T11:13:38.931733+0000","last_clean":"2023-08-27T11:13:38.931733+0000","last_became_active":"2023-08-27T11:13:37.880046+0000","last_became_peered":"2023-08-27T11:13:37.880046+0000","last_unstale":"2023-08-27T11:13:38.931733+0000","last_undegraded":"2023-08-27T11:13:38.931733+0000","last_fullsized":"2023-08-27T11:13:38.931733+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:02.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"41'6","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931946+0000","last_change":"2023-08-27T11:13:36.911943+0000","last_active":"2023-08-27T11:13:38.931946+0000","last_peered":"2023-08-27T11:13:38.931946+0000","last_clean":"2023-08-27T11:13:38.931946+0000","last_became_active":"2023-08-27T11:13:36.911642+0000","last_became_peered":"2023-08-27T11:13:36.911642+0000","last_unstale":"2023-08-27T11:13:38.931946+0000","last_undegraded":"2023-08-27T11:13:38.931946+0000","last_fullsized":"2023-08-27T11:13:38.931946+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:51.289621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931644+0000","last_change":"2023-08-27T11:13:37.883119+0000","last_active":"2023-08-27T11:13:38.931644+0000","last_peered":"2023-08-27T11:13:38.931644+0000","last_clean":"2023-08-27T11:13:38.931644+0000","last_became_active":"2023-08-27T11:13:37.882868+0000","last_became_peered":"2023-08-27T11:13:37.882868+0000","last_unstale":"2023-08-27T11:13:38.931644+0000","last_undegraded":"2023-08-27T11:13:38.931644+0000","last_fullsized":"2023-08-27T11:13:38.931644+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:02:26.545550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931758+0000","last_change":"2023-08-27T11:13:36.910332+0000","last_active":"2023-08-27T11:13:38.931758+0000","last_peered":"2023-08-27T11:13:38.931758+0000","last_clean":"2023-08-27T11:13:38.931758+0000","last_became_active":"2023-08-27T11:13:36.910032+0000","last_became_peered":"2023-08-27T11:13:36.910032+0000","last_unstale":"2023-08-27T11:13:38.931758+0000","last_undegraded":"2023-08-27T11:13:38.931758+0000","last_fullsized":"2023-08-27T11:13:38.931758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:03.703758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932316+0000","last_change":"2023-08-27T11:13:37.888907+0000","last_active":"2023-08-27T11:13:38.932316+0000","last_peered":"2023-08-27T11:13:38.932316+0000","last_clean":"2023-08-27T11:13:38.932316+0000","last_became_active":"2023-08-27T11:13:37.888600+0000","last_became_peered":"2023-08-27T11:13:37.888600+0000","last_unstale":"2023-08-27T11:13:38.932316+0000","last_undegraded":"2023-08-27T11:13:38.932316+0000","last_fullsized":"2023-08-27T11:13:38.932316+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:46:01.691567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931360+0000","last_change":"2023-08-27T11:13:36.908298+0000","last_active":"2023-08-27T11:13:38.931360+0000","last_peered":"2023-08-27T11:13:38.931360+0000","last_clean":"2023-08-27T11:13:38.931360+0000","last_became_active":"2023-08-27T11:13:36.908063+0000","last_became_peered":"2023-08-27T11:13:36.908063+0000","last_unstale":"2023-08-27T11:13:38.931360+0000","last_undegraded":"2023-08-27T11:13:38.931360+0000","last_fullsized":"2023-08-27T11:13:38.931360+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:37:34.113218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930724+0000","last_change":"2023-08-27T11:13:37.878146+0000","last_active":"2023-08-27T11:13:38.930724+0000","last_peered":"2023-08-27T11:13:38.930724+0000","last_clean":"2023-08-27T11:13:38.930724+0000","last_became_active":"2023-08-27T11:13:37.877835+0000","last_became_peered":"2023-08-27T11:13:37.877835+0000","last_unstale":"2023-08-27T11:13:38.930724+0000","last_undegraded":"2023-08-27T11:13:38.930724+0000","last_fullsized":"2023-08-27T11:13:38.930724+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:13:03.589442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931227+0000","last_change":"2023-08-27T11:13:36.909688+0000","last_active":"2023-08-27T11:13:38.931227+0000","last_peered":"2023-08-27T11:13:38.931227+0000","last_clean":"2023-08-27T11:13:38.931227+0000","last_became_active":"2023-08-27T11:13:36.908823+0000","last_became_peered":"2023-08-27T11:13:36.908823+0000","last_unstale":"2023-08-27T11:13:38.931227+0000","last_undegraded":"2023-08-27T11:13:38.931227+0000","last_fullsized":"2023-08-27T11:13:38.931227+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:13.768824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931906+0000","last_change":"2023-08-27T11:13:37.880286+0000","last_active":"2023-08-27T11:13:38.931906+0000","last_peered":"2023-08-27T11:13:38.931906+0000","last_clean":"2023-08-27T11:13:38.931906+0000","last_became_active":"2023-08-27T11:13:37.879965+0000","last_became_peered":"2023-08-27T11:13:37.879965+0000","last_unstale":"2023-08-27T11:13:38.931906+0000","last_undegraded":"2023-08-27T11:13:38.931906+0000","last_fullsized":"2023-08-27T11:13:38.931906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:59.902099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"41'8","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930987+0000","last_change":"2023-08-27T11:13:36.904679+0000","last_active":"2023-08-27T11:13:38.930987+0000","last_peered":"2023-08-27T11:13:38.930987+0000","last_clean":"2023-08-27T11:13:38.930987+0000","last_became_active":"2023-08-27T11:13:36.904382+0000","last_became_peered":"2023-08-27T11:13:36.904382+0000","last_unstale":"2023-08-27T11:13:38.930987+0000","last_undegraded":"2023-08-27T11:13:38.930987+0000","last_fullsized":"2023-08-27T11:13:38.930987+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:23:25.324897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931575+0000","last_change":"2023-08-27T11:13:37.887091+0000","last_active":"2023-08-27T11:13:38.931575+0000","last_peered":"2023-08-27T11:13:38.931575+0000","last_clean":"2023-08-27T11:13:38.931575+0000","last_became_active":"2023-08-27T11:13:37.886773+0000","last_became_peered":"2023-08-27T11:13:37.886773+0000","last_unstale":"2023-08-27T11:13:38.931575+0000","last_undegraded":"2023-08-27T11:13:38.931575+0000","last_fullsized":"2023-08-27T11:13:38.931575+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:32.609911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931819+0000","last_change":"2023-08-27T11:13:36.910961+0000","last_active":"2023-08-27T11:13:38.931819+0000","last_peered":"2023-08-27T11:13:38.931819+0000","last_clean":"2023-08-27T11:13:38.931819+0000","last_became_active":"2023-08-27T11:13:36.910645+0000","last_became_peered":"2023-08-27T11:13:36.910645+0000","last_unstale":"2023-08-27T11:13:38.931819+0000","last_undegraded":"2023-08-27T11:13:38.931819+0000","last_fullsized":"2023-08-27T11:13:38.931819+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:14.887604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932758+0000","last_change":"2023-08-27T11:13:37.883074+0000","last_active":"2023-08-27T11:13:38.932758+0000","last_peered":"2023-08-27T11:13:38.932758+0000","last_clean":"2023-08-27T11:13:38.932758+0000","last_became_active":"2023-08-27T11:13:37.882584+0000","last_became_peered":"2023-08-27T11:13:37.882584+0000","last_unstale":"2023-08-27T11:13:38.932758+0000","last_undegraded":"2023-08-27T11:13:38.932758+0000","last_fullsized":"2023-08-27T11:13:38.932758+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:11:15.935661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869186+0000","last_change":"2023-08-27T11:13:36.899322+0000","last_active":"2023-08-27T11:13:38.869186+0000","last_peered":"2023-08-27T11:13:38.869186+0000","last_clean":"2023-08-27T11:13:38.869186+0000","last_became_active":"2023-08-27T11:13:36.899044+0000","last_became_peered":"2023-08-27T11:13:36.899044+0000","last_unstale":"2023-08-27T11:13:38.869186+0000","last_undegraded":"2023-08-27T11:13:38.869186+0000","last_fullsized":"2023-08-27T11:13:38.869186+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:42.124218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931330+0000","last_change":"2023-08-27T11:13:37.886414+0000","last_active":"2023-08-27T11:13:38.931330+0000","last_peered":"2023-08-27T11:13:38.931330+0000","last_clean":"2023-08-27T11:13:38.931330+0000","last_became_active":"2023-08-27T11:13:37.886127+0000","last_became_peered":"2023-08-27T11:13:37.886127+0000","last_unstale":"2023-08-27T11:13:38.931330+0000","last_undegraded":"2023-08-27T11:13:38.931330+0000","last_fullsized":"2023-08-27T11:13:38.931330+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:38:32.087877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"41'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869553+0000","last_change":"2023-08-27T11:13:36.900525+0000","last_active":"2023-08-27T11:13:38.869553+0000","last_peered":"2023-08-27T11:13:38.869553+0000","last_clean":"2023-08-27T11:13:38.869553+0000","last_became_active":"2023-08-27T11:13:36.900261+0000","last_became_peered":"2023-08-27T11:13:36.900261+0000","last_unstale":"2023-08-27T11:13:38.869553+0000","last_undegraded":"2023-08-27T11:13:38.869553+0000","last_fullsized":"2023-08-27T11:13:38.869553+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:03.703758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932025+0000","last_change":"2023-08-27T11:13:37.880950+0000","last_active":"2023-08-27T11:13:38.932025+0000","last_peered":"2023-08-27T11:13:38.932025+0000","last_clean":"2023-08-27T11:13:38.932025+0000","last_became_active":"2023-08-27T11:13:37.880680+0000","last_became_peered":"2023-08-27T11:13:37.880680+0000","last_unstale":"2023-08-27T11:13:38.932025+0000","last_undegraded":"2023-08-27T11:13:38.932025+0000","last_fullsized":"2023-08-27T11:13:38.932025+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:37.476696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931570+0000","last_change":"2023-08-27T11:13:36.913330+0000","last_active":"2023-08-27T11:13:38.931570+0000","last_peered":"2023-08-27T11:13:38.931570+0000","last_clean":"2023-08-27T11:13:38.931570+0000","last_became_active":"2023-08-27T11:13:36.913018+0000","last_became_peered":"2023-08-27T11:13:36.913018+0000","last_unstale":"2023-08-27T11:13:38.931570+0000","last_undegraded":"2023-08-27T11:13:38.931570+0000","last_fullsized":"2023-08-27T11:13:38.931570+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:01.408597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931072+0000","last_change":"2023-08-27T11:13:37.878052+0000","last_active":"2023-08-27T11:13:38.931072+0000","last_peered":"2023-08-27T11:13:38.931072+0000","last_clean":"2023-08-27T11:13:38.931072+0000","last_became_active":"2023-08-27T11:13:37.877780+0000","last_became_peered":"2023-08-27T11:13:37.877780+0000","last_unstale":"2023-08-27T11:13:38.931072+0000","last_undegraded":"2023-08-27T11:13:38.931072+0000","last_fullsized":"2023-08-27T11:13:38.931072+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:34:03.294902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931762+0000","last_change":"2023-08-27T11:13:36.910074+0000","last_active":"2023-08-27T11:13:38.931762+0000","last_peered":"2023-08-27T11:13:38.931762+0000","last_clean":"2023-08-27T11:13:38.931762+0000","last_became_active":"2023-08-27T11:13:36.908889+0000","last_became_peered":"2023-08-27T11:13:36.908889+0000","last_unstale":"2023-08-27T11:13:38.931762+0000","last_undegraded":"2023-08-27T11:13:38.931762+0000","last_fullsized":"2023-08-27T11:13:38.931762+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:58.896094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865528+0000","last_change":"2023-08-27T11:13:37.882625+0000","last_active":"2023-08-27T11:13:38.865528+0000","last_peered":"2023-08-27T11:13:38.865528+0000","last_clean":"2023-08-27T11:13:38.865528+0000","last_became_active":"2023-08-27T11:13:37.882339+0000","last_became_peered":"2023-08-27T11:13:37.882339+0000","last_unstale":"2023-08-27T11:13:38.865528+0000","last_undegraded":"2023-08-27T11:13:38.865528+0000","last_fullsized":"2023-08-27T11:13:38.865528+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:57.229171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869230+0000","last_change":"2023-08-27T11:13:36.911780+0000","last_active":"2023-08-27T11:13:38.869230+0000","last_peered":"2023-08-27T11:13:38.869230+0000","last_clean":"2023-08-27T11:13:38.869230+0000","last_became_active":"2023-08-27T11:13:36.911506+0000","last_became_peered":"2023-08-27T11:13:36.911506+0000","last_unstale":"2023-08-27T11:13:38.869230+0000","last_undegraded":"2023-08-27T11:13:38.869230+0000","last_fullsized":"2023-08-27T11:13:38.869230+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:37:34.113218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931248+0000","last_change":"2023-08-27T11:13:37.875570+0000","last_active":"2023-08-27T11:13:38.931248+0000","last_peered":"2023-08-27T11:13:38.931248+0000","last_clean":"2023-08-27T11:13:38.931248+0000","last_became_active":"2023-08-27T11:13:37.875255+0000","last_became_peered":"2023-08-27T11:13:37.875255+0000","last_unstale":"2023-08-27T11:13:38.931248+0000","last_undegraded":"2023-08-27T11:13:38.931248+0000","last_fullsized":"2023-08-27T11:13:38.931248+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:06.895181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930860+0000","last_change":"2023-08-27T11:13:36.907510+0000","last_active":"2023-08-27T11:13:38.930860+0000","last_peered":"2023-08-27T11:13:38.930860+0000","last_clean":"2023-08-27T11:13:38.930860+0000","last_became_active":"2023-08-27T11:13:36.907218+0000","last_became_peered":"2023-08-27T11:13:36.907218+0000","last_unstale":"2023-08-27T11:13:38.930860+0000","last_undegraded":"2023-08-27T11:13:38.930860+0000","last_fullsized":"2023-08-27T11:13:38.930860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:56.223167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931602+0000","last_change":"2023-08-27T11:13:37.879236+0000","last_active":"2023-08-27T11:13:38.931602+0000","last_peered":"2023-08-27T11:13:38.931602+0000","last_clean":"2023-08-27T11:13:38.931602+0000","last_became_active":"2023-08-27T11:13:37.878942+0000","last_became_peered":"2023-08-27T11:13:37.878942+0000","last_unstale":"2023-08-27T11:13:38.931602+0000","last_undegraded":"2023-08-27T11:13:38.931602+0000","last_fullsized":"2023-08-27T11:13:38.931602+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:54:52.214478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932318+0000","last_change":"2023-08-27T11:13:36.908297+0000","last_active":"2023-08-27T11:13:38.932318+0000","last_peered":"2023-08-27T11:13:38.932318+0000","last_clean":"2023-08-27T11:13:38.932318+0000","last_became_active":"2023-08-27T11:13:36.908005+0000","last_became_peered":"2023-08-27T11:13:36.908005+0000","last_unstale":"2023-08-27T11:13:38.932318+0000","last_undegraded":"2023-08-27T11:13:38.932318+0000","last_fullsized":"2023-08-27T11:13:38.932318+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:57:13.418325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868886+0000","last_change":"2023-08-27T11:13:37.884316+0000","last_active":"2023-08-27T11:13:38.868886+0000","last_peered":"2023-08-27T11:13:38.868886+0000","last_clean":"2023-08-27T11:13:38.868886+0000","last_became_active":"2023-08-27T11:13:37.883967+0000","last_became_peered":"2023-08-27T11:13:37.883967+0000","last_unstale":"2023-08-27T11:13:38.868886+0000","last_undegraded":"2023-08-27T11:13:38.868886+0000","last_fullsized":"2023-08-27T11:13:38.868886+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:17:21.632858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931107+0000","last_change":"2023-08-27T11:13:36.909242+0000","last_active":"2023-08-27T11:13:38.931107+0000","last_peered":"2023-08-27T11:13:38.931107+0000","last_clean":"2023-08-27T11:13:38.931107+0000","last_became_active":"2023-08-27T11:13:36.908797+0000","last_became_peered":"2023-08-27T11:13:36.908797+0000","last_unstale":"2023-08-27T11:13:38.931107+0000","last_undegraded":"2023-08-27T11:13:38.931107+0000","last_fullsized":"2023-08-27T11:13:38.931107+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:50:48.793983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865671+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:13:38.865671+0000","last_peered":"2023-08-27T11:13:38.865671+0000","last_clean":"2023-08-27T11:13:38.865671+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:13:38.865671+0000","last_undegraded":"2023-08-27T11:13:38.865671+0000","last_fullsized":"2023-08-27T11:13:38.865671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:17.340508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932256+0000","last_change":"2023-08-27T11:13:37.883343+0000","last_active":"2023-08-27T11:13:38.932256+0000","last_peered":"2023-08-27T11:13:38.932256+0000","last_clean":"2023-08-27T11:13:38.932256+0000","last_became_active":"2023-08-27T11:13:37.882695+0000","last_became_peered":"2023-08-27T11:13:37.882695+0000","last_unstale":"2023-08-27T11:13:38.932256+0000","last_undegraded":"2023-08-27T11:13:38.932256+0000","last_fullsized":"2023-08-27T11:13:38.932256+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:03:31.199993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:39.003152+0000","last_change":"2023-08-27T11:13:36.906247+0000","last_active":"2023-08-27T11:13:39.003152+0000","last_peered":"2023-08-27T11:13:39.003152+0000","last_clean":"2023-08-27T11:13:39.003152+0000","last_became_active":"2023-08-27T11:13:36.905971+0000","last_became_peered":"2023-08-27T11:13:36.905971+0000","last_unstale":"2023-08-27T11:13:39.003152+0000","last_undegraded":"2023-08-27T11:13:39.003152+0000","last_fullsized":"2023-08-27T11:13:39.003152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:49.547584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869134+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:13:38.869134+0000","last_peered":"2023-08-27T11:13:38.869134+0000","last_clean":"2023-08-27T11:13:38.869134+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:13:38.869134+0000","last_undegraded":"2023-08-27T11:13:38.869134+0000","last_fullsized":"2023-08-27T11:13:38.869134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:37:12.054411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932390+0000","last_change":"2023-08-27T11:13:37.883076+0000","last_active":"2023-08-27T11:13:38.932390+0000","last_peered":"2023-08-27T11:13:38.932390+0000","last_clean":"2023-08-27T11:13:38.932390+0000","last_became_active":"2023-08-27T11:13:37.882793+0000","last_became_peered":"2023-08-27T11:13:37.882793+0000","last_unstale":"2023-08-27T11:13:38.932390+0000","last_undegraded":"2023-08-27T11:13:38.932390+0000","last_fullsized":"2023-08-27T11:13:38.932390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:01.061977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932139+0000","last_change":"2023-08-27T11:13:36.912325+0000","last_active":"2023-08-27T11:13:38.932139+0000","last_peered":"2023-08-27T11:13:38.932139+0000","last_clean":"2023-08-27T11:13:38.932139+0000","last_became_active":"2023-08-27T11:13:36.912054+0000","last_became_peered":"2023-08-27T11:13:36.912054+0000","last_unstale":"2023-08-27T11:13:38.932139+0000","last_undegraded":"2023-08-27T11:13:38.932139+0000","last_fullsized":"2023-08-27T11:13:38.932139+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:58:28.510547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931172+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:13:38.931172+0000","last_peered":"2023-08-27T11:13:38.931172+0000","last_clean":"2023-08-27T11:13:38.931172+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:13:38.931172+0000","last_undegraded":"2023-08-27T11:13:38.931172+0000","last_fullsized":"2023-08-27T11:13:38.931172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:57.057385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932653+0000","last_change":"2023-08-27T11:13:37.888938+0000","last_active":"2023-08-27T11:13:38.932653+0000","last_peered":"2023-08-27T11:13:38.932653+0000","last_clean":"2023-08-27T11:13:38.932653+0000","last_became_active":"2023-08-27T11:13:37.888625+0000","last_became_peered":"2023-08-27T11:13:37.888625+0000","last_unstale":"2023-08-27T11:13:38.932653+0000","last_undegraded":"2023-08-27T11:13:38.932653+0000","last_fullsized":"2023-08-27T11:13:38.932653+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:11:17.886182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"41'2","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869440+0000","last_change":"2023-08-27T11:13:36.898372+0000","last_active":"2023-08-27T11:13:38.869440+0000","last_peered":"2023-08-27T11:13:38.869440+0000","last_clean":"2023-08-27T11:13:38.869440+0000","last_became_active":"2023-08-27T11:13:36.898044+0000","last_became_peered":"2023-08-27T11:13:36.898044+0000","last_unstale":"2023-08-27T11:13:38.869440+0000","last_undegraded":"2023-08-27T11:13:38.869440+0000","last_fullsized":"2023-08-27T11:13:38.869440+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:12:49.599460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869367+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:13:38.869367+0000","last_peered":"2023-08-27T11:13:38.869367+0000","last_clean":"2023-08-27T11:13:38.869367+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:13:38.869367+0000","last_undegraded":"2023-08-27T11:13:38.869367+0000","last_fullsized":"2023-08-27T11:13:38.869367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:18:27.384111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931193+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:13:38.931193+0000","last_peered":"2023-08-27T11:13:38.931193+0000","last_clean":"2023-08-27T11:13:38.931193+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:13:38.931193+0000","last_undegraded":"2023-08-27T11:13:38.931193+0000","last_fullsized":"2023-08-27T11:13:38.931193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:08:00.473665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931874+0000","last_change":"2023-08-27T11:13:37.880908+0000","last_active":"2023-08-27T11:13:38.931874+0000","last_peered":"2023-08-27T11:13:38.931874+0000","last_clean":"2023-08-27T11:13:38.931874+0000","last_became_active":"2023-08-27T11:13:37.880615+0000","last_became_peered":"2023-08-27T11:13:37.880615+0000","last_unstale":"2023-08-27T11:13:38.931874+0000","last_undegraded":"2023-08-27T11:13:38.931874+0000","last_fullsized":"2023-08-27T11:13:38.931874+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:26:04.709762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865626+0000","last_change":"2023-08-27T11:13:36.905955+0000","last_active":"2023-08-27T11:13:38.865626+0000","last_peered":"2023-08-27T11:13:38.865626+0000","last_clean":"2023-08-27T11:13:38.865626+0000","last_became_active":"2023-08-27T11:13:36.905614+0000","last_became_peered":"2023-08-27T11:13:36.905614+0000","last_unstale":"2023-08-27T11:13:38.865626+0000","last_undegraded":"2023-08-27T11:13:38.865626+0000","last_fullsized":"2023-08-27T11:13:38.865626+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:23:25.324897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.930880+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:13:38.930880+0000","last_peered":"2023-08-27T11:13:38.930880+0000","last_clean":"2023-08-27T11:13:38.930880+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:13:38.930880+0000","last_undegraded":"2023-08-27T11:13:38.930880+0000","last_fullsized":"2023-08-27T11:13:38.930880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:27:22.608286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.868962+0000","last_change":"2023-08-27T11:13:37.884622+0000","last_active":"2023-08-27T11:13:38.868962+0000","last_peered":"2023-08-27T11:13:38.868962+0000","last_clean":"2023-08-27T11:13:38.868962+0000","last_became_active":"2023-08-27T11:13:37.884372+0000","last_became_peered":"2023-08-27T11:13:37.884372+0000","last_unstale":"2023-08-27T11:13:38.868962+0000","last_undegraded":"2023-08-27T11:13:38.868962+0000","last_fullsized":"2023-08-27T11:13:38.868962+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:42.147396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865921+0000","last_change":"2023-08-27T11:13:36.912357+0000","last_active":"2023-08-27T11:13:38.865921+0000","last_peered":"2023-08-27T11:13:38.865921+0000","last_clean":"2023-08-27T11:13:38.865921+0000","last_became_active":"2023-08-27T11:13:36.912075+0000","last_became_peered":"2023-08-27T11:13:36.912075+0000","last_unstale":"2023-08-27T11:13:38.865921+0000","last_undegraded":"2023-08-27T11:13:38.865921+0000","last_fullsized":"2023-08-27T11:13:38.865921+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:08:30.222629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932213+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:13:38.932213+0000","last_peered":"2023-08-27T11:13:38.932213+0000","last_clean":"2023-08-27T11:13:38.932213+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:13:38.932213+0000","last_undegraded":"2023-08-27T11:13:38.932213+0000","last_fullsized":"2023-08-27T11:13:38.932213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:27:49.048827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931619+0000","last_change":"2023-08-27T11:13:37.876107+0000","last_active":"2023-08-27T11:13:38.931619+0000","last_peered":"2023-08-27T11:13:38.931619+0000","last_clean":"2023-08-27T11:13:38.931619+0000","last_became_active":"2023-08-27T11:13:37.875821+0000","last_became_peered":"2023-08-27T11:13:37.875821+0000","last_unstale":"2023-08-27T11:13:38.931619+0000","last_undegraded":"2023-08-27T11:13:38.931619+0000","last_fullsized":"2023-08-27T11:13:38.931619+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:08:40.235315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.869389+0000","last_change":"2023-08-27T11:13:36.903987+0000","last_active":"2023-08-27T11:13:38.869389+0000","last_peered":"2023-08-27T11:13:38.869389+0000","last_clean":"2023-08-27T11:13:38.869389+0000","last_became_active":"2023-08-27T11:13:36.903637+0000","last_became_peered":"2023-08-27T11:13:36.903637+0000","last_unstale":"2023-08-27T11:13:38.869389+0000","last_undegraded":"2023-08-27T11:13:38.869389+0000","last_fullsized":"2023-08-27T11:13:38.869389+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:23.977411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.865608+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:13:38.865608+0000","last_peered":"2023-08-27T11:13:38.865608+0000","last_clean":"2023-08-27T11:13:38.865608+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:13:38.865608+0000","last_undegraded":"2023-08-27T11:13:38.865608+0000","last_fullsized":"2023-08-27T11:13:38.865608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:49:14.769496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931421+0000","last_change":"2023-08-27T11:13:37.882626+0000","last_active":"2023-08-27T11:13:38.931421+0000","last_peered":"2023-08-27T11:13:38.931421+0000","last_clean":"2023-08-27T11:13:38.931421+0000","last_became_active":"2023-08-27T11:13:37.882315+0000","last_became_peered":"2023-08-27T11:13:37.882315+0000","last_unstale":"2023-08-27T11:13:38.931421+0000","last_undegraded":"2023-08-27T11:13:38.931421+0000","last_fullsized":"2023-08-27T11:13:38.931421+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:36.884757+0000","last_clean_scrub_stamp":"2023-08-27T11:13:36.884757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:38:47.084902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.931471+0000","last_change":"2023-08-27T11:13:36.913330+0000","last_active":"2023-08-27T11:13:38.931471+0000","last_peered":"2023-08-27T11:13:38.931471+0000","last_clean":"2023-08-27T11:13:38.931471+0000","last_became_active":"2023-08-27T11:13:36.913062+0000","last_became_peered":"2023-08-27T11:13:36.913062+0000","last_unstale":"2023-08-27T11:13:38.931471+0000","last_undegraded":"2023-08-27T11:13:38.931471+0000","last_fullsized":"2023-08-27T11:13:38.931471+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:13:35.878753+0000","last_clean_scrub_stamp":"2023-08-27T11:13:35.878753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:54.056642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-08-27T11:13:38.932418+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:13:38.932418+0000","last_peered":"2023-08-27T11:13:38.932418+0000","last_clean":"2023-08-27T11:13:38.932418+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:13:38.932418+0000","last_undegraded":"2023-08-27T11:13:38.932418+0000","last_fullsized":"2023-08-27T11:13:38.932418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:45.131364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":809234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.495},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.59599999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65000000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.67100000000000004}]},{"osd":3,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.627},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41499999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.38700000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.36699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":219448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.50900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.3780000000000001,"5min":3.3780000000000001,"15min":3.3780000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.3490000000000002,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.57999999999999996}]},{"osd":2,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":9.3770000000000007,"5min":9.3770000000000007,"15min":9.3770000000000007},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.55200000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":8.3390000000000004,"5min":8.3390000000000004,"15min":8.3390000000000004},"last":0.748},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.72999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.77600000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.64400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":218846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41599999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.438}]},{"osd":4,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58999999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.622},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1073152,"data_stored":802205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.33100000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.252},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":8.1349999999999998,"5min":8.1349999999999998,"15min":8.1349999999999998},"last":0.30099999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.026999999999999,"5min":11.026999999999999,"15min":11.026999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":11.926,"5min":11.926,"15min":11.926},"last":0.35599999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":8.0779999999999994,"5min":8.0779999999999994,"15min":8.0779999999999994},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":10.922000000000001,"5min":10.922000000000001,"15min":10.922000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":8.0380000000000003,"5min":8.0380000000000003,"15min":8.0380000000000003},"last":0.55800000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:12:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":8.1069999999999993,"5min":8.1069999999999993,"15min":8.1069999999999993},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":10.996,"5min":10.996,"15min":10.996},"last":0.53600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":220410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.28799999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.307},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.24099999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.37},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.626}]},{"osd":5,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.496}]},{"osd":7,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.51900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":499712,"data_stored":219414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":11.885999999999999,"5min":11.885999999999999,"15min":11.885999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.317}]},{"osd":1,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":6.0830000000000002,"5min":6.0830000000000002,"15min":6.0830000000000002},"last":0.255},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":12.050000000000001,"5min":12.050000000000001,"15min":12.050000000000001},"last":0.36199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":11.939,"5min":11.939,"15min":11.939},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":11.007,"5min":11.007,"15min":11.007},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.624}]},{"osd":5,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":11.981,"5min":11.981,"15min":11.981},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":11.09,"5min":11.09,"15min":11.09},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":12.789999999999999,"5min":12.789999999999999,"15min":12.789999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":12.016,"5min":12.016,"15min":12.016},"last":0.72199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:12:57 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":11.054,"5min":11.054,"15min":11.054},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.59799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":499712,"data_stored":220388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41599999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55000000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.83699999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.85099999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":218846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67100000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65500000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68600000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70799999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56200000000000006}]},{"osd":6,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46600000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.434}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:13:42.466 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:13:42.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:13:42.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.630+0000 7fdcc0058700 1 Processor -- start 2023-08-27T11:13:42.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.631+0000 7fdcc0058700 1 -- start start 2023-08-27T11:13:42.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.631+0000 7fdcc0058700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdcb815d640 0x7fdcb80d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.631+0000 7fdcc0058700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdcb80b5170 0x7fdcb80b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcc0058700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb80d2a20 con 0x7fdcb80d3ad0 2023-08-27T11:13:42.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcc0058700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb815da20 con 0x7fdcb80b5170 2023-08-27T11:13:42.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcc0058700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb80d1e10 con 0x7fdcb815d640 2023-08-27T11:13:42.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbe5f5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdcb80b5170 0x7fdcb80b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbd5f3700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdcb80d3ad0 0x7fdcb80b4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41040/0 (socket says 172.21.15.45:41040) 2023-08-27T11:13:42.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbd5f3700 1 -- 172.21.15.45:0/2227087286 learned_addr learned my addr 172.21.15.45:0/2227087286 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:42.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbddf4700 1 --2- 172.21.15.45:0/2227087286 >> v2:172.21.15.135:3300/0 conn(0x7fdcb815d640 0x7fdcb80d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbe5f5700 1 -- 172.21.15.45:0/2227087286 >> v2:172.21.15.135:3300/0 conn(0x7fdcb815d640 msgr2=0x7fdcb80d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:42.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbe5f5700 1 --2- 172.21.15.45:0/2227087286 >> v2:172.21.15.135:3300/0 conn(0x7fdcb815d640 0x7fdcb80d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbcdf2700 1 -- 172.21.15.45:0/2227087286 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147169265 0 0) 0x7fdcb80d2a20 con 0x7fdcb80d3ad0 2023-08-27T11:13:42.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbe5f5700 1 -- 172.21.15.45:0/2227087286 >> v1:172.21.15.45:6789/0 conn(0x7fdcb80d3ad0 legacy=0x7fdcb80b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbe5f5700 1 -- 172.21.15.45:0/2227087286 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80d20f0 con 0x7fdcb80b5170 2023-08-27T11:13:42.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbe5f5700 1 --2- 172.21.15.45:0/2227087286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 0x7fdcb80b6860 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdcb806a4a0 tx=0x7fdcb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32d56ddde922fb0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbcdf2700 1 -- 172.21.15.45:0/2227087286 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb000e070 con 0x7fdcb80b5170 2023-08-27T11:13:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbcdf2700 1 -- 172.21.15.45:0/2227087286 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb000bbd0 con 0x7fdcb80b5170 2023-08-27T11:13:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.632+0000 7fdcbcdf2700 1 -- 172.21.15.45:0/2227087286 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0012430 con 0x7fdcb80b5170 2023-08-27T11:13:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.633+0000 7fdcc0058700 1 -- 172.21.15.45:0/2227087286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 msgr2=0x7fdcb80b6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.633+0000 7fdcc0058700 1 --2- 172.21.15.45:0/2227087286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 0x7fdcb80b6860 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdcb806a4a0 tx=0x7fdcb000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:42.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.633+0000 7fdcc0058700 1 -- 172.21.15.45:0/2227087286 shutdown_connections 2023-08-27T11:13:42.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.633+0000 7fdcc0058700 1 --2- 172.21.15.45:0/2227087286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 0x7fdcb80b6860 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.633+0000 7fdcc0058700 1 --2- 172.21.15.45:0/2227087286 >> v2:172.21.15.135:3300/0 conn(0x7fdcb815d640 0x7fdcb80d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.633+0000 7fdcc0058700 1 -- 172.21.15.45:0/2227087286 >> 172.21.15.45:0/2227087286 conn(0x7fdcb80d2fe0 msgr2=0x7fdcb80d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:42.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.633+0000 7fdcc0058700 1 -- 172.21.15.45:0/2227087286 shutdown_connections 2023-08-27T11:13:42.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.633+0000 7fdcc0058700 1 -- 172.21.15.45:0/2227087286 wait complete. 2023-08-27T11:13:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.634+0000 7fdcc0058700 1 Processor -- start 2023-08-27T11:13:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.634+0000 7fdcc0058700 1 -- start start 2023-08-27T11:13:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcc0058700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 0x7fdcb815e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcc0058700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcb80d3ad0 0x7fdcb8152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcc0058700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcb815d640 0x7fdcb81600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcc0058700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c7d50 con 0x7fdcb80b5170 2023-08-27T11:13:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcc0058700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c70d0 con 0x7fdcb80d3ad0 2023-08-27T11:13:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcc0058700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c73b0 con 0x7fdcb815d640 2023-08-27T11:13:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbddf4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 0x7fdcb815e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbddf4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 0x7fdcb815e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54268/0 (socket says 172.21.15.45:54268) 2023-08-27T11:13:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbddf4700 1 -- 172.21.15.45:0/2380732486 learned_addr learned my addr 172.21.15.45:0/2380732486 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbd5f3700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcb80d3ad0 0x7fdcb8152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbe5f5700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcb815d640 0x7fdcb81600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbddf4700 1 -- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcb815d640 msgr2=0x7fdcb81600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbddf4700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcb815d640 0x7fdcb81600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbddf4700 1 -- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcb80d3ad0 msgr2=0x7fdcb8152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbddf4700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcb80d3ad0 0x7fdcb8152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.635+0000 7fdcbddf4700 1 -- 172.21.15.45:0/2380732486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb0009a70 con 0x7fdcb80b5170 2023-08-27T11:13:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.636+0000 7fdcbddf4700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 0x7fdcb815e550 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c7f0 tx=0x7fdca800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.636+0000 7fdcae7fc700 1 -- 172.21.15.45:0/2380732486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8011800 con 0x7fdcb80b5170 2023-08-27T11:13:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.636+0000 7fdcae7fc700 1 -- 172.21.15.45:0/2380732486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca8011960 con 0x7fdcb80b5170 2023-08-27T11:13:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.636+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb80c7690 con 0x7fdcb80b5170 2023-08-27T11:13:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.636+0000 7fdcae7fc700 1 -- 172.21.15.45:0/2380732486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8011cf0 con 0x7fdcb80b5170 2023-08-27T11:13:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.636+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb815ea90 con 0x7fdcb80b5170 2023-08-27T11:13:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.637+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb80cd730 con 0x7fdcb80b5170 2023-08-27T11:13:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.637+0000 7fdcae7fc700 1 -- 172.21.15.45:0/2380732486 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdca8018360 con 0x7fdcb80b5170 2023-08-27T11:13:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.638+0000 7fdcae7fc700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc9c06a0e0 0x7fdc9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.638+0000 7fdcae7fc700 1 -- 172.21.15.45:0/2380732486 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdca8088620 con 0x7fdcb80b5170 2023-08-27T11:13:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.638+0000 7fdcbd5f3700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc9c06a0e0 0x7fdc9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:42.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.638+0000 7fdcbd5f3700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc9c06a0e0 0x7fdc9c06c5a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdcb81567e0 tx=0x7fdcb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:42.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.642+0000 7fdcae7fc700 1 -- 172.21.15.45:0/2380732486 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdca8053bd0 con 0x7fdcb80b5170 2023-08-27T11:13:42.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.929+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdcb80734a0 con 0x7fdcb80b5170 2023-08-27T11:13:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.933+0000 7fdcae7fc700 1 -- 172.21.15.45:0/2380732486 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdca80571f0 con 0x7fdcb80b5170 2023-08-27T11:13:42.935 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-08-27T11:13:42.935 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ== 2023-08-27T11:13:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2023-08-27T11:13:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2023-08-27T11:13:42.936 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:13:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc9c06a0e0 msgr2=0x7fdc9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc9c06a0e0 0x7fdc9c06c5a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdcb81567e0 tx=0x7fdcb4009040 comp rx=0 tx=0).stop 2023-08-27T11:13:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 msgr2=0x7fdcb815e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 0x7fdcb815e550 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c7f0 tx=0x7fdca800cbb0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 shutdown_connections 2023-08-27T11:13:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcb80b5170 0x7fdcb815e550 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fdc9c06a0e0 0x7fdc9c06c5a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcb815d640 0x7fdcb81600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 --2- 172.21.15.45:0/2380732486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcb80d3ad0 0x7fdcb8152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 >> 172.21.15.45:0/2380732486 conn(0x7fdcb80d2fe0 msgr2=0x7fdcb80d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 shutdown_connections 2023-08-27T11:13:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:42.935+0000 7fdcc0058700 1 -- 172.21.15.45:0/2380732486 wait complete. 2023-08-27T11:13:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-08-27T11:13:42.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:13:43.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.076+0000 7f2b6d6bc700 1 Processor -- start 2023-08-27T11:13:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.076+0000 7f2b6d6bc700 1 -- start start 2023-08-27T11:13:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.077+0000 7f2b6d6bc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2b6815d0f0 0x7f2b68160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.077+0000 7f2b6d6bc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b680d00e0 0x7f2b680d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.077+0000 7f2b6d6bc700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b680b5fc0 con 0x7f2b680b8cf0 2023-08-27T11:13:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.077+0000 7f2b6d6bc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b68161c30 con 0x7f2b680d00e0 2023-08-27T11:13:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.077+0000 7f2b6d6bc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b680d1430 con 0x7f2b6815d0f0 2023-08-27T11:13:43.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.077+0000 7f2b677fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b680d00e0 0x7f2b680d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b677fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b680d00e0 0x7f2b680d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54280/0 (socket says 172.21.15.45:54280) 2023-08-27T11:13:43.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b677fe700 1 -- 172.21.15.45:0/177749093 learned_addr learned my addr 172.21.15.45:0/177749093 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:43.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b66ffd700 1 --2- 172.21.15.45:0/177749093 >> v2:172.21.15.135:3300/0 conn(0x7f2b6815d0f0 0x7f2b68160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b677fe700 1 -- 172.21.15.45:0/177749093 >> v2:172.21.15.135:3300/0 conn(0x7f2b6815d0f0 msgr2=0x7f2b68160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:43.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b677fe700 1 --2- 172.21.15.45:0/177749093 >> v2:172.21.15.135:3300/0 conn(0x7f2b6815d0f0 0x7f2b68160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b677fe700 1 -- 172.21.15.45:0/177749093 >> v1:172.21.15.135:6789/0 conn(0x7f2b680b8cf0 legacy=0x7f2b680cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b677fe700 1 -- 172.21.15.45:0/177749093 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68161670 con 0x7f2b680d00e0 2023-08-27T11:13:43.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b677fe700 1 --2- 172.21.15.45:0/177749093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680d00e0 0x7f2b680d18c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806a9b0 tx=0x7f2b6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35eff59162072b2f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b65ffb700 1 -- 172.21.15.45:0/177749093 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000e070 con 0x7f2b680d00e0 2023-08-27T11:13:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b65ffb700 1 -- 172.21.15.45:0/177749093 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6000b9c0 con 0x7f2b680d00e0 2023-08-27T11:13:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.078+0000 7f2b65ffb700 1 -- 172.21.15.45:0/177749093 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000bd70 con 0x7f2b680d00e0 2023-08-27T11:13:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.079+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/177749093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680d00e0 msgr2=0x7f2b680d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.079+0000 7f2b6d6bc700 1 --2- 172.21.15.45:0/177749093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680d00e0 0x7f2b680d18c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2b6806a9b0 tx=0x7f2b6000b330 comp rx=0 tx=0).stop 2023-08-27T11:13:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.079+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/177749093 shutdown_connections 2023-08-27T11:13:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.079+0000 7f2b6d6bc700 1 --2- 172.21.15.45:0/177749093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680d00e0 0x7f2b680d18c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.079+0000 7f2b6d6bc700 1 --2- 172.21.15.45:0/177749093 >> v2:172.21.15.135:3300/0 conn(0x7f2b6815d0f0 0x7f2b68160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.079+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/177749093 >> 172.21.15.45:0/177749093 conn(0x7f2b6806ab40 msgr2=0x7f2b6806af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.079+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/177749093 shutdown_connections 2023-08-27T11:13:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.079+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/177749093 wait complete. 2023-08-27T11:13:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.080+0000 7f2b6d6bc700 1 Processor -- start 2023-08-27T11:13:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.080+0000 7f2b6d6bc700 1 -- start start 2023-08-27T11:13:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b6d6bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680b8cf0 0x7f2b68151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b6d6bc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b680d00e0 0x7f2b680c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b6d6bc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b6815d0f0 0x7f2b680c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b6d6bc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680cc950 con 0x7f2b680b8cf0 2023-08-27T11:13:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b6d6bc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680cb8d0 con 0x7f2b680d00e0 2023-08-27T11:13:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b6d6bc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680cbb90 con 0x7f2b6815d0f0 2023-08-27T11:13:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b66ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680b8cf0 0x7f2b68151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b66ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680b8cf0 0x7f2b68151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54282/0 (socket says 172.21.15.45:54282) 2023-08-27T11:13:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b66ffd700 1 -- 172.21.15.45:0/1998307845 learned_addr learned my addr 172.21.15.45:0/1998307845 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b667fc700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b680d00e0 0x7f2b680c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b677fe700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b6815d0f0 0x7f2b680c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b66ffd700 1 -- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b6815d0f0 msgr2=0x7f2b680c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b66ffd700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b6815d0f0 0x7f2b680c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b66ffd700 1 -- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b680d00e0 msgr2=0x7f2b680c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b66ffd700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b680d00e0 0x7f2b680c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b66ffd700 1 -- 172.21.15.45:0/1998307845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60009b30 con 0x7f2b680b8cf0 2023-08-27T11:13:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b677fe700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b6815d0f0 0x7f2b680c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.081+0000 7f2b66ffd700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680b8cf0 0x7f2b68151560 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00d800 tx=0x7f2b5c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.082+0000 7f2b4f7fe700 1 -- 172.21.15.45:0/1998307845 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c0125f0 con 0x7f2b680b8cf0 2023-08-27T11:13:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.082+0000 7f2b4f7fe700 1 -- 172.21.15.45:0/1998307845 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c012750 con 0x7f2b680b8cf0 2023-08-27T11:13:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.082+0000 7f2b4f7fe700 1 -- 172.21.15.45:0/1998307845 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c012b00 con 0x7f2b680b8cf0 2023-08-27T11:13:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.082+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b680cbe70 con 0x7f2b680b8cf0 2023-08-27T11:13:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.082+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b680cc430 con 0x7f2b680b8cf0 2023-08-27T11:13:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.083+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b680cc7a0 con 0x7f2b680b8cf0 2023-08-27T11:13:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.083+0000 7f2b4f7fe700 1 -- 172.21.15.45:0/1998307845 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2b5c012c60 con 0x7f2b680b8cf0 2023-08-27T11:13:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.083+0000 7f2b4f7fe700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2b5406a0e0 0x7f2b5406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.083+0000 7f2b4f7fe700 1 -- 172.21.15.45:0/1998307845 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2b5c088920 con 0x7f2b680b8cf0 2023-08-27T11:13:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.084+0000 7f2b667fc700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2b5406a0e0 0x7f2b5406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.084+0000 7f2b667fc700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2b5406a0e0 0x7f2b5406c5a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b68160900 tx=0x7f2b58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.088+0000 7f2b4f7fe700 1 -- 172.21.15.45:0/1998307845 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2b5c053f60 con 0x7f2b680b8cf0 2023-08-27T11:13:43.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.331+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b680cc5c0 con 0x7f2b680b8cf0 2023-08-27T11:13:43.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.331+0000 7f2b4f7fe700 1 -- 172.21.15.45:0/1998307845 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f2b5c057580 con 0x7f2b680b8cf0 2023-08-27T11:13:43.335 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:13:43.335 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1324661970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1324661970},{"type":"v1","addr":"172.21.15.135:6836","nonce":1324661970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2014390806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2014390806},{"type":"v1","addr":"172.21.15.135:6837","nonce":2014390806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2103056681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2103056681},{"type":"v1","addr":"172.21.15.45:6834","nonce":2103056681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:13:37.320852+0000","modified":"2023-08-27T11:13:40.917661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/2587151358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2587151358},{"type":"v1","addr":"172.21.15.45:6835","nonce":2587151358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:13:43.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2b5406a0e0 msgr2=0x7f2b5406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2b5406a0e0 0x7f2b5406c5a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b68160900 tx=0x7f2b58009040 comp rx=0 tx=0).stop 2023-08-27T11:13:43.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680b8cf0 msgr2=0x7f2b68151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680b8cf0 0x7f2b68151560 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00d800 tx=0x7f2b5c00db10 comp rx=0 tx=0).stop 2023-08-27T11:13:43.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 shutdown_connections 2023-08-27T11:13:43.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b680b8cf0 0x7f2b68151560 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f2b5406a0e0 0x7f2b5406c5a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b6815d0f0 0x7f2b680c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 --2- 172.21.15.45:0/1998307845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b680d00e0 0x7f2b680c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.333+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 >> 172.21.15.45:0/1998307845 conn(0x7f2b6806ab40 msgr2=0x7f2b680c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.334+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 shutdown_connections 2023-08-27T11:13:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.334+0000 7f2b6d6bc700 1 -- 172.21.15.45:0/1998307845 wait complete. 2023-08-27T11:13:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2023-08-27T11:13:43.752 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:13:37.320852+0000', 'modified': '2023-08-27T11:13:40.917661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5418}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5418': {'gid': 5418, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/2587151358', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 2587151358}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2587151358}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:13:43.753 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:13:43.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-27T11:13:43.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.889+0000 7fb1cc5d3700 1 Processor -- start 2023-08-27T11:13:43.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.889+0000 7fb1cc5d3700 1 -- start start 2023-08-27T11:13:43.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.890+0000 7fb1cc5d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40c9b00 0x7fb1c40c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.890+0000 7fb1cc5d3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40cd350 con 0x7fb1c40cdef0 2023-08-27T11:13:43.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.890+0000 7fb1cc5d3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40ca4a0 con 0x7fb1c41599e0 2023-08-27T11:13:43.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.890+0000 7fb1cc5d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c406bdc0 con 0x7fb1c40c9b00 2023-08-27T11:13:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.890+0000 7fb1cab70700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb1c40cdef0 0x7fb1c406cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41070/0 (socket says 172.21.15.45:41070) 2023-08-27T11:13:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.890+0000 7fb1cab70700 1 -- 172.21.15.45:0/257878431 learned_addr learned my addr 172.21.15.45:0/257878431 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.890+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/257878431 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40c9b00 0x7fb1c40c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.890+0000 7fb1c936d700 1 -- 172.21.15.45:0/257878431 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129918405 0 0) 0x7fb1c40cd350 con 0x7fb1c40cdef0 2023-08-27T11:13:43.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.890+0000 7fb1c936d700 1 -- 172.21.15.45:0/257878431 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1b00035b0 con 0x7fb1c40cdef0 2023-08-27T11:13:43.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c936d700 1 -- 172.21.15.45:0/257878431 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690791753 0 0) 0x7fb1c40ca4a0 con 0x7fb1c41599e0 2023-08-27T11:13:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c936d700 1 -- 172.21.15.45:0/257878431 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c40cd350 con 0x7fb1c41599e0 2023-08-27T11:13:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c9b6e700 1 -- 172.21.15.45:0/257878431 >> v1:172.21.15.135:6789/0 conn(0x7fb1c41599e0 legacy=0x7fb1c40700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c9b6e700 1 -- 172.21.15.45:0/257878431 >> v1:172.21.15.45:6789/0 conn(0x7fb1c40cdef0 legacy=0x7fb1c406cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c936d700 1 -- 172.21.15.45:0/257878431 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3227324395 0 0) 0x7fb1b00035b0 con 0x7fb1c40cdef0 2023-08-27T11:13:43.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c9b6e700 1 -- 172.21.15.45:0/257878431 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c406bfd0 con 0x7fb1c40c9b00 2023-08-27T11:13:43.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/257878431 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40c9b00 0x7fb1c40c9f60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009870 tx=0x7fb1c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5818d5132c37f31a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c936d700 1 -- 172.21.15.45:0/257878431 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0019f0 con 0x7fb1c40c9b00 2023-08-27T11:13:43.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c936d700 1 -- 172.21.15.45:0/257878431 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c0004030 con 0x7fb1c40c9b00 2023-08-27T11:13:43.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.891+0000 7fb1c936d700 1 -- 172.21.15.45:0/257878431 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00043a0 con 0x7fb1c40c9b00 2023-08-27T11:13:43.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.892+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/257878431 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40c9b00 msgr2=0x7fb1c40c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.892+0000 7fb1cc5d3700 1 --2- 172.21.15.45:0/257878431 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40c9b00 0x7fb1c40c9f60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009870 tx=0x7fb1c0009b80 comp rx=0 tx=0).stop 2023-08-27T11:13:43.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.892+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/257878431 shutdown_connections 2023-08-27T11:13:43.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.892+0000 7fb1cc5d3700 1 --2- 172.21.15.45:0/257878431 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40c9b00 0x7fb1c40c9f60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.892+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/257878431 >> 172.21.15.45:0/257878431 conn(0x7fb1c40b07c0 msgr2=0x7fb1c40b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:43.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.892+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/257878431 shutdown_connections 2023-08-27T11:13:43.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.892+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/257878431 wait complete. 2023-08-27T11:13:43.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.893+0000 7fb1cc5d3700 1 Processor -- start 2023-08-27T11:13:43.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.893+0000 7fb1cc5d3700 1 -- start start 2023-08-27T11:13:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.894+0000 7fb1cc5d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c40c9b00 0x7fb1c40ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.894+0000 7fb1cc5d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40cdef0 0x7fb1c40aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.894+0000 7fb1cc5d3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1c41599e0 0x7fb1c40c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.894+0000 7fb1cc5d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4144a40 con 0x7fb1c40c9b00 2023-08-27T11:13:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.894+0000 7fb1cc5d3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c41451f0 con 0x7fb1c41599e0 2023-08-27T11:13:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.894+0000 7fb1cc5d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c41454d0 con 0x7fb1c40cdef0 2023-08-27T11:13:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.894+0000 7fb1ca36f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c40c9b00 0x7fb1c40ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.894+0000 7fb1ca36f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c40c9b00 0x7fb1c40ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54296/0 (socket says 172.21.15.45:54296) 2023-08-27T11:13:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.894+0000 7fb1ca36f700 1 -- 172.21.15.45:0/3074539463 learned_addr learned my addr 172.21.15.45:0/3074539463 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40cdef0 0x7fb1c40aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1cab70700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1c41599e0 0x7fb1c40c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1ca36f700 1 -- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40cdef0 msgr2=0x7fb1c40aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40cdef0 0x7fb1c40aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1ca36f700 1 -- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1c41599e0 msgr2=0x7fb1c40c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1c41599e0 0x7fb1c40c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1ca36f700 1 -- 172.21.15.45:0/3074539463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00094a0 con 0x7fb1c40c9b00 2023-08-27T11:13:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c40c9b00 0x7fb1c40ae910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c520 tx=0x7fb1b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1baffd700 1 -- 172.21.15.45:0/3074539463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40052d0 con 0x7fb1c40c9b00 2023-08-27T11:13:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1baffd700 1 -- 172.21.15.45:0/3074539463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4005430 con 0x7fb1c40c9b00 2023-08-27T11:13:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/3074539463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c41457b0 con 0x7fb1c40c9b00 2023-08-27T11:13:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1baffd700 1 -- 172.21.15.45:0/3074539463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40038c0 con 0x7fb1c40c9b00 2023-08-27T11:13:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.895+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/3074539463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c40c6080 con 0x7fb1c40c9b00 2023-08-27T11:13:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.897+0000 7fb1baffd700 1 -- 172.21.15.45:0/3074539463 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb1b4003a20 con 0x7fb1c40c9b00 2023-08-27T11:13:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.897+0000 7fb1baffd700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1a806a0e0 0x7fb1a806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.897+0000 7fb1baffd700 1 -- 172.21.15.45:0/3074539463 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb1b4087ba0 con 0x7fb1c40c9b00 2023-08-27T11:13:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.897+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1a806a0e0 0x7fb1a806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.898+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1a806a0e0 0x7fb1a806c5a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00097c0 tx=0x7fb1c0013040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.899+0000 7fb1d8795b80 1 Processor -- start 2023-08-27T11:13:43.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.899+0000 7fb1d8795b80 1 -- start start 2023-08-27T11:13:43.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d29992590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 0x555d2998e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299b9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d298eb730 con 0x555d299903d0 2023-08-27T11:13:43.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d298e9050 con 0x555d299921b0 2023-08-27T11:13:43.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d298e4940 con 0x555d299d91d0 2023-08-27T11:13:43.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1cab70700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1cab70700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299b9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54308/0 (socket says 172.21.15.45:54308) 2023-08-27T11:13:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1cab70700 1 -- 172.21.15.45:0/3831965573 learned_addr learned my addr 172.21.15.45:0/3831965573 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/3831965573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 0x555d2998e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/3831965573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d29992590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1cab70700 1 -- 172.21.15.45:0/3831965573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 msgr2=0x555d2998e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1cab70700 1 --2- 172.21.15.45:0/3831965573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 0x555d2998e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1cab70700 1 -- 172.21.15.45:0/3831965573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 msgr2=0x555d29992590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1cab70700 1 --2- 172.21.15.45:0/3831965573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d29992590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.900+0000 7fb1cab70700 1 -- 172.21.15.45:0/3831965573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d297645e0 con 0x555d299903d0 2023-08-27T11:13:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1cab70700 1 --2- 172.21.15.45:0/3831965573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299b9520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc007050 tx=0x7fb1bc007360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dee340e0b7fdea4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1a7fff700 1 -- 172.21.15.45:0/3831965573 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc014480 con 0x555d299903d0 2023-08-27T11:13:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1a7fff700 1 -- 172.21.15.45:0/3831965573 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc0145e0 con 0x555d299903d0 2023-08-27T11:13:43.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1a7fff700 1 -- 172.21.15.45:0/3831965573 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc014990 con 0x555d299903d0 2023-08-27T11:13:43.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3831965573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 msgr2=0x555d299b9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3831965573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299b9520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc007050 tx=0x7fb1bc007360 comp rx=0 tx=0).stop 2023-08-27T11:13:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3831965573 shutdown_connections 2023-08-27T11:13:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3831965573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299b9520 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3831965573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d29992590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3831965573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 0x555d2998e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.901+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3831965573 >> 172.21.15.45:0/3831965573 conn(0x555d298f0c30 msgr2=0x555d299d7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.902+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3831965573 shutdown_connections 2023-08-27T11:13:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.902+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3831965573 wait complete. 2023-08-27T11:13:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.902+0000 7fb1d8795b80 1 -- start start 2023-08-27T11:13:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.902+0000 7fb1d8795b80 1 Processor -- start 2023-08-27T11:13:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299921b0 0x555d299d7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299d91d0 0x555d299d8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d29882b60 con 0x555d299921b0 2023-08-27T11:13:43.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d29882860 con 0x555d299d91d0 2023-08-27T11:13:43.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d297668d0 con 0x555d299903d0 2023-08-27T11:13:43.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1d8795b80 10 client.0 _ll_register_callbacks cb 0x555d29880630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:13:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1c9b6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299921b0 0x555d299d7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1c9b6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299921b0 0x555d299d7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54312/0 (socket says 172.21.15.45:54312) 2023-08-27T11:13:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1c9b6e700 1 -- 172.21.15.45:0/2443198307 learned_addr learned my addr 172.21.15.45:0/2443198307 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.903+0000 7fb1cab70700 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299d91d0 0x555d299d8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1c9b6e700 1 -- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 msgr2=0x555d299bdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:43.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1c9b6e700 1 -- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299d91d0 msgr2=0x555d299d8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:43.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299d91d0 0x555d299d8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1c9b6e700 1 -- 172.21.15.45:0/2443198307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc00f040 con 0x555d299921b0 2023-08-27T11:13:43.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1cab70700 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299d91d0 0x555d299d8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:43.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:43.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299921b0 0x555d299d7d50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40b1860 tx=0x7fb1c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55df45f49357b4db server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:13:43.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1a57fa700 1 -- 172.21.15.45:0/2443198307 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0011840 con 0x555d299921b0 2023-08-27T11:13:43.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1a57fa700 1 -- 172.21.15.45:0/2443198307 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb1c00119a0 con 0x555d299921b0 2023-08-27T11:13:43.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2443198307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d299854f0 con 0x555d299921b0 2023-08-27T11:13:43.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1a57fa700 1 -- 172.21.15.45:0/2443198307 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1c0011b60 con 0x555d299921b0 2023-08-27T11:13:43.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1d8795b80 10 client.5604 fetch_fsmap learned FSMap version 17 2023-08-27T11:13:43.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2443198307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x555d298fc750 con 0x555d299921b0 2023-08-27T11:13:43.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1a57fa700 1 -- 172.21.15.45:0/2443198307 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d299854f0 con 0x555d299921b0 2023-08-27T11:13:43.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1a57fa700 1 -- 172.21.15.45:0/2443198307 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb1c0011cc0 con 0x555d299921b0 2023-08-27T11:13:43.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1d8795b80 10 client.5604 fetch_fsmap finished waiting for FSMap version 17 2023-08-27T11:13:43.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1d8795b80 10 client.5604 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:13:43.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1d8795b80 20 client.5604 populate_metadata read hostname 'smithi045' 2023-08-27T11:13:43.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] conn(0x555d299bb600 0x555d299bbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1d8795b80 4 client.5604 mds_command: new command op to 5418 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:13:43.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.904+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2443198307 --> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d298dd920 con 0x555d299bb600 2023-08-27T11:13:43.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.905+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] conn(0x555d299bb600 0x555d299bbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.905+0000 7fb1a57fa700 10 client.5604 ms_handle_connect on v2:172.21.15.45:6833/2587151358 2023-08-27T11:13:43.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.905+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] conn(0x555d299bb600 0x555d299bbdf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2be89174b3dbde5 server_cookie=526176fd3011373b in_seq=0 out_seq=0 2023-08-27T11:13:43.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.905+0000 7fb1a57fa700 1 -- 172.21.15.45:0/2443198307 <== mds.0 v2:172.21.15.45:6833/2587151358 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555d298dd920 con 0x555d299bb600 2023-08-27T11:13:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.905+0000 7fb1a57fa700 10 client.5604 handle_command_reply: tid=0 2023-08-27T11:13:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.905+0000 7fb1d8795b80 1 client.5604 shutdown 2023-08-27T11:13:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2443198307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299921b0 msgr2=0x555d299d7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299921b0 0x555d299d7d50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40b1860 tx=0x7fb1c00047c0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2443198307 shutdown_connections 2023-08-27T11:13:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] conn(0x555d299bb600 0x555d299bbdf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299d91d0 0x555d299d8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2443198307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299921b0 0x555d299d7d50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1a57fa700 0 client.5604 ms_handle_reset on v2:172.21.15.45:6833/2587151358 2023-08-27T11:13:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2443198307 >> 172.21.15.45:0/2443198307 conn(0x555d298f0c30 msgr2=0x555d29990d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2443198307 shutdown_connections 2023-08-27T11:13:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2443198307 wait complete. 2023-08-27T11:13:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.906+0000 7fb1d8795b80 20 client.5604 trim_cache size 0 max 16384 2023-08-27T11:13:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.924+0000 7fb1d8795b80 1 Processor -- start 2023-08-27T11:13:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.924+0000 7fb1d8795b80 1 -- start start 2023-08-27T11:13:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.924+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.924+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d29992590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.924+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299d91d0 0x555d29992ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.924+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d299c0930 con 0x555d299d91d0 2023-08-27T11:13:43.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.924+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d299bf880 con 0x555d299921b0 2023-08-27T11:13:43.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.924+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d299bdf80 con 0x555d299903d0 2023-08-27T11:13:43.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1cab70700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299d91d0 0x555d29992ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1cab70700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299d91d0 0x555d29992ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54322/0 (socket says 172.21.15.45:54322) 2023-08-27T11:13:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1cab70700 1 -- 172.21.15.45:0/2172689905 learned_addr learned my addr 172.21.15.45:0/2172689905 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d29992590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1cab70700 1 -- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 msgr2=0x555d299bffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1cab70700 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1cab70700 1 -- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 msgr2=0x555d29992590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1cab70700 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d29992590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1cab70700 1 -- 172.21.15.45:0/2172689905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d299b53c0 con 0x555d299d91d0 2023-08-27T11:13:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d29992590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1cab70700 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299d91d0 0x555d29992ad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc017040 tx=0x7fb1bc00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb2db554ff1e209 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1b8ff9700 1 -- 172.21.15.45:0/2172689905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc001e60 con 0x555d299d91d0 2023-08-27T11:13:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1b8ff9700 1 -- 172.21.15.45:0/2172689905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc014750 con 0x555d299d91d0 2023-08-27T11:13:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.925+0000 7fb1b8ff9700 1 -- 172.21.15.45:0/2172689905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc014b00 con 0x555d299d91d0 2023-08-27T11:13:43.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2172689905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299d91d0 msgr2=0x555d29992ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299d91d0 0x555d29992ad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc017040 tx=0x7fb1bc00c000 comp rx=0 tx=0).stop 2023-08-27T11:13:43.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2172689905 shutdown_connections 2023-08-27T11:13:43.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299d91d0 0x555d29992ad0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299903d0 0x555d299bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/2172689905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d29992590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2172689905 >> 172.21.15.45:0/2172689905 conn(0x555d298f0c30 msgr2=0x555d2998d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:43.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2172689905 shutdown_connections 2023-08-27T11:13:43.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 -- 172.21.15.45:0/2172689905 wait complete. 2023-08-27T11:13:43.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 -- start start 2023-08-27T11:13:43.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.926+0000 7fb1d8795b80 1 Processor -- start 2023-08-27T11:13:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.927+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.927+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d299b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.927+0000 7fb1d8795b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 0x555d29992590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.927+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d299f3b50 con 0x555d299903d0 2023-08-27T11:13:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.927+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d299b8c20 con 0x555d299921b0 2023-08-27T11:13:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.927+0000 7fb1d8795b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d29a464c0 con 0x555d299d91d0 2023-08-27T11:13:43.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.927+0000 7fb1d8795b80 10 client.0 _ll_register_callbacks cb 0x555d29880630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:13:43.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.927+0000 7fb1ca36f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1ca36f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54324/0 (socket says 172.21.15.45:54324) 2023-08-27T11:13:43.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1ca36f700 1 -- 172.21.15.45:0/3072201217 learned_addr learned my addr 172.21.15.45:0/3072201217 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:43.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d299b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1cab70700 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 0x555d29992590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1ca36f700 1 -- 172.21.15.45:0/3072201217 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 msgr2=0x555d29992590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:43.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 0x555d29992590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1ca36f700 1 -- 172.21.15.45:0/3072201217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 msgr2=0x555d299b86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d299b86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1ca36f700 1 -- 172.21.15.45:0/3072201217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d297645e0 con 0x555d299903d0 2023-08-27T11:13:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1ca36f700 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299f1cd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40533a0 tx=0x7fb1b4085d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1e81f711560840 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:13:43.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1a57fa700 1 -- 172.21.15.45:0/3072201217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b405e720 con 0x555d299903d0 2023-08-27T11:13:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1a57fa700 1 -- 172.21.15.45:0/3072201217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb1b4057e40 con 0x555d299903d0 2023-08-27T11:13:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3072201217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1bc0145e0 con 0x555d299903d0 2023-08-27T11:13:43.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1a57fa700 1 -- 172.21.15.45:0/3072201217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1b40751a0 con 0x555d299903d0 2023-08-27T11:13:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1d8795b80 10 client.5610 fetch_fsmap learned FSMap version 17 2023-08-27T11:13:43.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3072201217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x555d299dbf50 con 0x555d299903d0 2023-08-27T11:13:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1a57fa700 1 -- 172.21.15.45:0/3072201217 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc0145e0 con 0x555d299903d0 2023-08-27T11:13:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1a57fa700 1 -- 172.21.15.45:0/3072201217 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb1b4052c90 con 0x555d299903d0 2023-08-27T11:13:43.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.928+0000 7fb1d8795b80 10 client.5610 fetch_fsmap finished waiting for FSMap version 17 2023-08-27T11:13:43.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 10 client.5610 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:13:43.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 20 client.5610 populate_metadata read hostname 'smithi045' 2023-08-27T11:13:43.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] conn(0x555d29a39140 0x555d29a39520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:43.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 4 client.5610 mds_command: new command op to 5418 tid=0[{"prefix": "status"}] 2023-08-27T11:13:43.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3072201217 --> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555d2989e000 con 0x555d29a39140 2023-08-27T11:13:43.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] conn(0x555d29a39140 0x555d29a39520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:43.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1a57fa700 10 client.5610 ms_handle_connect on v2:172.21.15.45:6833/2587151358 2023-08-27T11:13:43.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1c9b6e700 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] conn(0x555d29a39140 0x555d29a39520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2497021ed9bce125 server_cookie=7f1981f9cae44ae2 in_seq=0 out_seq=0 2023-08-27T11:13:43.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1a57fa700 1 -- 172.21.15.45:0/3072201217 <== mds.0 v2:172.21.15.45:6833/2587151358 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555d2989e000 con 0x555d29a39140 2023-08-27T11:13:43.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1a57fa700 10 client.5610 handle_command_reply: tid=0 2023-08-27T11:13:43.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 client.5610 shutdown 2023-08-27T11:13:43.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3072201217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 msgr2=0x555d299f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299f1cd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40533a0 tx=0x7fb1b4085d90 comp rx=0 tx=0).stop 2023-08-27T11:13:43.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3072201217 shutdown_connections 2023-08-27T11:13:43.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.45:6833/2587151358,v1:172.21.15.45:6835/2587151358] conn(0x555d29a39140 0x555d29a39520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d299903d0 0x555d299f1cd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x555d299d91d0 0x555d29992590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1a57fa700 0 client.5610 ms_handle_reset on v2:172.21.15.45:6833/2587151358 2023-08-27T11:13:43.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 --2- 172.21.15.45:0/3072201217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d299921b0 0x555d299b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3072201217 >> 172.21.15.45:0/3072201217 conn(0x555d298f0c30 msgr2=0x555d29989450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:43.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.929+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3072201217 shutdown_connections 2023-08-27T11:13:43.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1d8795b80 1 -- 172.21.15.45:0/3072201217 wait complete. 2023-08-27T11:13:43.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1d8795b80 20 client.5610 trim_cache size 0 max 16384 2023-08-27T11:13:43.974 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:13:43.975 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "8ea1ae34-1697-4b79-b2ce-3f741cd3c828", 2023-08-27T11:13:43.975 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-08-27T11:13:43.975 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5418, 2023-08-27T11:13:43.976 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-08-27T11:13:43.976 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-08-27T11:13:43.976 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-08-27T11:13:43.977 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.0392917290000003, 2023-08-27T11:13:43.977 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 17, 2023-08-27T11:13:43.977 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 42, 2023-08-27T11:13:43.978 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 42, 2023-08-27T11:13:43.978 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 14.931880573999999 2023-08-27T11:13:43.978 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:13:43.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1a806a0e0 msgr2=0x7fb1a806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1a806a0e0 0x7fb1a806c5a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00097c0 tx=0x7fb1c0013040 comp rx=0 tx=0).stop 2023-08-27T11:13:43.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/3074539463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c40c9b00 msgr2=0x7fb1c40ae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:43.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c40c9b00 0x7fb1c40ae910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c520 tx=0x7fb1b400c8e0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/3074539463 shutdown_connections 2023-08-27T11:13:43.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c40c9b00 0x7fb1c40ae910 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7fb1a806a0e0 0x7fb1a806c5a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1c41599e0 0x7fb1c40c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 --2- 172.21.15.45:0/3074539463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1c40cdef0 0x7fb1c40aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:43.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/3074539463 >> 172.21.15.45:0/3074539463 conn(0x7fb1c40b07c0 msgr2=0x7fb1c406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:43.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/3074539463 shutdown_connections 2023-08-27T11:13:43.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:43.930+0000 7fb1cc5d3700 1 -- 172.21.15.45:0/3074539463 wait complete. 2023-08-27T11:13:43.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:13:44.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.109+0000 7f4f7364d700 1 Processor -- start 2023-08-27T11:13:44.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.109+0000 7f4f7364d700 1 -- start start 2023-08-27T11:13:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f7364d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4f6c0cc650 0x7f4f6c0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f7364d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c0ccb80 con 0x7f4f6c15e990 2023-08-27T11:13:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f7364d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c0cce60 con 0x7f4f6c0adad0 2023-08-27T11:13:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f7364d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6c0cd600 con 0x7f4f6c0cc650 2023-08-27T11:13:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f713e9700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4f6c15e990 0x7f4f6c0ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41108/0 (socket says 172.21.15.45:41108) 2023-08-27T11:13:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f713e9700 1 -- 172.21.15.45:0/2275120594 learned_addr learned my addr 172.21.15.45:0/2275120594 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f70be8700 1 --2- 172.21.15.45:0/2275120594 >> v2:172.21.15.135:3300/0 conn(0x7f4f6c0cc650 0x7f4f6c0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094683495 0 0) 0x7f4f6c0ccb80 con 0x7f4f6c15e990 2023-08-27T11:13:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f580035b0 con 0x7f4f6c15e990 2023-08-27T11:13:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3593285957 0 0) 0x7f4f580035b0 con 0x7f4f6c15e990 2023-08-27T11:13:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 >> v2:172.21.15.135:3300/0 conn(0x7f4f6c0cc650 msgr2=0x7f4f6c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f63fff700 1 --2- 172.21.15.45:0/2275120594 >> v2:172.21.15.135:3300/0 conn(0x7f4f6c0cc650 0x7f4f6c0ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.110+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 >> v1:172.21.15.135:6789/0 conn(0x7f4f6c0adad0 legacy=0x7f4f6c0cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0cd8e0 con 0x7f4f6c15e990 2023-08-27T11:13:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f4f5c001670 con 0x7f4f6c15e990 2023-08-27T11:13:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 >> v1:172.21.15.45:6789/0 conn(0x7f4f6c15e990 legacy=0x7f4f6c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58003a00 0x7f4f58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f58004850 0x7f4f58004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f58005210 0x7f4f5800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0cd600 con 0x7f4f58004850 2023-08-27T11:13:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5800c110 con 0x7f4f58005210 2023-08-27T11:13:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5800c2d0 con 0x7f4f58003a00 2023-08-27T11:13:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f70be8700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f58004850 0x7f4f58004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f71bea700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f58005210 0x7f4f5800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f713e9700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58003a00 0x7f4f58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:44.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f70be8700 1 -- 172.21.15.45:0/2275120594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58003a00 msgr2=0x7f4f58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:44.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f70be8700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58003a00 0x7f4f58003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f70be8700 1 -- 172.21.15.45:0/2275120594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f58005210 msgr2=0x7f4f5800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:44.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f70be8700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f58005210 0x7f4f5800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f70be8700 1 -- 172.21.15.45:0/2275120594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f580035b0 con 0x7f4f58004850 2023-08-27T11:13:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f71bea700 1 -- 172.21.15.45:0/2275120594 reap_dead start 2023-08-27T11:13:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f70be8700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f58004850 0x7f4f58004cd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009b60 tx=0x7f4f58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd7f3dc76381e9e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800e6c0 con 0x7f4f58004850 2023-08-27T11:13:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6800e820 con 0x7f4f58004850 2023-08-27T11:13:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.111+0000 7f4f63fff700 1 -- 172.21.15.45:0/2275120594 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800eb90 con 0x7f4f58004850 2023-08-27T11:13:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.112+0000 7f4f7364d700 1 -- 172.21.15.45:0/2275120594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f58004850 msgr2=0x7f4f58004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.112+0000 7f4f7364d700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f58004850 0x7f4f58004cd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009b60 tx=0x7f4f58005880 comp rx=0 tx=0).stop 2023-08-27T11:13:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.112+0000 7f4f7364d700 1 -- 172.21.15.45:0/2275120594 shutdown_connections 2023-08-27T11:13:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.112+0000 7f4f7364d700 1 --2- 172.21.15.45:0/2275120594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f58004850 0x7f4f58004cd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.112+0000 7f4f7364d700 1 -- 172.21.15.45:0/2275120594 >> 172.21.15.45:0/2275120594 conn(0x7f4f6c0d0f60 msgr2=0x7f4f6c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:44.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.112+0000 7f4f7364d700 1 -- 172.21.15.45:0/2275120594 shutdown_connections 2023-08-27T11:13:44.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.112+0000 7f4f7364d700 1 -- 172.21.15.45:0/2275120594 wait complete. 2023-08-27T11:13:44.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.114+0000 7f4f7364d700 1 Processor -- start 2023-08-27T11:13:44.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.114+0000 7f4f7364d700 1 -- start start 2023-08-27T11:13:44.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f7364d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58004850 0x7f4f6c155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:44.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f7364d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f6c0b02c0 0x7f4f6c15d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:44.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f7364d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f6c0b0740 0x7f4f6c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:44.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f7364d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0b0ca0 con 0x7f4f6c0b0740 2023-08-27T11:13:44.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f7364d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c1f1ec0 con 0x7f4f6c0b02c0 2023-08-27T11:13:44.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f7364d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c1f21a0 con 0x7f4f58004850 2023-08-27T11:13:44.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f71bea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f6c0b0740 0x7f4f6c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:44.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f71bea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f6c0b0740 0x7f4f6c15d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54348/0 (socket says 172.21.15.45:54348) 2023-08-27T11:13:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f71bea700 1 -- 172.21.15.45:0/274546224 learned_addr learned my addr 172.21.15.45:0/274546224 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f713e9700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58004850 0x7f4f6c155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f70be8700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f6c0b02c0 0x7f4f6c15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f71bea700 1 -- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58004850 msgr2=0x7f4f6c155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f71bea700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58004850 0x7f4f6c155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f71bea700 1 -- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f6c0b02c0 msgr2=0x7f4f6c15d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f71bea700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f6c0b02c0 0x7f4f6c15d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f71bea700 1 -- 172.21.15.45:0/274546224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c1f2480 con 0x7f4f6c0b0740 2023-08-27T11:13:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f713e9700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58004850 0x7f4f6c155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.115+0000 7f4f71bea700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f6c0b0740 0x7f4f6c15d5c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c071640 tx=0x7f4f6400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.116+0000 7f4f61ffb700 1 -- 172.21.15.45:0/274546224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64003c10 con 0x7f4f6c0b0740 2023-08-27T11:13:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.116+0000 7f4f61ffb700 1 -- 172.21.15.45:0/274546224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f64003d70 con 0x7f4f6c0b0740 2023-08-27T11:13:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.116+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c0b0f80 con 0x7f4f6c0b0740 2023-08-27T11:13:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.116+0000 7f4f61ffb700 1 -- 172.21.15.45:0/274546224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6400db90 con 0x7f4f6c0b0740 2023-08-27T11:13:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.116+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c06f330 con 0x7f4f6c0b0740 2023-08-27T11:13:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.117+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c0cf320 con 0x7f4f6c0b0740 2023-08-27T11:13:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.117+0000 7f4f61ffb700 1 -- 172.21.15.45:0/274546224 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4f6400dcf0 con 0x7f4f6c0b0740 2023-08-27T11:13:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.117+0000 7f4f61ffb700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4f5006a0e0 0x7f4f5006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.118+0000 7f4f61ffb700 1 -- 172.21.15.45:0/274546224 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4f64088a20 con 0x7f4f6c0b0740 2023-08-27T11:13:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.118+0000 7f4f713e9700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4f5006a0e0 0x7f4f5006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.118+0000 7f4f713e9700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4f5006a0e0 0x7f4f5006c5a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c007d30 tx=0x7f4f5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:44.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.122+0000 7f4f61ffb700 1 -- 172.21.15.45:0/274546224 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4f64053fc0 con 0x7f4f6c0b0740 2023-08-27T11:13:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.368+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f6c0b6ee0 con 0x7f4f6c0b0740 2023-08-27T11:13:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.369+0000 7f4f61ffb700 1 -- 172.21.15.45:0/274546224 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4f640575e0 con 0x7f4f6c0b0740 2023-08-27T11:13:44.372 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:13:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4f5006a0e0 msgr2=0x7f4f5006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4f5006a0e0 0x7f4f5006c5a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c007d30 tx=0x7f4f5c009040 comp rx=0 tx=0).stop 2023-08-27T11:13:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f6c0b0740 msgr2=0x7f4f6c15d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f6c0b0740 0x7f4f6c15d5c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c071640 tx=0x7f4f6400d430 comp rx=0 tx=0).stop 2023-08-27T11:13:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 shutdown_connections 2023-08-27T11:13:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f6c0b0740 0x7f4f6c15d5c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f4f5006a0e0 0x7f4f5006c5a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4f58004850 0x7f4f6c155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 --2- 172.21.15.45:0/274546224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f6c0b02c0 0x7f4f6c15d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:44.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 >> 172.21.15.45:0/274546224 conn(0x7f4f6c0d0f60 msgr2=0x7f4f6c0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:44.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 shutdown_connections 2023-08-27T11:13:44.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:44.371+0000 7f4f7364d700 1 -- 172.21.15.45:0/274546224 wait complete. 2023-08-27T11:13:44.385 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:13:44.385 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:13:44.386 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-08-27T11:13:44.386 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:13:44.386 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:13:44.387 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:13:44.387 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:13:44.387 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:13:44.387 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:13:44.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:44.388 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-08-27T11:13:44.406 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:13:44.407 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:13:44.407 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:13:44.407 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:13:44.407 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:13:44.408 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:13:44.408 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:13:44.408 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:13:44.408 INFO:teuthology.orchestra.run.smithi045.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:13:44.409 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-08-27T11:13:44.409 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-27T11:13:44.409 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42317sec preferred_lft 42317sec 2023-08-27T11:13:44.410 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-08-27T11:13:44.410 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:13:44.410 INFO:teuthology.orchestra.run.smithi045.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:13:44.410 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:13:44.410 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:13:44.411 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-08-27T11:13:44.411 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:13:44.411 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:13:44.411 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:13:44.412 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:13:44.412 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a41d:a1ff:fed9:3bf2/64 scope link 2023-08-27T11:13:44.412 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:13:44.412 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:13:44.413 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether a6:e4:29:97:ce:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:13:44.413 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a4e4:29ff:fe97:ced1/64 scope link 2023-08-27T11:13:44.413 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:13:44.413 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:13:44.414 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:13:44.414 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c84:f3ff:fe1a:a84/64 scope link 2023-08-27T11:13:44.414 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:13:44.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:44.415 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:13:44.473 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-08-27T11:13:44.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:44.474 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-08-27T11:13:44.533 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-08-27T11:13:44.534 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:13:44.534 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-27T11:13:44.534 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-27T11:13:44.534 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:13:44.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:44.535 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:13:44.536 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:13:44.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-27T11:13:44.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-27T11:13:44.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-27T11:13:44.537 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:13:44.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:44.641 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:13:44.659 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:13:44.659 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:13:44.659 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:13:44.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:44.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-08-27T11:13:44.727 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:13:44.728 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:13:44.784 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:13:44.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:13:44.785 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:13:44.842 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:13:44.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:44.843 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:13:44.929 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:13:44.930 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:13:44.930 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:13:44.939 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:13:44.939 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:13:44.939 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:13:44.939 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:13:44.939 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:13:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:13:45.013 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:13:45.013 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:13:45.013 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:13:45.013 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:13:45.014 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790" 2023-08-27T11:13:45.014 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2023-08-27T11:13:45.014 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8ea1ae34-1697-4b79-b2ce-3f741cd3c828" will pass to kernel 2023-08-27T11:13:45.015 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:13:45.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:45.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:13:45.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:45.045 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:13:45.129 INFO:teuthology.orchestra.run.smithi045.stdout:client5126 2023-08-27T11:13:45.129 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:13:45.130 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:13:45.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:13:45.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.352+0000 7f488149d700 1 Processor -- start 2023-08-27T11:13:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.353+0000 7f488149d700 1 -- start start 2023-08-27T11:13:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.353+0000 7f488149d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f487c15d7d0 0x7f487c070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.353+0000 7f488149d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f487c161130 0x7f487c0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.353+0000 7f488149d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c0d0bf0 con 0x7f487c0cd770 2023-08-27T11:13:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.353+0000 7f488149d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487c162040 con 0x7f487c161130 2023-08-27T11:13:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.353+0000 7f488149d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487c1617a0 con 0x7f487c15d7d0 2023-08-27T11:13:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.353+0000 7f487b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f487c161130 0x7f487c0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f487b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f487c161130 0x7f487c0cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54360/0 (socket says 172.21.15.45:54360) 2023-08-27T11:13:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f487b7fe700 1 -- 172.21.15.45:0/2344701942 learned_addr learned my addr 172.21.15.45:0/2344701942 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f487a7fc700 1 --2- 172.21.15.45:0/2344701942 >> v2:172.21.15.135:3300/0 conn(0x7f487c15d7d0 0x7f487c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f487b7fe700 1 -- 172.21.15.45:0/2344701942 >> v2:172.21.15.135:3300/0 conn(0x7f487c15d7d0 msgr2=0x7f487c070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f487b7fe700 1 --2- 172.21.15.45:0/2344701942 >> v2:172.21.15.135:3300/0 conn(0x7f487c15d7d0 0x7f487c070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f487b7fe700 1 -- 172.21.15.45:0/2344701942 >> v1:172.21.15.135:6789/0 conn(0x7f487c0cd770 legacy=0x7f487c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f487b7fe700 1 -- 172.21.15.45:0/2344701942 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c161a80 con 0x7f487c161130 2023-08-27T11:13:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f487a7fc700 1 --2- 172.21.15.45:0/2344701942 >> v2:172.21.15.135:3300/0 conn(0x7f487c15d7d0 0x7f487c070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f487b7fe700 1 --2- 172.21.15.45:0/2344701942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c161130 0x7f487c0cb4a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f487c0afb70 tx=0x7f487400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb4e93793a590f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f4879ffb700 1 -- 172.21.15.45:0/2344701942 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487400e070 con 0x7f487c161130 2023-08-27T11:13:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f4879ffb700 1 -- 172.21.15.45:0/2344701942 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874003a20 con 0x7f487c161130 2023-08-27T11:13:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.354+0000 7f4879ffb700 1 -- 172.21.15.45:0/2344701942 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874003dd0 con 0x7f487c161130 2023-08-27T11:13:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.355+0000 7f488149d700 1 -- 172.21.15.45:0/2344701942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c161130 msgr2=0x7f487c0cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.355+0000 7f488149d700 1 --2- 172.21.15.45:0/2344701942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c161130 0x7f487c0cb4a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f487c0afb70 tx=0x7f487400b330 comp rx=0 tx=0).stop 2023-08-27T11:13:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.355+0000 7f488149d700 1 -- 172.21.15.45:0/2344701942 shutdown_connections 2023-08-27T11:13:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.355+0000 7f488149d700 1 --2- 172.21.15.45:0/2344701942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c161130 0x7f487c0cb4a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.355+0000 7f488149d700 1 --2- 172.21.15.45:0/2344701942 >> v2:172.21.15.135:3300/0 conn(0x7f487c15d7d0 0x7f487c070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.355+0000 7f488149d700 1 -- 172.21.15.45:0/2344701942 >> 172.21.15.45:0/2344701942 conn(0x7f487c06f960 msgr2=0x7f487c06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:45.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.355+0000 7f488149d700 1 -- 172.21.15.45:0/2344701942 shutdown_connections 2023-08-27T11:13:45.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.355+0000 7f488149d700 1 -- 172.21.15.45:0/2344701942 wait complete. 2023-08-27T11:13:45.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.356+0000 7f488149d700 1 Processor -- start 2023-08-27T11:13:45.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.357+0000 7f488149d700 1 -- start start 2023-08-27T11:13:45.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.357+0000 7f488149d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c0cd770 0x7f487c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.357+0000 7f488149d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f487c15d7d0 0x7f487c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.357+0000 7f488149d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487c161130 0x7f487c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.357+0000 7f488149d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0acb20 con 0x7f487c0cd770 2023-08-27T11:13:45.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.357+0000 7f488149d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c156550 con 0x7f487c161130 2023-08-27T11:13:45.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.357+0000 7f488149d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0b5ff0 con 0x7f487c15d7d0 2023-08-27T11:13:45.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c0cd770 0x7f487c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c0cd770 0x7f487c156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54362/0 (socket says 172.21.15.45:54362) 2023-08-27T11:13:45.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487affd700 1 -- 172.21.15.45:0/100916398 learned_addr learned my addr 172.21.15.45:0/100916398 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:45.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487a7fc700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f487c15d7d0 0x7f487c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487b7fe700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487c161130 0x7f487c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487affd700 1 -- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f487c15d7d0 msgr2=0x7f487c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:45.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487affd700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f487c15d7d0 0x7f487c0b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487affd700 1 -- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487c161130 msgr2=0x7f487c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487affd700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487c161130 0x7f487c15b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487affd700 1 -- 172.21.15.45:0/100916398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4874009b30 con 0x7f487c0cd770 2023-08-27T11:13:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f487affd700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c0cd770 0x7f487c156010 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f487000c4d0 tx=0x7f487000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:45.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f48637fe700 1 -- 172.21.15.45:0/100916398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870011790 con 0x7f487c0cd770 2023-08-27T11:13:45.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f48637fe700 1 -- 172.21.15.45:0/100916398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48700118f0 con 0x7f487c0cd770 2023-08-27T11:13:45.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c0b62d0 con 0x7f487c0cd770 2023-08-27T11:13:45.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f48637fe700 1 -- 172.21.15.45:0/100916398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870011ca0 con 0x7f487c0cd770 2023-08-27T11:13:45.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.358+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c0b6890 con 0x7f487c0cd770 2023-08-27T11:13:45.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.359+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c04e620 con 0x7f487c0cd770 2023-08-27T11:13:45.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.360+0000 7f48637fe700 1 -- 172.21.15.45:0/100916398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f487001c460 con 0x7f487c0cd770 2023-08-27T11:13:45.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.360+0000 7f48637fe700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f486806a0e0 0x7f486806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.360+0000 7f48637fe700 1 -- 172.21.15.45:0/100916398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f48700884e0 con 0x7f487c0cd770 2023-08-27T11:13:45.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.360+0000 7f487a7fc700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f486806a0e0 0x7f486806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.361+0000 7f487a7fc700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f486806a0e0 0x7f486806c5a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f486c006fd0 tx=0x7f486c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:45.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.365+0000 7f48637fe700 1 -- 172.21.15.45:0/100916398 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4870053a90 con 0x7f487c0cd770 2023-08-27T11:13:45.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.592+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 --> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f487c161d60 con 0x7f486806a0e0 2023-08-27T11:13:45.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.595+0000 7f48637fe700 1 -- 172.21.15.45:0/100916398 <== mgr.4100 v2:172.21.15.135:6832/4120550306 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f486c00c040 con 0x7f486806a0e0 2023-08-27T11:13:45.598 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:13:45.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:13:45.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephfs" 2023-08-27T11:13:45.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:13:45.599 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:13:45.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f486806a0e0 msgr2=0x7f486806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:45.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f486806a0e0 0x7f486806c5a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f486c006fd0 tx=0x7f486c008040 comp rx=0 tx=0).stop 2023-08-27T11:13:45.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c0cd770 msgr2=0x7f487c156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:45.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c0cd770 0x7f487c156010 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f487000c4d0 tx=0x7f487000c890 comp rx=0 tx=0).stop 2023-08-27T11:13:45.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 shutdown_connections 2023-08-27T11:13:45.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f487c0cd770 0x7f487c156010 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f486806a0e0 0x7f486806c5a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f487c161130 0x7f487c15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 --2- 172.21.15.45:0/100916398 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f487c15d7d0 0x7f487c0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 >> 172.21.15.45:0/100916398 conn(0x7f487c06f960 msgr2=0x7f487c0cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:45.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 shutdown_connections 2023-08-27T11:13:45.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.597+0000 7f488149d700 1 -- 172.21.15.45:0/100916398 wait complete. 2023-08-27T11:13:45.611 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-27T11:13:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.737+0000 7f85748ae700 1 Processor -- start 2023-08-27T11:13:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.737+0000 7f85748ae700 1 -- start start 2023-08-27T11:13:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f85748ae700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d480 0x7f85700cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f85748ae700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f85700ced40 0x7f85700cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f85748ae700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8570071d70 con 0x7f857015d860 2023-08-27T11:13:45.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f85748ae700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8570072050 con 0x7f85700ced40 2023-08-27T11:13:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f85748ae700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8570072330 con 0x7f857015d480 2023-08-27T11:13:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f856dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f857015d860 0x7f85700b0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41146/0 (socket says 172.21.15.45:41146) 2023-08-27T11:13:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f856dd9b700 1 -- 172.21.15.45:0/4036354122 learned_addr learned my addr 172.21.15.45:0/4036354122 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f856ed9d700 1 --2- 172.21.15.45:0/4036354122 >> v2:172.21.15.135:3300/0 conn(0x7f85700ced40 0x7f85700cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f856e59c700 1 --2- 172.21.15.45:0/4036354122 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d480 0x7f85700cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f856d59a700 1 -- 172.21.15.45:0/4036354122 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166202316 0 0) 0x7f8570071d70 con 0x7f857015d860 2023-08-27T11:13:45.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.738+0000 7f856d59a700 1 -- 172.21.15.45:0/4036354122 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85580035b0 con 0x7f857015d860 2023-08-27T11:13:45.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.739+0000 7f856ed9d700 1 -- 172.21.15.45:0/4036354122 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d480 msgr2=0x7f85700cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:45.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.739+0000 7f856ed9d700 1 --2- 172.21.15.45:0/4036354122 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d480 0x7f85700cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.739+0000 7f856ed9d700 1 -- 172.21.15.45:0/4036354122 >> v1:172.21.15.45:6789/0 conn(0x7f857015d860 legacy=0x7f85700b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:45.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.739+0000 7f856d59a700 1 -- 172.21.15.45:0/4036354122 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2998940982 0 0) 0x7f85580035b0 con 0x7f857015d860 2023-08-27T11:13:45.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.739+0000 7f856ed9d700 1 -- 172.21.15.45:0/4036354122 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8570072610 con 0x7f85700ced40 2023-08-27T11:13:45.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.739+0000 7f856ed9d700 1 --2- 172.21.15.45:0/4036354122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85700ced40 0x7f85700cc220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f856800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cb3aa323c076931 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:45.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.739+0000 7f856d59a700 1 -- 172.21.15.45:0/4036354122 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800e070 con 0x7f85700ced40 2023-08-27T11:13:45.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.739+0000 7f856d59a700 1 -- 172.21.15.45:0/4036354122 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568003a20 con 0x7f85700ced40 2023-08-27T11:13:45.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.739+0000 7f856d59a700 1 -- 172.21.15.45:0/4036354122 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560001670 con 0x7f85700ced40 2023-08-27T11:13:45.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.740+0000 7f85748ae700 1 -- 172.21.15.45:0/4036354122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85700ced40 msgr2=0x7f85700cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:45.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.740+0000 7f85748ae700 1 --2- 172.21.15.45:0/4036354122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85700ced40 0x7f85700cc220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f856800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:45.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.740+0000 7f85748ae700 1 -- 172.21.15.45:0/4036354122 shutdown_connections 2023-08-27T11:13:45.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.740+0000 7f85748ae700 1 --2- 172.21.15.45:0/4036354122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85700ced40 0x7f85700cc220 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.740+0000 7f85748ae700 1 --2- 172.21.15.45:0/4036354122 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d480 0x7f85700cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.740+0000 7f85748ae700 1 -- 172.21.15.45:0/4036354122 >> 172.21.15.45:0/4036354122 conn(0x7f85700b2440 msgr2=0x7f85700b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:45.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.740+0000 7f85748ae700 1 -- 172.21.15.45:0/4036354122 shutdown_connections 2023-08-27T11:13:45.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.740+0000 7f85748ae700 1 -- 172.21.15.45:0/4036354122 wait complete. 2023-08-27T11:13:45.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.741+0000 7f85748ae700 1 Processor -- start 2023-08-27T11:13:45.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.741+0000 7f85748ae700 1 -- start start 2023-08-27T11:13:45.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f85748ae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85700ced40 0x7f85700c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f85748ae700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f857015d480 0x7f85700ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f85748ae700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d860 0x7f85700ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f85748ae700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857008b0f0 con 0x7f85700ced40 2023-08-27T11:13:45.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f85748ae700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85700c5e70 con 0x7f857015d480 2023-08-27T11:13:45.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f85748ae700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85700c55d0 con 0x7f857015d860 2023-08-27T11:13:45.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85700ced40 0x7f85700c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85700ced40 0x7f85700c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54378/0 (socket says 172.21.15.45:54378) 2023-08-27T11:13:45.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856e59c700 1 -- 172.21.15.45:0/1246065558 learned_addr learned my addr 172.21.15.45:0/1246065558 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:45.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856ed9d700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d860 0x7f85700ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856dd9b700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f857015d480 0x7f85700ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856e59c700 1 -- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d860 msgr2=0x7f85700ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:45.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856e59c700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d860 0x7f85700ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856e59c700 1 -- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f857015d480 msgr2=0x7f85700ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:45.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856e59c700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f857015d480 0x7f85700ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:45.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856e59c700 1 -- 172.21.15.45:0/1246065558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8568009a70 con 0x7f85700ced40 2023-08-27T11:13:45.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856dd9b700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f857015d480 0x7f85700ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:45.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.742+0000 7f856e59c700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85700ced40 0x7f85700c4af0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f856400e990 tx=0x7f856400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:45.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.743+0000 7f855effd700 1 -- 172.21.15.45:0/1246065558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85640096f0 con 0x7f85700ced40 2023-08-27T11:13:45.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.743+0000 7f855effd700 1 -- 172.21.15.45:0/1246065558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856400c850 con 0x7f85700ced40 2023-08-27T11:13:45.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.743+0000 7f855effd700 1 -- 172.21.15.45:0/1246065558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856400eee0 con 0x7f85700ced40 2023-08-27T11:13:45.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.743+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85700c58b0 con 0x7f85700ced40 2023-08-27T11:13:45.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.743+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85700c31b0 con 0x7f85700ced40 2023-08-27T11:13:45.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.744+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857009d4b0 con 0x7f85700ced40 2023-08-27T11:13:45.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.744+0000 7f855effd700 1 -- 172.21.15.45:0/1246065558 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f856400cbc0 con 0x7f85700ced40 2023-08-27T11:13:45.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.744+0000 7f855effd700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f855006a0e0 0x7f855006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:45.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.745+0000 7f855effd700 1 -- 172.21.15.45:0/1246065558 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8564088a30 con 0x7f85700ced40 2023-08-27T11:13:45.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.745+0000 7f856dd9b700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f855006a0e0 0x7f855006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:45.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.745+0000 7f856dd9b700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f855006a0e0 0x7f855006c5a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8560001eb0 tx=0x7f8560001c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:45.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:45.749+0000 7f855effd700 1 -- 172.21.15.45:0/1246065558 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8564054070 con 0x7f85700ced40 2023-08-27T11:13:46.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.009+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f85700734a0 con 0x7f85700ced40 2023-08-27T11:13:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.948+0000 7f855effd700 1 -- 172.21.15.45:0/1246065558 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 6) v1 ==== 88358+0+0 (secure 0 0 0) 0x7f856405bd50 con 0x7f85700ced40 2023-08-27T11:13:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.949+0000 7f855effd700 1 -- 172.21.15.45:0/1246065558 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8564054070 con 0x7f85700ced40 2023-08-27T11:13:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f855006a0e0 msgr2=0x7f855006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f855006a0e0 0x7f855006c5a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8560001eb0 tx=0x7f8560001c60 comp rx=0 tx=0).stop 2023-08-27T11:13:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85700ced40 msgr2=0x7f85700c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85700ced40 0x7f85700c4af0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f856400e990 tx=0x7f856400eca0 comp rx=0 tx=0).stop 2023-08-27T11:13:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 shutdown_connections 2023-08-27T11:13:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85700ced40 0x7f85700c4af0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f855006a0e0 0x7f855006c5a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f857015d860 0x7f85700ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 --2- 172.21.15.45:0/1246065558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f857015d480 0x7f85700ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:46.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 >> 172.21.15.45:0/1246065558 conn(0x7f85700b2440 msgr2=0x7f85700cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:46.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 shutdown_connections 2023-08-27T11:13:46.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:46.952+0000 7f85748ae700 1 -- 172.21.15.45:0/1246065558 wait complete. 2023-08-27T11:13:46.969 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-27T11:13:47.091 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:47.089+0000 7f6c7f37e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:13:47.098 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:47.096+0000 7fc6a9293200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:13:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.104+0000 7f36a49f7700 1 Processor -- start 2023-08-27T11:13:47.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.104+0000 7f36a49f7700 1 -- start start 2023-08-27T11:13:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.105+0000 7f36a49f7700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f36a015d3f0 0x7f36a0160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.105+0000 7f36a49f7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a00af420 0x7f36a006ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f36a49f7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a0071de0 con 0x7f36a00aed60 2023-08-27T11:13:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f36a49f7700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a00c6ca0 con 0x7f36a015d3f0 2023-08-27T11:13:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f36a49f7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a015d7d0 con 0x7f36a00af420 2023-08-27T11:13:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f36a00aed60 0x7f36a0161390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41186/0 (socket says 172.21.15.45:41186) 2023-08-27T11:13:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369dd9b700 1 -- 172.21.15.45:0/3415076681 learned_addr learned my addr 172.21.15.45:0/3415076681 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:47.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369e59c700 1 --2- 172.21.15.45:0/3415076681 >> v2:172.21.15.135:3300/0 conn(0x7f36a015d3f0 0x7f36a0160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369ed9d700 1 --2- 172.21.15.45:0/3415076681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a00af420 0x7f36a006ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369d59a700 1 -- 172.21.15.45:0/3415076681 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977223417 0 0) 0x7f36a0071de0 con 0x7f36a00aed60 2023-08-27T11:13:47.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369d59a700 1 -- 172.21.15.45:0/3415076681 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36880035b0 con 0x7f36a00aed60 2023-08-27T11:13:47.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369e59c700 1 -- 172.21.15.45:0/3415076681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a00af420 msgr2=0x7f36a006ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369e59c700 1 --2- 172.21.15.45:0/3415076681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a00af420 0x7f36a006ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369e59c700 1 -- 172.21.15.45:0/3415076681 >> v1:172.21.15.45:6789/0 conn(0x7f36a00aed60 legacy=0x7f36a0161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369e59c700 1 -- 172.21.15.45:0/3415076681 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a00c5310 con 0x7f36a015d3f0 2023-08-27T11:13:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369ed9d700 1 --2- 172.21.15.45:0/3415076681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a00af420 0x7f36a006ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369e59c700 1 --2- 172.21.15.45:0/3415076681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a015d3f0 0x7f36a0160e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3690009870 tx=0x7f3690009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32bd7a6ca06ba035 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:47.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.106+0000 7f369d59a700 1 -- 172.21.15.45:0/3415076681 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690011830 con 0x7f36a015d3f0 2023-08-27T11:13:47.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f369d59a700 1 -- 172.21.15.45:0/3415076681 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3690011990 con 0x7f36a015d3f0 2023-08-27T11:13:47.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f369d59a700 1 -- 172.21.15.45:0/3415076681 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690011d20 con 0x7f36a015d3f0 2023-08-27T11:13:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f36a49f7700 1 -- 172.21.15.45:0/3415076681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a015d3f0 msgr2=0x7f36a0160e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f36a49f7700 1 --2- 172.21.15.45:0/3415076681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a015d3f0 0x7f36a0160e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3690009870 tx=0x7f3690009b80 comp rx=0 tx=0).stop 2023-08-27T11:13:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f36a49f7700 1 -- 172.21.15.45:0/3415076681 shutdown_connections 2023-08-27T11:13:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f36a49f7700 1 --2- 172.21.15.45:0/3415076681 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a00af420 0x7f36a006ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f36a49f7700 1 --2- 172.21.15.45:0/3415076681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a015d3f0 0x7f36a0160e50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f36a49f7700 1 -- 172.21.15.45:0/3415076681 >> 172.21.15.45:0/3415076681 conn(0x7f36a00cfdd0 msgr2=0x7f36a00d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f36a49f7700 1 -- 172.21.15.45:0/3415076681 shutdown_connections 2023-08-27T11:13:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.107+0000 7f36a49f7700 1 -- 172.21.15.45:0/3415076681 wait complete. 2023-08-27T11:13:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.109+0000 7f36a49f7700 1 Processor -- start 2023-08-27T11:13:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.109+0000 7f36a49f7700 1 -- start start 2023-08-27T11:13:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.109+0000 7f36a49f7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a00aed60 0x7f36a01f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.109+0000 7f36a49f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00af420 0x7f36a01f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.109+0000 7f36a49f7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a015d3f0 0x7f36a00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.109+0000 7f36a49f7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a00b53c0 con 0x7f36a00af420 2023-08-27T11:13:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.109+0000 7f36a49f7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01f39f0 con 0x7f36a00aed60 2023-08-27T11:13:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.109+0000 7f36a49f7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01f28c0 con 0x7f36a015d3f0 2023-08-27T11:13:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00af420 0x7f36a01f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00af420 0x7f36a01f2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54416/0 (socket says 172.21.15.45:54416) 2023-08-27T11:13:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369dd9b700 1 -- 172.21.15.45:0/2767651548 learned_addr learned my addr 172.21.15.45:0/2767651548 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369ed9d700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a015d3f0 0x7f36a00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369e59c700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a00aed60 0x7f36a01f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369dd9b700 1 -- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a015d3f0 msgr2=0x7f36a00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369dd9b700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a015d3f0 0x7f36a00ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369dd9b700 1 -- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a00aed60 msgr2=0x7f36a01f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369dd9b700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a00aed60 0x7f36a01f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369dd9b700 1 -- 172.21.15.45:0/2767651548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3698009a70 con 0x7f36a00af420 2023-08-27T11:13:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f369dd9b700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00af420 0x7f36a01f2380 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f369400c700 tx=0x7f369400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f368effd700 1 -- 172.21.15.45:0/2767651548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694005d70 con 0x7f36a00af420 2023-08-27T11:13:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f368effd700 1 -- 172.21.15.45:0/2767651548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36940070d0 con 0x7f36a00af420 2023-08-27T11:13:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f368effd700 1 -- 172.21.15.45:0/2767651548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694007460 con 0x7f36a00af420 2023-08-27T11:13:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36900094a0 con 0x7f36a00af420 2023-08-27T11:13:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.110+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a00ccf50 con 0x7f36a00af420 2023-08-27T11:13:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.111+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a00727c0 con 0x7f36a00af420 2023-08-27T11:13:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.112+0000 7f368effd700 1 -- 172.21.15.45:0/2767651548 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 6) v1 ==== 88358+0+0 (secure 0 0 0) 0x7f36940075c0 con 0x7f36a00af420 2023-08-27T11:13:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.112+0000 7f368effd700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f368006a130 0x7f368006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.112+0000 7f369e59c700 1 -- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f368006a130 msgr2=0x7f368006c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/4120550306 2023-08-27T11:13:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.112+0000 7f368effd700 1 -- 172.21.15.45:0/2767651548 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3694087bc0 con 0x7f36a00af420 2023-08-27T11:13:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.112+0000 7f369e59c700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f368006a130 0x7f368006c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:13:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.117+0000 7f368effd700 1 -- 172.21.15.45:0/2767651548 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3694053170 con 0x7f36a00af420 2023-08-27T11:13:47.158 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:47.157+0000 7f6c7f37e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:13:47.170 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:47.169+0000 7fc6a9293200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:13:47.229 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:47.228+0000 7f6c7f37e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:13:47.244 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:47.243+0000 7fc6a9293200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:13:47.306 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:47.305+0000 7f6c7f37e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:13:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.313+0000 7f369e59c700 1 -- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f368006a130 msgr2=0x7f368006c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/4120550306 2023-08-27T11:13:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.313+0000 7f369e59c700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f368006a130 0x7f368006c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:13:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.318+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f36a0160b90 con 0x7f36a00af420 2023-08-27T11:13:47.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.322+0000 7f368effd700 1 -- 172.21.15.45:0/2767651548 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3694056790 con 0x7f36a00af420 2023-08-27T11:13:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f368006a130 msgr2=0x7f368006c5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:13:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f368006a130 0x7f368006c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00af420 msgr2=0x7f36a01f2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00af420 0x7f36a01f2380 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f369400c700 tx=0x7f369400ca10 comp rx=0 tx=0).stop 2023-08-27T11:13:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 shutdown_connections 2023-08-27T11:13:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f368006a130 0x7f368006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36a015d3f0 0x7f36a00ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36a00aed60 0x7f36a01f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 --2- 172.21.15.45:0/2767651548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a00af420 0x7f36a01f2380 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 >> 172.21.15.45:0/2767651548 conn(0x7f36a00cfdd0 msgr2=0x7f36a015c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:47.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 shutdown_connections 2023-08-27T11:13:47.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.324+0000 7f36a49f7700 1 -- 172.21.15.45:0/2767651548 wait complete. 2023-08-27T11:13:47.328 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:47.324+0000 7fc6a9293200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:13:47.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-27T11:13:47.447 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:47.446+0000 7f6c7f37e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:13:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.466+0000 7f6ac1e2b700 1 Processor -- start 2023-08-27T11:13:47.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.466+0000 7f6ac1e2b700 1 -- start start 2023-08-27T11:13:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6ac1e2b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6abc157c80 0x7f6abc06e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6ac1e2b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15b170 0x7f6abc0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6ac1e2b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc06eee0 con 0x7f6abc15a600 2023-08-27T11:13:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6ac1e2b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6abc0ca3b0 con 0x7f6abc157c80 2023-08-27T11:13:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6ac1e2b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc06ecd0 con 0x7f6abc15b170 2023-08-27T11:13:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6abaffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6abc15a600 0x7f6abc15aa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41246/0 (socket says 172.21.15.45:41246) 2023-08-27T11:13:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6abaffd700 1 -- 172.21.15.45:0/2464184238 learned_addr learned my addr 172.21.15.45:0/2464184238 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:47.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6abb7fe700 1 --2- 172.21.15.45:0/2464184238 >> v2:172.21.15.135:3300/0 conn(0x7f6abc157c80 0x7f6abc06e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6abbfff700 1 --2- 172.21.15.45:0/2464184238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15b170 0x7f6abc0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6aba7fc700 1 -- 172.21.15.45:0/2464184238 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720356269 0 0) 0x7f6abc06eee0 con 0x7f6abc15a600 2023-08-27T11:13:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.467+0000 7f6aba7fc700 1 -- 172.21.15.45:0/2464184238 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aa40035b0 con 0x7f6abc15a600 2023-08-27T11:13:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.468+0000 7f6abb7fe700 1 -- 172.21.15.45:0/2464184238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15b170 msgr2=0x7f6abc0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.468+0000 7f6abb7fe700 1 --2- 172.21.15.45:0/2464184238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15b170 0x7f6abc0cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.468+0000 7f6abb7fe700 1 -- 172.21.15.45:0/2464184238 >> v1:172.21.15.45:6789/0 conn(0x7f6abc15a600 legacy=0x7f6abc15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.468+0000 7f6abb7fe700 1 -- 172.21.15.45:0/2464184238 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0ca5c0 con 0x7f6abc157c80 2023-08-27T11:13:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.468+0000 7f6abbfff700 1 --2- 172.21.15.45:0/2464184238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15b170 0x7f6abc0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:47.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.468+0000 7f6abb7fe700 1 --2- 172.21.15.45:0/2464184238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6abc157c80 0x7f6abc06e790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009840 tx=0x7f6aac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8780ba480a8b0759 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:47.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.468+0000 7f6aba7fc700 1 -- 172.21.15.45:0/2464184238 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac011830 con 0x7f6abc157c80 2023-08-27T11:13:47.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.468+0000 7f6aba7fc700 1 -- 172.21.15.45:0/2464184238 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aac011990 con 0x7f6abc157c80 2023-08-27T11:13:47.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.468+0000 7f6aba7fc700 1 -- 172.21.15.45:0/2464184238 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac011d00 con 0x7f6abc157c80 2023-08-27T11:13:47.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.469+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/2464184238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6abc157c80 msgr2=0x7f6abc06e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.469+0000 7f6ac1e2b700 1 --2- 172.21.15.45:0/2464184238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6abc157c80 0x7f6abc06e790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009840 tx=0x7f6aac009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:47.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.469+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/2464184238 shutdown_connections 2023-08-27T11:13:47.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.469+0000 7f6ac1e2b700 1 --2- 172.21.15.45:0/2464184238 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15b170 0x7f6abc0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.469+0000 7f6ac1e2b700 1 --2- 172.21.15.45:0/2464184238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6abc157c80 0x7f6abc06e790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.469+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/2464184238 >> 172.21.15.45:0/2464184238 conn(0x7f6abc0cc7e0 msgr2=0x7f6abc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:47.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.469+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/2464184238 shutdown_connections 2023-08-27T11:13:47.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.469+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/2464184238 wait complete. 2023-08-27T11:13:47.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.470+0000 7f6ac1e2b700 1 Processor -- start 2023-08-27T11:13:47.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.470+0000 7f6ac1e2b700 1 -- start start 2023-08-27T11:13:47.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6ac1e2b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6abc157c80 0x7f6abc0c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6ac1e2b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15a600 0x7f6abc1f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6ac1e2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6abc15b170 0x7f6abc158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6ac1e2b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc1e4900 con 0x7f6abc15b170 2023-08-27T11:13:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6ac1e2b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc1e5480 con 0x7f6abc157c80 2023-08-27T11:13:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6ac1e2b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc1e4b50 con 0x7f6abc15a600 2023-08-27T11:13:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6abc15b170 0x7f6abc158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6abc15b170 0x7f6abc158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54478/0 (socket says 172.21.15.45:54478) 2023-08-27T11:13:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abbfff700 1 -- 172.21.15.45:0/3796391618 learned_addr learned my addr 172.21.15.45:0/3796391618 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abb7fe700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6abc157c80 0x7f6abc0c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abaffd700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15a600 0x7f6abc1f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abbfff700 1 -- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15a600 msgr2=0x7f6abc1f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abbfff700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15a600 0x7f6abc1f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abbfff700 1 -- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6abc157c80 msgr2=0x7f6abc0c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abbfff700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6abc157c80 0x7f6abc0c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abbfff700 1 -- 172.21.15.45:0/3796391618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab4009a70 con 0x7f6abc15b170 2023-08-27T11:13:47.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.471+0000 7f6abbfff700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6abc15b170 0x7f6abc158760 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400efb0 tx=0x7f6ab400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:47.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.472+0000 7f6aa3fff700 1 -- 172.21.15.45:0/3796391618 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab400cc90 con 0x7f6abc15b170 2023-08-27T11:13:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.472+0000 7f6aa3fff700 1 -- 172.21.15.45:0/3796391618 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab400f040 con 0x7f6abc15b170 2023-08-27T11:13:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.472+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac0094a0 con 0x7f6abc15b170 2023-08-27T11:13:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.472+0000 7f6aa3fff700 1 -- 172.21.15.45:0/3796391618 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4014480 con 0x7f6abc15b170 2023-08-27T11:13:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.472+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc1e5110 con 0x7f6abc15b170 2023-08-27T11:13:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.473+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc1e5630 con 0x7f6abc15b170 2023-08-27T11:13:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.473+0000 7f6aa3fff700 1 -- 172.21.15.45:0/3796391618 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 6) v1 ==== 88358+0+0 (secure 0 0 0) 0x7f6ab40145e0 con 0x7f6abc15b170 2023-08-27T11:13:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.474+0000 7f6aa3fff700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f6aa806a130 0x7f6aa806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.474+0000 7f6abb7fe700 1 -- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f6aa806a130 msgr2=0x7f6aa806c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/4120550306 2023-08-27T11:13:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.474+0000 7f6aa3fff700 1 -- 172.21.15.45:0/3796391618 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6ab4089340 con 0x7f6abc15b170 2023-08-27T11:13:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.474+0000 7f6abb7fe700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f6aa806a130 0x7f6aa806c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:13:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.478+0000 7f6aa3fff700 1 -- 172.21.15.45:0/3796391618 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6abc1e5630 con 0x7f6abc15b170 2023-08-27T11:13:47.487 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:47.481+0000 7fc6a9293200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:13:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.675+0000 7f6abb7fe700 1 -- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f6aa806a130 msgr2=0x7f6aa806c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/4120550306 2023-08-27T11:13:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.675+0000 7f6abb7fe700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f6aa806a130 0x7f6aa806c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:13:47.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.683+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f6abc0b28c0 con 0x7f6abc15b170 2023-08-27T11:13:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.686+0000 7f6aa3fff700 1 -- 172.21.15.45:0/3796391618 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6abc0b28c0 con 0x7f6abc15b170 2023-08-27T11:13:47.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f6aa806a130 msgr2=0x7f6aa806c5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:13:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f6aa806a130 0x7f6aa806c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6abc15b170 msgr2=0x7f6abc158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6abc15b170 0x7f6abc158760 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400efb0 tx=0x7f6ab400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 shutdown_connections 2023-08-27T11:13:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6abc15b170 0x7f6abc158760 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f6aa806a130 0x7f6aa806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6abc157c80 0x7f6abc0c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 --2- 172.21.15.45:0/3796391618 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6abc15a600 0x7f6abc1f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.688+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 >> 172.21.15.45:0/3796391618 conn(0x7f6abc0cc7e0 msgr2=0x7f6abc0ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.689+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 shutdown_connections 2023-08-27T11:13:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.689+0000 7f6ac1e2b700 1 -- 172.21.15.45:0/3796391618 wait complete. 2023-08-27T11:13:47.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:47.703 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-27T11:13:47.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:47.727 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-08-27T11:13:47.773 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:47.772+0000 7f6c7f37e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:13:47.786 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-08-27T11:13:47.828 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:47.826+0000 7fc6a9293200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:13:47.888 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:47.887+0000 7f6c7f37e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:13:47.952 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:47.950+0000 7fc6a9293200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:13:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.958+0000 7f102e55c700 1 Processor -- start 2023-08-27T11:13:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.958+0000 7f102e55c700 1 -- start start 2023-08-27T11:13:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f102e55c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f102815a0f0 0x7f10280cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f102e55c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f102806ed90 0x7f102815e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f102e55c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280cfae0 con 0x7f10280cd300 2023-08-27T11:13:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f102e55c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102815e900 con 0x7f102815a0f0 2023-08-27T11:13:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f102e55c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10280cae60 con 0x7f102806ed90 2023-08-27T11:13:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f1027fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f102815a0f0 0x7f10280cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f1027fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f102815a0f0 0x7f10280cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54528/0 (socket says 172.21.15.45:54528) 2023-08-27T11:13:47.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f1027fff700 1 -- 172.21.15.45:0/270650466 learned_addr learned my addr 172.21.15.45:0/270650466 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f102caf9700 1 --2- 172.21.15.45:0/270650466 >> v2:172.21.15.135:3300/0 conn(0x7f102806ed90 0x7f102815e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f1027fff700 1 -- 172.21.15.45:0/270650466 >> v2:172.21.15.135:3300/0 conn(0x7f102806ed90 msgr2=0x7f102815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f1027fff700 1 --2- 172.21.15.45:0/270650466 >> v2:172.21.15.135:3300/0 conn(0x7f102806ed90 0x7f102815e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f1027fff700 1 -- 172.21.15.45:0/270650466 >> v1:172.21.15.135:6789/0 conn(0x7f10280cd300 legacy=0x7f102815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f1027fff700 1 -- 172.21.15.45:0/270650466 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280cb140 con 0x7f102815a0f0 2023-08-27T11:13:47.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.959+0000 7f1027fff700 1 --2- 172.21.15.45:0/270650466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102815a0f0 0x7f10280cbc20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1028070e00 tx=0x7f1018009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da41e1f6cdb026c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:47.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f1026ffd700 1 -- 172.21.15.45:0/270650466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018011830 con 0x7f102815a0f0 2023-08-27T11:13:47.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f1026ffd700 1 -- 172.21.15.45:0/270650466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018011990 con 0x7f102815a0f0 2023-08-27T11:13:47.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f1026ffd700 1 -- 172.21.15.45:0/270650466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018011d20 con 0x7f102815a0f0 2023-08-27T11:13:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f102e55c700 1 -- 172.21.15.45:0/270650466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102815a0f0 msgr2=0x7f10280cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f102e55c700 1 --2- 172.21.15.45:0/270650466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102815a0f0 0x7f10280cbc20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1028070e00 tx=0x7f1018009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f102e55c700 1 -- 172.21.15.45:0/270650466 shutdown_connections 2023-08-27T11:13:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f102e55c700 1 --2- 172.21.15.45:0/270650466 >> v2:172.21.15.135:3300/0 conn(0x7f102806ed90 0x7f102815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f102e55c700 1 --2- 172.21.15.45:0/270650466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102815a0f0 0x7f10280cbc20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f102e55c700 1 -- 172.21.15.45:0/270650466 >> 172.21.15.45:0/270650466 conn(0x7f10280cc7e0 msgr2=0x7f10280ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f102e55c700 1 -- 172.21.15.45:0/270650466 shutdown_connections 2023-08-27T11:13:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.960+0000 7f102e55c700 1 -- 172.21.15.45:0/270650466 wait complete. 2023-08-27T11:13:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.961+0000 7f102e55c700 1 Processor -- start 2023-08-27T11:13:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.962+0000 7f102e55c700 1 -- start start 2023-08-27T11:13:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.962+0000 7f102e55c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f102806ed90 0x7f10280c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.962+0000 7f102e55c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10280cd300 0x7f10280c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.962+0000 7f102e55c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f102815a0f0 0x7f10281536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.962+0000 7f102e55c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028153c10 con 0x7f10280cd300 2023-08-27T11:13:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.962+0000 7f102e55c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280c8660 con 0x7f102815a0f0 2023-08-27T11:13:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.962+0000 7f102e55c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280c8940 con 0x7f102806ed90 2023-08-27T11:13:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f10277fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10280cd300 0x7f10280c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f10277fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10280cd300 0x7f10280c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54534/0 (socket says 172.21.15.45:54534) 2023-08-27T11:13:47.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f10277fe700 1 -- 172.21.15.45:0/3312212213 learned_addr learned my addr 172.21.15.45:0/3312212213 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f102caf9700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f102815a0f0 0x7f10281536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f1027fff700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f102806ed90 0x7f10280c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f10277fe700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f102806ed90 msgr2=0x7f10280c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f10277fe700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f102806ed90 0x7f10280c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f10277fe700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f102815a0f0 msgr2=0x7f10281536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f10277fe700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f102815a0f0 0x7f10281536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f10277fe700 1 -- 172.21.15.45:0/3312212213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1018009550 con 0x7f10280cd300 2023-08-27T11:13:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f10277fe700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10280cd300 0x7f10280c62c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c4e0 tx=0x7f101c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c0052d0 con 0x7f10280cd300 2023-08-27T11:13:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c005430 con 0x7f10280cd300 2023-08-27T11:13:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10280c8c20 con 0x7f10280cd300 2023-08-27T11:13:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c0038c0 con 0x7f10280cd300 2023-08-27T11:13:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.963+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10281f2d60 con 0x7f10280cd300 2023-08-27T11:13:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.964+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10281f3350 con 0x7f10280cd300 2023-08-27T11:13:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.965+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 6) v1 ==== 88358+0+0 (secure 0 0 0) 0x7f101c003a20 con 0x7f10280cd300 2023-08-27T11:13:47.979 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:47.961+0000 7f6c7f37e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:13:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.965+0000 7f1024ff9700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 0x7f101406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.965+0000 7f1027fff700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 msgr2=0x7f101406c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/4120550306 2023-08-27T11:13:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.965+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f101c087ab0 con 0x7f10280cd300 2023-08-27T11:13:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.965+0000 7f1027fff700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 0x7f101406c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:13:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:47.970+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f10281f3350 con 0x7f10280cd300 2023-08-27T11:13:48.030 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:48.029+0000 7fc6a9293200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:13:48.043 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:48.042+0000 7f6c7f37e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:13:48.114 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:48.113+0000 7fc6a9293200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:13:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:48.166+0000 7f1027fff700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 msgr2=0x7f101406c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/4120550306 2023-08-27T11:13:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:48.166+0000 7f1027fff700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 0x7f101406c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:13:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:48.196+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 --> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f10280cb700 con 0x7f101406a130 2023-08-27T11:13:48.410 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:48.409+0000 7f6c7f37e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:13:48.499 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:48.497+0000 7fc6a9293200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:13:48.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:48.567+0000 7f1027fff700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 msgr2=0x7f101406c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/4120550306 2023-08-27T11:13:48.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:48.567+0000 7f1027fff700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 0x7f101406c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:13:48.977 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:48.976+0000 7f6c7f37e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:13:49.047 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:49.046+0000 7f6c7f37e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:13:49.103 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:49.101+0000 7fc6a9293200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:13:49.175 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:49.174+0000 7fc6a9293200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:13:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:49.368+0000 7f1027fff700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 msgr2=0x7f101406c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/4120550306 2023-08-27T11:13:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:49.368+0000 7f1027fff700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 0x7f101406c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:13:49.673 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:49.672+0000 7f6c7f37e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:13:49.750 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:49.749+0000 7f6c7f37e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:13:49.832 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:49.830+0000 7fc6a9293200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:13:49.893 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:49.892+0000 7f6c7f37e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:13:49.912 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:49.911+0000 7fc6a9293200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:13:50.061 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:50.060+0000 7fc6a9293200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:13:50.119 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:50.118+0000 7f6c7f37e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:13:50.193 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:50.192+0000 7f6c7f37e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:13:50.295 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:50.293+0000 7fc6a9293200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:13:50.370 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:50.369+0000 7fc6a9293200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:13:50.456 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:50.455+0000 7f6c7f37e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:13:50.644 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:50.642+0000 7fc6a9293200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:13:50.676 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:50.675+0000 7f6c7f37e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:13:50.825 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:50.824+0000 7f6c7f37e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:13:50.870 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:50.869+0000 7fc6a9293200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:13:50.937 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:50.936+0000 7f6c7f37e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:13:50.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:50.969+0000 7f1027fff700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 msgr2=0x7f101406c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/4120550306 2023-08-27T11:13:50.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:50.970+0000 7f1027fff700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 0x7f101406c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-27T11:13:51.023 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:51.021+0000 7fc6a9293200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:13:51.091 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:51.090+0000 7f6c7f37e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:13:51.140 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:51.139+0000 7fc6a9293200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:13:51.298 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:51.296+0000 7fc6a9293200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:13:51.354 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:51.353+0000 7f6c7f37e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:13:51.514 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:51.513+0000 7f6c7f37e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:13:51.569 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:51.567+0000 7fc6a9293200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:13:51.732 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:51.731+0000 7fc6a9293200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:13:52.034 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.033+0000 7f6c7f37e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:13:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:52.053+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 7) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f101c053430 con 0x7f10280cd300 2023-08-27T11:13:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:52.053+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 msgr2=0x7f101406c5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:13:52.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:52.053+0000 7f1024ff9700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/4120550306,v1:172.21.15.135:6833/4120550306] conn(0x7f101406a130 0x7f101406c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:52.176 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf23dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.176 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf23dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.176 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf23dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.177 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf23dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.177 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf23dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.177 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf53e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.178 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf53e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.178 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf53e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.178 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf53e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.179 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:13:52.174+0000 7f6bf53e2700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:13:52.293 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:13:52.290+0000 7fc6a9293200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:13:53.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.057+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 8) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f101c054410 con 0x7f10280cd300 2023-08-27T11:13:53.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.057+0000 7f1024ff9700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1014072b90 0x7f1014074f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.057+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1014027cf0 con 0x7f1014072b90 2023-08-27T11:13:53.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.061+0000 7f1027fff700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1014072b90 0x7f1014074f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.063+0000 7f1027fff700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1014072b90 0x7f1014074f80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1018014040 tx=0x7f1018008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:53.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.129+0000 7f1024ff9700 1 -- 172.21.15.45:0/3312212213 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1018004140 con 0x7f1014072b90 2023-08-27T11:13:53.131 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir0 2023-08-27T11:13:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.131+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1014072b90 msgr2=0x7f1014074f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.131+0000 7f102e55c700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1014072b90 0x7f1014074f80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1018014040 tx=0x7f1018008030 comp rx=0 tx=0).stop 2023-08-27T11:13:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.131+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10280cd300 msgr2=0x7f10280c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.131+0000 7f102e55c700 1 --2- 172.21.15.45:0/3312212213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10280cd300 0x7f10280c62c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c4e0 tx=0x7f101c00c7f0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.131+0000 7f102caf9700 1 -- 172.21.15.45:0/3312212213 reap_dead start 2023-08-27T11:13:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.131+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 shutdown_connections 2023-08-27T11:13:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.131+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 >> 172.21.15.45:0/3312212213 conn(0x7f10280cc7e0 msgr2=0x7f10280cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.131+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 shutdown_connections 2023-08-27T11:13:53.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.131+0000 7f102e55c700 1 -- 172.21.15.45:0/3312212213 wait complete. 2023-08-27T11:13:53.146 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-08-27T11:13:53.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:53.147 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-08-27T11:13:53.167 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-08-27T11:13:53.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.337+0000 7fae98edf700 1 Processor -- start 2023-08-27T11:13:53.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.337+0000 7fae98edf700 1 -- start start 2023-08-27T11:13:53.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.338+0000 7fae98edf700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae940ae630 0x7fae94073a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.338+0000 7fae98edf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae940723a0 con 0x7fae9406a280 2023-08-27T11:13:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.338+0000 7fae98edf700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae940cbdf0 con 0x7fae9415d560 2023-08-27T11:13:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.338+0000 7fae98edf700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9415d940 con 0x7fae940ae630 2023-08-27T11:13:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.338+0000 7fae92d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fae9406a280 0x7fae9406bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41394/0 (socket says 172.21.15.45:41394) 2023-08-27T11:13:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.338+0000 7fae92d9d700 1 -- 172.21.15.45:0/465694156 learned_addr learned my addr 172.21.15.45:0/465694156 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:53.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.338+0000 7fae91d9b700 1 --2- 172.21.15.45:0/465694156 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae940ae630 0x7fae94073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.338+0000 7fae9159a700 1 -- 172.21.15.45:0/465694156 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604227421 0 0) 0x7fae940723a0 con 0x7fae9406a280 2023-08-27T11:13:53.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.338+0000 7fae9159a700 1 -- 172.21.15.45:0/465694156 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7c0035b0 con 0x7fae9406a280 2023-08-27T11:13:53.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.339+0000 7fae9159a700 1 -- 172.21.15.45:0/465694156 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455941029 0 0) 0x7fae940cbdf0 con 0x7fae9415d560 2023-08-27T11:13:53.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.339+0000 7fae9159a700 1 -- 172.21.15.45:0/465694156 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae940723a0 con 0x7fae9415d560 2023-08-27T11:13:53.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.339+0000 7fae91d9b700 1 -- 172.21.15.45:0/465694156 >> v1:172.21.15.135:6789/0 conn(0x7fae9415d560 legacy=0x7fae940cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.339+0000 7fae91d9b700 1 -- 172.21.15.45:0/465694156 >> v1:172.21.15.45:6789/0 conn(0x7fae9406a280 legacy=0x7fae9406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.339+0000 7fae91d9b700 1 -- 172.21.15.45:0/465694156 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940cccc0 con 0x7fae940ae630 2023-08-27T11:13:53.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.339+0000 7fae91d9b700 1 --2- 172.21.15.45:0/465694156 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae940ae630 0x7fae94073a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae84009870 tx=0x7fae84009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=252cfd76c4b5912f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.339+0000 7fae9159a700 1 -- 172.21.15.45:0/465694156 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84011830 con 0x7fae940ae630 2023-08-27T11:13:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.339+0000 7fae9159a700 1 -- 172.21.15.45:0/465694156 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae84011990 con 0x7fae940ae630 2023-08-27T11:13:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.339+0000 7fae9159a700 1 -- 172.21.15.45:0/465694156 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84011d00 con 0x7fae940ae630 2023-08-27T11:13:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.340+0000 7fae98edf700 1 -- 172.21.15.45:0/465694156 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae940ae630 msgr2=0x7fae94073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.340+0000 7fae98edf700 1 --2- 172.21.15.45:0/465694156 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae940ae630 0x7fae94073a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae84009870 tx=0x7fae84009b80 comp rx=0 tx=0).stop 2023-08-27T11:13:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.340+0000 7fae98edf700 1 -- 172.21.15.45:0/465694156 shutdown_connections 2023-08-27T11:13:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.340+0000 7fae98edf700 1 --2- 172.21.15.45:0/465694156 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae940ae630 0x7fae94073a10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.340+0000 7fae98edf700 1 -- 172.21.15.45:0/465694156 >> 172.21.15.45:0/465694156 conn(0x7fae940b2420 msgr2=0x7fae940b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.340+0000 7fae98edf700 1 -- 172.21.15.45:0/465694156 shutdown_connections 2023-08-27T11:13:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.340+0000 7fae98edf700 1 -- 172.21.15.45:0/465694156 wait complete. 2023-08-27T11:13:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.341+0000 7fae98edf700 1 Processor -- start 2023-08-27T11:13:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.341+0000 7fae98edf700 1 -- start start 2023-08-27T11:13:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae98edf700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae9406a280 0x7fae941f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae98edf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae940ae630 0x7fae940af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae98edf700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae9415d560 0x7fae940b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae98edf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae941f2830 con 0x7fae940ae630 2023-08-27T11:13:53.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae98edf700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940b1de0 con 0x7fae9415d560 2023-08-27T11:13:53.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae98edf700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940afed0 con 0x7fae9406a280 2023-08-27T11:13:53.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae91d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae940ae630 0x7fae940af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae91d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae940ae630 0x7fae940af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54622/0 (socket says 172.21.15.45:54622) 2023-08-27T11:13:53.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae91d9b700 1 -- 172.21.15.45:0/2743439192 learned_addr learned my addr 172.21.15.45:0/2743439192 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae92d9d700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae9415d560 0x7fae940b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae9259c700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae9406a280 0x7fae941f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae91d9b700 1 -- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae9406a280 msgr2=0x7fae941f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae91d9b700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae9406a280 0x7fae941f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae91d9b700 1 -- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae9415d560 msgr2=0x7fae940b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae91d9b700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae9415d560 0x7fae940b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.342+0000 7fae91d9b700 1 -- 172.21.15.45:0/2743439192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae840094a0 con 0x7fae940ae630 2023-08-27T11:13:53.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.343+0000 7fae91d9b700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae940ae630 0x7fae940af7f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fae840097c0 tx=0x7fae84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:53.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.343+0000 7fae82ffd700 1 -- 172.21.15.45:0/2743439192 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840044d0 con 0x7fae940ae630 2023-08-27T11:13:53.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.343+0000 7fae82ffd700 1 -- 172.21.15.45:0/2743439192 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8401f070 con 0x7fae940ae630 2023-08-27T11:13:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.343+0000 7fae98edf700 1 -- 172.21.15.45:0/2743439192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae940b01b0 con 0x7fae940ae630 2023-08-27T11:13:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.343+0000 7fae82ffd700 1 -- 172.21.15.45:0/2743439192 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840038e0 con 0x7fae940ae630 2023-08-27T11:13:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.343+0000 7fae98edf700 1 -- 172.21.15.45:0/2743439192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae940b0770 con 0x7fae940ae630 2023-08-27T11:13:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.344+0000 7fae80ff9700 1 -- 172.21.15.45:0/2743439192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6c0052f0 con 0x7fae940ae630 2023-08-27T11:13:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.344+0000 7fae82ffd700 1 -- 172.21.15.45:0/2743439192 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 88196+0+0 (secure 0 0 0) 0x7fae84003a40 con 0x7fae940ae630 2023-08-27T11:13:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.345+0000 7fae82ffd700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fae74069f80 0x7fae7406c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.345+0000 7fae82ffd700 1 -- 172.21.15.45:0/2743439192 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fae84021070 con 0x7fae940ae630 2023-08-27T11:13:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.345+0000 7fae9259c700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fae74069f80 0x7fae7406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.345+0000 7fae9259c700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fae74069f80 0x7fae7406c440 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae88007fd0 tx=0x7fae88009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:53.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.349+0000 7fae82ffd700 1 -- 172.21.15.45:0/2743439192 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fae84058e50 con 0x7fae940ae630 2023-08-27T11:13:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.576+0000 7fae80ff9700 1 -- 172.21.15.45:0/2743439192 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fae6c000bc0 con 0x7fae74069f80 2023-08-27T11:13:53.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.646+0000 7fae82ffd700 1 -- 172.21.15.45:0/2743439192 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fae6c000bc0 con 0x7fae74069f80 2023-08-27T11:13:53.648 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir1 2023-08-27T11:13:53.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 -- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fae74069f80 msgr2=0x7fae7406c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fae74069f80 0x7fae7406c440 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae88007fd0 tx=0x7fae88009040 comp rx=0 tx=0).stop 2023-08-27T11:13:53.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 -- 172.21.15.45:0/2743439192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae940ae630 msgr2=0x7fae940af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae940ae630 0x7fae940af7f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fae840097c0 tx=0x7fae84004750 comp rx=0 tx=0).stop 2023-08-27T11:13:53.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 -- 172.21.15.45:0/2743439192 shutdown_connections 2023-08-27T11:13:53.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fae74069f80 0x7fae7406c440 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae9415d560 0x7fae940b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fae9406a280 0x7fae941f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 --2- 172.21.15.45:0/2743439192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae940ae630 0x7fae940af7f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 -- 172.21.15.45:0/2743439192 >> 172.21.15.45:0/2743439192 conn(0x7fae940b2420 msgr2=0x7fae940cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:53.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 -- 172.21.15.45:0/2743439192 shutdown_connections 2023-08-27T11:13:53.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.648+0000 7fae80ff9700 1 -- 172.21.15.45:0/2743439192 wait complete. 2023-08-27T11:13:53.662 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-08-27T11:13:53.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:53.663 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-08-27T11:13:53.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-08-27T11:13:53.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.850+0000 7f554a419700 1 Processor -- start 2023-08-27T11:13:53.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.850+0000 7f554a419700 1 -- start start 2023-08-27T11:13:53.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f554a419700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440ca620 0x7f55440cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f554a419700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f55440caa00 0x7f55440acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f554a419700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55440ce340 con 0x7f55440af200 2023-08-27T11:13:53.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f554a419700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55440ceec0 con 0x7f55440caa00 2023-08-27T11:13:53.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f554a419700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554415d7c0 con 0x7f55440ca620 2023-08-27T11:13:53.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f55489b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f55440caa00 0x7f55440acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f55489b6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f55440caa00 0x7f55440acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54632/0 (socket says 172.21.15.45:54632) 2023-08-27T11:13:53.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f55489b6700 1 -- 172.21.15.45:0/2279785282 learned_addr learned my addr 172.21.15.45:0/2279785282 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:53.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f5543fff700 1 --2- 172.21.15.45:0/2279785282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440ca620 0x7f55440cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f55489b6700 1 -- 172.21.15.45:0/2279785282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440ca620 msgr2=0x7f55440cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:53.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f55489b6700 1 --2- 172.21.15.45:0/2279785282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440ca620 0x7f55440cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f55489b6700 1 -- 172.21.15.45:0/2279785282 >> v1:172.21.15.135:6789/0 conn(0x7f55440af200 legacy=0x7f55440ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:53.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f55489b6700 1 -- 172.21.15.45:0/2279785282 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440cc570 con 0x7f55440caa00 2023-08-27T11:13:53.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f55489b6700 1 --2- 172.21.15.45:0/2279785282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440caa00 0x7f55440acfc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f553c009e50 tx=0x7f553c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132e2dee2cda661f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f5542ffd700 1 -- 172.21.15.45:0/2279785282 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00e070 con 0x7f55440caa00 2023-08-27T11:13:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.851+0000 7f5542ffd700 1 -- 172.21.15.45:0/2279785282 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c003a20 con 0x7f55440caa00 2023-08-27T11:13:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.852+0000 7f5542ffd700 1 -- 172.21.15.45:0/2279785282 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c003dd0 con 0x7f55440caa00 2023-08-27T11:13:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.852+0000 7f554a419700 1 -- 172.21.15.45:0/2279785282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440caa00 msgr2=0x7f55440acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.852+0000 7f554a419700 1 --2- 172.21.15.45:0/2279785282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440caa00 0x7f55440acfc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f553c009e50 tx=0x7f553c00b330 comp rx=0 tx=0).stop 2023-08-27T11:13:53.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.852+0000 7f554a419700 1 -- 172.21.15.45:0/2279785282 shutdown_connections 2023-08-27T11:13:53.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.852+0000 7f554a419700 1 --2- 172.21.15.45:0/2279785282 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440ca620 0x7f55440cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.852+0000 7f554a419700 1 --2- 172.21.15.45:0/2279785282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440caa00 0x7f55440acfc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.852+0000 7f554a419700 1 -- 172.21.15.45:0/2279785282 >> 172.21.15.45:0/2279785282 conn(0x7f55440cc7e0 msgr2=0x7f55440ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:53.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.852+0000 7f554a419700 1 -- 172.21.15.45:0/2279785282 shutdown_connections 2023-08-27T11:13:53.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.852+0000 7f554a419700 1 -- 172.21.15.45:0/2279785282 wait complete. 2023-08-27T11:13:53.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.853+0000 7f554a419700 1 Processor -- start 2023-08-27T11:13:53.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f554a419700 1 -- start start 2023-08-27T11:13:53.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f554a419700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440af200 0x7f55440b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f554a419700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440ca620 0x7f55440c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f554a419700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55440caa00 0x7f55440c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f554a419700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55440c7370 con 0x7f55440ca620 2023-08-27T11:13:53.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f554a419700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55440c75d0 con 0x7f55440caa00 2023-08-27T11:13:53.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f554a419700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55440c78b0 con 0x7f55440af200 2023-08-27T11:13:53.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f55437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440ca620 0x7f55440c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f55437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440ca620 0x7f55440c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54638/0 (socket says 172.21.15.45:54638) 2023-08-27T11:13:53.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.854+0000 7f55437fe700 1 -- 172.21.15.45:0/2067232560 learned_addr learned my addr 172.21.15.45:0/2067232560 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:53.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f5543fff700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440af200 0x7f55440b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f55489b6700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55440caa00 0x7f55440c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f55437fe700 1 -- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440af200 msgr2=0x7f55440b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f55437fe700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440af200 0x7f55440b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f55437fe700 1 -- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55440caa00 msgr2=0x7f55440c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:53.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f55437fe700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55440caa00 0x7f55440c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:53.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f55437fe700 1 -- 172.21.15.45:0/2067232560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c009b30 con 0x7f55440ca620 2023-08-27T11:13:53.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f55489b6700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55440caa00 0x7f55440c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:53.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f5543fff700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440af200 0x7f55440b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:53.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f55437fe700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440ca620 0x7f55440c6e30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f553800c4a0 tx=0x7f553800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:53.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f5540ff9700 1 -- 172.21.15.45:0/2067232560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538011790 con 0x7f55440ca620 2023-08-27T11:13:53.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f5540ff9700 1 -- 172.21.15.45:0/2067232560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55380118f0 con 0x7f55440ca620 2023-08-27T11:13:53.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f5540ff9700 1 -- 172.21.15.45:0/2067232560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538011ca0 con 0x7f55440ca620 2023-08-27T11:13:53.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55440c7b90 con 0x7f55440ca620 2023-08-27T11:13:53.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.855+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55440c8150 con 0x7f55440ca620 2023-08-27T11:13:53.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.856+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55440cbce0 con 0x7f55440ca620 2023-08-27T11:13:53.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.856+0000 7f5540ff9700 1 -- 172.21.15.45:0/2067232560 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f553801c460 con 0x7f55440ca620 2023-08-27T11:13:53.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.857+0000 7f5540ff9700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5530069f80 0x7f553006c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:53.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.857+0000 7f5540ff9700 1 -- 172.21.15.45:0/2067232560 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f5538088280 con 0x7f55440ca620 2023-08-27T11:13:53.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.857+0000 7f5543fff700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5530069f80 0x7f553006c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:53.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.858+0000 7f5543fff700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5530069f80 0x7f553006c440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55340099f0 tx=0x7f5534008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:53.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:53.861+0000 7f5540ff9700 1 -- 172.21.15.45:0/2067232560 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5538053a70 con 0x7f55440ca620 2023-08-27T11:13:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.092+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f55440ce870 con 0x7f5530069f80 2023-08-27T11:13:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.165+0000 7f5540ff9700 1 -- 172.21.15.45:0/2067232560 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5534009680 con 0x7f5530069f80 2023-08-27T11:13:54.167 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir2 2023-08-27T11:13:54.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5530069f80 msgr2=0x7f553006c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5530069f80 0x7f553006c440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55340099f0 tx=0x7f5534008040 comp rx=0 tx=0).stop 2023-08-27T11:13:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440ca620 msgr2=0x7f55440c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440ca620 0x7f55440c6e30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f553800c4a0 tx=0x7f553800c860 comp rx=0 tx=0).stop 2023-08-27T11:13:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 shutdown_connections 2023-08-27T11:13:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5530069f80 0x7f553006c440 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55440caa00 0x7f55440c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55440af200 0x7f55440b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 --2- 172.21.15.45:0/2067232560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55440ca620 0x7f55440c6e30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 >> 172.21.15.45:0/2067232560 conn(0x7f55440cc7e0 msgr2=0x7f55440b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 shutdown_connections 2023-08-27T11:13:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.167+0000 7f554a419700 1 -- 172.21.15.45:0/2067232560 wait complete. 2023-08-27T11:13:54.181 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-08-27T11:13:54.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:54.182 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-08-27T11:13:54.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-08-27T11:13:54.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.369+0000 7f1e6b9fd700 1 Processor -- start 2023-08-27T11:13:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.369+0000 7f1e6b9fd700 1 -- start start 2023-08-27T11:13:54.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e6b9fd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1e640aeb10 0x7f1e64072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e6b9fd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1e6406aa70 0x7f1e6406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e6b9fd700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6406ddf0 con 0x7f1e640ae730 2023-08-27T11:13:54.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e6b9fd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6406b440 con 0x7f1e6406aa70 2023-08-27T11:13:54.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e6b9fd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6406d830 con 0x7f1e640aeb10 2023-08-27T11:13:54.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e69f9a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1e6406aa70 0x7f1e6406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:54.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e69f9a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1e6406aa70 0x7f1e6406af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54646/0 (socket says 172.21.15.45:54646) 2023-08-27T11:13:54.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e69f9a700 1 -- 172.21.15.45:0/2267137703 learned_addr learned my addr 172.21.15.45:0/2267137703 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:54.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e68f98700 1 --2- 172.21.15.45:0/2267137703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1e640aeb10 0x7f1e64072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:54.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e69f9a700 1 -- 172.21.15.45:0/2267137703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1e640aeb10 msgr2=0x7f1e64072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:54.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e69f9a700 1 --2- 172.21.15.45:0/2267137703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1e640aeb10 0x7f1e64072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e69f9a700 1 -- 172.21.15.45:0/2267137703 >> v1:172.21.15.135:6789/0 conn(0x7f1e640ae730 legacy=0x7f1e640ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.370+0000 7f1e69f9a700 1 -- 172.21.15.45:0/2267137703 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6406db10 con 0x7f1e6406aa70 2023-08-27T11:13:54.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e69f9a700 1 --2- 172.21.15.45:0/2267137703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 0x7f1e6406af00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00d150 tx=0x7f1e5c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9af5ed7aa557d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:54.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e5bfff700 1 -- 172.21.15.45:0/2267137703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c010070 con 0x7f1e6406aa70 2023-08-27T11:13:54.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e5bfff700 1 -- 172.21.15.45:0/2267137703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5c00dbb0 con 0x7f1e6406aa70 2023-08-27T11:13:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e5bfff700 1 -- 172.21.15.45:0/2267137703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c016400 con 0x7f1e6406aa70 2023-08-27T11:13:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/2267137703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 msgr2=0x7f1e6406af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e6b9fd700 1 --2- 172.21.15.45:0/2267137703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 0x7f1e6406af00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00d150 tx=0x7f1e5c00d460 comp rx=0 tx=0).stop 2023-08-27T11:13:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/2267137703 shutdown_connections 2023-08-27T11:13:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e6b9fd700 1 --2- 172.21.15.45:0/2267137703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 0x7f1e6406af00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e6b9fd700 1 --2- 172.21.15.45:0/2267137703 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1e640aeb10 0x7f1e64072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/2267137703 >> 172.21.15.45:0/2267137703 conn(0x7f1e640b2440 msgr2=0x7f1e640b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/2267137703 shutdown_connections 2023-08-27T11:13:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.371+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/2267137703 wait complete. 2023-08-27T11:13:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e6b9fd700 1 Processor -- start 2023-08-27T11:13:54.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e6b9fd700 1 -- start start 2023-08-27T11:13:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e6b9fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 0x7f1e640cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e6b9fd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1e640ae730 0x7f1e641f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e6b9fd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e640aeb10 0x7f1e640aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e6b9fd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e640c4320 con 0x7f1e6406aa70 2023-08-27T11:13:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e6b9fd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e641f31b0 con 0x7f1e640aeb10 2023-08-27T11:13:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e6b9fd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e641f3c60 con 0x7f1e640ae730 2023-08-27T11:13:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e69799700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 0x7f1e640cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e69799700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 0x7f1e640cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54648/0 (socket says 172.21.15.45:54648) 2023-08-27T11:13:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.373+0000 7f1e69799700 1 -- 172.21.15.45:0/481388547 learned_addr learned my addr 172.21.15.45:0/481388547 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:54.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e69799700 1 -- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1e640ae730 msgr2=0x7f1e641f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e69799700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1e640ae730 0x7f1e641f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e69799700 1 -- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e640aeb10 msgr2=0x7f1e640aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e69799700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e640aeb10 0x7f1e640aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e69799700 1 -- 172.21.15.45:0/481388547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c00bd40 con 0x7f1e6406aa70 2023-08-27T11:13:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e69799700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 0x7f1e640cfb50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000c4d0 tx=0x7f1e6000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e59ffb700 1 -- 172.21.15.45:0/481388547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600052d0 con 0x7f1e6406aa70 2023-08-27T11:13:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e59ffb700 1 -- 172.21.15.45:0/481388547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e60005430 con 0x7f1e6406aa70 2023-08-27T11:13:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e59ffb700 1 -- 172.21.15.45:0/481388547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600038f0 con 0x7f1e6406aa70 2023-08-27T11:13:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e641f33c0 con 0x7f1e6406aa70 2023-08-27T11:13:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.374+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e641f3980 con 0x7f1e6406aa70 2023-08-27T11:13:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.375+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6409d4b0 con 0x7f1e6406aa70 2023-08-27T11:13:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.375+0000 7f1e59ffb700 1 -- 172.21.15.45:0/481388547 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f1e60003a50 con 0x7f1e6406aa70 2023-08-27T11:13:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.376+0000 7f1e59ffb700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1e48069f80 0x7f1e4806c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.376+0000 7f1e59ffb700 1 -- 172.21.15.45:0/481388547 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f1e600878a0 con 0x7f1e6406aa70 2023-08-27T11:13:54.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.376+0000 7f1e68f98700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1e48069f80 0x7f1e4806c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:54.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.377+0000 7f1e68f98700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1e48069f80 0x7f1e4806c440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e6409d450 tx=0x7f1e54008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.380+0000 7f1e59ffb700 1 -- 172.21.15.45:0/481388547 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1e60053090 con 0x7f1e6406aa70 2023-08-27T11:13:54.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.608+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e64071e10 con 0x7f1e48069f80 2023-08-27T11:13:54.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.671+0000 7f1e59ffb700 1 -- 172.21.15.45:0/481388547 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1e54015440 con 0x7f1e48069f80 2023-08-27T11:13:54.674 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir3 2023-08-27T11:13:54.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1e48069f80 msgr2=0x7f1e4806c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1e48069f80 0x7f1e4806c440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e6409d450 tx=0x7f1e54008040 comp rx=0 tx=0).stop 2023-08-27T11:13:54.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 msgr2=0x7f1e640cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 0x7f1e640cfb50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000c4d0 tx=0x7f1e6000c890 comp rx=0 tx=0).stop 2023-08-27T11:13:54.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 shutdown_connections 2023-08-27T11:13:54.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f1e48069f80 0x7f1e4806c440 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e6406aa70 0x7f1e640cfb50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e640aeb10 0x7f1e640aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 --2- 172.21.15.45:0/481388547 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1e640ae730 0x7f1e641f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 >> 172.21.15.45:0/481388547 conn(0x7f1e640b2440 msgr2=0x7f1e640ac210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:54.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 shutdown_connections 2023-08-27T11:13:54.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.674+0000 7f1e6b9fd700 1 -- 172.21.15.45:0/481388547 wait complete. 2023-08-27T11:13:54.687 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-08-27T11:13:54.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:54.688 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-08-27T11:13:54.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-08-27T11:13:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.875+0000 7f663022f700 1 Processor -- start 2023-08-27T11:13:54.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.876+0000 7f663022f700 1 -- start start 2023-08-27T11:13:54.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.876+0000 7f663022f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66280ca3b0 0x7f66280ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f663022f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6628161d10 con 0x7f662815d600 2023-08-27T11:13:54.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f663022f700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66280cacd0 con 0x7f66280aed20 2023-08-27T11:13:54.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f663022f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66280af270 con 0x7f66280ca3b0 2023-08-27T11:13:54.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662d7ca700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f662815d600 0x7f6628160ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41432/0 (socket says 172.21.15.45:41432) 2023-08-27T11:13:54.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662d7ca700 1 -- 172.21.15.45:0/4185317058 learned_addr learned my addr 172.21.15.45:0/4185317058 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:54.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662dfcb700 1 --2- 172.21.15.45:0/4185317058 >> v2:172.21.15.135:3300/0 conn(0x7f66280ca3b0 0x7f66280ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:54.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662cfc9700 1 -- 172.21.15.45:0/4185317058 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359380690 0 0) 0x7f6628161d10 con 0x7f662815d600 2023-08-27T11:13:54.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662cfc9700 1 -- 172.21.15.45:0/4185317058 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66140035b0 con 0x7f662815d600 2023-08-27T11:13:54.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662cfc9700 1 -- 172.21.15.45:0/4185317058 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068432153 0 0) 0x7f66280cacd0 con 0x7f66280aed20 2023-08-27T11:13:54.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662cfc9700 1 -- 172.21.15.45:0/4185317058 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6628161d10 con 0x7f66280aed20 2023-08-27T11:13:54.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662cfc9700 1 -- 172.21.15.45:0/4185317058 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2785642897 0 0) 0x7f66140035b0 con 0x7f662815d600 2023-08-27T11:13:54.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662dfcb700 1 -- 172.21.15.45:0/4185317058 >> v1:172.21.15.135:6789/0 conn(0x7f66280aed20 legacy=0x7f6628161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662dfcb700 1 -- 172.21.15.45:0/4185317058 >> v1:172.21.15.45:6789/0 conn(0x7f662815d600 legacy=0x7f6628160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.877+0000 7f662dfcb700 1 -- 172.21.15.45:0/4185317058 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66280af550 con 0x7f66280ca3b0 2023-08-27T11:13:54.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f662dfcb700 1 --2- 172.21.15.45:0/4185317058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66280ca3b0 0x7f66280ca790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6624009840 tx=0x7f6624009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc231be65984149 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:54.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f662cfc9700 1 -- 172.21.15.45:0/4185317058 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618001670 con 0x7f66280ca3b0 2023-08-27T11:13:54.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f662cfc9700 1 -- 172.21.15.45:0/4185317058 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6624004030 con 0x7f66280ca3b0 2023-08-27T11:13:54.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f662cfc9700 1 -- 172.21.15.45:0/4185317058 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66240043a0 con 0x7f66280ca3b0 2023-08-27T11:13:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f663022f700 1 -- 172.21.15.45:0/4185317058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66280ca3b0 msgr2=0x7f66280ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f663022f700 1 --2- 172.21.15.45:0/4185317058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66280ca3b0 0x7f66280ca790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6624009840 tx=0x7f6624009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f663022f700 1 -- 172.21.15.45:0/4185317058 shutdown_connections 2023-08-27T11:13:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f663022f700 1 --2- 172.21.15.45:0/4185317058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66280ca3b0 0x7f66280ca790 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f663022f700 1 -- 172.21.15.45:0/4185317058 >> 172.21.15.45:0/4185317058 conn(0x7f66280cc7e0 msgr2=0x7f66280ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:54.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f663022f700 1 -- 172.21.15.45:0/4185317058 shutdown_connections 2023-08-27T11:13:54.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.878+0000 7f663022f700 1 -- 172.21.15.45:0/4185317058 wait complete. 2023-08-27T11:13:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f663022f700 1 Processor -- start 2023-08-27T11:13:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f663022f700 1 -- start start 2023-08-27T11:13:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f663022f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280aed20 0x7f662815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f663022f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66280ca3b0 0x7f6628151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f663022f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66281600f0 0x7f66280c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f663022f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628160660 con 0x7f66280aed20 2023-08-27T11:13:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f663022f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280c5ea0 con 0x7f66281600f0 2023-08-27T11:13:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f663022f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66280c6180 con 0x7f66280ca3b0 2023-08-27T11:13:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f662dfcb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280aed20 0x7f662815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f662dfcb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280aed20 0x7f662815bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54662/0 (socket says 172.21.15.45:54662) 2023-08-27T11:13:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.880+0000 7f662dfcb700 1 -- 172.21.15.45:0/1737789548 learned_addr learned my addr 172.21.15.45:0/1737789548 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f662d7ca700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66280ca3b0 0x7f6628151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f662e7cc700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66281600f0 0x7f66280c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f662dfcb700 1 -- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66280ca3b0 msgr2=0x7f6628151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f662dfcb700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66280ca3b0 0x7f6628151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f662dfcb700 1 -- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66281600f0 msgr2=0x7f66280c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:54.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f662dfcb700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66281600f0 0x7f66280c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:54.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f662dfcb700 1 -- 172.21.15.45:0/1737789548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240094a0 con 0x7f66280aed20 2023-08-27T11:13:54.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f662dfcb700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280aed20 0x7f662815bab0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6624009790 tx=0x7f6624004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:54.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f661e7fc700 1 -- 172.21.15.45:0/1737789548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624004570 con 0x7f66280aed20 2023-08-27T11:13:54.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f661e7fc700 1 -- 172.21.15.45:0/1737789548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662401f070 con 0x7f66280aed20 2023-08-27T11:13:54.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66280c6460 con 0x7f66280aed20 2023-08-27T11:13:54.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f661e7fc700 1 -- 172.21.15.45:0/1737789548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624011870 con 0x7f66280aed20 2023-08-27T11:13:54.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.881+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66280c39f0 con 0x7f66280aed20 2023-08-27T11:13:54.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.882+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66280c3be0 con 0x7f66280aed20 2023-08-27T11:13:54.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.883+0000 7f661e7fc700 1 -- 172.21.15.45:0/1737789548 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f66240119d0 con 0x7f66280aed20 2023-08-27T11:13:54.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.883+0000 7f661e7fc700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f660c069fb0 0x7f660c06c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:54.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.883+0000 7f661e7fc700 1 -- 172.21.15.45:0/1737789548 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f6624021070 con 0x7f66280aed20 2023-08-27T11:13:54.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.883+0000 7f662d7ca700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f660c069fb0 0x7f660c06c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:54.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.884+0000 7f662d7ca700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f660c069fb0 0x7f660c06c470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f66280c8b00 tx=0x7f6618001c00 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:54.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:54.887+0000 7f661e7fc700 1 -- 172.21.15.45:0/1737789548 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f66240555a0 con 0x7f66280aed20 2023-08-27T11:13:55.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.062+0000 7f661e7fc700 1 -- 172.21.15.45:0/1737789548 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 9) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f6624055970 con 0x7f66280aed20 2023-08-27T11:13:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.116+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f662806af70 con 0x7f660c069fb0 2023-08-27T11:13:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.171+0000 7f661e7fc700 1 -- 172.21.15.45:0/1737789548 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6618001510 con 0x7f660c069fb0 2023-08-27T11:13:55.173 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir4 2023-08-27T11:13:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f660c069fb0 msgr2=0x7f660c06c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f660c069fb0 0x7f660c06c470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f66280c8b00 tx=0x7f6618001c00 comp rx=0 tx=0).stop 2023-08-27T11:13:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280aed20 msgr2=0x7f662815bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280aed20 0x7f662815bab0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6624009790 tx=0x7f6624004750 comp rx=0 tx=0).stop 2023-08-27T11:13:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 shutdown_connections 2023-08-27T11:13:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f660c069fb0 0x7f660c06c470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280aed20 0x7f662815bab0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66281600f0 0x7f66280c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 --2- 172.21.15.45:0/1737789548 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f66280ca3b0 0x7f6628151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 >> 172.21.15.45:0/1737789548 conn(0x7f66280cc7e0 msgr2=0x7f662815c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 shutdown_connections 2023-08-27T11:13:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.173+0000 7f663022f700 1 -- 172.21.15.45:0/1737789548 wait complete. 2023-08-27T11:13:55.188 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-08-27T11:13:55.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:55.188 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-08-27T11:13:55.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-08-27T11:13:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.375+0000 7fc01cb43700 1 Processor -- start 2023-08-27T11:13:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.375+0000 7fc01cb43700 1 -- start start 2023-08-27T11:13:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01cb43700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc01815d340 0x7fc018160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01cb43700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01806b160 con 0x7fc01806a990 2023-08-27T11:13:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01cb43700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0180af590 con 0x7fc0180cd880 2023-08-27T11:13:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01cb43700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc018072010 con 0x7fc01815d340 2023-08-27T11:13:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc016d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc01806a990 0x7fc0180b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41450/0 (socket says 172.21.15.45:41450) 2023-08-27T11:13:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc016d9d700 1 -- 172.21.15.45:0/2657517290 learned_addr learned my addr 172.21.15.45:0/2657517290 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:55.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466632401 0 0) 0x7fc01806b160 con 0x7fc01806a990 2023-08-27T11:13:55.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01659c700 1 --2- 172.21.15.45:0/2657517290 >> v2:172.21.15.135:3300/0 conn(0x7fc01815d340 0x7fc018160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0000035b0 con 0x7fc01806a990 2023-08-27T11:13:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1733510919 0 0) 0x7fc0000035b0 con 0x7fc01806a990 2023-08-27T11:13:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 >> v2:172.21.15.135:3300/0 conn(0x7fc01815d340 msgr2=0x7fc018160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 --2- 172.21.15.45:0/2657517290 >> v2:172.21.15.135:3300/0 conn(0x7fc01815d340 0x7fc018160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 >> v1:172.21.15.135:6789/0 conn(0x7fc0180cd880 legacy=0x7fc01806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0180af2b0 con 0x7fc01806a990 2023-08-27T11:13:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fc0100019f0 con 0x7fc01806a990 2023-08-27T11:13:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 >> v1:172.21.15.45:6789/0 conn(0x7fc01806a990 legacy=0x7fc0180b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc000003a00 0x7fc000003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc000004850 0x7fc000004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000005210 0x7fc00000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018072010 con 0x7fc000004850 2023-08-27T11:13:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00000c1b0 con 0x7fc000005210 2023-08-27T11:13:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.376+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00000c370 con 0x7fc000003a00 2023-08-27T11:13:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc015d9b700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc000004850 0x7fc000004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc016d9d700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000005210 0x7fc00000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc01659c700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc000003a00 0x7fc000003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc015d9b700 1 -- 172.21.15.45:0/2657517290 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc000003a00 msgr2=0x7fc000003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:55.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc015d9b700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc000003a00 0x7fc000003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc015d9b700 1 -- 172.21.15.45:0/2657517290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000005210 msgr2=0x7fc00000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:55.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc015d9b700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000005210 0x7fc00000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc015d9b700 1 -- 172.21.15.45:0/2657517290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000035b0 con 0x7fc000004850 2023-08-27T11:13:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc016d9d700 1 -- 172.21.15.45:0/2657517290 reap_dead start 2023-08-27T11:13:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc015d9b700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc000004850 0x7fc000004cd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0017c0 tx=0x7fc000005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0382b7eee81f5c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0052d0 con 0x7fc000004850 2023-08-27T11:13:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c005450 con 0x7fc000004850 2023-08-27T11:13:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.377+0000 7fc01559a700 1 -- 172.21.15.45:0/2657517290 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0038c0 con 0x7fc000004850 2023-08-27T11:13:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.378+0000 7fc01cb43700 1 -- 172.21.15.45:0/2657517290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc000004850 msgr2=0x7fc000004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.378+0000 7fc01cb43700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc000004850 0x7fc000004cd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0017c0 tx=0x7fc000005880 comp rx=0 tx=0).stop 2023-08-27T11:13:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.378+0000 7fc01cb43700 1 -- 172.21.15.45:0/2657517290 shutdown_connections 2023-08-27T11:13:55.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.378+0000 7fc01cb43700 1 --2- 172.21.15.45:0/2657517290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc000004850 0x7fc000004cd0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.378+0000 7fc01cb43700 1 -- 172.21.15.45:0/2657517290 >> 172.21.15.45:0/2657517290 conn(0x7fc0180cc7c0 msgr2=0x7fc0180ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:55.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.378+0000 7fc01cb43700 1 -- 172.21.15.45:0/2657517290 shutdown_connections 2023-08-27T11:13:55.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.378+0000 7fc01cb43700 1 -- 172.21.15.45:0/2657517290 wait complete. 2023-08-27T11:13:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.379+0000 7fc01cb43700 1 Processor -- start 2023-08-27T11:13:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.379+0000 7fc01cb43700 1 -- start start 2023-08-27T11:13:55.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc01cb43700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000004850 0x7fc018151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc01cb43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0180cad10 0x7fc0180adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc01cb43700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0180ae3e0 0x7fc0180c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc01cb43700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0180c6270 con 0x7fc0180cad10 2023-08-27T11:13:55.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc01cb43700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0180ae860 con 0x7fc000004850 2023-08-27T11:13:55.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc01cb43700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0180c9fe0 con 0x7fc0180ae3e0 2023-08-27T11:13:55.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc015d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0180cad10 0x7fc0180adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc015d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0180cad10 0x7fc0180adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54684/0 (socket says 172.21.15.45:54684) 2023-08-27T11:13:55.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc015d9b700 1 -- 172.21.15.45:0/953694830 learned_addr learned my addr 172.21.15.45:0/953694830 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc01659c700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000004850 0x7fc018151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc016d9d700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0180ae3e0 0x7fc0180c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc015d9b700 1 -- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0180ae3e0 msgr2=0x7fc0180c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc015d9b700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0180ae3e0 0x7fc0180c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc015d9b700 1 -- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000004850 msgr2=0x7fc018151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc015d9b700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000004850 0x7fc018151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc015d9b700 1 -- 172.21.15.45:0/953694830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0180ca880 con 0x7fc0180cad10 2023-08-27T11:13:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.380+0000 7fc01659c700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000004850 0x7fc018151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.381+0000 7fc015d9b700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0180cad10 0x7fc0180adea0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc00c000c00 tx=0x7fc00c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.381+0000 7fc006ffd700 1 -- 172.21.15.45:0/953694830 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c001290 con 0x7fc0180cad10 2023-08-27T11:13:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.381+0000 7fc006ffd700 1 -- 172.21.15.45:0/953694830 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c007040 con 0x7fc0180cad10 2023-08-27T11:13:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.381+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc018151850 con 0x7fc0180cad10 2023-08-27T11:13:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.381+0000 7fc006ffd700 1 -- 172.21.15.45:0/953694830 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0073f0 con 0x7fc0180cad10 2023-08-27T11:13:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.381+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0180ca230 con 0x7fc0180cad10 2023-08-27T11:13:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.382+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0180ceb30 con 0x7fc0180cad10 2023-08-27T11:13:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.382+0000 7fc006ffd700 1 -- 172.21.15.45:0/953694830 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) v1 ==== 88461+0+0 (secure 0 0 0) 0x7fc00c007550 con 0x7fc0180cad10 2023-08-27T11:13:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.383+0000 7fc006ffd700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbff806a250 0x7fbff806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.383+0000 7fc006ffd700 1 -- 172.21.15.45:0/953694830 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fc00c089290 con 0x7fc0180cad10 2023-08-27T11:13:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.383+0000 7fc01659c700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbff806a250 0x7fbff806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.383+0000 7fc01659c700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbff806a250 0x7fbff806c710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc008009c60 tx=0x7fc008008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.387+0000 7fc006ffd700 1 -- 172.21.15.45:0/953694830 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc00c0546c0 con 0x7fc0180cad10 2023-08-27T11:13:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.613+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc018071c30 con 0x7fbff806a250 2023-08-27T11:13:55.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.675+0000 7fc006ffd700 1 -- 172.21.15.45:0/953694830 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc008009da0 con 0x7fbff806a250 2023-08-27T11:13:55.677 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir5 2023-08-27T11:13:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbff806a250 msgr2=0x7fbff806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbff806a250 0x7fbff806c710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc008009c60 tx=0x7fc008008040 comp rx=0 tx=0).stop 2023-08-27T11:13:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0180cad10 msgr2=0x7fc0180adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0180cad10 0x7fc0180adea0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc00c000c00 tx=0x7fc00c001120 comp rx=0 tx=0).stop 2023-08-27T11:13:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 shutdown_connections 2023-08-27T11:13:55.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbff806a250 0x7fbff806c710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0180ae3e0 0x7fc0180c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc000004850 0x7fc018151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 --2- 172.21.15.45:0/953694830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0180cad10 0x7fc0180adea0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 >> 172.21.15.45:0/953694830 conn(0x7fc0180cc7c0 msgr2=0x7fc0180af910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:55.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 shutdown_connections 2023-08-27T11:13:55.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.677+0000 7fc01cb43700 1 -- 172.21.15.45:0/953694830 wait complete. 2023-08-27T11:13:55.692 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-08-27T11:13:55.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:55.692 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-08-27T11:13:55.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-08-27T11:13:55.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.880+0000 7f22ec77a700 1 Processor -- start 2023-08-27T11:13:55.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.880+0000 7f22ec77a700 1 -- start start 2023-08-27T11:13:55.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22ec77a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e40af420 0x7f22e4161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22ec77a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e4071de0 con 0x7f22e415d3f0 2023-08-27T11:13:55.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22ec77a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40c5310 con 0x7f22e40aed60 2023-08-27T11:13:55.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22ec77a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40c6ca0 con 0x7f22e40af420 2023-08-27T11:13:55.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22ea516700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f22e415d3f0 0x7f22e4160e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41466/0 (socket says 172.21.15.45:41466) 2023-08-27T11:13:55.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22ea516700 1 -- 172.21.15.45:0/3790914633 learned_addr learned my addr 172.21.15.45:0/3790914633 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:55.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493720706 0 0) 0x7f22e4071de0 con 0x7f22e415d3f0 2023-08-27T11:13:55.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22ead17700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e40af420 0x7f22e4161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d00035b0 con 0x7f22e415d3f0 2023-08-27T11:13:55.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 455155954 0 0) 0x7f22d00035b0 con 0x7f22e415d3f0 2023-08-27T11:13:55.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e40af420 msgr2=0x7f22e4161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:55.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e40af420 0x7f22e4161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 >> v1:172.21.15.135:6789/0 conn(0x7f22e40aed60 legacy=0x7f22e406ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4161c60 con 0x7f22e415d3f0 2023-08-27T11:13:55.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f22e0001670 con 0x7f22e415d3f0 2023-08-27T11:13:55.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 >> v1:172.21.15.45:6789/0 conn(0x7f22e415d3f0 legacy=0x7f22e4160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d0003a30 0x7f22d0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e40af420 0x7f22d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40c6ca0 con 0x7f22d0004d90 2023-08-27T11:13:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d000bba0 con 0x7f22d0003a30 2023-08-27T11:13:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.881+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d000bd60 con 0x7f22e40af420 2023-08-27T11:13:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22ead17700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22ea516700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d0003a30 0x7f22d0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22e9d15700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e40af420 0x7f22d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22ead17700 1 -- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e40af420 msgr2=0x7f22d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22ead17700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e40af420 0x7f22d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22ead17700 1 -- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d0003a30 msgr2=0x7f22d0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22ead17700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d0003a30 0x7f22d0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22ead17700 1 -- 172.21.15.45:0/3790914633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00035b0 con 0x7f22d0004d90 2023-08-27T11:13:55.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22ead17700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22d0005210 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009fc0 tx=0x7f22d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3f41f9bd0c715b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:55.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00ec30 con 0x7f22d0004d90 2023-08-27T11:13:55.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc00f040 con 0x7f22d0004d90 2023-08-27T11:13:55.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.882+0000 7f22e9514700 1 -- 172.21.15.45:0/3790914633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc014480 con 0x7f22d0004d90 2023-08-27T11:13:55.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.883+0000 7f22ec77a700 1 -- 172.21.15.45:0/3790914633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 msgr2=0x7f22d0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.883+0000 7f22ec77a700 1 --2- 172.21.15.45:0/3790914633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22d0005210 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009fc0 tx=0x7f22d0005880 comp rx=0 tx=0).stop 2023-08-27T11:13:55.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.883+0000 7f22ead17700 1 -- 172.21.15.45:0/3790914633 reap_dead start 2023-08-27T11:13:55.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.883+0000 7f22ec77a700 1 -- 172.21.15.45:0/3790914633 shutdown_connections 2023-08-27T11:13:55.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.883+0000 7f22ec77a700 1 -- 172.21.15.45:0/3790914633 >> 172.21.15.45:0/3790914633 conn(0x7f22e40cfdd0 msgr2=0x7f22e40d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:55.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.883+0000 7f22ec77a700 1 -- 172.21.15.45:0/3790914633 shutdown_connections 2023-08-27T11:13:55.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.883+0000 7f22ec77a700 1 -- 172.21.15.45:0/3790914633 wait complete. 2023-08-27T11:13:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.884+0000 7f22ec77a700 1 Processor -- start 2023-08-27T11:13:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.884+0000 7f22ec77a700 1 -- start start 2023-08-27T11:13:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ec77a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22e414ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ec77a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e414cf90 0x7f22e41f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ec77a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e41f2fa0 0x7f22e41f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ec77a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41f3990 con 0x7f22d0004d90 2023-08-27T11:13:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ec77a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e414e130 con 0x7f22e414cf90 2023-08-27T11:13:55.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ec77a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e414de50 con 0x7f22e41f2fa0 2023-08-27T11:13:55.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ea516700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22e414ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ea516700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22e414ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54702/0 (socket says 172.21.15.45:54702) 2023-08-27T11:13:55.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ea516700 1 -- 172.21.15.45:0/3419107479 learned_addr learned my addr 172.21.15.45:0/3419107479 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:55.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22e9d15700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e414cf90 0x7f22e41f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ead17700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e41f2fa0 0x7f22e41f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ea516700 1 -- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e41f2fa0 msgr2=0x7f22e41f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ea516700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e41f2fa0 0x7f22e41f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ea516700 1 -- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e414cf90 msgr2=0x7f22e41f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:55.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ea516700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e414cf90 0x7f22e41f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ea516700 1 -- 172.21.15.45:0/3419107479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e414d520 con 0x7f22d0004d90 2023-08-27T11:13:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22ea516700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22e414ca50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f22e00072c0 tx=0x7f22e00075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.885+0000 7f22daffd700 1 -- 172.21.15.45:0/3419107479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000f5e0 con 0x7f22d0004d90 2023-08-27T11:13:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.886+0000 7f22daffd700 1 -- 172.21.15.45:0/3419107479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e000f740 con 0x7f22d0004d90 2023-08-27T11:13:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.886+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e414d800 con 0x7f22d0004d90 2023-08-27T11:13:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.886+0000 7f22daffd700 1 -- 172.21.15.45:0/3419107479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000faf0 con 0x7f22d0004d90 2023-08-27T11:13:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.886+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e40b32a0 con 0x7f22d0004d90 2023-08-27T11:13:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.887+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40727c0 con 0x7f22d0004d90 2023-08-27T11:13:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.887+0000 7f22daffd700 1 -- 172.21.15.45:0/3419107479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f22e000fc50 con 0x7f22d0004d90 2023-08-27T11:13:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.887+0000 7f22daffd700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22c806a250 0x7f22c806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.887+0000 7f22daffd700 1 -- 172.21.15.45:0/3419107479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f22e0088900 con 0x7f22d0004d90 2023-08-27T11:13:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.888+0000 7f22e9d15700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22c806a250 0x7f22c806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.888+0000 7f22e9d15700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22c806a250 0x7f22c806c710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22e40ae990 tx=0x7f22d4009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:55.892+0000 7f22daffd700 1 -- 172.21.15.45:0/3419107479 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22e0053cb0 con 0x7f22d0004d90 2023-08-27T11:13:56.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.116+0000 7f22daffd700 1 -- 172.21.15.45:0/3419107479 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f22e0054080 con 0x7f22d0004d90 2023-08-27T11:13:56.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.121+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f22e4071b00 con 0x7f22c806a250 2023-08-27T11:13:56.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.172+0000 7f22daffd700 1 -- 172.21.15.45:0/3419107479 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f22d4001d60 con 0x7f22c806a250 2023-08-27T11:13:56.174 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir6 2023-08-27T11:13:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22c806a250 msgr2=0x7f22c806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22c806a250 0x7f22c806c710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22e40ae990 tx=0x7f22d4009040 comp rx=0 tx=0).stop 2023-08-27T11:13:56.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 msgr2=0x7f22e414ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22e414ca50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f22e00072c0 tx=0x7f22e00075d0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 shutdown_connections 2023-08-27T11:13:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22c806a250 0x7f22c806c710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d0004d90 0x7f22e414ca50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e41f2fa0 0x7f22e41f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 --2- 172.21.15.45:0/3419107479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e414cf90 0x7f22e41f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.174+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 >> 172.21.15.45:0/3419107479 conn(0x7f22e40cfdd0 msgr2=0x7f22e40b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.175+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 shutdown_connections 2023-08-27T11:13:56.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.175+0000 7f22ec77a700 1 -- 172.21.15.45:0/3419107479 wait complete. 2023-08-27T11:13:56.188 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-08-27T11:13:56.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:56.189 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-08-27T11:13:56.208 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-08-27T11:13:56.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.376+0000 7f9f89f95700 1 Processor -- start 2023-08-27T11:13:56.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.376+0000 7f9f89f95700 1 -- start start 2023-08-27T11:13:56.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.377+0000 7f9f89f95700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f84157be0 0x7f9f840716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.377+0000 7f9f89f95700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 0x7f9f840b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.377+0000 7f9f89f95700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8415b460 con 0x7f9f840b17c0 2023-08-27T11:13:56.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.377+0000 7f9f89f95700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f84157fc0 con 0x7f9f84157be0 2023-08-27T11:13:56.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.377+0000 7f9f89f95700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840b2280 con 0x7f9f8406d5e0 2023-08-27T11:13:56.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.377+0000 7f9f82ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9f840b17c0 0x7f9f8406ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41486/0 (socket says 172.21.15.45:41486) 2023-08-27T11:13:56.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.377+0000 7f9f82ffd700 1 -- 172.21.15.45:0/256149765 learned_addr learned my addr 172.21.15.45:0/256149765 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:56.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.377+0000 7f9f83fff700 1 --2- 172.21.15.45:0/256149765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 0x7f9f840b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.377+0000 7f9f837fe700 1 --2- 172.21.15.45:0/256149765 >> v2:172.21.15.135:3300/0 conn(0x7f9f84157be0 0x7f9f840716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f827fc700 1 -- 172.21.15.45:0/256149765 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031448968 0 0) 0x7f9f8415b460 con 0x7f9f840b17c0 2023-08-27T11:13:56.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f827fc700 1 -- 172.21.15.45:0/256149765 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f700035b0 con 0x7f9f840b17c0 2023-08-27T11:13:56.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f827fc700 1 -- 172.21.15.45:0/256149765 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131016293 0 0) 0x7f9f700035b0 con 0x7f9f840b17c0 2023-08-27T11:13:56.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f837fe700 1 -- 172.21.15.45:0/256149765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 msgr2=0x7f9f840b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:56.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f837fe700 1 --2- 172.21.15.45:0/256149765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 0x7f9f840b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f837fe700 1 -- 172.21.15.45:0/256149765 >> v1:172.21.15.45:6789/0 conn(0x7f9f840b17c0 legacy=0x7f9f8406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f837fe700 1 -- 172.21.15.45:0/256149765 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f840b2560 con 0x7f9f84157be0 2023-08-27T11:13:56.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f83fff700 1 --2- 172.21.15.45:0/256149765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 0x7f9f840b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:56.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f837fe700 1 --2- 172.21.15.45:0/256149765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84157be0 0x7f9f840716b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009840 tx=0x7f9f74009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bcc5fa97e09be70 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:56.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f827fc700 1 -- 172.21.15.45:0/256149765 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c001670 con 0x7f9f84157be0 2023-08-27T11:13:56.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f827fc700 1 -- 172.21.15.45:0/256149765 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f74004030 con 0x7f9f84157be0 2023-08-27T11:13:56.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.378+0000 7f9f827fc700 1 -- 172.21.15.45:0/256149765 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f740043c0 con 0x7f9f84157be0 2023-08-27T11:13:56.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.379+0000 7f9f89f95700 1 -- 172.21.15.45:0/256149765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84157be0 msgr2=0x7f9f840716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.379+0000 7f9f89f95700 1 --2- 172.21.15.45:0/256149765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84157be0 0x7f9f840716b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009840 tx=0x7f9f74009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:56.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.379+0000 7f9f89f95700 1 -- 172.21.15.45:0/256149765 shutdown_connections 2023-08-27T11:13:56.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.379+0000 7f9f89f95700 1 --2- 172.21.15.45:0/256149765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 0x7f9f840b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.379+0000 7f9f89f95700 1 --2- 172.21.15.45:0/256149765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84157be0 0x7f9f840716b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.379+0000 7f9f89f95700 1 -- 172.21.15.45:0/256149765 >> 172.21.15.45:0/256149765 conn(0x7f9f840cc7c0 msgr2=0x7f9f840ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:56.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.379+0000 7f9f89f95700 1 -- 172.21.15.45:0/256149765 shutdown_connections 2023-08-27T11:13:56.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.379+0000 7f9f89f95700 1 -- 172.21.15.45:0/256149765 wait complete. 2023-08-27T11:13:56.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.380+0000 7f9f89f95700 1 Processor -- start 2023-08-27T11:13:56.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.380+0000 7f9f89f95700 1 -- start start 2023-08-27T11:13:56.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f89f95700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 0x7f9f840c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f89f95700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840b17c0 0x7f9f840b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f89f95700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84157be0 0x7f9f840c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f89f95700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840b3810 con 0x7f9f840b17c0 2023-08-27T11:13:56.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f89f95700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840b5a50 con 0x7f9f84157be0 2023-08-27T11:13:56.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f89f95700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840c6090 con 0x7f9f8406d5e0 2023-08-27T11:13:56.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f82ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840b17c0 0x7f9f840b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f82ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840b17c0 0x7f9f840b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54718/0 (socket says 172.21.15.45:54718) 2023-08-27T11:13:56.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f82ffd700 1 -- 172.21.15.45:0/1959705857 learned_addr learned my addr 172.21.15.45:0/1959705857 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:56.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f837fe700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 0x7f9f840c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f83fff700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84157be0 0x7f9f840c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.381+0000 7f9f82ffd700 1 -- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 msgr2=0x7f9f840c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f82ffd700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 0x7f9f840c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f82ffd700 1 -- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84157be0 msgr2=0x7f9f840c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f82ffd700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84157be0 0x7f9f840c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f82ffd700 1 -- 172.21.15.45:0/1959705857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c009a70 con 0x7f9f840b17c0 2023-08-27T11:13:56.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f82ffd700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840b17c0 0x7f9f840b5510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c750 tx=0x7f9f6c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:56.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f7bfff700 1 -- 172.21.15.45:0/1959705857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c004c80 con 0x7f9f840b17c0 2023-08-27T11:13:56.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f7bfff700 1 -- 172.21.15.45:0/1959705857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c004de0 con 0x7f9f840b17c0 2023-08-27T11:13:56.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f7bfff700 1 -- 172.21.15.45:0/1959705857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0072d0 con 0x7f9f840b17c0 2023-08-27T11:13:56.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f740094a0 con 0x7f9f840b17c0 2023-08-27T11:13:56.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.382+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f840b2e70 con 0x7f9f840b17c0 2023-08-27T11:13:56.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.383+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f840b3950 con 0x7f9f840b17c0 2023-08-27T11:13:56.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.383+0000 7f9f7bfff700 1 -- 172.21.15.45:0/1959705857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f9f6c007430 con 0x7f9f840b17c0 2023-08-27T11:13:56.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.384+0000 7f9f7bfff700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f9f6406a2e0 0x7f9f6406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.384+0000 7f9f7bfff700 1 -- 172.21.15.45:0/1959705857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f9f6c088810 con 0x7f9f840b17c0 2023-08-27T11:13:56.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.384+0000 7f9f837fe700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f9f6406a2e0 0x7f9f6406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.384+0000 7f9f837fe700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f9f6406a2e0 0x7f9f6406c7a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9f740090b0 tx=0x7f9f74008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:56.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.388+0000 7f9f7bfff700 1 -- 172.21.15.45:0/1959705857 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9f840b3950 con 0x7f9f840b17c0 2023-08-27T11:13:56.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.612+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f84071190 con 0x7f9f6406a2e0 2023-08-27T11:13:56.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.676+0000 7f9f7bfff700 1 -- 172.21.15.45:0/1959705857 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9f74004460 con 0x7f9f6406a2e0 2023-08-27T11:13:56.678 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir7 2023-08-27T11:13:56.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f9f6406a2e0 msgr2=0x7f9f6406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f9f6406a2e0 0x7f9f6406c7a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9f740090b0 tx=0x7f9f74008030 comp rx=0 tx=0).stop 2023-08-27T11:13:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840b17c0 msgr2=0x7f9f840b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840b17c0 0x7f9f840b5510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c750 tx=0x7f9f6c00ca60 comp rx=0 tx=0).stop 2023-08-27T11:13:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 shutdown_connections 2023-08-27T11:13:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f9f6406a2e0 0x7f9f6406c7a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84157be0 0x7f9f840c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9f8406d5e0 0x7f9f840c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 --2- 172.21.15.45:0/1959705857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840b17c0 0x7f9f840b5510 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 >> 172.21.15.45:0/1959705857 conn(0x7f9f840cc7c0 msgr2=0x7f9f8406ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 shutdown_connections 2023-08-27T11:13:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.678+0000 7f9f89f95700 1 -- 172.21.15.45:0/1959705857 wait complete. 2023-08-27T11:13:56.693 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-08-27T11:13:56.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:56.694 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-08-27T11:13:56.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-08-27T11:13:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.881+0000 7f828de83700 1 Processor -- start 2023-08-27T11:13:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.881+0000 7f828de83700 1 -- start start 2023-08-27T11:13:56.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.882+0000 7f828de83700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f828806a280 0x7f828806a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.882+0000 7f828de83700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8288071400 0x7f8288071890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.882+0000 7f828de83700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82880b2290 con 0x7f828806ced0 2023-08-27T11:13:56.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.882+0000 7f828de83700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828815c5c0 con 0x7f8288071400 2023-08-27T11:13:56.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.882+0000 7f828de83700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828806aba0 con 0x7f828806a280 2023-08-27T11:13:56.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.882+0000 7f8286ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8288071400 0x7f8288071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.882+0000 7f8286ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8288071400 0x7f8288071890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54726/0 (socket says 172.21.15.45:54726) 2023-08-27T11:13:56.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f8286ffd700 1 -- 172.21.15.45:0/529401275 learned_addr learned my addr 172.21.15.45:0/529401275 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:56.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f82877fe700 1 --2- 172.21.15.45:0/529401275 >> v2:172.21.15.135:3300/0 conn(0x7f828806a280 0x7f828806a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f8286ffd700 1 -- 172.21.15.45:0/529401275 >> v2:172.21.15.135:3300/0 conn(0x7f828806a280 msgr2=0x7f828806a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f8286ffd700 1 --2- 172.21.15.45:0/529401275 >> v2:172.21.15.135:3300/0 conn(0x7f828806a280 0x7f828806a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f8286ffd700 1 -- 172.21.15.45:0/529401275 >> v1:172.21.15.135:6789/0 conn(0x7f828806ced0 legacy=0x7f828806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f8286ffd700 1 -- 172.21.15.45:0/529401275 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82880b1960 con 0x7f8288071400 2023-08-27T11:13:56.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f82877fe700 1 --2- 172.21.15.45:0/529401275 >> v2:172.21.15.135:3300/0 conn(0x7f828806a280 0x7f828806a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:56.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f8286ffd700 1 --2- 172.21.15.45:0/529401275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8288071400 0x7f8288071890 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f82880cd830 tx=0x7f827c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15be538aae0aa2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:56.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f82867fc700 1 -- 172.21.15.45:0/529401275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00b870 con 0x7f8288071400 2023-08-27T11:13:56.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f82867fc700 1 -- 172.21.15.45:0/529401275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c00b9d0 con 0x7f8288071400 2023-08-27T11:13:56.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.883+0000 7f82867fc700 1 -- 172.21.15.45:0/529401275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00bd60 con 0x7f8288071400 2023-08-27T11:13:56.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.884+0000 7f828de83700 1 -- 172.21.15.45:0/529401275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8288071400 msgr2=0x7f8288071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.884+0000 7f828de83700 1 --2- 172.21.15.45:0/529401275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8288071400 0x7f8288071890 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f82880cd830 tx=0x7f827c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.884+0000 7f828de83700 1 -- 172.21.15.45:0/529401275 shutdown_connections 2023-08-27T11:13:56.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.884+0000 7f828de83700 1 --2- 172.21.15.45:0/529401275 >> v2:172.21.15.135:3300/0 conn(0x7f828806a280 0x7f828806a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.884+0000 7f828de83700 1 --2- 172.21.15.45:0/529401275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8288071400 0x7f8288071890 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.884+0000 7f828de83700 1 -- 172.21.15.45:0/529401275 >> 172.21.15.45:0/529401275 conn(0x7f82880cc7c0 msgr2=0x7f82880ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:56.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.884+0000 7f828de83700 1 -- 172.21.15.45:0/529401275 shutdown_connections 2023-08-27T11:13:56.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.884+0000 7f828de83700 1 -- 172.21.15.45:0/529401275 wait complete. 2023-08-27T11:13:56.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.885+0000 7f828de83700 1 Processor -- start 2023-08-27T11:13:56.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.885+0000 7f828de83700 1 -- start start 2023-08-27T11:13:56.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f828de83700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828806a280 0x7f8288157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f828de83700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f828806ced0 0x7f8288152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f828de83700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288071400 0x7f828814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f828de83700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8288152db0 con 0x7f828806a280 2023-08-27T11:13:56.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f828de83700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828814fd20 con 0x7f8288071400 2023-08-27T11:13:56.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f828de83700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828814b7d0 con 0x7f828806ced0 2023-08-27T11:13:56.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f82877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828806a280 0x7f8288157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f82877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828806a280 0x7f8288157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54730/0 (socket says 172.21.15.45:54730) 2023-08-27T11:13:56.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f82877fe700 1 -- 172.21.15.45:0/2504800213 learned_addr learned my addr 172.21.15.45:0/2504800213 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:56.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f8287fff700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288071400 0x7f828814b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f8286ffd700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f828806ced0 0x7f8288152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f82877fe700 1 -- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f828806ced0 msgr2=0x7f8288152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f82877fe700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f828806ced0 0x7f8288152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f82877fe700 1 -- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288071400 msgr2=0x7f828814b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:56.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f82877fe700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288071400 0x7f828814b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f82877fe700 1 -- 172.21.15.45:0/2504800213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c009550 con 0x7f828806a280 2023-08-27T11:13:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f8287fff700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288071400 0x7f828814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:13:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.886+0000 7f82877fe700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828806a280 0x7f8288157c50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f827800e810 tx=0x7f827800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.887+0000 7f826ffff700 1 -- 172.21.15.45:0/2504800213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827800c020 con 0x7f828806a280 2023-08-27T11:13:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.887+0000 7f826ffff700 1 -- 172.21.15.45:0/2504800213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8278004250 con 0x7f828806a280 2023-08-27T11:13:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.887+0000 7f826ffff700 1 -- 172.21.15.45:0/2504800213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780036a0 con 0x7f828806a280 2023-08-27T11:13:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.887+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8288152fc0 con 0x7f828806a280 2023-08-27T11:13:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.887+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8288153580 con 0x7f828806a280 2023-08-27T11:13:56.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.888+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8288150300 con 0x7f828806a280 2023-08-27T11:13:56.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.888+0000 7f826ffff700 1 -- 172.21.15.45:0/2504800213 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f8278003800 con 0x7f828806a280 2023-08-27T11:13:56.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.888+0000 7f826ffff700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f827406a2e0 0x7f827406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:56.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.889+0000 7f826ffff700 1 -- 172.21.15.45:0/2504800213 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f82780886d0 con 0x7f828806a280 2023-08-27T11:13:56.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.891+0000 7f8286ffd700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f827406a2e0 0x7f827406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:56.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.892+0000 7f8286ffd700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f827406a2e0 0x7f827406c7a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f827c009c10 tx=0x7f827c008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:56.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:56.893+0000 7f826ffff700 1 -- 172.21.15.45:0/2504800213 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8278053880 con 0x7f828806a280 2023-08-27T11:13:57.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.120+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f82880b1f20 con 0x7f827406a2e0 2023-08-27T11:13:57.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.183+0000 7f826ffff700 1 -- 172.21.15.45:0/2504800213 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f827c00bb40 con 0x7f827406a2e0 2023-08-27T11:13:57.185 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir8 2023-08-27T11:13:57.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f827406a2e0 msgr2=0x7f827406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f827406a2e0 0x7f827406c7a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f827c009c10 tx=0x7f827c008030 comp rx=0 tx=0).stop 2023-08-27T11:13:57.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828806a280 msgr2=0x7f8288157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828806a280 0x7f8288157c50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f827800e810 tx=0x7f827800ebd0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 shutdown_connections 2023-08-27T11:13:57.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f827406a2e0 0x7f827406c7a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f828806a280 0x7f8288157c50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8288071400 0x7f828814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 --2- 172.21.15.45:0/2504800213 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f828806ced0 0x7f8288152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 >> 172.21.15.45:0/2504800213 conn(0x7f82880cc7c0 msgr2=0x7f828815b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:57.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 shutdown_connections 2023-08-27T11:13:57.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.185+0000 7f828de83700 1 -- 172.21.15.45:0/2504800213 wait complete. 2023-08-27T11:13:57.199 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-08-27T11:13:57.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:13:57.200 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-08-27T11:13:57.218 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-08-27T11:13:57.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.387+0000 7ff8fab87700 1 Processor -- start 2023-08-27T11:13:57.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.387+0000 7ff8fab87700 1 -- start start 2023-08-27T11:13:57.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.387+0000 7ff8fab87700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff8f415af30 0x7ff8f40cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.387+0000 7ff8fab87700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff8f40cf960 0x7ff8f40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.387+0000 7ff8fab87700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f40cb8b0 con 0x7ff8f415b310 2023-08-27T11:13:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.387+0000 7ff8fab87700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f406f9c0 con 0x7ff8f40cf960 2023-08-27T11:13:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.387+0000 7ff8fab87700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f40cecd0 con 0x7ff8f415af30 2023-08-27T11:13:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff8f40cf960 0x7ff8f40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff8f40cf960 0x7ff8f40cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54738/0 (socket says 172.21.15.45:54738) 2023-08-27T11:13:57.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f3fff700 1 -- 172.21.15.45:0/4225847609 learned_addr learned my addr 172.21.15.45:0/4225847609 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f8923700 1 --2- 172.21.15.45:0/4225847609 >> v2:172.21.15.135:3300/0 conn(0x7ff8f415af30 0x7ff8f40cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f3fff700 1 -- 172.21.15.45:0/4225847609 >> v2:172.21.15.135:3300/0 conn(0x7ff8f415af30 msgr2=0x7ff8f40cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f3fff700 1 --2- 172.21.15.45:0/4225847609 >> v2:172.21.15.135:3300/0 conn(0x7ff8f415af30 0x7ff8f40cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f3fff700 1 -- 172.21.15.45:0/4225847609 >> v1:172.21.15.135:6789/0 conn(0x7ff8f415b310 legacy=0x7ff8f415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f3fff700 1 -- 172.21.15.45:0/4225847609 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f40cbae0 con 0x7ff8f40cf960 2023-08-27T11:13:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f3fff700 1 --2- 172.21.15.45:0/4225847609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f40cf960 0x7ff8f40cd770 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff8f409a8d0 tx=0x7ff8e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6a1b4802a32034 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f37fe700 1 -- 172.21.15.45:0/4225847609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000b870 con 0x7ff8f40cf960 2023-08-27T11:13:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f37fe700 1 -- 172.21.15.45:0/4225847609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e000b9d0 con 0x7ff8f40cf960 2023-08-27T11:13:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.388+0000 7ff8f37fe700 1 -- 172.21.15.45:0/4225847609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000bd60 con 0x7ff8f40cf960 2023-08-27T11:13:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.389+0000 7ff8fab87700 1 -- 172.21.15.45:0/4225847609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f40cf960 msgr2=0x7ff8f40cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.389+0000 7ff8fab87700 1 --2- 172.21.15.45:0/4225847609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f40cf960 0x7ff8f40cd770 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff8f409a8d0 tx=0x7ff8e0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.389+0000 7ff8fab87700 1 -- 172.21.15.45:0/4225847609 shutdown_connections 2023-08-27T11:13:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.389+0000 7ff8fab87700 1 --2- 172.21.15.45:0/4225847609 >> v2:172.21.15.135:3300/0 conn(0x7ff8f415af30 0x7ff8f40cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.389+0000 7ff8fab87700 1 --2- 172.21.15.45:0/4225847609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f40cf960 0x7ff8f40cd770 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.389+0000 7ff8fab87700 1 -- 172.21.15.45:0/4225847609 >> 172.21.15.45:0/4225847609 conn(0x7ff8f409a1a0 msgr2=0x7ff8f40d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.389+0000 7ff8fab87700 1 -- 172.21.15.45:0/4225847609 shutdown_connections 2023-08-27T11:13:57.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.389+0000 7ff8fab87700 1 -- 172.21.15.45:0/4225847609 wait complete. 2023-08-27T11:13:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.390+0000 7ff8fab87700 1 Processor -- start 2023-08-27T11:13:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.390+0000 7ff8fab87700 1 -- start start 2023-08-27T11:13:57.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8fab87700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8f40cf960 0x7ff8f414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8fab87700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f415af30 0x7ff8f41f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8fab87700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f415b310 0x7ff8f40ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8fab87700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40b5be0 con 0x7ff8f415af30 2023-08-27T11:13:57.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8fab87700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40b4f60 con 0x7ff8f415b310 2023-08-27T11:13:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8fab87700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40b5240 con 0x7ff8f40cf960 2023-08-27T11:13:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f415af30 0x7ff8f41f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f415af30 0x7ff8f41f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54746/0 (socket says 172.21.15.45:54746) 2023-08-27T11:13:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f3fff700 1 -- 172.21.15.45:0/1887079425 learned_addr learned my addr 172.21.15.45:0/1887079425 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f8923700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8f40cf960 0x7ff8f414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f9124700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f415b310 0x7ff8f40ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f3fff700 1 -- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8f40cf960 msgr2=0x7ff8f414f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f3fff700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8f40cf960 0x7ff8f414f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f3fff700 1 -- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f415b310 msgr2=0x7ff8f40ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f3fff700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f415b310 0x7ff8f40ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.391+0000 7ff8f3fff700 1 -- 172.21.15.45:0/1887079425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e0009550 con 0x7ff8f415af30 2023-08-27T11:13:57.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.392+0000 7ff8f9124700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f415b310 0x7ff8f40ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:57.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.392+0000 7ff8f3fff700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f415af30 0x7ff8f41f0f70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0013000 tx=0x7ff8e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:57.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.392+0000 7ff8f17fa700 1 -- 172.21.15.45:0/1887079425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0004310 con 0x7ff8f415af30 2023-08-27T11:13:57.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.392+0000 7ff8f17fa700 1 -- 172.21.15.45:0/1887079425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e001f070 con 0x7ff8f415af30 2023-08-27T11:13:57.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.392+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f40b5520 con 0x7ff8f415af30 2023-08-27T11:13:57.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.392+0000 7ff8f17fa700 1 -- 172.21.15.45:0/1887079425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00036a0 con 0x7ff8f415af30 2023-08-27T11:13:57.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.392+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f41585b0 con 0x7ff8f415af30 2023-08-27T11:13:57.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.393+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f40b5d20 con 0x7ff8f415af30 2023-08-27T11:13:57.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.393+0000 7ff8f17fa700 1 -- 172.21.15.45:0/1887079425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7ff8e0021030 con 0x7ff8f415af30 2023-08-27T11:13:57.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.394+0000 7ff8f17fa700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff8d806a2e0 0x7ff8d806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.394+0000 7ff8f17fa700 1 -- 172.21.15.45:0/1887079425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7ff8e0089df0 con 0x7ff8f415af30 2023-08-27T11:13:57.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.394+0000 7ff8f8923700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff8d806a2e0 0x7ff8d806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.394+0000 7ff8f8923700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff8d806a2e0 0x7ff8d806c7a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009930 tx=0x7ff8e8008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:57.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.398+0000 7ff8f17fa700 1 -- 172.21.15.45:0/1887079425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff8f40b5d20 con 0x7ff8f415af30 2023-08-27T11:13:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.625+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8f40cc0a0 con 0x7ff8d806a2e0 2023-08-27T11:13:57.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.688+0000 7ff8f17fa700 1 -- 172.21.15.45:0/1887079425 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff8e8009e00 con 0x7ff8d806a2e0 2023-08-27T11:13:57.690 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /snap_test_dir1/dir9 2023-08-27T11:13:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff8d806a2e0 msgr2=0x7ff8d806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff8d806a2e0 0x7ff8d806c7a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009930 tx=0x7ff8e8008040 comp rx=0 tx=0).stop 2023-08-27T11:13:57.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f415af30 msgr2=0x7ff8f41f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f415af30 0x7ff8f41f0f70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0013000 tx=0x7ff8e0004750 comp rx=0 tx=0).stop 2023-08-27T11:13:57.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 shutdown_connections 2023-08-27T11:13:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff8d806a2e0 0x7ff8d806c7a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f415b310 0x7ff8f40ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff8f40cf960 0x7ff8f414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 --2- 172.21.15.45:0/1887079425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8f415af30 0x7ff8f41f0f70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 >> 172.21.15.45:0/1887079425 conn(0x7ff8f409a1a0 msgr2=0x7ff8f40b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:57.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 shutdown_connections 2023-08-27T11:13:57.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.691+0000 7ff8fab87700 1 -- 172.21.15.45:0/1887079425 wait complete. 2023-08-27T11:13:57.704 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-08-27T11:13:57.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-08-27T11:13:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.834+0000 7fbdb6ead700 1 Processor -- start 2023-08-27T11:13:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.834+0000 7fbdb6ead700 1 -- start start 2023-08-27T11:13:57.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.834+0000 7fbdb6ead700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00b65d0 0x7fbdb00ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.834+0000 7fbdb6ead700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbdb00d0520 0x7fbdb015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.834+0000 7fbdb6ead700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb015ec50 con 0x7fbdb00cb5c0 2023-08-27T11:13:57.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.834+0000 7fbdb6ead700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb015c620 con 0x7fbdb00d0520 2023-08-27T11:13:57.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.834+0000 7fbdb6ead700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb015eea0 con 0x7fbdb00b65d0 2023-08-27T11:13:57.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdb544a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbdb00d0520 0x7fbdb015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdb544a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbdb00d0520 0x7fbdb015c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54756/0 (socket says 172.21.15.45:54756) 2023-08-27T11:13:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdb544a700 1 -- 172.21.15.45:0/283095601 learned_addr learned my addr 172.21.15.45:0/283095601 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdb4c49700 1 --2- 172.21.15.45:0/283095601 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00b65d0 0x7fbdb00ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdb544a700 1 -- 172.21.15.45:0/283095601 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00b65d0 msgr2=0x7fbdb00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdb544a700 1 --2- 172.21.15.45:0/283095601 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00b65d0 0x7fbdb00ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdb544a700 1 -- 172.21.15.45:0/283095601 >> v1:172.21.15.135:6789/0 conn(0x7fbdb00cb5c0 legacy=0x7fbdb00cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:57.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdb544a700 1 -- 172.21.15.45:0/283095601 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb015f180 con 0x7fbdb00d0520 2023-08-27T11:13:57.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdb544a700 1 --2- 172.21.15.45:0/283095601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00d0520 0x7fbdb015c0e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009e50 tx=0x7fbda800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d1e2be55b682cb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:57.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdaf7fe700 1 -- 172.21.15.45:0/283095601 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda800e070 con 0x7fbdb00d0520 2023-08-27T11:13:57.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdaf7fe700 1 -- 172.21.15.45:0/283095601 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda8003a20 con 0x7fbdb00d0520 2023-08-27T11:13:57.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.835+0000 7fbdaf7fe700 1 -- 172.21.15.45:0/283095601 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8003dd0 con 0x7fbdb00d0520 2023-08-27T11:13:57.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.836+0000 7fbdb6ead700 1 -- 172.21.15.45:0/283095601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00d0520 msgr2=0x7fbdb015c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.836+0000 7fbdb6ead700 1 --2- 172.21.15.45:0/283095601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00d0520 0x7fbdb015c0e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009e50 tx=0x7fbda800b330 comp rx=0 tx=0).stop 2023-08-27T11:13:57.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.836+0000 7fbdb6ead700 1 -- 172.21.15.45:0/283095601 shutdown_connections 2023-08-27T11:13:57.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.836+0000 7fbdb6ead700 1 --2- 172.21.15.45:0/283095601 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00b65d0 0x7fbdb00ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.836+0000 7fbdb6ead700 1 --2- 172.21.15.45:0/283095601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00d0520 0x7fbdb015c0e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.836+0000 7fbdb6ead700 1 -- 172.21.15.45:0/283095601 >> 172.21.15.45:0/283095601 conn(0x7fbdb006f960 msgr2=0x7fbdb006fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:57.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.836+0000 7fbdb6ead700 1 -- 172.21.15.45:0/283095601 shutdown_connections 2023-08-27T11:13:57.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.836+0000 7fbdb6ead700 1 -- 172.21.15.45:0/283095601 wait complete. 2023-08-27T11:13:57.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.837+0000 7fbdb6ead700 1 Processor -- start 2023-08-27T11:13:57.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.838+0000 7fbdb6ead700 1 -- start start 2023-08-27T11:13:57.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb6ead700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00b65d0 0x7fbdb014a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb6ead700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb00cb5c0 0x7fbdb00c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb6ead700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00d0520 0x7fbdb01f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb6ead700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb014a870 con 0x7fbdb00b65d0 2023-08-27T11:13:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb6ead700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb01f41f0 con 0x7fbdb00cb5c0 2023-08-27T11:13:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb6ead700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00ca6a0 con 0x7fbdb00d0520 2023-08-27T11:13:57.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb4c49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00b65d0 0x7fbdb014a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb4c49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00b65d0 0x7fbdb014a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54758/0 (socket says 172.21.15.45:54758) 2023-08-27T11:13:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb4c49700 1 -- 172.21.15.45:0/3072207585 learned_addr learned my addr 172.21.15.45:0/3072207585 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb4c49700 1 -- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00d0520 msgr2=0x7fbdb01f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb544a700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00d0520 0x7fbdb01f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdaffff700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb00cb5c0 0x7fbdb00c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb4c49700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00d0520 0x7fbdb01f49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb4c49700 1 -- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb00cb5c0 msgr2=0x7fbdb00c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:57.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb4c49700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb00cb5c0 0x7fbdb00c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:57.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb4c49700 1 -- 172.21.15.45:0/3072207585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda8009b30 con 0x7fbdb00b65d0 2023-08-27T11:13:57.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdaffff700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb00cb5c0 0x7fbdb00c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:57.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb4c49700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00b65d0 0x7fbdb014a330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbda400c7f0 tx=0x7fbda400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:57.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdad7fa700 1 -- 172.21.15.45:0/3072207585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda4011800 con 0x7fbdb00b65d0 2023-08-27T11:13:57.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdad7fa700 1 -- 172.21.15.45:0/3072207585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda4011960 con 0x7fbdb00b65d0 2023-08-27T11:13:57.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdad7fa700 1 -- 172.21.15.45:0/3072207585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda4011cf0 con 0x7fbdb00b65d0 2023-08-27T11:13:57.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.839+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb00ca0e0 con 0x7fbdb00b65d0 2023-08-27T11:13:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.840+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb01f4490 con 0x7fbdb00b65d0 2023-08-27T11:13:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.840+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb01f46b0 con 0x7fbdb00b65d0 2023-08-27T11:13:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.841+0000 7fbdad7fa700 1 -- 172.21.15.45:0/3072207585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fbda4018360 con 0x7fbdb00b65d0 2023-08-27T11:13:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.841+0000 7fbdad7fa700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbd9406a2e0 0x7fbd9406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:57.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.842+0000 7fbdad7fa700 1 -- 172.21.15.45:0/3072207585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fbda4088cc0 con 0x7fbdb00b65d0 2023-08-27T11:13:57.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.842+0000 7fbdaffff700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbd9406a2e0 0x7fbd9406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:57.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.842+0000 7fbdaffff700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbd9406a2e0 0x7fbd9406c7a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0154890 tx=0x7fbd9c009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:57.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:57.846+0000 7fbdad7fa700 1 -- 172.21.15.45:0/3072207585 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbda408f080 con 0x7fbdb00b65d0 2023-08-27T11:13:58.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.074+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdb015f740 con 0x7fbd9406a2e0 2023-08-27T11:13:58.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.118+0000 7fbdad7fa700 1 -- 172.21.15.45:0/3072207585 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbd9c006860 con 0x7fbd9406a2e0 2023-08-27T11:13:58.121 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-08-27T11:13:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbd9406a2e0 msgr2=0x7fbd9406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbd9406a2e0 0x7fbd9406c7a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0154890 tx=0x7fbd9c009040 comp rx=0 tx=0).stop 2023-08-27T11:13:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00b65d0 msgr2=0x7fbdb014a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00b65d0 0x7fbdb014a330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbda400c7f0 tx=0x7fbda400cbb0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 shutdown_connections 2023-08-27T11:13:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbd9406a2e0 0x7fbd9406c7a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb00b65d0 0x7fbdb014a330 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbdb00d0520 0x7fbdb01f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 --2- 172.21.15.45:0/3072207585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb00cb5c0 0x7fbdb00c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 >> 172.21.15.45:0/3072207585 conn(0x7fbdb006f960 msgr2=0x7fbdb00cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 shutdown_connections 2023-08-27T11:13:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.121+0000 7fbdb6ead700 1 -- 172.21.15.45:0/3072207585 wait complete. 2023-08-27T11:13:58.135 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-08-27T11:13:58.136 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-08-27T11:13:58.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.264+0000 7f6821e56700 1 Processor -- start 2023-08-27T11:13:58.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.264+0000 7f6821e56700 1 -- start start 2023-08-27T11:13:58.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f6821e56700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 0x7f681c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f6821e56700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f681c0b62c0 0x7f681c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f6821e56700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f681c160c70 0x7f681c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f6821e56700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c0cdf40 con 0x7f681c160c70 2023-08-27T11:13:58.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f6821e56700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c0d0fd0 con 0x7f681c0b62c0 2023-08-27T11:13:58.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f6821e56700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0ce190 con 0x7f681c160890 2023-08-27T11:13:58.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f681c160c70 0x7f681c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f681c160c70 0x7f681c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54770/0 (socket says 172.21.15.45:54770) 2023-08-27T11:13:58.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681bfff700 1 -- 172.21.15.45:0/3243132977 learned_addr learned my addr 172.21.15.45:0/3243132977 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:58.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681b7fe700 1 --2- 172.21.15.45:0/3243132977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 0x7f681c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681affd700 1 --2- 172.21.15.45:0/3243132977 >> v2:172.21.15.135:3300/0 conn(0x7f681c0b62c0 0x7f681c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681bfff700 1 -- 172.21.15.45:0/3243132977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 msgr2=0x7f681c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:58.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681bfff700 1 --2- 172.21.15.45:0/3243132977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 0x7f681c0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681bfff700 1 -- 172.21.15.45:0/3243132977 >> v2:172.21.15.135:3300/0 conn(0x7f681c0b62c0 msgr2=0x7f681c0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:58.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681bfff700 1 --2- 172.21.15.45:0/3243132977 >> v2:172.21.15.135:3300/0 conn(0x7f681c0b62c0 0x7f681c0b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681bfff700 1 -- 172.21.15.45:0/3243132977 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0ce470 con 0x7f681c160c70 2023-08-27T11:13:58.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.265+0000 7f681affd700 1 --2- 172.21.15.45:0/3243132977 >> v2:172.21.15.135:3300/0 conn(0x7f681c0b62c0 0x7f681c0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:58.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f681bfff700 1 --2- 172.21.15.45:0/3243132977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c160c70 0x7f681c0b5050 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f681c067ea0 tx=0x7f681400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f02f7c31ba51ae3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:58.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f681a7fc700 1 -- 172.21.15.45:0/3243132977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681400e070 con 0x7f681c160c70 2023-08-27T11:13:58.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f681a7fc700 1 -- 172.21.15.45:0/3243132977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681400b9c0 con 0x7f681c160c70 2023-08-27T11:13:58.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f681a7fc700 1 -- 172.21.15.45:0/3243132977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681400bd70 con 0x7f681c160c70 2023-08-27T11:13:58.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f6821e56700 1 -- 172.21.15.45:0/3243132977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c160c70 msgr2=0x7f681c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f6821e56700 1 --2- 172.21.15.45:0/3243132977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c160c70 0x7f681c0b5050 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f681c067ea0 tx=0x7f681400b330 comp rx=0 tx=0).stop 2023-08-27T11:13:58.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f6821e56700 1 -- 172.21.15.45:0/3243132977 shutdown_connections 2023-08-27T11:13:58.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f6821e56700 1 --2- 172.21.15.45:0/3243132977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 0x7f681c0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f6821e56700 1 --2- 172.21.15.45:0/3243132977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c160c70 0x7f681c0b5050 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f6821e56700 1 --2- 172.21.15.45:0/3243132977 >> v2:172.21.15.135:3300/0 conn(0x7f681c0b62c0 0x7f681c0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f6821e56700 1 -- 172.21.15.45:0/3243132977 >> 172.21.15.45:0/3243132977 conn(0x7f681c0cfb10 msgr2=0x7f681c0cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:58.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f6821e56700 1 -- 172.21.15.45:0/3243132977 shutdown_connections 2023-08-27T11:13:58.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.266+0000 7f6821e56700 1 -- 172.21.15.45:0/3243132977 wait complete. 2023-08-27T11:13:58.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.268+0000 7f6821e56700 1 Processor -- start 2023-08-27T11:13:58.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.268+0000 7f6821e56700 1 -- start start 2023-08-27T11:13:58.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6821e56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c0b62c0 0x7f681c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6821e56700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 0x7f681c15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6821e56700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c160c70 0x7f681c0c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6821e56700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c1f2d20 con 0x7f681c0b62c0 2023-08-27T11:13:58.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6821e56700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c15e660 con 0x7f681c160c70 2023-08-27T11:13:58.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6821e56700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c15cd60 con 0x7f681c160890 2023-08-27T11:13:58.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c0b62c0 0x7f681c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c0b62c0 0x7f681c154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54772/0 (socket says 172.21.15.45:54772) 2023-08-27T11:13:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681b7fe700 1 -- 172.21.15.45:0/3123420995 learned_addr learned my addr 172.21.15.45:0/3123420995 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681affd700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 0x7f681c15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681bfff700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c160c70 0x7f681c0c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681b7fe700 1 -- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 msgr2=0x7f681c15dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681b7fe700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 0x7f681c15dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681b7fe700 1 -- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c160c70 msgr2=0x7f681c0c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681b7fe700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c160c70 0x7f681c0c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681b7fe700 1 -- 172.21.15.45:0/3123420995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6814009b30 con 0x7f681c0b62c0 2023-08-27T11:13:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681bfff700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c160c70 0x7f681c0c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f681b7fe700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c0b62c0 0x7f681c154c60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f680c00e810 tx=0x7f680c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6813fff700 1 -- 172.21.15.45:0/3123420995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c013610 con 0x7f681c0b62c0 2023-08-27T11:13:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6813fff700 1 -- 172.21.15.45:0/3123420995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c013770 con 0x7f681c0b62c0 2023-08-27T11:13:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c15d040 con 0x7f681c0b62c0 2023-08-27T11:13:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6813fff700 1 -- 172.21.15.45:0/3123420995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c013b20 con 0x7f681c0b62c0 2023-08-27T11:13:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.269+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c15d600 con 0x7f681c0b62c0 2023-08-27T11:13:58.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.270+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681c06c120 con 0x7f681c0b62c0 2023-08-27T11:13:58.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.271+0000 7f6813fff700 1 -- 172.21.15.45:0/3123420995 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f680c013c80 con 0x7f681c0b62c0 2023-08-27T11:13:58.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.271+0000 7f6813fff700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f67fc06a2e0 0x7f67fc06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.271+0000 7f6813fff700 1 -- 172.21.15.45:0/3123420995 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f680c088920 con 0x7f681c0b62c0 2023-08-27T11:13:58.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.271+0000 7f681affd700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f67fc06a2e0 0x7f67fc06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.272+0000 7f681affd700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f67fc06a2e0 0x7f67fc06c7a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6804006ba0 tx=0x7f6804008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:58.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.276+0000 7f6813fff700 1 -- 172.21.15.45:0/3123420995 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f680c053a50 con 0x7f681c0b62c0 2023-08-27T11:13:58.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.504+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f681c0d0c60 con 0x7f67fc06a2e0 2023-08-27T11:13:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.558+0000 7f6813fff700 1 -- 172.21.15.45:0/3123420995 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f680400a6a0 con 0x7f67fc06a2e0 2023-08-27T11:13:58.560 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-08-27T11:13:58.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f67fc06a2e0 msgr2=0x7f67fc06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f67fc06a2e0 0x7f67fc06c7a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6804006ba0 tx=0x7f6804008040 comp rx=0 tx=0).stop 2023-08-27T11:13:58.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c0b62c0 msgr2=0x7f681c154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c0b62c0 0x7f681c154c60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f680c00e810 tx=0x7f680c00ebd0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 shutdown_connections 2023-08-27T11:13:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f67fc06a2e0 0x7f67fc06c7a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f681c0b62c0 0x7f681c154c60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f681c160c70 0x7f681c0c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 --2- 172.21.15.45:0/3123420995 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f681c160890 0x7f681c15dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 >> 172.21.15.45:0/3123420995 conn(0x7f681c0cfb10 msgr2=0x7f681c0c6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 shutdown_connections 2023-08-27T11:13:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.560+0000 7f6821e56700 1 -- 172.21.15.45:0/3123420995 wait complete. 2023-08-27T11:13:58.574 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-08-27T11:13:58.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-08-27T11:13:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.703+0000 7fa6ab08a700 1 Processor -- start 2023-08-27T11:13:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.703+0000 7fa6ab08a700 1 -- start start 2023-08-27T11:13:58.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.703+0000 7fa6ab08a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a4157be0 0x7fa6a40716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.703+0000 7fa6ab08a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6a406d5e0 0x7fa6a40b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6ab08a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a415b460 con 0x7fa6a40b17c0 2023-08-27T11:13:58.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6ab08a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a4157fc0 con 0x7fa6a406d5e0 2023-08-27T11:13:58.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6ab08a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40b2280 con 0x7fa6a4157be0 2023-08-27T11:13:58.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a9627700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6a406d5e0 0x7fa6a40b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a9627700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6a406d5e0 0x7fa6a40b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54784/0 (socket says 172.21.15.45:54784) 2023-08-27T11:13:58.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a9627700 1 -- 172.21.15.45:0/3323175154 learned_addr learned my addr 172.21.15.45:0/3323175154 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:58.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a8e26700 1 --2- 172.21.15.45:0/3323175154 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a4157be0 0x7fa6a40716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a9627700 1 -- 172.21.15.45:0/3323175154 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a4157be0 msgr2=0x7fa6a40716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:58.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a9627700 1 --2- 172.21.15.45:0/3323175154 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a4157be0 0x7fa6a40716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a9627700 1 -- 172.21.15.45:0/3323175154 >> v1:172.21.15.135:6789/0 conn(0x7fa6a40b17c0 legacy=0x7fa6a406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:58.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a9627700 1 -- 172.21.15.45:0/3323175154 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40b2560 con 0x7fa6a406d5e0 2023-08-27T11:13:58.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a9627700 1 --2- 172.21.15.45:0/3323175154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a406d5e0 0x7fa6a40b1c40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009e50 tx=0x7fa69c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb67b6f2de9c9c60 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:58.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a37fe700 1 -- 172.21.15.45:0/3323175154 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00e070 con 0x7fa6a406d5e0 2023-08-27T11:13:58.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a37fe700 1 -- 172.21.15.45:0/3323175154 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c003a20 con 0x7fa6a406d5e0 2023-08-27T11:13:58.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.704+0000 7fa6a37fe700 1 -- 172.21.15.45:0/3323175154 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c003dd0 con 0x7fa6a406d5e0 2023-08-27T11:13:58.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.705+0000 7fa6ab08a700 1 -- 172.21.15.45:0/3323175154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a406d5e0 msgr2=0x7fa6a40b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.705+0000 7fa6ab08a700 1 --2- 172.21.15.45:0/3323175154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a406d5e0 0x7fa6a40b1c40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009e50 tx=0x7fa69c00b330 comp rx=0 tx=0).stop 2023-08-27T11:13:58.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.705+0000 7fa6ab08a700 1 -- 172.21.15.45:0/3323175154 shutdown_connections 2023-08-27T11:13:58.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.705+0000 7fa6ab08a700 1 --2- 172.21.15.45:0/3323175154 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a4157be0 0x7fa6a40716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.705+0000 7fa6ab08a700 1 --2- 172.21.15.45:0/3323175154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a406d5e0 0x7fa6a40b1c40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.705+0000 7fa6ab08a700 1 -- 172.21.15.45:0/3323175154 >> 172.21.15.45:0/3323175154 conn(0x7fa6a40cc7c0 msgr2=0x7fa6a40ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:58.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.705+0000 7fa6ab08a700 1 -- 172.21.15.45:0/3323175154 shutdown_connections 2023-08-27T11:13:58.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.705+0000 7fa6ab08a700 1 -- 172.21.15.45:0/3323175154 wait complete. 2023-08-27T11:13:58.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.706+0000 7fa6ab08a700 1 Processor -- start 2023-08-27T11:13:58.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.706+0000 7fa6ab08a700 1 -- start start 2023-08-27T11:13:58.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6ab08a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a406d5e0 0x7fa6a4154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6ab08a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6a40b17c0 0x7fa6a4154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6ab08a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a4157be0 0x7fa6a415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6ab08a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a41efa10 con 0x7fa6a4157be0 2023-08-27T11:13:58.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6ab08a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a414fcb0 con 0x7fa6a40b17c0 2023-08-27T11:13:58.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6ab08a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a415d060 con 0x7fa6a406d5e0 2023-08-27T11:13:58.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a9627700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a4157be0 0x7fa6a415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a9627700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a4157be0 0x7fa6a415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54790/0 (socket says 172.21.15.45:54790) 2023-08-27T11:13:58.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a9627700 1 -- 172.21.15.45:0/1464333019 learned_addr learned my addr 172.21.15.45:0/1464333019 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:58.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a8e26700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a406d5e0 0x7fa6a4154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a3fff700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6a40b17c0 0x7fa6a4154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a9627700 1 -- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a406d5e0 msgr2=0x7fa6a4154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a9627700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a406d5e0 0x7fa6a4154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a9627700 1 -- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6a40b17c0 msgr2=0x7fa6a4154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:58.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a9627700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6a40b17c0 0x7fa6a4154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:58.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.707+0000 7fa6a9627700 1 -- 172.21.15.45:0/1464333019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c009b30 con 0x7fa6a4157be0 2023-08-27T11:13:58.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.708+0000 7fa6a8e26700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a406d5e0 0x7fa6a4154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:58.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.708+0000 7fa6a9627700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a4157be0 0x7fa6a415c090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa6a406c3e0 tx=0x7fa69c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:58.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.708+0000 7fa6a17fa700 1 -- 172.21.15.45:0/1464333019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c004440 con 0x7fa6a4157be0 2023-08-27T11:13:58.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.708+0000 7fa6a17fa700 1 -- 172.21.15.45:0/1464333019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c0045a0 con 0x7fa6a4157be0 2023-08-27T11:13:58.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.708+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a415cd80 con 0x7fa6a4157be0 2023-08-27T11:13:58.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.708+0000 7fa6a17fa700 1 -- 172.21.15.45:0/1464333019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c004950 con 0x7fa6a4157be0 2023-08-27T11:13:58.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.708+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a40b2ff0 con 0x7fa6a4157be0 2023-08-27T11:13:58.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.709+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a40b2b70 con 0x7fa6a4157be0 2023-08-27T11:13:58.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.709+0000 7fa6a17fa700 1 -- 172.21.15.45:0/1464333019 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fa69c025450 con 0x7fa6a4157be0 2023-08-27T11:13:58.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.710+0000 7fa6a17fa700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa68806a2e0 0x7fa68806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:58.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.710+0000 7fa6a17fa700 1 -- 172.21.15.45:0/1464333019 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fa69c08a460 con 0x7fa6a4157be0 2023-08-27T11:13:58.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.710+0000 7fa6a8e26700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa68806a2e0 0x7fa68806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:58.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.711+0000 7fa6a8e26700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa68806a2e0 0x7fa68806c7a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:58.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.714+0000 7fa6a17fa700 1 -- 172.21.15.45:0/1464333019 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa6a40b2b70 con 0x7fa6a4157be0 2023-08-27T11:13:58.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:58.945+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6a4071190 con 0x7fa68806a2e0 2023-08-27T11:13:59.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.001+0000 7fa6a17fa700 1 -- 172.21.15.45:0/1464333019 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa69800c040 con 0x7fa68806a2e0 2023-08-27T11:13:59.004 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-08-27T11:13:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa68806a2e0 msgr2=0x7fa68806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa68806a2e0 0x7fa68806c7a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698008040 comp rx=0 tx=0).stop 2023-08-27T11:13:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a4157be0 msgr2=0x7fa6a415c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a4157be0 0x7fa6a415c090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa6a406c3e0 tx=0x7fa69c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 shutdown_connections 2023-08-27T11:13:59.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa68806a2e0 0x7fa68806c7a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6a4157be0 0x7fa6a415c090 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6a406d5e0 0x7fa6a4154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 --2- 172.21.15.45:0/1464333019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6a40b17c0 0x7fa6a4154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 >> 172.21.15.45:0/1464333019 conn(0x7fa6a40cc7c0 msgr2=0x7fa6a40cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:59.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 shutdown_connections 2023-08-27T11:13:59.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.003+0000 7fa6ab08a700 1 -- 172.21.15.45:0/1464333019 wait complete. 2023-08-27T11:13:59.017 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-08-27T11:13:59.018 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-08-27T11:13:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.149+0000 7fc3daf45700 1 Processor -- start 2023-08-27T11:13:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.149+0000 7fc3daf45700 1 -- start start 2023-08-27T11:13:59.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.149+0000 7fc3daf45700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc3d415b270 0x7fc3d406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.149+0000 7fc3daf45700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d406f8b0 con 0x7fc3d406cf40 2023-08-27T11:13:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.149+0000 7fc3daf45700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d406f010 con 0x7fc3d4071e80 2023-08-27T11:13:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.149+0000 7fc3daf45700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d40ca650 con 0x7fc3d415b270 2023-08-27T11:13:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d8ce1700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc3d406cf40 0x7fc3d406d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41570/0 (socket says 172.21.15.45:41570) 2023-08-27T11:13:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d8ce1700 1 -- 172.21.15.45:0/643690010 learned_addr learned my addr 172.21.15.45:0/643690010 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d3fff700 1 --2- 172.21.15.45:0/643690010 >> v2:172.21.15.135:3300/0 conn(0x7fc3d415b270 0x7fc3d406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885733305 0 0) 0x7fc3c8001110 con 0x7fc3d406cf40 2023-08-27T11:13:59.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3c40035b0 con 0x7fc3d406cf40 2023-08-27T11:13:59.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210982649 0 0) 0x7fc3d406f010 con 0x7fc3d4071e80 2023-08-27T11:13:59.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3c8001110 con 0x7fc3d4071e80 2023-08-27T11:13:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1456238211 0 0) 0x7fc3c40035b0 con 0x7fc3d406cf40 2023-08-27T11:13:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 >> v2:172.21.15.135:3300/0 conn(0x7fc3d415b270 msgr2=0x7fc3d406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 --2- 172.21.15.45:0/643690010 >> v2:172.21.15.135:3300/0 conn(0x7fc3d415b270 0x7fc3d406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 >> v1:172.21.15.135:6789/0 conn(0x7fc3d4071e80 legacy=0x7fc3d40caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d406f2f0 con 0x7fc3d406cf40 2023-08-27T11:13:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fc3c8001800 con 0x7fc3d406cf40 2023-08-27T11:13:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d3fff700 1 --2- 172.21.15.45:0/643690010 >> v2:172.21.15.135:3300/0 conn(0x7fc3d415b270 0x7fc3d406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:13:59.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 >> v1:172.21.15.45:6789/0 conn(0x7fc3d406cf40 legacy=0x7fc3d406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4003be0 0x7fc3c4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3c4004850 0x7fc3c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c4005260 0x7fc3c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40ca650 con 0x7fc3c4005260 2023-08-27T11:13:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c400c190 con 0x7fc3c4004850 2023-08-27T11:13:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c400c350 con 0x7fc3c4003be0 2023-08-27T11:13:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d94e2700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c4005260 0x7fc3c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d3fff700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3c4004850 0x7fc3c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.150+0000 7fc3d8ce1700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4003be0 0x7fc3c4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d94e2700 1 -- 172.21.15.45:0/643690010 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4003be0 msgr2=0x7fc3c4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d94e2700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4003be0 0x7fc3c4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d94e2700 1 -- 172.21.15.45:0/643690010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3c4004850 msgr2=0x7fc3c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:13:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d94e2700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3c4004850 0x7fc3c4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d94e2700 1 -- 172.21.15.45:0/643690010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40035b0 con 0x7fc3c4005260 2023-08-27T11:13:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d94e2700 1 -- 172.21.15.45:0/643690010 reap_dead start 2023-08-27T11:13:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d94e2700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c4005260 0x7fc3c400ba10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00c2f0 tx=0x7fc3c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85b3c3a9a34b66a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc001f30 con 0x7fc3c4005260 2023-08-27T11:13:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3cc003a00 con 0x7fc3c4005260 2023-08-27T11:13:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3d37fe700 1 -- 172.21.15.45:0/643690010 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc003db0 con 0x7fc3c4005260 2023-08-27T11:13:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.151+0000 7fc3daf45700 1 -- 172.21.15.45:0/643690010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c4005260 msgr2=0x7fc3c400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.152+0000 7fc3daf45700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c4005260 0x7fc3c400ba10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00c2f0 tx=0x7fc3c4005880 comp rx=0 tx=0).stop 2023-08-27T11:13:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.152+0000 7fc3daf45700 1 -- 172.21.15.45:0/643690010 shutdown_connections 2023-08-27T11:13:59.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.152+0000 7fc3daf45700 1 --2- 172.21.15.45:0/643690010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3c4005260 0x7fc3c400ba10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.152+0000 7fc3daf45700 1 -- 172.21.15.45:0/643690010 >> 172.21.15.45:0/643690010 conn(0x7fc3d40cc7e0 msgr2=0x7fc3d40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:59.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.152+0000 7fc3daf45700 1 -- 172.21.15.45:0/643690010 shutdown_connections 2023-08-27T11:13:59.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.152+0000 7fc3daf45700 1 -- 172.21.15.45:0/643690010 wait complete. 2023-08-27T11:13:59.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.153+0000 7fc3daf45700 1 Processor -- start 2023-08-27T11:13:59.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.153+0000 7fc3daf45700 1 -- start start 2023-08-27T11:13:59.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3daf45700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4005260 0x7fc3d4148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3daf45700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3d40b3380 0x7fc3d40b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3daf45700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3d40b0cb0 0x7fc3d41f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3daf45700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40b1140 con 0x7fc3d40b3380 2023-08-27T11:13:59.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3daf45700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40b5850 con 0x7fc3d40b0cb0 2023-08-27T11:13:59.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3daf45700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40b4560 con 0x7fc3c4005260 2023-08-27T11:13:59.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3d40b3380 0x7fc3d40b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3d40b3380 0x7fc3d40b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54808/0 (socket says 172.21.15.45:54808) 2023-08-27T11:13:59.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d3fff700 1 -- 172.21.15.45:0/3482676345 learned_addr learned my addr 172.21.15.45:0/3482676345 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:59.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d94e2700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3d40b0cb0 0x7fc3d41f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d3fff700 1 -- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4005260 msgr2=0x7fc3d4148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d8ce1700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4005260 0x7fc3d4148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d3fff700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4005260 0x7fc3d4148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d3fff700 1 -- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3d40b0cb0 msgr2=0x7fc3d41f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d3fff700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3d40b0cb0 0x7fc3d41f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d3fff700 1 -- 172.21.15.45:0/3482676345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d40b53c0 con 0x7fc3d40b3380 2023-08-27T11:13:59.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d94e2700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3d40b0cb0 0x7fc3d41f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:59.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d8ce1700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4005260 0x7fc3d4148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:13:59.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.154+0000 7fc3d3fff700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3d40b3380 0x7fc3d40b0770 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0011a90 tx=0x7fc3c0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:59.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.155+0000 7fc3d17fa700 1 -- 172.21.15.45:0/3482676345 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000fa60 con 0x7fc3d40b3380 2023-08-27T11:13:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.155+0000 7fc3d17fa700 1 -- 172.21.15.45:0/3482676345 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c000fbc0 con 0x7fc3d40b3380 2023-08-27T11:13:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.155+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d40b47b0 con 0x7fc3d40b3380 2023-08-27T11:13:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.155+0000 7fc3d17fa700 1 -- 172.21.15.45:0/3482676345 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0014440 con 0x7fc3d40b3380 2023-08-27T11:13:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.155+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d40b4d70 con 0x7fc3d40b3380 2023-08-27T11:13:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.156+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d40b5550 con 0x7fc3d40b3380 2023-08-27T11:13:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.156+0000 7fc3d17fa700 1 -- 172.21.15.45:0/3482676345 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fc3c00145a0 con 0x7fc3d40b3380 2023-08-27T11:13:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.157+0000 7fc3d17fa700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc3b806a2e0 0x7fc3b806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.157+0000 7fc3d17fa700 1 -- 172.21.15.45:0/3482676345 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fc3c0089350 con 0x7fc3d40b3380 2023-08-27T11:13:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.157+0000 7fc3d8ce1700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc3b806a2e0 0x7fc3b806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.157+0000 7fc3d8ce1700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc3b806a2e0 0x7fc3b806c7a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc3d415a120 tx=0x7fc3c8009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.161+0000 7fc3d17fa700 1 -- 172.21.15.45:0/3482676345 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc3c0054580 con 0x7fc3d40b3380 2023-08-27T11:13:59.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.389+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3d406c110 con 0x7fc3b806a2e0 2023-08-27T11:13:59.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.441+0000 7fc3d17fa700 1 -- 172.21.15.45:0/3482676345 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc3c8007e10 con 0x7fc3b806a2e0 2023-08-27T11:13:59.446 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-08-27T11:13:59.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.443+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc3b806a2e0 msgr2=0x7fc3b806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.443+0000 7fc3daf45700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc3b806a2e0 0x7fc3b806c7a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc3d415a120 tx=0x7fc3c8009040 comp rx=0 tx=0).stop 2023-08-27T11:13:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.443+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3d40b3380 msgr2=0x7fc3d40b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.443+0000 7fc3daf45700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3d40b3380 0x7fc3d40b0770 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0011a90 tx=0x7fc3c0011e50 comp rx=0 tx=0).stop 2023-08-27T11:13:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.444+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 shutdown_connections 2023-08-27T11:13:59.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.444+0000 7fc3daf45700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc3b806a2e0 0x7fc3b806c7a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.444+0000 7fc3daf45700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3d40b0cb0 0x7fc3d41f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.444+0000 7fc3daf45700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc3c4005260 0x7fc3d4148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.444+0000 7fc3daf45700 1 --2- 172.21.15.45:0/3482676345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3d40b3380 0x7fc3d40b0770 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.444+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 >> 172.21.15.45:0/3482676345 conn(0x7fc3d40cc7e0 msgr2=0x7fc3d40caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:59.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.444+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 shutdown_connections 2023-08-27T11:13:59.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.444+0000 7fc3daf45700 1 -- 172.21.15.45:0/3482676345 wait complete. 2023-08-27T11:13:59.458 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-08-27T11:13:59.459 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-08-27T11:13:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.595+0000 7fb1eb5a9700 1 Processor -- start 2023-08-27T11:13:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.595+0000 7fb1eb5a9700 1 -- start start 2023-08-27T11:13:59.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.596+0000 7fb1eb5a9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb1e40af200 0x7fb1e40cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.596+0000 7fb1eb5a9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40ce340 con 0x7fb1e40caa00 2023-08-27T11:13:59.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.596+0000 7fb1eb5a9700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40ceec0 con 0x7fb1e40ca620 2023-08-27T11:13:59.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.596+0000 7fb1eb5a9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e415d7c0 con 0x7fb1e40af200 2023-08-27T11:13:59.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.596+0000 7fb1e8b44700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb1e40af200 0x7fb1e40cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.596+0000 7fb1e9b46700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb1e40caa00 0x7fb1e40acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41594/0 (socket says 172.21.15.45:41594) 2023-08-27T11:13:59.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.596+0000 7fb1e9b46700 1 -- 172.21.15.45:0/1057586835 learned_addr learned my addr 172.21.15.45:0/1057586835 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:13:59.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.597+0000 7fb1e8b44700 1 -- 172.21.15.45:0/1057586835 >> v1:172.21.15.135:6789/0 conn(0x7fb1e40ca620 legacy=0x7fb1e40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.597+0000 7fb1e8b44700 1 -- 172.21.15.45:0/1057586835 >> v1:172.21.15.45:6789/0 conn(0x7fb1e40caa00 legacy=0x7fb1e40acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.597+0000 7fb1e8b44700 1 -- 172.21.15.45:0/1057586835 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40cc570 con 0x7fb1e40af200 2023-08-27T11:13:59.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.597+0000 7fb1e8b44700 1 --2- 172.21.15.45:0/1057586835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40af200 0x7fb1e40cf350 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009840 tx=0x7fb1e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340a11eeb21734c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:59.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.597+0000 7fb1dbfff700 1 -- 172.21.15.45:0/1057586835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0011830 con 0x7fb1e40af200 2023-08-27T11:13:59.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.597+0000 7fb1dbfff700 1 -- 172.21.15.45:0/1057586835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0011990 con 0x7fb1e40af200 2023-08-27T11:13:59.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.597+0000 7fb1dbfff700 1 -- 172.21.15.45:0/1057586835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0011d20 con 0x7fb1e40af200 2023-08-27T11:13:59.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.597+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/1057586835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40af200 msgr2=0x7fb1e40cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.597+0000 7fb1eb5a9700 1 --2- 172.21.15.45:0/1057586835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40af200 0x7fb1e40cf350 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009840 tx=0x7fb1e0009b50 comp rx=0 tx=0).stop 2023-08-27T11:13:59.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.598+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/1057586835 shutdown_connections 2023-08-27T11:13:59.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.598+0000 7fb1eb5a9700 1 --2- 172.21.15.45:0/1057586835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40af200 0x7fb1e40cf350 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.598+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/1057586835 >> 172.21.15.45:0/1057586835 conn(0x7fb1e40cc7e0 msgr2=0x7fb1e40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:59.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.598+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/1057586835 shutdown_connections 2023-08-27T11:13:59.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.598+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/1057586835 wait complete. 2023-08-27T11:13:59.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.599+0000 7fb1eb5a9700 1 Processor -- start 2023-08-27T11:13:59.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.599+0000 7fb1eb5a9700 1 -- start start 2023-08-27T11:13:59.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.600+0000 7fb1eb5a9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1e40af200 0x7fb1e41f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.600+0000 7fb1eb5a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40ca620 0x7fb1e4155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.600+0000 7fb1eb5a9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e40caa00 0x7fb1e4070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.600+0000 7fb1eb5a9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4155ad0 con 0x7fb1e40ca620 2023-08-27T11:13:59.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.600+0000 7fb1eb5a9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c7960 con 0x7fb1e40caa00 2023-08-27T11:13:59.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.600+0000 7fb1eb5a9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40c7c40 con 0x7fb1e40af200 2023-08-27T11:13:59.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.600+0000 7fb1e8b44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40ca620 0x7fb1e4155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.600+0000 7fb1e8b44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40ca620 0x7fb1e4155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54822/0 (socket says 172.21.15.45:54822) 2023-08-27T11:13:59.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.600+0000 7fb1e8b44700 1 -- 172.21.15.45:0/3598685488 learned_addr learned my addr 172.21.15.45:0/3598685488 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:13:59.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1e9345700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1e40af200 0x7fb1e41f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1e9b46700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e40caa00 0x7fb1e4070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1e8b44700 1 -- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1e40af200 msgr2=0x7fb1e41f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1e8b44700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1e40af200 0x7fb1e41f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1e8b44700 1 -- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e40caa00 msgr2=0x7fb1e4070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1e8b44700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e40caa00 0x7fb1e4070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1e8b44700 1 -- 172.21.15.45:0/3598685488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e00094a0 con 0x7fb1e40ca620 2023-08-27T11:13:59.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1e8b44700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40ca620 0x7fb1e4155590 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00090b0 tx=0x7fb1e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:59.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1d9ffb700 1 -- 172.21.15.45:0/3598685488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00038a0 con 0x7fb1e40ca620 2023-08-27T11:13:59.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1d9ffb700 1 -- 172.21.15.45:0/3598685488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e001f070 con 0x7fb1e40ca620 2023-08-27T11:13:59.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1d9ffb700 1 -- 172.21.15.45:0/3598685488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0003c60 con 0x7fb1e40ca620 2023-08-27T11:13:59.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e40c7f20 con 0x7fb1e40ca620 2023-08-27T11:13:59.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.601+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e40c84e0 con 0x7fb1e40ca620 2023-08-27T11:13:59.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.602+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e40734a0 con 0x7fb1e40ca620 2023-08-27T11:13:59.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.603+0000 7fb1d9ffb700 1 -- 172.21.15.45:0/3598685488 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fb1e001a430 con 0x7fb1e40ca620 2023-08-27T11:13:59.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.603+0000 7fb1d9ffb700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb1c806a2e0 0x7fb1c806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:13:59.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.603+0000 7fb1d9ffb700 1 -- 172.21.15.45:0/3598685488 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fb1e0021070 con 0x7fb1e40ca620 2023-08-27T11:13:59.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.604+0000 7fb1e9345700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb1c806a2e0 0x7fb1c806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:13:59.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.604+0000 7fb1e9345700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb1c806a2e0 0x7fb1c806c7a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1e415cdc0 tx=0x7fb1d4009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:13:59.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.608+0000 7fb1d9ffb700 1 -- 172.21.15.45:0/3598685488 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb1e0055840 con 0x7fb1e40ca620 2023-08-27T11:13:59.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.839+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1e40ce870 con 0x7fb1c806a2e0 2023-08-27T11:13:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.889+0000 7fb1d9ffb700 1 -- 172.21.15.45:0/3598685488 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb1d40050d0 con 0x7fb1c806a2e0 2023-08-27T11:13:59.892 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-08-27T11:13:59.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb1c806a2e0 msgr2=0x7fb1c806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb1c806a2e0 0x7fb1c806c7a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1e415cdc0 tx=0x7fb1d4009040 comp rx=0 tx=0).stop 2023-08-27T11:13:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40ca620 msgr2=0x7fb1e4155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:13:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40ca620 0x7fb1e4155590 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00090b0 tx=0x7fb1e0004750 comp rx=0 tx=0).stop 2023-08-27T11:13:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 shutdown_connections 2023-08-27T11:13:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb1c806a2e0 0x7fb1c806c7a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1e40caa00 0x7fb1e4070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb1e40af200 0x7fb1e41f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 --2- 172.21.15.45:0/3598685488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e40ca620 0x7fb1e4155590 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:13:59.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 >> 172.21.15.45:0/3598685488 conn(0x7fb1e40cc7e0 msgr2=0x7fb1e40cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:13:59.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 shutdown_connections 2023-08-27T11:13:59.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:13:59.892+0000 7fb1eb5a9700 1 -- 172.21.15.45:0/3598685488 wait complete. 2023-08-27T11:13:59.906 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-08-27T11:13:59.907 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-08-27T11:14:00.012 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:14:00.011+0000 7f6be43c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:14:00.012 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:14:00.011+0000 7f6be43c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:14:00.013 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:14:00.011+0000 7f6be43c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:14:00.013 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:14:00.011+0000 7f6be43c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:14:00.013 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:14:00.011+0000 7f6be43c0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:14:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.041+0000 7f5ef0a9d700 1 Processor -- start 2023-08-27T11:14:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.041+0000 7f5ef0a9d700 1 -- start start 2023-08-27T11:14:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5ef0a9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec157be0 0x7f5eec0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5ef0a9d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5eec06d5e0 0x7f5eec0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5ef0a9d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec15b460 con 0x7f5eec0b17c0 2023-08-27T11:14:00.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5ef0a9d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec157fc0 con 0x7f5eec06d5e0 2023-08-27T11:14:00.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5ef0a9d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0b2280 con 0x7f5eec157be0 2023-08-27T11:14:00.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5eea59c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec157be0 0x7f5eec0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5ee9d9b700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f5eec0b17c0 0x7f5eec06ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:37270/0 (socket says 172.21.15.45:37270) 2023-08-27T11:14:00.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5eea59c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec157be0 0x7f5eec0716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:52738/0 (socket says 172.21.15.45:52738) 2023-08-27T11:14:00.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5ee9d9b700 1 -- 172.21.15.45:0/1183015708 learned_addr learned my addr 172.21.15.45:0/1183015708 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:14:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.042+0000 7f5eead9d700 1 --2- 172.21.15.45:0/1183015708 >> v2:172.21.15.135:3300/0 conn(0x7f5eec06d5e0 0x7f5eec0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.043+0000 7f5ee959a700 1 -- 172.21.15.45:0/1183015708 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43825249 0 0) 0x7f5eec15b460 con 0x7f5eec0b17c0 2023-08-27T11:14:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.043+0000 7f5eea59c700 1 -- 172.21.15.45:0/1183015708 >> v2:172.21.15.135:3300/0 conn(0x7f5eec06d5e0 msgr2=0x7f5eec0b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:14:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.043+0000 7f5eea59c700 1 --2- 172.21.15.45:0/1183015708 >> v2:172.21.15.135:3300/0 conn(0x7f5eec06d5e0 0x7f5eec0b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.043+0000 7f5eea59c700 1 -- 172.21.15.45:0/1183015708 >> v1:172.21.15.135:6789/0 conn(0x7f5eec0b17c0 legacy=0x7f5eec06ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.043+0000 7f5eea59c700 1 -- 172.21.15.45:0/1183015708 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0b2560 con 0x7f5eec157be0 2023-08-27T11:14:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.043+0000 7f5eea59c700 1 --2- 172.21.15.45:0/1183015708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec157be0 0x7f5eec0716b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00bae0 tx=0x7f5edc00bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b44beadcf723c103 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.043+0000 7f5ee959a700 1 -- 172.21.15.45:0/1183015708 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc00dc70 con 0x7f5eec157be0 2023-08-27T11:14:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.043+0000 7f5ee959a700 1 -- 172.21.15.45:0/1183015708 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc00ddd0 con 0x7f5eec157be0 2023-08-27T11:14:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.043+0000 7f5ee959a700 1 -- 172.21.15.45:0/1183015708 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc010690 con 0x7f5eec157be0 2023-08-27T11:14:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.044+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/1183015708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec157be0 msgr2=0x7f5eec0716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.044+0000 7f5ef0a9d700 1 --2- 172.21.15.45:0/1183015708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec157be0 0x7f5eec0716b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00bae0 tx=0x7f5edc00bdf0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.044+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/1183015708 shutdown_connections 2023-08-27T11:14:00.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.044+0000 7f5ef0a9d700 1 --2- 172.21.15.45:0/1183015708 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec157be0 0x7f5eec0716b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.044+0000 7f5ef0a9d700 1 --2- 172.21.15.45:0/1183015708 >> v2:172.21.15.135:3300/0 conn(0x7f5eec06d5e0 0x7f5eec0b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.044+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/1183015708 >> 172.21.15.45:0/1183015708 conn(0x7f5eec0cc7c0 msgr2=0x7f5eec0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:00.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.044+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/1183015708 shutdown_connections 2023-08-27T11:14:00.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.044+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/1183015708 wait complete. 2023-08-27T11:14:00.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.045+0000 7f5ef0a9d700 1 Processor -- start 2023-08-27T11:14:00.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.045+0000 7f5ef0a9d700 1 -- start start 2023-08-27T11:14:00.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5ef0a9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eec06d5e0 0x7f5eec1ee170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5ef0a9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec0b17c0 0x7f5eec14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5ef0a9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec157be0 0x7f5eec159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5ef0a9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0b35f0 con 0x7f5eec157be0 2023-08-27T11:14:00.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5ef0a9d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0b38d0 con 0x7f5eec06d5e0 2023-08-27T11:14:00.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5ef0a9d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0b3bb0 con 0x7f5eec0b17c0 2023-08-27T11:14:00.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5eead9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec157be0 0x7f5eec159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5eead9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec157be0 0x7f5eec159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54836/0 (socket says 172.21.15.45:54836) 2023-08-27T11:14:00.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5eead9d700 1 -- 172.21.15.45:0/869038211 learned_addr learned my addr 172.21.15.45:0/869038211 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:14:00.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.046+0000 7f5eea59c700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eec06d5e0 0x7f5eec1ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5eead9d700 1 -- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec0b17c0 msgr2=0x7f5eec14f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5eead9d700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec0b17c0 0x7f5eec14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5eead9d700 1 -- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eec06d5e0 msgr2=0x7f5eec1ee170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5eead9d700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eec06d5e0 0x7f5eec1ee170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5eead9d700 1 -- 172.21.15.45:0/869038211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc00b740 con 0x7f5eec157be0 2023-08-27T11:14:00.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5eead9d700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec157be0 0x7f5eec159150 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400efc0 tx=0x7f5ee4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:00.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5edaffd700 1 -- 172.21.15.45:0/869038211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4009b20 con 0x7f5eec157be0 2023-08-27T11:14:00.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5edaffd700 1 -- 172.21.15.45:0/869038211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee400f040 con 0x7f5eec157be0 2023-08-27T11:14:00.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5edaffd700 1 -- 172.21.15.45:0/869038211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400cd40 con 0x7f5eec157be0 2023-08-27T11:14:00.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec0b3e90 con 0x7f5eec157be0 2023-08-27T11:14:00.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.047+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec0b5970 con 0x7f5eec157be0 2023-08-27T11:14:00.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.048+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec1539f0 con 0x7f5eec157be0 2023-08-27T11:14:00.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.049+0000 7f5edaffd700 1 -- 172.21.15.45:0/869038211 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f5ee4014480 con 0x7f5eec157be0 2023-08-27T11:14:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.049+0000 7f5edaffd700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ecc06a2e0 0x7f5ecc06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.049+0000 7f5edaffd700 1 -- 172.21.15.45:0/869038211 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f5ee4089660 con 0x7f5eec157be0 2023-08-27T11:14:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.049+0000 7f5eea59c700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ecc06a2e0 0x7f5ecc06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.050+0000 7f5eea59c700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ecc06a2e0 0x7f5ecc06c7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00b2c0 tx=0x7f5edc013000 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:00.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.054+0000 7f5edaffd700 1 -- 172.21.15.45:0/869038211 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5eec1539f0 con 0x7f5eec157be0 2023-08-27T11:14:00.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.312+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5eec0b3a40 con 0x7f5ecc06a2e0 2023-08-27T11:14:00.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.353+0000 7f5edaffd700 1 -- 172.21.15.45:0/869038211 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5eec0b3a40 con 0x7f5ecc06a2e0 2023-08-27T11:14:00.355 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-08-27T11:14:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ecc06a2e0 msgr2=0x7f5ecc06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ecc06a2e0 0x7f5ecc06c7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00b2c0 tx=0x7f5edc013000 comp rx=0 tx=0).stop 2023-08-27T11:14:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec157be0 msgr2=0x7f5eec159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec157be0 0x7f5eec159150 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400efc0 tx=0x7f5ee4009f60 comp rx=0 tx=0).stop 2023-08-27T11:14:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 shutdown_connections 2023-08-27T11:14:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ecc06a2e0 0x7f5ecc06c7a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5eec157be0 0x7f5eec159150 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eec06d5e0 0x7f5eec1ee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 --2- 172.21.15.45:0/869038211 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5eec0b17c0 0x7f5eec14f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 >> 172.21.15.45:0/869038211 conn(0x7f5eec0cc7c0 msgr2=0x7f5eec156690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 shutdown_connections 2023-08-27T11:14:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.355+0000 7f5ef0a9d700 1 -- 172.21.15.45:0/869038211 wait complete. 2023-08-27T11:14:00.369 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-08-27T11:14:00.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-08-27T11:14:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.506+0000 7fef99651700 1 Processor -- start 2023-08-27T11:14:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.506+0000 7fef99651700 1 -- start start 2023-08-27T11:14:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.507+0000 7fef99651700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef94157be0 0x7fef940716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.507+0000 7fef99651700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9415b460 con 0x7fef940b17c0 2023-08-27T11:14:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.507+0000 7fef99651700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef940b2280 con 0x7fef9406d5e0 2023-08-27T11:14:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.507+0000 7fef99651700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef94157fc0 con 0x7fef94157be0 2023-08-27T11:14:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.507+0000 7fef927fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef940b17c0 0x7fef9406ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41618/0 (socket says 172.21.15.45:41618) 2023-08-27T11:14:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.507+0000 7fef927fc700 1 -- 172.21.15.45:0/169749317 learned_addr learned my addr 172.21.15.45:0/169749317 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:14:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef92ffd700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef94157be0 0x7fef940716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960073568 0 0) 0x7fef9415b460 con 0x7fef940b17c0 2023-08-27T11:14:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef7c0035b0 con 0x7fef940b17c0 2023-08-27T11:14:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3091127537 0 0) 0x7fef7c0035b0 con 0x7fef940b17c0 2023-08-27T11:14:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef94157be0 msgr2=0x7fef940716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:14:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef94157be0 0x7fef940716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 >> v1:172.21.15.135:6789/0 conn(0x7fef9406d5e0 legacy=0x7fef940cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940b2560 con 0x7fef940b17c0 2023-08-27T11:14:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fef84001670 con 0x7fef940b17c0 2023-08-27T11:14:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 >> v1:172.21.15.45:6789/0 conn(0x7fef940b17c0 legacy=0x7fef9406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef7c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef94157be0 0x7fef7c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef7c004d90 0x7fef7c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef94157fc0 con 0x7fef7c003a30 2023-08-27T11:14:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c00bac0 con 0x7fef7c004d90 2023-08-27T11:14:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c00bc80 con 0x7fef94157be0 2023-08-27T11:14:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef92ffd700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef7c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef927fc700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef94157be0 0x7fef7c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef937fe700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef7c004d90 0x7fef7c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef92ffd700 1 -- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef94157be0 msgr2=0x7fef7c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:14:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef92ffd700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef94157be0 0x7fef7c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef92ffd700 1 -- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef7c004d90 msgr2=0x7fef7c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:14:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef92ffd700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef7c004d90 0x7fef7c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.508+0000 7fef92ffd700 1 -- 172.21.15.45:0/169749317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c0035b0 con 0x7fef7c003a30 2023-08-27T11:14:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef92ffd700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef7c003e50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fef88009b30 tx=0x7fef7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f8810471dca6fd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800e6c0 con 0x7fef7c003a30 2023-08-27T11:14:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800e820 con 0x7fef7c003a30 2023-08-27T11:14:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef91ffb700 1 -- 172.21.15.45:0/169749317 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800eb90 con 0x7fef7c003a30 2023-08-27T11:14:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef99651700 1 -- 172.21.15.45:0/169749317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 msgr2=0x7fef7c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef99651700 1 --2- 172.21.15.45:0/169749317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef7c003e50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fef88009b30 tx=0x7fef7c005880 comp rx=0 tx=0).stop 2023-08-27T11:14:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef937fe700 1 -- 172.21.15.45:0/169749317 reap_dead start 2023-08-27T11:14:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef99651700 1 -- 172.21.15.45:0/169749317 shutdown_connections 2023-08-27T11:14:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef99651700 1 -- 172.21.15.45:0/169749317 >> 172.21.15.45:0/169749317 conn(0x7fef940cc7c0 msgr2=0x7fef940ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:00.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef99651700 1 -- 172.21.15.45:0/169749317 shutdown_connections 2023-08-27T11:14:00.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.509+0000 7fef99651700 1 -- 172.21.15.45:0/169749317 wait complete. 2023-08-27T11:14:00.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.511+0000 7fef99651700 1 Processor -- start 2023-08-27T11:14:00.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.511+0000 7fef99651700 1 -- start start 2023-08-27T11:14:00.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef99651700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef940c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef99651700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9414d4d0 0x7fef940c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef99651700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef941f0b00 0x7fef941f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef99651700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef941f1930 con 0x7fef7c003a30 2023-08-27T11:14:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef99651700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef940b2b90 con 0x7fef9414d4d0 2023-08-27T11:14:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef99651700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef940b2de0 con 0x7fef941f0b00 2023-08-27T11:14:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef92ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef940c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef92ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef940c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54854/0 (socket says 172.21.15.45:54854) 2023-08-27T11:14:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef92ffd700 1 -- 172.21.15.45:0/1306886629 learned_addr learned my addr 172.21.15.45:0/1306886629 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:14:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef927fc700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9414d4d0 0x7fef940c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef92ffd700 1 -- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef941f0b00 msgr2=0x7fef941f0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef937fe700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef941f0b00 0x7fef941f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef92ffd700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef941f0b00 0x7fef941f0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef92ffd700 1 -- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9414d4d0 msgr2=0x7fef940c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef92ffd700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9414d4d0 0x7fef940c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef92ffd700 1 -- 172.21.15.45:0/1306886629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940b30c0 con 0x7fef7c003a30 2023-08-27T11:14:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.512+0000 7fef927fc700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9414d4d0 0x7fef940c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:14:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.513+0000 7fef937fe700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef941f0b00 0x7fef941f0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:14:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.513+0000 7fef92ffd700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef940c8f90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0047b0 tx=0x7fef880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.513+0000 7fef7b7fe700 1 -- 172.21.15.45:0/1306886629 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88021070 con 0x7fef7c003a30 2023-08-27T11:14:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.513+0000 7fef7b7fe700 1 -- 172.21.15.45:0/1306886629 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef880045d0 con 0x7fef7c003a30 2023-08-27T11:14:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.513+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef940b33a0 con 0x7fef7c003a30 2023-08-27T11:14:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.513+0000 7fef7b7fe700 1 -- 172.21.15.45:0/1306886629 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800e6c0 con 0x7fef7c003a30 2023-08-27T11:14:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.513+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef941f14c0 con 0x7fef7c003a30 2023-08-27T11:14:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.514+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef941f1650 con 0x7fef7c003a30 2023-08-27T11:14:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.514+0000 7fef7b7fe700 1 -- 172.21.15.45:0/1306886629 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fef88025070 con 0x7fef7c003a30 2023-08-27T11:14:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.515+0000 7fef7b7fe700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef8006a2e0 0x7fef8006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.515+0000 7fef7b7fe700 1 -- 172.21.15.45:0/1306886629 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fef88089be0 con 0x7fef7c003a30 2023-08-27T11:14:00.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.515+0000 7fef927fc700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef8006a2e0 0x7fef8006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:00.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.515+0000 7fef927fc700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef8006a2e0 0x7fef8006c7a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef840017a0 tx=0x7fef84009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:00.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.519+0000 7fef7b7fe700 1 -- 172.21.15.45:0/1306886629 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fef88054f40 con 0x7fef7c003a30 2023-08-27T11:14:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.766+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fef94071190 con 0x7fef8006a2e0 2023-08-27T11:14:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.843+0000 7fef7b7fe700 1 -- 172.21.15.45:0/1306886629 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fef84001620 con 0x7fef8006a2e0 2023-08-27T11:14:00.846 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-08-27T11:14:00.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.845+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef8006a2e0 msgr2=0x7fef8006c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.845+0000 7fef99651700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef8006a2e0 0x7fef8006c7a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef840017a0 tx=0x7fef84009040 comp rx=0 tx=0).stop 2023-08-27T11:14:00.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.845+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 msgr2=0x7fef940c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:00.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.845+0000 7fef99651700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef940c8f90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0047b0 tx=0x7fef880047c0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.845+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 shutdown_connections 2023-08-27T11:14:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.845+0000 7fef99651700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef8006a2e0 0x7fef8006c7a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.846+0000 7fef99651700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef7c003a30 0x7fef940c8f90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.846+0000 7fef99651700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef941f0b00 0x7fef941f0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.846+0000 7fef99651700 1 --2- 172.21.15.45:0/1306886629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9414d4d0 0x7fef940c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.846+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 >> 172.21.15.45:0/1306886629 conn(0x7fef940cc7c0 msgr2=0x7fef940b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.846+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 shutdown_connections 2023-08-27T11:14:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:00.846+0000 7fef99651700 1 -- 172.21.15.45:0/1306886629 wait complete. 2023-08-27T11:14:00.860 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-08-27T11:14:00.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-08-27T11:14:01.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.000+0000 7f3ffb3e8700 1 Processor -- start 2023-08-27T11:14:01.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.000+0000 7f3ffb3e8700 1 -- start start 2023-08-27T11:14:01.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ffb3e8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415b5e0 0x7f3ff406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ffb3e8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3ff406f450 0x7f3ff40cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ffb3e8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff40cb060 con 0x7f3ff415a700 2023-08-27T11:14:01.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ffb3e8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff406f8e0 con 0x7f3ff406f450 2023-08-27T11:14:01.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ffb3e8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff406e320 con 0x7f3ff415b5e0 2023-08-27T11:14:01.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ff9184700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3ff415a700 0x7f3ff415aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41638/0 (socket says 172.21.15.45:41638) 2023-08-27T11:14:01.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ff9985700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3ff406f450 0x7f3ff40cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ff9184700 1 -- 172.21.15.45:0/3940500478 learned_addr learned my addr 172.21.15.45:0/3940500478 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:14:01.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ff8983700 1 --2- 172.21.15.45:0/3940500478 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415b5e0 0x7f3ff406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3febfff700 1 -- 172.21.15.45:0/3940500478 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220791203 0 0) 0x7f3ff40cb060 con 0x7f3ff415a700 2023-08-27T11:14:01.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3febfff700 1 -- 172.21.15.45:0/3940500478 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fe00035b0 con 0x7f3ff415a700 2023-08-27T11:14:01.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ff9985700 1 -- 172.21.15.45:0/3940500478 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415b5e0 msgr2=0x7f3ff406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:14:01.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ff9985700 1 --2- 172.21.15.45:0/3940500478 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415b5e0 0x7f3ff406ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ff9985700 1 -- 172.21.15.45:0/3940500478 >> v1:172.21.15.45:6789/0 conn(0x7f3ff415a700 legacy=0x7f3ff415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ff9985700 1 -- 172.21.15.45:0/3940500478 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff415d7d0 con 0x7f3ff406f450 2023-08-27T11:14:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3ff9985700 1 --2- 172.21.15.45:0/3940500478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff406f450 0x7f3ff40cb4f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009d90 tx=0x7f3fec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c742b0c0b9cea7d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3febfff700 1 -- 172.21.15.45:0/3940500478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec00e070 con 0x7f3ff406f450 2023-08-27T11:14:01.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3febfff700 1 -- 172.21.15.45:0/3940500478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec00bb20 con 0x7f3ff406f450 2023-08-27T11:14:01.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.001+0000 7f3febfff700 1 -- 172.21.15.45:0/3940500478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec012430 con 0x7f3ff406f450 2023-08-27T11:14:01.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.002+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/3940500478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff406f450 msgr2=0x7f3ff40cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.002+0000 7f3ffb3e8700 1 --2- 172.21.15.45:0/3940500478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff406f450 0x7f3ff40cb4f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009d90 tx=0x7f3fec00b330 comp rx=0 tx=0).stop 2023-08-27T11:14:01.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.002+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/3940500478 shutdown_connections 2023-08-27T11:14:01.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.002+0000 7f3ffb3e8700 1 --2- 172.21.15.45:0/3940500478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff406f450 0x7f3ff40cb4f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.002+0000 7f3ffb3e8700 1 --2- 172.21.15.45:0/3940500478 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415b5e0 0x7f3ff406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.002+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/3940500478 >> 172.21.15.45:0/3940500478 conn(0x7f3ff40cc7e0 msgr2=0x7f3ff40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:01.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.002+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/3940500478 shutdown_connections 2023-08-27T11:14:01.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.002+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/3940500478 wait complete. 2023-08-27T11:14:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.003+0000 7f3ffb3e8700 1 Processor -- start 2023-08-27T11:14:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.003+0000 7f3ffb3e8700 1 -- start start 2023-08-27T11:14:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ffb3e8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ff406f450 0x7f3ff415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ffb3e8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415a700 0x7f3ff4152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ffb3e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff415b5e0 0x7f3ff4158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ffb3e8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff40cd5a0 con 0x7f3ff415b5e0 2023-08-27T11:14:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ffb3e8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff40ca1c0 con 0x7f3ff406f450 2023-08-27T11:14:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ffb3e8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff415c760 con 0x7f3ff415a700 2023-08-27T11:14:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ff9985700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff415b5e0 0x7f3ff4158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ff9985700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff415b5e0 0x7f3ff4158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54872/0 (socket says 172.21.15.45:54872) 2023-08-27T11:14:01.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ff9985700 1 -- 172.21.15.45:0/524050147 learned_addr learned my addr 172.21.15.45:0/524050147 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:14:01.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ff9184700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ff406f450 0x7f3ff415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.004+0000 7f3ff8983700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415a700 0x7f3ff4152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3ff9985700 1 -- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415a700 msgr2=0x7f3ff4152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3ff9985700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415a700 0x7f3ff4152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3ff9985700 1 -- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ff406f450 msgr2=0x7f3ff415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3ff9985700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ff406f450 0x7f3ff415c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3ff9985700 1 -- 172.21.15.45:0/524050147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec009a70 con 0x7f3ff415b5e0 2023-08-27T11:14:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3ff9985700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff415b5e0 0x7f3ff4158ed0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3fec014040 tx=0x7f3fec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3fe9ffb700 1 -- 172.21.15.45:0/524050147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0129f0 con 0x7f3ff415b5e0 2023-08-27T11:14:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3fe9ffb700 1 -- 172.21.15.45:0/524050147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec020070 con 0x7f3ff415b5e0 2023-08-27T11:14:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3fe9ffb700 1 -- 172.21.15.45:0/524050147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec012db0 con 0x7f3ff415b5e0 2023-08-27T11:14:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff415ca40 con 0x7f3ff415b5e0 2023-08-27T11:14:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.005+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff40cda90 con 0x7f3ff415b5e0 2023-08-27T11:14:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.006+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff40726d0 con 0x7f3ff415b5e0 2023-08-27T11:14:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.007+0000 7f3fe9ffb700 1 -- 172.21.15.45:0/524050147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 11) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f3fec004040 con 0x7f3ff415b5e0 2023-08-27T11:14:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.007+0000 7f3fe9ffb700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3fd806a370 0x7f3fd806c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.007+0000 7f3fe9ffb700 1 -- 172.21.15.45:0/524050147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f3fec022070 con 0x7f3ff415b5e0 2023-08-27T11:14:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.007+0000 7f3ff9184700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3fd806a370 0x7f3fd806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.008+0000 7f3ff9184700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3fd806a370 0x7f3fd806c830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00011e0 tx=0x7f3ff0009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:01.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.012+0000 7f3fe9ffb700 1 -- 172.21.15.45:0/524050147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3fec055b00 con 0x7f3ff415b5e0 2023-08-27T11:14:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.241+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ff40ca730 con 0x7f3fd806a370 2023-08-27T11:14:01.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.283+0000 7f3fe9ffb700 1 -- 172.21.15.45:0/524050147 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3ff00058e0 con 0x7f3fd806a370 2023-08-27T11:14:01.285 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-08-27T11:14:01.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3fd806a370 msgr2=0x7f3fd806c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3fd806a370 0x7f3fd806c830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00011e0 tx=0x7f3ff0009040 comp rx=0 tx=0).stop 2023-08-27T11:14:01.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff415b5e0 msgr2=0x7f3ff4158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff415b5e0 0x7f3ff4158ed0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3fec014040 tx=0x7f3fec004ad0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 shutdown_connections 2023-08-27T11:14:01.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3fd806a370 0x7f3fd806c830 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ff415b5e0 0x7f3ff4158ed0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ff406f450 0x7f3ff415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 --2- 172.21.15.45:0/524050147 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3ff415a700 0x7f3ff4152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 >> 172.21.15.45:0/524050147 conn(0x7f3ff40cc7e0 msgr2=0x7f3ff4070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:01.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 shutdown_connections 2023-08-27T11:14:01.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.285+0000 7f3ffb3e8700 1 -- 172.21.15.45:0/524050147 wait complete. 2023-08-27T11:14:01.299 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-08-27T11:14:01.299 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-08-27T11:14:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.429+0000 7f44732bd700 1 Processor -- start 2023-08-27T11:14:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.429+0000 7f44732bd700 1 -- start start 2023-08-27T11:14:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f44732bd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 0x7f446c0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f44732bd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c15b460 con 0x7f446c06d5e0 2023-08-27T11:14:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f44732bd700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c0b2280 con 0x7f446c0b17c0 2023-08-27T11:14:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f44732bd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c157fc0 con 0x7f446c157be0 2023-08-27T11:14:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f447185a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f446c06d5e0 0x7f446c0cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41656/0 (socket says 172.21.15.45:41656) 2023-08-27T11:14:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f447185a700 1 -- 172.21.15.45:0/3220770694 learned_addr learned my addr 172.21.15.45:0/3220770694 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:14:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f4463fff700 1 -- 172.21.15.45:0/3220770694 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840949325 0 0) 0x7f446c15b460 con 0x7f446c06d5e0 2023-08-27T11:14:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f4463fff700 1 -- 172.21.15.45:0/3220770694 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44580035b0 con 0x7f446c06d5e0 2023-08-27T11:14:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f4471059700 1 --2- 172.21.15.45:0/3220770694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 0x7f446c0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f4463fff700 1 -- 172.21.15.45:0/3220770694 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877200579 0 0) 0x7f446c0b2280 con 0x7f446c0b17c0 2023-08-27T11:14:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f4463fff700 1 -- 172.21.15.45:0/3220770694 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f446c15b460 con 0x7f446c0b17c0 2023-08-27T11:14:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f4463fff700 1 -- 172.21.15.45:0/3220770694 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409087188 0 0) 0x7f44580035b0 con 0x7f446c06d5e0 2023-08-27T11:14:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f4471059700 1 -- 172.21.15.45:0/3220770694 >> v1:172.21.15.135:6789/0 conn(0x7f446c0b17c0 legacy=0x7f446c06ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f4471059700 1 -- 172.21.15.45:0/3220770694 >> v1:172.21.15.45:6789/0 conn(0x7f446c06d5e0 legacy=0x7f446c0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.430+0000 7f4471059700 1 -- 172.21.15.45:0/3220770694 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0b2560 con 0x7f446c157be0 2023-08-27T11:14:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.431+0000 7f4471059700 1 --2- 172.21.15.45:0/3220770694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 0x7f446c0716b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4468009870 tx=0x7f4468009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5474cbdb2e155b88 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:01.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.431+0000 7f4463fff700 1 -- 172.21.15.45:0/3220770694 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640019f0 con 0x7f446c157be0 2023-08-27T11:14:01.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.431+0000 7f4463fff700 1 -- 172.21.15.45:0/3220770694 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468004030 con 0x7f446c157be0 2023-08-27T11:14:01.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.431+0000 7f4463fff700 1 -- 172.21.15.45:0/3220770694 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44680043a0 con 0x7f446c157be0 2023-08-27T11:14:01.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.431+0000 7f44732bd700 1 -- 172.21.15.45:0/3220770694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 msgr2=0x7f446c0716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.431+0000 7f44732bd700 1 --2- 172.21.15.45:0/3220770694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 0x7f446c0716b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4468009870 tx=0x7f4468009b80 comp rx=0 tx=0).stop 2023-08-27T11:14:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.431+0000 7f44732bd700 1 -- 172.21.15.45:0/3220770694 shutdown_connections 2023-08-27T11:14:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.431+0000 7f44732bd700 1 --2- 172.21.15.45:0/3220770694 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 0x7f446c0716b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.431+0000 7f44732bd700 1 -- 172.21.15.45:0/3220770694 >> 172.21.15.45:0/3220770694 conn(0x7f446c0cc7c0 msgr2=0x7f446c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.432+0000 7f44732bd700 1 -- 172.21.15.45:0/3220770694 shutdown_connections 2023-08-27T11:14:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.432+0000 7f44732bd700 1 -- 172.21.15.45:0/3220770694 wait complete. 2023-08-27T11:14:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.433+0000 7f44732bd700 1 Processor -- start 2023-08-27T11:14:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.433+0000 7f44732bd700 1 -- start start 2023-08-27T11:14:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f44732bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c06d5e0 0x7f446c1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f44732bd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f446c0b17c0 0x7f446c0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f44732bd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 0x7f446c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f44732bd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0c8300 con 0x7f446c06d5e0 2023-08-27T11:14:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f44732bd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0c85e0 con 0x7f446c0b17c0 2023-08-27T11:14:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f44732bd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0c88c0 con 0x7f446c157be0 2023-08-27T11:14:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4471059700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c06d5e0 0x7f446c1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4471059700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c06d5e0 0x7f446c1553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54882/0 (socket says 172.21.15.45:54882) 2023-08-27T11:14:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4471059700 1 -- 172.21.15.45:0/3006772501 learned_addr learned my addr 172.21.15.45:0/3006772501 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:14:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4471059700 1 -- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 msgr2=0x7f446c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4471059700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 0x7f446c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4471059700 1 -- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f446c0b17c0 msgr2=0x7f446c0c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4471059700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f446c0b17c0 0x7f446c0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4471059700 1 -- 172.21.15.45:0/3006772501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44680094a0 con 0x7f446c06d5e0 2023-08-27T11:14:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4471059700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c06d5e0 0x7f446c1553c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4468000c00 tx=0x7f4468004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4461ffb700 1 -- 172.21.15.45:0/3006772501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468003e40 con 0x7f446c06d5e0 2023-08-27T11:14:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4461ffb700 1 -- 172.21.15.45:0/3006772501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446801f070 con 0x7f446c06d5e0 2023-08-27T11:14:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c0c8ba0 con 0x7f446c06d5e0 2023-08-27T11:14:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.434+0000 7f4461ffb700 1 -- 172.21.15.45:0/3006772501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468011870 con 0x7f446c06d5e0 2023-08-27T11:14:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.435+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c155900 con 0x7f446c06d5e0 2023-08-27T11:14:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.435+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c071170 con 0x7f446c06d5e0 2023-08-27T11:14:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.436+0000 7f4461ffb700 1 -- 172.21.15.45:0/3006772501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 11) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f44680119d0 con 0x7f446c06d5e0 2023-08-27T11:14:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.436+0000 7f4461ffb700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f445006a370 0x7f445006c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.436+0000 7f4461ffb700 1 -- 172.21.15.45:0/3006772501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f4468021070 con 0x7f446c06d5e0 2023-08-27T11:14:01.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.437+0000 7f4470858700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f445006a370 0x7f445006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.437+0000 7f4470858700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f445006a370 0x7f445006c830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f445c007fd0 tx=0x7f445c007980 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:01.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.441+0000 7f4461ffb700 1 -- 172.21.15.45:0/3006772501 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f446c071170 con 0x7f446c06d5e0 2023-08-27T11:14:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.674+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f446c0c8750 con 0x7f445006a370 2023-08-27T11:14:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.719+0000 7f4461ffb700 1 -- 172.21.15.45:0/3006772501 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f446c0c8750 con 0x7f445006a370 2023-08-27T11:14:01.722 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-08-27T11:14:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f445006a370 msgr2=0x7f445006c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f445006a370 0x7f445006c830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f445c007fd0 tx=0x7f445c007980 comp rx=0 tx=0).stop 2023-08-27T11:14:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c06d5e0 msgr2=0x7f446c1553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c06d5e0 0x7f446c1553c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4468000c00 tx=0x7f4468004750 comp rx=0 tx=0).stop 2023-08-27T11:14:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 shutdown_connections 2023-08-27T11:14:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f445006a370 0x7f445006c830 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c06d5e0 0x7f446c1553c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f446c157be0 0x7f446c0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 --2- 172.21.15.45:0/3006772501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f446c0b17c0 0x7f446c0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 >> 172.21.15.45:0/3006772501 conn(0x7f446c0cc7c0 msgr2=0x7f446c06ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 shutdown_connections 2023-08-27T11:14:01.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.722+0000 7f44732bd700 1 -- 172.21.15.45:0/3006772501 wait complete. 2023-08-27T11:14:01.736 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-08-27T11:14:01.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-08-27T11:14:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.865+0000 7f991e498700 1 Processor -- start 2023-08-27T11:14:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.865+0000 7f991e498700 1 -- start start 2023-08-27T11:14:01.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f991e498700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f99180cd300 0x7f99180cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f991e498700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f991806ed90 0x7f991815e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f991e498700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99180cfae0 con 0x7f991815a0f0 2023-08-27T11:14:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f991e498700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991815e900 con 0x7f99180cd300 2023-08-27T11:14:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f991e498700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99180cae60 con 0x7f991806ed90 2023-08-27T11:14:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f9917fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f991815a0f0 0x7f991815daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41666/0 (socket says 172.21.15.45:41666) 2023-08-27T11:14:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f9917fff700 1 -- 172.21.15.45:0/3929554739 learned_addr learned my addr 172.21.15.45:0/3929554739 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:14:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f99177fe700 1 --2- 172.21.15.45:0/3929554739 >> v2:172.21.15.135:3300/0 conn(0x7f99180cd300 0x7f99180cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f991ca35700 1 --2- 172.21.15.45:0/3929554739 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f991806ed90 0x7f991815e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f9916ffd700 1 -- 172.21.15.45:0/3929554739 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793664242 0 0) 0x7f99180cfae0 con 0x7f991815a0f0 2023-08-27T11:14:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f9916ffd700 1 -- 172.21.15.45:0/3929554739 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99000035b0 con 0x7f991815a0f0 2023-08-27T11:14:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.866+0000 7f9916ffd700 1 -- 172.21.15.45:0/3929554739 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3582535422 0 0) 0x7f99000035b0 con 0x7f991815a0f0 2023-08-27T11:14:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f99177fe700 1 -- 172.21.15.45:0/3929554739 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f991806ed90 msgr2=0x7f991815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:14:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f99177fe700 1 --2- 172.21.15.45:0/3929554739 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f991806ed90 0x7f991815e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f99177fe700 1 -- 172.21.15.45:0/3929554739 >> v1:172.21.15.45:6789/0 conn(0x7f991815a0f0 legacy=0x7f991815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f99177fe700 1 -- 172.21.15.45:0/3929554739 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180cb140 con 0x7f99180cd300 2023-08-27T11:14:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f991ca35700 1 --2- 172.21.15.45:0/3929554739 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f991806ed90 0x7f991815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:14:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f99177fe700 1 --2- 172.21.15.45:0/3929554739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99180cd300 0x7f99180cbc20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9908009840 tx=0x7f9908009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47149f0b1959e9b0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f9916ffd700 1 -- 172.21.15.45:0/3929554739 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c001670 con 0x7f99180cd300 2023-08-27T11:14:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f9916ffd700 1 -- 172.21.15.45:0/3929554739 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9908004030 con 0x7f99180cd300 2023-08-27T11:14:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f9916ffd700 1 -- 172.21.15.45:0/3929554739 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99080043a0 con 0x7f99180cd300 2023-08-27T11:14:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f991e498700 1 -- 172.21.15.45:0/3929554739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99180cd300 msgr2=0x7f99180cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.867+0000 7f991e498700 1 --2- 172.21.15.45:0/3929554739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99180cd300 0x7f99180cbc20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9908009840 tx=0x7f9908009b50 comp rx=0 tx=0).stop 2023-08-27T11:14:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.868+0000 7f991e498700 1 -- 172.21.15.45:0/3929554739 shutdown_connections 2023-08-27T11:14:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.868+0000 7f991e498700 1 --2- 172.21.15.45:0/3929554739 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f991806ed90 0x7f991815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.868+0000 7f991e498700 1 --2- 172.21.15.45:0/3929554739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99180cd300 0x7f99180cbc20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.868+0000 7f991e498700 1 -- 172.21.15.45:0/3929554739 >> 172.21.15.45:0/3929554739 conn(0x7f99180cc7e0 msgr2=0x7f99180ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.868+0000 7f991e498700 1 -- 172.21.15.45:0/3929554739 shutdown_connections 2023-08-27T11:14:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.868+0000 7f991e498700 1 -- 172.21.15.45:0/3929554739 wait complete. 2023-08-27T11:14:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.869+0000 7f991e498700 1 Processor -- start 2023-08-27T11:14:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.869+0000 7f991e498700 1 -- start start 2023-08-27T11:14:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f991e498700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806ed90 0x7f99180b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f991e498700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99180cd300 0x7f9918156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f991e498700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815a0f0 0x7f99180c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f991e498700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99181ef460 con 0x7f991806ed90 2023-08-27T11:14:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f991e498700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99181ef740 con 0x7f991815a0f0 2023-08-27T11:14:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f991e498700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99181efa20 con 0x7f99180cd300 2023-08-27T11:14:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9917fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806ed90 0x7f99180b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9917fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806ed90 0x7f99180b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54896/0 (socket says 172.21.15.45:54896) 2023-08-27T11:14:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9917fff700 1 -- 172.21.15.45:0/3340259528 learned_addr learned my addr 172.21.15.45:0/3340259528 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:14:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f991ca35700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815a0f0 0x7f99180c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f99177fe700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99180cd300 0x7f9918156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9917fff700 1 -- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99180cd300 msgr2=0x7f9918156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9917fff700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99180cd300 0x7f9918156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9917fff700 1 -- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815a0f0 msgr2=0x7f99180c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9917fff700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815a0f0 0x7f99180c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9917fff700 1 -- 172.21.15.45:0/3340259528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9910009a70 con 0x7f991806ed90 2023-08-27T11:14:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f991ca35700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815a0f0 0x7f99180c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:14:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f99177fe700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99180cd300 0x7f9918156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:14:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9917fff700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806ed90 0x7f99180b29c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f990c00c520 tx=0x7f990c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.870+0000 7f9914ff9700 1 -- 172.21.15.45:0/3340259528 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c004c80 con 0x7f991806ed90 2023-08-27T11:14:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.871+0000 7f9914ff9700 1 -- 172.21.15.45:0/3340259528 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c004e20 con 0x7f991806ed90 2023-08-27T11:14:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.871+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99080094a0 con 0x7f991806ed90 2023-08-27T11:14:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.871+0000 7f9914ff9700 1 -- 172.21.15.45:0/3340259528 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c007310 con 0x7f991806ed90 2023-08-27T11:14:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.871+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99181effe0 con 0x7f991806ed90 2023-08-27T11:14:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.871+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99180726d0 con 0x7f991806ed90 2023-08-27T11:14:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.872+0000 7f9914ff9700 1 -- 172.21.15.45:0/3340259528 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 11) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f990c007470 con 0x7f991806ed90 2023-08-27T11:14:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.872+0000 7f9914ff9700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f990406a370 0x7f990406c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:14:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.872+0000 7f9914ff9700 1 -- 172.21.15.45:0/3340259528 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f990c088a60 con 0x7f991806ed90 2023-08-27T11:14:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.873+0000 7f99177fe700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f990406a370 0x7f990406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:14:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.873+0000 7f99177fe700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f990406a370 0x7f990406c830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f99080090b0 tx=0x7f9908008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:14:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:01.877+0000 7f9914ff9700 1 -- 172.21.15.45:0/3340259528 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f990c053bb0 con 0x7f991806ed90 2023-08-27T11:14:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.106+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f99180cb700 con 0x7f990406a370 2023-08-27T11:14:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.156+0000 7f9914ff9700 1 -- 172.21.15.45:0/3340259528 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9908004360 con 0x7f990406a370 2023-08-27T11:14:02.159 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-08-27T11:14:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f990406a370 msgr2=0x7f990406c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f990406a370 0x7f990406c830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f99080090b0 tx=0x7f9908008030 comp rx=0 tx=0).stop 2023-08-27T11:14:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806ed90 msgr2=0x7f99180b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:14:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806ed90 0x7f99180b29c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f990c00c520 tx=0x7f990c00c8e0 comp rx=0 tx=0).stop 2023-08-27T11:14:02.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 shutdown_connections 2023-08-27T11:14:02.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f990406a370 0x7f990406c830 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:02.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806ed90 0x7f99180b29c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:02.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815a0f0 0x7f99180c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 --2- 172.21.15.45:0/3340259528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99180cd300 0x7f9918156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:14:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 >> 172.21.15.45:0/3340259528 conn(0x7f99180cc7e0 msgr2=0x7f99180ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:14:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 shutdown_connections 2023-08-27T11:14:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:14:02.159+0000 7f991e498700 1 -- 172.21.15.45:0/3340259528 wait complete. 2023-08-27T11:14:02.173 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-08-27T11:14:03.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:14:03.653 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:14:33.721 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:14:33.725 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:15:03.877 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:15:03.880 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:15:33.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:15:33.957 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:04.033 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:04.037 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:34.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:16:34.106 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:17:04.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:17:04.162 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:17:34.254 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:17:34.257 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:18:04.336 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:18:04.399 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:18:34.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:18:34.470 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:19:04.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:19:04.539 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:19:34.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:19:34.657 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:20:04.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:20:05.454 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:20:35.637 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:20:35.642 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:21:00.713 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:21:00.711+0000 7f6bdc330700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:21:00.713 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:21:00.711+0000 7f6bdc330700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:21:00.713 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:21:00.711+0000 7f6bdc330700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:21:00.714 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:21:00.711+0000 7f6bdc330700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:21:00.714 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:21:00.711+0000 7f6bdc330700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:21:05.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:21:05.712 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:21:35.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:21:35.812 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:22:05.877 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:22:05.880 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:22:35.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:22:35.971 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:23:06.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:23:06.044 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:23:36.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:23:36.255 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:24:06.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:24:06.349 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:24:20.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-08-27T11:24:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.317+0000 7fb62235d700 1 Processor -- start 2023-08-27T11:24:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.317+0000 7fb62235d700 1 -- start start 2023-08-27T11:24:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.318+0000 7fb62235d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb61c15a220 0x7fb61c0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.318+0000 7fb62235d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb61c072080 0x7fb61c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.318+0000 7fb62235d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61c0b3220 con 0x7fb61c0aea00 2023-08-27T11:24:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.318+0000 7fb62235d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c0b2c60 con 0x7fb61c15a220 2023-08-27T11:24:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.318+0000 7fb62235d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c0b2f40 con 0x7fb61c072080 2023-08-27T11:24:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.318+0000 7fb61bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb61c15a220 0x7fb61c0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.318+0000 7fb61bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb61c15a220 0x7fb61c0cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54904/0 (socket says 172.21.15.45:54904) 2023-08-27T11:24:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.318+0000 7fb61bfff700 1 -- 172.21.15.45:0/1400902678 learned_addr learned my addr 172.21.15.45:0/1400902678 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb6208fa700 1 --2- 172.21.15.45:0/1400902678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb61c072080 0x7fb61c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb61bfff700 1 -- 172.21.15.45:0/1400902678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb61c072080 msgr2=0x7fb61c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb61bfff700 1 --2- 172.21.15.45:0/1400902678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb61c072080 0x7fb61c06a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb61bfff700 1 -- 172.21.15.45:0/1400902678 >> v1:172.21.15.135:6789/0 conn(0x7fb61c0aea00 legacy=0x7fb61c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb61bfff700 1 -- 172.21.15.45:0/1400902678 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c0cec70 con 0x7fb61c15a220 2023-08-27T11:24:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb61bfff700 1 --2- 172.21.15.45:0/1400902678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c15a220 0x7fb61c0cc780 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00bb90 tx=0x7fb60c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e325f7281a28a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb61affd700 1 -- 172.21.15.45:0/1400902678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00dd10 con 0x7fb61c15a220 2023-08-27T11:24:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb61affd700 1 -- 172.21.15.45:0/1400902678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c00de70 con 0x7fb61c15a220 2023-08-27T11:24:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb61affd700 1 -- 172.21.15.45:0/1400902678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c004230 con 0x7fb61c15a220 2023-08-27T11:24:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb62235d700 1 -- 172.21.15.45:0/1400902678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c15a220 msgr2=0x7fb61c0cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.319+0000 7fb62235d700 1 --2- 172.21.15.45:0/1400902678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c15a220 0x7fb61c0cc780 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00bb90 tx=0x7fb60c00bea0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.320+0000 7fb62235d700 1 -- 172.21.15.45:0/1400902678 shutdown_connections 2023-08-27T11:24:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.320+0000 7fb62235d700 1 --2- 172.21.15.45:0/1400902678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb61c072080 0x7fb61c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.320+0000 7fb62235d700 1 --2- 172.21.15.45:0/1400902678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c15a220 0x7fb61c0cc780 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.320+0000 7fb62235d700 1 -- 172.21.15.45:0/1400902678 >> 172.21.15.45:0/1400902678 conn(0x7fb61c0b4650 msgr2=0x7fb61c0b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.320+0000 7fb62235d700 1 -- 172.21.15.45:0/1400902678 shutdown_connections 2023-08-27T11:24:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.320+0000 7fb62235d700 1 -- 172.21.15.45:0/1400902678 wait complete. 2023-08-27T11:24:20.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.321+0000 7fb62235d700 1 Processor -- start 2023-08-27T11:24:20.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.321+0000 7fb62235d700 1 -- start start 2023-08-27T11:24:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb62235d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c072080 0x7fb61c0c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb62235d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb61c0aea00 0x7fb61c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb62235d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb61c15a220 0x7fb61c0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb62235d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c152180 con 0x7fb61c072080 2023-08-27T11:24:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb62235d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c151970 con 0x7fb61c0aea00 2023-08-27T11:24:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb62235d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c15b450 con 0x7fb61c15a220 2023-08-27T11:24:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c072080 0x7fb61c0c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c072080 0x7fb61c0c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54910/0 (socket says 172.21.15.45:54910) 2023-08-27T11:24:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61bfff700 1 -- 172.21.15.45:0/1817768705 learned_addr learned my addr 172.21.15.45:0/1817768705 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61b7fe700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb61c0aea00 0x7fb61c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61bfff700 1 -- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb61c15a220 msgr2=0x7fb61c0af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61bfff700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb61c15a220 0x7fb61c0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61bfff700 1 -- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb61c0aea00 msgr2=0x7fb61c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61bfff700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb61c0aea00 0x7fb61c0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61bfff700 1 -- 172.21.15.45:0/1817768705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c00b7f0 con 0x7fb61c072080 2023-08-27T11:24:20.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61b7fe700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb61c0aea00 0x7fb61c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:20.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.322+0000 7fb61bfff700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c072080 0x7fb61c0c2a60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00b340 tx=0x7fb60c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:20.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.323+0000 7fb618ff9700 1 -- 172.21.15.45:0/1817768705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c020070 con 0x7fb61c072080 2023-08-27T11:24:20.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.323+0000 7fb618ff9700 1 -- 172.21.15.45:0/1817768705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c003bd0 con 0x7fb61c072080 2023-08-27T11:24:20.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.323+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c151bc0 con 0x7fb61c072080 2023-08-27T11:24:20.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.323+0000 7fb618ff9700 1 -- 172.21.15.45:0/1817768705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c010470 con 0x7fb61c072080 2023-08-27T11:24:20.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.323+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c0cc160 con 0x7fb61c072080 2023-08-27T11:24:20.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.324+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c0ccf10 con 0x7fb61c072080 2023-08-27T11:24:20.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.325+0000 7fb618ff9700 1 -- 172.21.15.45:0/1817768705 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fb60c02d3f0 con 0x7fb61c072080 2023-08-27T11:24:20.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.325+0000 7fb618ff9700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb5fc06a310 0x7fb5fc06c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.325+0000 7fb618ff9700 1 -- 172.21.15.45:0/1817768705 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fb60c08c380 con 0x7fb61c072080 2023-08-27T11:24:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.326+0000 7fb61b7fe700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb5fc06a310 0x7fb5fc06c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.326+0000 7fb61b7fe700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb5fc06a310 0x7fb5fc06c7d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb61c158be0 tx=0x7fb604009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.329+0000 7fb618ff9700 1 -- 172.21.15.45:0/1817768705 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb60c0576f0 con 0x7fb61c072080 2023-08-27T11:24:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.560+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb61c0cccc0 con 0x7fb5fc06a310 2023-08-27T11:24:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.628+0000 7fb618ff9700 1 -- 172.21.15.45:0/1817768705 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb604003ba0 con 0x7fb5fc06a310 2023-08-27T11:24:20.630 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-08-27T11:24:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb5fc06a310 msgr2=0x7fb5fc06c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb5fc06a310 0x7fb5fc06c7d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb61c158be0 tx=0x7fb604009040 comp rx=0 tx=0).stop 2023-08-27T11:24:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c072080 msgr2=0x7fb61c0c2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c072080 0x7fb61c0c2a60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00b340 tx=0x7fb60c004750 comp rx=0 tx=0).stop 2023-08-27T11:24:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 shutdown_connections 2023-08-27T11:24:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb5fc06a310 0x7fb5fc06c7d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb61c072080 0x7fb61c0c2a60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb61c15a220 0x7fb61c0af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 --2- 172.21.15.45:0/1817768705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb61c0aea00 0x7fb61c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 >> 172.21.15.45:0/1817768705 conn(0x7fb61c0b4650 msgr2=0x7fb61c0ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:20.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 shutdown_connections 2023-08-27T11:24:20.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.630+0000 7fb62235d700 1 -- 172.21.15.45:0/1817768705 wait complete. 2023-08-27T11:24:20.644 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-08-27T11:24:20.645 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-08-27T11:24:20.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.774+0000 7f75a56d0700 1 Processor -- start 2023-08-27T11:24:20.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.774+0000 7f75a56d0700 1 -- start start 2023-08-27T11:24:20.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.775+0000 7f75a56d0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a006d5e0 0x7f75a00b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.775+0000 7f75a56d0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a015b460 con 0x7f75a0157be0 2023-08-27T11:24:20.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.775+0000 7f75a56d0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a00b2280 con 0x7f75a00b17c0 2023-08-27T11:24:20.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.775+0000 7f75a56d0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a0157fc0 con 0x7f75a006d5e0 2023-08-27T11:24:20.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.775+0000 7f759effd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f75a0157be0 0x7f75a00716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41694/0 (socket says 172.21.15.45:41694) 2023-08-27T11:24:20.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.775+0000 7f759effd700 1 -- 172.21.15.45:0/3525298446 learned_addr learned my addr 172.21.15.45:0/3525298446 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:20.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759f7fe700 1 --2- 172.21.15.45:0/3525298446 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a006d5e0 0x7f75a00b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759dffb700 1 -- 172.21.15.45:0/3525298446 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538628734 0 0) 0x7f75a015b460 con 0x7f75a0157be0 2023-08-27T11:24:20.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759dffb700 1 -- 172.21.15.45:0/3525298446 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75880035b0 con 0x7f75a0157be0 2023-08-27T11:24:20.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759dffb700 1 -- 172.21.15.45:0/3525298446 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445058213 0 0) 0x7f75a00b2280 con 0x7f75a00b17c0 2023-08-27T11:24:20.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759dffb700 1 -- 172.21.15.45:0/3525298446 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a015b460 con 0x7f75a00b17c0 2023-08-27T11:24:20.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759dffb700 1 -- 172.21.15.45:0/3525298446 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2827649990 0 0) 0x7f75880035b0 con 0x7f75a0157be0 2023-08-27T11:24:20.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759f7fe700 1 -- 172.21.15.45:0/3525298446 >> v1:172.21.15.135:6789/0 conn(0x7f75a00b17c0 legacy=0x7f75a006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759f7fe700 1 -- 172.21.15.45:0/3525298446 >> v1:172.21.15.45:6789/0 conn(0x7f75a0157be0 legacy=0x7f75a00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759f7fe700 1 -- 172.21.15.45:0/3525298446 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00b2560 con 0x7f75a006d5e0 2023-08-27T11:24:20.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759f7fe700 1 --2- 172.21.15.45:0/3525298446 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a006d5e0 0x7f75a00b1c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7598009dc0 tx=0x7f759800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e793506f52c1ccc2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:20.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759dffb700 1 -- 172.21.15.45:0/3525298446 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759800e070 con 0x7f75a006d5e0 2023-08-27T11:24:20.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759dffb700 1 -- 172.21.15.45:0/3525298446 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598003a20 con 0x7f75a006d5e0 2023-08-27T11:24:20.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.776+0000 7f759dffb700 1 -- 172.21.15.45:0/3525298446 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594001670 con 0x7f75a006d5e0 2023-08-27T11:24:20.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.777+0000 7f75a56d0700 1 -- 172.21.15.45:0/3525298446 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a006d5e0 msgr2=0x7f75a00b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.777+0000 7f75a56d0700 1 --2- 172.21.15.45:0/3525298446 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a006d5e0 0x7f75a00b1c70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7598009dc0 tx=0x7f759800b330 comp rx=0 tx=0).stop 2023-08-27T11:24:20.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.777+0000 7f75a56d0700 1 -- 172.21.15.45:0/3525298446 shutdown_connections 2023-08-27T11:24:20.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.777+0000 7f75a56d0700 1 --2- 172.21.15.45:0/3525298446 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a006d5e0 0x7f75a00b1c70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.777+0000 7f75a56d0700 1 -- 172.21.15.45:0/3525298446 >> 172.21.15.45:0/3525298446 conn(0x7f75a00cc7c0 msgr2=0x7f75a00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:20.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.777+0000 7f75a56d0700 1 -- 172.21.15.45:0/3525298446 shutdown_connections 2023-08-27T11:24:20.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.777+0000 7f75a56d0700 1 -- 172.21.15.45:0/3525298446 wait complete. 2023-08-27T11:24:20.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.778+0000 7f75a56d0700 1 Processor -- start 2023-08-27T11:24:20.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.778+0000 7f75a56d0700 1 -- start start 2023-08-27T11:24:20.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f75a56d0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75a006d5e0 0x7f75a01f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f75a56d0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a00b17c0 0x7f75a00b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f75a56d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75a0157be0 0x7f75a00b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f75a56d0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a0149cc0 con 0x7f75a0157be0 2023-08-27T11:24:20.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f75a56d0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a01f6be0 con 0x7f75a006d5e0 2023-08-27T11:24:20.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f75a56d0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a01f2b60 con 0x7f75a00b17c0 2023-08-27T11:24:20.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f759f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75a0157be0 0x7f75a00b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f759f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75a0157be0 0x7f75a00b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54928/0 (socket says 172.21.15.45:54928) 2023-08-27T11:24:20.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f759f7fe700 1 -- 172.21.15.45:0/2316356121 learned_addr learned my addr 172.21.15.45:0/2316356121 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:20.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.779+0000 7f759effd700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75a006d5e0 0x7f75a01f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f759e7fc700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a00b17c0 0x7f75a00b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f759f7fe700 1 -- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a00b17c0 msgr2=0x7f75a00b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f759f7fe700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a00b17c0 0x7f75a00b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f759f7fe700 1 -- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75a006d5e0 msgr2=0x7f75a01f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:20.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f759f7fe700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75a006d5e0 0x7f75a01f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:20.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f759f7fe700 1 -- 172.21.15.45:0/2316356121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7598009a70 con 0x7f75a0157be0 2023-08-27T11:24:20.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f759effd700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75a006d5e0 0x7f75a01f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:20.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f759f7fe700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75a0157be0 0x7f75a00b4540 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7598015040 tx=0x7f7598004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:20.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f75877fe700 1 -- 172.21.15.45:0/2316356121 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75980043c0 con 0x7f75a0157be0 2023-08-27T11:24:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f75877fe700 1 -- 172.21.15.45:0/2316356121 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598004520 con 0x7f75a0157be0 2023-08-27T11:24:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f75877fe700 1 -- 172.21.15.45:0/2316356121 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598004890 con 0x7f75a0157be0 2023-08-27T11:24:20.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a01f2e40 con 0x7f75a0157be0 2023-08-27T11:24:20.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.780+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a01f3400 con 0x7f75a0157be0 2023-08-27T11:24:20.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.781+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a0072690 con 0x7f75a0157be0 2023-08-27T11:24:20.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.781+0000 7f75877fe700 1 -- 172.21.15.45:0/2316356121 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f759801c460 con 0x7f75a0157be0 2023-08-27T11:24:20.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.782+0000 7f75877fe700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f758c06a2e0 0x7f758c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:20.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.782+0000 7f75877fe700 1 -- 172.21.15.45:0/2316356121 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f759808a310 con 0x7f75a0157be0 2023-08-27T11:24:20.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.782+0000 7f759effd700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f758c06a2e0 0x7f758c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:20.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.783+0000 7f759effd700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f758c06a2e0 0x7f758c06c7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7594001eb0 tx=0x7f7594001ca0 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:20.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:20.786+0000 7f75877fe700 1 -- 172.21.15.45:0/2316356121 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7598055480 con 0x7f75a0157be0 2023-08-27T11:24:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.014+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f75a0070eb0 con 0x7f758c06a2e0 2023-08-27T11:24:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.076+0000 7f75877fe700 1 -- 172.21.15.45:0/2316356121 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f759400e620 con 0x7f758c06a2e0 2023-08-27T11:24:21.077 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-08-27T11:24:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f758c06a2e0 msgr2=0x7f758c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f758c06a2e0 0x7f758c06c7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7594001eb0 tx=0x7f7594001ca0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75a0157be0 msgr2=0x7f75a00b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75a0157be0 0x7f75a00b4540 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7598015040 tx=0x7f7598004ad0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 shutdown_connections 2023-08-27T11:24:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f758c06a2e0 0x7f758c06c7a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75a0157be0 0x7f75a00b4540 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75a006d5e0 0x7f75a01f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 --2- 172.21.15.45:0/2316356121 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f75a00b17c0 0x7f75a00b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 >> 172.21.15.45:0/2316356121 conn(0x7f75a00cc7c0 msgr2=0x7f75a00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 shutdown_connections 2023-08-27T11:24:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.078+0000 7f75a56d0700 1 -- 172.21.15.45:0/2316356121 wait complete. 2023-08-27T11:24:21.091 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-08-27T11:24:21.092 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-08-27T11:24:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.226+0000 7f5e5c5fd700 1 Processor -- start 2023-08-27T11:24:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.226+0000 7f5e5c5fd700 1 -- start start 2023-08-27T11:24:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.227+0000 7f5e5c5fd700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5e5406a280 0x7f5e5406a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.227+0000 7f5e5c5fd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e5415b5b0 0x7f5e5415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.227+0000 7f5e5c5fd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e540b2290 con 0x7f5e54071400 2023-08-27T11:24:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.227+0000 7f5e5c5fd700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e54071fa0 con 0x7f5e5415b5b0 2023-08-27T11:24:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.227+0000 7f5e5c5fd700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5406aba0 con 0x7f5e5406a280 2023-08-27T11:24:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.227+0000 7f5e5ab9a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e5415b5b0 0x7f5e5415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.227+0000 7f5e59b98700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5e54071400 0x7f5e54071890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41708/0 (socket says 172.21.15.45:41708) 2023-08-27T11:24:21.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.227+0000 7f5e59b98700 1 -- 172.21.15.45:0/3723837601 learned_addr learned my addr 172.21.15.45:0/3723837601 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:21.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.227+0000 7f5e5a399700 1 --2- 172.21.15.45:0/3723837601 >> v2:172.21.15.135:3300/0 conn(0x7f5e5406a280 0x7f5e5406a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5ab9a700 1 -- 172.21.15.45:0/3723837601 >> v2:172.21.15.135:3300/0 conn(0x7f5e5406a280 msgr2=0x7f5e5406a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5ab9a700 1 --2- 172.21.15.45:0/3723837601 >> v2:172.21.15.135:3300/0 conn(0x7f5e5406a280 0x7f5e5406a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5ab9a700 1 -- 172.21.15.45:0/3723837601 >> v1:172.21.15.45:6789/0 conn(0x7f5e54071400 legacy=0x7f5e54071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5ab9a700 1 -- 172.21.15.45:0/3723837601 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540b1960 con 0x7f5e5415b5b0 2023-08-27T11:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e59397700 1 -- 172.21.15.45:0/3723837601 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277913933 0 0) 0x7f5e540b2290 con 0x7f5e54071400 2023-08-27T11:24:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5ab9a700 1 --2- 172.21.15.45:0/3723837601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 0x7f5e5415ba00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5e540cd830 tx=0x7f5e4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8861bc9d6ffe564 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e59397700 1 -- 172.21.15.45:0/3723837601 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00e070 con 0x7f5e5415b5b0 2023-08-27T11:24:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e59397700 1 -- 172.21.15.45:0/3723837601 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c00bb20 con 0x7f5e5415b5b0 2023-08-27T11:24:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e59397700 1 -- 172.21.15.45:0/3723837601 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c012430 con 0x7f5e5415b5b0 2023-08-27T11:24:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/3723837601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 msgr2=0x7f5e5415ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5c5fd700 1 --2- 172.21.15.45:0/3723837601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 0x7f5e5415ba00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5e540cd830 tx=0x7f5e4c00b330 comp rx=0 tx=0).stop 2023-08-27T11:24:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/3723837601 shutdown_connections 2023-08-27T11:24:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5c5fd700 1 --2- 172.21.15.45:0/3723837601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 0x7f5e5415ba00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5c5fd700 1 --2- 172.21.15.45:0/3723837601 >> v2:172.21.15.135:3300/0 conn(0x7f5e5406a280 0x7f5e5406a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.228+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/3723837601 >> 172.21.15.45:0/3723837601 conn(0x7f5e540cc7c0 msgr2=0x7f5e540ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.229+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/3723837601 shutdown_connections 2023-08-27T11:24:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.229+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/3723837601 wait complete. 2023-08-27T11:24:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.230+0000 7f5e5c5fd700 1 Processor -- start 2023-08-27T11:24:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.230+0000 7f5e5c5fd700 1 -- start start 2023-08-27T11:24:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.230+0000 7f5e5c5fd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e5406a280 0x7f5e540cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.230+0000 7f5e5c5fd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e54071400 0x7f5e5414b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.230+0000 7f5e5c5fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 0x7f5e540cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.230+0000 7f5e5c5fd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5414b6d0 con 0x7f5e5415b5b0 2023-08-27T11:24:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.230+0000 7f5e5c5fd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540a9590 con 0x7f5e54071400 2023-08-27T11:24:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.230+0000 7f5e5c5fd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540a92b0 con 0x7f5e5406a280 2023-08-27T11:24:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5ab9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 0x7f5e540cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5ab9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 0x7f5e540cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54942/0 (socket says 172.21.15.45:54942) 2023-08-27T11:24:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5ab9a700 1 -- 172.21.15.45:0/2134439929 learned_addr learned my addr 172.21.15.45:0/2134439929 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5ab9a700 1 -- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e5406a280 msgr2=0x7f5e540cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5ab9a700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e5406a280 0x7f5e540cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5ab9a700 1 -- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e54071400 msgr2=0x7f5e5414b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e59b98700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e54071400 0x7f5e5414b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5ab9a700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e54071400 0x7f5e5414b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5ab9a700 1 -- 172.21.15.45:0/2134439929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c009a70 con 0x7f5e5415b5b0 2023-08-27T11:24:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5ab9a700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 0x7f5e540cbc20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5e5406cbb0 tx=0x7f5e4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e4affd700 1 -- 172.21.15.45:0/2134439929 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c012430 con 0x7f5e5415b5b0 2023-08-27T11:24:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e4affd700 1 -- 172.21.15.45:0/2134439929 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c020070 con 0x7f5e5415b5b0 2023-08-27T11:24:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e540ce430 con 0x7f5e5415b5b0 2023-08-27T11:24:21.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e4affd700 1 -- 172.21.15.45:0/2134439929 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c004480 con 0x7f5e5415b5b0 2023-08-27T11:24:21.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.231+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e540ce9f0 con 0x7f5e5415b5b0 2023-08-27T11:24:21.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.232+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e540ce290 con 0x7f5e5415b5b0 2023-08-27T11:24:21.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.233+0000 7f5e4affd700 1 -- 172.21.15.45:0/2134439929 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f5e4c0045e0 con 0x7f5e5415b5b0 2023-08-27T11:24:21.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.233+0000 7f5e4affd700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e3806a2e0 0x7f5e3806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.233+0000 7f5e4affd700 1 -- 172.21.15.45:0/2134439929 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f5e4c022070 con 0x7f5e5415b5b0 2023-08-27T11:24:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.234+0000 7f5e5a399700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e3806a2e0 0x7f5e3806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.234+0000 7f5e5a399700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e3806a2e0 0x7f5e3806c7a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e44006b20 tx=0x7f5e44006ab0 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:21.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.238+0000 7f5e4affd700 1 -- 172.21.15.45:0/2134439929 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5e540ce290 con 0x7f5e5415b5b0 2023-08-27T11:24:21.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.465+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e540b1f20 con 0x7f5e3806a2e0 2023-08-27T11:24:21.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.533+0000 7f5e4affd700 1 -- 172.21.15.45:0/2134439929 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5e440096c0 con 0x7f5e3806a2e0 2023-08-27T11:24:21.534 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-08-27T11:24:21.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e3806a2e0 msgr2=0x7f5e3806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e3806a2e0 0x7f5e3806c7a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e44006b20 tx=0x7f5e44006ab0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 msgr2=0x7f5e540cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 0x7f5e540cbc20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5e5406cbb0 tx=0x7f5e4c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 shutdown_connections 2023-08-27T11:24:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e3806a2e0 0x7f5e3806c7a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e5415b5b0 0x7f5e540cbc20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e5406a280 0x7f5e540cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 --2- 172.21.15.45:0/2134439929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e54071400 0x7f5e5414b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 >> 172.21.15.45:0/2134439929 conn(0x7f5e540cc7c0 msgr2=0x7f5e5415a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 shutdown_connections 2023-08-27T11:24:21.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.535+0000 7f5e5c5fd700 1 -- 172.21.15.45:0/2134439929 wait complete. 2023-08-27T11:24:21.548 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-08-27T11:24:21.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-08-27T11:24:21.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.677+0000 7fc5ad9b2700 1 Processor -- start 2023-08-27T11:24:21.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.677+0000 7fc5ad9b2700 1 -- start start 2023-08-27T11:24:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.678+0000 7fc5ad9b2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc5a80b17c0 0x7fc5a80b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.678+0000 7fc5ad9b2700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc5a806ced0 0x7fc5a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.678+0000 7fc5ad9b2700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a806d840 con 0x7fc5a8157be0 2023-08-27T11:24:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.678+0000 7fc5ad9b2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a8157fc0 con 0x7fc5a80b17c0 2023-08-27T11:24:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.678+0000 7fc5ad9b2700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a8071dc0 con 0x7fc5a806ced0 2023-08-27T11:24:21.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.678+0000 7fc5a67fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc5a80b17c0 0x7fc5a80b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.678+0000 7fc5a67fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc5a80b17c0 0x7fc5a80b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54948/0 (socket says 172.21.15.45:54948) 2023-08-27T11:24:21.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.678+0000 7fc5a67fc700 1 -- 172.21.15.45:0/2291505285 learned_addr learned my addr 172.21.15.45:0/2291505285 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:21.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.678+0000 7fc5a77fe700 1 --2- 172.21.15.45:0/2291505285 >> v2:172.21.15.135:3300/0 conn(0x7fc5a806ced0 0x7fc5a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5a67fc700 1 -- 172.21.15.45:0/2291505285 >> v2:172.21.15.135:3300/0 conn(0x7fc5a806ced0 msgr2=0x7fc5a806d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:21.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5a67fc700 1 --2- 172.21.15.45:0/2291505285 >> v2:172.21.15.135:3300/0 conn(0x7fc5a806ced0 0x7fc5a806d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5a67fc700 1 -- 172.21.15.45:0/2291505285 >> v1:172.21.15.135:6789/0 conn(0x7fc5a8157be0 legacy=0x7fc5a80716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5a67fc700 1 -- 172.21.15.45:0/2291505285 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a815b460 con 0x7fc5a80b17c0 2023-08-27T11:24:21.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5a67fc700 1 --2- 172.21.15.45:0/2291505285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 0x7fc5a80b1c70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc5a815b650 tx=0x7fc598009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4644f35398ab051 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:21.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5a5ffb700 1 -- 172.21.15.45:0/2291505285 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598004230 con 0x7fc5a80b17c0 2023-08-27T11:24:21.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5a5ffb700 1 -- 172.21.15.45:0/2291505285 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598004390 con 0x7fc5a80b17c0 2023-08-27T11:24:21.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5a5ffb700 1 -- 172.21.15.45:0/2291505285 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980036a0 con 0x7fc5a80b17c0 2023-08-27T11:24:21.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2291505285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 msgr2=0x7fc5a80b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5ad9b2700 1 --2- 172.21.15.45:0/2291505285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 0x7fc5a80b1c70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc5a815b650 tx=0x7fc598009bd0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2291505285 shutdown_connections 2023-08-27T11:24:21.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5ad9b2700 1 --2- 172.21.15.45:0/2291505285 >> v2:172.21.15.135:3300/0 conn(0x7fc5a806ced0 0x7fc5a806d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.679+0000 7fc5ad9b2700 1 --2- 172.21.15.45:0/2291505285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 0x7fc5a80b1c70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.680+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2291505285 >> 172.21.15.45:0/2291505285 conn(0x7fc5a80cc7c0 msgr2=0x7fc5a80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:21.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.680+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2291505285 shutdown_connections 2023-08-27T11:24:21.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.680+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2291505285 wait complete. 2023-08-27T11:24:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.681+0000 7fc5ad9b2700 1 Processor -- start 2023-08-27T11:24:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.681+0000 7fc5ad9b2700 1 -- start start 2023-08-27T11:24:21.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5ad9b2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5a806ced0 0x7fc5a8144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5ad9b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 0x7fc5a80c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5ad9b2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a8157be0 0x7fc5a80c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5ad9b2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80c7e40 con 0x7fc5a80b17c0 2023-08-27T11:24:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5ad9b2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80c8120 con 0x7fc5a8157be0 2023-08-27T11:24:21.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5ad9b2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80c8400 con 0x7fc5a806ced0 2023-08-27T11:24:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 0x7fc5a80c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 0x7fc5a80c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54954/0 (socket says 172.21.15.45:54954) 2023-08-27T11:24:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a67fc700 1 -- 172.21.15.45:0/2350122971 learned_addr learned my addr 172.21.15.45:0/2350122971 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:21.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a77fe700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a8157be0 0x7fc5a80c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a6ffd700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5a806ced0 0x7fc5a8144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a67fc700 1 -- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5a806ced0 msgr2=0x7fc5a8144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a67fc700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5a806ced0 0x7fc5a8144e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a67fc700 1 -- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a8157be0 msgr2=0x7fc5a80c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a67fc700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a8157be0 0x7fc5a80c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a67fc700 1 -- 172.21.15.45:0/2350122971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc598009550 con 0x7fc5a80b17c0 2023-08-27T11:24:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5a67fc700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 0x7fc5a80c70d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc598009c10 tx=0x7fc598004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc58ffff700 1 -- 172.21.15.45:0/2350122971 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598003d60 con 0x7fc5a80b17c0 2023-08-27T11:24:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc58ffff700 1 -- 172.21.15.45:0/2350122971 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59800b670 con 0x7fc5a80b17c0 2023-08-27T11:24:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc58ffff700 1 -- 172.21.15.45:0/2350122971 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59800ba00 con 0x7fc5a80b17c0 2023-08-27T11:24:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.682+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a80c86e0 con 0x7fc5a80b17c0 2023-08-27T11:24:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.683+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a80c7b50 con 0x7fc5a80b17c0 2023-08-27T11:24:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.683+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a80c8c00 con 0x7fc5a80b17c0 2023-08-27T11:24:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.684+0000 7fc58ffff700 1 -- 172.21.15.45:0/2350122971 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fc598021070 con 0x7fc5a80b17c0 2023-08-27T11:24:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.684+0000 7fc58ffff700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc59406a2e0 0x7fc59406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.684+0000 7fc58ffff700 1 -- 172.21.15.45:0/2350122971 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fc59808a420 con 0x7fc5a80b17c0 2023-08-27T11:24:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.685+0000 7fc5a6ffd700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc59406a2e0 0x7fc59406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.685+0000 7fc5a6ffd700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc59406a2e0 0x7fc59406c7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0016c0 tx=0x7fc59c009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.689+0000 7fc58ffff700 1 -- 172.21.15.45:0/2350122971 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc5a80c8c00 con 0x7fc5a80b17c0 2023-08-27T11:24:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.916+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5a80c8290 con 0x7fc59406a2e0 2023-08-27T11:24:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.979+0000 7fc58ffff700 1 -- 172.21.15.45:0/2350122971 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc5a80c8290 con 0x7fc59406a2e0 2023-08-27T11:24:21.983 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-08-27T11:24:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc59406a2e0 msgr2=0x7fc59406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc59406a2e0 0x7fc59406c7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0016c0 tx=0x7fc59c009040 comp rx=0 tx=0).stop 2023-08-27T11:24:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 msgr2=0x7fc5a80c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:21.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 0x7fc5a80c70d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc598009c10 tx=0x7fc598004750 comp rx=0 tx=0).stop 2023-08-27T11:24:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 shutdown_connections 2023-08-27T11:24:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc59406a2e0 0x7fc59406c7a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5a8157be0 0x7fc5a80c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5a806ced0 0x7fc5a8144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 --2- 172.21.15.45:0/2350122971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a80b17c0 0x7fc5a80c70d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 >> 172.21.15.45:0/2350122971 conn(0x7fc5a80cc7c0 msgr2=0x7fc5a80caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 shutdown_connections 2023-08-27T11:24:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:21.982+0000 7fc5ad9b2700 1 -- 172.21.15.45:0/2350122971 wait complete. 2023-08-27T11:24:21.995 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-08-27T11:24:21.996 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-08-27T11:24:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.127+0000 7f9355151700 1 Processor -- start 2023-08-27T11:24:22.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.127+0000 7f9355151700 1 -- start start 2023-08-27T11:24:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f9355151700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9350157be0 0x7f93500716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f9355151700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f935006d5e0 0x7f93500b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f9355151700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935015b460 con 0x7f93500b17c0 2023-08-27T11:24:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f9355151700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9350157fc0 con 0x7f935006d5e0 2023-08-27T11:24:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f9355151700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b2280 con 0x7f9350157be0 2023-08-27T11:24:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f934f59e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f935006d5e0 0x7f93500b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f934f59e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f935006d5e0 0x7f93500b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54962/0 (socket says 172.21.15.45:54962) 2023-08-27T11:24:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f934f59e700 1 -- 172.21.15.45:0/1753951908 learned_addr learned my addr 172.21.15.45:0/1753951908 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f934ed9d700 1 --2- 172.21.15.45:0/1753951908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9350157be0 0x7f93500716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f934f59e700 1 -- 172.21.15.45:0/1753951908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9350157be0 msgr2=0x7f93500716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f934f59e700 1 --2- 172.21.15.45:0/1753951908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9350157be0 0x7f93500716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f934f59e700 1 -- 172.21.15.45:0/1753951908 >> v1:172.21.15.135:6789/0 conn(0x7f93500b17c0 legacy=0x7f935006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.128+0000 7f934f59e700 1 -- 172.21.15.45:0/1753951908 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500b2560 con 0x7f935006d5e0 2023-08-27T11:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f934f59e700 1 --2- 172.21.15.45:0/1753951908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f935006d5e0 0x7f93500b1c40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9348009e50 tx=0x7f934800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4315de0b96a16a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f934dd9b700 1 -- 172.21.15.45:0/1753951908 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934800e070 con 0x7f935006d5e0 2023-08-27T11:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f934dd9b700 1 -- 172.21.15.45:0/1753951908 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934800b9c0 con 0x7f935006d5e0 2023-08-27T11:24:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f934dd9b700 1 -- 172.21.15.45:0/1753951908 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934800bd70 con 0x7f935006d5e0 2023-08-27T11:24:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f9355151700 1 -- 172.21.15.45:0/1753951908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f935006d5e0 msgr2=0x7f93500b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f9355151700 1 --2- 172.21.15.45:0/1753951908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f935006d5e0 0x7f93500b1c40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9348009e50 tx=0x7f934800b330 comp rx=0 tx=0).stop 2023-08-27T11:24:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f9355151700 1 -- 172.21.15.45:0/1753951908 shutdown_connections 2023-08-27T11:24:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f9355151700 1 --2- 172.21.15.45:0/1753951908 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9350157be0 0x7f93500716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f9355151700 1 --2- 172.21.15.45:0/1753951908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f935006d5e0 0x7f93500b1c40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f9355151700 1 -- 172.21.15.45:0/1753951908 >> 172.21.15.45:0/1753951908 conn(0x7f93500cc7c0 msgr2=0x7f93500ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f9355151700 1 -- 172.21.15.45:0/1753951908 shutdown_connections 2023-08-27T11:24:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.129+0000 7f9355151700 1 -- 172.21.15.45:0/1753951908 wait complete. 2023-08-27T11:24:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.131+0000 7f9355151700 1 Processor -- start 2023-08-27T11:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.131+0000 7f9355151700 1 -- start start 2023-08-27T11:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f9355151700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f935006d5e0 0x7f935014db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f9355151700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93500b17c0 0x7f935015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f9355151700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9350157be0 0x7f935006a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f9355151700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500b4670 con 0x7f9350157be0 2023-08-27T11:24:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f9355151700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500acc50 con 0x7f93500b17c0 2023-08-27T11:24:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f9355151700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500ac950 con 0x7f935006d5e0 2023-08-27T11:24:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934f59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9350157be0 0x7f935006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934f59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9350157be0 0x7f935006a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54968/0 (socket says 172.21.15.45:54968) 2023-08-27T11:24:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934f59e700 1 -- 172.21.15.45:0/2153668525 learned_addr learned my addr 172.21.15.45:0/2153668525 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934ed9d700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f935006d5e0 0x7f935014db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934e59c700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93500b17c0 0x7f935015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934f59e700 1 -- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f935006d5e0 msgr2=0x7f935014db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934f59e700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f935006d5e0 0x7f935014db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934f59e700 1 -- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93500b17c0 msgr2=0x7f935015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934f59e700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93500b17c0 0x7f935015cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934f59e700 1 -- 172.21.15.45:0/2153668525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9348009b30 con 0x7f9350157be0 2023-08-27T11:24:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934e59c700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93500b17c0 0x7f935015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934ed9d700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f935006d5e0 0x7f935014db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f934f59e700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9350157be0 0x7f935006a280 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f935006c3e0 tx=0x7f9348004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.132+0000 7f93377fe700 1 -- 172.21.15.45:0/2153668525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348004960 con 0x7f9350157be0 2023-08-27T11:24:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.133+0000 7f93377fe700 1 -- 172.21.15.45:0/2153668525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93480099b0 con 0x7f9350157be0 2023-08-27T11:24:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.133+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93500ae390 con 0x7f9350157be0 2023-08-27T11:24:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.133+0000 7f93377fe700 1 -- 172.21.15.45:0/2153668525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934801c680 con 0x7f9350157be0 2023-08-27T11:24:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.133+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93500b4ba0 con 0x7f9350157be0 2023-08-27T11:24:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.133+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93500aca90 con 0x7f9350157be0 2023-08-27T11:24:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.134+0000 7f93377fe700 1 -- 172.21.15.45:0/2153668525 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f934801c7e0 con 0x7f9350157be0 2023-08-27T11:24:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.134+0000 7f93377fe700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f933c06a2e0 0x7f933c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.134+0000 7f93377fe700 1 -- 172.21.15.45:0/2153668525 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f934808a950 con 0x7f9350157be0 2023-08-27T11:24:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.135+0000 7f934ed9d700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f933c06a2e0 0x7f933c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.135+0000 7f934ed9d700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f933c06a2e0 0x7f933c06c7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93500c69c0 tx=0x7f9344008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.139+0000 7f93377fe700 1 -- 172.21.15.45:0/2153668525 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f93500aca90 con 0x7f9350157be0 2023-08-27T11:24:22.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.371+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9350071190 con 0x7f933c06a2e0 2023-08-27T11:24:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.435+0000 7f93377fe700 1 -- 172.21.15.45:0/2153668525 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9344007660 con 0x7f933c06a2e0 2023-08-27T11:24:22.436 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-08-27T11:24:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f933c06a2e0 msgr2=0x7f933c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f933c06a2e0 0x7f933c06c7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93500c69c0 tx=0x7f9344008040 comp rx=0 tx=0).stop 2023-08-27T11:24:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9350157be0 msgr2=0x7f935006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9350157be0 0x7f935006a280 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f935006c3e0 tx=0x7f9348004ad0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 shutdown_connections 2023-08-27T11:24:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f933c06a2e0 0x7f933c06c7a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9350157be0 0x7f935006a280 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f935006d5e0 0x7f935014db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 --2- 172.21.15.45:0/2153668525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93500b17c0 0x7f935015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 >> 172.21.15.45:0/2153668525 conn(0x7f93500cc7c0 msgr2=0x7f93500cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 shutdown_connections 2023-08-27T11:24:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.437+0000 7f9355151700 1 -- 172.21.15.45:0/2153668525 wait complete. 2023-08-27T11:24:22.450 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-08-27T11:24:22.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-08-27T11:24:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.580+0000 7f00b64dc700 1 Processor -- start 2023-08-27T11:24:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.580+0000 7f00b64dc700 1 -- start start 2023-08-27T11:24:22.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.581+0000 7f00b64dc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f00b00aea00 0x7f00b0071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.581+0000 7f00b64dc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b00b3220 con 0x7f00b015a220 2023-08-27T11:24:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.581+0000 7f00b64dc700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b00b2c60 con 0x7f00b0071eb0 2023-08-27T11:24:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.581+0000 7f00b64dc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b0072340 con 0x7f00b00aea00 2023-08-27T11:24:22.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.581+0000 7f00affff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f00b015a220 0x7f00b00cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41748/0 (socket says 172.21.15.45:41748) 2023-08-27T11:24:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.581+0000 7f00affff700 1 -- 172.21.15.45:0/1911690483 learned_addr learned my addr 172.21.15.45:0/1911690483 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.581+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810810525 0 0) 0x7f00b00b3220 con 0x7f00b015a220 2023-08-27T11:24:22.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.581+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00980035b0 con 0x7f00b015a220 2023-08-27T11:24:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.581+0000 7f00af7fe700 1 --2- 172.21.15.45:0/1911690483 >> v2:172.21.15.135:3300/0 conn(0x7f00b00aea00 0x7f00b0071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 169971447 0 0) 0x7f00980035b0 con 0x7f00b015a220 2023-08-27T11:24:22.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 >> v2:172.21.15.135:3300/0 conn(0x7f00b00aea00 msgr2=0x7f00b0071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:22.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 --2- 172.21.15.45:0/1911690483 >> v2:172.21.15.135:3300/0 conn(0x7f00b00aea00 0x7f00b0071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 >> v1:172.21.15.135:6789/0 conn(0x7f00b0071eb0 legacy=0x7f00b006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00b2f40 con 0x7f00b015a220 2023-08-27T11:24:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f00a0001670 con 0x7f00b015a220 2023-08-27T11:24:22.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 >> v1:172.21.15.45:6789/0 conn(0x7f00b015a220 legacy=0x7f00b00cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0098003a00 0x7f0098003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f0098004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0098005210 0x7f009800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b0072340 con 0x7f0098004850 2023-08-27T11:24:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009800c0b0 con 0x7f0098005210 2023-08-27T11:24:22.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009800c270 con 0x7f0098003a00 2023-08-27T11:24:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00af7fe700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f0098004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00b4a79700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0098005210 0x7f009800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00affff700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0098003a00 0x7f0098003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00af7fe700 1 -- 172.21.15.45:0/1911690483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0098003a00 msgr2=0x7f0098003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00af7fe700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0098003a00 0x7f0098003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00af7fe700 1 -- 172.21.15.45:0/1911690483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0098005210 msgr2=0x7f009800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00af7fe700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0098005210 0x7f009800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00b4a79700 1 -- 172.21.15.45:0/1911690483 reap_dead start 2023-08-27T11:24:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00af7fe700 1 -- 172.21.15.45:0/1911690483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00980035b0 con 0x7f0098004850 2023-08-27T11:24:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00affff700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0098003a00 0x7f0098003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:22.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.582+0000 7f00af7fe700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f0098004cd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009b30 tx=0x7f0098005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343b91cf94d60d5b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:22.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a400e6c0 con 0x7f0098004850 2023-08-27T11:24:22.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a400e820 con 0x7f0098004850 2023-08-27T11:24:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00aeffd700 1 -- 172.21.15.45:0/1911690483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a400eb90 con 0x7f0098004850 2023-08-27T11:24:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00b64dc700 1 -- 172.21.15.45:0/1911690483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 msgr2=0x7f0098004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00b64dc700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f0098004cd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009b30 tx=0x7f0098005880 comp rx=0 tx=0).stop 2023-08-27T11:24:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00b64dc700 1 -- 172.21.15.45:0/1911690483 shutdown_connections 2023-08-27T11:24:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00b64dc700 1 --2- 172.21.15.45:0/1911690483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f0098004cd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00b64dc700 1 -- 172.21.15.45:0/1911690483 >> 172.21.15.45:0/1911690483 conn(0x7f00b00b4650 msgr2=0x7f00b00b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00b64dc700 1 -- 172.21.15.45:0/1911690483 shutdown_connections 2023-08-27T11:24:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.583+0000 7f00b64dc700 1 -- 172.21.15.45:0/1911690483 wait complete. 2023-08-27T11:24:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.584+0000 7f00b64dc700 1 Processor -- start 2023-08-27T11:24:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00b64dc700 1 -- start start 2023-08-27T11:24:22.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00b64dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f00b015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00b64dc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b015c8f0 0x7f00b015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00b64dc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00b015adb0 0x7f00b0150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00b64dc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b015b310 con 0x7f0098004850 2023-08-27T11:24:22.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00b64dc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b015b5f0 con 0x7f00b015c8f0 2023-08-27T11:24:22.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00b64dc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b015b8d0 con 0x7f00b015adb0 2023-08-27T11:24:22.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00affff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f00b015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00affff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f00b015c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54984/0 (socket says 172.21.15.45:54984) 2023-08-27T11:24:22.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00affff700 1 -- 172.21.15.45:0/3363747698 learned_addr learned my addr 172.21.15.45:0/3363747698 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00af7fe700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b015c8f0 0x7f00b015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.585+0000 7f00b4a79700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00b015adb0 0x7f00b0150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00affff700 1 -- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00b015adb0 msgr2=0x7f00b0150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00affff700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00b015adb0 0x7f00b0150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00affff700 1 -- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b015c8f0 msgr2=0x7f00b015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00affff700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b015c8f0 0x7f00b015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00affff700 1 -- 172.21.15.45:0/3363747698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b015bbb0 con 0x7f0098004850 2023-08-27T11:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00af7fe700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b015c8f0 0x7f00b015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00affff700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f00b015c3b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f00a0001c90 tx=0x7f00a000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00acff9700 1 -- 172.21.15.45:0/3363747698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000c9b0 con 0x7f0098004850 2023-08-27T11:24:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00acff9700 1 -- 172.21.15.45:0/3363747698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000cb10 con 0x7f0098004850 2023-08-27T11:24:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b015be90 con 0x7f0098004850 2023-08-27T11:24:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00acff9700 1 -- 172.21.15.45:0/3363747698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000ce80 con 0x7f0098004850 2023-08-27T11:24:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.586+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b01f2e30 con 0x7f0098004850 2023-08-27T11:24:22.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.587+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b0159720 con 0x7f0098004850 2023-08-27T11:24:22.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.588+0000 7f00acff9700 1 -- 172.21.15.45:0/3363747698 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f00a0012450 con 0x7f0098004850 2023-08-27T11:24:22.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.588+0000 7f00acff9700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f009c06a2e0 0x7f009c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:22.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.588+0000 7f00acff9700 1 -- 172.21.15.45:0/3363747698 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f00a008a9b0 con 0x7f0098004850 2023-08-27T11:24:22.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.588+0000 7f00af7fe700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f009c06a2e0 0x7f009c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:22.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.589+0000 7f00af7fe700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f009c06a2e0 0x7f009c06c7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00a40090b0 tx=0x7f00a401e040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:22.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.592+0000 7f00acff9700 1 -- 172.21.15.45:0/3363747698 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f00a0055c30 con 0x7f0098004850 2023-08-27T11:24:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.819+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f00b015b760 con 0x7f009c06a2e0 2023-08-27T11:24:22.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.883+0000 7f00acff9700 1 -- 172.21.15.45:0/3363747698 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f00b015b760 con 0x7f009c06a2e0 2023-08-27T11:24:22.884 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-08-27T11:24:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f009c06a2e0 msgr2=0x7f009c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f009c06a2e0 0x7f009c06c7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00a40090b0 tx=0x7f00a401e040 comp rx=0 tx=0).stop 2023-08-27T11:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 msgr2=0x7f00b015c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f00b015c3b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f00a0001c90 tx=0x7f00a000edd0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 shutdown_connections 2023-08-27T11:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f009c06a2e0 0x7f009c06c7a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0098004850 0x7f00b015c3b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00b015adb0 0x7f00b0150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 --2- 172.21.15.45:0/3363747698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b015c8f0 0x7f00b015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 >> 172.21.15.45:0/3363747698 conn(0x7f00b00b4650 msgr2=0x7f00b00cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 shutdown_connections 2023-08-27T11:24:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:22.885+0000 7f00b64dc700 1 -- 172.21.15.45:0/3363747698 wait complete. 2023-08-27T11:24:22.898 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-08-27T11:24:22.898 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-08-27T11:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.029+0000 7f39b1b58700 1 Processor -- start 2023-08-27T11:24:23.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.029+0000 7f39b1b58700 1 -- start start 2023-08-27T11:24:23.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39b1b58700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f39ac15b2e0 0x7f39ac0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39b1b58700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ac0cb9b0 con 0x7f39ac0cd540 2023-08-27T11:24:23.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39b1b58700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ac0ce2e0 con 0x7f39ac0cf950 2023-08-27T11:24:23.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39b1b58700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac0cc230 con 0x7f39ac15b2e0 2023-08-27T11:24:23.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39ab7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f39ac15b2e0 0x7f39ac0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39aaffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f39ac0cd540 0x7f39ac0cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41770/0 (socket says 172.21.15.45:41770) 2023-08-27T11:24:23.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39aaffd700 1 -- 172.21.15.45:0/2983071943 learned_addr learned my addr 172.21.15.45:0/2983071943 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:23.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39ab7fe700 1 -- 172.21.15.45:0/2983071943 >> v1:172.21.15.135:6789/0 conn(0x7f39ac0cf950 legacy=0x7f39ac06da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:23.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39ab7fe700 1 -- 172.21.15.45:0/2983071943 >> v1:172.21.15.45:6789/0 conn(0x7f39ac0cd540 legacy=0x7f39ac0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39ab7fe700 1 -- 172.21.15.45:0/2983071943 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac0ce000 con 0x7f39ac15b2e0 2023-08-27T11:24:23.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2983071943 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151482955 0 0) 0x7f39ac0cb9b0 con 0x7f39ac0cd540 2023-08-27T11:24:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.030+0000 7f39ab7fe700 1 --2- 172.21.15.45:0/2983071943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 0x7f39ac0cc6c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f399c009840 tx=0x7f399c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e532b97d35ac94a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2983071943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c004230 con 0x7f39ac15b2e0 2023-08-27T11:24:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2983071943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c004390 con 0x7f39ac15b2e0 2023-08-27T11:24:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39aa7fc700 1 -- 172.21.15.45:0/2983071943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c0036a0 con 0x7f39ac15b2e0 2023-08-27T11:24:23.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39b1b58700 1 -- 172.21.15.45:0/2983071943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 msgr2=0x7f39ac0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39b1b58700 1 --2- 172.21.15.45:0/2983071943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 0x7f39ac0cc6c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f399c009840 tx=0x7f399c009b50 comp rx=0 tx=0).stop 2023-08-27T11:24:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39b1b58700 1 -- 172.21.15.45:0/2983071943 shutdown_connections 2023-08-27T11:24:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39b1b58700 1 --2- 172.21.15.45:0/2983071943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 0x7f39ac0cc6c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39b1b58700 1 -- 172.21.15.45:0/2983071943 >> 172.21.15.45:0/2983071943 conn(0x7f39ac09a1a0 msgr2=0x7f39ac0d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:23.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39b1b58700 1 -- 172.21.15.45:0/2983071943 shutdown_connections 2023-08-27T11:24:23.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.031+0000 7f39b1b58700 1 -- 172.21.15.45:0/2983071943 wait complete. 2023-08-27T11:24:23.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.032+0000 7f39b1b58700 1 Processor -- start 2023-08-27T11:24:23.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39b1b58700 1 -- start start 2023-08-27T11:24:23.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39b1b58700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac0cd540 0x7f39ac0b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39b1b58700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac0cf950 0x7f39ac0b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39b1b58700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 0x7f39ac150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39b1b58700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b2230 con 0x7f39ac15b2e0 2023-08-27T11:24:23.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39b1b58700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b15b0 con 0x7f39ac0cd540 2023-08-27T11:24:23.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39b1b58700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b1890 con 0x7f39ac0cf950 2023-08-27T11:24:23.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39abfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 0x7f39ac150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39abfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 0x7f39ac150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55002/0 (socket says 172.21.15.45:55002) 2023-08-27T11:24:23.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39abfff700 1 -- 172.21.15.45:0/2812817573 learned_addr learned my addr 172.21.15.45:0/2812817573 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:23.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.033+0000 7f39ab7fe700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac0cd540 0x7f39ac0b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39aaffd700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac0cf950 0x7f39ac0b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39abfff700 1 -- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac0cf950 msgr2=0x7f39ac0b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39abfff700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac0cf950 0x7f39ac0b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39abfff700 1 -- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac0cd540 msgr2=0x7f39ac0b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39abfff700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac0cd540 0x7f39ac0b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39abfff700 1 -- 172.21.15.45:0/2812817573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0094a0 con 0x7f39ac15b2e0 2023-08-27T11:24:23.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39abfff700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 0x7f39ac150310 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f39a400cc10 tx=0x7f39a400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:23.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39a3fff700 1 -- 172.21.15.45:0/2812817573 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4004b90 con 0x7f39ac15b2e0 2023-08-27T11:24:23.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39a3fff700 1 -- 172.21.15.45:0/2812817573 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a4004cf0 con 0x7f39ac15b2e0 2023-08-27T11:24:23.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac0b1b70 con 0x7f39ac15b2e0 2023-08-27T11:24:23.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39a3fff700 1 -- 172.21.15.45:0/2812817573 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a400aae0 con 0x7f39ac15b2e0 2023-08-27T11:24:23.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.034+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac0b12a0 con 0x7f39ac15b2e0 2023-08-27T11:24:23.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.035+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac0b6d60 con 0x7f39ac15b2e0 2023-08-27T11:24:23.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.036+0000 7f39a3fff700 1 -- 172.21.15.45:0/2812817573 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f39a400ac40 con 0x7f39ac15b2e0 2023-08-27T11:24:23.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.036+0000 7f39a3fff700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f398c06a2e0 0x7f398c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.036+0000 7f39a3fff700 1 -- 172.21.15.45:0/2812817573 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f39a4088440 con 0x7f39ac15b2e0 2023-08-27T11:24:23.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.036+0000 7f39ab7fe700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f398c06a2e0 0x7f398c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.037+0000 7f39ab7fe700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f398c06a2e0 0x7f398c06c7a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f399c0090b0 tx=0x7f399c01c040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:23.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.041+0000 7f39a3fff700 1 -- 172.21.15.45:0/2812817573 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f39a40536c0 con 0x7f39ac15b2e0 2023-08-27T11:24:23.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.269+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f39ac0cbec0 con 0x7f398c06a2e0 2023-08-27T11:24:23.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.334+0000 7f39a3fff700 1 -- 172.21.15.45:0/2812817573 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f399c004130 con 0x7f398c06a2e0 2023-08-27T11:24:23.336 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-08-27T11:24:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f398c06a2e0 msgr2=0x7f398c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f398c06a2e0 0x7f398c06c7a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f399c0090b0 tx=0x7f399c01c040 comp rx=0 tx=0).stop 2023-08-27T11:24:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 msgr2=0x7f39ac150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 0x7f39ac150310 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f39a400cc10 tx=0x7f39a400cf20 comp rx=0 tx=0).stop 2023-08-27T11:24:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 shutdown_connections 2023-08-27T11:24:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f398c06a2e0 0x7f398c06c7a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39ac15b2e0 0x7f39ac150310 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39ac0cd540 0x7f39ac0b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 --2- 172.21.15.45:0/2812817573 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f39ac0cf950 0x7f39ac0b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 >> 172.21.15.45:0/2812817573 conn(0x7f39ac09a1a0 msgr2=0x7f39ac06f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 shutdown_connections 2023-08-27T11:24:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.337+0000 7f39b1b58700 1 -- 172.21.15.45:0/2812817573 wait complete. 2023-08-27T11:24:23.349 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-08-27T11:24:23.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-08-27T11:24:23.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.479+0000 7f178b251700 1 Processor -- start 2023-08-27T11:24:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.479+0000 7f178b251700 1 -- start start 2023-08-27T11:24:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.480+0000 7f178b251700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f178415b5e0 0x7f178406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.480+0000 7f178b251700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178406f450 0x7f17840cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.480+0000 7f178b251700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17840cb060 con 0x7f178415a700 2023-08-27T11:24:23.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.480+0000 7f178b251700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178406e320 con 0x7f178415b5e0 2023-08-27T11:24:23.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.480+0000 7f178b251700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178406f8c0 con 0x7f178406f450 2023-08-27T11:24:23.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.480+0000 7f1783fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f178415b5e0 0x7f178406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.480+0000 7f1783fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f178415b5e0 0x7f178406ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55008/0 (socket says 172.21.15.45:55008) 2023-08-27T11:24:23.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.480+0000 7f1783fff700 1 -- 172.21.15.45:0/843125097 learned_addr learned my addr 172.21.15.45:0/843125097 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.480+0000 7f17897ee700 1 --2- 172.21.15.45:0/843125097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178406f450 0x7f17840cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f1783fff700 1 -- 172.21.15.45:0/843125097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178406f450 msgr2=0x7f17840cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f1783fff700 1 --2- 172.21.15.45:0/843125097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178406f450 0x7f17840cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f1783fff700 1 -- 172.21.15.45:0/843125097 >> v1:172.21.15.135:6789/0 conn(0x7f178415a700 legacy=0x7f178415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f1783fff700 1 -- 172.21.15.45:0/843125097 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178415d7d0 con 0x7f178415b5e0 2023-08-27T11:24:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f1783fff700 1 --2- 172.21.15.45:0/843125097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178415b5e0 0x7f178406ef10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f17700098f0 tx=0x7f1770009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5040c84059684bc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f17837fe700 1 -- 172.21.15.45:0/843125097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177000b8a0 con 0x7f178415b5e0 2023-08-27T11:24:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f17837fe700 1 -- 172.21.15.45:0/843125097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177000ba00 con 0x7f178415b5e0 2023-08-27T11:24:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f17837fe700 1 -- 172.21.15.45:0/843125097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177000bd90 con 0x7f178415b5e0 2023-08-27T11:24:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f178b251700 1 -- 172.21.15.45:0/843125097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178415b5e0 msgr2=0x7f178406ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.481+0000 7f178b251700 1 --2- 172.21.15.45:0/843125097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178415b5e0 0x7f178406ef10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f17700098f0 tx=0x7f1770009c00 comp rx=0 tx=0).stop 2023-08-27T11:24:23.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.482+0000 7f178b251700 1 -- 172.21.15.45:0/843125097 shutdown_connections 2023-08-27T11:24:23.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.482+0000 7f178b251700 1 --2- 172.21.15.45:0/843125097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178406f450 0x7f17840cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.482+0000 7f178b251700 1 --2- 172.21.15.45:0/843125097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178415b5e0 0x7f178406ef10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.482+0000 7f178b251700 1 -- 172.21.15.45:0/843125097 >> 172.21.15.45:0/843125097 conn(0x7f17840cc7e0 msgr2=0x7f17840ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:23.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.482+0000 7f178b251700 1 -- 172.21.15.45:0/843125097 shutdown_connections 2023-08-27T11:24:23.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.482+0000 7f178b251700 1 -- 172.21.15.45:0/843125097 wait complete. 2023-08-27T11:24:23.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.483+0000 7f178b251700 1 Processor -- start 2023-08-27T11:24:23.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.483+0000 7f178b251700 1 -- start start 2023-08-27T11:24:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f178b251700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178406f450 0x7f17841522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f178b251700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178415a700 0x7f17840c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f178b251700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178415b5e0 0x7f1784154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f178b251700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17841527f0 con 0x7f178406f450 2023-08-27T11:24:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f178b251700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840c8b30 con 0x7f178415b5e0 2023-08-27T11:24:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f178b251700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1784158ad0 con 0x7f178415a700 2023-08-27T11:24:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f1788fed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178406f450 0x7f17841522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f1788fed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178406f450 0x7f17841522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55012/0 (socket says 172.21.15.45:55012) 2023-08-27T11:24:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f1788fed700 1 -- 172.21.15.45:0/2749136388 learned_addr learned my addr 172.21.15.45:0/2749136388 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f17897ee700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178415b5e0 0x7f1784154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f1788fed700 1 -- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178415a700 msgr2=0x7f17840c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f1788fed700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178415a700 0x7f17840c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f1788fed700 1 -- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178415b5e0 msgr2=0x7f1784154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f1788fed700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178415b5e0 0x7f1784154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f1788fed700 1 -- 172.21.15.45:0/2749136388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1770009550 con 0x7f178406f450 2023-08-27T11:24:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f1788fed700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178406f450 0x7f17841522b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f177800c450 tx=0x7f177800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f17817fa700 1 -- 172.21.15.45:0/2749136388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780052d0 con 0x7f178406f450 2023-08-27T11:24:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f17817fa700 1 -- 172.21.15.45:0/2749136388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1778005430 con 0x7f178406f450 2023-08-27T11:24:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f17817fa700 1 -- 172.21.15.45:0/2749136388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780038c0 con 0x7f178406f450 2023-08-27T11:24:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1784158db0 con 0x7f178406f450 2023-08-27T11:24:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.484+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1784159090 con 0x7f178406f450 2023-08-27T11:24:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.485+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17840c8990 con 0x7f178406f450 2023-08-27T11:24:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.486+0000 7f17817fa700 1 -- 172.21.15.45:0/2749136388 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f1778003a20 con 0x7f178406f450 2023-08-27T11:24:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.486+0000 7f17817fa700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f176806a2e0 0x7f176806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.486+0000 7f17817fa700 1 -- 172.21.15.45:0/2749136388 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f1778087ef0 con 0x7f178406f450 2023-08-27T11:24:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.487+0000 7f1783fff700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f176806a2e0 0x7f176806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.487+0000 7f1783fff700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f176806a2e0 0x7f176806c7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1784157c60 tx=0x7f1770010000 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:23.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.491+0000 7f17817fa700 1 -- 172.21.15.45:0/2749136388 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17840c8990 con 0x7f178406f450 2023-08-27T11:24:23.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.720+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f17840ca730 con 0x7f176806a2e0 2023-08-27T11:24:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.782+0000 7f17817fa700 1 -- 172.21.15.45:0/2749136388 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f177001f070 con 0x7f176806a2e0 2023-08-27T11:24:23.783 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-08-27T11:24:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f176806a2e0 msgr2=0x7f176806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f176806a2e0 0x7f176806c7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1784157c60 tx=0x7f1770010000 comp rx=0 tx=0).stop 2023-08-27T11:24:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178406f450 msgr2=0x7f17841522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178406f450 0x7f17841522b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f177800c450 tx=0x7f177800c810 comp rx=0 tx=0).stop 2023-08-27T11:24:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 shutdown_connections 2023-08-27T11:24:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f176806a2e0 0x7f176806c7a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178406f450 0x7f17841522b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178415b5e0 0x7f1784154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 --2- 172.21.15.45:0/2749136388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178415a700 0x7f17840c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.784+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 >> 172.21.15.45:0/2749136388 conn(0x7f17840cc7e0 msgr2=0x7f17840af920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.785+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 shutdown_connections 2023-08-27T11:24:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.785+0000 7f178b251700 1 -- 172.21.15.45:0/2749136388 wait complete. 2023-08-27T11:24:23.797 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-08-27T11:24:23.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-08-27T11:24:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.927+0000 7fac074bf700 1 Processor -- start 2023-08-27T11:24:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.927+0000 7fac074bf700 1 -- start start 2023-08-27T11:24:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.928+0000 7fac074bf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac0006a990 0x7fac000b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.928+0000 7fac074bf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0006b160 con 0x7fac0015d340 2023-08-27T11:24:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.928+0000 7fac074bf700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac000cdf90 con 0x7fac00160d00 2023-08-27T11:24:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.928+0000 7fac074bf700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac00072010 con 0x7fac0006a990 2023-08-27T11:24:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.928+0000 7fac05a5c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac0006a990 0x7fac000b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.928+0000 7fac0525b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fac0015d340 0x7fac000cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41796/0 (socket says 172.21.15.45:41796) 2023-08-27T11:24:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.928+0000 7fac0525b700 1 -- 172.21.15.45:0/3580450302 learned_addr learned my addr 172.21.15.45:0/3580450302 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.928+0000 7fac05a5c700 1 -- 172.21.15.45:0/3580450302 >> v1:172.21.15.135:6789/0 conn(0x7fac00160d00 legacy=0x7fac0006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.928+0000 7fac05a5c700 1 -- 172.21.15.45:0/3580450302 >> v1:172.21.15.45:6789/0 conn(0x7fac0015d340 legacy=0x7fac000cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fac05a5c700 1 -- 172.21.15.45:0/3580450302 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac000af590 con 0x7fac0006a990 2023-08-27T11:24:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fac05a5c700 1 --2- 172.21.15.45:0/3580450302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0006a990 0x7fac000b18a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fac0006b2f0 tx=0x7fabf800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1702d7e51f91b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fabf7fff700 1 -- 172.21.15.45:0/3580450302 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8016070 con 0x7fac0006a990 2023-08-27T11:24:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fabf7fff700 1 -- 172.21.15.45:0/3580450302 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf800bbd0 con 0x7fac0006a990 2023-08-27T11:24:23.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fabf7fff700 1 -- 172.21.15.45:0/3580450302 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf800e070 con 0x7fac0006a990 2023-08-27T11:24:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fac074bf700 1 -- 172.21.15.45:0/3580450302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0006a990 msgr2=0x7fac000b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fac074bf700 1 --2- 172.21.15.45:0/3580450302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0006a990 0x7fac000b18a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fac0006b2f0 tx=0x7fabf800b330 comp rx=0 tx=0).stop 2023-08-27T11:24:23.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fac074bf700 1 -- 172.21.15.45:0/3580450302 shutdown_connections 2023-08-27T11:24:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fac074bf700 1 --2- 172.21.15.45:0/3580450302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0006a990 0x7fac000b18a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.929+0000 7fac074bf700 1 -- 172.21.15.45:0/3580450302 >> 172.21.15.45:0/3580450302 conn(0x7fac000cc7c0 msgr2=0x7fac000ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.930+0000 7fac074bf700 1 -- 172.21.15.45:0/3580450302 shutdown_connections 2023-08-27T11:24:23.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.930+0000 7fac074bf700 1 -- 172.21.15.45:0/3580450302 wait complete. 2023-08-27T11:24:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.931+0000 7fac074bf700 1 Processor -- start 2023-08-27T11:24:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.931+0000 7fac074bf700 1 -- start start 2023-08-27T11:24:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.931+0000 7fac074bf700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0006a990 0x7fac000cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.931+0000 7fac074bf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0015d340 0x7fac001f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.931+0000 7fac074bf700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac00160d00 0x7fac001f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.931+0000 7fac074bf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac000c6d80 con 0x7fac0015d340 2023-08-27T11:24:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.931+0000 7fac074bf700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac001f3040 con 0x7fac0006a990 2023-08-27T11:24:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.931+0000 7fac074bf700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac000c78c0 con 0x7fac00160d00 2023-08-27T11:24:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac04a5a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0015d340 0x7fac001f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac04a5a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0015d340 0x7fac001f25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55028/0 (socket says 172.21.15.45:55028) 2023-08-27T11:24:23.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac04a5a700 1 -- 172.21.15.45:0/1673315086 learned_addr learned my addr 172.21.15.45:0/1673315086 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:23.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac04a5a700 1 -- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac00160d00 msgr2=0x7fac001f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac04a5a700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac00160d00 0x7fac001f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac04a5a700 1 -- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0006a990 msgr2=0x7fac000cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:23.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac04a5a700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0006a990 0x7fac000cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:23.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac04a5a700 1 -- 172.21.15.45:0/1673315086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf8009a70 con 0x7fac0015d340 2023-08-27T11:24:23.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac04a5a700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0015d340 0x7fac001f25c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00c510 tx=0x7fabfc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:23.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fabf5ffb700 1 -- 172.21.15.45:0/1673315086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0052d0 con 0x7fac0015d340 2023-08-27T11:24:23.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fabf5ffb700 1 -- 172.21.15.45:0/1673315086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc005430 con 0x7fac0015d340 2023-08-27T11:24:23.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac000c6f90 con 0x7fac0015d340 2023-08-27T11:24:23.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fabf5ffb700 1 -- 172.21.15.45:0/1673315086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0038c0 con 0x7fac0015d340 2023-08-27T11:24:23.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.932+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac000c7550 con 0x7fac0015d340 2023-08-27T11:24:23.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.933+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac00072690 con 0x7fac0015d340 2023-08-27T11:24:23.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.934+0000 7fabf5ffb700 1 -- 172.21.15.45:0/1673315086 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fabfc003a20 con 0x7fac0015d340 2023-08-27T11:24:23.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.934+0000 7fabf5ffb700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fabe406a2e0 0x7fabe406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:23.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.934+0000 7fabf5ffb700 1 -- 172.21.15.45:0/1673315086 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fabfc088040 con 0x7fac0015d340 2023-08-27T11:24:23.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.934+0000 7fac0525b700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fabe406a2e0 0x7fabe406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:23.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.935+0000 7fac0525b700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fabe406a2e0 0x7fabe406c7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fabf00015d0 tx=0x7fabf0007ad0 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:23.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:23.938+0000 7fabf5ffb700 1 -- 172.21.15.45:0/1673315086 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fabfc053230 con 0x7fac0015d340 2023-08-27T11:24:24.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.166+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fac00071950 con 0x7fabe406a2e0 2023-08-27T11:24:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.230+0000 7fabf5ffb700 1 -- 172.21.15.45:0/1673315086 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fabf0005940 con 0x7fabe406a2e0 2023-08-27T11:24:24.232 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-08-27T11:24:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fabe406a2e0 msgr2=0x7fabe406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fabe406a2e0 0x7fabe406c7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fabf00015d0 tx=0x7fabf0007ad0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0015d340 msgr2=0x7fac001f25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0015d340 0x7fac001f25c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00c510 tx=0x7fabfc00c8d0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 shutdown_connections 2023-08-27T11:24:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fabe406a2e0 0x7fabe406c7a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fac00160d00 0x7fac001f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fac0006a990 0x7fac000cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 --2- 172.21.15.45:0/1673315086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac0015d340 0x7fac001f25c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 >> 172.21.15.45:0/1673315086 conn(0x7fac000cc7c0 msgr2=0x7fac000af780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 shutdown_connections 2023-08-27T11:24:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.233+0000 7fac074bf700 1 -- 172.21.15.45:0/1673315086 wait complete. 2023-08-27T11:24:24.245 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-08-27T11:24:24.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-08-27T11:24:24.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.376+0000 7f174869d700 1 Processor -- start 2023-08-27T11:24:24.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.376+0000 7f174869d700 1 -- start start 2023-08-27T11:24:24.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f174869d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1740157be0 0x7f17400716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f174869d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f174006d5e0 0x7f17400b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f174869d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174015b460 con 0x7f17400b17c0 2023-08-27T11:24:24.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f174869d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1740157fc0 con 0x7f174006d5e0 2023-08-27T11:24:24.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f174869d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400b2280 con 0x7f1740157be0 2023-08-27T11:24:24.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f1745c38700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f17400b17c0 0x7f174006ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41810/0 (socket says 172.21.15.45:41810) 2023-08-27T11:24:24.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f1745c38700 1 -- 172.21.15.45:0/3827520974 learned_addr learned my addr 172.21.15.45:0/3827520974 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:24.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f1746c3a700 1 --2- 172.21.15.45:0/3827520974 >> v2:172.21.15.135:3300/0 conn(0x7f174006d5e0 0x7f17400b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f1746439700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1740157be0 0x7f17400716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049067030 0 0) 0x7f174015b460 con 0x7f17400b17c0 2023-08-27T11:24:24.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.377+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f172c0035b0 con 0x7f17400b17c0 2023-08-27T11:24:24.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1509523838 0 0) 0x7f172c0035b0 con 0x7f17400b17c0 2023-08-27T11:24:24.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1740157be0 msgr2=0x7f17400716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:24.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1740157be0 0x7f17400716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 >> v2:172.21.15.135:3300/0 conn(0x7f174006d5e0 msgr2=0x7f17400b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:24.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 --2- 172.21.15.45:0/3827520974 >> v2:172.21.15.135:3300/0 conn(0x7f174006d5e0 0x7f17400b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400b2560 con 0x7f17400b17c0 2023-08-27T11:24:24.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f173c001670 con 0x7f17400b17c0 2023-08-27T11:24:24.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 >> v1:172.21.15.45:6789/0 conn(0x7f17400b17c0 legacy=0x7f174006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172c003a00 0x7f172c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1740157be0 0x7f172c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f172c004d90 0x7f172c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1740157fc0 con 0x7f172c003a00 2023-08-27T11:24:24.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400b2280 con 0x7f172c004d90 2023-08-27T11:24:24.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c00ba10 con 0x7f1740157be0 2023-08-27T11:24:24.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1746439700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172c003a00 0x7f172c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745c38700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1740157be0 0x7f172c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1746c3a700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f172c004d90 0x7f172c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1746439700 1 -- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1740157be0 msgr2=0x7f172c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:24.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1746439700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1740157be0 0x7f172c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1746439700 1 -- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f172c004d90 msgr2=0x7f172c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:24.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1746439700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f172c004d90 0x7f172c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1746439700 1 -- 172.21.15.45:0/3827520974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c0035b0 con 0x7f172c003a00 2023-08-27T11:24:24.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1746439700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172c003a00 0x7f172c003e20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1730009b60 tx=0x7f172c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c99f31a8be73426 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:24.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000e6c0 con 0x7f172c003a00 2023-08-27T11:24:24.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173000e820 con 0x7f172c003a00 2023-08-27T11:24:24.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.378+0000 7f1745437700 1 -- 172.21.15.45:0/3827520974 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000eb90 con 0x7f172c003a00 2023-08-27T11:24:24.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.379+0000 7f174869d700 1 -- 172.21.15.45:0/3827520974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172c003a00 msgr2=0x7f172c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.379+0000 7f174869d700 1 --2- 172.21.15.45:0/3827520974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f172c003a00 0x7f172c003e20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1730009b60 tx=0x7f172c005880 comp rx=0 tx=0).stop 2023-08-27T11:24:24.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.379+0000 7f1746c3a700 1 -- 172.21.15.45:0/3827520974 reap_dead start 2023-08-27T11:24:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.379+0000 7f174869d700 1 -- 172.21.15.45:0/3827520974 shutdown_connections 2023-08-27T11:24:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.379+0000 7f174869d700 1 -- 172.21.15.45:0/3827520974 >> 172.21.15.45:0/3827520974 conn(0x7f17400cc7c0 msgr2=0x7f17400ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.379+0000 7f174869d700 1 -- 172.21.15.45:0/3827520974 shutdown_connections 2023-08-27T11:24:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.379+0000 7f174869d700 1 -- 172.21.15.45:0/3827520974 wait complete. 2023-08-27T11:24:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.380+0000 7f174869d700 1 Processor -- start 2023-08-27T11:24:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.380+0000 7f174869d700 1 -- start start 2023-08-27T11:24:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f174869d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f172c003a00 0x7f174014d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f174869d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17401f2de0 0x7f17401f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f174869d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1740155a70 0x7f1740155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f174869d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400c9bd0 con 0x7f17401f2de0 2023-08-27T11:24:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f174869d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17401f37a0 con 0x7f1740155a70 2023-08-27T11:24:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f174869d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1740156430 con 0x7f172c003a00 2023-08-27T11:24:24.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1745c38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17401f2de0 0x7f17401f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1745c38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17401f2de0 0x7f17401f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55048/0 (socket says 172.21.15.45:55048) 2023-08-27T11:24:24.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1746439700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f172c003a00 0x7f174014d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1745c38700 1 -- 172.21.15.45:0/3699416016 learned_addr learned my addr 172.21.15.45:0/3699416016 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:24.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1746c3a700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1740155a70 0x7f1740155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1745c38700 1 -- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f172c003a00 msgr2=0x7f174014d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1745c38700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f172c003a00 0x7f174014d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1745c38700 1 -- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1740155a70 msgr2=0x7f1740155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1745c38700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1740155a70 0x7f1740155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1745c38700 1 -- 172.21.15.45:0/3699416016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17401f3cd0 con 0x7f17401f2de0 2023-08-27T11:24:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.381+0000 7f1746439700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f172c003a00 0x7f174014d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.382+0000 7f1745c38700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17401f2de0 0x7f17401f3260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f173c00e8f0 tx=0x7f173c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.382+0000 7f1736ffd700 1 -- 172.21.15.45:0/3699416016 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c012590 con 0x7f17401f2de0 2023-08-27T11:24:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.382+0000 7f1736ffd700 1 -- 172.21.15.45:0/3699416016 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c0126f0 con 0x7f17401f2de0 2023-08-27T11:24:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.382+0000 7f1736ffd700 1 -- 172.21.15.45:0/3699416016 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c012aa0 con 0x7f17401f2de0 2023-08-27T11:24:24.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.382+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17401f39f0 con 0x7f17401f2de0 2023-08-27T11:24:24.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.382+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17400b4040 con 0x7f17401f2de0 2023-08-27T11:24:24.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.383+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1740072690 con 0x7f17401f2de0 2023-08-27T11:24:24.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.383+0000 7f1736ffd700 1 -- 172.21.15.45:0/3699416016 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f173c012c00 con 0x7f17401f2de0 2023-08-27T11:24:24.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.384+0000 7f1736ffd700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f172406a2e0 0x7f172406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.384+0000 7f1736ffd700 1 -- 172.21.15.45:0/3699416016 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f173c088920 con 0x7f17401f2de0 2023-08-27T11:24:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.384+0000 7f1746439700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f172406a2e0 0x7f172406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.384+0000 7f1746439700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f172406a2e0 0x7f172406c7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17300090b0 tx=0x7f173001f040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.388+0000 7f1736ffd700 1 -- 172.21.15.45:0/3699416016 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f173c053a70 con 0x7f17401f2de0 2023-08-27T11:24:24.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.619+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1740071190 con 0x7f172406a2e0 2023-08-27T11:24:24.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.686+0000 7f1736ffd700 1 -- 172.21.15.45:0/3699416016 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f173000fe50 con 0x7f172406a2e0 2023-08-27T11:24:24.687 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-08-27T11:24:24.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.688+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f172406a2e0 msgr2=0x7f172406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.688+0000 7f174869d700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f172406a2e0 0x7f172406c7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17300090b0 tx=0x7f173001f040 comp rx=0 tx=0).stop 2023-08-27T11:24:24.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.688+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17401f2de0 msgr2=0x7f17401f3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.688+0000 7f174869d700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17401f2de0 0x7f17401f3260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f173c00e8f0 tx=0x7f173c00ec00 comp rx=0 tx=0).stop 2023-08-27T11:24:24.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.689+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 shutdown_connections 2023-08-27T11:24:24.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.689+0000 7f174869d700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f172406a2e0 0x7f172406c7a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.689+0000 7f174869d700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1740155a70 0x7f1740155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.689+0000 7f174869d700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f172c003a00 0x7f174014d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.689+0000 7f174869d700 1 --2- 172.21.15.45:0/3699416016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17401f2de0 0x7f17401f3260 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.689+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 >> 172.21.15.45:0/3699416016 conn(0x7f17400cc7c0 msgr2=0x7f17400af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.689+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 shutdown_connections 2023-08-27T11:24:24.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.689+0000 7f174869d700 1 -- 172.21.15.45:0/3699416016 wait complete. 2023-08-27T11:24:24.701 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-08-27T11:24:24.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:24.703 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-08-27T11:24:24.721 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:24.721 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:24.722 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:24.722 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:24.722 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:24.722 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:24.722 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:24.723 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:24.723 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:24.723 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:24.723 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:24.724 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_14_00_UTC', 'scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:24.724 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-08-27T11:24:24.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.889+0000 7fbf4136d700 1 Processor -- start 2023-08-27T11:24:24.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.889+0000 7fbf4136d700 1 -- start start 2023-08-27T11:24:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.890+0000 7fbf4136d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbf3c157be0 0x7fbf3c0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.890+0000 7fbf4136d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf3c0b17c0 0x7fbf3c0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.890+0000 7fbf4136d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c15b460 con 0x7fbf3c06ced0 2023-08-27T11:24:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.890+0000 7fbf4136d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c157fc0 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.890+0000 7fbf4136d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0b2280 con 0x7fbf3c157be0 2023-08-27T11:24:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.890+0000 7fbf3a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf3c0b17c0 0x7fbf3c0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.890+0000 7fbf3a7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf3c0b17c0 0x7fbf3c0b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55056/0 (socket says 172.21.15.45:55056) 2023-08-27T11:24:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.890+0000 7fbf3b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbf3c06ced0 0x7fbf3c0cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41834/0 (socket says 172.21.15.45:41834) 2023-08-27T11:24:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.890+0000 7fbf3a7fc700 1 -- 172.21.15.45:0/3279627484 learned_addr learned my addr 172.21.15.45:0/3279627484 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf3affd700 1 --2- 172.21.15.45:0/3279627484 >> v2:172.21.15.135:3300/0 conn(0x7fbf3c157be0 0x7fbf3c0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf3a7fc700 1 -- 172.21.15.45:0/3279627484 >> v2:172.21.15.135:3300/0 conn(0x7fbf3c157be0 msgr2=0x7fbf3c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf3a7fc700 1 --2- 172.21.15.45:0/3279627484 >> v2:172.21.15.135:3300/0 conn(0x7fbf3c157be0 0x7fbf3c0716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf3a7fc700 1 -- 172.21.15.45:0/3279627484 >> v1:172.21.15.45:6789/0 conn(0x7fbf3c06ced0 legacy=0x7fbf3c0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf3a7fc700 1 -- 172.21.15.45:0/3279627484 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0b2560 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf39ffb700 1 -- 172.21.15.45:0/3279627484 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881936671 0 0) 0x7fbf3c15b460 con 0x7fbf3c06ced0 2023-08-27T11:24:24.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf3affd700 1 --2- 172.21.15.45:0/3279627484 >> v2:172.21.15.135:3300/0 conn(0x7fbf3c157be0 0x7fbf3c0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:24.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf3a7fc700 1 --2- 172.21.15.45:0/3279627484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 0x7fbf3c0b1c70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c066e70 tx=0x7fbf2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd3c3c0d0663157 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:24.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf39ffb700 1 -- 172.21.15.45:0/3279627484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c00b870 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf39ffb700 1 -- 172.21.15.45:0/3279627484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c00b9d0 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf39ffb700 1 -- 172.21.15.45:0/3279627484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c00bd60 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf4136d700 1 -- 172.21.15.45:0/3279627484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 msgr2=0x7fbf3c0b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.891+0000 7fbf4136d700 1 --2- 172.21.15.45:0/3279627484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 0x7fbf3c0b1c70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c066e70 tx=0x7fbf2c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.892+0000 7fbf4136d700 1 -- 172.21.15.45:0/3279627484 shutdown_connections 2023-08-27T11:24:24.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.892+0000 7fbf4136d700 1 --2- 172.21.15.45:0/3279627484 >> v2:172.21.15.135:3300/0 conn(0x7fbf3c157be0 0x7fbf3c0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.892+0000 7fbf4136d700 1 --2- 172.21.15.45:0/3279627484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 0x7fbf3c0b1c70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.892+0000 7fbf4136d700 1 -- 172.21.15.45:0/3279627484 >> 172.21.15.45:0/3279627484 conn(0x7fbf3c0cc7c0 msgr2=0x7fbf3c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:24.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.892+0000 7fbf4136d700 1 -- 172.21.15.45:0/3279627484 shutdown_connections 2023-08-27T11:24:24.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.892+0000 7fbf4136d700 1 -- 172.21.15.45:0/3279627484 wait complete. 2023-08-27T11:24:24.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.893+0000 7fbf4136d700 1 Processor -- start 2023-08-27T11:24:24.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.893+0000 7fbf4136d700 1 -- start start 2023-08-27T11:24:24.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf4136d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3c06ced0 0x7fbf3c0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf4136d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 0x7fbf3c154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf4136d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c157be0 0x7fbf3c15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf4136d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0c83e0 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf4136d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0c7760 con 0x7fbf3c157be0 2023-08-27T11:24:24.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf4136d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0c7a40 con 0x7fbf3c06ced0 2023-08-27T11:24:24.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 0x7fbf3c154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 0x7fbf3c154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55062/0 (socket says 172.21.15.45:55062) 2023-08-27T11:24:24.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3a7fc700 1 -- 172.21.15.45:0/2467526726 learned_addr learned my addr 172.21.15.45:0/2467526726 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:24.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3b7fe700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c157be0 0x7fbf3c15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3a7fc700 1 -- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3c06ced0 msgr2=0x7fbf3c0c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3affd700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3c06ced0 0x7fbf3c0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3a7fc700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3c06ced0 0x7fbf3c0c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3a7fc700 1 -- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c157be0 msgr2=0x7fbf3c15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3a7fc700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c157be0 0x7fbf3c15c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3a7fc700 1 -- 172.21.15.45:0/2467526726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c009550 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3b7fe700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c157be0 0x7fbf3c15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3affd700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3c06ced0 0x7fbf3c0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.894+0000 7fbf3a7fc700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 0x7fbf3c154640 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c014040 tx=0x7fbf2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.895+0000 7fbf237fe700 1 -- 172.21.15.45:0/2467526726 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c004220 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.895+0000 7fbf237fe700 1 -- 172.21.15.45:0/2467526726 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c01e070 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.895+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c0c7d20 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.895+0000 7fbf237fe700 1 -- 172.21.15.45:0/2467526726 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c004580 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.895+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c0c7450 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.896+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c0c8520 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.896+0000 7fbf237fe700 1 -- 172.21.15.45:0/2467526726 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fbf2c012800 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.896+0000 7fbf237fe700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbf2806a2e0 0x7fbf2806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:24.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.896+0000 7fbf237fe700 1 -- 172.21.15.45:0/2467526726 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fbf2c089dd0 con 0x7fbf3c0b17c0 2023-08-27T11:24:24.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.897+0000 7fbf3affd700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbf2806a2e0 0x7fbf2806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:24.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.897+0000 7fbf3affd700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbf2806a2e0 0x7fbf2806c7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0b5c90 tx=0x7fbf30008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:24.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:24.901+0000 7fbf237fe700 1 -- 172.21.15.45:0/2467526726 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf3c0c8520 con 0x7fbf3c0b17c0 2023-08-27T11:24:25.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.125+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf3c071190 con 0x7fbf2806a2e0 2023-08-27T11:24:25.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.135+0000 7fbf237fe700 1 -- 172.21.15.45:0/2467526726 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fbf3000c040 con 0x7fbf2806a2e0 2023-08-27T11:24:25.136 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:25.136 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:53", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbf2806a2e0 msgr2=0x7fbf2806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbf2806a2e0 0x7fbf2806c7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0b5c90 tx=0x7fbf30008040 comp rx=0 tx=0).stop 2023-08-27T11:24:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 msgr2=0x7fbf3c154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 0x7fbf3c154640 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c014040 tx=0x7fbf2c004750 comp rx=0 tx=0).stop 2023-08-27T11:24:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 shutdown_connections 2023-08-27T11:24:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fbf2806a2e0 0x7fbf2806c7a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf3c157be0 0x7fbf3c15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbf3c06ced0 0x7fbf3c0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 --2- 172.21.15.45:0/2467526726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf3c0b17c0 0x7fbf3c154640 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 >> 172.21.15.45:0/2467526726 conn(0x7fbf3c0cc7c0 msgr2=0x7fbf3c0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:25.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 shutdown_connections 2023-08-27T11:24:25.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.137+0000 7fbf4136d700 1 -- 172.21.15.45:0/2467526726 wait complete. 2023-08-27T11:24:25.150 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:53", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:25.151 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:53', 'first': '2023-08-27T11:14:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:25.151 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-27T11:24:25.151 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-27T11:24:25.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.152 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-08-27T11:24:25.170 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:25.171 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:25.171 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:25.171 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:25.171 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:25.172 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:25.172 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:25.172 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:25.172 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:25.172 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:25.173 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:25.173 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_14_00_UTC', 'scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:25.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-08-27T11:24:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.339+0000 7f2820c94700 1 Processor -- start 2023-08-27T11:24:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.339+0000 7f2820c94700 1 -- start start 2023-08-27T11:24:25.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f2820c94700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c158ab0 0x7f281c0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f2820c94700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f281c158e90 0x7f281c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f2820c94700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f281c0b3310 0x7f281c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f2820c94700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281c0cc8f0 con 0x7f281c0b3310 2023-08-27T11:24:25.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f2820c94700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281c15b700 con 0x7f281c158e90 2023-08-27T11:24:25.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f2820c94700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0707d0 con 0x7f281c158ab0 2023-08-27T11:24:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f281ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f281c0b3310 0x7f281c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f281ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f281c0b3310 0x7f281c0d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55072/0 (socket says 172.21.15.45:55072) 2023-08-27T11:24:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f281ad9d700 1 -- 172.21.15.45:0/991939620 learned_addr learned my addr 172.21.15.45:0/991939620 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f281a59c700 1 --2- 172.21.15.45:0/991939620 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c158ab0 0x7f281c0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f2819d9b700 1 --2- 172.21.15.45:0/991939620 >> v2:172.21.15.135:3300/0 conn(0x7f281c158e90 0x7f281c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f281ad9d700 1 -- 172.21.15.45:0/991939620 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c158ab0 msgr2=0x7f281c0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f281ad9d700 1 --2- 172.21.15.45:0/991939620 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c158ab0 0x7f281c0cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f281ad9d700 1 -- 172.21.15.45:0/991939620 >> v2:172.21.15.135:3300/0 conn(0x7f281c158e90 msgr2=0x7f281c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f281ad9d700 1 --2- 172.21.15.45:0/991939620 >> v2:172.21.15.135:3300/0 conn(0x7f281c158e90 0x7f281c0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.340+0000 7f281ad9d700 1 -- 172.21.15.45:0/991939620 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c15b910 con 0x7f281c0b3310 2023-08-27T11:24:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f281ad9d700 1 --2- 172.21.15.45:0/991939620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c0b3310 0x7f281c0d0460 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f281c0d1f80 tx=0x7f281400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b43f93aea6a740 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f281959a700 1 -- 172.21.15.45:0/991939620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281400e070 con 0x7f281c0b3310 2023-08-27T11:24:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f281959a700 1 -- 172.21.15.45:0/991939620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281400b9c0 con 0x7f281c0b3310 2023-08-27T11:24:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f281959a700 1 -- 172.21.15.45:0/991939620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281400bd70 con 0x7f281c0b3310 2023-08-27T11:24:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f2820c94700 1 -- 172.21.15.45:0/991939620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c0b3310 msgr2=0x7f281c0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f2820c94700 1 --2- 172.21.15.45:0/991939620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c0b3310 0x7f281c0d0460 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f281c0d1f80 tx=0x7f281400b330 comp rx=0 tx=0).stop 2023-08-27T11:24:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f2820c94700 1 -- 172.21.15.45:0/991939620 shutdown_connections 2023-08-27T11:24:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f2820c94700 1 --2- 172.21.15.45:0/991939620 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c158ab0 0x7f281c0cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f2820c94700 1 --2- 172.21.15.45:0/991939620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c0b3310 0x7f281c0d0460 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f2820c94700 1 --2- 172.21.15.45:0/991939620 >> v2:172.21.15.135:3300/0 conn(0x7f281c158e90 0x7f281c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f2820c94700 1 -- 172.21.15.45:0/991939620 >> 172.21.15.45:0/991939620 conn(0x7f281c06f9d0 msgr2=0x7f281c06fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:25.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f2820c94700 1 -- 172.21.15.45:0/991939620 shutdown_connections 2023-08-27T11:24:25.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.341+0000 7f2820c94700 1 -- 172.21.15.45:0/991939620 wait complete. 2023-08-27T11:24:25.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.343+0000 7f2820c94700 1 Processor -- start 2023-08-27T11:24:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.343+0000 7f2820c94700 1 -- start start 2023-08-27T11:24:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.343+0000 7f2820c94700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c0b3310 0x7f281c14e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2820c94700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c158ab0 0x7f281c0c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2820c94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c158e90 0x7f281c14f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2820c94700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c14ef40 con 0x7f281c158e90 2023-08-27T11:24:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2820c94700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c14f500 con 0x7f281c158ab0 2023-08-27T11:24:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2820c94700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0c5430 con 0x7f281c0b3310 2023-08-27T11:24:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c158e90 0x7f281c14f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c158e90 0x7f281c14f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55078/0 (socket says 172.21.15.45:55078) 2023-08-27T11:24:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281ad9d700 1 -- 172.21.15.45:0/4212380331 learned_addr learned my addr 172.21.15.45:0/4212380331 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2819d9b700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c158ab0 0x7f281c0c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281a59c700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c0b3310 0x7f281c14e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281ad9d700 1 -- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c0b3310 msgr2=0x7f281c14e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281ad9d700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c0b3310 0x7f281c14e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281ad9d700 1 -- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c158ab0 msgr2=0x7f281c0c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281ad9d700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c158ab0 0x7f281c0c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281ad9d700 1 -- 172.21.15.45:0/4212380331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2814009b30 con 0x7f281c158e90 2023-08-27T11:24:25.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2819d9b700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c158ab0 0x7f281c0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:25.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281a59c700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c0b3310 0x7f281c14e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:25.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f281ad9d700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c158e90 0x7f281c14f990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f281c0b3a70 tx=0x7f2814004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:25.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2812ffd700 1 -- 172.21.15.45:0/4212380331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814004250 con 0x7f281c158e90 2023-08-27T11:24:25.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2812ffd700 1 -- 172.21.15.45:0/4212380331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28140043d0 con 0x7f281c158e90 2023-08-27T11:24:25.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2812ffd700 1 -- 172.21.15.45:0/4212380331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814004780 con 0x7f281c158e90 2023-08-27T11:24:25.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c0c5710 con 0x7f281c158e90 2023-08-27T11:24:25.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.344+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c151970 con 0x7f281c158e90 2023-08-27T11:24:25.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.345+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c151b00 con 0x7f281c158e90 2023-08-27T11:24:25.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.346+0000 7f2812ffd700 1 -- 172.21.15.45:0/4212380331 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f281401b430 con 0x7f281c158e90 2023-08-27T11:24:25.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.346+0000 7f2812ffd700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f27fc06a2e0 0x7f27fc06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.346+0000 7f2812ffd700 1 -- 172.21.15.45:0/4212380331 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f2814089e00 con 0x7f281c158e90 2023-08-27T11:24:25.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.346+0000 7f281a59c700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f27fc06a2e0 0x7f27fc06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.347+0000 7f281a59c700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f27fc06a2e0 0x7f27fc06c7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f280c009970 tx=0x7f280c008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:25.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.351+0000 7f2812ffd700 1 -- 172.21.15.45:0/4212380331 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2814054f70 con 0x7f281c158e90 2023-08-27T11:24:25.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.574+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f281c15bed0 con 0x7f27fc06a2e0 2023-08-27T11:24:25.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.583+0000 7f2812ffd700 1 -- 172.21.15.45:0/4212380331 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f280c009e40 con 0x7f27fc06a2e0 2023-08-27T11:24:25.585 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:25.585 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:53", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f27fc06a2e0 msgr2=0x7f27fc06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f27fc06a2e0 0x7f27fc06c7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f280c009970 tx=0x7f280c008040 comp rx=0 tx=0).stop 2023-08-27T11:24:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c158e90 msgr2=0x7f281c14f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c158e90 0x7f281c14f990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f281c0b3a70 tx=0x7f2814004ad0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 shutdown_connections 2023-08-27T11:24:25.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f27fc06a2e0 0x7f27fc06c7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f281c158e90 0x7f281c14f990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f281c0b3310 0x7f281c14e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 --2- 172.21.15.45:0/4212380331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f281c158ab0 0x7f281c0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 >> 172.21.15.45:0/4212380331 conn(0x7f281c06f9d0 msgr2=0x7f281c15e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 shutdown_connections 2023-08-27T11:24:25.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.585+0000 7f2820c94700 1 -- 172.21.15.45:0/4212380331 wait complete. 2023-08-27T11:24:25.598 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:53", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:25.598 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:53', 'first': '2023-08-27T11:14:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:25.599 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-27T11:24:25.599 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-27T11:24:25.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:25.600 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-08-27T11:24:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:25.620 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:25.620 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:25.620 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:25.620 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:25.621 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:25.621 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:25.622 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:25.622 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:25.622 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:25.623 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:25.623 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_14_00_UTC', 'scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:25.624 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-08-27T11:24:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.788+0000 7f2449990700 1 Processor -- start 2023-08-27T11:24:25.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.788+0000 7f2449990700 1 -- start start 2023-08-27T11:24:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2449990700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24440ce4b0 0x7f2444167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2449990700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244415d6a0 con 0x7f24440b8c20 2023-08-27T11:24:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2449990700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24440d2c90 con 0x7f244415d2c0 2023-08-27T11:24:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2449990700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440d1bf0 con 0x7f24440ce4b0 2023-08-27T11:24:25.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f24427fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f24440b8c20 0x7f24440cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41860/0 (socket says 172.21.15.45:41860) 2023-08-27T11:24:25.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f24427fc700 1 -- 172.21.15.45:0/1869026202 learned_addr learned my addr 172.21.15.45:0/1869026202 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:25.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335620580 0 0) 0x7f2434001110 con 0x7f24440b8c20 2023-08-27T11:24:25.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f24437fe700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24440ce4b0 0x7f2444167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f242c0035b0 con 0x7f24440b8c20 2023-08-27T11:24:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2521069345 0 0) 0x7f242c0035b0 con 0x7f24440b8c20 2023-08-27T11:24:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24440ce4b0 msgr2=0x7f2444167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24440ce4b0 0x7f2444167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 >> v1:172.21.15.135:6789/0 conn(0x7f244415d2c0 legacy=0x7f24440d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440d1ed0 con 0x7f24440b8c20 2023-08-27T11:24:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f2434001800 con 0x7f24440b8c20 2023-08-27T11:24:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 >> v1:172.21.15.45:6789/0 conn(0x7f24440b8c20 legacy=0x7f24440cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f242c003a30 0x7f242c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c004850 0x7f242c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24440ce4b0 0x7f242c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440d1bf0 con 0x7f242c003a30 2023-08-27T11:24:25.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c00bb70 con 0x7f242c004850 2023-08-27T11:24:25.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.789+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c00bd30 con 0x7f24440ce4b0 2023-08-27T11:24:25.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2442ffd700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f242c003a30 0x7f242c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f24427fc700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c004850 0x7f242c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f24437fe700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24440ce4b0 0x7f242c005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2442ffd700 1 -- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24440ce4b0 msgr2=0x7f242c005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:25.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2442ffd700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24440ce4b0 0x7f242c005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2442ffd700 1 -- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c004850 msgr2=0x7f242c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:25.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2442ffd700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c004850 0x7f242c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2442ffd700 1 -- 172.21.15.45:0/1869026202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0035b0 con 0x7f242c003a30 2023-08-27T11:24:25.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2442ffd700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f242c003a30 0x7f242c003e50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f24380017c0 tx=0x7f242c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1b28d27bbe4446 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:25.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24380052d0 con 0x7f242c003a30 2023-08-27T11:24:25.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2438005430 con 0x7f242c003a30 2023-08-27T11:24:25.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.790+0000 7f2441ffb700 1 -- 172.21.15.45:0/1869026202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24380038c0 con 0x7f242c003a30 2023-08-27T11:24:25.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.791+0000 7f2449990700 1 -- 172.21.15.45:0/1869026202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f242c003a30 msgr2=0x7f242c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.791+0000 7f2449990700 1 --2- 172.21.15.45:0/1869026202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f242c003a30 0x7f242c003e50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f24380017c0 tx=0x7f242c005880 comp rx=0 tx=0).stop 2023-08-27T11:24:25.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.791+0000 7f24437fe700 1 -- 172.21.15.45:0/1869026202 reap_dead start 2023-08-27T11:24:25.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.791+0000 7f2449990700 1 -- 172.21.15.45:0/1869026202 shutdown_connections 2023-08-27T11:24:25.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.791+0000 7f2449990700 1 -- 172.21.15.45:0/1869026202 >> 172.21.15.45:0/1869026202 conn(0x7f24440d2fd0 msgr2=0x7f24440d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:25.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.791+0000 7f2449990700 1 -- 172.21.15.45:0/1869026202 shutdown_connections 2023-08-27T11:24:25.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.791+0000 7f2449990700 1 -- 172.21.15.45:0/1869026202 wait complete. 2023-08-27T11:24:25.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.792+0000 7f2449990700 1 Processor -- start 2023-08-27T11:24:25.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.792+0000 7f2449990700 1 -- start start 2023-08-27T11:24:25.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f2449990700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c003a30 0x7f24440c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f2449990700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24441527a0 0x7f2444150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f2449990700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24441f27b0 0x7f2444153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f2449990700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2444152380 con 0x7f24441f27b0 2023-08-27T11:24:25.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f2449990700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24441514f0 con 0x7f242c003a30 2023-08-27T11:24:25.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f2449990700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2444151700 con 0x7f24441527a0 2023-08-27T11:24:25.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24441f27b0 0x7f2444153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24437fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24441f27b0 0x7f2444153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55096/0 (socket says 172.21.15.45:55096) 2023-08-27T11:24:25.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24437fe700 1 -- 172.21.15.45:0/2093141040 learned_addr learned my addr 172.21.15.45:0/2093141040 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:25.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24427fc700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24441527a0 0x7f2444150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f2442ffd700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c003a30 0x7f24440c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24437fe700 1 -- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24441527a0 msgr2=0x7f2444150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24437fe700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24441527a0 0x7f2444150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24437fe700 1 -- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c003a30 msgr2=0x7f24440c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:25.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24437fe700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c003a30 0x7f24440c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:25.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24437fe700 1 -- 172.21.15.45:0/2093141040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24441519e0 con 0x7f24441f27b0 2023-08-27T11:24:25.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f2442ffd700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c003a30 0x7f24440c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:25.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.793+0000 7f24437fe700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24441f27b0 0x7f2444153050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f24440d0e00 tx=0x7f243c0048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:25.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.794+0000 7f242bfff700 1 -- 172.21.15.45:0/2093141040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c007900 con 0x7f24441f27b0 2023-08-27T11:24:25.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.794+0000 7f242bfff700 1 -- 172.21.15.45:0/2093141040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c007a60 con 0x7f24441f27b0 2023-08-27T11:24:25.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.794+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2444151cc0 con 0x7f24441f27b0 2023-08-27T11:24:25.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.794+0000 7f242bfff700 1 -- 172.21.15.45:0/2093141040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c007e10 con 0x7f24441f27b0 2023-08-27T11:24:25.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.794+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24441f2c40 con 0x7f24441f27b0 2023-08-27T11:24:25.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.795+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24441524c0 con 0x7f24441f27b0 2023-08-27T11:24:25.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.795+0000 7f242bfff700 1 -- 172.21.15.45:0/2093141040 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f243c015420 con 0x7f24441f27b0 2023-08-27T11:24:25.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.795+0000 7f242bfff700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f243006a2e0 0x7f243006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:25.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.796+0000 7f242bfff700 1 -- 172.21.15.45:0/2093141040 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f243c088e10 con 0x7f24441f27b0 2023-08-27T11:24:25.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.796+0000 7f2442ffd700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f243006a2e0 0x7f243006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:25.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.796+0000 7f2442ffd700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f243006a2e0 0x7f243006c7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2438015040 tx=0x7f2438009030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:25.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:25.800+0000 7f242bfff700 1 -- 172.21.15.45:0/2093141040 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f24441524c0 con 0x7f24441f27b0 2023-08-27T11:24:26.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.024+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24440d2490 con 0x7f243006a2e0 2023-08-27T11:24:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.034+0000 7f242bfff700 1 -- 172.21.15.45:0/2093141040 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f2438005270 con 0x7f243006a2e0 2023-08-27T11:24:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:26.035 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:54", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f243006a2e0 msgr2=0x7f243006c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f243006a2e0 0x7f243006c7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2438015040 tx=0x7f2438009030 comp rx=0 tx=0).stop 2023-08-27T11:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24441f27b0 msgr2=0x7f2444153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24441f27b0 0x7f2444153050 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f24440d0e00 tx=0x7f243c0048c0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 shutdown_connections 2023-08-27T11:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f243006a2e0 0x7f243006c7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24441f27b0 0x7f2444153050 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f242c003a30 0x7f24440c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 --2- 172.21.15.45:0/2093141040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f24441527a0 0x7f2444150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 >> 172.21.15.45:0/2093141040 conn(0x7f24440d2fd0 msgr2=0x7f24440cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:26.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 shutdown_connections 2023-08-27T11:24:26.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.036+0000 7f2449990700 1 -- 172.21.15.45:0/2093141040 wait complete. 2023-08-27T11:24:26.049 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:54", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:26.050 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:54', 'first': '2023-08-27T11:14:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:26.050 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-27T11:24:26.050 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-27T11:24:26.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.051 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-08-27T11:24:26.069 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:26.071 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:26.071 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:26.071 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:26.071 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:26.071 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:26.072 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:26.072 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_14_00_UTC', 'scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:26.072 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-08-27T11:24:26.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.238+0000 7f4b44df3700 1 Processor -- start 2023-08-27T11:24:26.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.238+0000 7f4b44df3700 1 -- start start 2023-08-27T11:24:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b44df3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4b4015b2e0 0x7f4b400cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b44df3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cf950 0x7f4b400cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b44df3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b400cb9b0 con 0x7f4b400cd540 2023-08-27T11:24:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b44df3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b400ce2e0 con 0x7f4b4015b2e0 2023-08-27T11:24:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b44df3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400cc230 con 0x7f4b400cf950 2023-08-27T11:24:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4b400cd540 0x7f4b400cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41878/0 (socket says 172.21.15.45:41878) 2023-08-27T11:24:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3dd9b700 1 -- 172.21.15.45:0/1086032194 learned_addr learned my addr 172.21.15.45:0/1086032194 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:26.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3e59c700 1 --2- 172.21.15.45:0/1086032194 >> v2:172.21.15.135:3300/0 conn(0x7f4b4015b2e0 0x7f4b400cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3ed9d700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cf950 0x7f4b400cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054289887 0 0) 0x7f4b400cb9b0 con 0x7f4b400cd540 2023-08-27T11:24:26.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b280035b0 con 0x7f4b400cd540 2023-08-27T11:24:26.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1650398511 0 0) 0x7f4b280035b0 con 0x7f4b400cd540 2023-08-27T11:24:26.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cf950 msgr2=0x7f4b400cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:26.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cf950 0x7f4b400cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 >> v2:172.21.15.135:3300/0 conn(0x7f4b4015b2e0 msgr2=0x7f4b400cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:26.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 --2- 172.21.15.45:0/1086032194 >> v2:172.21.15.135:3300/0 conn(0x7f4b4015b2e0 0x7f4b400cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b400ce000 con 0x7f4b400cd540 2023-08-27T11:24:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f4b34001670 con 0x7f4b400cd540 2023-08-27T11:24:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 >> v1:172.21.15.45:6789/0 conn(0x7f4b400cd540 legacy=0x7f4b400cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28003a00 0x7f4b28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004850 0x7f4b28004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cf950 0x7f4b28005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400ce2e0 con 0x7f4b28003a00 2023-08-27T11:24:26.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400cc230 con 0x7f4b28004850 2023-08-27T11:24:26.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2800ba10 con 0x7f4b400cf950 2023-08-27T11:24:26.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.239+0000 7f4b3e59c700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28003a00 0x7f4b28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3dd9b700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004850 0x7f4b28004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3ed9d700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cf950 0x7f4b28005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3e59c700 1 -- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cf950 msgr2=0x7f4b28005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:26.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3e59c700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400cf950 0x7f4b28005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3e59c700 1 -- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004850 msgr2=0x7f4b28004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:26.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3e59c700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004850 0x7f4b28004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3e59c700 1 -- 172.21.15.45:0/1086032194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b280035b0 con 0x7f4b28003a00 2023-08-27T11:24:26.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3dd9b700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28004850 0x7f4b28004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:26.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3e59c700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28003a00 0x7f4b28003e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009b60 tx=0x7f4b28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252f23eddf039497 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:26.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000e6c0 con 0x7f4b28003a00 2023-08-27T11:24:26.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3000e820 con 0x7f4b28003a00 2023-08-27T11:24:26.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.240+0000 7f4b3d59a700 1 -- 172.21.15.45:0/1086032194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000eb90 con 0x7f4b28003a00 2023-08-27T11:24:26.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.241+0000 7f4b44df3700 1 -- 172.21.15.45:0/1086032194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28003a00 msgr2=0x7f4b28003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.241+0000 7f4b44df3700 1 --2- 172.21.15.45:0/1086032194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28003a00 0x7f4b28003e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009b60 tx=0x7f4b28005880 comp rx=0 tx=0).stop 2023-08-27T11:24:26.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.241+0000 7f4b3ed9d700 1 -- 172.21.15.45:0/1086032194 reap_dead start 2023-08-27T11:24:26.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.241+0000 7f4b44df3700 1 -- 172.21.15.45:0/1086032194 shutdown_connections 2023-08-27T11:24:26.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.241+0000 7f4b44df3700 1 -- 172.21.15.45:0/1086032194 >> 172.21.15.45:0/1086032194 conn(0x7f4b4009a1a0 msgr2=0x7f4b400d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:26.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.241+0000 7f4b44df3700 1 -- 172.21.15.45:0/1086032194 shutdown_connections 2023-08-27T11:24:26.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.241+0000 7f4b44df3700 1 -- 172.21.15.45:0/1086032194 wait complete. 2023-08-27T11:24:26.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.242+0000 7f4b44df3700 1 Processor -- start 2023-08-27T11:24:26.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.242+0000 7f4b44df3700 1 -- start start 2023-08-27T11:24:26.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b44df3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28003a00 0x7f4b401584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b44df3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400c7ee0 0x7f4b400c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b44df3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400c5840 0x7f4b400c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b44df3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400b4e70 con 0x7f4b400c5840 2023-08-27T11:24:26.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b44df3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400b5110 con 0x7f4b28003a00 2023-08-27T11:24:26.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b44df3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400b53f0 con 0x7f4b400c7ee0 2023-08-27T11:24:26.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400c5840 0x7f4b400c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400c5840 0x7f4b400c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55118/0 (socket says 172.21.15.45:55118) 2023-08-27T11:24:26.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3ed9d700 1 -- 172.21.15.45:0/2230547550 learned_addr learned my addr 172.21.15.45:0/2230547550 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:26.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3e59c700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28003a00 0x7f4b401584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3dd9b700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400c7ee0 0x7f4b400c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3ed9d700 1 -- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400c7ee0 msgr2=0x7f4b400c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3ed9d700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400c7ee0 0x7f4b400c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3ed9d700 1 -- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28003a00 msgr2=0x7f4b401584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3ed9d700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28003a00 0x7f4b401584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3ed9d700 1 -- 172.21.15.45:0/2230547550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b400b56d0 con 0x7f4b400c5840 2023-08-27T11:24:26.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3e59c700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28003a00 0x7f4b401584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:26.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3dd9b700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400c7ee0 0x7f4b400c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:26.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.243+0000 7f4b3ed9d700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400c5840 0x7f4b400c6e30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4b40071a30 tx=0x7f4b380114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:26.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.244+0000 7f4b2effd700 1 -- 172.21.15.45:0/2230547550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3800e070 con 0x7f4b400c5840 2023-08-27T11:24:26.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.244+0000 7f4b2effd700 1 -- 172.21.15.45:0/2230547550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b38003c30 con 0x7f4b400c5840 2023-08-27T11:24:26.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.244+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b400b59b0 con 0x7f4b400c5840 2023-08-27T11:24:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.244+0000 7f4b2effd700 1 -- 172.21.15.45:0/2230547550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38003de0 con 0x7f4b400c5840 2023-08-27T11:24:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.244+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b400c6b50 con 0x7f4b400c5840 2023-08-27T11:24:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.245+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4009eda0 con 0x7f4b400c5840 2023-08-27T11:24:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.245+0000 7f4b2effd700 1 -- 172.21.15.45:0/2230547550 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f4b38011a10 con 0x7f4b400c5840 2023-08-27T11:24:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.246+0000 7f4b2effd700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f4b2006a2e0 0x7f4b2006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.246+0000 7f4b2effd700 1 -- 172.21.15.45:0/2230547550 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f4b38089ae0 con 0x7f4b400c5840 2023-08-27T11:24:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.246+0000 7f4b3e59c700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f4b2006a2e0 0x7f4b2006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.247+0000 7f4b3e59c700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f4b2006a2e0 0x7f4b2006c7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009a80 tx=0x7f4b30008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.250+0000 7f4b2effd700 1 -- 172.21.15.45:0/2230547550 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4b38054ca0 con 0x7f4b400c5840 2023-08-27T11:24:26.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.471+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b400cbec0 con 0x7f4b2006a2e0 2023-08-27T11:24:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.481+0000 7f4b2effd700 1 -- 172.21.15.45:0/2230547550 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f4b3000e990 con 0x7f4b2006a2e0 2023-08-27T11:24:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:26.482 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:54", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f4b2006a2e0 msgr2=0x7f4b2006c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f4b2006a2e0 0x7f4b2006c7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009a80 tx=0x7f4b30008030 comp rx=0 tx=0).stop 2023-08-27T11:24:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400c5840 msgr2=0x7f4b400c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400c5840 0x7f4b400c6e30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4b40071a30 tx=0x7f4b380114b0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 shutdown_connections 2023-08-27T11:24:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f4b2006a2e0 0x7f4b2006c7a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b400c5840 0x7f4b400c6e30 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b28003a00 0x7f4b401584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 --2- 172.21.15.45:0/2230547550 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b400c7ee0 0x7f4b400c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 >> 172.21.15.45:0/2230547550 conn(0x7f4b4009a1a0 msgr2=0x7f4b400cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 shutdown_connections 2023-08-27T11:24:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.483+0000 7f4b44df3700 1 -- 172.21.15.45:0/2230547550 wait complete. 2023-08-27T11:24:26.495 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:54", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:26.496 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:54', 'first': '2023-08-27T11:14:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:26.496 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-27T11:24:26.497 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-27T11:24:26.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.497 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-08-27T11:24:26.515 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:26.516 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:26.516 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:26.516 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:26.516 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:26.516 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:26.517 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:26.517 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:26.517 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:26.517 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:26.517 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:26.518 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_14_00_UTC', 'scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:26.518 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-08-27T11:24:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.683+0000 7fa85601e700 1 Processor -- start 2023-08-27T11:24:26.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.684+0000 7fa85601e700 1 -- start start 2023-08-27T11:24:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.684+0000 7fa85601e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 0x7fa8500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.684+0000 7fa85601e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa85006c130 0x7fa85006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.684+0000 7fa85601e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa85006a7c0 0x7fa8500b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.684+0000 7fa85601e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85006b7c0 con 0x7fa85006a7c0 2023-08-27T11:24:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.684+0000 7fa85601e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85006ac30 con 0x7fa85006c130 2023-08-27T11:24:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.684+0000 7fa85601e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500cf7d0 con 0x7fa85015b2e0 2023-08-27T11:24:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.684+0000 7fa84ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa85006a7c0 0x7fa8500b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa85006a7c0 0x7fa8500b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55124/0 (socket says 172.21.15.45:55124) 2023-08-27T11:24:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84ffff700 1 -- 172.21.15.45:0/2080648131 learned_addr learned my addr 172.21.15.45:0/2080648131 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84effd700 1 --2- 172.21.15.45:0/2080648131 >> v2:172.21.15.135:3300/0 conn(0x7fa85006c130 0x7fa85006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84f7fe700 1 --2- 172.21.15.45:0/2080648131 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 0x7fa8500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84ffff700 1 -- 172.21.15.45:0/2080648131 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 msgr2=0x7fa8500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84ffff700 1 --2- 172.21.15.45:0/2080648131 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 0x7fa8500cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84ffff700 1 -- 172.21.15.45:0/2080648131 >> v2:172.21.15.135:3300/0 conn(0x7fa85006c130 msgr2=0x7fa85006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84ffff700 1 --2- 172.21.15.45:0/2080648131 >> v2:172.21.15.135:3300/0 conn(0x7fa85006c130 0x7fa85006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84ffff700 1 -- 172.21.15.45:0/2080648131 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8500cfa40 con 0x7fa85006a7c0 2023-08-27T11:24:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84ffff700 1 --2- 172.21.15.45:0/2080648131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 0x7fa8500b0f10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa850072a10 tx=0x7fa84800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35a2a411bb2d2a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84e7fc700 1 -- 172.21.15.45:0/2080648131 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84800e070 con 0x7fa85006a7c0 2023-08-27T11:24:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84e7fc700 1 -- 172.21.15.45:0/2080648131 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84800b9c0 con 0x7fa85006a7c0 2023-08-27T11:24:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.685+0000 7fa84e7fc700 1 -- 172.21.15.45:0/2080648131 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84800bd70 con 0x7fa85006a7c0 2023-08-27T11:24:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.686+0000 7fa85601e700 1 -- 172.21.15.45:0/2080648131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 msgr2=0x7fa8500b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.686+0000 7fa85601e700 1 --2- 172.21.15.45:0/2080648131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 0x7fa8500b0f10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa850072a10 tx=0x7fa84800b330 comp rx=0 tx=0).stop 2023-08-27T11:24:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.686+0000 7fa85601e700 1 -- 172.21.15.45:0/2080648131 shutdown_connections 2023-08-27T11:24:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.686+0000 7fa85601e700 1 --2- 172.21.15.45:0/2080648131 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 0x7fa8500cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.686+0000 7fa85601e700 1 --2- 172.21.15.45:0/2080648131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 0x7fa8500b0f10 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.686+0000 7fa85601e700 1 --2- 172.21.15.45:0/2080648131 >> v2:172.21.15.135:3300/0 conn(0x7fa85006c130 0x7fa85006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.686+0000 7fa85601e700 1 -- 172.21.15.45:0/2080648131 >> 172.21.15.45:0/2080648131 conn(0x7fa8500b23e0 msgr2=0x7fa8500b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.686+0000 7fa85601e700 1 -- 172.21.15.45:0/2080648131 shutdown_connections 2023-08-27T11:24:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.686+0000 7fa85601e700 1 -- 172.21.15.45:0/2080648131 wait complete. 2023-08-27T11:24:26.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.687+0000 7fa85601e700 1 Processor -- start 2023-08-27T11:24:26.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.687+0000 7fa85601e700 1 -- start start 2023-08-27T11:24:26.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa85601e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 0x7fa850098a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa85601e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85006c130 0x7fa8500c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa85601e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 0x7fa8500c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa85601e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8501f3880 con 0x7fa85006a7c0 2023-08-27T11:24:26.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa85601e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8501f29b0 con 0x7fa85006c130 2023-08-27T11:24:26.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa85601e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8501f2c00 con 0x7fa85015b2e0 2023-08-27T11:24:26.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 0x7fa850098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 0x7fa850098a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55126/0 (socket says 172.21.15.45:55126) 2023-08-27T11:24:26.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84f7fe700 1 -- 172.21.15.45:0/4005307763 learned_addr learned my addr 172.21.15.45:0/4005307763 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:26.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84effd700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85006c130 0x7fa8500c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84ffff700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 0x7fa8500c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84f7fe700 1 -- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 msgr2=0x7fa8500c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84f7fe700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 0x7fa8500c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84f7fe700 1 -- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85006c130 msgr2=0x7fa8500c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84f7fe700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85006c130 0x7fa8500c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.688+0000 7fa84f7fe700 1 -- 172.21.15.45:0/4005307763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848009b30 con 0x7fa85006a7c0 2023-08-27T11:24:26.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.689+0000 7fa84f7fe700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 0x7fa850098a80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa84000e810 tx=0x7fa84000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:26.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.689+0000 7fa837fff700 1 -- 172.21.15.45:0/4005307763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84000c070 con 0x7fa85006a7c0 2023-08-27T11:24:26.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.689+0000 7fa837fff700 1 -- 172.21.15.45:0/4005307763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa840013610 con 0x7fa85006a7c0 2023-08-27T11:24:26.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.689+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8501f2ee0 con 0x7fa85006a7c0 2023-08-27T11:24:26.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.689+0000 7fa837fff700 1 -- 172.21.15.45:0/4005307763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8400139c0 con 0x7fa85006a7c0 2023-08-27T11:24:26.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.689+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8501f34a0 con 0x7fa85006a7c0 2023-08-27T11:24:26.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.690+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85006afb0 con 0x7fa85006a7c0 2023-08-27T11:24:26.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.690+0000 7fa837fff700 1 -- 172.21.15.45:0/4005307763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fa840013b20 con 0x7fa85006a7c0 2023-08-27T11:24:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.691+0000 7fa837fff700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa83c06a2e0 0x7fa83c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.691+0000 7fa837fff700 1 -- 172.21.15.45:0/4005307763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fa840088a00 con 0x7fa85006a7c0 2023-08-27T11:24:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.691+0000 7fa84effd700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa83c06a2e0 0x7fa83c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.692+0000 7fa84effd700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa83c06a2e0 0x7fa83c06c7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa8500ad730 tx=0x7fa844008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.695+0000 7fa837fff700 1 -- 172.21.15.45:0/4005307763 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa840053a50 con 0x7fa85006a7c0 2023-08-27T11:24:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.920+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8500cff80 con 0x7fa83c06a2e0 2023-08-27T11:24:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.930+0000 7fa837fff700 1 -- 172.21.15.45:0/4005307763 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fa844009e80 con 0x7fa83c06a2e0 2023-08-27T11:24:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:26.932 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:55", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa83c06a2e0 msgr2=0x7fa83c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa83c06a2e0 0x7fa83c06c7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa8500ad730 tx=0x7fa844008040 comp rx=0 tx=0).stop 2023-08-27T11:24:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 msgr2=0x7fa850098a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 0x7fa850098a80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa84000e810 tx=0x7fa84000eb20 comp rx=0 tx=0).stop 2023-08-27T11:24:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 shutdown_connections 2023-08-27T11:24:26.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa83c06a2e0 0x7fa83c06c7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85006a7c0 0x7fa850098a80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa85015b2e0 0x7fa8500c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 --2- 172.21.15.45:0/4005307763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa85006c130 0x7fa8500c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.932+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 >> 172.21.15.45:0/4005307763 conn(0x7fa8500b23e0 msgr2=0x7fa8500ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.933+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 shutdown_connections 2023-08-27T11:24:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:26.933+0000 7fa85601e700 1 -- 172.21.15.45:0/4005307763 wait complete. 2023-08-27T11:24:26.946 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:55", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:26.946 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:55', 'first': '2023-08-27T11:14:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:26.947 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-27T11:24:26.947 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-27T11:24:26.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:26.949 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-08-27T11:24:26.967 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:26.967 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:26.968 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:26.968 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:26.968 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:26.968 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:26.969 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:26.969 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:26.969 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:26.969 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:26.970 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:26.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-08-27T11:24:27.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.137+0000 7fa06bc8c700 1 Processor -- start 2023-08-27T11:24:27.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.138+0000 7fa06bc8c700 1 -- start start 2023-08-27T11:24:27.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.138+0000 7fa06bc8c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa064157be0 0x7fa0640716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.138+0000 7fa06bc8c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa0640b17c0 0x7fa0640b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.138+0000 7fa06bc8c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06415b460 con 0x7fa06406ced0 2023-08-27T11:24:27.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.138+0000 7fa06bc8c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa064157fc0 con 0x7fa0640b17c0 2023-08-27T11:24:27.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.138+0000 7fa06bc8c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640b2280 con 0x7fa064157be0 2023-08-27T11:24:27.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.138+0000 7fa06a229700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa06406ced0 0x7fa0640cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41914/0 (socket says 172.21.15.45:41914) 2023-08-27T11:24:27.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.138+0000 7fa06a229700 1 -- 172.21.15.45:0/3206609611 learned_addr learned my addr 172.21.15.45:0/3206609611 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:27.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288126740 0 0) 0x7fa06415b460 con 0x7fa06406ced0 2023-08-27T11:24:27.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa069a28700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa064157be0 0x7fa0640716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0500035b0 con 0x7fa06406ced0 2023-08-27T11:24:27.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa069227700 1 --2- 172.21.15.45:0/3206609611 >> v2:172.21.15.135:3300/0 conn(0x7fa0640b17c0 0x7fa0640b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4165665186 0 0) 0x7fa0500035b0 con 0x7fa06406ced0 2023-08-27T11:24:27.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa064157be0 msgr2=0x7fa0640716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:27.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa064157be0 0x7fa0640716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 >> v2:172.21.15.135:3300/0 conn(0x7fa0640b17c0 msgr2=0x7fa0640b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:27.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 --2- 172.21.15.45:0/3206609611 >> v2:172.21.15.135:3300/0 conn(0x7fa0640b17c0 0x7fa0640b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0640b2560 con 0x7fa06406ced0 2023-08-27T11:24:27.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fa05c0019f0 con 0x7fa06406ced0 2023-08-27T11:24:27.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa069227700 1 --2- 172.21.15.45:0/3206609611 >> v2:172.21.15.135:3300/0 conn(0x7fa0640b17c0 0x7fa0640b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:27.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 >> v1:172.21.15.45:6789/0 conn(0x7fa06406ced0 legacy=0x7fa0640cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa064157be0 0x7fa050003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa050004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa050004d90 0x7fa050005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa064157fc0 con 0x7fa050003f80 2023-08-27T11:24:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640b2280 con 0x7fa050004d90 2023-08-27T11:24:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05000ba60 con 0x7fa064157be0 2023-08-27T11:24:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa069227700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa050004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa069a28700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa064157be0 0x7fa050003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.139+0000 7fa06a229700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa050004d90 0x7fa050005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa069227700 1 -- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa064157be0 msgr2=0x7fa050003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa069227700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa064157be0 0x7fa050003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa069227700 1 -- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa050004d90 msgr2=0x7fa050005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa069227700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa050004d90 0x7fa050005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa069227700 1 -- 172.21.15.45:0/3206609611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0500035b0 con 0x7fa050003f80 2023-08-27T11:24:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa06a229700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa050004d90 0x7fa050005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa069227700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa050004850 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa060009950 tx=0x7fa050005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c863f7c314cd777 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000c070 con 0x7fa050003f80 2023-08-27T11:24:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa060013610 con 0x7fa050003f80 2023-08-27T11:24:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa068a26700 1 -- 172.21.15.45:0/3206609611 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060013980 con 0x7fa050003f80 2023-08-27T11:24:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa06bc8c700 1 -- 172.21.15.45:0/3206609611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 msgr2=0x7fa050004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa06bc8c700 1 --2- 172.21.15.45:0/3206609611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa050004850 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa060009950 tx=0x7fa050005880 comp rx=0 tx=0).stop 2023-08-27T11:24:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa06a229700 1 -- 172.21.15.45:0/3206609611 reap_dead start 2023-08-27T11:24:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa06bc8c700 1 -- 172.21.15.45:0/3206609611 shutdown_connections 2023-08-27T11:24:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa06bc8c700 1 -- 172.21.15.45:0/3206609611 >> 172.21.15.45:0/3206609611 conn(0x7fa0640cc7c0 msgr2=0x7fa0640ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa06bc8c700 1 -- 172.21.15.45:0/3206609611 shutdown_connections 2023-08-27T11:24:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.140+0000 7fa06bc8c700 1 -- 172.21.15.45:0/3206609611 wait complete. 2023-08-27T11:24:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.142+0000 7fa06bc8c700 1 Processor -- start 2023-08-27T11:24:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.142+0000 7fa06bc8c700 1 -- start start 2023-08-27T11:24:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.142+0000 7fa06bc8c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa0640c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.142+0000 7fa06bc8c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa06414c930 0x7fa064158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.142+0000 7fa06bc8c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0640b0980 0x7fa0640b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.142+0000 7fa06bc8c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06415a000 con 0x7fa050003f80 2023-08-27T11:24:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.142+0000 7fa06bc8c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640b1370 con 0x7fa0640b0980 2023-08-27T11:24:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.142+0000 7fa06bc8c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640b3610 con 0x7fa06414c930 2023-08-27T11:24:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.142+0000 7fa069a28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa0640c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa069a28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa0640c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55146/0 (socket says 172.21.15.45:55146) 2023-08-27T11:24:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa069a28700 1 -- 172.21.15.45:0/1492223376 learned_addr learned my addr 172.21.15.45:0/1492223376 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa069a28700 1 -- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa06414c930 msgr2=0x7fa064158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa069a28700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa06414c930 0x7fa064158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa069a28700 1 -- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0640b0980 msgr2=0x7fa0640b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa069a28700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0640b0980 0x7fa0640b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa069a28700 1 -- 172.21.15.45:0/1492223376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0640c6ae0 con 0x7fa050003f80 2023-08-27T11:24:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa069a28700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa0640c65a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa054011ad0 tx=0x7fa054011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa05a7fc700 1 -- 172.21.15.45:0/1492223376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05400f920 con 0x7fa050003f80 2023-08-27T11:24:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa05a7fc700 1 -- 172.21.15.45:0/1492223376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05400fa80 con 0x7fa050003f80 2023-08-27T11:24:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0641593f0 con 0x7fa050003f80 2023-08-27T11:24:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa05a7fc700 1 -- 172.21.15.45:0/1492223376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05400fe30 con 0x7fa050003f80 2023-08-27T11:24:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.143+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0641599b0 con 0x7fa050003f80 2023-08-27T11:24:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.144+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0640b38b0 con 0x7fa050003f80 2023-08-27T11:24:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.144+0000 7fa05a7fc700 1 -- 172.21.15.45:0/1492223376 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fa054014440 con 0x7fa050003f80 2023-08-27T11:24:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.145+0000 7fa05a7fc700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa04806a2e0 0x7fa04806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.145+0000 7fa05a7fc700 1 -- 172.21.15.45:0/1492223376 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fa054089110 con 0x7fa050003f80 2023-08-27T11:24:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.145+0000 7fa069227700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa04806a2e0 0x7fa04806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.146+0000 7fa069227700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa04806a2e0 0x7fa04806c7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa064157390 tx=0x7fa060013000 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:27.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.149+0000 7fa05a7fc700 1 -- 172.21.15.45:0/1492223376 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa054054520 con 0x7fa050003f80 2023-08-27T11:24:27.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.376+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa064070eb0 con 0x7fa04806a2e0 2023-08-27T11:24:27.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.385+0000 7fa05a7fc700 1 -- 172.21.15.45:0/1492223376 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fa060013490 con 0x7fa04806a2e0 2023-08-27T11:24:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:27.386 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:55", "first": "2023-08-27T11:15:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 10, "pruned_count": 0, "active": false}] 2023-08-27T11:24:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa04806a2e0 msgr2=0x7fa04806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa04806a2e0 0x7fa04806c7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa064157390 tx=0x7fa060013000 comp rx=0 tx=0).stop 2023-08-27T11:24:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 msgr2=0x7fa0640c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa0640c65a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa054011ad0 tx=0x7fa054011e90 comp rx=0 tx=0).stop 2023-08-27T11:24:27.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 shutdown_connections 2023-08-27T11:24:27.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fa04806a2e0 0x7fa04806c7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa050003f80 0x7fa0640c65a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0640b0980 0x7fa0640b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 --2- 172.21.15.45:0/1492223376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa06414c930 0x7fa064158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 >> 172.21.15.45:0/1492223376 conn(0x7fa0640cc7c0 msgr2=0x7fa0640cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 shutdown_connections 2023-08-27T11:24:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.387+0000 7fa06bc8c700 1 -- 172.21.15.45:0/1492223376 wait complete. 2023-08-27T11:24:27.400 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:55", "first": "2023-08-27T11:15:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 10, "pruned_count": 0, "active": false}] 2023-08-27T11:24:27.400 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:55', 'first': '2023-08-27T11:15:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 10, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:27.400 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 10 2023-08-27T11:24:27.401 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 10 2023-08-27T11:24:27.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.402 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-08-27T11:24:27.421 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:27.421 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:27.422 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:27.422 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:27.422 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:27.422 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:27.423 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:27.423 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:27.423 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:27.423 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:27.423 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:27.424 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_14_00_UTC', 'scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:27.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-08-27T11:24:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.590+0000 7fec63a4d700 1 Processor -- start 2023-08-27T11:24:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.591+0000 7fec63a4d700 1 -- start start 2023-08-27T11:24:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.591+0000 7fec63a4d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fec5c15d340 0x7fec5c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.591+0000 7fec63a4d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec5c06b160 con 0x7fec5c0cd880 2023-08-27T11:24:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.591+0000 7fec63a4d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec5c0af590 con 0x7fec5c06a990 2023-08-27T11:24:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.591+0000 7fec63a4d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec5c072010 con 0x7fec5c15d340 2023-08-27T11:24:27.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.591+0000 7fec60fe8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fec5c0cd880 0x7fec5c06a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41932/0 (socket says 172.21.15.45:41932) 2023-08-27T11:24:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.591+0000 7fec60fe8700 1 -- 172.21.15.45:0/2883322647 learned_addr learned my addr 172.21.15.45:0/2883322647 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec617e9700 1 --2- 172.21.15.45:0/2883322647 >> v2:172.21.15.135:3300/0 conn(0x7fec5c15d340 0x7fec5c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501261965 0 0) 0x7fec5c06b160 con 0x7fec5c0cd880 2023-08-27T11:24:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec480035b0 con 0x7fec5c0cd880 2023-08-27T11:24:27.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2667040773 0 0) 0x7fec480035b0 con 0x7fec5c0cd880 2023-08-27T11:24:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 >> v2:172.21.15.135:3300/0 conn(0x7fec5c15d340 msgr2=0x7fec5c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 --2- 172.21.15.45:0/2883322647 >> v2:172.21.15.135:3300/0 conn(0x7fec5c15d340 0x7fec5c160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 >> v1:172.21.15.135:6789/0 conn(0x7fec5c06a990 legacy=0x7fec5c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c0af2b0 con 0x7fec5c0cd880 2023-08-27T11:24:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fec50001670 con 0x7fec5c0cd880 2023-08-27T11:24:27.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 >> v1:172.21.15.45:6789/0 conn(0x7fec5c0cd880 legacy=0x7fec5c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec48003a00 0x7fec48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec48004850 0x7fec48004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c072010 con 0x7fec48005210 2023-08-27T11:24:27.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4800c1b0 con 0x7fec48003a00 2023-08-27T11:24:27.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4800c370 con 0x7fec48004850 2023-08-27T11:24:27.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec61fea700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec617e9700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec48003a00 0x7fec48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.592+0000 7fec60fe8700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec48004850 0x7fec48004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec61fea700 1 -- 172.21.15.45:0/2883322647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec48004850 msgr2=0x7fec48004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:27.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec61fea700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec48004850 0x7fec48004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec61fea700 1 -- 172.21.15.45:0/2883322647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec48003a00 msgr2=0x7fec48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:27.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec61fea700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec48003a00 0x7fec48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec61fea700 1 -- 172.21.15.45:0/2883322647 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec480035b0 con 0x7fec48005210 2023-08-27T11:24:27.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec60fe8700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec48004850 0x7fec48004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:27.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec61fea700 1 -- 172.21.15.45:0/2883322647 reap_dead start 2023-08-27T11:24:27.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec61fea700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec4800ba10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fec5400c160 tx=0x7fec48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191d87485ec38fdf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:27.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54001db0 con 0x7fec48005210 2023-08-27T11:24:27.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5400ccc0 con 0x7fec48005210 2023-08-27T11:24:27.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec4ffff700 1 -- 172.21.15.45:0/2883322647 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54003c40 con 0x7fec48005210 2023-08-27T11:24:27.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec63a4d700 1 -- 172.21.15.45:0/2883322647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 msgr2=0x7fec4800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec63a4d700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec4800ba10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fec5400c160 tx=0x7fec48005880 comp rx=0 tx=0).stop 2023-08-27T11:24:27.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec63a4d700 1 -- 172.21.15.45:0/2883322647 shutdown_connections 2023-08-27T11:24:27.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec63a4d700 1 --2- 172.21.15.45:0/2883322647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec4800ba10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.593+0000 7fec63a4d700 1 -- 172.21.15.45:0/2883322647 >> 172.21.15.45:0/2883322647 conn(0x7fec5c0cc7c0 msgr2=0x7fec5c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:27.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.594+0000 7fec63a4d700 1 -- 172.21.15.45:0/2883322647 shutdown_connections 2023-08-27T11:24:27.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.594+0000 7fec63a4d700 1 -- 172.21.15.45:0/2883322647 wait complete. 2023-08-27T11:24:27.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.595+0000 7fec63a4d700 1 Processor -- start 2023-08-27T11:24:27.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.595+0000 7fec63a4d700 1 -- start start 2023-08-27T11:24:27.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.595+0000 7fec63a4d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec5c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.595+0000 7fec63a4d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec5c1f6970 0x7fec5c151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec63a4d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec5c0c41f0 0x7fec5c0ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec63a4d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c0c8360 con 0x7fec48005210 2023-08-27T11:24:27.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec63a4d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c0ca630 con 0x7fec5c0c41f0 2023-08-27T11:24:27.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec63a4d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c0ca910 con 0x7fec5c1f6970 2023-08-27T11:24:27.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec617e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec5c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec617e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec5c0c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55166/0 (socket says 172.21.15.45:55166) 2023-08-27T11:24:27.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec617e9700 1 -- 172.21.15.45:0/1486115883 learned_addr learned my addr 172.21.15.45:0/1486115883 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:27.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec60fe8700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec5c1f6970 0x7fec5c151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec617e9700 1 -- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec5c1f6970 msgr2=0x7fec5c151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec617e9700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec5c1f6970 0x7fec5c151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec61fea700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec5c0c41f0 0x7fec5c0ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec617e9700 1 -- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec5c0c41f0 msgr2=0x7fec5c0ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec617e9700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec5c0c41f0 0x7fec5c0ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec617e9700 1 -- 172.21.15.45:0/1486115883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c0cabf0 con 0x7fec48005210 2023-08-27T11:24:27.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec617e9700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec5c0c9fe0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fec580119a0 tx=0x7fec58011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:27.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec4dffb700 1 -- 172.21.15.45:0/1486115883 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5800f940 con 0x7fec48005210 2023-08-27T11:24:27.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec4dffb700 1 -- 172.21.15.45:0/1486115883 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5800faa0 con 0x7fec48005210 2023-08-27T11:24:27.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5c0caed0 con 0x7fec48005210 2023-08-27T11:24:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec4dffb700 1 -- 172.21.15.45:0/1486115883 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5800fe30 con 0x7fec48005210 2023-08-27T11:24:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.596+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c1517c0 con 0x7fec48005210 2023-08-27T11:24:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.597+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5c0c8640 con 0x7fec48005210 2023-08-27T11:24:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.598+0000 7fec4dffb700 1 -- 172.21.15.45:0/1486115883 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fec58014440 con 0x7fec48005210 2023-08-27T11:24:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.598+0000 7fec4dffb700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fec4006a2e0 0x7fec4006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:27.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.598+0000 7fec4dffb700 1 -- 172.21.15.45:0/1486115883 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fec580892a0 con 0x7fec48005210 2023-08-27T11:24:27.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.599+0000 7fec60fe8700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fec4006a2e0 0x7fec4006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:27.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.599+0000 7fec60fe8700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fec4006a2e0 0x7fec4006c7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec50000f60 tx=0x7fec50001110 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:27.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.602+0000 7fec4dffb700 1 -- 172.21.15.45:0/1486115883 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fec58054480 con 0x7fec48005210 2023-08-27T11:24:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.831+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec5c0caa80 con 0x7fec4006a2e0 2023-08-27T11:24:27.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.840+0000 7fec4dffb700 1 -- 172.21.15.45:0/1486115883 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fec5c0caa80 con 0x7fec4006a2e0 2023-08-27T11:24:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:27.842 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:56", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:27.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.842+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fec4006a2e0 msgr2=0x7fec4006c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.842+0000 7fec63a4d700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fec4006a2e0 0x7fec4006c7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec50000f60 tx=0x7fec50001110 comp rx=0 tx=0).stop 2023-08-27T11:24:27.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 msgr2=0x7fec5c0c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:27.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec5c0c9fe0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fec580119a0 tx=0x7fec58011d60 comp rx=0 tx=0).stop 2023-08-27T11:24:27.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 shutdown_connections 2023-08-27T11:24:27.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fec4006a2e0 0x7fec4006c7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec48005210 0x7fec5c0c9fe0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec5c0c41f0 0x7fec5c0ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 --2- 172.21.15.45:0/1486115883 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fec5c1f6970 0x7fec5c151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:27.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 >> 172.21.15.45:0/1486115883 conn(0x7fec5c0cc7c0 msgr2=0x7fec5c0af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:27.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 shutdown_connections 2023-08-27T11:24:27.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:27.843+0000 7fec63a4d700 1 -- 172.21.15.45:0/1486115883 wait complete. 2023-08-27T11:24:27.855 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:56", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:27.856 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:56', 'first': '2023-08-27T11:14:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:27.856 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-27T11:24:27.856 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-27T11:24:27.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:27.857 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-08-27T11:24:27.875 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:27.875 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:27.876 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:27.876 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:27.876 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:27.876 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:27.876 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:27.877 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:27.877 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:27.877 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:27.877 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:27.877 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_14_00_UTC', 'scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:27.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-08-27T11:24:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.044+0000 7f5e41db7700 1 Processor -- start 2023-08-27T11:24:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.044+0000 7f5e41db7700 1 -- start start 2023-08-27T11:24:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e41db7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c15d3f0 0x7f5e3c160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e41db7700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5e3c0af420 0x7f5e3c06ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e41db7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c071de0 con 0x7f5e3c0aed60 2023-08-27T11:24:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e41db7700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0c6ca0 con 0x7f5e3c0af420 2023-08-27T11:24:28.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e41db7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c15d7d0 con 0x7f5e3c15d3f0 2023-08-27T11:24:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5e3c0aed60 0x7f5e3c161390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41950/0 (socket says 172.21.15.45:41950) 2023-08-27T11:24:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3affd700 1 -- 172.21.15.45:0/3248823128 learned_addr learned my addr 172.21.15.45:0/3248823128 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3b7fe700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c15d3f0 0x7f5e3c160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3bfff700 1 --2- 172.21.15.45:0/3248823128 >> v2:172.21.15.135:3300/0 conn(0x7f5e3c0af420 0x7f5e3c06ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219724083 0 0) 0x7f5e3c071de0 con 0x7f5e3c0aed60 2023-08-27T11:24:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e240035b0 con 0x7f5e3c0aed60 2023-08-27T11:24:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3927029328 0 0) 0x7f5e240035b0 con 0x7f5e3c0aed60 2023-08-27T11:24:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c15d3f0 msgr2=0x7f5e3c160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c15d3f0 0x7f5e3c160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 >> v2:172.21.15.135:3300/0 conn(0x7f5e3c0af420 msgr2=0x7f5e3c06ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 --2- 172.21.15.45:0/3248823128 >> v2:172.21.15.135:3300/0 conn(0x7f5e3c0af420 0x7f5e3c06ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0c5310 con 0x7f5e3c0aed60 2023-08-27T11:24:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f5e2c001670 con 0x7f5e3c0aed60 2023-08-27T11:24:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 >> v1:172.21.15.45:6789/0 conn(0x7f5e3c0aed60 legacy=0x7f5e3c161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c15d3f0 0x7f5e24003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e24003f80 0x7f5e24004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.045+0000 7f5e3a7fc700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e24004d90 0x7f5e24005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0c6ca0 con 0x7f5e24004d90 2023-08-27T11:24:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c15d7d0 con 0x7f5e24003f80 2023-08-27T11:24:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2400ba60 con 0x7f5e3c15d3f0 2023-08-27T11:24:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3bfff700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e24004d90 0x7f5e24005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3b7fe700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c15d3f0 0x7f5e24003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3affd700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e24003f80 0x7f5e24004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3bfff700 1 -- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c15d3f0 msgr2=0x7f5e24003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3bfff700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c15d3f0 0x7f5e24003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3bfff700 1 -- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e24003f80 msgr2=0x7f5e24004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:28.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3bfff700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e24003f80 0x7f5e24004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3bfff700 1 -- 172.21.15.45:0/3248823128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240035b0 con 0x7f5e24004d90 2023-08-27T11:24:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3bfff700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e24004d90 0x7f5e24005210 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400e170 tx=0x7f5e24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09c4187c2b6efc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400ec90 con 0x7f5e24004d90 2023-08-27T11:24:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3400f040 con 0x7f5e24004d90 2023-08-27T11:24:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.046+0000 7f5e3a7fc700 1 -- 172.21.15.45:0/3248823128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34014480 con 0x7f5e24004d90 2023-08-27T11:24:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.047+0000 7f5e41db7700 1 -- 172.21.15.45:0/3248823128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e24004d90 msgr2=0x7f5e24005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.047+0000 7f5e41db7700 1 --2- 172.21.15.45:0/3248823128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e24004d90 0x7f5e24005210 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400e170 tx=0x7f5e24005880 comp rx=0 tx=0).stop 2023-08-27T11:24:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.047+0000 7f5e3bfff700 1 -- 172.21.15.45:0/3248823128 reap_dead start 2023-08-27T11:24:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.047+0000 7f5e41db7700 1 -- 172.21.15.45:0/3248823128 shutdown_connections 2023-08-27T11:24:28.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.047+0000 7f5e41db7700 1 -- 172.21.15.45:0/3248823128 >> 172.21.15.45:0/3248823128 conn(0x7f5e3c0cfdd0 msgr2=0x7f5e3c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:28.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.047+0000 7f5e41db7700 1 -- 172.21.15.45:0/3248823128 shutdown_connections 2023-08-27T11:24:28.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.047+0000 7f5e41db7700 1 -- 172.21.15.45:0/3248823128 wait complete. 2023-08-27T11:24:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.048+0000 7f5e41db7700 1 Processor -- start 2023-08-27T11:24:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.049+0000 7f5e41db7700 1 -- start start 2023-08-27T11:24:28.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e41db7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e3c147800 0x7f5e3c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e41db7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c1562b0 0x7f5e3c0b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e41db7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3c0b3850 0x7f5e3c1f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e41db7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c1575b0 con 0x7f5e3c0b3850 2023-08-27T11:24:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e41db7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b3cd0 con 0x7f5e3c147800 2023-08-27T11:24:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e41db7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c156a30 con 0x7f5e3c1562b0 2023-08-27T11:24:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3c0b3850 0x7f5e3c1f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3c0b3850 0x7f5e3c1f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55190/0 (socket says 172.21.15.45:55190) 2023-08-27T11:24:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3bfff700 1 -- 172.21.15.45:0/4028475385 learned_addr learned my addr 172.21.15.45:0/4028475385 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3b7fe700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e3c147800 0x7f5e3c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3affd700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c1562b0 0x7f5e3c0b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3bfff700 1 -- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c1562b0 msgr2=0x7f5e3c0b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3bfff700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c1562b0 0x7f5e3c0b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3bfff700 1 -- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e3c147800 msgr2=0x7f5e3c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3bfff700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e3c147800 0x7f5e3c0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3bfff700 1 -- 172.21.15.45:0/4028475385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c156d10 con 0x7f5e3c0b3850 2023-08-27T11:24:28.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e3bfff700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3c0b3850 0x7f5e3c1f6750 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5e24004790 tx=0x7f5e34009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:28.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e23fff700 1 -- 172.21.15.45:0/4028475385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34007000 con 0x7f5e3c0b3850 2023-08-27T11:24:28.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e23fff700 1 -- 172.21.15.45:0/4028475385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3400ee80 con 0x7f5e3c0b3850 2023-08-27T11:24:28.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c156ff0 con 0x7f5e3c0b3850 2023-08-27T11:24:28.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e23fff700 1 -- 172.21.15.45:0/4028475385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34004490 con 0x7f5e3c0b3850 2023-08-27T11:24:28.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.050+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c156730 con 0x7f5e3c0b3850 2023-08-27T11:24:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.051+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c0727c0 con 0x7f5e3c0b3850 2023-08-27T11:24:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.052+0000 7f5e23fff700 1 -- 172.21.15.45:0/4028475385 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f5e340045f0 con 0x7f5e3c0b3850 2023-08-27T11:24:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.052+0000 7f5e23fff700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e2806a2e0 0x7f5e2806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.052+0000 7f5e23fff700 1 -- 172.21.15.45:0/4028475385 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f5e3408a090 con 0x7f5e3c0b3850 2023-08-27T11:24:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.053+0000 7f5e3b7fe700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e2806a2e0 0x7f5e2806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.053+0000 7f5e3b7fe700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e2806a2e0 0x7f5e2806c7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009ba0 tx=0x7f5e30008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:28.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.057+0000 7f5e23fff700 1 -- 172.21.15.45:0/4028475385 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5e34055200 con 0x7f5e3c0b3850 2023-08-27T11:24:28.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.285+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e3c161c60 con 0x7f5e2806a2e0 2023-08-27T11:24:28.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.294+0000 7f5e23fff700 1 -- 172.21.15.45:0/4028475385 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f5e300097f0 con 0x7f5e2806a2e0 2023-08-27T11:24:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:28.296 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:56", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e2806a2e0 msgr2=0x7f5e2806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e2806a2e0 0x7f5e2806c7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009ba0 tx=0x7f5e30008040 comp rx=0 tx=0).stop 2023-08-27T11:24:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3c0b3850 msgr2=0x7f5e3c1f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3c0b3850 0x7f5e3c1f6750 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5e24004790 tx=0x7f5e34009750 comp rx=0 tx=0).stop 2023-08-27T11:24:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 shutdown_connections 2023-08-27T11:24:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5e2806a2e0 0x7f5e2806c7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3c0b3850 0x7f5e3c1f6750 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e3c147800 0x7f5e3c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 --2- 172.21.15.45:0/4028475385 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5e3c1562b0 0x7f5e3c0b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 >> 172.21.15.45:0/4028475385 conn(0x7f5e3c0cfdd0 msgr2=0x7f5e3c15b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.296+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 shutdown_connections 2023-08-27T11:24:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.297+0000 7f5e41db7700 1 -- 172.21.15.45:0/4028475385 wait complete. 2023-08-27T11:24:28.309 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:56", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:28.310 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:56', 'first': '2023-08-27T11:14:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:28.310 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-27T11:24:28.310 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-27T11:24:28.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.311 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-08-27T11:24:28.329 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:28.330 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:28.330 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:28.330 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:28.331 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:28.331 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:28.331 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:28.332 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:28.332 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:28.332 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:28.333 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:28.333 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_14_00_UTC', 'scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:28.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-08-27T11:24:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.498+0000 7f90e77a9700 1 Processor -- start 2023-08-27T11:24:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.498+0000 7f90e77a9700 1 -- start start 2023-08-27T11:24:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e77a9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90e015d340 0x7f90e0160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e77a9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 0x7f90e006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e77a9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e006b160 con 0x7f90e00cdd00 2023-08-27T11:24:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e77a9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e0072010 con 0x7f90e015d340 2023-08-27T11:24:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e77a9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00af590 con 0x7f90e00cd880 2023-08-27T11:24:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e5545700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90e015d340 0x7f90e0160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e5545700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90e015d340 0x7f90e0160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55194/0 (socket says 172.21.15.45:55194) 2023-08-27T11:24:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e5d46700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f90e00cdd00 0x7f90e00b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41974/0 (socket says 172.21.15.45:41974) 2023-08-27T11:24:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e5545700 1 -- 172.21.15.45:0/3296364015 learned_addr learned my addr 172.21.15.45:0/3296364015 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e4d44700 1 --2- 172.21.15.45:0/3296364015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 0x7f90e006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e5545700 1 -- 172.21.15.45:0/3296364015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 msgr2=0x7f90e006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e5545700 1 --2- 172.21.15.45:0/3296364015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 0x7f90e006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e5545700 1 -- 172.21.15.45:0/3296364015 >> v1:172.21.15.45:6789/0 conn(0x7f90e00cdd00 legacy=0x7f90e00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90d7fff700 1 -- 172.21.15.45:0/3296364015 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703555119 0 0) 0x7f90e006b160 con 0x7f90e00cdd00 2023-08-27T11:24:28.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.499+0000 7f90e5545700 1 -- 172.21.15.45:0/3296364015 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e00af2b0 con 0x7f90e015d340 2023-08-27T11:24:28.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90e5545700 1 --2- 172.21.15.45:0/3296364015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e015d340 0x7f90e0160d00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f90dc0098f0 tx=0x7f90dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38034581dd845a55 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:28.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90d7fff700 1 -- 172.21.15.45:0/3296364015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00b8a0 con 0x7f90e015d340 2023-08-27T11:24:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90d7fff700 1 -- 172.21.15.45:0/3296364015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc00ba00 con 0x7f90e015d340 2023-08-27T11:24:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90d7fff700 1 -- 172.21.15.45:0/3296364015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00bd90 con 0x7f90e015d340 2023-08-27T11:24:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90e77a9700 1 -- 172.21.15.45:0/3296364015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e015d340 msgr2=0x7f90e0160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90e77a9700 1 --2- 172.21.15.45:0/3296364015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e015d340 0x7f90e0160d00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f90dc0098f0 tx=0x7f90dc009c00 comp rx=0 tx=0).stop 2023-08-27T11:24:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90e77a9700 1 -- 172.21.15.45:0/3296364015 shutdown_connections 2023-08-27T11:24:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90e77a9700 1 --2- 172.21.15.45:0/3296364015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e015d340 0x7f90e0160d00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90e77a9700 1 --2- 172.21.15.45:0/3296364015 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 0x7f90e006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90e77a9700 1 -- 172.21.15.45:0/3296364015 >> 172.21.15.45:0/3296364015 conn(0x7f90e00cc7c0 msgr2=0x7f90e00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90e77a9700 1 -- 172.21.15.45:0/3296364015 shutdown_connections 2023-08-27T11:24:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.500+0000 7f90e77a9700 1 -- 172.21.15.45:0/3296364015 wait complete. 2023-08-27T11:24:28.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e77a9700 1 Processor -- start 2023-08-27T11:24:28.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e77a9700 1 -- start start 2023-08-27T11:24:28.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e77a9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 0x7f90e0151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e77a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e00cdd00 0x7f90e00ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e77a9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015d340 0x7f90e00adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e77a9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00c6d80 con 0x7f90e00cdd00 2023-08-27T11:24:28.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e77a9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00c6f90 con 0x7f90e015d340 2023-08-27T11:24:28.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e77a9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00c7270 con 0x7f90e00cd880 2023-08-27T11:24:28.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e4d44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e00cdd00 0x7f90e00ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e4d44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e00cdd00 0x7f90e00ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55202/0 (socket says 172.21.15.45:55202) 2023-08-27T11:24:28.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.502+0000 7f90e4d44700 1 -- 172.21.15.45:0/2871080606 learned_addr learned my addr 172.21.15.45:0/2871080606 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:28.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e5545700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 0x7f90e0151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e5d46700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015d340 0x7f90e00adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e4d44700 1 -- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 msgr2=0x7f90e0151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e4d44700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 0x7f90e0151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e4d44700 1 -- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015d340 msgr2=0x7f90e00adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e4d44700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015d340 0x7f90e00adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e4d44700 1 -- 172.21.15.45:0/2871080606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc009550 con 0x7f90e00cdd00 2023-08-27T11:24:28.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e5d46700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015d340 0x7f90e00adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:28.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e4d44700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e00cdd00 0x7f90e00ada30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f90d000d800 tx=0x7f90d000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:28.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90d5ffb700 1 -- 172.21.15.45:0/2871080606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d00125f0 con 0x7f90e00cdd00 2023-08-27T11:24:28.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90d5ffb700 1 -- 172.21.15.45:0/2871080606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d0012750 con 0x7f90e00cdd00 2023-08-27T11:24:28.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e00c7550 con 0x7f90e00cdd00 2023-08-27T11:24:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90d5ffb700 1 -- 172.21.15.45:0/2871080606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0012b00 con 0x7f90e00cdd00 2023-08-27T11:24:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.503+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e01f39b0 con 0x7f90e00cdd00 2023-08-27T11:24:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.504+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e00aef10 con 0x7f90e00cdd00 2023-08-27T11:24:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.505+0000 7f90d5ffb700 1 -- 172.21.15.45:0/2871080606 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f90d0012c60 con 0x7f90e00cdd00 2023-08-27T11:24:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.505+0000 7f90d5ffb700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f90c406a2e0 0x7f90c406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.505+0000 7f90d5ffb700 1 -- 172.21.15.45:0/2871080606 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f90d0088f00 con 0x7f90e00cdd00 2023-08-27T11:24:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.505+0000 7f90e5545700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f90c406a2e0 0x7f90c406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.506+0000 7f90e5545700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f90c406a2e0 0x7f90c406c7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f90dc000c00 tx=0x7f90dc008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.509+0000 7f90d5ffb700 1 -- 172.21.15.45:0/2871080606 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f90e00aef10 con 0x7f90e00cdd00 2023-08-27T11:24:28.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.735+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90e0071c30 con 0x7f90c406a2e0 2023-08-27T11:24:28.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.744+0000 7f90d5ffb700 1 -- 172.21.15.45:0/2871080606 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f90dc00bb70 con 0x7f90c406a2e0 2023-08-27T11:24:28.746 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:28.746 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:57", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:28.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f90c406a2e0 msgr2=0x7f90c406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f90c406a2e0 0x7f90c406c7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f90dc000c00 tx=0x7f90dc008030 comp rx=0 tx=0).stop 2023-08-27T11:24:28.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e00cdd00 msgr2=0x7f90e00ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e00cdd00 0x7f90e00ada30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f90d000d800 tx=0x7f90d000dbc0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 shutdown_connections 2023-08-27T11:24:28.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f90c406a2e0 0x7f90c406c7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90e015d340 0x7f90e00adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f90e00cd880 0x7f90e0151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 --2- 172.21.15.45:0/2871080606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e00cdd00 0x7f90e00ada30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 >> 172.21.15.45:0/2871080606 conn(0x7f90e00cc7c0 msgr2=0x7f90e015e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 shutdown_connections 2023-08-27T11:24:28.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.746+0000 7f90e77a9700 1 -- 172.21.15.45:0/2871080606 wait complete. 2023-08-27T11:24:28.759 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:57", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:28.760 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:57', 'first': '2023-08-27T11:14:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:28.760 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-27T11:24:28.760 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-27T11:24:28.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:28.761 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-08-27T11:24:28.779 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:28.780 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:28.780 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:28.780 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:28.780 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:28.780 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:28.781 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:28.781 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:28.781 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:28.781 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:28.781 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:28.782 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-27-11_14_00_UTC', 'scheduled-2023-08-27-11_15_00_UTC', 'scheduled-2023-08-27-11_16_00_UTC', 'scheduled-2023-08-27-11_17_00_UTC', 'scheduled-2023-08-27-11_18_00_UTC', 'scheduled-2023-08-27-11_19_00_UTC', 'scheduled-2023-08-27-11_20_00_UTC', 'scheduled-2023-08-27-11_21_00_UTC', 'scheduled-2023-08-27-11_22_00_UTC', 'scheduled-2023-08-27-11_23_00_UTC', 'scheduled-2023-08-27-11_24_00_UTC'] 2023-08-27T11:24:28.782 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-08-27T11:24:28.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.948+0000 7fc087ff2700 1 Processor -- start 2023-08-27T11:24:28.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.948+0000 7fc087ff2700 1 -- start start 2023-08-27T11:24:28.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc087ff2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0800aea00 0x7fc080071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc087ff2700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc080071eb0 0x7fc08006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc087ff2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0800b3220 con 0x7fc08015a220 2023-08-27T11:24:28.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc087ff2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc080072340 con 0x7fc0800aea00 2023-08-27T11:24:28.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc087ff2700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0800b2c60 con 0x7fc080071eb0 2023-08-27T11:24:28.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc08558d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0800aea00 0x7fc080071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc085d8e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc08015a220 0x7fc0800cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41984/0 (socket says 172.21.15.45:41984) 2023-08-27T11:24:28.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc085d8e700 1 -- 172.21.15.45:0/2409751208 learned_addr learned my addr 172.21.15.45:0/2409751208 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:28.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc08658f700 1 --2- 172.21.15.45:0/2409751208 >> v2:172.21.15.135:3300/0 conn(0x7fc080071eb0 0x7fc08006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc08558d700 1 -- 172.21.15.45:0/2409751208 >> v2:172.21.15.135:3300/0 conn(0x7fc080071eb0 msgr2=0x7fc08006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:28.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc08558d700 1 --2- 172.21.15.45:0/2409751208 >> v2:172.21.15.135:3300/0 conn(0x7fc080071eb0 0x7fc08006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc08558d700 1 -- 172.21.15.45:0/2409751208 >> v1:172.21.15.45:6789/0 conn(0x7fc08015a220 legacy=0x7fc0800cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc084d8c700 1 -- 172.21.15.45:0/2409751208 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876694410 0 0) 0x7fc0800b3220 con 0x7fc08015a220 2023-08-27T11:24:28.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.949+0000 7fc08558d700 1 -- 172.21.15.45:0/2409751208 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0800b2f40 con 0x7fc0800aea00 2023-08-27T11:24:28.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc08558d700 1 --2- 172.21.15.45:0/2409751208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0800aea00 0x7fc080071970 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc080073890 tx=0x7fc07c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911b320ddc7c026f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:28.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc084d8c700 1 -- 172.21.15.45:0/2409751208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c011830 con 0x7fc0800aea00 2023-08-27T11:24:28.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc084d8c700 1 -- 172.21.15.45:0/2409751208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c011990 con 0x7fc0800aea00 2023-08-27T11:24:28.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc084d8c700 1 -- 172.21.15.45:0/2409751208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c011d20 con 0x7fc0800aea00 2023-08-27T11:24:28.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc087ff2700 1 -- 172.21.15.45:0/2409751208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0800aea00 msgr2=0x7fc080071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc087ff2700 1 --2- 172.21.15.45:0/2409751208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0800aea00 0x7fc080071970 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc080073890 tx=0x7fc07c009b20 comp rx=0 tx=0).stop 2023-08-27T11:24:28.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc087ff2700 1 -- 172.21.15.45:0/2409751208 shutdown_connections 2023-08-27T11:24:28.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc087ff2700 1 --2- 172.21.15.45:0/2409751208 >> v2:172.21.15.135:3300/0 conn(0x7fc080071eb0 0x7fc08006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc087ff2700 1 --2- 172.21.15.45:0/2409751208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0800aea00 0x7fc080071970 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc087ff2700 1 -- 172.21.15.45:0/2409751208 >> 172.21.15.45:0/2409751208 conn(0x7fc0800b4650 msgr2=0x7fc0800b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:28.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc087ff2700 1 -- 172.21.15.45:0/2409751208 shutdown_connections 2023-08-27T11:24:28.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.950+0000 7fc087ff2700 1 -- 172.21.15.45:0/2409751208 wait complete. 2023-08-27T11:24:28.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc087ff2700 1 Processor -- start 2023-08-27T11:24:28.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc087ff2700 1 -- start start 2023-08-27T11:24:28.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc087ff2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc080071eb0 0x7fc080098300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc087ff2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800aea00 0x7fc0801582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc087ff2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc08015a220 0x7fc080158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc087ff2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08015cb30 con 0x7fc08015a220 2023-08-27T11:24:28.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc087ff2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc080098840 con 0x7fc080071eb0 2023-08-27T11:24:28.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc087ff2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc080156c60 con 0x7fc0800aea00 2023-08-27T11:24:28.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc08658f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc08015a220 0x7fc080158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.952+0000 7fc08658f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc08015a220 0x7fc080158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55218/0 (socket says 172.21.15.45:55218) 2023-08-27T11:24:28.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc08658f700 1 -- 172.21.15.45:0/2952054102 learned_addr learned my addr 172.21.15.45:0/2952054102 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:28.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc085d8e700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc080071eb0 0x7fc080098300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc08558d700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800aea00 0x7fc0801582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc08658f700 1 -- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800aea00 msgr2=0x7fc0801582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc08658f700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800aea00 0x7fc0801582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc08658f700 1 -- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc080071eb0 msgr2=0x7fc080098300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc08658f700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc080071eb0 0x7fc080098300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc08658f700 1 -- 172.21.15.45:0/2952054102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0094a0 con 0x7fc08015a220 2023-08-27T11:24:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc08558d700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800aea00 0x7fc0801582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc08658f700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc08015a220 0x7fc080158800 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc078007e70 tx=0x7fc07800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc0767fc700 1 -- 172.21.15.45:0/2952054102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800ee50 con 0x7fc08015a220 2023-08-27T11:24:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc0767fc700 1 -- 172.21.15.45:0/2952054102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc078014420 con 0x7fc08015a220 2023-08-27T11:24:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc0767fc700 1 -- 172.21.15.45:0/2952054102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780147d0 con 0x7fc08015a220 2023-08-27T11:24:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc080098cd0 con 0x7fc08015a220 2023-08-27T11:24:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.953+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08015c190 con 0x7fc08015a220 2023-08-27T11:24:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.954+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0800ccf10 con 0x7fc08015a220 2023-08-27T11:24:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.955+0000 7fc0767fc700 1 -- 172.21.15.45:0/2952054102 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fc078014930 con 0x7fc08015a220 2023-08-27T11:24:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.955+0000 7fc0767fc700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc06406a2e0 0x7fc06406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.955+0000 7fc0767fc700 1 -- 172.21.15.45:0/2952054102 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fc0780892b0 con 0x7fc08015a220 2023-08-27T11:24:28.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.955+0000 7fc085d8e700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc06406a2e0 0x7fc06406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.956+0000 7fc085d8e700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc06406a2e0 0x7fc06406c7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc070000f60 tx=0x7fc070007c90 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:28.959+0000 7fc0767fc700 1 -- 172.21.15.45:0/2952054102 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc078054530 con 0x7fc08015a220 2023-08-27T11:24:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.184+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0800cef50 con 0x7fc06406a2e0 2023-08-27T11:24:29.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.194+0000 7fc0767fc700 1 -- 172.21.15.45:0/2952054102 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fc070013420 con 0x7fc06406a2e0 2023-08-27T11:24:29.195 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:29.195 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:57", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:29.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc06406a2e0 msgr2=0x7fc06406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:29.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc06406a2e0 0x7fc06406c7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc070000f60 tx=0x7fc070007c90 comp rx=0 tx=0).stop 2023-08-27T11:24:29.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc08015a220 msgr2=0x7fc080158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:29.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc08015a220 0x7fc080158800 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc078007e70 tx=0x7fc07800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 shutdown_connections 2023-08-27T11:24:29.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc06406a2e0 0x7fc06406c7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc08015a220 0x7fc080158800 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc080071eb0 0x7fc080098300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 --2- 172.21.15.45:0/2952054102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0800aea00 0x7fc0801582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 >> 172.21.15.45:0/2952054102 conn(0x7fc0800b4650 msgr2=0x7fc0800ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 shutdown_connections 2023-08-27T11:24:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.196+0000 7fc087ff2700 1 -- 172.21.15.45:0/2952054102 wait complete. 2023-08-27T11:24:29.209 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:13:57", "first": "2023-08-27T11:14:00", "last": "2023-08-27T11:24:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-27T11:24:29.209 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-08-27T00:00:00', 'created': '2023-08-27T11:13:57', 'first': '2023-08-27T11:14:00', 'last': '2023-08-27T11:24:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-27T11:24:29.210 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-27T11:24:29.210 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-27T11:24:29.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-08-27T11:24:29.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.340+0000 7fd72a36a700 1 Processor -- start 2023-08-27T11:24:29.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.340+0000 7fd72a36a700 1 -- start start 2023-08-27T11:24:29.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd72a36a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd7240aefb0 0x7fd7240ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:29.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd72a36a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72406b010 con 0x7fd72415d4f0 2023-08-27T11:24:29.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd72a36a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7240ad9f0 con 0x7fd7240cacd0 2023-08-27T11:24:29.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd72a36a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72415d8d0 con 0x7fd7240aefb0 2023-08-27T11:24:29.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd723fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd72415d4f0 0x7fd7240ca5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41998/0 (socket says 172.21.15.45:41998) 2023-08-27T11:24:29.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd723fff700 1 -- 172.21.15.45:0/2793889303 learned_addr learned my addr 172.21.15.45:0/2793889303 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:29.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003526203 0 0) 0x7fd72406b010 con 0x7fd72415d4f0 2023-08-27T11:24:29.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd728907700 1 --2- 172.21.15.45:0/2793889303 >> v2:172.21.15.135:3300/0 conn(0x7fd7240aefb0 0x7fd7240ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:29.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd70c0035b0 con 0x7fd72415d4f0 2023-08-27T11:24:29.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285466395 0 0) 0x7fd70c0035b0 con 0x7fd72415d4f0 2023-08-27T11:24:29.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 >> v2:172.21.15.135:3300/0 conn(0x7fd7240aefb0 msgr2=0x7fd7240ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:29.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 --2- 172.21.15.45:0/2793889303 >> v2:172.21.15.135:3300/0 conn(0x7fd7240aefb0 0x7fd7240ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 >> v1:172.21.15.135:6789/0 conn(0x7fd7240cacd0 legacy=0x7fd7240cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:29.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72406ad30 con 0x7fd72415d4f0 2023-08-27T11:24:29.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fd714001670 con 0x7fd72415d4f0 2023-08-27T11:24:29.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 >> v1:172.21.15.45:6789/0 conn(0x7fd72415d4f0 legacy=0x7fd7240ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:29.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd70c003a00 0x7fd70c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.341+0000 7fd722ffd700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd70c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd722ffd700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd70c005330 0x7fd70c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72415d8d0 con 0x7fd70c004850 2023-08-27T11:24:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c00c200 con 0x7fd70c005330 2023-08-27T11:24:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c00c3c0 con 0x7fd70c003a00 2023-08-27T11:24:29.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd7237fe700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd70c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:29.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd728907700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd70c005330 0x7fd70c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:29.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd723fff700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd70c003a00 0x7fd70c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:29.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd7237fe700 1 -- 172.21.15.45:0/2793889303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd70c003a00 msgr2=0x7fd70c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd7237fe700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd70c003a00 0x7fd70c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd7237fe700 1 -- 172.21.15.45:0/2793889303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd70c005330 msgr2=0x7fd70c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd7237fe700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd70c005330 0x7fd70c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd7237fe700 1 -- 172.21.15.45:0/2793889303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c0035b0 con 0x7fd70c004850 2023-08-27T11:24:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd728907700 1 -- 172.21.15.45:0/2793889303 reap_dead start 2023-08-27T11:24:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd7237fe700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd70c004cd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd718007fd0 tx=0x7fd70c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d74df7b52252c11 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180052d0 con 0x7fd70c004850 2023-08-27T11:24:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd718005450 con 0x7fd70c004850 2023-08-27T11:24:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.342+0000 7fd722ffd700 1 -- 172.21.15.45:0/2793889303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180038c0 con 0x7fd70c004850 2023-08-27T11:24:29.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.343+0000 7fd72a36a700 1 -- 172.21.15.45:0/2793889303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 msgr2=0x7fd70c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:29.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.343+0000 7fd72a36a700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd70c004cd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd718007fd0 tx=0x7fd70c005880 comp rx=0 tx=0).stop 2023-08-27T11:24:29.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.343+0000 7fd72a36a700 1 -- 172.21.15.45:0/2793889303 shutdown_connections 2023-08-27T11:24:29.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.343+0000 7fd72a36a700 1 --2- 172.21.15.45:0/2793889303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd70c004cd0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.343+0000 7fd72a36a700 1 -- 172.21.15.45:0/2793889303 >> 172.21.15.45:0/2793889303 conn(0x7fd7240cc7e0 msgr2=0x7fd7240ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:29.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.343+0000 7fd72a36a700 1 -- 172.21.15.45:0/2793889303 shutdown_connections 2023-08-27T11:24:29.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.343+0000 7fd72a36a700 1 -- 172.21.15.45:0/2793889303 wait complete. 2023-08-27T11:24:29.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.344+0000 7fd72a36a700 1 Processor -- start 2023-08-27T11:24:29.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.344+0000 7fd72a36a700 1 -- start start 2023-08-27T11:24:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd72a36a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd72414da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd72a36a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd72415da10 0x7fd72414d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:29.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd72a36a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7241502d0 0x7fd724150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:29.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd72a36a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240c4a20 con 0x7fd70c004850 2023-08-27T11:24:29.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd72a36a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd724160120 con 0x7fd72415da10 2023-08-27T11:24:29.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd72a36a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd724150c90 con 0x7fd7241502d0 2023-08-27T11:24:29.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd723fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd72414da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:29.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd723fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd72414da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55234/0 (socket says 172.21.15.45:55234) 2023-08-27T11:24:29.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd723fff700 1 -- 172.21.15.45:0/3610638479 learned_addr learned my addr 172.21.15.45:0/3610638479 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:29.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd7237fe700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd72415da10 0x7fd72414d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:29.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd728907700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7241502d0 0x7fd724150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:29.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd723fff700 1 -- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7241502d0 msgr2=0x7fd724150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:29.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd723fff700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7241502d0 0x7fd724150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd723fff700 1 -- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd72415da10 msgr2=0x7fd72414d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:29.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd723fff700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd72415da10 0x7fd72414d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd723fff700 1 -- 172.21.15.45:0/3610638479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd724160930 con 0x7fd70c004850 2023-08-27T11:24:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.345+0000 7fd728907700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7241502d0 0x7fd724150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.346+0000 7fd723fff700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd72414da10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd7140072f0 tx=0x7fd714007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.346+0000 7fd720ff9700 1 -- 172.21.15.45:0/3610638479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400f6c0 con 0x7fd70c004850 2023-08-27T11:24:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.346+0000 7fd720ff9700 1 -- 172.21.15.45:0/3610638479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71400f820 con 0x7fd70c004850 2023-08-27T11:24:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.346+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd724160370 con 0x7fd70c004850 2023-08-27T11:24:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.346+0000 7fd720ff9700 1 -- 172.21.15.45:0/3610638479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400fbd0 con 0x7fd70c004850 2023-08-27T11:24:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.346+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72415de90 con 0x7fd70c004850 2023-08-27T11:24:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.347+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72415e020 con 0x7fd70c004850 2023-08-27T11:24:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.347+0000 7fd720ff9700 1 -- 172.21.15.45:0/3610638479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fd71400fd30 con 0x7fd70c004850 2023-08-27T11:24:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.348+0000 7fd720ff9700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fd71006a2e0 0x7fd71006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.348+0000 7fd720ff9700 1 -- 172.21.15.45:0/3610638479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fd714088440 con 0x7fd70c004850 2023-08-27T11:24:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.348+0000 7fd7237fe700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fd71006a2e0 0x7fd71006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:29.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.348+0000 7fd7237fe700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fd71006a2e0 0x7fd71006c7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd718005040 tx=0x7fd718009030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:29.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.352+0000 7fd720ff9700 1 -- 172.21.15.45:0/3610638479 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd72415e020 con 0x7fd70c004850 2023-08-27T11:24:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.577+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7240adf70 con 0x7fd71006a2e0 2023-08-27T11:24:29.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.650+0000 7fd720ff9700 1 -- 172.21.15.45:0/3610638479 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fd718022070 con 0x7fd71006a2e0 2023-08-27T11:24:29.651 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-08-27T11:24:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fd71006a2e0 msgr2=0x7fd71006c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fd71006a2e0 0x7fd71006c7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd718005040 tx=0x7fd718009030 comp rx=0 tx=0).stop 2023-08-27T11:24:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 msgr2=0x7fd72414da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd72414da10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd7140072f0 tx=0x7fd714007600 comp rx=0 tx=0).stop 2023-08-27T11:24:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 shutdown_connections 2023-08-27T11:24:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fd71006a2e0 0x7fd71006c7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70c004850 0x7fd72414da10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd7241502d0 0x7fd724150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 --2- 172.21.15.45:0/3610638479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd72415da10 0x7fd72414d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 >> 172.21.15.45:0/3610638479 conn(0x7fd7240cc7e0 msgr2=0x7fd7240ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 shutdown_connections 2023-08-27T11:24:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:29.652+0000 7fd72a36a700 1 -- 172.21.15.45:0/3610638479 wait complete. 2023-08-27T11:24:29.665 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-08-27T11:24:29.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.666 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-08-27T11:24:29.684 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:29.684 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:29.684 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:29.685 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:29.685 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:29.685 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:29.685 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:29.686 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:29.686 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:29.686 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:29.686 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:29.686 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:29.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.687 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_14_00_UTC) 2023-08-27T11:24:29.748 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:29.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.749 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:29.811 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:29.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.812 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:29.875 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:29.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.876 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:29.938 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:29.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:29.939 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:30.002 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:30.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.003 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:30.067 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:30.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.067 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:30.130 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:30.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.131 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:30.194 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:30.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.195 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:30.259 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:30.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.260 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:30.321 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:30.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.322 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:30.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.386 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-08-27T11:24:30.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-08-27T11:24:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.615+0000 7f331a0b7700 1 Processor -- start 2023-08-27T11:24:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.615+0000 7f331a0b7700 1 -- start start 2023-08-27T11:24:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f331a0b7700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f33140ca620 0x7f33140cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f331a0b7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140caa00 0x7f33140acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f331a0b7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33140ce340 con 0x7f33140af200 2023-08-27T11:24:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f331a0b7700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33140ceec0 con 0x7f33140ca620 2023-08-27T11:24:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f331a0b7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331415d7c0 con 0x7f33140caa00 2023-08-27T11:24:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f3312ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f33140af200 0x7f33140ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42018/0 (socket says 172.21.15.45:42018) 2023-08-27T11:24:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f3312ffd700 1 -- 172.21.15.45:0/1412933310 learned_addr learned my addr 172.21.15.45:0/1412933310 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f33137fe700 1 --2- 172.21.15.45:0/1412933310 >> v2:172.21.15.135:3300/0 conn(0x7f33140ca620 0x7f33140cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:30.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f3313fff700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140caa00 0x7f33140acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:30.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297504944 0 0) 0x7f33140ce340 con 0x7f33140af200 2023-08-27T11:24:30.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.616+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33000035b0 con 0x7f33140af200 2023-08-27T11:24:30.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3330219070 0 0) 0x7f33000035b0 con 0x7f33140af200 2023-08-27T11:24:30.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140caa00 msgr2=0x7f33140acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:30.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140caa00 0x7f33140acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 >> v2:172.21.15.135:3300/0 conn(0x7f33140ca620 msgr2=0x7f33140cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:30.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 --2- 172.21.15.45:0/1412933310 >> v2:172.21.15.135:3300/0 conn(0x7f33140ca620 0x7f33140cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33140cc570 con 0x7f33140af200 2023-08-27T11:24:30.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f32fc001670 con 0x7f33140af200 2023-08-27T11:24:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 >> v1:172.21.15.45:6789/0 conn(0x7f33140af200 legacy=0x7f33140ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3300003a00 0x7f3300003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3300004850 0x7f3300004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140caa00 0x7f3300005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33140ceec0 con 0x7f3300004850 2023-08-27T11:24:30.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331415d7c0 con 0x7f3300003a00 2023-08-27T11:24:30.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330000ba10 con 0x7f33140caa00 2023-08-27T11:24:30.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f3312ffd700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3300004850 0x7f3300004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:30.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33137fe700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3300003a00 0x7f3300003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:30.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f3313fff700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140caa00 0x7f3300005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:30.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f3312ffd700 1 -- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140caa00 msgr2=0x7f3300005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:30.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f3312ffd700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140caa00 0x7f3300005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f3312ffd700 1 -- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3300003a00 msgr2=0x7f3300003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:30.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f3312ffd700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3300003a00 0x7f3300003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f3312ffd700 1 -- 172.21.15.45:0/1412933310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33000035b0 con 0x7f3300004850 2023-08-27T11:24:30.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f3312ffd700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3300004850 0x7f3300004cd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f33140af640 tx=0x7f3300005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6345cc5669bec0ee server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.617+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc007c60 con 0x7f3300004850 2023-08-27T11:24:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.618+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32fc00ce80 con 0x7f3300004850 2023-08-27T11:24:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.618+0000 7f33127fc700 1 -- 172.21.15.45:0/1412933310 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc005ba0 con 0x7f3300004850 2023-08-27T11:24:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.618+0000 7f331a0b7700 1 -- 172.21.15.45:0/1412933310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3300004850 msgr2=0x7f3300004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.618+0000 7f331a0b7700 1 --2- 172.21.15.45:0/1412933310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3300004850 0x7f3300004cd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f33140af640 tx=0x7f3300005880 comp rx=0 tx=0).stop 2023-08-27T11:24:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.618+0000 7f3313fff700 1 -- 172.21.15.45:0/1412933310 reap_dead start 2023-08-27T11:24:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.618+0000 7f331a0b7700 1 -- 172.21.15.45:0/1412933310 shutdown_connections 2023-08-27T11:24:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.618+0000 7f331a0b7700 1 -- 172.21.15.45:0/1412933310 >> 172.21.15.45:0/1412933310 conn(0x7f33140cc7e0 msgr2=0x7f33140ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.618+0000 7f331a0b7700 1 -- 172.21.15.45:0/1412933310 shutdown_connections 2023-08-27T11:24:30.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.618+0000 7f331a0b7700 1 -- 172.21.15.45:0/1412933310 wait complete. 2023-08-27T11:24:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.619+0000 7f331a0b7700 1 Processor -- start 2023-08-27T11:24:30.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f331a0b7700 1 -- start start 2023-08-27T11:24:30.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f331a0b7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3300004850 0x7f33140b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:30.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f331a0b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3314148f50 0x7f33140c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:30.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f331a0b7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140b48a0 0x7f33140b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:30.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f331a0b7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33141f3380 con 0x7f3314148f50 2023-08-27T11:24:30.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f331a0b7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33140b5260 con 0x7f3300004850 2023-08-27T11:24:30.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f331a0b7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33140c6e30 con 0x7f33140b48a0 2023-08-27T11:24:30.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f3312ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3314148f50 0x7f33140c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:30.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f3312ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3314148f50 0x7f33140c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55256/0 (socket says 172.21.15.45:55256) 2023-08-27T11:24:30.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.620+0000 7f3312ffd700 1 -- 172.21.15.45:0/926497724 learned_addr learned my addr 172.21.15.45:0/926497724 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:30.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f3312ffd700 1 -- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140b48a0 msgr2=0x7f33140b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:30.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f33137fe700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3300004850 0x7f33140b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:30.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f3312ffd700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140b48a0 0x7f33140b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f3312ffd700 1 -- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3300004850 msgr2=0x7f33140b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f3312ffd700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3300004850 0x7f33140b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f3312ffd700 1 -- 172.21.15.45:0/926497724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33140c7110 con 0x7f3314148f50 2023-08-27T11:24:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f3312ffd700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3314148f50 0x7f33140c9680 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c930 tx=0x7f32fc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f330bfff700 1 -- 172.21.15.45:0/926497724 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0070d0 con 0x7f3314148f50 2023-08-27T11:24:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f330bfff700 1 -- 172.21.15.45:0/926497724 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32fc007250 con 0x7f3314148f50 2023-08-27T11:24:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f330bfff700 1 -- 172.21.15.45:0/926497724 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc007600 con 0x7f3314148f50 2023-08-27T11:24:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33140b39b0 con 0x7f3314148f50 2023-08-27T11:24:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.621+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33140b3f70 con 0x7f3314148f50 2023-08-27T11:24:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.622+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33140c72a0 con 0x7f3314148f50 2023-08-27T11:24:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.623+0000 7f330bfff700 1 -- 172.21.15.45:0/926497724 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f32fc01e430 con 0x7f3314148f50 2023-08-27T11:24:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.623+0000 7f330bfff700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f32f406a2e0 0x7f32f406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.623+0000 7f330bfff700 1 -- 172.21.15.45:0/926497724 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f32fc08a160 con 0x7f3314148f50 2023-08-27T11:24:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.623+0000 7f33137fe700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f32f406a2e0 0x7f32f406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.624+0000 7f33137fe700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f32f406a2e0 0x7f32f406c7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3304004530 tx=0x7f3304008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.627+0000 7f330bfff700 1 -- 172.21.15.45:0/926497724 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f32fc055490 con 0x7f3314148f50 2023-08-27T11:24:30.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.853+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f331415c2a0 con 0x7f32f406a2e0 2023-08-27T11:24:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.931+0000 7f330bfff700 1 -- 172.21.15.45:0/926497724 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f331415c2a0 con 0x7f32f406a2e0 2023-08-27T11:24:30.933 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-08-27T11:24:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f32f406a2e0 msgr2=0x7f32f406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f32f406a2e0 0x7f32f406c7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3304004530 tx=0x7f3304008040 comp rx=0 tx=0).stop 2023-08-27T11:24:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3314148f50 msgr2=0x7f33140c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3314148f50 0x7f33140c9680 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c930 tx=0x7f32fc000fe0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 shutdown_connections 2023-08-27T11:24:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f32f406a2e0 0x7f32f406c7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33140b48a0 0x7f33140b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3300004850 0x7f33140b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 --2- 172.21.15.45:0/926497724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3314148f50 0x7f33140c9680 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.933+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 >> 172.21.15.45:0/926497724 conn(0x7f33140cc7e0 msgr2=0x7f331406ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.934+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 shutdown_connections 2023-08-27T11:24:30.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:30.934+0000 7f331a0b7700 1 -- 172.21.15.45:0/926497724 wait complete. 2023-08-27T11:24:30.946 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-08-27T11:24:30.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.947 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-08-27T11:24:30.965 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:30.966 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:30.966 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:30.966 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:30.966 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:30.966 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:30.967 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:30.967 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:30.967 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:30.967 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:30.967 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:30.968 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:30.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:30.968 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_14_00_UTC) 2023-08-27T11:24:31.030 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:31.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.031 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:31.093 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:31.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.094 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:31.156 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:31.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.157 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:31.220 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:31.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.221 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:31.282 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:31.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.283 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:31.345 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:31.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.346 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:31.410 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:31.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.411 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:31.474 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:31.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.475 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:31.537 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:31.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.538 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:31.603 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:31.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.604 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:31.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:31.667 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-08-27T11:24:31.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-08-27T11:24:31.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.898+0000 7fb6af0a9700 1 Processor -- start 2023-08-27T11:24:31.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.899+0000 7fb6af0a9700 1 -- start start 2023-08-27T11:24:31.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.899+0000 7fb6af0a9700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb6a80b8c20 0x7fb6a80cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:31.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.899+0000 7fb6af0a9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80ce2e0 0x7fb6a80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:31.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.899+0000 7fb6af0a9700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a815d6a0 con 0x7fb6a815d2c0 2023-08-27T11:24:31.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.899+0000 7fb6af0a9700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a80ce770 con 0x7fb6a80b8c20 2023-08-27T11:24:31.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.899+0000 7fb6af0a9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80d2c90 con 0x7fb6a80ce2e0 2023-08-27T11:24:31.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6a7fff700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb6a80b8c20 0x7fb6a80cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:31.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6ad646700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80ce2e0 0x7fb6a80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:31.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6a7fff700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb6a80b8c20 0x7fb6a80cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:44120/0 (socket says 172.21.15.45:44120) 2023-08-27T11:24:31.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6a7fff700 1 -- 172.21.15.45:0/2004410683 learned_addr learned my addr 172.21.15.45:0/2004410683 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:31.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6a7fff700 1 -- 172.21.15.45:0/2004410683 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80ce2e0 msgr2=0x7fb6a80b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:31.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6a7fff700 1 --2- 172.21.15.45:0/2004410683 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80ce2e0 0x7fb6a80b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:31.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6a7fff700 1 -- 172.21.15.45:0/2004410683 >> v1:172.21.15.135:6789/0 conn(0x7fb6a815d2c0 legacy=0x7fb6a80d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:31.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6a7fff700 1 -- 172.21.15.45:0/2004410683 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80d1bf0 con 0x7fb6a80b8c20 2023-08-27T11:24:31.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6a77fe700 1 -- 172.21.15.45:0/2004410683 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874364961 0 0) 0x7fb69c001110 con 0x7fb6a815d2c0 2023-08-27T11:24:31.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6ad646700 1 --2- 172.21.15.45:0/2004410683 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80ce2e0 0x7fb6a80b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:31.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.900+0000 7fb6a7fff700 1 --2- 172.21.15.45:0/2004410683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6a80b8c20 0x7fb6a80cdda0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb694009920 tx=0x7fb694009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae13c7a07115a45 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:31.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6a77fe700 1 -- 172.21.15.45:0/2004410683 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694011830 con 0x7fb6a80b8c20 2023-08-27T11:24:31.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6a77fe700 1 -- 172.21.15.45:0/2004410683 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb694011990 con 0x7fb6a80b8c20 2023-08-27T11:24:31.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6a77fe700 1 -- 172.21.15.45:0/2004410683 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694011d20 con 0x7fb6a80b8c20 2023-08-27T11:24:31.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6af0a9700 1 -- 172.21.15.45:0/2004410683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6a80b8c20 msgr2=0x7fb6a80cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:31.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6af0a9700 1 --2- 172.21.15.45:0/2004410683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6a80b8c20 0x7fb6a80cdda0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb694009920 tx=0x7fb694009c30 comp rx=0 tx=0).stop 2023-08-27T11:24:31.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6af0a9700 1 -- 172.21.15.45:0/2004410683 shutdown_connections 2023-08-27T11:24:31.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6af0a9700 1 --2- 172.21.15.45:0/2004410683 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80ce2e0 0x7fb6a80b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:31.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6af0a9700 1 --2- 172.21.15.45:0/2004410683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6a80b8c20 0x7fb6a80cdda0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:31.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6af0a9700 1 -- 172.21.15.45:0/2004410683 >> 172.21.15.45:0/2004410683 conn(0x7fb6a80d2fd0 msgr2=0x7fb6a80d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:31.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6af0a9700 1 -- 172.21.15.45:0/2004410683 shutdown_connections 2023-08-27T11:24:31.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.901+0000 7fb6af0a9700 1 -- 172.21.15.45:0/2004410683 wait complete. 2023-08-27T11:24:31.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.902+0000 7fb6af0a9700 1 Processor -- start 2023-08-27T11:24:31.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.903+0000 7fb6af0a9700 1 -- start start 2023-08-27T11:24:31.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.903+0000 7fb6af0a9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80b8c20 0x7fb6a814dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:31.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.903+0000 7fb6af0a9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6a80ce2e0 0x7fb6a81600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:31.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.903+0000 7fb6af0a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6a815d2c0 0x7fb6a80b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:31.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.903+0000 7fb6af0a9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80c88e0 con 0x7fb6a815d2c0 2023-08-27T11:24:31.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.903+0000 7fb6af0a9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80c8450 con 0x7fb6a80ce2e0 2023-08-27T11:24:31.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.903+0000 7fb6af0a9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8160620 con 0x7fb6a80b8c20 2023-08-27T11:24:31.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ad646700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6a815d2c0 0x7fb6a80b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:31.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ad646700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6a815d2c0 0x7fb6a80b7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55272/0 (socket says 172.21.15.45:55272) 2023-08-27T11:24:31.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ad646700 1 -- 172.21.15.45:0/1299875678 learned_addr learned my addr 172.21.15.45:0/1299875678 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:31.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6a7fff700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6a80ce2e0 0x7fb6a81600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:31.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ace45700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80b8c20 0x7fb6a814dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:31.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ad646700 1 -- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80b8c20 msgr2=0x7fb6a814dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:31.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ad646700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80b8c20 0x7fb6a814dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:31.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ad646700 1 -- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6a80ce2e0 msgr2=0x7fb6a81600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:31.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ad646700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6a80ce2e0 0x7fb6a81600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:31.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ad646700 1 -- 172.21.15.45:0/1299875678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0009a70 con 0x7fb6a815d2c0 2023-08-27T11:24:31.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6ad646700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6a815d2c0 0x7fb6a80b7790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009f00 tx=0x7fb6a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:31.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6a57fa700 1 -- 172.21.15.45:0/1299875678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000ce60 con 0x7fb6a815d2c0 2023-08-27T11:24:31.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6a57fa700 1 -- 172.21.15.45:0/1299875678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a000f040 con 0x7fb6a815d2c0 2023-08-27T11:24:31.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb694009550 con 0x7fb6a815d2c0 2023-08-27T11:24:31.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6a57fa700 1 -- 172.21.15.45:0/1299875678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0014680 con 0x7fb6a815d2c0 2023-08-27T11:24:31.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.904+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a80c7840 con 0x7fb6a815d2c0 2023-08-27T11:24:31.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.905+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a804e620 con 0x7fb6a815d2c0 2023-08-27T11:24:31.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.906+0000 7fb6a57fa700 1 -- 172.21.15.45:0/1299875678 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fb6a00147e0 con 0x7fb6a815d2c0 2023-08-27T11:24:31.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.906+0000 7fb6a57fa700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb68c06a2e0 0x7fb68c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:31.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.906+0000 7fb6a57fa700 1 -- 172.21.15.45:0/1299875678 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fb6a0089b00 con 0x7fb6a815d2c0 2023-08-27T11:24:31.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.906+0000 7fb6ace45700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb68c06a2e0 0x7fb68c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:31.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.907+0000 7fb6ace45700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb68c06a2e0 0x7fb68c06c7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb69c007fd0 tx=0x7fb69c009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:31.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:31.911+0000 7fb6a57fa700 1 -- 172.21.15.45:0/1299875678 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb6a0054be0 con 0x7fb6a815d2c0 2023-08-27T11:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.135+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6a80d21b0 con 0x7fb68c06a2e0 2023-08-27T11:24:32.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.210+0000 7fb6a57fa700 1 -- 172.21.15.45:0/1299875678 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb69c001c70 con 0x7fb68c06a2e0 2023-08-27T11:24:32.212 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-08-27T11:24:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb68c06a2e0 msgr2=0x7fb68c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:32.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb68c06a2e0 0x7fb68c06c7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb69c007fd0 tx=0x7fb69c009040 comp rx=0 tx=0).stop 2023-08-27T11:24:32.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6a815d2c0 msgr2=0x7fb6a80b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:32.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6a815d2c0 0x7fb6a80b7790 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009f00 tx=0x7fb6a000c3a0 comp rx=0 tx=0).stop 2023-08-27T11:24:32.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 shutdown_connections 2023-08-27T11:24:32.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb68c06a2e0 0x7fb68c06c7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:32.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6a815d2c0 0x7fb6a80b7790 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:32.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb6a80b8c20 0x7fb6a814dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:32.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 --2- 172.21.15.45:0/1299875678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6a80ce2e0 0x7fb6a81600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:32.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 >> 172.21.15.45:0/1299875678 conn(0x7fb6a80d2fd0 msgr2=0x7fb6a815a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:32.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 shutdown_connections 2023-08-27T11:24:32.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:32.212+0000 7fb6af0a9700 1 -- 172.21.15.45:0/1299875678 wait complete. 2023-08-27T11:24:32.225 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-08-27T11:24:32.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.226 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-08-27T11:24:32.244 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:32.244 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:32.245 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:32.245 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:32.245 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:32.245 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:32.245 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:32.246 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:32.246 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:32.246 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:32.246 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:32.246 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:32.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.249 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_14_00_UTC) 2023-08-27T11:24:32.313 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:32.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.314 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:32.376 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:32.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.377 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:32.440 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:32.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.440 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:32.503 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:32.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.504 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:32.568 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:32.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.569 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:32.631 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:32.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.633 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:32.695 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:32.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.696 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:32.760 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:32.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.761 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:32.823 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:32.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.824 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:32.885 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:32.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.886 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:32.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:32.949 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-08-27T11:24:33.008 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-08-27T11:24:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.178+0000 7ffaeb1ab700 1 Processor -- start 2023-08-27T11:24:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.178+0000 7ffaeb1ab700 1 -- start start 2023-08-27T11:24:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffaeb1ab700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffae40b65d0 0x7ffae40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffaeb1ab700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40cb5c0 0x7ffae415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffaeb1ab700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae415ec50 con 0x7ffae40cfe10 2023-08-27T11:24:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffaeb1ab700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae415c620 con 0x7ffae40b65d0 2023-08-27T11:24:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffaeb1ab700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae415eea0 con 0x7ffae40cb5c0 2023-08-27T11:24:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffae9748700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffae40cfe10 0x7ffae40cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42056/0 (socket says 172.21.15.45:42056) 2023-08-27T11:24:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffae9748700 1 -- 172.21.15.45:0/2086313343 learned_addr learned my addr 172.21.15.45:0/2086313343 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffae8f47700 1 --2- 172.21.15.45:0/2086313343 >> v2:172.21.15.135:3300/0 conn(0x7ffae40b65d0 0x7ffae40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307945854 0 0) 0x7ffae415ec50 con 0x7ffae40cfe10 2023-08-27T11:24:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad40035b0 con 0x7ffae40cfe10 2023-08-27T11:24:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.179+0000 7ffae3fff700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40cb5c0 0x7ffae415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1978474194 0 0) 0x7ffad40035b0 con 0x7ffae40cfe10 2023-08-27T11:24:33.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40cb5c0 msgr2=0x7ffae415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:33.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40cb5c0 0x7ffae415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 >> v2:172.21.15.135:3300/0 conn(0x7ffae40b65d0 msgr2=0x7ffae40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:33.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 --2- 172.21.15.45:0/2086313343 >> v2:172.21.15.135:3300/0 conn(0x7ffae40b65d0 0x7ffae40ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae415f180 con 0x7ffae40cfe10 2023-08-27T11:24:33.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7ffadc0019f0 con 0x7ffae40cfe10 2023-08-27T11:24:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 >> v1:172.21.15.45:6789/0 conn(0x7ffae40cfe10 legacy=0x7ffae40cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad4003a00 0x7ffad4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffad4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40cb5c0 0x7ffad4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae415c620 con 0x7ffad4004850 2023-08-27T11:24:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae415eea0 con 0x7ffad4003a00 2023-08-27T11:24:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad400ba40 con 0x7ffae40cb5c0 2023-08-27T11:24:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae3fff700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffad4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae8f47700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad4003a00 0x7ffad4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae9748700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40cb5c0 0x7ffad4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae3fff700 1 -- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40cb5c0 msgr2=0x7ffad4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae3fff700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40cb5c0 0x7ffad4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae3fff700 1 -- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad4003a00 msgr2=0x7ffad4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae3fff700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad4003a00 0x7ffad4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae3fff700 1 -- 172.21.15.45:0/2086313343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad40035b0 con 0x7ffad4004850 2023-08-27T11:24:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae9748700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40cb5c0 0x7ffad4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae3fff700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffad4004cd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009b60 tx=0x7ffad4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92bd2c273873eda server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.180+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad000e660 con 0x7ffad4004850 2023-08-27T11:24:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.181+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad000e7c0 con 0x7ffad4004850 2023-08-27T11:24:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.181+0000 7ffae37fe700 1 -- 172.21.15.45:0/2086313343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad000eb30 con 0x7ffad4004850 2023-08-27T11:24:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.181+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/2086313343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 msgr2=0x7ffad4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.181+0000 7ffaeb1ab700 1 --2- 172.21.15.45:0/2086313343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffad4004cd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009b60 tx=0x7ffad4005880 comp rx=0 tx=0).stop 2023-08-27T11:24:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.181+0000 7ffae9748700 1 -- 172.21.15.45:0/2086313343 reap_dead start 2023-08-27T11:24:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.181+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/2086313343 shutdown_connections 2023-08-27T11:24:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.181+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/2086313343 >> 172.21.15.45:0/2086313343 conn(0x7ffae406f960 msgr2=0x7ffae406fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:33.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.181+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/2086313343 shutdown_connections 2023-08-27T11:24:33.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.181+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/2086313343 wait complete. 2023-08-27T11:24:33.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.182+0000 7ffaeb1ab700 1 Processor -- start 2023-08-27T11:24:33.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.182+0000 7ffaeb1ab700 1 -- start start 2023-08-27T11:24:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffaeb1ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffae41f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffaeb1ab700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40c9aa0 0x7ffae40b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffaeb1ab700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae40c9f20 0x7ffae40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffaeb1ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b1a20 con 0x7ffad4004850 2023-08-27T11:24:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffaeb1ab700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae41f43c0 con 0x7ffae40c9f20 2023-08-27T11:24:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffaeb1ab700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b1390 con 0x7ffae40c9aa0 2023-08-27T11:24:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffae8f47700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffae41f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffae8f47700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffae41f0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55288/0 (socket says 172.21.15.45:55288) 2023-08-27T11:24:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffae8f47700 1 -- 172.21.15.45:0/962232911 learned_addr learned my addr 172.21.15.45:0/962232911 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffae3fff700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40c9aa0 0x7ffae40b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.183+0000 7ffae9748700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae40c9f20 0x7ffae40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae8f47700 1 -- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40c9aa0 msgr2=0x7ffae40b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae8f47700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40c9aa0 0x7ffae40b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae8f47700 1 -- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae40c9f20 msgr2=0x7ffae40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae8f47700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae40c9f20 0x7ffae40b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae8f47700 1 -- 172.21.15.45:0/962232911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae40b1670 con 0x7ffad4004850 2023-08-27T11:24:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae9748700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae40c9f20 0x7ffae40b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae8f47700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffae41f0aa0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ffad8011aa0 tx=0x7ffad8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae17fa700 1 -- 172.21.15.45:0/962232911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad800f940 con 0x7ffad4004850 2023-08-27T11:24:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae17fa700 1 -- 172.21.15.45:0/962232911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad800faa0 con 0x7ffad4004850 2023-08-27T11:24:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffae17fa700 1 -- 172.21.15.45:0/962232911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad800fe50 con 0x7ffad4004850 2023-08-27T11:24:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae40b1c30 con 0x7ffad4004850 2023-08-27T11:24:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.184+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae41f0fe0 con 0x7ffad4004850 2023-08-27T11:24:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.185+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae40b2280 con 0x7ffad4004850 2023-08-27T11:24:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.185+0000 7ffae17fa700 1 -- 172.21.15.45:0/962232911 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7ffad8014440 con 0x7ffad4004850 2023-08-27T11:24:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.186+0000 7ffae17fa700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffac806a2e0 0x7ffac806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.186+0000 7ffae17fa700 1 -- 172.21.15.45:0/962232911 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6698+0+0 (secure 0 0 0) 0x7ffad8089450 con 0x7ffad4004850 2023-08-27T11:24:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.186+0000 7ffae3fff700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffac806a2e0 0x7ffac806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.186+0000 7ffae3fff700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffac806a2e0 0x7ffac806c7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffad00090b0 tx=0x7ffad0008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.190+0000 7ffae17fa700 1 -- 172.21.15.45:0/962232911 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffad80545a0 con 0x7ffad4004850 2023-08-27T11:24:33.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.419+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffae415f740 con 0x7ffac806a2e0 2023-08-27T11:24:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.492+0000 7ffae17fa700 1 -- 172.21.15.45:0/962232911 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7ffad0022040 con 0x7ffac806a2e0 2023-08-27T11:24:33.494 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-08-27T11:24:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffac806a2e0 msgr2=0x7ffac806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffac806a2e0 0x7ffac806c7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffad00090b0 tx=0x7ffad0008030 comp rx=0 tx=0).stop 2023-08-27T11:24:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 msgr2=0x7ffae41f0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffae41f0aa0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ffad8011aa0 tx=0x7ffad8011db0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 shutdown_connections 2023-08-27T11:24:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffac806a2e0 0x7ffac806c7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffad4004850 0x7ffae41f0aa0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae40c9f20 0x7ffae40b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 --2- 172.21.15.45:0/962232911 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffae40c9aa0 0x7ffae40b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 >> 172.21.15.45:0/962232911 conn(0x7ffae406f960 msgr2=0x7ffae415dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 shutdown_connections 2023-08-27T11:24:33.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:33.494+0000 7ffaeb1ab700 1 -- 172.21.15.45:0/962232911 wait complete. 2023-08-27T11:24:33.507 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-08-27T11:24:33.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.508 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-08-27T11:24:33.526 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:33.526 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:33.527 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:33.527 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:33.527 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:33.527 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:33.527 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:33.528 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:33.528 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:33.528 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:33.528 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:33.528 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:33.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.529 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_14_00_UTC) 2023-08-27T11:24:33.591 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:33.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.591 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:33.654 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:33.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.655 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:33.718 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:33.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.718 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:33.781 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:33.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.781 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:33.844 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:33.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.845 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:33.910 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:33.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.911 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:33.977 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:33.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:33.978 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:34.039 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:34.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.040 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:34.136 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:34.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.137 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:34.195 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:34.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.196 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:34.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.246 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-08-27T11:24:34.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-08-27T11:24:34.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.514+0000 7fef6c694700 1 Processor -- start 2023-08-27T11:24:34.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.514+0000 7fef6c694700 1 -- start start 2023-08-27T11:24:34.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.515+0000 7fef6c694700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef6406aa70 0x7fef6406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.515+0000 7fef6c694700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6406ddf0 con 0x7fef640ae730 2023-08-27T11:24:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.515+0000 7fef6c694700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6406d830 con 0x7fef640aeb10 2023-08-27T11:24:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.515+0000 7fef6c694700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6406b440 con 0x7fef6406aa70 2023-08-27T11:24:34.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.516+0000 7fef6a430700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef640ae730 0x7fef640ced60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42072/0 (socket says 172.21.15.45:42072) 2023-08-27T11:24:34.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.516+0000 7fef6a430700 1 -- 172.21.15.45:0/3447659485 learned_addr learned my addr 172.21.15.45:0/3447659485 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:34.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.516+0000 7fef6ac31700 1 --2- 172.21.15.45:0/3447659485 >> v2:172.21.15.45:3300/0 conn(0x7fef6406aa70 0x7fef6406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.516+0000 7fef6942e700 1 -- 172.21.15.45:0/3447659485 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729197469 0 0) 0x7fef6406d830 con 0x7fef640aeb10 2023-08-27T11:24:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.516+0000 7fef6ac31700 1 -- 172.21.15.45:0/3447659485 >> v1:172.21.15.135:6789/0 conn(0x7fef640aeb10 legacy=0x7fef64072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:34.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.516+0000 7fef6ac31700 1 -- 172.21.15.45:0/3447659485 >> v1:172.21.15.45:6789/0 conn(0x7fef640ae730 legacy=0x7fef640ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:34.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.516+0000 7fef6ac31700 1 -- 172.21.15.45:0/3447659485 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6406db10 con 0x7fef6406aa70 2023-08-27T11:24:34.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.516+0000 7fef6ac31700 1 --2- 172.21.15.45:0/3447659485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 0x7fef6406af00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00a950 tx=0x7fef5c00ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106bfa7f8065f9da server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:34.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.517+0000 7fef6942e700 1 -- 172.21.15.45:0/3447659485 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00aea0 con 0x7fef6406aa70 2023-08-27T11:24:34.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.517+0000 7fef6942e700 1 -- 172.21.15.45:0/3447659485 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c005e30 con 0x7fef6406aa70 2023-08-27T11:24:34.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.517+0000 7fef6942e700 1 -- 172.21.15.45:0/3447659485 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c004250 con 0x7fef6406aa70 2023-08-27T11:24:34.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.518+0000 7fef6c694700 1 -- 172.21.15.45:0/3447659485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 msgr2=0x7fef6406af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:34.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.518+0000 7fef6c694700 1 --2- 172.21.15.45:0/3447659485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 0x7fef6406af00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00a950 tx=0x7fef5c00ac60 comp rx=0 tx=0).stop 2023-08-27T11:24:34.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.518+0000 7fef6c694700 1 -- 172.21.15.45:0/3447659485 shutdown_connections 2023-08-27T11:24:34.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.518+0000 7fef6c694700 1 --2- 172.21.15.45:0/3447659485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 0x7fef6406af00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:34.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.518+0000 7fef6c694700 1 -- 172.21.15.45:0/3447659485 >> 172.21.15.45:0/3447659485 conn(0x7fef640b2440 msgr2=0x7fef640b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:34.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.519+0000 7fef6c694700 1 -- 172.21.15.45:0/3447659485 shutdown_connections 2023-08-27T11:24:34.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.519+0000 7fef6c694700 1 -- 172.21.15.45:0/3447659485 wait complete. 2023-08-27T11:24:34.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.521+0000 7fef6c694700 1 Processor -- start 2023-08-27T11:24:34.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.521+0000 7fef6c694700 1 -- start start 2023-08-27T11:24:34.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.522+0000 7fef6c694700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 0x7fef64152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:34.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.522+0000 7fef6c694700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef640ae730 0x7fef64156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:34.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.522+0000 7fef6c694700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef640aeb10 0x7fef64157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:34.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.522+0000 7fef6c694700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef64158e60 con 0x7fef6406aa70 2023-08-27T11:24:34.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.522+0000 7fef6c694700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef64098e70 con 0x7fef640ae730 2023-08-27T11:24:34.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.522+0000 7fef6c694700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef64098b90 con 0x7fef640aeb10 2023-08-27T11:24:34.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef6a430700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 0x7fef64152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:34.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef69c2f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef640ae730 0x7fef64156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:34.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef6a430700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 0x7fef64152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55302/0 (socket says 172.21.15.45:55302) 2023-08-27T11:24:34.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef6a430700 1 -- 172.21.15.45:0/2641926980 learned_addr learned my addr 172.21.15.45:0/2641926980 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:34.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef6ac31700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef640aeb10 0x7fef64157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:34.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef69c2f700 1 -- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef640aeb10 msgr2=0x7fef64157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:34.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef69c2f700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef640aeb10 0x7fef64157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:34.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef69c2f700 1 -- 172.21.15.45:0/2641926980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 msgr2=0x7fef64152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:34.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef69c2f700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 0x7fef64152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:34.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef69c2f700 1 -- 172.21.15.45:0/2641926980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c00a5b0 con 0x7fef640ae730 2023-08-27T11:24:34.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.523+0000 7fef6ac31700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef640aeb10 0x7fef64157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:34.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.524+0000 7fef69c2f700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef640ae730 0x7fef64156d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fef5400c520 tx=0x7fef5400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:34.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.524+0000 7fef5affd700 1 -- 172.21.15.45:0/2641926980 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef540054e0 con 0x7fef640ae730 2023-08-27T11:24:34.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.524+0000 7fef5affd700 1 -- 172.21.15.45:0/2641926980 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef54003680 con 0x7fef640ae730 2023-08-27T11:24:34.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.524+0000 7fef5affd700 1 -- 172.21.15.45:0/2641926980 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54003a30 con 0x7fef640ae730 2023-08-27T11:24:34.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.524+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef640b1720 con 0x7fef640ae730 2023-08-27T11:24:34.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.524+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef640b1ce0 con 0x7fef640ae730 2023-08-27T11:24:34.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.526+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef640b1e70 con 0x7fef640ae730 2023-08-27T11:24:34.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.526+0000 7fef5affd700 1 -- 172.21.15.45:0/2641926980 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fef54003b90 con 0x7fef640ae730 2023-08-27T11:24:34.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.527+0000 7fef5affd700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef4806a330 0x7fef4806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:34.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.527+0000 7fef5affd700 1 -- 172.21.15.45:0/2641926980 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7746+0+0 (secure 0 0 0) 0x7fef54088180 con 0x7fef640ae730 2023-08-27T11:24:34.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.527+0000 7fef6a430700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef4806a330 0x7fef4806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:34.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.528+0000 7fef6a430700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef4806a330 0x7fef4806c7f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fef60001270 tx=0x7fef60009040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:34.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.532+0000 7fef5affd700 1 -- 172.21.15.45:0/2641926980 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fef54053170 con 0x7fef640ae730 2023-08-27T11:24:34.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.799+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fef64071e10 con 0x7fef4806a330 2023-08-27T11:24:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.863+0000 7fef5affd700 1 -- 172.21.15.45:0/2641926980 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fef6000f9d0 con 0x7fef4806a330 2023-08-27T11:24:34.865 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-08-27T11:24:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef4806a330 msgr2=0x7fef4806c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef4806a330 0x7fef4806c7f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fef60001270 tx=0x7fef60009040 comp rx=0 tx=0).stop 2023-08-27T11:24:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef640ae730 msgr2=0x7fef64156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef640ae730 0x7fef64156d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fef5400c520 tx=0x7fef5400c830 comp rx=0 tx=0).stop 2023-08-27T11:24:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 shutdown_connections 2023-08-27T11:24:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fef4806a330 0x7fef4806c7f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6406aa70 0x7fef64152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef640aeb10 0x7fef64157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 --2- 172.21.15.45:0/2641926980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef640ae730 0x7fef64156d50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 >> 172.21.15.45:0/2641926980 conn(0x7fef640b2440 msgr2=0x7fef640b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 shutdown_connections 2023-08-27T11:24:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:34.866+0000 7fef6c694700 1 -- 172.21.15.45:0/2641926980 wait complete. 2023-08-27T11:24:34.878 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-08-27T11:24:34.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.879 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-08-27T11:24:34.897 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:34.897 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:34.897 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:34.897 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:34.898 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:34.898 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:34.898 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:34.898 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:34.898 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:34.899 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:34.899 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:34.900 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:34.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.901 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_14_00_UTC) 2023-08-27T11:24:34.965 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:34.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:34.966 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:35.027 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:35.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.028 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:35.101 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:35.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.102 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:35.164 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:35.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.165 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:35.228 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:35.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.229 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:35.291 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:35.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.293 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:35.355 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:35.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.356 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:35.418 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:35.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.419 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:35.481 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:35.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.482 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:35.546 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:35.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.547 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:35.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:35.610 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-08-27T11:24:35.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-08-27T11:24:35.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.839+0000 7efc885c5700 1 Processor -- start 2023-08-27T11:24:35.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.839+0000 7efc885c5700 1 -- start start 2023-08-27T11:24:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc885c5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 0x7efc8006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc885c5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efc80071e80 0x7efc800caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc885c5700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8006f8b0 con 0x7efc8015b270 2023-08-27T11:24:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc885c5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc800ca650 con 0x7efc80071e80 2023-08-27T11:24:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc885c5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8006f010 con 0x7efc8006cf40 2023-08-27T11:24:35.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc86b62700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efc80071e80 0x7efc800caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:35.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc86b62700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efc80071e80 0x7efc800caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55314/0 (socket says 172.21.15.45:55314) 2023-08-27T11:24:35.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc86b62700 1 -- 172.21.15.45:0/3889400906 learned_addr learned my addr 172.21.15.45:0/3889400906 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:35.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc86361700 1 --2- 172.21.15.45:0/3889400906 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 0x7efc8006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:35.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc86b62700 1 -- 172.21.15.45:0/3889400906 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 msgr2=0x7efc8006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:35.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc86b62700 1 --2- 172.21.15.45:0/3889400906 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 0x7efc8006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc86b62700 1 -- 172.21.15.45:0/3889400906 >> v1:172.21.15.135:6789/0 conn(0x7efc8015b270 legacy=0x7efc8006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc86b62700 1 -- 172.21.15.45:0/3889400906 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8006f2f0 con 0x7efc80071e80 2023-08-27T11:24:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.840+0000 7efc86361700 1 --2- 172.21.15.45:0/3889400906 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 0x7efc8006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc86b62700 1 --2- 172.21.15.45:0/3889400906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc80071e80 0x7efc800caa70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efc78009e50 tx=0x7efc7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8b17a4995217d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc8535f700 1 -- 172.21.15.45:0/3889400906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7800e070 con 0x7efc80071e80 2023-08-27T11:24:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc8535f700 1 -- 172.21.15.45:0/3889400906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7800b9c0 con 0x7efc80071e80 2023-08-27T11:24:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc8535f700 1 -- 172.21.15.45:0/3889400906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7800bd70 con 0x7efc80071e80 2023-08-27T11:24:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc885c5700 1 -- 172.21.15.45:0/3889400906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc80071e80 msgr2=0x7efc800caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc885c5700 1 --2- 172.21.15.45:0/3889400906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc80071e80 0x7efc800caa70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efc78009e50 tx=0x7efc7800b330 comp rx=0 tx=0).stop 2023-08-27T11:24:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc885c5700 1 -- 172.21.15.45:0/3889400906 shutdown_connections 2023-08-27T11:24:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc885c5700 1 --2- 172.21.15.45:0/3889400906 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 0x7efc8006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc885c5700 1 --2- 172.21.15.45:0/3889400906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc80071e80 0x7efc800caa70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc885c5700 1 -- 172.21.15.45:0/3889400906 >> 172.21.15.45:0/3889400906 conn(0x7efc800cc7e0 msgr2=0x7efc800ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.841+0000 7efc885c5700 1 -- 172.21.15.45:0/3889400906 shutdown_connections 2023-08-27T11:24:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.842+0000 7efc885c5700 1 -- 172.21.15.45:0/3889400906 wait complete. 2023-08-27T11:24:35.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.843+0000 7efc885c5700 1 Processor -- start 2023-08-27T11:24:35.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.843+0000 7efc885c5700 1 -- start start 2023-08-27T11:24:35.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc885c5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 0x7efc80141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc885c5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc80071e80 0x7efc800b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc885c5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc8015b270 0x7efc801f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc885c5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc801f4b70 con 0x7efc8015b270 2023-08-27T11:24:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc885c5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc801f45b0 con 0x7efc80071e80 2023-08-27T11:24:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc885c5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc801f4890 con 0x7efc8006cf40 2023-08-27T11:24:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc86b62700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc8015b270 0x7efc801f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc86b62700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc8015b270 0x7efc801f3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55320/0 (socket says 172.21.15.45:55320) 2023-08-27T11:24:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc86b62700 1 -- 172.21.15.45:0/3080665996 learned_addr learned my addr 172.21.15.45:0/3080665996 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc86361700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 0x7efc80141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc85b60700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc80071e80 0x7efc800b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:35.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc86b62700 1 -- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 msgr2=0x7efc80141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:35.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc86b62700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 0x7efc80141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:35.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc86b62700 1 -- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc80071e80 msgr2=0x7efc800b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:35.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc86b62700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc80071e80 0x7efc800b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:35.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.844+0000 7efc86b62700 1 -- 172.21.15.45:0/3080665996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc78009b30 con 0x7efc8015b270 2023-08-27T11:24:35.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.845+0000 7efc85b60700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc80071e80 0x7efc800b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:35.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.845+0000 7efc86b62700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc8015b270 0x7efc801f3c40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efc78012000 tx=0x7efc78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:35.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.845+0000 7efc76ffd700 1 -- 172.21.15.45:0/3080665996 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780048b0 con 0x7efc8015b270 2023-08-27T11:24:35.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.845+0000 7efc76ffd700 1 -- 172.21.15.45:0/3080665996 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc780099b0 con 0x7efc8015b270 2023-08-27T11:24:35.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.845+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc80140f00 con 0x7efc8015b270 2023-08-27T11:24:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.845+0000 7efc76ffd700 1 -- 172.21.15.45:0/3080665996 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7801c680 con 0x7efc8015b270 2023-08-27T11:24:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.845+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc80141c40 con 0x7efc8015b270 2023-08-27T11:24:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.846+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8006dca0 con 0x7efc8015b270 2023-08-27T11:24:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.846+0000 7efc76ffd700 1 -- 172.21.15.45:0/3080665996 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7efc7801c7e0 con 0x7efc8015b270 2023-08-27T11:24:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.847+0000 7efc76ffd700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7efc6406a2e0 0x7efc6406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.847+0000 7efc76ffd700 1 -- 172.21.15.45:0/3080665996 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7266+0+0 (secure 0 0 0) 0x7efc7808a7c0 con 0x7efc8015b270 2023-08-27T11:24:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.847+0000 7efc86361700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7efc6406a2e0 0x7efc6406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.847+0000 7efc86361700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7efc6406a2e0 0x7efc6406c7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efc700099d0 tx=0x7efc70008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:35.851+0000 7efc76ffd700 1 -- 172.21.15.45:0/3080665996 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efc780556f0 con 0x7efc8015b270 2023-08-27T11:24:36.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.076+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efc80140d90 con 0x7efc6406a2e0 2023-08-27T11:24:36.184 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.184+0000 7efc76ffd700 1 -- 172.21.15.45:0/3080665996 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7efc80140d90 con 0x7efc6406a2e0 2023-08-27T11:24:36.185 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-08-27T11:24:36.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7efc6406a2e0 msgr2=0x7efc6406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7efc6406a2e0 0x7efc6406c7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efc700099d0 tx=0x7efc70008040 comp rx=0 tx=0).stop 2023-08-27T11:24:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc8015b270 msgr2=0x7efc801f3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc8015b270 0x7efc801f3c40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efc78012000 tx=0x7efc78004ad0 comp rx=0 tx=0).stop 2023-08-27T11:24:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 shutdown_connections 2023-08-27T11:24:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7efc6406a2e0 0x7efc6406c7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:36.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc8015b270 0x7efc801f3c40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:36.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7efc8006cf40 0x7efc80141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:36.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 --2- 172.21.15.45:0/3080665996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc80071e80 0x7efc800b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:36.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 >> 172.21.15.45:0/3080665996 conn(0x7efc800cc7e0 msgr2=0x7efc800c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:36.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 shutdown_connections 2023-08-27T11:24:36.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:36.186+0000 7efc885c5700 1 -- 172.21.15.45:0/3080665996 wait complete. 2023-08-27T11:24:36.199 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-08-27T11:24:36.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.200 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-08-27T11:24:36.218 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:36.218 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:36.218 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:36.218 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:36.219 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:36.219 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:36.219 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:36.219 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:36.220 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:36.220 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:36.220 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:36.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.221 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:36.283 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:36.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.284 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:36.348 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:36.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.348 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:36.411 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:36.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.412 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:36.423 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:24:36.455 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:24:36.476 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:36.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.477 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:36.577 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:36.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.578 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:36.642 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:36.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.643 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:36.708 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:36.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.709 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:36.770 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:36.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.771 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:36.833 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:36.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.835 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:36.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:36.899 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-08-27T11:24:36.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-08-27T11:24:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.154+0000 7fc29ba7f700 1 Processor -- start 2023-08-27T11:24:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.154+0000 7fc29ba7f700 1 -- start start 2023-08-27T11:24:37.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.155+0000 7fc29ba7f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc29415e930 0x7fc2940c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:37.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.155+0000 7fc29ba7f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc2940cb920 0x7fc29406bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:37.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.155+0000 7fc29ba7f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc2940cbda0 0x7fc29406c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:37.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.155+0000 7fc29ba7f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29415ed10 con 0x7fc2940cbda0 2023-08-27T11:24:37.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.155+0000 7fc29ba7f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2940ce810 con 0x7fc2940cb920 2023-08-27T11:24:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.155+0000 7fc29ba7f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc294070090 con 0x7fc29415e930 2023-08-27T11:24:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.155+0000 7fc29a01c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc2940cbda0 0x7fc29406c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.155+0000 7fc29a01c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc2940cbda0 0x7fc29406c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55328/0 (socket says 172.21.15.45:55328) 2023-08-27T11:24:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.156+0000 7fc29a01c700 1 -- 172.21.15.45:0/1851446043 learned_addr learned my addr 172.21.15.45:0/1851446043 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.156+0000 7fc29981b700 1 --2- 172.21.15.45:0/1851446043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc29415e930 0x7fc2940c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.156+0000 7fc29901a700 1 --2- 172.21.15.45:0/1851446043 >> v2:172.21.15.135:3300/0 conn(0x7fc2940cb920 0x7fc29406bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.156+0000 7fc29981b700 1 -- 172.21.15.45:0/1851446043 >> v2:172.21.15.135:3300/0 conn(0x7fc2940cb920 msgr2=0x7fc29406bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.156+0000 7fc29981b700 1 --2- 172.21.15.45:0/1851446043 >> v2:172.21.15.135:3300/0 conn(0x7fc2940cb920 0x7fc29406bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.156+0000 7fc29981b700 1 -- 172.21.15.45:0/1851446043 >> v2:172.21.15.45:3300/0 conn(0x7fc2940cbda0 msgr2=0x7fc29406c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.156+0000 7fc29981b700 1 --2- 172.21.15.45:0/1851446043 >> v2:172.21.15.45:3300/0 conn(0x7fc2940cbda0 0x7fc29406c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.156+0000 7fc29981b700 1 -- 172.21.15.45:0/1851446043 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294070370 con 0x7fc29415e930 2023-08-27T11:24:37.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.156+0000 7fc29981b700 1 --2- 172.21.15.45:0/1851446043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc29415e930 0x7fc2940c9c90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc2940ceb20 tx=0x7fc28c009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4c5e9a807fcd49 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:37.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.157+0000 7fc298819700 1 -- 172.21.15.45:0/1851446043 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c011830 con 0x7fc29415e930 2023-08-27T11:24:37.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.157+0000 7fc298819700 1 -- 172.21.15.45:0/1851446043 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c011990 con 0x7fc29415e930 2023-08-27T11:24:37.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.157+0000 7fc298819700 1 -- 172.21.15.45:0/1851446043 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c011d20 con 0x7fc29415e930 2023-08-27T11:24:37.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.157+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1851446043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc29415e930 msgr2=0x7fc2940c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:37.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.157+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1851446043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc29415e930 0x7fc2940c9c90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc2940ceb20 tx=0x7fc28c009b20 comp rx=0 tx=0).stop 2023-08-27T11:24:37.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.158+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1851446043 shutdown_connections 2023-08-27T11:24:37.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.158+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1851446043 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc29415e930 0x7fc2940c9c90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.158+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1851446043 >> v2:172.21.15.45:3300/0 conn(0x7fc2940cbda0 0x7fc29406c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.158+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1851446043 >> v2:172.21.15.135:3300/0 conn(0x7fc2940cb920 0x7fc29406bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.158+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1851446043 >> 172.21.15.45:0/1851446043 conn(0x7fc2940b07c0 msgr2=0x7fc2940b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:37.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.158+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1851446043 shutdown_connections 2023-08-27T11:24:37.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.158+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1851446043 wait complete. 2023-08-27T11:24:37.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.159+0000 7fc29ba7f700 1 Processor -- start 2023-08-27T11:24:37.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.159+0000 7fc29ba7f700 1 -- start start 2023-08-27T11:24:37.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.160+0000 7fc29ba7f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2940cb920 0x7fc294145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:37.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.160+0000 7fc29ba7f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2940cbda0 0x7fc2940cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:37.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.160+0000 7fc29ba7f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc29415e930 0x7fc294146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:37.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.160+0000 7fc29ba7f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29414a360 con 0x7fc29415e930 2023-08-27T11:24:37.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.160+0000 7fc29ba7f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29414a640 con 0x7fc2940cb920 2023-08-27T11:24:37.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.160+0000 7fc29ba7f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29414a920 con 0x7fc2940cbda0 2023-08-27T11:24:37.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29981b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2940cb920 0x7fc294145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:37.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29981b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2940cb920 0x7fc294145dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:44186/0 (socket says 172.21.15.45:44186) 2023-08-27T11:24:37.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29981b700 1 -- 172.21.15.45:0/1151214020 learned_addr learned my addr 172.21.15.45:0/1151214020 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:37.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29901a700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2940cbda0 0x7fc2940cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:37.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29a01c700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc29415e930 0x7fc294146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:37.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29a01c700 1 -- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2940cbda0 msgr2=0x7fc2940cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:37.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29a01c700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2940cbda0 0x7fc2940cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29a01c700 1 -- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2940cb920 msgr2=0x7fc294145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:37.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29a01c700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2940cb920 0x7fc294145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29a01c700 1 -- 172.21.15.45:0/1151214020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c0094a0 con 0x7fc29415e930 2023-08-27T11:24:37.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29901a700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2940cbda0 0x7fc2940cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:37.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.161+0000 7fc29981b700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2940cb920 0x7fc294145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.162+0000 7fc29a01c700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc29415e930 0x7fc294146f90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc29000ef90 tx=0x7fc29000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.162+0000 7fc28a7fc700 1 -- 172.21.15.45:0/1151214020 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290003c20 con 0x7fc29415e930 2023-08-27T11:24:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.162+0000 7fc28a7fc700 1 -- 172.21.15.45:0/1151214020 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29000f040 con 0x7fc29415e930 2023-08-27T11:24:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.162+0000 7fc28a7fc700 1 -- 172.21.15.45:0/1151214020 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290009e60 con 0x7fc29415e930 2023-08-27T11:24:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.162+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29414ac00 con 0x7fc29415e930 2023-08-27T11:24:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.162+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2940ce410 con 0x7fc29415e930 2023-08-27T11:24:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.163+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2940b4c40 con 0x7fc29415e930 2023-08-27T11:24:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.164+0000 7fc28a7fc700 1 -- 172.21.15.45:0/1151214020 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fc29000c880 con 0x7fc29415e930 2023-08-27T11:24:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.164+0000 7fc28a7fc700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc27806a2e0 0x7fc27806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.165+0000 7fc28a7fc700 1 -- 172.21.15.45:0/1151214020 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7378+0+0 (secure 0 0 0) 0x7fc2900890c0 con 0x7fc29415e930 2023-08-27T11:24:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.165+0000 7fc29981b700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc27806a2e0 0x7fc27806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.165+0000 7fc29981b700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc27806a2e0 0x7fc27806c7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009b60 tx=0x7fc28c008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.172+0000 7fc28a7fc700 1 -- 172.21.15.45:0/1151214020 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc290053f80 con 0x7fc29415e930 2023-08-27T11:24:37.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.415+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc29414a7b0 con 0x7fc27806a2e0 2023-08-27T11:24:37.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.487+0000 7fc28a7fc700 1 -- 172.21.15.45:0/1151214020 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc29414a7b0 con 0x7fc27806a2e0 2023-08-27T11:24:37.488 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-08-27T11:24:37.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc27806a2e0 msgr2=0x7fc27806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:37.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc27806a2e0 0x7fc27806c7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009b60 tx=0x7fc28c008030 comp rx=0 tx=0).stop 2023-08-27T11:24:37.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc29415e930 msgr2=0x7fc294146f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc29415e930 0x7fc294146f90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc29000ef90 tx=0x7fc29000c3a0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 shutdown_connections 2023-08-27T11:24:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc27806a2e0 0x7fc27806c7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc29415e930 0x7fc294146f90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2940cb920 0x7fc294145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 --2- 172.21.15.45:0/1151214020 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc2940cbda0 0x7fc2940cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:37.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 >> 172.21.15.45:0/1151214020 conn(0x7fc2940b07c0 msgr2=0x7fc294065fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:37.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 shutdown_connections 2023-08-27T11:24:37.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:37.489+0000 7fc29ba7f700 1 -- 172.21.15.45:0/1151214020 wait complete. 2023-08-27T11:24:37.502 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-08-27T11:24:37.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.503 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-08-27T11:24:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:37.521 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:37.522 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:37.523 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:37.523 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:37.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.524 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_14_00_UTC) 2023-08-27T11:24:37.584 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:37.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.586 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:37.647 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:37.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.648 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:37.710 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:37.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.711 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:37.773 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:37.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.774 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:37.837 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:37.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.838 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:37.900 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:37.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.901 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:37.963 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:37.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:37.964 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:38.026 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:38.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.027 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:38.090 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:38.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.091 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:38.156 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:38.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.157 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:38.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.220 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-08-27T11:24:38.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-08-27T11:24:38.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.482+0000 7fcdb565a700 1 Processor -- start 2023-08-27T11:24:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.482+0000 7fcdb565a700 1 -- start start 2023-08-27T11:24:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.482+0000 7fcdb565a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcdb0160890 0x7fcdb00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.482+0000 7fcdb565a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcdb00b62c0 0x7fcdb00b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.482+0000 7fcdb565a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160c70 0x7fcdb00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.482+0000 7fcdb565a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb00cdf40 con 0x7fcdb0160890 2023-08-27T11:24:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdb565a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb00d0fd0 con 0x7fcdb00b62c0 2023-08-27T11:24:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdb565a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb00ce190 con 0x7fcdb0160c70 2023-08-27T11:24:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaeffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcdb0160890 0x7fcdb00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaeffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcdb0160890 0x7fcdb00cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55338/0 (socket says 172.21.15.45:55338) 2023-08-27T11:24:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaeffd700 1 -- 172.21.15.45:0/2749409975 learned_addr learned my addr 172.21.15.45:0/2749409975 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdae7fc700 1 --2- 172.21.15.45:0/2749409975 >> v2:172.21.15.135:3300/0 conn(0x7fcdb00b62c0 0x7fcdb00b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaf7fe700 1 --2- 172.21.15.45:0/2749409975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160c70 0x7fcdb00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaeffd700 1 -- 172.21.15.45:0/2749409975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160c70 msgr2=0x7fcdb00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaeffd700 1 --2- 172.21.15.45:0/2749409975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160c70 0x7fcdb00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaeffd700 1 -- 172.21.15.45:0/2749409975 >> v2:172.21.15.135:3300/0 conn(0x7fcdb00b62c0 msgr2=0x7fcdb00b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaeffd700 1 --2- 172.21.15.45:0/2749409975 >> v2:172.21.15.135:3300/0 conn(0x7fcdb00b62c0 0x7fcdb00b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaeffd700 1 -- 172.21.15.45:0/2749409975 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb00ce470 con 0x7fcdb0160890 2023-08-27T11:24:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdaeffd700 1 --2- 172.21.15.45:0/2749409975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160890 0x7fcdb00cd8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcdb006c0f0 tx=0x7fcda0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5dfdf8b622452a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdadffb700 1 -- 172.21.15.45:0/2749409975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000b870 con 0x7fcdb0160890 2023-08-27T11:24:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdadffb700 1 -- 172.21.15.45:0/2749409975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda000b9d0 con 0x7fcdb0160890 2023-08-27T11:24:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.483+0000 7fcdadffb700 1 -- 172.21.15.45:0/2749409975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000bd60 con 0x7fcdb0160890 2023-08-27T11:24:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.484+0000 7fcdb565a700 1 -- 172.21.15.45:0/2749409975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160890 msgr2=0x7fcdb00cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.484+0000 7fcdb565a700 1 --2- 172.21.15.45:0/2749409975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160890 0x7fcdb00cd8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcdb006c0f0 tx=0x7fcda0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.484+0000 7fcdb565a700 1 -- 172.21.15.45:0/2749409975 shutdown_connections 2023-08-27T11:24:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.484+0000 7fcdb565a700 1 --2- 172.21.15.45:0/2749409975 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160c70 0x7fcdb00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.484+0000 7fcdb565a700 1 --2- 172.21.15.45:0/2749409975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160890 0x7fcdb00cd8f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.484+0000 7fcdb565a700 1 --2- 172.21.15.45:0/2749409975 >> v2:172.21.15.135:3300/0 conn(0x7fcdb00b62c0 0x7fcdb00b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.484+0000 7fcdb565a700 1 -- 172.21.15.45:0/2749409975 >> 172.21.15.45:0/2749409975 conn(0x7fcdb00cfb10 msgr2=0x7fcdb00cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:38.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.484+0000 7fcdb565a700 1 -- 172.21.15.45:0/2749409975 shutdown_connections 2023-08-27T11:24:38.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.484+0000 7fcdb565a700 1 -- 172.21.15.45:0/2749409975 wait complete. 2023-08-27T11:24:38.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.485+0000 7fcdb565a700 1 Processor -- start 2023-08-27T11:24:38.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.485+0000 7fcdb565a700 1 -- start start 2023-08-27T11:24:38.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdb565a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb00b62c0 0x7fcdb00cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:38.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdb565a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160890 0x7fcdb01f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:38.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdb565a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160c70 0x7fcdb01f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:38.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdb565a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb00a9e30 con 0x7fcdb0160c70 2023-08-27T11:24:38.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdb565a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb015a2a0 con 0x7fcdb00b62c0 2023-08-27T11:24:38.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdb565a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb00cc410 con 0x7fcdb0160890 2023-08-27T11:24:38.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdaf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160c70 0x7fcdb01f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:38.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdaf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160c70 0x7fcdb01f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55348/0 (socket says 172.21.15.45:55348) 2023-08-27T11:24:38.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdaf7fe700 1 -- 172.21.15.45:0/1491330135 learned_addr learned my addr 172.21.15.45:0/1491330135 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:38.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdaeffd700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb00b62c0 0x7fcdb00cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:38.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.486+0000 7fcdae7fc700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160890 0x7fcdb01f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:38.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcdaf7fe700 1 -- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160890 msgr2=0x7fcdb01f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:38.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcdaf7fe700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160890 0x7fcdb01f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcdaf7fe700 1 -- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb00b62c0 msgr2=0x7fcdb00cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:38.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcdaf7fe700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb00b62c0 0x7fcdb00cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcdaf7fe700 1 -- 172.21.15.45:0/1491330135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0009550 con 0x7fcdb0160c70 2023-08-27T11:24:38.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcdaf7fe700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160c70 0x7fcdb01f35e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009fd0 tx=0x7fcda800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:38.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcd977fe700 1 -- 172.21.15.45:0/1491330135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8007ea0 con 0x7fcdb0160c70 2023-08-27T11:24:38.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcd977fe700 1 -- 172.21.15.45:0/1491330135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda800ed30 con 0x7fcdb0160c70 2023-08-27T11:24:38.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcd977fe700 1 -- 172.21.15.45:0/1491330135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda800eea0 con 0x7fcdb0160c70 2023-08-27T11:24:38.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb00cc6f0 con 0x7fcdb0160c70 2023-08-27T11:24:38.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.487+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb015a540 con 0x7fcdb0160c70 2023-08-27T11:24:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.488+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb015a730 con 0x7fcdb0160c70 2023-08-27T11:24:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.489+0000 7fcd977fe700 1 -- 172.21.15.45:0/1491330135 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fcda8014660 con 0x7fcdb0160c70 2023-08-27T11:24:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.489+0000 7fcd977fe700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcd9c06a2e0 0x7fcd9c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.489+0000 7fcd977fe700 1 -- 172.21.15.45:0/1491330135 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fcda80893b0 con 0x7fcdb0160c70 2023-08-27T11:24:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.489+0000 7fcdaeffd700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcd9c06a2e0 0x7fcd9c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:38.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.490+0000 7fcdaeffd700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcd9c06a2e0 0x7fcd9c06c7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcda0000c00 tx=0x7fcda0008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:38.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.493+0000 7fcd977fe700 1 -- 172.21.15.45:0/1491330135 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcda8054360 con 0x7fcdb0160c70 2023-08-27T11:24:38.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.721+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdb00d0c60 con 0x7fcd9c06a2e0 2023-08-27T11:24:38.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.786+0000 7fcd977fe700 1 -- 172.21.15.45:0/1491330135 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fcda000bb40 con 0x7fcd9c06a2e0 2023-08-27T11:24:38.787 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-08-27T11:24:38.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcd9c06a2e0 msgr2=0x7fcd9c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:38.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcd9c06a2e0 0x7fcd9c06c7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcda0000c00 tx=0x7fcda0008030 comp rx=0 tx=0).stop 2023-08-27T11:24:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160c70 msgr2=0x7fcdb01f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160c70 0x7fcdb01f35e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009fd0 tx=0x7fcda800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 shutdown_connections 2023-08-27T11:24:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcd9c06a2e0 0x7fcd9c06c7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcdb0160c70 0x7fcdb01f35e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdb00b62c0 0x7fcdb00cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 --2- 172.21.15.45:0/1491330135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcdb0160890 0x7fcdb01f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:38.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 >> 172.21.15.45:0/1491330135 conn(0x7fcdb00cfb10 msgr2=0x7fcdb00c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:38.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 shutdown_connections 2023-08-27T11:24:38.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:38.788+0000 7fcdb565a700 1 -- 172.21.15.45:0/1491330135 wait complete. 2023-08-27T11:24:38.802 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-08-27T11:24:38.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.803 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-08-27T11:24:38.821 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:38.822 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:38.823 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:38.824 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:38.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.825 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_14_00_UTC) 2023-08-27T11:24:38.889 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:38.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.890 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:38.957 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:38.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:38.958 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:39.022 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:39.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.023 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:39.089 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:39.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.090 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:39.169 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:39.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.169 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:39.257 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:39.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.258 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:39.285 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:39.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.285 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:39.348 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:39.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.349 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:39.411 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:39.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.412 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:39.474 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:39.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.475 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:39.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:39.539 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-08-27T11:24:39.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-08-27T11:24:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.767+0000 7f29661c0700 1 Processor -- start 2023-08-27T11:24:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.767+0000 7f29661c0700 1 -- start start 2023-08-27T11:24:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.768+0000 7f29661c0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f296015d340 0x7f2960160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.768+0000 7f29661c0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f296006a990 0x7f29600b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.768+0000 7f29661c0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296006b160 con 0x7f29600cd880 2023-08-27T11:24:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.768+0000 7f29661c0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2960072010 con 0x7f296015d340 2023-08-27T11:24:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.768+0000 7f29661c0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29600af590 con 0x7f296006a990 2023-08-27T11:24:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.768+0000 7f295f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f296015d340 0x7f2960160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.768+0000 7f295f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f296015d340 0x7f2960160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55352/0 (socket says 172.21.15.45:55352) 2023-08-27T11:24:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.768+0000 7f295f7fe700 1 -- 172.21.15.45:0/2487186227 learned_addr learned my addr 172.21.15.45:0/2487186227 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.769+0000 7f295ffff700 1 --2- 172.21.15.45:0/2487186227 >> v2:172.21.15.135:3300/0 conn(0x7f296006a990 0x7f29600b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.769+0000 7f295f7fe700 1 -- 172.21.15.45:0/2487186227 >> v2:172.21.15.135:3300/0 conn(0x7f296006a990 msgr2=0x7f29600b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.769+0000 7f295f7fe700 1 --2- 172.21.15.45:0/2487186227 >> v2:172.21.15.135:3300/0 conn(0x7f296006a990 0x7f29600b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.769+0000 7f295f7fe700 1 -- 172.21.15.45:0/2487186227 >> v1:172.21.15.135:6789/0 conn(0x7f29600cd880 legacy=0x7f296006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.769+0000 7f295f7fe700 1 -- 172.21.15.45:0/2487186227 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29600af2b0 con 0x7f296015d340 2023-08-27T11:24:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.769+0000 7f295f7fe700 1 --2- 172.21.15.45:0/2487186227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f296015d340 0x7f2960160d00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2960072af0 tx=0x7f2950009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2df150bbc7a8ce server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.769+0000 7f295e7fc700 1 -- 172.21.15.45:0/2487186227 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950004230 con 0x7f296015d340 2023-08-27T11:24:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.769+0000 7f295e7fc700 1 -- 172.21.15.45:0/2487186227 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2950004390 con 0x7f296015d340 2023-08-27T11:24:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.769+0000 7f295e7fc700 1 -- 172.21.15.45:0/2487186227 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29500036a0 con 0x7f296015d340 2023-08-27T11:24:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.770+0000 7f29661c0700 1 -- 172.21.15.45:0/2487186227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f296015d340 msgr2=0x7f2960160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.770+0000 7f29661c0700 1 --2- 172.21.15.45:0/2487186227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f296015d340 0x7f2960160d00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2960072af0 tx=0x7f2950009bd0 comp rx=0 tx=0).stop 2023-08-27T11:24:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.770+0000 7f29661c0700 1 -- 172.21.15.45:0/2487186227 shutdown_connections 2023-08-27T11:24:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.770+0000 7f29661c0700 1 --2- 172.21.15.45:0/2487186227 >> v2:172.21.15.135:3300/0 conn(0x7f296006a990 0x7f29600b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.770+0000 7f29661c0700 1 --2- 172.21.15.45:0/2487186227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f296015d340 0x7f2960160d00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.770+0000 7f29661c0700 1 -- 172.21.15.45:0/2487186227 >> 172.21.15.45:0/2487186227 conn(0x7f29600cc7c0 msgr2=0x7f29600ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.770+0000 7f29661c0700 1 -- 172.21.15.45:0/2487186227 shutdown_connections 2023-08-27T11:24:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.770+0000 7f29661c0700 1 -- 172.21.15.45:0/2487186227 wait complete. 2023-08-27T11:24:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.771+0000 7f29661c0700 1 Processor -- start 2023-08-27T11:24:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.771+0000 7f29661c0700 1 -- start start 2023-08-27T11:24:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f29661c0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f296006a990 0x7f29600adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f29661c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29600cd880 0x7f29600c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f29661c0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f296015d340 0x7f2960151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f29661c0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29600c6eb0 con 0x7f29600cd880 2023-08-27T11:24:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f29661c0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29600c7190 con 0x7f296015d340 2023-08-27T11:24:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f29661c0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29600c7470 con 0x7f296006a990 2023-08-27T11:24:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29600cd880 0x7f29600c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29600cd880 0x7f29600c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55360/0 (socket says 172.21.15.45:55360) 2023-08-27T11:24:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295effd700 1 -- 172.21.15.45:0/2918464092 learned_addr learned my addr 172.21.15.45:0/2918464092 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295ffff700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f296015d340 0x7f2960151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295f7fe700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f296006a990 0x7f29600adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295effd700 1 -- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f296006a990 msgr2=0x7f29600adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295effd700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f296006a990 0x7f29600adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295effd700 1 -- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f296015d340 msgr2=0x7f2960151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295effd700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f296015d340 0x7f2960151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:39.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295effd700 1 -- 172.21.15.45:0/2918464092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2950009550 con 0x7f29600cd880 2023-08-27T11:24:39.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.772+0000 7f295effd700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29600cd880 0x7f29600c84b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f294800b4a0 tx=0x7f294800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:39.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.773+0000 7f2957fff700 1 -- 172.21.15.45:0/2918464092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948011830 con 0x7f29600cd880 2023-08-27T11:24:39.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.773+0000 7f2957fff700 1 -- 172.21.15.45:0/2918464092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948011990 con 0x7f29600cd880 2023-08-27T11:24:39.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.773+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29600c7750 con 0x7f29600cd880 2023-08-27T11:24:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.773+0000 7f2957fff700 1 -- 172.21.15.45:0/2918464092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948011d40 con 0x7f29600cd880 2023-08-27T11:24:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.773+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29600cc390 con 0x7f29600cd880 2023-08-27T11:24:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.774+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29600c7c70 con 0x7f29600cd880 2023-08-27T11:24:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.774+0000 7f2957fff700 1 -- 172.21.15.45:0/2918464092 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f294801a400 con 0x7f29600cd880 2023-08-27T11:24:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.774+0000 7f2957fff700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f294006a2e0 0x7f294006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:39.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.775+0000 7f2957fff700 1 -- 172.21.15.45:0/2918464092 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7442+0+0 (secure 0 0 0) 0x7f2948088be0 con 0x7f29600cd880 2023-08-27T11:24:39.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.775+0000 7f295f7fe700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f294006a2e0 0x7f294006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:39.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.775+0000 7f295f7fe700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f294006a2e0 0x7f294006c7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2950000c00 tx=0x7f2950008030 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:39.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:39.779+0000 7f2957fff700 1 -- 172.21.15.45:0/2918464092 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29600c7c70 con 0x7f29600cd880 2023-08-27T11:24:40.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.012+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2960071c30 con 0x7f294006a2e0 2023-08-27T11:24:40.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.092+0000 7f2957fff700 1 -- 172.21.15.45:0/2918464092 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f295001f070 con 0x7f294006a2e0 2023-08-27T11:24:40.096 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-08-27T11:24:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f294006a2e0 msgr2=0x7f294006c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f294006a2e0 0x7f294006c7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2950000c00 tx=0x7f2950008030 comp rx=0 tx=0).stop 2023-08-27T11:24:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29600cd880 msgr2=0x7f29600c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29600cd880 0x7f29600c84b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f294800b4a0 tx=0x7f294800b860 comp rx=0 tx=0).stop 2023-08-27T11:24:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 shutdown_connections 2023-08-27T11:24:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f294006a2e0 0x7f294006c7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f296015d340 0x7f2960151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f296006a990 0x7f29600adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 --2- 172.21.15.45:0/2918464092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29600cd880 0x7f29600c84b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.094+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 >> 172.21.15.45:0/2918464092 conn(0x7f29600cc7c0 msgr2=0x7f2960071000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.095+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 shutdown_connections 2023-08-27T11:24:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:40.095+0000 7f29661c0700 1 -- 172.21.15.45:0/2918464092 wait complete. 2023-08-27T11:24:40.109 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-08-27T11:24:40.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.110 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-08-27T11:24:40.128 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:40.129 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:40.129 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:40.129 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:40.129 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:40.129 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:40.130 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:40.130 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:40.130 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:40.130 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:40.130 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:40.131 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:40.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.131 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_14_00_UTC) 2023-08-27T11:24:40.197 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:40.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.198 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:40.261 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:40.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.262 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:40.325 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:40.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.326 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:40.393 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:40.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.394 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:40.458 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:40.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.459 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:40.525 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:40.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.527 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:40.589 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:40.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.590 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:40.654 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:40.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.655 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:40.717 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:40.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.718 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:40.781 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:40.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.781 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:40.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:40.844 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-08-27T11:24:40.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-08-27T11:24:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.072+0000 7f94059c0700 1 Processor -- start 2023-08-27T11:24:41.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.072+0000 7f94059c0700 1 -- start start 2023-08-27T11:24:41.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f94059c0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f940015d2c0 0x7f94000d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:41.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f94059c0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94000b8c20 0x7f94000cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:41.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f94059c0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f940015d6a0 con 0x7f94000ce2e0 2023-08-27T11:24:41.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f94059c0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94000ce730 con 0x7f94000b8c20 2023-08-27T11:24:41.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f94059c0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94000d2c90 con 0x7f940015d2c0 2023-08-27T11:24:41.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f93fe7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94000b8c20 0x7f94000cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:41.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f93ff7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f94000ce2e0 0x7f940006c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42146/0 (socket says 172.21.15.45:42146) 2023-08-27T11:24:41.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f93ff7fe700 1 -- 172.21.15.45:0/1905286685 learned_addr learned my addr 172.21.15.45:0/1905286685 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:41.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f93feffd700 1 --2- 172.21.15.45:0/1905286685 >> v2:172.21.15.135:3300/0 conn(0x7f940015d2c0 0x7f94000d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:41.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f93fe7fc700 1 -- 172.21.15.45:0/1905286685 >> v2:172.21.15.135:3300/0 conn(0x7f940015d2c0 msgr2=0x7f94000d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:41.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f93fe7fc700 1 --2- 172.21.15.45:0/1905286685 >> v2:172.21.15.135:3300/0 conn(0x7f940015d2c0 0x7f94000d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:41.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f93fe7fc700 1 -- 172.21.15.45:0/1905286685 >> v1:172.21.15.45:6789/0 conn(0x7f94000ce2e0 legacy=0x7f940006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:41.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.073+0000 7f93fdffb700 1 -- 172.21.15.45:0/1905286685 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190093571 0 0) 0x7f93f8001490 con 0x7f94000ce2e0 2023-08-27T11:24:41.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f93fe7fc700 1 -- 172.21.15.45:0/1905286685 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000d1bf0 con 0x7f94000b8c20 2023-08-27T11:24:41.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f93fe7fc700 1 --2- 172.21.15.45:0/1905286685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000b8c20 0x7f94000cdda0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009840 tx=0x7f93f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7f702ac93de24b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:41.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f93fdffb700 1 -- 172.21.15.45:0/1905286685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0011830 con 0x7f94000b8c20 2023-08-27T11:24:41.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f93fdffb700 1 -- 172.21.15.45:0/1905286685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f0011990 con 0x7f94000b8c20 2023-08-27T11:24:41.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f93fdffb700 1 -- 172.21.15.45:0/1905286685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0011d20 con 0x7f94000b8c20 2023-08-27T11:24:41.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f94059c0700 1 -- 172.21.15.45:0/1905286685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000b8c20 msgr2=0x7f94000cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:41.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f94059c0700 1 --2- 172.21.15.45:0/1905286685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000b8c20 0x7f94000cdda0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009840 tx=0x7f93f0009b50 comp rx=0 tx=0).stop 2023-08-27T11:24:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f94059c0700 1 -- 172.21.15.45:0/1905286685 shutdown_connections 2023-08-27T11:24:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f94059c0700 1 --2- 172.21.15.45:0/1905286685 >> v2:172.21.15.135:3300/0 conn(0x7f940015d2c0 0x7f94000d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f94059c0700 1 --2- 172.21.15.45:0/1905286685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000b8c20 0x7f94000cdda0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f94059c0700 1 -- 172.21.15.45:0/1905286685 >> 172.21.15.45:0/1905286685 conn(0x7f94000d2fd0 msgr2=0x7f94000d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f94059c0700 1 -- 172.21.15.45:0/1905286685 shutdown_connections 2023-08-27T11:24:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.074+0000 7f94059c0700 1 -- 172.21.15.45:0/1905286685 wait complete. 2023-08-27T11:24:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.076+0000 7f94059c0700 1 Processor -- start 2023-08-27T11:24:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.076+0000 7f94059c0700 1 -- start start 2023-08-27T11:24:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.076+0000 7f94059c0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94000b8c20 0x7f94001f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.076+0000 7f94059c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000ce2e0 0x7f94000c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.076+0000 7f94059c0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f940015d2c0 0x7f94000c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.076+0000 7f94059c0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001518d0 con 0x7f94000ce2e0 2023-08-27T11:24:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.076+0000 7f94059c0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940015e710 con 0x7f940015d2c0 2023-08-27T11:24:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.076+0000 7f94059c0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940015db00 con 0x7f94000b8c20 2023-08-27T11:24:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93fe7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000ce2e0 0x7f94000c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93fe7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000ce2e0 0x7f94000c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55374/0 (socket says 172.21.15.45:55374) 2023-08-27T11:24:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93fe7fc700 1 -- 172.21.15.45:0/2585773309 learned_addr learned my addr 172.21.15.45:0/2585773309 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93ff7fe700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f940015d2c0 0x7f94000c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93feffd700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94000b8c20 0x7f94001f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93fe7fc700 1 -- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94000b8c20 msgr2=0x7f94001f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93fe7fc700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94000b8c20 0x7f94001f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93fe7fc700 1 -- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f940015d2c0 msgr2=0x7f94000c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:41.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93fe7fc700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f940015d2c0 0x7f94000c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93fe7fc700 1 -- 172.21.15.45:0/2585773309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f00094a0 con 0x7f94000ce2e0 2023-08-27T11:24:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93ff7fe700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f940015d2c0 0x7f94000c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93feffd700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94000b8c20 0x7f94001f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93fe7fc700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000ce2e0 0x7f94000c9080 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f93f00090b0 tx=0x7f93f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93e7fff700 1 -- 172.21.15.45:0/2585773309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f00038a0 con 0x7f94000ce2e0 2023-08-27T11:24:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93e7fff700 1 -- 172.21.15.45:0/2585773309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f001e070 con 0x7f94000ce2e0 2023-08-27T11:24:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940015dde0 con 0x7f94000ce2e0 2023-08-27T11:24:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f93e7fff700 1 -- 172.21.15.45:0/2585773309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0003c20 con 0x7f94000ce2e0 2023-08-27T11:24:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.077+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940015e3a0 con 0x7f94000ce2e0 2023-08-27T11:24:41.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.078+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940015e880 con 0x7f94000ce2e0 2023-08-27T11:24:41.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.079+0000 7f93e7fff700 1 -- 172.21.15.45:0/2585773309 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f93f002d450 con 0x7f94000ce2e0 2023-08-27T11:24:41.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.079+0000 7f93e7fff700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f93ec06a2e0 0x7f93ec06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:41.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.079+0000 7f93e7fff700 1 -- 172.21.15.45:0/2585773309 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7234+0+0 (secure 0 0 0) 0x7f93f0089da0 con 0x7f94000ce2e0 2023-08-27T11:24:41.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.079+0000 7f93feffd700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f93ec06a2e0 0x7f93ec06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:41.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.080+0000 7f93feffd700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f93ec06a2e0 0x7f93ec06c7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009a00 tx=0x7f93f4008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:41.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.083+0000 7f93e7fff700 1 -- 172.21.15.45:0/2585773309 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f93f0054e00 con 0x7f94000ce2e0 2023-08-27T11:24:41.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.327+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 --> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f94000d21b0 con 0x7f93ec06a2e0 2023-08-27T11:24:41.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.433+0000 7f93e7fff700 1 -- 172.21.15.45:0/2585773309 <== mgr.5141 v2:172.21.15.135:6832/584312721 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f93f4009610 con 0x7f93ec06a2e0 2023-08-27T11:24:41.435 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-08-27T11:24:41.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f93ec06a2e0 msgr2=0x7f93ec06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:41.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f93ec06a2e0 0x7f93ec06c7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009a00 tx=0x7f93f4008040 comp rx=0 tx=0).stop 2023-08-27T11:24:41.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000ce2e0 msgr2=0x7f94000c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:41.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000ce2e0 0x7f94000c9080 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f93f00090b0 tx=0x7f93f0004750 comp rx=0 tx=0).stop 2023-08-27T11:24:41.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 shutdown_connections 2023-08-27T11:24:41.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f93ec06a2e0 0x7f93ec06c7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:41.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f940015d2c0 0x7f94000c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:41.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94000b8c20 0x7f94001f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:41.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 --2- 172.21.15.45:0/2585773309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94000ce2e0 0x7f94000c9080 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:41.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.435+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 >> 172.21.15.45:0/2585773309 conn(0x7f94000d2fd0 msgr2=0x7f94000b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:41.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.436+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 shutdown_connections 2023-08-27T11:24:41.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:41.436+0000 7f94059c0700 1 -- 172.21.15.45:0/2585773309 wait complete. 2023-08-27T11:24:41.449 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-08-27T11:24:41.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.450 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-08-27T11:24:41.468 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:41.469 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:41.469 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:41.469 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:41.469 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:41.470 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:41.470 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:41.470 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:41.470 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:41.470 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:41.470 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:41.471 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_14_00_UTC 2023-08-27T11:24:41.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.472 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_14_00_UTC) 2023-08-27T11:24:41.534 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_15_00_UTC 2023-08-27T11:24:41.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.535 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_15_00_UTC) 2023-08-27T11:24:41.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_16_00_UTC 2023-08-27T11:24:41.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.598 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_16_00_UTC) 2023-08-27T11:24:41.661 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_17_00_UTC 2023-08-27T11:24:41.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.662 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_17_00_UTC) 2023-08-27T11:24:41.724 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_18_00_UTC 2023-08-27T11:24:41.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.725 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_18_00_UTC) 2023-08-27T11:24:41.787 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_19_00_UTC 2023-08-27T11:24:41.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.788 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_19_00_UTC) 2023-08-27T11:24:41.850 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_20_00_UTC 2023-08-27T11:24:41.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.851 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_20_00_UTC) 2023-08-27T11:24:41.917 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_21_00_UTC 2023-08-27T11:24:41.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.917 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_21_00_UTC) 2023-08-27T11:24:41.981 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_22_00_UTC 2023-08-27T11:24:41.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:41.982 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_22_00_UTC) 2023-08-27T11:24:42.047 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_23_00_UTC 2023-08-27T11:24:42.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.048 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_23_00_UTC) 2023-08-27T11:24:42.113 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_24_00_UTC 2023-08-27T11:24:42.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.114 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-27-11_24_00_UTC) 2023-08-27T11:24:42.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:42.192 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-08-27T11:24:42.254 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-27T11:24:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.449+0000 7f22eef79700 1 Processor -- start 2023-08-27T11:24:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.449+0000 7f22eef79700 1 -- start start 2023-08-27T11:24:42.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22eef79700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80b2d60 0x7f22e806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:42.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22eef79700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f22e80aea50 0x7f22e806a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:42.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22eef79700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e80cbd70 con 0x7f22e815d480 2023-08-27T11:24:42.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22eef79700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e815d860 con 0x7f22e80aea50 2023-08-27T11:24:42.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22eef79700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80cc330 con 0x7f22e80b2d60 2023-08-27T11:24:42.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22ecd15700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f22e815d480 0x7f22e80cc7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42156/0 (socket says 172.21.15.45:42156) 2023-08-27T11:24:42.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22ecd15700 1 -- 172.21.15.45:0/3659987239 learned_addr learned my addr 172.21.15.45:0/3659987239 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:42.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22ed516700 1 --2- 172.21.15.45:0/3659987239 >> v2:172.21.15.135:3300/0 conn(0x7f22e80aea50 0x7f22e806a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:42.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151556690 0 0) 0x7f22e80cbd70 con 0x7f22e815d480 2023-08-27T11:24:42.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.450+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d80035b0 con 0x7f22e815d480 2023-08-27T11:24:42.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e7fff700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80b2d60 0x7f22e806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:42.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1981152426 0 0) 0x7f22d80035b0 con 0x7f22e815d480 2023-08-27T11:24:42.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80b2d60 msgr2=0x7f22e806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:42.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80b2d60 0x7f22e806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:42.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 >> v2:172.21.15.135:3300/0 conn(0x7f22e80aea50 msgr2=0x7f22e806a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:42.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 --2- 172.21.15.45:0/3659987239 >> v2:172.21.15.135:3300/0 conn(0x7f22e80aea50 0x7f22e806a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:42.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e8073ab0 con 0x7f22e815d480 2023-08-27T11:24:42.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f22dc001670 con 0x7f22e815d480 2023-08-27T11:24:42.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 >> v1:172.21.15.45:6789/0 conn(0x7f22e815d480 legacy=0x7f22e80cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:42.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8003a00 0x7f22d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:42.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8004850 0x7f22d8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:42.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80b2d60 0x7f22d8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:42.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e815d860 con 0x7f22d8003a00 2023-08-27T11:24:42.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80cc330 con 0x7f22d8004850 2023-08-27T11:24:42.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d800ba10 con 0x7f22e80b2d60 2023-08-27T11:24:42.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22ecd15700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8003a00 0x7f22d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:42.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22e7fff700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8004850 0x7f22d8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:42.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22ed516700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80b2d60 0x7f22d8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:42.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22ecd15700 1 -- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80b2d60 msgr2=0x7f22d8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:42.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22ecd15700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80b2d60 0x7f22d8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:42.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22ecd15700 1 -- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8004850 msgr2=0x7f22d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:42.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22ecd15700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22d8004850 0x7f22d8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:42.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22ecd15700 1 -- 172.21.15.45:0/3659987239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80035b0 con 0x7f22d8003a00 2023-08-27T11:24:42.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.451+0000 7f22ecd15700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8003a00 0x7f22d8003e20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c2f0 tx=0x7f22d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9840ad0d1b41d2e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:42.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc007c40 con 0x7f22d8003a00 2023-08-27T11:24:42.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc00ce30 con 0x7f22d8003a00 2023-08-27T11:24:42.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22e77fe700 1 -- 172.21.15.45:0/3659987239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc005ba0 con 0x7f22d8003a00 2023-08-27T11:24:42.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22eef79700 1 -- 172.21.15.45:0/3659987239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8003a00 msgr2=0x7f22d8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:42.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22eef79700 1 --2- 172.21.15.45:0/3659987239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8003a00 0x7f22d8003e20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c2f0 tx=0x7f22d8005880 comp rx=0 tx=0).stop 2023-08-27T11:24:42.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22ed516700 1 -- 172.21.15.45:0/3659987239 reap_dead start 2023-08-27T11:24:42.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22eef79700 1 -- 172.21.15.45:0/3659987239 shutdown_connections 2023-08-27T11:24:42.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22eef79700 1 -- 172.21.15.45:0/3659987239 >> 172.21.15.45:0/3659987239 conn(0x7f22e80b4650 msgr2=0x7f22e80b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:42.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22eef79700 1 -- 172.21.15.45:0/3659987239 shutdown_connections 2023-08-27T11:24:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.452+0000 7f22eef79700 1 -- 172.21.15.45:0/3659987239 wait complete. 2023-08-27T11:24:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.453+0000 7f22eef79700 1 Processor -- start 2023-08-27T11:24:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.453+0000 7f22eef79700 1 -- start start 2023-08-27T11:24:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.454+0000 7f22eef79700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e81f2290 0x7f22e80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.454+0000 7f22eef79700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80acd70 0x7f22e815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:42.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.454+0000 7f22eef79700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e815e4e0 0x7f22e80b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:42.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.454+0000 7f22eef79700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80c49b0 con 0x7f22e81f2290 2023-08-27T11:24:42.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.454+0000 7f22eef79700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80c5ec0 con 0x7f22e815e4e0 2023-08-27T11:24:42.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.454+0000 7f22eef79700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80c4bc0 con 0x7f22e80acd70 2023-08-27T11:24:42.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.454+0000 7f22ecd15700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e81f2290 0x7f22e80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:42.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.454+0000 7f22ecd15700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e81f2290 0x7f22e80c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55392/0 (socket says 172.21.15.45:55392) 2023-08-27T11:24:42.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.454+0000 7f22ecd15700 1 -- 172.21.15.45:0/4100539511 learned_addr learned my addr 172.21.15.45:0/4100539511 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:42.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22e7fff700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80acd70 0x7f22e815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:42.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22ed516700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e815e4e0 0x7f22e80b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:42.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22ecd15700 1 -- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80acd70 msgr2=0x7f22e815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:42.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22ecd15700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80acd70 0x7f22e815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:42.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22ecd15700 1 -- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e815e4e0 msgr2=0x7f22e80b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:42.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22ecd15700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e815e4e0 0x7f22e80b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:42.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22ecd15700 1 -- 172.21.15.45:0/4100539511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e80c4ea0 con 0x7f22e81f2290 2023-08-27T11:24:42.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22ecd15700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e81f2290 0x7f22e80c5980 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0048d0 tx=0x7f22e8072a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:42.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22e57fa700 1 -- 172.21.15.45:0/4100539511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0070d0 con 0x7f22e81f2290 2023-08-27T11:24:42.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22e57fa700 1 -- 172.21.15.45:0/4100539511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc007250 con 0x7f22e81f2290 2023-08-27T11:24:42.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22e57fa700 1 -- 172.21.15.45:0/4100539511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc007600 con 0x7f22e81f2290 2023-08-27T11:24:42.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e80c5180 con 0x7f22e81f2290 2023-08-27T11:24:42.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.455+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e80ad1f0 con 0x7f22e81f2290 2023-08-27T11:24:42.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.456+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e80c57d0 con 0x7f22e81f2290 2023-08-27T11:24:42.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.456+0000 7f22e57fa700 1 -- 172.21.15.45:0/4100539511 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f22dc012400 con 0x7f22e81f2290 2023-08-27T11:24:42.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.457+0000 7f22e57fa700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22cc06a2e0 0x7f22cc06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:42.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.457+0000 7f22e57fa700 1 -- 172.21.15.45:0/4100539511 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7106+0+0 (secure 0 0 0) 0x7f22dc089060 con 0x7f22e81f2290 2023-08-27T11:24:42.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.457+0000 7f22e7fff700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22cc06a2e0 0x7f22cc06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:42.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.458+0000 7f22e7fff700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22cc06a2e0 0x7f22cc06c7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f22e806c830 tx=0x7f22d4008040 comp rx=0 tx=0).ready entity=mgr.5141 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:42.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.462+0000 7f22e57fa700 1 -- 172.21.15.45:0/4100539511 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22dc054140 con 0x7f22e81f2290 2023-08-27T11:24:42.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:42.719+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f22e80ad380 con 0x7f22e81f2290 2023-08-27T11:24:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.124+0000 7f22e57fa700 1 -- 172.21.15.45:0/4100539511 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7f22dc054510 con 0x7f22e81f2290 2023-08-27T11:24:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.125+0000 7f22e57fa700 1 -- 172.21.15.45:0/4100539511 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v14) v1 ==== 93+0+0 (secure 0 0 0) 0x7f22dc057bd0 con 0x7f22e81f2290 2023-08-27T11:24:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22cc06a2e0 msgr2=0x7f22cc06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22cc06a2e0 0x7f22cc06c7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f22e806c830 tx=0x7f22d4008040 comp rx=0 tx=0).stop 2023-08-27T11:24:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e81f2290 msgr2=0x7f22e80c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e81f2290 0x7f22e80c5980 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0048d0 tx=0x7f22e8072a60 comp rx=0 tx=0).stop 2023-08-27T11:24:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 shutdown_connections 2023-08-27T11:24:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f22cc06a2e0 0x7f22cc06c7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22e81f2290 0x7f22e80c5980 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22e815e4e0 0x7f22e80b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 --2- 172.21.15.45:0/4100539511 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f22e80acd70 0x7f22e815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 >> 172.21.15.45:0/4100539511 conn(0x7f22e80b4650 msgr2=0x7f22e806a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 shutdown_connections 2023-08-27T11:24:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.127+0000 7f22eef79700 1 -- 172.21.15.45:0/4100539511 wait complete. 2023-08-27T11:24:43.142 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:24:43.143 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:24:43.198 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:24:43.240 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:24:43.241 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:24:43.241 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.259 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:24:43.259 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:43.291 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:43.289+0000 7fcc44fe2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:24:43.291 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:43.291+0000 7f086130e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:24:43.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:24:43.359 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:43.358+0000 7f086130e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:24:43.366 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:43.364+0000 7fcc44fe2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:24:43.437 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:43.436+0000 7f086130e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:24:43.469 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:43.467+0000 7fcc44fe2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:24:43.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.498+0000 7f3faccf4700 1 Processor -- start 2023-08-27T11:24:43.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.498+0000 7f3faccf4700 1 -- start start 2023-08-27T11:24:43.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3faccf4700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3fa815d450 0x7f3fa80cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3faccf4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa80cbaf0 0x7f3fa80cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3faccf4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3fa80b2140 0x7f3fa80d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3faccf4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa80ce1c0 con 0x7f3fa80b2140 2023-08-27T11:24:43.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3faccf4700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa80cf290 con 0x7f3fa815d450 2023-08-27T11:24:43.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3faccf4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa80ce440 con 0x7f3fa80cbaf0 2023-08-27T11:24:43.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa6d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3fa80b2140 0x7f3fa80d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa6d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3fa80b2140 0x7f3fa80d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55448/0 (socket says 172.21.15.45:55448) 2023-08-27T11:24:43.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa6d9d700 1 -- 172.21.15.45:0/1659295787 learned_addr learned my addr 172.21.15.45:0/1659295787 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:43.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa659c700 1 --2- 172.21.15.45:0/1659295787 >> v2:172.21.15.135:3300/0 conn(0x7f3fa815d450 0x7f3fa80cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa5d9b700 1 --2- 172.21.15.45:0/1659295787 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa80cbaf0 0x7f3fa80cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa6d9d700 1 -- 172.21.15.45:0/1659295787 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa80cbaf0 msgr2=0x7f3fa80cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:43.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa6d9d700 1 --2- 172.21.15.45:0/1659295787 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa80cbaf0 0x7f3fa80cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa6d9d700 1 -- 172.21.15.45:0/1659295787 >> v2:172.21.15.135:3300/0 conn(0x7f3fa815d450 msgr2=0x7f3fa80cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:43.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa6d9d700 1 --2- 172.21.15.45:0/1659295787 >> v2:172.21.15.135:3300/0 conn(0x7f3fa815d450 0x7f3fa80cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.499+0000 7f3fa6d9d700 1 -- 172.21.15.45:0/1659295787 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa80ce720 con 0x7f3fa80b2140 2023-08-27T11:24:43.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3fa6d9d700 1 --2- 172.21.15.45:0/1659295787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 0x7f3fa80d0270 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80b3b50 tx=0x7f3fa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3355e313ca0797bd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:43.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3fa559a700 1 -- 172.21.15.45:0/1659295787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa000e070 con 0x7f3fa80b2140 2023-08-27T11:24:43.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3fa559a700 1 -- 172.21.15.45:0/1659295787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa000b910 con 0x7f3fa80b2140 2023-08-27T11:24:43.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3fa559a700 1 -- 172.21.15.45:0/1659295787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa000bcc0 con 0x7f3fa80b2140 2023-08-27T11:24:43.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3faccf4700 1 -- 172.21.15.45:0/1659295787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 msgr2=0x7f3fa80d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3faccf4700 1 --2- 172.21.15.45:0/1659295787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 0x7f3fa80d0270 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80b3b50 tx=0x7f3fa000b330 comp rx=0 tx=0).stop 2023-08-27T11:24:43.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3faccf4700 1 -- 172.21.15.45:0/1659295787 shutdown_connections 2023-08-27T11:24:43.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3faccf4700 1 --2- 172.21.15.45:0/1659295787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 0x7f3fa80d0270 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3faccf4700 1 --2- 172.21.15.45:0/1659295787 >> v2:172.21.15.135:3300/0 conn(0x7f3fa815d450 0x7f3fa80cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3faccf4700 1 --2- 172.21.15.45:0/1659295787 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa80cbaf0 0x7f3fa80cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.500+0000 7f3faccf4700 1 -- 172.21.15.45:0/1659295787 >> 172.21.15.45:0/1659295787 conn(0x7f3fa80d0f60 msgr2=0x7f3fa80d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.501+0000 7f3faccf4700 1 -- 172.21.15.45:0/1659295787 shutdown_connections 2023-08-27T11:24:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.501+0000 7f3faccf4700 1 -- 172.21.15.45:0/1659295787 wait complete. 2023-08-27T11:24:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.502+0000 7f3faccf4700 1 Processor -- start 2023-08-27T11:24:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.502+0000 7f3faccf4700 1 -- start start 2023-08-27T11:24:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3faccf4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 0x7f3fa8159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3faccf4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fa80cbaf0 0x7f3fa806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3faccf4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa815d450 0x7f3fa806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3faccf4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa80b51d0 con 0x7f3fa80b2140 2023-08-27T11:24:43.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3faccf4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa80c8cb0 con 0x7f3fa80cbaf0 2023-08-27T11:24:43.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3faccf4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa80c8f90 con 0x7f3fa815d450 2023-08-27T11:24:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 0x7f3fa8159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 0x7f3fa8159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55450/0 (socket says 172.21.15.45:55450) 2023-08-27T11:24:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa659c700 1 -- 172.21.15.45:0/3095249117 learned_addr learned my addr 172.21.15.45:0/3095249117 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa5d9b700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fa80cbaf0 0x7f3fa806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa6d9d700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa815d450 0x7f3fa806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa659c700 1 -- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa815d450 msgr2=0x7f3fa806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa659c700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa815d450 0x7f3fa806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa659c700 1 -- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fa80cbaf0 msgr2=0x7f3fa806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa659c700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fa80cbaf0 0x7f3fa806e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa659c700 1 -- 172.21.15.45:0/3095249117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0009b30 con 0x7f3fa80b2140 2023-08-27T11:24:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3fa659c700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 0x7f3fa8159810 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00e870 tx=0x7f3f9c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3f96ffd700 1 -- 172.21.15.45:0/3095249117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c009590 con 0x7f3fa80b2140 2023-08-27T11:24:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3f96ffd700 1 -- 172.21.15.45:0/3095249117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c013680 con 0x7f3fa80b2140 2023-08-27T11:24:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.503+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa80c9270 con 0x7f3fa80b2140 2023-08-27T11:24:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.504+0000 7f3f96ffd700 1 -- 172.21.15.45:0/3095249117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c0139f0 con 0x7f3fa80b2140 2023-08-27T11:24:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.504+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa8159d50 con 0x7f3fa80b2140 2023-08-27T11:24:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.504+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa80c9790 con 0x7f3fa80b2140 2023-08-27T11:24:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.505+0000 7f3f96ffd700 1 -- 172.21.15.45:0/3095249117 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7f3f9c013b90 con 0x7f3fa80b2140 2023-08-27T11:24:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.505+0000 7f3f96ffd700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3f8806a290 0x7f3f8806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.505+0000 7f3f96ffd700 1 -- 172.21.15.45:0/3095249117 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7902+0+0 (secure 0 0 0) 0x7f3f9c088e20 con 0x7f3fa80b2140 2023-08-27T11:24:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.505+0000 7f3fa5d9b700 1 -- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3f8806a290 msgr2=0x7f3f8806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.505+0000 7f3fa5d9b700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3f8806a290 0x7f3f8806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:43.517 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:43.515+0000 7f086130e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:24:43.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.510+0000 7f3f96ffd700 1 -- 172.21.15.45:0/3095249117 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3fa80c9790 con 0x7f3fa80b2140 2023-08-27T11:24:43.556 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:43.555+0000 7fcc44fe2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:24:43.657 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:43.656+0000 7f086130e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:24:43.706 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:43.704+0000 7fcc44fe2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:24:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.706+0000 7f3fa5d9b700 1 -- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3f8806a290 msgr2=0x7f3f8806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:43.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.706+0000 7f3fa5d9b700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3f8806a290 0x7f3f8806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:43.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.714+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fa80c9790 con 0x7f3fa80b2140 2023-08-27T11:24:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.716+0000 7f3f96ffd700 1 -- 172.21.15.45:0/3095249117 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+20834 (secure 0 0 0) 0x7f3fa80c9790 con 0x7f3fa80b2140 2023-08-27T11:24:43.718 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:43.718 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":55,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:24:42.997994+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:13:35.610850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-27T11:13:36.282109+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":217,"snap_epoch":55,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/2587284130":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1110669507":"2023-08-28T11:13:52.047816+0000","172.21.15.135:6833/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:6832/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1692978501":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/2898860171":"2023-08-28T11:13:52.047816+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":4,"length":2},{"begin":8,"length":2},{"begin":12,"length":1},{"begin":14,"length":2},{"begin":18,"length":2},{"begin":23,"length":1},{"begin":25,"length":2},{"begin":29,"length":2},{"begin":33,"length":1},{"begin":35,"length":2},{"begin":39,"length":3},{"begin":44,"length":1},{"begin":46,"length":2},{"begin":50,"length":2},{"begin":54,"length":1},{"begin":56,"length":2},{"begin":60,"length":2},{"begin":64,"length":1},{"begin":66,"length":2},{"begin":70,"length":3},{"begin":75,"length":1},{"begin":77,"length":2},{"begin":81,"length":2},{"begin":85,"length":1},{"begin":87,"length":2},{"begin":92,"length":2},{"begin":98,"length":2},{"begin":102,"length":3},{"begin":108,"length":2},{"begin":165,"length":53}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":9,"length":1},{"begin":12,"length":1},{"begin":23,"length":1},{"begin":33,"length":1},{"begin":44,"length":1},{"begin":54,"length":1},{"begin":64,"length":1},{"begin":75,"length":1},{"begin":85,"length":1},{"begin":102,"length":1},{"begin":208,"length":10}]}],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2},{"begin":10,"length":2},{"begin":13,"length":1},{"begin":16,"length":2},{"begin":20,"length":3},{"begin":24,"length":1},{"begin":27,"length":2},{"begin":31,"length":2},{"begin":34,"length":1},{"begin":37,"length":2},{"begin":42,"length":2},{"begin":45,"length":1},{"begin":48,"length":2},{"begin":52,"length":2},{"begin":55,"length":1},{"begin":58,"length":2},{"begin":62,"length":2},{"begin":65,"length":1},{"begin":68,"length":2},{"begin":73,"length":2},{"begin":76,"length":1},{"begin":79,"length":2},{"begin":83,"length":2},{"begin":86,"length":1},{"begin":89,"length":3},{"begin":94,"length":2},{"begin":97,"length":1},{"begin":100,"length":2},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":110,"length":55}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:24:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3f8806a290 msgr2=0x7f3f8806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3f8806a290 0x7f3f8806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 msgr2=0x7f3fa8159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 0x7f3fa8159810 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00e870 tx=0x7f3f9c00eb80 comp rx=0 tx=0).stop 2023-08-27T11:24:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 shutdown_connections 2023-08-27T11:24:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3f8806a290 0x7f3f8806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fa80b2140 0x7f3fa8159810 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3fa815d450 0x7f3fa806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 --2- 172.21.15.45:0/3095249117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fa80cbaf0 0x7f3fa806e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 >> 172.21.15.45:0/3095249117 conn(0x7f3fa80d0f60 msgr2=0x7f3fa80d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 shutdown_connections 2023-08-27T11:24:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.718+0000 7f3faccf4700 1 -- 172.21.15.45:0/3095249117 wait complete. 2023-08-27T11:24:43.733 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:13:35.610850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:13:36.282109+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 217, 'snap_epoch': 55, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:24:43.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:24:43.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.902+0000 7f9955702700 1 Processor -- start 2023-08-27T11:24:43.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.902+0000 7f9955702700 1 -- start start 2023-08-27T11:24:43.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f9955702700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f995015d3a0 0x7f99500d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f9955702700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f99500d02c0 0x7f995006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f9955702700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995015d780 con 0x7f99500b0320 2023-08-27T11:24:43.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f9955702700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500cfe30 con 0x7f995015d3a0 2023-08-27T11:24:43.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f9955702700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500cf220 con 0x7f99500d02c0 2023-08-27T11:24:43.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f995015d3a0 0x7f99500d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f99500b0320 0x7f99500cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42258/0 (socket says 172.21.15.45:42258) 2023-08-27T11:24:43.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994f7fe700 1 -- 172.21.15.45:0/4080888350 learned_addr learned my addr 172.21.15.45:0/4080888350 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:43.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994e7fc700 1 --2- 172.21.15.45:0/4080888350 >> v2:172.21.15.135:3300/0 conn(0x7f99500d02c0 0x7f995006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994effd700 1 -- 172.21.15.45:0/4080888350 >> v2:172.21.15.135:3300/0 conn(0x7f99500d02c0 msgr2=0x7f995006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:43.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994effd700 1 --2- 172.21.15.45:0/4080888350 >> v2:172.21.15.135:3300/0 conn(0x7f99500d02c0 0x7f995006bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994effd700 1 -- 172.21.15.45:0/4080888350 >> v1:172.21.15.45:6789/0 conn(0x7f99500b0320 legacy=0x7f99500cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994dffb700 1 -- 172.21.15.45:0/4080888350 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167367513 0 0) 0x7f9948001490 con 0x7f99500b0320 2023-08-27T11:24:43.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994effd700 1 -- 172.21.15.45:0/4080888350 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500cf500 con 0x7f995015d3a0 2023-08-27T11:24:43.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994e7fc700 1 --2- 172.21.15.45:0/4080888350 >> v2:172.21.15.135:3300/0 conn(0x7f99500d02c0 0x7f995006bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:43.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.903+0000 7f994effd700 1 --2- 172.21.15.45:0/4080888350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015d3a0 0x7f99500d1550 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f99500719f0 tx=0x7f9940009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c503491ba15f8f06 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:43.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f994dffb700 1 -- 172.21.15.45:0/4080888350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940011830 con 0x7f995015d3a0 2023-08-27T11:24:43.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f994dffb700 1 -- 172.21.15.45:0/4080888350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9940011990 con 0x7f995015d3a0 2023-08-27T11:24:43.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f994dffb700 1 -- 172.21.15.45:0/4080888350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940011d20 con 0x7f995015d3a0 2023-08-27T11:24:43.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f9955702700 1 -- 172.21.15.45:0/4080888350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015d3a0 msgr2=0x7f99500d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f9955702700 1 --2- 172.21.15.45:0/4080888350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015d3a0 0x7f99500d1550 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f99500719f0 tx=0x7f9940009b20 comp rx=0 tx=0).stop 2023-08-27T11:24:43.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f9955702700 1 -- 172.21.15.45:0/4080888350 shutdown_connections 2023-08-27T11:24:43.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f9955702700 1 --2- 172.21.15.45:0/4080888350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015d3a0 0x7f99500d1550 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f9955702700 1 --2- 172.21.15.45:0/4080888350 >> v2:172.21.15.135:3300/0 conn(0x7f99500d02c0 0x7f995006bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f9955702700 1 -- 172.21.15.45:0/4080888350 >> 172.21.15.45:0/4080888350 conn(0x7f995009a1a0 msgr2=0x7f99500d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:43.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f9955702700 1 -- 172.21.15.45:0/4080888350 shutdown_connections 2023-08-27T11:24:43.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.904+0000 7f9955702700 1 -- 172.21.15.45:0/4080888350 wait complete. 2023-08-27T11:24:43.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.906+0000 7f9955702700 1 Processor -- start 2023-08-27T11:24:43.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.906+0000 7f9955702700 1 -- start start 2023-08-27T11:24:43.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.906+0000 7f9955702700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99500b0320 0x7f99500b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.906+0000 7f9955702700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500d02c0 0x7f9950154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.906+0000 7f9955702700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f995015d3a0 0x7f99500b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.906+0000 7f9955702700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99500b3630 con 0x7f99500d02c0 2023-08-27T11:24:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.906+0000 7f9955702700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950155530 con 0x7f995015d3a0 2023-08-27T11:24:43.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.906+0000 7f9955702700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99500b5650 con 0x7f99500b0320 2023-08-27T11:24:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500d02c0 0x7f9950154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500d02c0 0x7f9950154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55486/0 (socket says 172.21.15.45:55486) 2023-08-27T11:24:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994e7fc700 1 -- 172.21.15.45:0/2645805052 learned_addr learned my addr 172.21.15.45:0/2645805052 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994effd700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99500b0320 0x7f99500b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994f7fe700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f995015d3a0 0x7f99500b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994e7fc700 1 -- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99500b0320 msgr2=0x7f99500b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994e7fc700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99500b0320 0x7f99500b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994e7fc700 1 -- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f995015d3a0 msgr2=0x7f99500b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994e7fc700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f995015d3a0 0x7f99500b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:43.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994e7fc700 1 -- 172.21.15.45:0/2645805052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400094a0 con 0x7f99500d02c0 2023-08-27T11:24:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f994e7fc700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500d02c0 0x7f9950154ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f994400e840 tx=0x7f994400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f99377fe700 1 -- 172.21.15.45:0/2645805052 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944013610 con 0x7f99500d02c0 2023-08-27T11:24:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f99377fe700 1 -- 172.21.15.45:0/2645805052 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9944013770 con 0x7f99500d02c0 2023-08-27T11:24:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99500b5930 con 0x7f99500d02c0 2023-08-27T11:24:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f99377fe700 1 -- 172.21.15.45:0/2645805052 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944013b20 con 0x7f99500d02c0 2023-08-27T11:24:43.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.907+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99500b5ef0 con 0x7f99500d02c0 2023-08-27T11:24:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.908+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99500b37b0 con 0x7f99500d02c0 2023-08-27T11:24:43.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.909+0000 7f99377fe700 1 -- 172.21.15.45:0/2645805052 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7f9944013c80 con 0x7f99500d02c0 2023-08-27T11:24:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.909+0000 7f99377fe700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f993c06a290 0x7f993c06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.909+0000 7f99377fe700 1 -- 172.21.15.45:0/2645805052 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7902+0+0 (secure 0 0 0) 0x7f9944088c30 con 0x7f99500d02c0 2023-08-27T11:24:43.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.909+0000 7f994effd700 1 -- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f993c06a290 msgr2=0x7f993c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.910+0000 7f994effd700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f993c06a290 0x7f993c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:43.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:43.914+0000 7f99377fe700 1 -- 172.21.15.45:0/2645805052 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f99440537e0 con 0x7f99500d02c0 2023-08-27T11:24:43.982 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:43.981+0000 7f086130e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:24:44.067 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:44.065+0000 7fcc44fe2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:24:44.107 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:44.106+0000 7f086130e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:24:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.111+0000 7f994effd700 1 -- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f993c06a290 msgr2=0x7f993c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:44.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.111+0000 7f994effd700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f993c06a290 0x7f993c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:44.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.124+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99500734a0 con 0x7f99500d02c0 2023-08-27T11:24:44.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.125+0000 7f99377fe700 1 -- 172.21.15.45:0/2645805052 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9944056e00 con 0x7f99500d02c0 2023-08-27T11:24:44.127 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:24:44.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f993c06a290 msgr2=0x7f993c06c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:44.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f993c06a290 0x7f993c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500d02c0 msgr2=0x7f9950154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500d02c0 0x7f9950154ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f994400e840 tx=0x7f994400eb50 comp rx=0 tx=0).stop 2023-08-27T11:24:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 shutdown_connections 2023-08-27T11:24:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f993c06a290 0x7f993c06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f995015d3a0 0x7f99500b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f99500b0320 0x7f99500b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 --2- 172.21.15.45:0/2645805052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99500d02c0 0x7f9950154ff0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 >> 172.21.15.45:0/2645805052 conn(0x7f995009a1a0 msgr2=0x7f9950156470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 shutdown_connections 2023-08-27T11:24:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.127+0000 7f9955702700 1 -- 172.21.15.45:0/2645805052 wait complete. 2023-08-27T11:24:44.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:24:44.182 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:44.181+0000 7f086130e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:24:44.192 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:44.191+0000 7fcc44fe2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:24:44.263 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:44.262+0000 7f086130e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:24:44.273 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:44.272+0000 7fcc44fe2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.275+0000 7f3b04635700 1 Processor -- start 2023-08-27T11:24:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.275+0000 7f3b04635700 1 -- start start 2023-08-27T11:24:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b04635700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3afc0cff50 0x7f3afc0b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b04635700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3afc0cb490 con 0x7f3afc0cea10 2023-08-27T11:24:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b04635700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3afc0d0c40 con 0x7f3afc15b2f0 2023-08-27T11:24:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b04635700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3afc0cdaa0 con 0x7f3afc0cff50 2023-08-27T11:24:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b02bd2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3afc0cff50 0x7f3afc0b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b023d1700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3afc0cea10 0x7f3afc0cedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42270/0 (socket says 172.21.15.45:42270) 2023-08-27T11:24:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b023d1700 1 -- 172.21.15.45:0/3871798175 learned_addr learned my addr 172.21.15.45:0/3871798175 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b02bd2700 1 -- 172.21.15.45:0/3871798175 >> v1:172.21.15.135:6789/0 conn(0x7f3afc15b2f0 legacy=0x7f3afc0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b02bd2700 1 -- 172.21.15.45:0/3871798175 >> v1:172.21.15.45:6789/0 conn(0x7f3afc0cea10 legacy=0x7f3afc0cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b013cf700 1 -- 172.21.15.45:0/3871798175 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432534980 0 0) 0x7f3afc0d0c40 con 0x7f3afc15b2f0 2023-08-27T11:24:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b02bd2700 1 -- 172.21.15.45:0/3871798175 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc06ab80 con 0x7f3afc0cff50 2023-08-27T11:24:44.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.276+0000 7f3b02bd2700 1 --2- 172.21.15.45:0/3871798175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 0x7f3afc0b43a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009d90 tx=0x7f3af400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863264fa9440bbc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b013cf700 1 -- 172.21.15.45:0/3871798175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af400e070 con 0x7f3afc0cff50 2023-08-27T11:24:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b013cf700 1 -- 172.21.15.45:0/3871798175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af400bad0 con 0x7f3afc0cff50 2023-08-27T11:24:44.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b013cf700 1 -- 172.21.15.45:0/3871798175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af400be80 con 0x7f3afc0cff50 2023-08-27T11:24:44.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b04635700 1 -- 172.21.15.45:0/3871798175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 msgr2=0x7f3afc0b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b04635700 1 --2- 172.21.15.45:0/3871798175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 0x7f3afc0b43a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009d90 tx=0x7f3af400b330 comp rx=0 tx=0).stop 2023-08-27T11:24:44.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b04635700 1 -- 172.21.15.45:0/3871798175 shutdown_connections 2023-08-27T11:24:44.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b04635700 1 --2- 172.21.15.45:0/3871798175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 0x7f3afc0b43a0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b04635700 1 -- 172.21.15.45:0/3871798175 >> 172.21.15.45:0/3871798175 conn(0x7f3afc0b2990 msgr2=0x7f3afc0b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:44.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b04635700 1 -- 172.21.15.45:0/3871798175 shutdown_connections 2023-08-27T11:24:44.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.277+0000 7f3b04635700 1 -- 172.21.15.45:0/3871798175 wait complete. 2023-08-27T11:24:44.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b04635700 1 Processor -- start 2023-08-27T11:24:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b04635700 1 -- start start 2023-08-27T11:24:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b04635700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3afc0cea10 0x7f3afc0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b04635700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 0x7f3afc1f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b04635700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3afc15b2f0 0x7f3afc1f0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b04635700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1f4d60 con 0x7f3afc0cff50 2023-08-27T11:24:44.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b04635700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc0c8840 con 0x7f3afc0cea10 2023-08-27T11:24:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b04635700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1f0750 con 0x7f3afc15b2f0 2023-08-27T11:24:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b01bd0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 0x7f3afc1f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b01bd0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 0x7f3afc1f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55500/0 (socket says 172.21.15.45:55500) 2023-08-27T11:24:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b01bd0700 1 -- 172.21.15.45:0/2637540874 learned_addr learned my addr 172.21.15.45:0/2637540874 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:44.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.279+0000 7f3b023d1700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3afc0cea10 0x7f3afc0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3b02bd2700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3afc15b2f0 0x7f3afc1f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3b01bd0700 1 -- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3afc15b2f0 msgr2=0x7f3afc1f0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3b01bd0700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3afc15b2f0 0x7f3afc1f0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3b01bd0700 1 -- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3afc0cea10 msgr2=0x7f3afc0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3b01bd0700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3afc0cea10 0x7f3afc0c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3b01bd0700 1 -- 172.21.15.45:0/2637540874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af4009a70 con 0x7f3afc0cff50 2023-08-27T11:24:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3b01bd0700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 0x7f3afc1f4820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00c520 tx=0x7f3aec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3af2ffd700 1 -- 172.21.15.45:0/2637540874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0054e0 con 0x7f3afc0cff50 2023-08-27T11:24:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3af2ffd700 1 -- 172.21.15.45:0/2637540874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec003680 con 0x7f3afc0cff50 2023-08-27T11:24:44.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc0b4020 con 0x7f3afc0cff50 2023-08-27T11:24:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3af2ffd700 1 -- 172.21.15.45:0/2637540874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec003a30 con 0x7f3afc0cff50 2023-08-27T11:24:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.280+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc0c81f0 con 0x7f3afc0cff50 2023-08-27T11:24:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.281+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc06f560 con 0x7f3afc0cff50 2023-08-27T11:24:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.282+0000 7f3af2ffd700 1 -- 172.21.15.45:0/2637540874 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7f3aec003b90 con 0x7f3afc0cff50 2023-08-27T11:24:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.282+0000 7f3af2ffd700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3ae006a290 0x7f3ae006c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.282+0000 7f3b023d1700 1 -- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3ae006a290 msgr2=0x7f3ae006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.282+0000 7f3af2ffd700 1 -- 172.21.15.45:0/2637540874 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7f3aec088150 con 0x7f3afc0cff50 2023-08-27T11:24:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.282+0000 7f3b023d1700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3ae006a290 0x7f3ae006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:44.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.287+0000 7f3af2ffd700 1 -- 172.21.15.45:0/2637540874 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3aec053110 con 0x7f3afc0cff50 2023-08-27T11:24:44.361 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:44.359+0000 7fcc44fe2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:24:44.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.483+0000 7f3b023d1700 1 -- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3ae006a290 msgr2=0x7f3ae006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:44.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.483+0000 7f3b023d1700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3ae006a290 0x7f3ae006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.488+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3afc0c8980 con 0x7f3afc0cff50 2023-08-27T11:24:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.489+0000 7f3af2ffd700 1 -- 172.21.15.45:0/2637540874 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3aec056730 con 0x7f3afc0cff50 2023-08-27T11:24:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:24:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3ae006a290 msgr2=0x7f3ae006c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3ae006a290 0x7f3ae006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 msgr2=0x7f3afc1f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 0x7f3afc1f4820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00c520 tx=0x7f3aec00c830 comp rx=0 tx=0).stop 2023-08-27T11:24:44.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 shutdown_connections 2023-08-27T11:24:44.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f3ae006a290 0x7f3ae006c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3afc15b2f0 0x7f3afc1f0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3afc0cea10 0x7f3afc0c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 --2- 172.21.15.45:0/2637540874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3afc0cff50 0x7f3afc1f4820 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 >> 172.21.15.45:0/2637540874 conn(0x7f3afc0b2990 msgr2=0x7f3afc0b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:44.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 shutdown_connections 2023-08-27T11:24:44.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.491+0000 7f3b04635700 1 -- 172.21.15.45:0/2637540874 wait complete. 2023-08-27T11:24:44.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:24:44.635 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:44.634+0000 7f086130e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:24:44.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.634+0000 7f7e80f9d700 1 Processor -- start 2023-08-27T11:24:44.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.634+0000 7f7e80f9d700 1 -- start start 2023-08-27T11:24:44.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.635+0000 7f7e80f9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e7c15af30 0x7f7e7c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.635+0000 7f7e80f9d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7c0cb8b0 con 0x7f7e7c0cde80 2023-08-27T11:24:44.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.635+0000 7f7e80f9d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7c0cecd0 con 0x7f7e7c0cf960 2023-08-27T11:24:44.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.635+0000 7f7e80f9d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7c06f9c0 con 0x7f7e7c15af30 2023-08-27T11:24:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.635+0000 7f7e7a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e7c15af30 0x7f7e7c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.635+0000 7f7e7ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7e7c0cde80 0x7f7e7c164fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42308/0 (socket says 172.21.15.45:42308) 2023-08-27T11:24:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.635+0000 7f7e7ad9d700 1 -- 172.21.15.45:0/981430558 learned_addr learned my addr 172.21.15.45:0/981430558 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.636+0000 7f7e7a59c700 1 -- 172.21.15.45:0/981430558 >> v1:172.21.15.135:6789/0 conn(0x7f7e7c0cf960 legacy=0x7f7e7c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.636+0000 7f7e7a59c700 1 -- 172.21.15.45:0/981430558 >> v1:172.21.15.45:6789/0 conn(0x7f7e7c0cde80 legacy=0x7f7e7c164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.636+0000 7f7e7a59c700 1 -- 172.21.15.45:0/981430558 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c0cbae0 con 0x7f7e7c15af30 2023-08-27T11:24:44.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.636+0000 7f7e7a59c700 1 --2- 172.21.15.45:0/981430558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c15af30 0x7f7e7c0cf420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009840 tx=0x7f7e6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e278a41532d9e782 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:44.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.636+0000 7f7e7959a700 1 -- 172.21.15.45:0/981430558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c011830 con 0x7f7e7c15af30 2023-08-27T11:24:44.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.636+0000 7f7e7959a700 1 -- 172.21.15.45:0/981430558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c011990 con 0x7f7e7c15af30 2023-08-27T11:24:44.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.636+0000 7f7e7959a700 1 -- 172.21.15.45:0/981430558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c011d20 con 0x7f7e7c15af30 2023-08-27T11:24:44.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.637+0000 7f7e80f9d700 1 -- 172.21.15.45:0/981430558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c15af30 msgr2=0x7f7e7c0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.637+0000 7f7e80f9d700 1 --2- 172.21.15.45:0/981430558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c15af30 0x7f7e7c0cf420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009840 tx=0x7f7e6c009b50 comp rx=0 tx=0).stop 2023-08-27T11:24:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.637+0000 7f7e80f9d700 1 -- 172.21.15.45:0/981430558 shutdown_connections 2023-08-27T11:24:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.637+0000 7f7e80f9d700 1 --2- 172.21.15.45:0/981430558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c15af30 0x7f7e7c0cf420 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.637+0000 7f7e80f9d700 1 -- 172.21.15.45:0/981430558 >> 172.21.15.45:0/981430558 conn(0x7f7e7c09a1a0 msgr2=0x7f7e7c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.637+0000 7f7e80f9d700 1 -- 172.21.15.45:0/981430558 shutdown_connections 2023-08-27T11:24:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.637+0000 7f7e80f9d700 1 -- 172.21.15.45:0/981430558 wait complete. 2023-08-27T11:24:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.638+0000 7f7e80f9d700 1 Processor -- start 2023-08-27T11:24:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.638+0000 7f7e80f9d700 1 -- start start 2023-08-27T11:24:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e80f9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c0cde80 0x7f7e7c06b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e80f9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e7c0cf960 0x7f7e7c0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e80f9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e7c15af30 0x7f7e7c0c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e80f9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c06c330 con 0x7f7e7c0cde80 2023-08-27T11:24:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e80f9d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0b6240 con 0x7f7e7c15af30 2023-08-27T11:24:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e80f9d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0c8cd0 con 0x7f7e7c0cf960 2023-08-27T11:24:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e7a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c0cde80 0x7f7e7c06b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e7a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c0cde80 0x7f7e7c06b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55534/0 (socket says 172.21.15.45:55534) 2023-08-27T11:24:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e7a59c700 1 -- 172.21.15.45:0/358140270 learned_addr learned my addr 172.21.15.45:0/358140270 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e7ad9d700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e7c15af30 0x7f7e7c0c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.639+0000 7f7e79d9b700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e7c0cf960 0x7f7e7c0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e7a59c700 1 -- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e7c0cf960 msgr2=0x7f7e7c0b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e7a59c700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e7c0cf960 0x7f7e7c0b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e7a59c700 1 -- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e7c15af30 msgr2=0x7f7e7c0c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e7a59c700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e7c15af30 0x7f7e7c0c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e7a59c700 1 -- 172.21.15.45:0/358140270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0094a0 con 0x7f7e7c0cde80 2023-08-27T11:24:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e7a59c700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c0cde80 0x7f7e7c06b390 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0090b0 tx=0x7f7e6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e6affd700 1 -- 172.21.15.45:0/358140270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0036a0 con 0x7f7e7c0cde80 2023-08-27T11:24:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e6affd700 1 -- 172.21.15.45:0/358140270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c01e070 con 0x7f7e7c0cde80 2023-08-27T11:24:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c06ba90 con 0x7f7e7c0cde80 2023-08-27T11:24:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e6affd700 1 -- 172.21.15.45:0/358140270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c003a60 con 0x7f7e7c0cde80 2023-08-27T11:24:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.640+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c06c050 con 0x7f7e7c0cde80 2023-08-27T11:24:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.641+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c0b5410 con 0x7f7e7c0cde80 2023-08-27T11:24:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.642+0000 7f7e6affd700 1 -- 172.21.15.45:0/358140270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7f7e6c003bc0 con 0x7f7e7c0cde80 2023-08-27T11:24:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.642+0000 7f7e6affd700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f7e5c06a290 0x7f7e5c06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.642+0000 7f7e6affd700 1 -- 172.21.15.45:0/358140270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7f7e6c089d70 con 0x7f7e7c0cde80 2023-08-27T11:24:44.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.642+0000 7f7e79d9b700 1 -- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f7e5c06a290 msgr2=0x7f7e5c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:44.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.642+0000 7f7e79d9b700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f7e5c06a290 0x7f7e5c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:44.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.647+0000 7f7e6affd700 1 -- 172.21.15.45:0/358140270 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7e6c054d30 con 0x7f7e7c0cde80 2023-08-27T11:24:44.759 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:44.757+0000 7fcc44fe2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:24:44.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.843+0000 7f7e79d9b700 1 -- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f7e5c06a290 msgr2=0x7f7e5c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:44.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.843+0000 7f7e79d9b700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f7e5c06a290 0x7f7e5c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.849+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e7c15f430 con 0x7f7e7c0cde80 2023-08-27T11:24:44.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.849+0000 7f7e6affd700 1 -- 172.21.15.45:0/358140270 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e6c020020 con 0x7f7e7c0cde80 2023-08-27T11:24:44.851 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:24:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f7e5c06a290 msgr2=0x7f7e5c06c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f7e5c06a290 0x7f7e5c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c0cde80 msgr2=0x7f7e7c06b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c0cde80 0x7f7e7c06b390 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0090b0 tx=0x7f7e6c004750 comp rx=0 tx=0).stop 2023-08-27T11:24:44.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 shutdown_connections 2023-08-27T11:24:44.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f7e5c06a290 0x7f7e5c06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e7c0cde80 0x7f7e7c06b390 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e7c15af30 0x7f7e7c0c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 --2- 172.21.15.45:0/358140270 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7e7c0cf960 0x7f7e7c0b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:44.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 >> 172.21.15.45:0/358140270 conn(0x7f7e7c09a1a0 msgr2=0x7f7e7c0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:44.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 shutdown_connections 2023-08-27T11:24:44.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.851+0000 7f7e80f9d700 1 -- 172.21.15.45:0/358140270 wait complete. 2023-08-27T11:24:44.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:24:44.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.992+0000 7f5eded2c700 1 Processor -- start 2023-08-27T11:24:44.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.992+0000 7f5eded2c700 1 -- start start 2023-08-27T11:24:44.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.993+0000 7f5eded2c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed8159a70 0x7f5ed806aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:44.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.993+0000 7f5eded2c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed80cceb0 con 0x7f5ed80b1620 2023-08-27T11:24:44.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.993+0000 7f5eded2c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed806b3f0 con 0x7f5ed806afb0 2023-08-27T11:24:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.993+0000 7f5eded2c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80cc970 con 0x7f5ed8159a70 2023-08-27T11:24:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.993+0000 7f5edd2c9700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5ed80b1620 0x7f5ed8163980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42324/0 (socket says 172.21.15.45:42324) 2023-08-27T11:24:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.993+0000 7f5edd2c9700 1 -- 172.21.15.45:0/2569085462 learned_addr learned my addr 172.21.15.45:0/2569085462 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.993+0000 7f5edcac8700 1 --2- 172.21.15.45:0/2569085462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed8159a70 0x7f5ed806aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5ed77fe700 1 -- 172.21.15.45:0/2569085462 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810281153 0 0) 0x7f5ed80cceb0 con 0x7f5ed80b1620 2023-08-27T11:24:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5ed77fe700 1 -- 172.21.15.45:0/2569085462 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ec80035b0 con 0x7f5ed80b1620 2023-08-27T11:24:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5ed77fe700 1 -- 172.21.15.45:0/2569085462 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109197049 0 0) 0x7f5ed806b3f0 con 0x7f5ed806afb0 2023-08-27T11:24:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5ed77fe700 1 -- 172.21.15.45:0/2569085462 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed80cceb0 con 0x7f5ed806afb0 2023-08-27T11:24:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5edcac8700 1 -- 172.21.15.45:0/2569085462 >> v1:172.21.15.135:6789/0 conn(0x7f5ed806afb0 legacy=0x7f5ed80b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5edcac8700 1 -- 172.21.15.45:0/2569085462 >> v1:172.21.15.45:6789/0 conn(0x7f5ed80b1620 legacy=0x7f5ed8163980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5edcac8700 1 -- 172.21.15.45:0/2569085462 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed80ccc10 con 0x7f5ed8159a70 2023-08-27T11:24:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5edcac8700 1 --2- 172.21.15.45:0/2569085462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed8159a70 0x7f5ed806aa70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009870 tx=0x7f5ecc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b1433b58a749de server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5ed77fe700 1 -- 172.21.15.45:0/2569085462 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc011830 con 0x7f5ed8159a70 2023-08-27T11:24:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5ed77fe700 1 -- 172.21.15.45:0/2569085462 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ecc011990 con 0x7f5ed8159a70 2023-08-27T11:24:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.994+0000 7f5ed77fe700 1 -- 172.21.15.45:0/2569085462 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc011d00 con 0x7f5ed8159a70 2023-08-27T11:24:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.995+0000 7f5eded2c700 1 -- 172.21.15.45:0/2569085462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed8159a70 msgr2=0x7f5ed806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.995+0000 7f5eded2c700 1 --2- 172.21.15.45:0/2569085462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed8159a70 0x7f5ed806aa70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009870 tx=0x7f5ecc009b80 comp rx=0 tx=0).stop 2023-08-27T11:24:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.995+0000 7f5eded2c700 1 -- 172.21.15.45:0/2569085462 shutdown_connections 2023-08-27T11:24:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.995+0000 7f5eded2c700 1 --2- 172.21.15.45:0/2569085462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed8159a70 0x7f5ed806aa70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.995+0000 7f5eded2c700 1 -- 172.21.15.45:0/2569085462 >> 172.21.15.45:0/2569085462 conn(0x7f5ed80b23e0 msgr2=0x7f5ed80b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.995+0000 7f5eded2c700 1 -- 172.21.15.45:0/2569085462 shutdown_connections 2023-08-27T11:24:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.995+0000 7f5eded2c700 1 -- 172.21.15.45:0/2569085462 wait complete. 2023-08-27T11:24:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.996+0000 7f5eded2c700 1 Processor -- start 2023-08-27T11:24:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.996+0000 7f5eded2c700 1 -- start start 2023-08-27T11:24:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5eded2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed806afb0 0x7f5ed806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5eded2c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed80b1620 0x7f5ed81f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5eded2c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed8159a70 0x7f5ed80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5eded2c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed806db20 con 0x7f5ed806afb0 2023-08-27T11:24:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5eded2c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed8158c60 con 0x7f5ed8159a70 2023-08-27T11:24:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5eded2c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed81f1ee0 con 0x7f5ed80b1620 2023-08-27T11:24:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5edcac8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed806afb0 0x7f5ed806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5edcac8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed806afb0 0x7f5ed806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55550/0 (socket says 172.21.15.45:55550) 2023-08-27T11:24:45.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5edcac8700 1 -- 172.21.15.45:0/2038879501 learned_addr learned my addr 172.21.15.45:0/2038879501 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:45.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5edd2c9700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed8159a70 0x7f5ed80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5ed7fff700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed80b1620 0x7f5ed81f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5edcac8700 1 -- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed80b1620 msgr2=0x7f5ed81f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5edcac8700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed80b1620 0x7f5ed81f19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5edcac8700 1 -- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed8159a70 msgr2=0x7f5ed80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5edcac8700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed8159a70 0x7f5ed80c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5edcac8700 1 -- 172.21.15.45:0/2038879501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc0094a0 con 0x7f5ed806afb0 2023-08-27T11:24:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.997+0000 7f5ed7fff700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed80b1620 0x7f5ed81f19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.998+0000 7f5edcac8700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed806afb0 0x7f5ed806d5e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0090b0 tx=0x7f5ecc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.998+0000 7f5ed57fa700 1 -- 172.21.15.45:0/2038879501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc004450 con 0x7f5ed806afb0 2023-08-27T11:24:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.998+0000 7f5ed57fa700 1 -- 172.21.15.45:0/2038879501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ecc0045d0 con 0x7f5ed806afb0 2023-08-27T11:24:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.998+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed80c3da0 con 0x7f5ed806afb0 2023-08-27T11:24:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.998+0000 7f5ed57fa700 1 -- 172.21.15.45:0/2038879501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc0038e0 con 0x7f5ed806afb0 2023-08-27T11:24:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.998+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed80c4860 con 0x7f5ed806afb0 2023-08-27T11:24:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:44.999+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed80c46c0 con 0x7f5ed806afb0 2023-08-27T11:24:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.000+0000 7f5ed57fa700 1 -- 172.21.15.45:0/2038879501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7f5ecc003a40 con 0x7f5ed806afb0 2023-08-27T11:24:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.000+0000 7f5ed57fa700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ebc06a290 0x7f5ebc06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.000+0000 7f5ed57fa700 1 -- 172.21.15.45:0/2038879501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7f5ecc089fe0 con 0x7f5ed806afb0 2023-08-27T11:24:45.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.000+0000 7f5ed7fff700 1 -- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ebc06a290 msgr2=0x7f5ebc06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:45.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.000+0000 7f5ed7fff700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ebc06a290 0x7f5ebc06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:45.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.005+0000 7f5ed57fa700 1 -- 172.21.15.45:0/2038879501 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5ed80c46c0 con 0x7f5ed806afb0 2023-08-27T11:24:45.204 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:45.203+0000 7f086130e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:24:45.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.204+0000 7f5ed7fff700 1 -- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ebc06a290 msgr2=0x7f5ebc06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:45.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.204+0000 7f5ed7fff700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ebc06a290 0x7f5ebc06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.252+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ed80c46c0 con 0x7f5ed806afb0 2023-08-27T11:24:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.253+0000 7f5ed57fa700 1 -- 172.21.15.45:0/2038879501 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ed80c46c0 con 0x7f5ed806afb0 2023-08-27T11:24:45.256 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:24:45.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ebc06a290 msgr2=0x7f5ebc06c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:45.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ebc06a290 0x7f5ebc06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed806afb0 msgr2=0x7f5ed806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed806afb0 0x7f5ed806d5e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0090b0 tx=0x7f5ecc004750 comp rx=0 tx=0).stop 2023-08-27T11:24:45.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 shutdown_connections 2023-08-27T11:24:45.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7f5ebc06a290 0x7f5ebc06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ed806afb0 0x7f5ed806d5e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ed8159a70 0x7f5ed80c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 --2- 172.21.15.45:0/2038879501 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ed80b1620 0x7f5ed81f19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 >> 172.21.15.45:0/2038879501 conn(0x7f5ed80b23e0 msgr2=0x7f5ed80cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:45.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 shutdown_connections 2023-08-27T11:24:45.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.255+0000 7f5eded2c700 1 -- 172.21.15.45:0/2038879501 wait complete. 2023-08-27T11:24:45.270 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:24:45.274 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:45.273+0000 7f086130e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:24:45.378 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:45.376+0000 7fcc44fe2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:24:45.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.399+0000 7ffbacf21700 1 Processor -- start 2023-08-27T11:24:45.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.399+0000 7ffbacf21700 1 -- start start 2023-08-27T11:24:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.400+0000 7ffbacf21700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffba80cacd0 0x7ffba80ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.400+0000 7ffbacf21700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffba80aefb0 0x7ffba80cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.400+0000 7ffbacf21700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba806b010 con 0x7ffba815d4f0 2023-08-27T11:24:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.400+0000 7ffbacf21700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba815d8d0 con 0x7ffba80aefb0 2023-08-27T11:24:45.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.400+0000 7ffbacf21700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba80ad9f0 con 0x7ffba80cacd0 2023-08-27T11:24:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.400+0000 7ffba6d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffba80aefb0 0x7ffba80cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.400+0000 7ffba659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffba815d4f0 0x7ffba80ca5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42336/0 (socket says 172.21.15.45:42336) 2023-08-27T11:24:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.400+0000 7ffba659c700 1 -- 172.21.15.45:0/929992091 learned_addr learned my addr 172.21.15.45:0/929992091 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba5d9b700 1 --2- 172.21.15.45:0/929992091 >> v2:172.21.15.135:3300/0 conn(0x7ffba80cacd0 0x7ffba80ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba6d9d700 1 -- 172.21.15.45:0/929992091 >> v2:172.21.15.135:3300/0 conn(0x7ffba80cacd0 msgr2=0x7ffba80ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba6d9d700 1 --2- 172.21.15.45:0/929992091 >> v2:172.21.15.135:3300/0 conn(0x7ffba80cacd0 0x7ffba80ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba6d9d700 1 -- 172.21.15.45:0/929992091 >> v1:172.21.15.45:6789/0 conn(0x7ffba815d4f0 legacy=0x7ffba80ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba6d9d700 1 -- 172.21.15.45:0/929992091 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba806ad30 con 0x7ffba80aefb0 2023-08-27T11:24:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba559a700 1 -- 172.21.15.45:0/929992091 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138344134 0 0) 0x7ffba806b010 con 0x7ffba815d4f0 2023-08-27T11:24:45.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba6d9d700 1 --2- 172.21.15.45:0/929992091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80aefb0 0x7ffba80cd930 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ffba806ccb0 tx=0x7ffba000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d252f9b841c90110 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:45.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba559a700 1 -- 172.21.15.45:0/929992091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000e070 con 0x7ffba80aefb0 2023-08-27T11:24:45.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba559a700 1 -- 172.21.15.45:0/929992091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000bb20 con 0x7ffba80aefb0 2023-08-27T11:24:45.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.401+0000 7ffba559a700 1 -- 172.21.15.45:0/929992091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0012430 con 0x7ffba80aefb0 2023-08-27T11:24:45.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.402+0000 7ffbacf21700 1 -- 172.21.15.45:0/929992091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80aefb0 msgr2=0x7ffba80cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.402+0000 7ffbacf21700 1 --2- 172.21.15.45:0/929992091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80aefb0 0x7ffba80cd930 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ffba806ccb0 tx=0x7ffba000b330 comp rx=0 tx=0).stop 2023-08-27T11:24:45.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.402+0000 7ffbacf21700 1 -- 172.21.15.45:0/929992091 shutdown_connections 2023-08-27T11:24:45.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.402+0000 7ffbacf21700 1 --2- 172.21.15.45:0/929992091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80aefb0 0x7ffba80cd930 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.402+0000 7ffbacf21700 1 --2- 172.21.15.45:0/929992091 >> v2:172.21.15.135:3300/0 conn(0x7ffba80cacd0 0x7ffba80ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.402+0000 7ffbacf21700 1 -- 172.21.15.45:0/929992091 >> 172.21.15.45:0/929992091 conn(0x7ffba80cc7e0 msgr2=0x7ffba80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:45.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.402+0000 7ffbacf21700 1 -- 172.21.15.45:0/929992091 shutdown_connections 2023-08-27T11:24:45.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.402+0000 7ffbacf21700 1 -- 172.21.15.45:0/929992091 wait complete. 2023-08-27T11:24:45.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.403+0000 7ffbacf21700 1 Processor -- start 2023-08-27T11:24:45.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.403+0000 7ffbacf21700 1 -- start start 2023-08-27T11:24:45.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffbacf21700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffba80aefb0 0x7ffba80c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffbacf21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80cacd0 0x7ffba80c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffbacf21700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffba815d4f0 0x7ffba80c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffbacf21700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80b2be0 con 0x7ffba80cacd0 2023-08-27T11:24:45.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffbacf21700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80b3480 con 0x7ffba80aefb0 2023-08-27T11:24:45.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffbacf21700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80c8e40 con 0x7ffba815d4f0 2023-08-27T11:24:45.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffba5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80cacd0 0x7ffba80c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffba5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80cacd0 0x7ffba80c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55568/0 (socket says 172.21.15.45:55568) 2023-08-27T11:24:45.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffba5d9b700 1 -- 172.21.15.45:0/3776220634 learned_addr learned my addr 172.21.15.45:0/3776220634 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffba659c700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffba80aefb0 0x7ffba80c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.404+0000 7ffba6d9d700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffba815d4f0 0x7ffba80c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffba5d9b700 1 -- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffba815d4f0 msgr2=0x7ffba80c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffba5d9b700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffba815d4f0 0x7ffba80c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffba5d9b700 1 -- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffba80aefb0 msgr2=0x7ffba80c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffba5d9b700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffba80aefb0 0x7ffba80c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffba5d9b700 1 -- 172.21.15.45:0/3776220634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba0009a70 con 0x7ffba80cacd0 2023-08-27T11:24:45.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffba5d9b700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80cacd0 0x7ffba80c73c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800e9e0 tx=0x7ffb9800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:45.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffb9effd700 1 -- 172.21.15.45:0/3776220634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98009710 con 0x7ffba80cacd0 2023-08-27T11:24:45.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffb9effd700 1 -- 172.21.15.45:0/3776220634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9800ee80 con 0x7ffba80cacd0 2023-08-27T11:24:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba80c6a40 con 0x7ffba80cacd0 2023-08-27T11:24:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffb9effd700 1 -- 172.21.15.45:0/3776220634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800ca50 con 0x7ffba80cacd0 2023-08-27T11:24:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.405+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba80b3110 con 0x7ffba80cacd0 2023-08-27T11:24:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.406+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba80726d0 con 0x7ffba80cacd0 2023-08-27T11:24:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.407+0000 7ffb9effd700 1 -- 172.21.15.45:0/3776220634 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7ffb9800cbb0 con 0x7ffba80cacd0 2023-08-27T11:24:45.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.407+0000 7ffb9effd700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffb8806a290 0x7ffb8806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.407+0000 7ffb9effd700 1 -- 172.21.15.45:0/3776220634 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7ffb98089000 con 0x7ffba80cacd0 2023-08-27T11:24:45.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.407+0000 7ffba659c700 1 -- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffb8806a290 msgr2=0x7ffb8806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:45.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.407+0000 7ffba659c700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffb8806a290 0x7ffb8806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:45.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.411+0000 7ffb9effd700 1 -- 172.21.15.45:0/3776220634 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffb98053fc0 con 0x7ffba80cacd0 2023-08-27T11:24:45.455 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:45.453+0000 7fcc44fe2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:24:45.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.608+0000 7ffba659c700 1 -- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffb8806a290 msgr2=0x7ffb8806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:45.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.608+0000 7ffba659c700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffb8806a290 0x7ffb8806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:45.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.628+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffba80734a0 con 0x7ffba80cacd0 2023-08-27T11:24:45.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.628+0000 7ffb9effd700 1 -- 172.21.15.45:0/3776220634 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v56) v1 ==== 80+0+360 (secure 0 0 0) 0x7ffb980575e0 con 0x7ffba80cacd0 2023-08-27T11:24:45.631 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/2587284130 2023-08-28T11:13:52.047816+0000 2023-08-27T11:24:45.631 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/1110669507 2023-08-28T11:13:52.047816+0000 2023-08-27T11:24:45.631 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6833/4120550306 2023-08-28T11:13:52.047816+0000 2023-08-27T11:24:45.631 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6832/4120550306 2023-08-28T11:13:52.047816+0000 2023-08-27T11:24:45.631 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/1692978501 2023-08-28T11:13:52.047816+0000 2023-08-27T11:24:45.632 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/2898860171 2023-08-28T11:13:52.047816+0000 2023-08-27T11:24:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffb8806a290 msgr2=0x7ffb8806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffb8806a290 0x7ffb8806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80cacd0 msgr2=0x7ffba80c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80cacd0 0x7ffba80c73c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800e9e0 tx=0x7ffb9800ecf0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 shutdown_connections 2023-08-27T11:24:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ffb8806a290 0x7ffb8806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffba815d4f0 0x7ffba80c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffba80aefb0 0x7ffba80c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 --2- 172.21.15.45:0/3776220634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffba80cacd0 0x7ffba80c73c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 >> 172.21.15.45:0/3776220634 conn(0x7ffba80cc7e0 msgr2=0x7ffba806ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 shutdown_connections 2023-08-27T11:24:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.630+0000 7ffbacf21700 1 -- 172.21.15.45:0/3776220634 wait complete. 2023-08-27T11:24:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:listed 6 entries 2023-08-27T11:24:45.645 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:24:45.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:24:45.689 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:24:45.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:24:45.690 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:24:45.746 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:24:45.746 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.764 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:24:45.765 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:24:45.765 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.803 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:24:45.804 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:24:45.804 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:45.821 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:24:45.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:24:45.907 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:45.906+0000 7f086130e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:24:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.970+0000 7fcf74c2b700 1 Processor -- start 2023-08-27T11:24:45.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.970+0000 7fcf74c2b700 1 -- start start 2023-08-27T11:24:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.970+0000 7fcf74c2b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcf700cf980 0x7fcf700d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf74c2b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf700ceae0 con 0x7fcf700cfd60 2023-08-27T11:24:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf74c2b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf700cedc0 con 0x7fcf700b45d0 2023-08-27T11:24:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf74c2b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf700cf0a0 con 0x7fcf700cf980 2023-08-27T11:24:45.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcf700cf980 0x7fcf700d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6ed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcf700cfd60 0x7fcf700b2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42356/0 (socket says 172.21.15.45:42356) 2023-08-27T11:24:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcf700cf980 0x7fcf700d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55576/0 (socket says 172.21.15.45:55576) 2023-08-27T11:24:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6ed9d700 1 -- 172.21.15.45:0/434196077 learned_addr learned my addr 172.21.15.45:0/434196077 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6e59c700 1 -- 172.21.15.45:0/434196077 >> v1:172.21.15.135:6789/0 conn(0x7fcf700b45d0 legacy=0x7fcf700d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6e59c700 1 -- 172.21.15.45:0/434196077 >> v1:172.21.15.45:6789/0 conn(0x7fcf700cfd60 legacy=0x7fcf700b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6e59c700 1 -- 172.21.15.45:0/434196077 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf700b22c0 con 0x7fcf700cf980 2023-08-27T11:24:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6d59a700 1 -- 172.21.15.45:0/434196077 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539048357 0 0) 0x7fcf700ceae0 con 0x7fcf700cfd60 2023-08-27T11:24:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6e59c700 1 --2- 172.21.15.45:0/434196077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 0x7fcf700d0f20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009840 tx=0x7fcf60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ddfe77f96556d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6d59a700 1 -- 172.21.15.45:0/434196077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60011830 con 0x7fcf700cf980 2023-08-27T11:24:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6d59a700 1 -- 172.21.15.45:0/434196077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60011990 con 0x7fcf700cf980 2023-08-27T11:24:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.971+0000 7fcf6d59a700 1 -- 172.21.15.45:0/434196077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60011d20 con 0x7fcf700cf980 2023-08-27T11:24:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.972+0000 7fcf74c2b700 1 -- 172.21.15.45:0/434196077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 msgr2=0x7fcf700d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.972+0000 7fcf74c2b700 1 --2- 172.21.15.45:0/434196077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 0x7fcf700d0f20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009840 tx=0x7fcf60009b50 comp rx=0 tx=0).stop 2023-08-27T11:24:45.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.972+0000 7fcf74c2b700 1 -- 172.21.15.45:0/434196077 shutdown_connections 2023-08-27T11:24:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.972+0000 7fcf74c2b700 1 --2- 172.21.15.45:0/434196077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 0x7fcf700d0f20 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.972+0000 7fcf74c2b700 1 -- 172.21.15.45:0/434196077 >> 172.21.15.45:0/434196077 conn(0x7fcf700d3020 msgr2=0x7fcf700d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.972+0000 7fcf74c2b700 1 -- 172.21.15.45:0/434196077 shutdown_connections 2023-08-27T11:24:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.972+0000 7fcf74c2b700 1 -- 172.21.15.45:0/434196077 wait complete. 2023-08-27T11:24:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.974+0000 7fcf74c2b700 1 Processor -- start 2023-08-27T11:24:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.974+0000 7fcf74c2b700 1 -- start start 2023-08-27T11:24:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.974+0000 7fcf74c2b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf700b45d0 0x7fcf7014b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.974+0000 7fcf74c2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 0x7fcf700b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.974+0000 7fcf74c2b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf700cfd60 0x7fcf700c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.974+0000 7fcf74c2b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700b74d0 con 0x7fcf700cf980 2023-08-27T11:24:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.974+0000 7fcf74c2b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7014bd00 con 0x7fcf700b45d0 2023-08-27T11:24:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.974+0000 7fcf74c2b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7014bfe0 con 0x7fcf700cfd60 2023-08-27T11:24:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 0x7fcf700b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6e59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf700b45d0 0x7fcf7014b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 0x7fcf700b6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55584/0 (socket says 172.21.15.45:55584) 2023-08-27T11:24:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6dd9b700 1 -- 172.21.15.45:0/2461432478 learned_addr learned my addr 172.21.15.45:0/2461432478 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6ed9d700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf700cfd60 0x7fcf700c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:45.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6dd9b700 1 -- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf700cfd60 msgr2=0x7fcf700c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6dd9b700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf700cfd60 0x7fcf700c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6dd9b700 1 -- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf700b45d0 msgr2=0x7fcf7014b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6dd9b700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf700b45d0 0x7fcf7014b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6dd9b700 1 -- 172.21.15.45:0/2461432478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf600094a0 con 0x7fcf700cf980 2023-08-27T11:24:45.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6e59c700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf700b45d0 0x7fcf7014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf6dd9b700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 0x7fcf700b6ae0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c4b0 tx=0x7fcf6400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf5effd700 1 -- 172.21.15.45:0/2461432478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640052d0 con 0x7fcf700cf980 2023-08-27T11:24:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf5effd700 1 -- 172.21.15.45:0/2461432478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf64005430 con 0x7fcf700cf980 2023-08-27T11:24:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7014c2c0 con 0x7fcf700cf980 2023-08-27T11:24:45.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf5effd700 1 -- 172.21.15.45:0/2461432478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640038c0 con 0x7fcf700cf980 2023-08-27T11:24:45.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.975+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf700b5cf0 con 0x7fcf700cf980 2023-08-27T11:24:45.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.976+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf700a1030 con 0x7fcf700cf980 2023-08-27T11:24:45.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.977+0000 7fcf5effd700 1 -- 172.21.15.45:0/2461432478 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fcf64003a20 con 0x7fcf700cf980 2023-08-27T11:24:45.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.978+0000 7fcf5effd700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcf5006a290 0x7fcf5006c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:45.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.978+0000 7fcf5effd700 1 -- 172.21.15.45:0/2461432478 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7fcf640883f0 con 0x7fcf700cf980 2023-08-27T11:24:45.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.978+0000 7fcf6e59c700 1 -- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcf5006a290 msgr2=0x7fcf5006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:45.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.978+0000 7fcf6e59c700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcf5006a290 0x7fcf5006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:45.992 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:45.984+0000 7f086130e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:24:45.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:45.982+0000 7fcf5effd700 1 -- 172.21.15.45:0/2461432478 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcf64053490 con 0x7fcf700cf980 2023-08-27T11:24:46.132 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:46.130+0000 7f086130e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:24:46.138 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:46.136+0000 7fcc44fe2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:24:46.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.179+0000 7fcf6e59c700 1 -- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcf5006a290 msgr2=0x7fcf5006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:46.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.179+0000 7fcf6e59c700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcf5006a290 0x7fcf5006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:46.221 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:46.220+0000 7fcc44fe2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:24:46.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.232+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf700734a0 con 0x7fcf700cf980 2023-08-27T11:24:46.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.233+0000 7fcf5effd700 1 -- 172.21.15.45:0/2461432478 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4881 (secure 0 0 0) 0x7fcf64017020 con 0x7fcf700cf980 2023-08-27T11:24:46.235 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:46.235 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1324661970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1324661970},{"type":"v1","addr":"172.21.15.135:6836","nonce":1324661970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2014390806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2014390806},{"type":"v1","addr":"172.21.15.135:6837","nonce":2014390806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2103056681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2103056681},{"type":"v1","addr":"172.21.15.45:6834","nonce":2103056681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:13:37.320852+0000","modified":"2023-08-27T11:13:40.917661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/2587151358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2587151358},{"type":"v1","addr":"172.21.15.45:6835","nonce":2587151358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:24:46.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcf5006a290 msgr2=0x7fcf5006c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:46.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcf5006a290 0x7fcf5006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 msgr2=0x7fcf700b6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 0x7fcf700b6ae0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400c4b0 tx=0x7fcf6400c7c0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 shutdown_connections 2023-08-27T11:24:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fcf5006a290 0x7fcf5006c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcf700cfd60 0x7fcf700c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf700b45d0 0x7fcf7014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 --2- 172.21.15.45:0/2461432478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf700cf980 0x7fcf700b6ae0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 >> 172.21.15.45:0/2461432478 conn(0x7fcf700d3020 msgr2=0x7fcf7006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 shutdown_connections 2023-08-27T11:24:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.235+0000 7fcf74c2b700 1 -- 172.21.15.45:0/2461432478 wait complete. 2023-08-27T11:24:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2023-08-27T11:24:46.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:24:46.360 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:46.359+0000 7f086130e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:24:46.378 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:46.377+0000 7fcc44fe2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:24:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.377+0000 7fe6571d7700 1 Processor -- start 2023-08-27T11:24:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.377+0000 7fe6571d7700 1 -- start start 2023-08-27T11:24:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.377+0000 7fe6571d7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6500b8c20 0x7fe6500cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe6571d7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65015d6a0 con 0x7fe65015d2c0 2023-08-27T11:24:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe6571d7700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6500ce730 con 0x7fe6500ce2e0 2023-08-27T11:24:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe6571d7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6500d2c90 con 0x7fe6500b8c20 2023-08-27T11:24:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe64ffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6500b8c20 0x7fe6500cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe654f73700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe65015d2c0 0x7fe6500d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42386/0 (socket says 172.21.15.45:42386) 2023-08-27T11:24:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe654f73700 1 -- 172.21.15.45:0/3881460371 learned_addr learned my addr 172.21.15.45:0/3881460371 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe64ffff700 1 -- 172.21.15.45:0/3881460371 >> v1:172.21.15.135:6789/0 conn(0x7fe6500ce2e0 legacy=0x7fe6500b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe64ffff700 1 -- 172.21.15.45:0/3881460371 >> v1:172.21.15.45:6789/0 conn(0x7fe65015d2c0 legacy=0x7fe6500d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe64ffff700 1 -- 172.21.15.45:0/3881460371 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6500d1bf0 con 0x7fe6500b8c20 2023-08-27T11:24:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe64f7fe700 1 -- 172.21.15.45:0/3881460371 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169262109 0 0) 0x7fe644001110 con 0x7fe65015d2c0 2023-08-27T11:24:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe64ffff700 1 --2- 172.21.15.45:0/3881460371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500b8c20 0x7fe6500cdda0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009870 tx=0x7fe63c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414e4f9a73601a19 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe64f7fe700 1 -- 172.21.15.45:0/3881460371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c011830 con 0x7fe6500b8c20 2023-08-27T11:24:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe64f7fe700 1 -- 172.21.15.45:0/3881460371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c011990 con 0x7fe6500b8c20 2023-08-27T11:24:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.378+0000 7fe64f7fe700 1 -- 172.21.15.45:0/3881460371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c011d20 con 0x7fe6500b8c20 2023-08-27T11:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.379+0000 7fe6571d7700 1 -- 172.21.15.45:0/3881460371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500b8c20 msgr2=0x7fe6500cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.379+0000 7fe6571d7700 1 --2- 172.21.15.45:0/3881460371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500b8c20 0x7fe6500cdda0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009870 tx=0x7fe63c009b80 comp rx=0 tx=0).stop 2023-08-27T11:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.379+0000 7fe6571d7700 1 -- 172.21.15.45:0/3881460371 shutdown_connections 2023-08-27T11:24:46.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.379+0000 7fe6571d7700 1 --2- 172.21.15.45:0/3881460371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500b8c20 0x7fe6500cdda0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.379+0000 7fe6571d7700 1 -- 172.21.15.45:0/3881460371 >> 172.21.15.45:0/3881460371 conn(0x7fe6500d2fd0 msgr2=0x7fe6500d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.379+0000 7fe6571d7700 1 -- 172.21.15.45:0/3881460371 shutdown_connections 2023-08-27T11:24:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.379+0000 7fe6571d7700 1 -- 172.21.15.45:0/3881460371 wait complete. 2023-08-27T11:24:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.380+0000 7fe6571d7700 1 Processor -- start 2023-08-27T11:24:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.380+0000 7fe6571d7700 1 -- start start 2023-08-27T11:24:46.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe6571d7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6500b8c20 0x7fe65015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe6571d7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6500ce2e0 0x7fe65015d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe6571d7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe65015d2c0 0x7fe650148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe6571d7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65015e300 con 0x7fe65015d2c0 2023-08-27T11:24:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe6571d7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65015e5e0 con 0x7fe6500b8c20 2023-08-27T11:24:46.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe6571d7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65015e8c0 con 0x7fe6500ce2e0 2023-08-27T11:24:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe655774700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe65015d2c0 0x7fe650148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe655774700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe65015d2c0 0x7fe650148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55620/0 (socket says 172.21.15.45:55620) 2023-08-27T11:24:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe655774700 1 -- 172.21.15.45:0/417227976 learned_addr learned my addr 172.21.15.45:0/417227976 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe654f73700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6500b8c20 0x7fe65015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.381+0000 7fe64ffff700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6500ce2e0 0x7fe65015d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe655774700 1 -- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6500ce2e0 msgr2=0x7fe65015d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe655774700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6500ce2e0 0x7fe65015d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe655774700 1 -- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6500b8c20 msgr2=0x7fe65015eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe655774700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6500b8c20 0x7fe65015eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe655774700 1 -- 172.21.15.45:0/417227976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0094a0 con 0x7fe65015d2c0 2023-08-27T11:24:46.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe655774700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe65015d2c0 0x7fe650148ea0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe64800cc50 tx=0x7fe64800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:46.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe64d7fa700 1 -- 172.21.15.45:0/417227976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe648003c00 con 0x7fe65015d2c0 2023-08-27T11:24:46.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe64d7fa700 1 -- 172.21.15.45:0/417227976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe648003d60 con 0x7fe65015d2c0 2023-08-27T11:24:46.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6500b61a0 con 0x7fe65015d2c0 2023-08-27T11:24:46.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe64d7fa700 1 -- 172.21.15.45:0/417227976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64800aab0 con 0x7fe65015d2c0 2023-08-27T11:24:46.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.382+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe650149c70 con 0x7fe65015d2c0 2023-08-27T11:24:46.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.383+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6500b66c0 con 0x7fe65015d2c0 2023-08-27T11:24:46.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.384+0000 7fe64d7fa700 1 -- 172.21.15.45:0/417227976 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fe64800ac10 con 0x7fe65015d2c0 2023-08-27T11:24:46.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.384+0000 7fe64d7fa700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe63406a290 0x7fe63406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.384+0000 7fe654f73700 1 -- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe63406a290 msgr2=0x7fe63406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:46.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.384+0000 7fe64d7fa700 1 -- 172.21.15.45:0/417227976 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7fe648088b00 con 0x7fe65015d2c0 2023-08-27T11:24:46.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.384+0000 7fe654f73700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe63406a290 0x7fe63406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:46.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.389+0000 7fe64d7fa700 1 -- 172.21.15.45:0/417227976 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe6500b66c0 con 0x7fe65015d2c0 2023-08-27T11:24:46.435 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:46.434+0000 7f086130e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:24:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.585+0000 7fe654f73700 1 -- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe63406a290 msgr2=0x7fe63406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.585+0000 7fe654f73700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe63406a290 0x7fe63406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:46.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.592+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6500ce730 con 0x7fe65015d2c0 2023-08-27T11:24:46.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.594+0000 7fe64d7fa700 1 -- 172.21.15.45:0/417227976 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+19850 (secure 0 0 0) 0x7fe6500ce730 con 0x7fe65015d2c0 2023-08-27T11:24:46.596 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:46.596 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":56,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:24:44.000524+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:13:35.610850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-27T11:13:36.282109+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":218,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/2587284130":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1110669507":"2023-08-28T11:13:52.047816+0000","172.21.15.135:6833/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:6832/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1692978501":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/2898860171":"2023-08-28T11:13:52.047816+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":4,"length":2},{"begin":8,"length":2},{"begin":12,"length":1},{"begin":14,"length":2},{"begin":18,"length":2},{"begin":23,"length":1},{"begin":25,"length":2},{"begin":29,"length":2},{"begin":33,"length":1},{"begin":35,"length":2},{"begin":39,"length":3},{"begin":44,"length":1},{"begin":46,"length":2},{"begin":50,"length":2},{"begin":54,"length":1},{"begin":56,"length":2},{"begin":60,"length":2},{"begin":64,"length":1},{"begin":66,"length":2},{"begin":70,"length":3},{"begin":75,"length":1},{"begin":77,"length":2},{"begin":81,"length":2},{"begin":85,"length":1},{"begin":87,"length":2},{"begin":92,"length":2},{"begin":96,"length":1},{"begin":98,"length":2},{"begin":102,"length":3},{"begin":108,"length":2},{"begin":165,"length":54}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":96,"length":1},{"begin":218,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:24:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe63406a290 msgr2=0x7fe63406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:46.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe63406a290 0x7fe63406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe65015d2c0 msgr2=0x7fe650148ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe65015d2c0 0x7fe650148ea0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe64800cc50 tx=0x7fe64800cf60 comp rx=0 tx=0).stop 2023-08-27T11:24:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 shutdown_connections 2023-08-27T11:24:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe63406a290 0x7fe63406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe65015d2c0 0x7fe650148ea0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6500b8c20 0x7fe65015eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 --2- 172.21.15.45:0/417227976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6500ce2e0 0x7fe65015d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 >> 172.21.15.45:0/417227976 conn(0x7fe6500d2fd0 msgr2=0x7fe650159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 shutdown_connections 2023-08-27T11:24:46.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.596+0000 7fe6571d7700 1 -- 172.21.15.45:0/417227976 wait complete. 2023-08-27T11:24:46.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:13:35.610850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:13:36.282109+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 218, 'snap_epoch': 56, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:24:46.612 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:24:46.624 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:46.622+0000 7fcc44fe2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:24:46.701 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:46.699+0000 7f086130e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:24:46.706 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:46.704+0000 7fcc44fe2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:24:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.739+0000 7fc690f08700 1 Processor -- start 2023-08-27T11:24:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.740+0000 7fc690f08700 1 -- start start 2023-08-27T11:24:46.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.740+0000 7fc690f08700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc68c0ca620 0x7fc68c0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.740+0000 7fc690f08700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0caa00 0x7fc68c0acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.740+0000 7fc690f08700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68c0ce340 con 0x7fc68c0af200 2023-08-27T11:24:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.740+0000 7fc690f08700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68c0ceec0 con 0x7fc68c0ca620 2023-08-27T11:24:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.740+0000 7fc690f08700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c15d7c0 con 0x7fc68c0caa00 2023-08-27T11:24:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc68c0ca620 0x7fc68c0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc68c0ca620 0x7fc68c0cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55626/0 (socket says 172.21.15.45:55626) 2023-08-27T11:24:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68a59c700 1 -- 172.21.15.45:0/793516969 learned_addr learned my addr 172.21.15.45:0/793516969 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68ad9d700 1 --2- 172.21.15.45:0/793516969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0caa00 0x7fc68c0acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68a59c700 1 -- 172.21.15.45:0/793516969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0caa00 msgr2=0x7fc68c0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68a59c700 1 --2- 172.21.15.45:0/793516969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0caa00 0x7fc68c0acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68a59c700 1 -- 172.21.15.45:0/793516969 >> v1:172.21.15.135:6789/0 conn(0x7fc68c0af200 legacy=0x7fc68c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68a59c700 1 -- 172.21.15.45:0/793516969 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c0cc570 con 0x7fc68c0ca620 2023-08-27T11:24:46.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68a59c700 1 --2- 172.21.15.45:0/793516969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0ca620 0x7fc68c0cf350 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0098f0 tx=0x7fc67c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93182bf3e3b498b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:46.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68959a700 1 -- 172.21.15.45:0/793516969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c011830 con 0x7fc68c0ca620 2023-08-27T11:24:46.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68959a700 1 -- 172.21.15.45:0/793516969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67c011990 con 0x7fc68c0ca620 2023-08-27T11:24:46.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.741+0000 7fc68959a700 1 -- 172.21.15.45:0/793516969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c011d20 con 0x7fc68c0ca620 2023-08-27T11:24:46.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.742+0000 7fc690f08700 1 -- 172.21.15.45:0/793516969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0ca620 msgr2=0x7fc68c0cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.742+0000 7fc690f08700 1 --2- 172.21.15.45:0/793516969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0ca620 0x7fc68c0cf350 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0098f0 tx=0x7fc67c009c00 comp rx=0 tx=0).stop 2023-08-27T11:24:46.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.742+0000 7fc690f08700 1 -- 172.21.15.45:0/793516969 shutdown_connections 2023-08-27T11:24:46.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.742+0000 7fc690f08700 1 --2- 172.21.15.45:0/793516969 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0caa00 0x7fc68c0acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.742+0000 7fc690f08700 1 --2- 172.21.15.45:0/793516969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0ca620 0x7fc68c0cf350 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.742+0000 7fc690f08700 1 -- 172.21.15.45:0/793516969 >> 172.21.15.45:0/793516969 conn(0x7fc68c0cc7e0 msgr2=0x7fc68c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:46.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.742+0000 7fc690f08700 1 -- 172.21.15.45:0/793516969 shutdown_connections 2023-08-27T11:24:46.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.742+0000 7fc690f08700 1 -- 172.21.15.45:0/793516969 wait complete. 2023-08-27T11:24:46.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.743+0000 7fc690f08700 1 Processor -- start 2023-08-27T11:24:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.743+0000 7fc690f08700 1 -- start start 2023-08-27T11:24:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc690f08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0af200 0x7fc68c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc690f08700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0ca620 0x7fc68c1f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc690f08700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc68c0caa00 0x7fc68c0b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc690f08700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c159ec0 con 0x7fc68c0af200 2023-08-27T11:24:46.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc690f08700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c15a1a0 con 0x7fc68c0caa00 2023-08-27T11:24:46.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc690f08700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c15a480 con 0x7fc68c0ca620 2023-08-27T11:24:46.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc68a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0af200 0x7fc68c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:46.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc68a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0af200 0x7fc68c159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55630/0 (socket says 172.21.15.45:55630) 2023-08-27T11:24:46.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc68a59c700 1 -- 172.21.15.45:0/2332458337 learned_addr learned my addr 172.21.15.45:0/2332458337 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:46.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc68ad9d700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc68c0caa00 0x7fc68c0b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:46.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.744+0000 7fc689d9b700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0ca620 0x7fc68c1f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:46.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc68a59c700 1 -- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0ca620 msgr2=0x7fc68c1f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc68a59c700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0ca620 0x7fc68c1f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc68a59c700 1 -- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc68c0caa00 msgr2=0x7fc68c0b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc68a59c700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc68c0caa00 0x7fc68c0b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc68a59c700 1 -- 172.21.15.45:0/2332458337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c009550 con 0x7fc68c0af200 2023-08-27T11:24:46.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc68a59c700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0af200 0x7fc68c159820 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc67c000c00 tx=0x7fc67c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:46.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc682ffd700 1 -- 172.21.15.45:0/2332458337 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c004300 con 0x7fc68c0af200 2023-08-27T11:24:46.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc682ffd700 1 -- 172.21.15.45:0/2332458337 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67c01f070 con 0x7fc68c0af200 2023-08-27T11:24:46.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68c15a760 con 0x7fc68c0af200 2023-08-27T11:24:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc682ffd700 1 -- 172.21.15.45:0/2332458337 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0036a0 con 0x7fc68c0af200 2023-08-27T11:24:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.745+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c1f3640 con 0x7fc68c0af200 2023-08-27T11:24:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.746+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68c15ac80 con 0x7fc68c0af200 2023-08-27T11:24:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.746+0000 7fc682ffd700 1 -- 172.21.15.45:0/2332458337 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fc67c021030 con 0x7fc68c0af200 2023-08-27T11:24:46.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.747+0000 7fc682ffd700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc66c06a290 0x7fc66c06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.747+0000 7fc689d9b700 1 -- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc66c06a290 msgr2=0x7fc66c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.747+0000 7fc682ffd700 1 -- 172.21.15.45:0/2332458337 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7fc67c089ec0 con 0x7fc68c0af200 2023-08-27T11:24:46.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.747+0000 7fc689d9b700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc66c06a290 0x7fc66c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.752+0000 7fc682ffd700 1 -- 172.21.15.45:0/2332458337 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc68c15ac80 con 0x7fc68c0af200 2023-08-27T11:24:46.920 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:46.919+0000 7f086130e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:24:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.948+0000 7fc689d9b700 1 -- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc66c06a290 msgr2=0x7fc66c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.948+0000 7fc689d9b700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc66c06a290 0x7fc66c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.952+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc68c15ac80 con 0x7fc68c0af200 2023-08-27T11:24:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.952+0000 7fc682ffd700 1 -- 172.21.15.45:0/2332458337 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc68c15ac80 con 0x7fc68c0af200 2023-08-27T11:24:46.954 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:24:46.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc66c06a290 msgr2=0x7fc66c06c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc66c06a290 0x7fc66c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0af200 msgr2=0x7fc68c159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0af200 0x7fc68c159820 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc67c000c00 tx=0x7fc67c004750 comp rx=0 tx=0).stop 2023-08-27T11:24:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 shutdown_connections 2023-08-27T11:24:46.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fc66c06a290 0x7fc66c06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc68c0af200 0x7fc68c159820 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc68c0caa00 0x7fc68c0b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 --2- 172.21.15.45:0/2332458337 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc68c0ca620 0x7fc68c1f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:46.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 >> 172.21.15.45:0/2332458337 conn(0x7fc68c0cc7e0 msgr2=0x7fc68c0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:46.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 shutdown_connections 2023-08-27T11:24:46.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:46.954+0000 7fc690f08700 1 -- 172.21.15.45:0/2332458337 wait complete. 2023-08-27T11:24:46.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:24:46.987 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:46.985+0000 7fcc44fe2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:24:47.069 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:47.068+0000 7f086130e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:24:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.115+0000 7ff42ed22700 1 Processor -- start 2023-08-27T11:24:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.116+0000 7ff42ed22700 1 -- start start 2023-08-27T11:24:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff42ed22700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff42806a990 0x7ff42806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff42ed22700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42806f840 con 0x7ff4280c33f0 2023-08-27T11:24:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff42ed22700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4280ca4e0 con 0x7ff42815b160 2023-08-27T11:24:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff42ed22700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42806f280 con 0x7ff42806a990 2023-08-27T11:24:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff42d2bf700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff4280c33f0 0x7ff4280caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42420/0 (socket says 172.21.15.45:42420) 2023-08-27T11:24:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff42d2bf700 1 -- 172.21.15.45:0/3867092384 learned_addr learned my addr 172.21.15.45:0/3867092384 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff427fff700 1 --2- 172.21.15.45:0/3867092384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff42806a990 0x7ff42806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff4277fe700 1 -- 172.21.15.45:0/3867092384 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034077233 0 0) 0x7ff42806f840 con 0x7ff4280c33f0 2023-08-27T11:24:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff4277fe700 1 -- 172.21.15.45:0/3867092384 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4180035b0 con 0x7ff4280c33f0 2023-08-27T11:24:47.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff4277fe700 1 -- 172.21.15.45:0/3867092384 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608543381 0 0) 0x7ff4280ca4e0 con 0x7ff42815b160 2023-08-27T11:24:47.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.117+0000 7ff4277fe700 1 -- 172.21.15.45:0/3867092384 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff42806f840 con 0x7ff42815b160 2023-08-27T11:24:47.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.118+0000 7ff427fff700 1 -- 172.21.15.45:0/3867092384 >> v1:172.21.15.135:6789/0 conn(0x7ff42815b160 legacy=0x7ff42806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.118+0000 7ff427fff700 1 -- 172.21.15.45:0/3867092384 >> v1:172.21.15.45:6789/0 conn(0x7ff4280c33f0 legacy=0x7ff4280caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.118+0000 7ff427fff700 1 -- 172.21.15.45:0/3867092384 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42806f560 con 0x7ff42806a990 2023-08-27T11:24:47.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.118+0000 7ff427fff700 1 --2- 172.21.15.45:0/3867092384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff42806a990 0x7ff42806d900 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff414009870 tx=0x7ff414009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6896a771533ced30 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:47.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.118+0000 7ff4277fe700 1 -- 172.21.15.45:0/3867092384 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414011830 con 0x7ff42806a990 2023-08-27T11:24:47.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.118+0000 7ff4277fe700 1 -- 172.21.15.45:0/3867092384 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff414011990 con 0x7ff42806a990 2023-08-27T11:24:47.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.118+0000 7ff4277fe700 1 -- 172.21.15.45:0/3867092384 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414011d20 con 0x7ff42806a990 2023-08-27T11:24:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.119+0000 7ff42ed22700 1 -- 172.21.15.45:0/3867092384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff42806a990 msgr2=0x7ff42806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.119+0000 7ff42ed22700 1 --2- 172.21.15.45:0/3867092384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff42806a990 0x7ff42806d900 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff414009870 tx=0x7ff414009b80 comp rx=0 tx=0).stop 2023-08-27T11:24:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.119+0000 7ff42ed22700 1 -- 172.21.15.45:0/3867092384 shutdown_connections 2023-08-27T11:24:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.119+0000 7ff42ed22700 1 --2- 172.21.15.45:0/3867092384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff42806a990 0x7ff42806d900 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.119+0000 7ff42ed22700 1 -- 172.21.15.45:0/3867092384 >> 172.21.15.45:0/3867092384 conn(0x7ff4280cc7e0 msgr2=0x7ff4280ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.119+0000 7ff42ed22700 1 -- 172.21.15.45:0/3867092384 shutdown_connections 2023-08-27T11:24:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.119+0000 7ff42ed22700 1 -- 172.21.15.45:0/3867092384 wait complete. 2023-08-27T11:24:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.121+0000 7ff42ed22700 1 Processor -- start 2023-08-27T11:24:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.121+0000 7ff42ed22700 1 -- start start 2023-08-27T11:24:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.122+0000 7ff42ed22700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff42806a990 0x7ff4281493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.122+0000 7ff42ed22700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4280c33f0 0x7ff4280aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42ed22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff42815b160 0x7ff4280b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42ed22700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4281517e0 con 0x7ff42815b160 2023-08-27T11:24:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42ed22700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff428150bd0 con 0x7ff42806a990 2023-08-27T11:24:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42ed22700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff428150eb0 con 0x7ff4280c33f0 2023-08-27T11:24:47.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42d2bf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff42815b160 0x7ff4280b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42d2bf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff42815b160 0x7ff4280b4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55650/0 (socket says 172.21.15.45:55650) 2023-08-27T11:24:47.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42d2bf700 1 -- 172.21.15.45:0/1088144603 learned_addr learned my addr 172.21.15.45:0/1088144603 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:47.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff427fff700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4280c33f0 0x7ff4280aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42cabe700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff42806a990 0x7ff4281493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42d2bf700 1 -- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4280c33f0 msgr2=0x7ff4280aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42d2bf700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4280c33f0 0x7ff4280aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42d2bf700 1 -- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff42806a990 msgr2=0x7ff4281493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42d2bf700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff42806a990 0x7ff4281493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42d2bf700 1 -- 172.21.15.45:0/1088144603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4140094a0 con 0x7ff42815b160 2023-08-27T11:24:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff42d2bf700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff42815b160 0x7ff4280b4c50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff42000cac0 tx=0x7ff42000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff4257fa700 1 -- 172.21.15.45:0/1088144603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42000a9e0 con 0x7ff42815b160 2023-08-27T11:24:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.123+0000 7ff4257fa700 1 -- 172.21.15.45:0/1088144603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42000ab40 con 0x7ff42815b160 2023-08-27T11:24:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.124+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff428151190 con 0x7ff42815b160 2023-08-27T11:24:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.124+0000 7ff4257fa700 1 -- 172.21.15.45:0/1088144603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff420012400 con 0x7ff42815b160 2023-08-27T11:24:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.124+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4280c7e80 con 0x7ff42815b160 2023-08-27T11:24:47.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.125+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4280726d0 con 0x7ff42815b160 2023-08-27T11:24:47.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.125+0000 7ff4257fa700 1 -- 172.21.15.45:0/1088144603 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7ff42001b3b0 con 0x7ff42815b160 2023-08-27T11:24:47.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.126+0000 7ff4257fa700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff40c06a290 0x7ff40c06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.126+0000 7ff42cabe700 1 -- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff40c06a290 msgr2=0x7ff40c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:47.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.126+0000 7ff4257fa700 1 -- 172.21.15.45:0/1088144603 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7ff4200888c0 con 0x7ff42815b160 2023-08-27T11:24:47.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.126+0000 7ff42cabe700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff40c06a290 0x7ff40c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:47.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.131+0000 7ff4257fa700 1 -- 172.21.15.45:0/1088144603 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff420053880 con 0x7ff42815b160 2023-08-27T11:24:47.183 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:47.182+0000 7f086130e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:24:47.236 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:47.234+0000 7fcc44fe2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:24:47.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.327+0000 7ff42cabe700 1 -- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff40c06a290 msgr2=0x7ff40c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:47.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.327+0000 7ff42cabe700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff40c06a290 0x7ff40c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:47.337 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:47.336+0000 7f086130e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:24:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.340+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4280734a0 con 0x7ff42815b160 2023-08-27T11:24:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.341+0000 7ff4257fa700 1 -- 172.21.15.45:0/1088144603 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff420056ea0 con 0x7ff42815b160 2023-08-27T11:24:47.343 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:24:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff40c06a290 msgr2=0x7ff40c06c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff40c06a290 0x7ff40c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff42815b160 msgr2=0x7ff4280b4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff42815b160 0x7ff4280b4c50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff42000cac0 tx=0x7ff42000cdd0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 shutdown_connections 2023-08-27T11:24:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7ff40c06a290 0x7ff40c06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff42815b160 0x7ff4280b4c50 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff42806a990 0x7ff4281493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 --2- 172.21.15.45:0/1088144603 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff4280c33f0 0x7ff4280aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 >> 172.21.15.45:0/1088144603 conn(0x7ff4280cc7e0 msgr2=0x7ff4280b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:47.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 shutdown_connections 2023-08-27T11:24:47.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.343+0000 7ff42ed22700 1 -- 172.21.15.45:0/1088144603 wait complete. 2023-08-27T11:24:47.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:24:47.396 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:47.395+0000 7fcc44fe2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:24:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.485+0000 7fe6a234c700 1 Processor -- start 2023-08-27T11:24:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.485+0000 7fe6a234c700 1 -- start start 2023-08-27T11:24:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.486+0000 7fe6a234c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe69c15d3a0 0x7fe69c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.486+0000 7fe6a234c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c15d780 con 0x7fe69c06c4b0 2023-08-27T11:24:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.486+0000 7fe6a234c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69c0cfe30 con 0x7fe69c0d02c0 2023-08-27T11:24:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.486+0000 7fe6a234c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe69c0cf220 con 0x7fe69c15d3a0 2023-08-27T11:24:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.486+0000 7fe6a08e9700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe69c06c4b0 0x7fe69c0cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42434/0 (socket says 172.21.15.45:42434) 2023-08-27T11:24:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.486+0000 7fe6a08e9700 1 -- 172.21.15.45:0/1175367947 learned_addr learned my addr 172.21.15.45:0/1175367947 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69bfff700 1 --2- 172.21.15.45:0/1175367947 >> v2:172.21.15.135:3300/0 conn(0x7fe69c15d3a0 0x7fe69c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69affd700 1 -- 172.21.15.45:0/1175367947 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886023093 0 0) 0x7fe694001490 con 0x7fe69c06c4b0 2023-08-27T11:24:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69affd700 1 -- 172.21.15.45:0/1175367947 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6840035b0 con 0x7fe69c06c4b0 2023-08-27T11:24:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69bfff700 1 -- 172.21.15.45:0/1175367947 >> v1:172.21.15.135:6789/0 conn(0x7fe69c0d02c0 legacy=0x7fe69c06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69affd700 1 -- 172.21.15.45:0/1175367947 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289321473 0 0) 0x7fe69c0cfe30 con 0x7fe69c0d02c0 2023-08-27T11:24:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69bfff700 1 -- 172.21.15.45:0/1175367947 >> v1:172.21.15.45:6789/0 conn(0x7fe69c06c4b0 legacy=0x7fe69c0cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69bfff700 1 -- 172.21.15.45:0/1175367947 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c0cf500 con 0x7fe69c15d3a0 2023-08-27T11:24:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69bfff700 1 --2- 172.21.15.45:0/1175367947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe69c15d3a0 0x7fe69c0d1550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009840 tx=0x7fe68c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b7bdcc793e418e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69affd700 1 -- 172.21.15.45:0/1175367947 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c011830 con 0x7fe69c15d3a0 2023-08-27T11:24:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69affd700 1 -- 172.21.15.45:0/1175367947 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68c011990 con 0x7fe69c15d3a0 2023-08-27T11:24:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.487+0000 7fe69affd700 1 -- 172.21.15.45:0/1175367947 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c011d20 con 0x7fe69c15d3a0 2023-08-27T11:24:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.488+0000 7fe6a234c700 1 -- 172.21.15.45:0/1175367947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe69c15d3a0 msgr2=0x7fe69c0d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.488+0000 7fe6a234c700 1 --2- 172.21.15.45:0/1175367947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe69c15d3a0 0x7fe69c0d1550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009840 tx=0x7fe68c009b50 comp rx=0 tx=0).stop 2023-08-27T11:24:47.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.488+0000 7fe6a234c700 1 -- 172.21.15.45:0/1175367947 shutdown_connections 2023-08-27T11:24:47.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.488+0000 7fe6a234c700 1 --2- 172.21.15.45:0/1175367947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe69c15d3a0 0x7fe69c0d1550 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.488+0000 7fe6a234c700 1 -- 172.21.15.45:0/1175367947 >> 172.21.15.45:0/1175367947 conn(0x7fe69c09a1a0 msgr2=0x7fe69c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:47.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.488+0000 7fe6a234c700 1 -- 172.21.15.45:0/1175367947 shutdown_connections 2023-08-27T11:24:47.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.488+0000 7fe6a234c700 1 -- 172.21.15.45:0/1175367947 wait complete. 2023-08-27T11:24:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.489+0000 7fe6a234c700 1 Processor -- start 2023-08-27T11:24:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.489+0000 7fe6a234c700 1 -- start start 2023-08-27T11:24:47.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe6a234c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe69c06c4b0 0x7fe69c1e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe6a234c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe69c0d02c0 0x7fe69c14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe6a234c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69c15d3a0 0x7fe69c0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe6a234c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c1e7160 con 0x7fe69c15d3a0 2023-08-27T11:24:47.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe6a234c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0b4e90 con 0x7fe69c0d02c0 2023-08-27T11:24:47.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe6a234c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c0b50e0 con 0x7fe69c06c4b0 2023-08-27T11:24:47.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe6a08e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69c15d3a0 0x7fe69c0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe6a08e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69c15d3a0 0x7fe69c0b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55664/0 (socket says 172.21.15.45:55664) 2023-08-27T11:24:47.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe6a08e9700 1 -- 172.21.15.45:0/4014504870 learned_addr learned my addr 172.21.15.45:0/4014504870 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:47.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe69b7fe700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe69c0d02c0 0x7fe69c14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.490+0000 7fe69bfff700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe69c06c4b0 0x7fe69c1e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe6a08e9700 1 -- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe69c06c4b0 msgr2=0x7fe69c1e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe6a08e9700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe69c06c4b0 0x7fe69c1e6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe6a08e9700 1 -- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe69c0d02c0 msgr2=0x7fe69c14b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe6a08e9700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe69c0d02c0 0x7fe69c14b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe6a08e9700 1 -- 172.21.15.45:0/4014504870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c0094a0 con 0x7fe69c15d3a0 2023-08-27T11:24:47.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe6a08e9700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69c15d3a0 0x7fe69c0b6180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe69400cb40 tx=0x7fe69400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:47.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe698ff9700 1 -- 172.21.15.45:0/4014504870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69400a9e0 con 0x7fe69c15d3a0 2023-08-27T11:24:47.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe698ff9700 1 -- 172.21.15.45:0/4014504870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69400ab40 con 0x7fe69c15d3a0 2023-08-27T11:24:47.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c0b53c0 con 0x7fe69c15d3a0 2023-08-27T11:24:47.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe698ff9700 1 -- 172.21.15.45:0/4014504870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694012400 con 0x7fe69c15d3a0 2023-08-27T11:24:47.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.491+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69c0b5980 con 0x7fe69c15d3a0 2023-08-27T11:24:47.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.492+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69c0d1a10 con 0x7fe69c15d3a0 2023-08-27T11:24:47.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.493+0000 7fe698ff9700 1 -- 172.21.15.45:0/4014504870 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fe69401b3b0 con 0x7fe69c15d3a0 2023-08-27T11:24:47.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.493+0000 7fe698ff9700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe68806a290 0x7fe68806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.493+0000 7fe698ff9700 1 -- 172.21.15.45:0/4014504870 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7fe6940886f0 con 0x7fe69c15d3a0 2023-08-27T11:24:47.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.493+0000 7fe69bfff700 1 -- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe68806a290 msgr2=0x7fe68806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:47.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.493+0000 7fe69bfff700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe68806a290 0x7fe68806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.497+0000 7fe698ff9700 1 -- 172.21.15.45:0/4014504870 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe694053760 con 0x7fe69c15d3a0 2023-08-27T11:24:47.521 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:47.519+0000 7fcc44fe2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:24:47.601 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:47.600+0000 7f086130e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:24:47.688 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:47.683+0000 7fcc44fe2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:24:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.694+0000 7fe69bfff700 1 -- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe68806a290 msgr2=0x7fe68806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.694+0000 7fe69bfff700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe68806a290 0x7fe68806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:47.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.697+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe69c15d780 con 0x7fe69c15d3a0 2023-08-27T11:24:47.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.697+0000 7fe698ff9700 1 -- 172.21.15.45:0/4014504870 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe69c15d780 con 0x7fe69c15d3a0 2023-08-27T11:24:47.699 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:24:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.699+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe68806a290 msgr2=0x7fe68806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.699+0000 7fe6a234c700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe68806a290 0x7fe68806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.699+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69c15d3a0 msgr2=0x7fe69c0b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.699+0000 7fe6a234c700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69c15d3a0 0x7fe69c0b6180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe69400cb40 tx=0x7fe69400ce50 comp rx=0 tx=0).stop 2023-08-27T11:24:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.699+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 shutdown_connections 2023-08-27T11:24:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.699+0000 7fe6a234c700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fe68806a290 0x7fe68806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.699+0000 7fe6a234c700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69c15d3a0 0x7fe69c0b6180 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.699+0000 7fe6a234c700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe69c06c4b0 0x7fe69c1e6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.699+0000 7fe6a234c700 1 --2- 172.21.15.45:0/4014504870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe69c0d02c0 0x7fe69c14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.700+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 >> 172.21.15.45:0/4014504870 conn(0x7fe69c09a1a0 msgr2=0x7fe69c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.700+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 shutdown_connections 2023-08-27T11:24:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.700+0000 7fe6a234c700 1 -- 172.21.15.45:0/4014504870 wait complete. 2023-08-27T11:24:47.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:24:47.760 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:47.759+0000 7f086130e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:24:47.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.885+0000 7fb767c8b700 1 Processor -- start 2023-08-27T11:24:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.886+0000 7fb767c8b700 1 -- start start 2023-08-27T11:24:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.886+0000 7fb767c8b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb76015af30 0x7fb7600cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb767c8b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb760164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb767c8b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7600cb8b0 con 0x7fb7600cf960 2023-08-27T11:24:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb767c8b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76006f9c0 con 0x7fb76015af30 2023-08-27T11:24:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb767c8b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7600cecd0 con 0x7fb7600cde80 2023-08-27T11:24:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb765a27700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb76015af30 0x7fb7600cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb765a27700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb76015af30 0x7fb7600cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:44526/0 (socket says 172.21.15.45:44526) 2023-08-27T11:24:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb766228700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb760164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb765226700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb7600cf960 0x7fb7600cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:38110/0 (socket says 172.21.15.45:38110) 2023-08-27T11:24:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb765a27700 1 -- 172.21.15.45:0/1331468005 learned_addr learned my addr 172.21.15.45:0/1331468005 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb765a27700 1 -- 172.21.15.45:0/1331468005 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 msgr2=0x7fb760164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb765a27700 1 --2- 172.21.15.45:0/1331468005 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb760164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb765a27700 1 -- 172.21.15.45:0/1331468005 >> v1:172.21.15.135:6789/0 conn(0x7fb7600cf960 legacy=0x7fb7600cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb764a25700 1 -- 172.21.15.45:0/1331468005 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492447693 0 0) 0x7fb7600cb8b0 con 0x7fb7600cf960 2023-08-27T11:24:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb765a27700 1 -- 172.21.15.45:0/1331468005 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7600cbae0 con 0x7fb76015af30 2023-08-27T11:24:47.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb766228700 1 --2- 172.21.15.45:0/1331468005 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb760164fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:47.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.887+0000 7fb765a27700 1 --2- 172.21.15.45:0/1331468005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb76015af30 0x7fb7600cf420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098f0 tx=0x7fb75c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afc34858bc7af48f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:47.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb764a25700 1 -- 172.21.15.45:0/1331468005 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c011830 con 0x7fb76015af30 2023-08-27T11:24:47.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb764a25700 1 -- 172.21.15.45:0/1331468005 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75c011990 con 0x7fb76015af30 2023-08-27T11:24:47.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb764a25700 1 -- 172.21.15.45:0/1331468005 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c011d20 con 0x7fb76015af30 2023-08-27T11:24:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb767c8b700 1 -- 172.21.15.45:0/1331468005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb76015af30 msgr2=0x7fb7600cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb767c8b700 1 --2- 172.21.15.45:0/1331468005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb76015af30 0x7fb7600cf420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098f0 tx=0x7fb75c009c00 comp rx=0 tx=0).stop 2023-08-27T11:24:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb767c8b700 1 -- 172.21.15.45:0/1331468005 shutdown_connections 2023-08-27T11:24:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb767c8b700 1 --2- 172.21.15.45:0/1331468005 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb760164fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb767c8b700 1 --2- 172.21.15.45:0/1331468005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb76015af30 0x7fb7600cf420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb767c8b700 1 -- 172.21.15.45:0/1331468005 >> 172.21.15.45:0/1331468005 conn(0x7fb76009a1a0 msgr2=0x7fb7600d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:47.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb767c8b700 1 -- 172.21.15.45:0/1331468005 shutdown_connections 2023-08-27T11:24:47.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.888+0000 7fb767c8b700 1 -- 172.21.15.45:0/1331468005 wait complete. 2023-08-27T11:24:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.890+0000 7fb767c8b700 1 Processor -- start 2023-08-27T11:24:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.890+0000 7fb767c8b700 1 -- start start 2023-08-27T11:24:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.890+0000 7fb767c8b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb76006bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.890+0000 7fb767c8b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7600cf960 0x7fb76006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.890+0000 7fb767c8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb76015af30 0x7fb7601f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.890+0000 7fb767c8b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76006e860 con 0x7fb76015af30 2023-08-27T11:24:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.890+0000 7fb767c8b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601f25a0 con 0x7fb7600cf960 2023-08-27T11:24:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.890+0000 7fb767c8b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76006c360 con 0x7fb7600cde80 2023-08-27T11:24:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb766228700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb76015af30 0x7fb7601f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb765226700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7600cf960 0x7fb76006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb765a27700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb76006bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb766228700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb76015af30 0x7fb7601f2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55678/0 (socket says 172.21.15.45:55678) 2023-08-27T11:24:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb766228700 1 -- 172.21.15.45:0/187977671 learned_addr learned my addr 172.21.15.45:0/187977671 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb765226700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7600cf960 0x7fb76006e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:44532/0 (socket says 172.21.15.45:44532) 2023-08-27T11:24:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb766228700 1 -- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 msgr2=0x7fb76006bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb766228700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb76006bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb766228700 1 -- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7600cf960 msgr2=0x7fb76006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb766228700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7600cf960 0x7fb76006e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:47.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb766228700 1 -- 172.21.15.45:0/187977671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758009a70 con 0x7fb76015af30 2023-08-27T11:24:47.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb765226700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7600cf960 0x7fb76006e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:47.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb765a27700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb76006bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:47.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb766228700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb76015af30 0x7fb7601f2060 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb75800ef90 tx=0x7fb75800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:47.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb7567fc700 1 -- 172.21.15.45:0/187977671 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75800cc90 con 0x7fb76015af30 2023-08-27T11:24:47.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb7567fc700 1 -- 172.21.15.45:0/187977671 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75800f040 con 0x7fb76015af30 2023-08-27T11:24:47.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c009550 con 0x7fb76015af30 2023-08-27T11:24:47.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb7567fc700 1 -- 172.21.15.45:0/187977671 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7580146c0 con 0x7fb76015af30 2023-08-27T11:24:47.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.891+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76006c920 con 0x7fb76015af30 2023-08-27T11:24:47.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.892+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7600734a0 con 0x7fb76015af30 2023-08-27T11:24:47.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.894+0000 7fb7567fc700 1 -- 172.21.15.45:0/187977671 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fb758014820 con 0x7fb76015af30 2023-08-27T11:24:47.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.894+0000 7fb7567fc700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb74406a290 0x7fb74406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:47.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.894+0000 7fb7567fc700 1 -- 172.21.15.45:0/187977671 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7fb7580899c0 con 0x7fb76015af30 2023-08-27T11:24:47.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.894+0000 7fb765a27700 1 -- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb74406a290 msgr2=0x7fb74406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:47.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.894+0000 7fb765a27700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb74406a290 0x7fb74406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:47.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:47.898+0000 7fb7567fc700 1 -- 172.21.15.45:0/187977671 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb758054870 con 0x7fb76015af30 2023-08-27T11:24:47.964 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:47.962+0000 7fcc44fe2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:24:48.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.095+0000 7fb765a27700 1 -- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb74406a290 msgr2=0x7fb74406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:48.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.095+0000 7fb765a27700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb74406a290 0x7fb74406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:24:48.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.102+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7600ce4c0 con 0x7fb76015af30 2023-08-27T11:24:48.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.103+0000 7fb7567fc700 1 -- 172.21.15.45:0/187977671 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb758009cd0 con 0x7fb76015af30 2023-08-27T11:24:48.105 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:24:48.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb74406a290 msgr2=0x7fb74406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:48.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb74406a290 0x7fb74406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb76015af30 msgr2=0x7fb7601f2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb76015af30 0x7fb7601f2060 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb75800ef90 tx=0x7fb75800c3a0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 shutdown_connections 2023-08-27T11:24:48.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fb74406a290 0x7fb74406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb76015af30 0x7fb7601f2060 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7600cde80 0x7fb76006bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 --2- 172.21.15.45:0/187977671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7600cf960 0x7fb76006e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 >> 172.21.15.45:0/187977671 conn(0x7fb76009a1a0 msgr2=0x7fb7600b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:48.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 shutdown_connections 2023-08-27T11:24:48.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.105+0000 7fb767c8b700 1 -- 172.21.15.45:0/187977671 wait complete. 2023-08-27T11:24:48.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:24:48.136 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:48.135+0000 7fcc44fe2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.246+0000 7fcab6a99700 1 Processor -- start 2023-08-27T11:24:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.246+0000 7fcab6a99700 1 -- start start 2023-08-27T11:24:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcab6a99700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcab00cdb80 0x7fcab0165da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcab6a99700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab00ada50 con 0x7fcab015b120 2023-08-27T11:24:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcab6a99700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab00ccd60 con 0x7fcab00d1a20 2023-08-27T11:24:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcab6a99700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab00ccb40 con 0x7fcab00cdb80 2023-08-27T11:24:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcab5036700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcab00cdb80 0x7fcab0165da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcab4835700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcab015b120 0x7fcab00cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42460/0 (socket says 172.21.15.45:42460) 2023-08-27T11:24:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcab4835700 1 -- 172.21.15.45:0/400519899 learned_addr learned my addr 172.21.15.45:0/400519899 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcab5036700 1 -- 172.21.15.45:0/400519899 >> v1:172.21.15.135:6789/0 conn(0x7fcab00d1a20 legacy=0x7fcab00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcab5036700 1 -- 172.21.15.45:0/400519899 >> v1:172.21.15.45:6789/0 conn(0x7fcab015b120 legacy=0x7fcab00cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcab5036700 1 -- 172.21.15.45:0/400519899 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00b0860 con 0x7fcab00cdb80 2023-08-27T11:24:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.247+0000 7fcaaf7fe700 1 -- 172.21.15.45:0/400519899 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010542932 0 0) 0x7fcab00ada50 con 0x7fcab015b120 2023-08-27T11:24:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcab5036700 1 --2- 172.21.15.45:0/400519899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab00cdb80 0x7fcab0165da0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009d90 tx=0x7fcaa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5334cb72115f80e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcaaf7fe700 1 -- 172.21.15.45:0/400519899 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa800e070 con 0x7fcab00cdb80 2023-08-27T11:24:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcaaf7fe700 1 -- 172.21.15.45:0/400519899 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa800bbd0 con 0x7fcab00cdb80 2023-08-27T11:24:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcaaf7fe700 1 -- 172.21.15.45:0/400519899 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8012430 con 0x7fcab00cdb80 2023-08-27T11:24:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcab6a99700 1 -- 172.21.15.45:0/400519899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab00cdb80 msgr2=0x7fcab0165da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcab6a99700 1 --2- 172.21.15.45:0/400519899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab00cdb80 0x7fcab0165da0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009d90 tx=0x7fcaa800b330 comp rx=0 tx=0).stop 2023-08-27T11:24:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcab6a99700 1 -- 172.21.15.45:0/400519899 shutdown_connections 2023-08-27T11:24:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcab6a99700 1 --2- 172.21.15.45:0/400519899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab00cdb80 0x7fcab0165da0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.248+0000 7fcab6a99700 1 -- 172.21.15.45:0/400519899 >> 172.21.15.45:0/400519899 conn(0x7fcab009a160 msgr2=0x7fcab00d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.249+0000 7fcab6a99700 1 -- 172.21.15.45:0/400519899 shutdown_connections 2023-08-27T11:24:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.249+0000 7fcab6a99700 1 -- 172.21.15.45:0/400519899 wait complete. 2023-08-27T11:24:48.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.250+0000 7fcab6a99700 1 Processor -- start 2023-08-27T11:24:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.250+0000 7fcab6a99700 1 -- start start 2023-08-27T11:24:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab6a99700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcab00cdb80 0x7fcab015bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab6a99700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcab00d1a20 0x7fcab01506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab6a99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab015b120 0x7fcab00c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab6a99700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab015c450 con 0x7fcab015b120 2023-08-27T11:24:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab6a99700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0150c00 con 0x7fcab00cdb80 2023-08-27T11:24:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab6a99700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0157930 con 0x7fcab00d1a20 2023-08-27T11:24:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab5036700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab015b120 0x7fcab00c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab5036700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab015b120 0x7fcab00c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55694/0 (socket says 172.21.15.45:55694) 2023-08-27T11:24:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab5036700 1 -- 172.21.15.45:0/1135952855 learned_addr learned my addr 172.21.15.45:0/1135952855 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab4835700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcab00cdb80 0x7fcab015bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcaaffff700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcab00d1a20 0x7fcab01506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab5036700 1 -- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcab00d1a20 msgr2=0x7fcab01506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab5036700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcab00d1a20 0x7fcab01506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab5036700 1 -- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcab00cdb80 msgr2=0x7fcab015bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab5036700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcab00cdb80 0x7fcab015bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab5036700 1 -- 172.21.15.45:0/1135952855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa8009a70 con 0x7fcab015b120 2023-08-27T11:24:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcab5036700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab015b120 0x7fcab00c95b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8014040 tx=0x7fcaa8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.251+0000 7fcaad7fa700 1 -- 172.21.15.45:0/1135952855 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8012430 con 0x7fcab015b120 2023-08-27T11:24:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.252+0000 7fcaad7fa700 1 -- 172.21.15.45:0/1135952855 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa8020070 con 0x7fcab015b120 2023-08-27T11:24:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.252+0000 7fcab6a99700 1 -- 172.21.15.45:0/1135952855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab015d1d0 con 0x7fcab015b120 2023-08-27T11:24:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.252+0000 7fcaad7fa700 1 -- 172.21.15.45:0/1135952855 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8004280 con 0x7fcab015b120 2023-08-27T11:24:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.252+0000 7fcab6a99700 1 -- 172.21.15.45:0/1135952855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab015d790 con 0x7fcab015b120 2023-08-27T11:24:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.252+0000 7fcab6a99700 1 -- 172.21.15.45:0/1135952855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab009edd0 con 0x7fcab015b120 2023-08-27T11:24:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.254+0000 7fcaad7fa700 1 -- 172.21.15.45:0/1135952855 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fcaa80043e0 con 0x7fcab015b120 2023-08-27T11:24:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.254+0000 7fcaad7fa700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fca9406a290 0x7fca9406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.254+0000 7fcaad7fa700 1 -- 172.21.15.45:0/1135952855 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7250+0+0 (secure 0 0 0) 0x7fcaa8022070 con 0x7fcab015b120 2023-08-27T11:24:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.254+0000 7fcab4835700 1 -- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fca9406a290 msgr2=0x7fca9406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/584312721 2023-08-27T11:24:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.254+0000 7fcab4835700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fca9406a290 0x7fca9406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:24:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.262+0000 7fcaad7fa700 1 -- 172.21.15.45:0/1135952855 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcaa80561a0 con 0x7fcab015b120 2023-08-27T11:24:48.288 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.287+0000 7f086130e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:24:48.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.308+0000 7fcaad7fa700 1 -- 172.21.15.45:0/1135952855 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 15) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fcaa8056570 con 0x7fcab015b120 2023-08-27T11:24:48.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.309+0000 7fcaad7fa700 1 -- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fca9406a290 msgr2=0x7fca9406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:24:48.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.309+0000 7fcaad7fa700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fca9406a290 0x7fca9406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.370 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07d9376700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.370 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07d9376700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.371 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07d9376700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.371 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07d9376700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.371 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07d9376700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.371 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07ddb7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.371 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07ddb7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.372 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07ddb7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.372 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07ddb7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.372 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:24:48.369+0000 7f07ddb7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:24:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.519+0000 7fcab6a99700 1 -- 172.21.15.45:0/1135952855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcab00ce000 con 0x7fcab015b120 2023-08-27T11:24:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.519+0000 7fcaad7fa700 1 -- 172.21.15.45:0/1135952855 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4881 (secure 0 0 0) 0x7fcaa8082fc0 con 0x7fcab015b120 2023-08-27T11:24:48.521 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:48.522 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1324661970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1324661970},{"type":"v1","addr":"172.21.15.135:6836","nonce":1324661970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2014390806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2014390806},{"type":"v1","addr":"172.21.15.135:6837","nonce":2014390806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2103056681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2103056681},{"type":"v1","addr":"172.21.15.45:6834","nonce":2103056681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:13:37.320852+0000","modified":"2023-08-27T11:13:40.917661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/2587151358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2587151358},{"type":"v1","addr":"172.21.15.45:6835","nonce":2587151358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:24:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.521+0000 7fcab6a99700 1 -- 172.21.15.45:0/1135952855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab015b120 msgr2=0x7fcab00c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.521+0000 7fcab6a99700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab015b120 0x7fcab00c95b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8014040 tx=0x7fcaa8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.522+0000 7fcab6a99700 1 -- 172.21.15.45:0/1135952855 shutdown_connections 2023-08-27T11:24:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.522+0000 7fcab6a99700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:6832/584312721,v1:172.21.15.135:6833/584312721] conn(0x7fca9406a290 0x7fca9406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.522+0000 7fcab6a99700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcab015b120 0x7fcab00c95b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.522+0000 7fcab6a99700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcab00cdb80 0x7fcab015bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.522+0000 7fcab6a99700 1 --2- 172.21.15.45:0/1135952855 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fcab00d1a20 0x7fcab01506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.522+0000 7fcab6a99700 1 -- 172.21.15.45:0/1135952855 >> 172.21.15.45:0/1135952855 conn(0x7fcab009a160 msgr2=0x7fcab015a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.522+0000 7fcab6a99700 1 -- 172.21.15.45:0/1135952855 shutdown_connections 2023-08-27T11:24:48.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.522+0000 7fcab6a99700 1 -- 172.21.15.45:0/1135952855 wait complete. 2023-08-27T11:24:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2023-08-27T11:24:48.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:24:48.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.702+0000 7ff9b6b36700 1 Processor -- start 2023-08-27T11:24:48.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.702+0000 7ff9b6b36700 1 -- start start 2023-08-27T11:24:48.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.703+0000 7ff9b6b36700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff9b015d140 0x7ff9b00ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:48.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.703+0000 7ff9b6b36700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b00cff50 con 0x7ff9b00cc790 2023-08-27T11:24:48.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.703+0000 7ff9b6b36700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b00cc4e0 con 0x7ff9b006bda0 2023-08-27T11:24:48.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.703+0000 7ff9b6b36700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b00cd330 con 0x7ff9b015d140 2023-08-27T11:24:48.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.703+0000 7ff9b50d3700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff9b00cc790 0x7ff9b00ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42478/0 (socket says 172.21.15.45:42478) 2023-08-27T11:24:48.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.703+0000 7ff9b50d3700 1 -- 172.21.15.45:0/1333119850 learned_addr learned my addr 172.21.15.45:0/1333119850 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.703+0000 7ff9b48d2700 1 --2- 172.21.15.45:0/1333119850 >> v2:172.21.15.135:3300/0 conn(0x7ff9b015d140 0x7ff9b00ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.703+0000 7ff9af7fe700 1 -- 172.21.15.45:0/1333119850 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124790794 0 0) 0x7ff9b00cff50 con 0x7ff9b00cc790 2023-08-27T11:24:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.703+0000 7ff9af7fe700 1 -- 172.21.15.45:0/1333119850 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a00035b0 con 0x7ff9b00cc790 2023-08-27T11:24:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9af7fe700 1 -- 172.21.15.45:0/1333119850 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639175417 0 0) 0x7ff9b00cc4e0 con 0x7ff9b006bda0 2023-08-27T11:24:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9af7fe700 1 -- 172.21.15.45:0/1333119850 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b00cff50 con 0x7ff9b006bda0 2023-08-27T11:24:48.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9b48d2700 1 -- 172.21.15.45:0/1333119850 >> v1:172.21.15.135:6789/0 conn(0x7ff9b006bda0 legacy=0x7ff9b006c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9b48d2700 1 -- 172.21.15.45:0/1333119850 >> v1:172.21.15.45:6789/0 conn(0x7ff9b00cc790 legacy=0x7ff9b00ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9af7fe700 1 -- 172.21.15.45:0/1333119850 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2463566807 0 0) 0x7ff9a00035b0 con 0x7ff9b00cc790 2023-08-27T11:24:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9b48d2700 1 -- 172.21.15.45:0/1333119850 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00cf0a0 con 0x7ff9b015d140 2023-08-27T11:24:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9b48d2700 1 --2- 172.21.15.45:0/1333119850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9b015d140 0x7ff9b00ce110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009840 tx=0x7ff9a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d96e0c56aa2385ab server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9af7fe700 1 -- 172.21.15.45:0/1333119850 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80019f0 con 0x7ff9b015d140 2023-08-27T11:24:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9af7fe700 1 -- 172.21.15.45:0/1333119850 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a4004030 con 0x7ff9b015d140 2023-08-27T11:24:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.704+0000 7ff9af7fe700 1 -- 172.21.15.45:0/1333119850 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40043c0 con 0x7ff9b015d140 2023-08-27T11:24:48.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.705+0000 7ff9b6b36700 1 -- 172.21.15.45:0/1333119850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9b015d140 msgr2=0x7ff9b00ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.705+0000 7ff9b6b36700 1 --2- 172.21.15.45:0/1333119850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9b015d140 0x7ff9b00ce110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009840 tx=0x7ff9a4009b50 comp rx=0 tx=0).stop 2023-08-27T11:24:48.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.705+0000 7ff9b6b36700 1 -- 172.21.15.45:0/1333119850 shutdown_connections 2023-08-27T11:24:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.705+0000 7ff9b6b36700 1 --2- 172.21.15.45:0/1333119850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9b015d140 0x7ff9b00ce110 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.705+0000 7ff9b6b36700 1 -- 172.21.15.45:0/1333119850 >> 172.21.15.45:0/1333119850 conn(0x7ff9b009a160 msgr2=0x7ff9b00d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.705+0000 7ff9b6b36700 1 -- 172.21.15.45:0/1333119850 shutdown_connections 2023-08-27T11:24:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.705+0000 7ff9b6b36700 1 -- 172.21.15.45:0/1333119850 wait complete. 2023-08-27T11:24:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.706+0000 7ff9b6b36700 1 Processor -- start 2023-08-27T11:24:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.706+0000 7ff9b6b36700 1 -- start start 2023-08-27T11:24:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9b6b36700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9b006bda0 0x7ff9b01e6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9b6b36700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cc790 0x7ff9b00c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9b6b36700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff9b015d140 0x7ff9b01581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9b6b36700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b015ee30 con 0x7ff9b00cc790 2023-08-27T11:24:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9b6b36700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00c9020 con 0x7ff9b006bda0 2023-08-27T11:24:48.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9b6b36700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b01e6a60 con 0x7ff9b015d140 2023-08-27T11:24:48.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9affff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cc790 0x7ff9b00c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:48.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9affff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cc790 0x7ff9b00c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55706/0 (socket says 172.21.15.45:55706) 2023-08-27T11:24:48.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9affff700 1 -- 172.21.15.45:0/2684810759 learned_addr learned my addr 172.21.15.45:0/2684810759 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:48.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9b48d2700 1 --2- 172.21.15.45:0/2684810759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9b006bda0 0x7ff9b01e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:48.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9b50d3700 1 --2- 172.21.15.45:0/2684810759 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff9b015d140 0x7ff9b01581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:48.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9affff700 1 -- 172.21.15.45:0/2684810759 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff9b015d140 msgr2=0x7ff9b01581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9affff700 1 --2- 172.21.15.45:0/2684810759 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff9b015d140 0x7ff9b01581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9affff700 1 -- 172.21.15.45:0/2684810759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9b006bda0 msgr2=0x7ff9b01e6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9affff700 1 --2- 172.21.15.45:0/2684810759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9b006bda0 0x7ff9b01e6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.707+0000 7ff9affff700 1 -- 172.21.15.45:0/2684810759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a40094a0 con 0x7ff9b00cc790 2023-08-27T11:24:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.708+0000 7ff9affff700 1 --2- 172.21.15.45:0/2684810759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cc790 0x7ff9b00c7f50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c500 tx=0x7ff99c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.708+0000 7ff9ad7fa700 1 -- 172.21.15.45:0/2684810759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0052d0 con 0x7ff9b00cc790 2023-08-27T11:24:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.708+0000 7ff9ad7fa700 1 -- 172.21.15.45:0/2684810759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c005450 con 0x7ff9b00cc790 2023-08-27T11:24:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.708+0000 7ff9b6b36700 1 -- 172.21.15.45:0/2684810759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b00c8780 con 0x7ff9b00cc790 2023-08-27T11:24:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.708+0000 7ff9ad7fa700 1 -- 172.21.15.45:0/2684810759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0038c0 con 0x7ff9b00cc790 2023-08-27T11:24:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.708+0000 7ff9b6b36700 1 -- 172.21.15.45:0/2684810759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b00c8d40 con 0x7ff9b00cc790 2023-08-27T11:24:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.709+0000 7ff9b6b36700 1 -- 172.21.15.45:0/2684810759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b015ef70 con 0x7ff9b00cc790 2023-08-27T11:24:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.709+0000 7ff9ad7fa700 1 -- 172.21.15.45:0/2684810759 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) v1 ==== 88052+0+0 (secure 0 0 0) 0x7ff99c003a20 con 0x7ff9b00cc790 2023-08-27T11:24:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.710+0000 7ff9ad7fa700 1 -- 172.21.15.45:0/2684810759 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7593+0+0 (secure 0 0 0) 0x7ff99c087ab0 con 0x7ff9b00cc790 2023-08-27T11:24:48.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.718+0000 7ff9ad7fa700 1 -- 172.21.15.45:0/2684810759 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff9b015ef70 con 0x7ff9b00cc790 2023-08-27T11:24:48.726 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:24:48.721+0000 7fcc44fe2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:24:48.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.922+0000 7ff9b6b36700 1 -- 172.21.15.45:0/2684810759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9b015ef70 con 0x7ff9b00cc790 2023-08-27T11:24:48.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.923+0000 7ff9ad7fa700 1 -- 172.21.15.45:0/2684810759 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+20350 (secure 0 0 0) 0x7ff9b015ef70 con 0x7ff9b00cc790 2023-08-27T11:24:48.925 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":57,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:24:48.301697+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:13:35.610850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-27T11:13:36.282109+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":218,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:6833/584312721":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/3750381439":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/459759971":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/1050128524":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/2208777569":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/918212180":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/2898860171":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1692978501":"2023-08-28T11:13:52.047816+0000","172.21.15.135:6832/584312721":"2023-08-28T11:24:48.301631+0000","172.21.15.135:6832/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1902421107":"2023-08-28T11:24:48.301631+0000","172.21.15.135:6833/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1110669507":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/4257866035":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/2587284130":"2023-08-28T11:13:52.047816+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":4,"length":2},{"begin":8,"length":2},{"begin":12,"length":1},{"begin":14,"length":2},{"begin":18,"length":2},{"begin":23,"length":1},{"begin":25,"length":2},{"begin":29,"length":2},{"begin":33,"length":1},{"begin":35,"length":2},{"begin":39,"length":3},{"begin":44,"length":1},{"begin":46,"length":2},{"begin":50,"length":2},{"begin":54,"length":1},{"begin":56,"length":2},{"begin":60,"length":2},{"begin":64,"length":1},{"begin":66,"length":2},{"begin":70,"length":3},{"begin":75,"length":1},{"begin":77,"length":2},{"begin":81,"length":2},{"begin":85,"length":1},{"begin":87,"length":2},{"begin":92,"length":2},{"begin":96,"length":1},{"begin":98,"length":2},{"begin":102,"length":3},{"begin":108,"length":2},{"begin":165,"length":54}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:24:48.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.925+0000 7ff9b6b36700 1 -- 172.21.15.45:0/2684810759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cc790 msgr2=0x7ff9b00c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:48.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.925+0000 7ff9b6b36700 1 --2- 172.21.15.45:0/2684810759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cc790 0x7ff9b00c7f50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c500 tx=0x7ff99c00c8c0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.925+0000 7ff9b6b36700 1 -- 172.21.15.45:0/2684810759 shutdown_connections 2023-08-27T11:24:48.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.925+0000 7ff9b6b36700 1 --2- 172.21.15.45:0/2684810759 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff9b015d140 0x7ff9b01581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.925+0000 7ff9b6b36700 1 --2- 172.21.15.45:0/2684810759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9b006bda0 0x7ff9b01e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.925+0000 7ff9b6b36700 1 --2- 172.21.15.45:0/2684810759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cc790 0x7ff9b00c7f50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:48.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.925+0000 7ff9b6b36700 1 -- 172.21.15.45:0/2684810759 >> 172.21.15.45:0/2684810759 conn(0x7ff9b009a160 msgr2=0x7ff9b00b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:48.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.925+0000 7ff9b6b36700 1 -- 172.21.15.45:0/2684810759 shutdown_connections 2023-08-27T11:24:48.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:48.925+0000 7ff9b6b36700 1 -- 172.21.15.45:0/2684810759 wait complete. 2023-08-27T11:24:48.940 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:24:48.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:24:49.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.065+0000 7f64b5270700 1 Processor -- start 2023-08-27T11:24:49.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.065+0000 7f64b5270700 1 -- start start 2023-08-27T11:24:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.065+0000 7f64b5270700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00ce4b0 0x7f64b0167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.065+0000 7f64b5270700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b015d6a0 con 0x7f64b00b8c20 2023-08-27T11:24:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.065+0000 7f64b5270700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b00d2c90 con 0x7f64b015d2c0 2023-08-27T11:24:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.065+0000 7f64b5270700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b00d1bf0 con 0x7f64b00ce4b0 2023-08-27T11:24:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64ae7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f64b00b8c20 0x7f64b00cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42488/0 (socket says 172.21.15.45:42488) 2023-08-27T11:24:49.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64ae7fc700 1 -- 172.21.15.45:0/3476452856 learned_addr learned my addr 172.21.15.45:0/3476452856 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:49.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64af7fe700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00ce4b0 0x7f64b0167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583155668 0 0) 0x7f64a0001110 con 0x7f64b00b8c20 2023-08-27T11:24:49.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64980035b0 con 0x7f64b00b8c20 2023-08-27T11:24:49.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896085428 0 0) 0x7f64b00d2c90 con 0x7f64b015d2c0 2023-08-27T11:24:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a0001110 con 0x7f64b015d2c0 2023-08-27T11:24:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 92102798 0 0) 0x7f64980035b0 con 0x7f64b00b8c20 2023-08-27T11:24:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00ce4b0 msgr2=0x7f64b0167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:49.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00ce4b0 0x7f64b0167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 >> v1:172.21.15.135:6789/0 conn(0x7f64b015d2c0 legacy=0x7f64b00d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b00d1ed0 con 0x7f64b00b8c20 2023-08-27T11:24:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f64a0001800 con 0x7f64b00b8c20 2023-08-27T11:24:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64af7fe700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00ce4b0 0x7f64b0167d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 >> v1:172.21.15.45:6789/0 conn(0x7f64b00b8c20 legacy=0x7f64b00cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6498003bc0 0x7f6498003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00ce4b0 0x7f6498004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6498004d90 0x7f6498005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b00d1bf0 con 0x7f6498004d90 2023-08-27T11:24:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649800bac0 con 0x7f6498003bc0 2023-08-27T11:24:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649800bc30 con 0x7f64b00ce4b0 2023-08-27T11:24:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64af7fe700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6498004d90 0x7f6498005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64ae7fc700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00ce4b0 0x7f6498004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.066+0000 7f64aeffd700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6498003bc0 0x7f6498003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64af7fe700 1 -- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00ce4b0 msgr2=0x7f6498004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64af7fe700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00ce4b0 0x7f6498004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64af7fe700 1 -- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6498003bc0 msgr2=0x7f6498003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64af7fe700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6498003bc0 0x7f6498003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64af7fe700 1 -- 172.21.15.45:0/3476452856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8009a70 con 0x7f6498004d90 2023-08-27T11:24:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64af7fe700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6498004d90 0x7f6498005210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f64b006d570 tx=0x7f6498005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a089ddccf45c32e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a800ed30 con 0x7f6498004d90 2023-08-27T11:24:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a800f040 con 0x7f6498004d90 2023-08-27T11:24:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64adffb700 1 -- 172.21.15.45:0/3476452856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80146a0 con 0x7f6498004d90 2023-08-27T11:24:49.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64b5270700 1 -- 172.21.15.45:0/3476452856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6498004d90 msgr2=0x7f6498005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64b5270700 1 --2- 172.21.15.45:0/3476452856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6498004d90 0x7f6498005210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f64b006d570 tx=0x7f6498005880 comp rx=0 tx=0).stop 2023-08-27T11:24:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.067+0000 7f64af7fe700 1 -- 172.21.15.45:0/3476452856 reap_dead start 2023-08-27T11:24:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.068+0000 7f64b5270700 1 -- 172.21.15.45:0/3476452856 shutdown_connections 2023-08-27T11:24:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.068+0000 7f64b5270700 1 -- 172.21.15.45:0/3476452856 >> 172.21.15.45:0/3476452856 conn(0x7f64b00d2fd0 msgr2=0x7f64b00d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.068+0000 7f64b5270700 1 -- 172.21.15.45:0/3476452856 shutdown_connections 2023-08-27T11:24:49.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.068+0000 7f64b5270700 1 -- 172.21.15.45:0/3476452856 wait complete. 2023-08-27T11:24:49.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.069+0000 7f64b5270700 1 Processor -- start 2023-08-27T11:24:49.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.069+0000 7f64b5270700 1 -- start start 2023-08-27T11:24:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64b5270700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00c9490 0x7f64b015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64b5270700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64b015ffc0 0x7f64b00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64b5270700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b00ca0a0 0x7f64b00ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64b5270700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b00caa70 con 0x7f64b00ca0a0 2023-08-27T11:24:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64b5270700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b00cd070 con 0x7f64b015ffc0 2023-08-27T11:24:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64b5270700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b01f2b10 con 0x7f64b00c9490 2023-08-27T11:24:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b00ca0a0 0x7f64b00ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64af7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b00ca0a0 0x7f64b00ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55728/0 (socket says 172.21.15.45:55728) 2023-08-27T11:24:49.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64af7fe700 1 -- 172.21.15.45:0/1625845619 learned_addr learned my addr 172.21.15.45:0/1625845619 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:49.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64af7fe700 1 -- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00c9490 msgr2=0x7f64b015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64aeffd700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00c9490 0x7f64b015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64af7fe700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00c9490 0x7f64b015e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64ae7fc700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64b015ffc0 0x7f64b00ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64af7fe700 1 -- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64b015ffc0 msgr2=0x7f64b00ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64af7fe700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64b015ffc0 0x7f64b00ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64af7fe700 1 -- 172.21.15.45:0/1625845619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b015ef20 con 0x7f64b00ca0a0 2023-08-27T11:24:49.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64af7fe700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b00ca0a0 0x7f64b00ca530 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f64a8002230 tx=0x7f64a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:49.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.070+0000 7f64977fe700 1 -- 172.21.15.45:0/1625845619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8009410 con 0x7f64b00ca0a0 2023-08-27T11:24:49.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.071+0000 7f64977fe700 1 -- 172.21.15.45:0/1625845619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a8022070 con 0x7f64b00ca0a0 2023-08-27T11:24:49.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.071+0000 7f64977fe700 1 -- 172.21.15.45:0/1625845619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80296c0 con 0x7f64b00ca0a0 2023-08-27T11:24:49.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.071+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b015f200 con 0x7f64b00ca0a0 2023-08-27T11:24:49.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.071+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b015f7c0 con 0x7f64b00ca0a0 2023-08-27T11:24:49.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.072+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b004e620 con 0x7f64b00ca0a0 2023-08-27T11:24:49.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.072+0000 7f64977fe700 1 -- 172.21.15.45:0/1625845619 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f64a801e430 con 0x7f64b00ca0a0 2023-08-27T11:24:49.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.072+0000 7f64977fe700 1 -- 172.21.15.45:0/1625845619 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7593+0+0 (secure 0 0 0) 0x7f64a8024070 con 0x7f64b00ca0a0 2023-08-27T11:24:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.077+0000 7f64977fe700 1 -- 172.21.15.45:0/1625845619 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f64a80554f0 con 0x7f64b00ca0a0 2023-08-27T11:24:49.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.318+0000 7f64977fe700 1 -- 172.21.15.45:0/1625845619 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 16) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f64a8009ca0 con 0x7f64b00ca0a0 2023-08-27T11:24:49.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.319+0000 7f64977fe700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f649c070a00 0x7f649c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.321+0000 7f64aeffd700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f649c070a00 0x7f649c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.325+0000 7f64aeffd700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f649c070a00 0x7f649c072df0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64a4007fd0 tx=0x7f64a4009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.329+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64b00c68a0 con 0x7f64b00ca0a0 2023-08-27T11:24:49.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.329+0000 7f64977fe700 1 -- 172.21.15.45:0/1625845619 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f64a806f810 con 0x7f64b00ca0a0 2023-08-27T11:24:49.331 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:49.331 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1324661970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1324661970},{"type":"v1","addr":"172.21.15.135:6836","nonce":1324661970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2014390806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2014390806},{"type":"v1","addr":"172.21.15.135:6837","nonce":2014390806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2103056681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2103056681},{"type":"v1","addr":"172.21.15.45:6834","nonce":2103056681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:13:37.320852+0000","modified":"2023-08-27T11:13:40.917661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/2587151358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2587151358},{"type":"v1","addr":"172.21.15.45:6835","nonce":2587151358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:24:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f649c070a00 msgr2=0x7f649c072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f649c070a00 0x7f649c072df0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64a4007fd0 tx=0x7f64a4009040 comp rx=0 tx=0).stop 2023-08-27T11:24:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b00ca0a0 msgr2=0x7f64b00ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b00ca0a0 0x7f64b00ca530 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f64a8002230 tx=0x7f64a8009750 comp rx=0 tx=0).stop 2023-08-27T11:24:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 shutdown_connections 2023-08-27T11:24:49.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f649c070a00 0x7f649c072df0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64b00ca0a0 0x7f64b00ca530 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f64b00c9490 0x7f64b015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 --2- 172.21.15.45:0/1625845619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64b015ffc0 0x7f64b00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 >> 172.21.15.45:0/1625845619 conn(0x7f64b00d2fd0 msgr2=0x7f64b00b6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 shutdown_connections 2023-08-27T11:24:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.331+0000 7f64b5270700 1 -- 172.21.15.45:0/1625845619 wait complete. 2023-08-27T11:24:49.336 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2023-08-27T11:24:49.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:24:49.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.473+0000 7f97fcdd8700 1 Processor -- start 2023-08-27T11:24:49.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.473+0000 7f97fcdd8700 1 -- start start 2023-08-27T11:24:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.474+0000 7f97fcdd8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97f815d450 0x7f97f80cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.474+0000 7f97fcdd8700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f97f80d0440 0x7f97f8161250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.474+0000 7f97fcdd8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80ce1c0 con 0x7f97f80cbaf0 2023-08-27T11:24:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.474+0000 7f97fcdd8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80cf290 con 0x7f97f815d450 2023-08-27T11:24:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.474+0000 7f97fcdd8700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80ce440 con 0x7f97f80d0440 2023-08-27T11:24:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.474+0000 7f97f659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97f815d450 0x7f97f80cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.474+0000 7f97f5d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f97f80cbaf0 0x7f97f80cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42522/0 (socket says 172.21.15.45:42522) 2023-08-27T11:24:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.474+0000 7f97f5d9b700 1 -- 172.21.15.45:0/2515580050 learned_addr learned my addr 172.21.15.45:0/2515580050 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.474+0000 7f97f6d9d700 1 --2- 172.21.15.45:0/2515580050 >> v2:172.21.15.135:3300/0 conn(0x7f97f80d0440 0x7f97f8161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97f659c700 1 -- 172.21.15.45:0/2515580050 >> v2:172.21.15.135:3300/0 conn(0x7f97f80d0440 msgr2=0x7f97f8161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:49.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97f659c700 1 --2- 172.21.15.45:0/2515580050 >> v2:172.21.15.135:3300/0 conn(0x7f97f80d0440 0x7f97f8161250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97f659c700 1 -- 172.21.15.45:0/2515580050 >> v1:172.21.15.45:6789/0 conn(0x7f97f80cbaf0 legacy=0x7f97f80cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97f659c700 1 -- 172.21.15.45:0/2515580050 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80ce720 con 0x7f97f815d450 2023-08-27T11:24:49.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97f559a700 1 -- 172.21.15.45:0/2515580050 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 961896880 0 0) 0x7f97f80ce1c0 con 0x7f97f80cbaf0 2023-08-27T11:24:49.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97f659c700 1 --2- 172.21.15.45:0/2515580050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 0x7f97f80cb5b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f97f80d0f30 tx=0x7f97ec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9593adb2ce9533b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:49.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97f559a700 1 -- 172.21.15.45:0/2515580050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec011830 con 0x7f97f815d450 2023-08-27T11:24:49.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97f559a700 1 -- 172.21.15.45:0/2515580050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec011990 con 0x7f97f815d450 2023-08-27T11:24:49.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97f559a700 1 -- 172.21.15.45:0/2515580050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec011d20 con 0x7f97f815d450 2023-08-27T11:24:49.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97fcdd8700 1 -- 172.21.15.45:0/2515580050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 msgr2=0x7f97f80cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.475+0000 7f97fcdd8700 1 --2- 172.21.15.45:0/2515580050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 0x7f97f80cb5b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f97f80d0f30 tx=0x7f97ec009b20 comp rx=0 tx=0).stop 2023-08-27T11:24:49.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.476+0000 7f97fcdd8700 1 -- 172.21.15.45:0/2515580050 shutdown_connections 2023-08-27T11:24:49.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.476+0000 7f97fcdd8700 1 --2- 172.21.15.45:0/2515580050 >> v2:172.21.15.135:3300/0 conn(0x7f97f80d0440 0x7f97f8161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.476+0000 7f97fcdd8700 1 --2- 172.21.15.45:0/2515580050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 0x7f97f80cb5b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.476+0000 7f97fcdd8700 1 -- 172.21.15.45:0/2515580050 >> 172.21.15.45:0/2515580050 conn(0x7f97f80d0f60 msgr2=0x7f97f80d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:49.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.476+0000 7f97fcdd8700 1 -- 172.21.15.45:0/2515580050 shutdown_connections 2023-08-27T11:24:49.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.476+0000 7f97fcdd8700 1 -- 172.21.15.45:0/2515580050 wait complete. 2023-08-27T11:24:49.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.477+0000 7f97fcdd8700 1 Processor -- start 2023-08-27T11:24:49.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.477+0000 7f97fcdd8700 1 -- start start 2023-08-27T11:24:49.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97fcdd8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97f80cbaf0 0x7f97f8160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97fcdd8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f97f80d0440 0x7f97f80c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97fcdd8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 0x7f97f814ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97fcdd8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80c6020 con 0x7f97f815d450 2023-08-27T11:24:49.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97fcdd8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f814b1d0 con 0x7f97f80cbaf0 2023-08-27T11:24:49.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97fcdd8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80c62c0 con 0x7f97f80d0440 2023-08-27T11:24:49.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 0x7f97f814ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 0x7f97f814ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55756/0 (socket says 172.21.15.45:55756) 2023-08-27T11:24:49.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f6d9d700 1 -- 172.21.15.45:0/3980506895 learned_addr learned my addr 172.21.15.45:0/3980506895 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:49.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f659c700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97f80cbaf0 0x7f97f8160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f5d9b700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f97f80d0440 0x7f97f80c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f6d9d700 1 -- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f97f80d0440 msgr2=0x7f97f80c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f6d9d700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f97f80d0440 0x7f97f80c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f6d9d700 1 -- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97f80cbaf0 msgr2=0x7f97f8160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f6d9d700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97f80cbaf0 0x7f97f8160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f6d9d700 1 -- 172.21.15.45:0/3980506895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0094a0 con 0x7f97f815d450 2023-08-27T11:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97f6d9d700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 0x7f97f814ac90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f97f0007de0 tx=0x7f97f000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97e6ffd700 1 -- 172.21.15.45:0/3980506895 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000ed30 con 0x7f97f815d450 2023-08-27T11:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.478+0000 7f97e6ffd700 1 -- 172.21.15.45:0/3980506895 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0014420 con 0x7f97f815d450 2023-08-27T11:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.479+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f80c65a0 con 0x7f97f815d450 2023-08-27T11:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.479+0000 7f97e6ffd700 1 -- 172.21.15.45:0/3980506895 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00147d0 con 0x7f97f815d450 2023-08-27T11:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.479+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f80c6b60 con 0x7f97f815d450 2023-08-27T11:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.480+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f809ee50 con 0x7f97f815d450 2023-08-27T11:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.480+0000 7f97e6ffd700 1 -- 172.21.15.45:0/3980506895 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f97f0014930 con 0x7f97f815d450 2023-08-27T11:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.480+0000 7f97e6ffd700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f97d8069f30 0x7f97d806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.480+0000 7f97e6ffd700 1 -- 172.21.15.45:0/3980506895 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7653+0+0 (secure 0 0 0) 0x7f97f0088f10 con 0x7f97f815d450 2023-08-27T11:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.480+0000 7f97f659c700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f97d8069f30 0x7f97d806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.481+0000 7f97f659c700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f97d8069f30 0x7f97d806c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f97ec000c00 tx=0x7f97ec01c040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.485+0000 7f97e6ffd700 1 -- 172.21.15.45:0/3980506895 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f97f0054300 con 0x7f97f815d450 2023-08-27T11:24:49.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.727+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97f80b6ee0 con 0x7f97f815d450 2023-08-27T11:24:49.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.728+0000 7f97e6ffd700 1 -- 172.21.15.45:0/3980506895 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f97f0057920 con 0x7f97f815d450 2023-08-27T11:24:49.730 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:49.730 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1324661970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1324661970},{"type":"v1","addr":"172.21.15.135:6836","nonce":1324661970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2014390806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2014390806},{"type":"v1","addr":"172.21.15.135:6837","nonce":2014390806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2103056681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2103056681},{"type":"v1","addr":"172.21.15.45:6834","nonce":2103056681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:13:37.320852+0000","modified":"2023-08-27T11:13:40.917661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/2587151358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2587151358},{"type":"v1","addr":"172.21.15.45:6835","nonce":2587151358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-27T11:24:49.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f97d8069f30 msgr2=0x7f97d806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f97d8069f30 0x7f97d806c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f97ec000c00 tx=0x7f97ec01c040 comp rx=0 tx=0).stop 2023-08-27T11:24:49.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 msgr2=0x7f97f814ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 0x7f97f814ac90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f97f0007de0 tx=0x7f97f000e3c0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 shutdown_connections 2023-08-27T11:24:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f97d8069f30 0x7f97d806c3f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97f815d450 0x7f97f814ac90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97f80cbaf0 0x7f97f8160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 --2- 172.21.15.45:0/3980506895 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f97f80d0440 0x7f97f80c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 >> 172.21.15.45:0/3980506895 conn(0x7f97f80d0f60 msgr2=0x7f97f80cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:49.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 shutdown_connections 2023-08-27T11:24:49.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.730+0000 7f97fcdd8700 1 -- 172.21.15.45:0/3980506895 wait complete. 2023-08-27T11:24:49.735 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2023-08-27T11:24:49.744 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:24:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.869+0000 7fa09af9f700 1 Processor -- start 2023-08-27T11:24:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.869+0000 7fa09af9f700 1 -- start start 2023-08-27T11:24:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.870+0000 7fa09af9f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa09415a240 0x7fa0940cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.870+0000 7fa09af9f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0940cbe70 0x7fa0940ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.870+0000 7fa09af9f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940ce9d0 0x7fa094070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.870+0000 7fa09af9f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09406d690 con 0x7fa0940cbe70 2023-08-27T11:24:49.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.870+0000 7fa09af9f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0940cee20 con 0x7fa09415a240 2023-08-27T11:24:49.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.870+0000 7fa09af9f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940cc440 con 0x7fa0940ce9d0 2023-08-27T11:24:49.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.870+0000 7fa093fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0940cbe70 0x7fa0940ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa093fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0940cbe70 0x7fa0940ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55764/0 (socket says 172.21.15.45:55764) 2023-08-27T11:24:49.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa093fff700 1 -- 172.21.15.45:0/4056992185 learned_addr learned my addr 172.21.15.45:0/4056992185 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa098d3b700 1 --2- 172.21.15.45:0/4056992185 >> v2:172.21.15.135:3300/0 conn(0x7fa09415a240 0x7fa0940cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa09953c700 1 --2- 172.21.15.45:0/4056992185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940ce9d0 0x7fa094070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa093fff700 1 -- 172.21.15.45:0/4056992185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940ce9d0 msgr2=0x7fa094070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa093fff700 1 --2- 172.21.15.45:0/4056992185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940ce9d0 0x7fa094070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa093fff700 1 -- 172.21.15.45:0/4056992185 >> v2:172.21.15.135:3300/0 conn(0x7fa09415a240 msgr2=0x7fa0940cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa093fff700 1 --2- 172.21.15.45:0/4056992185 >> v2:172.21.15.135:3300/0 conn(0x7fa09415a240 0x7fa0940cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa093fff700 1 -- 172.21.15.45:0/4056992185 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09406d040 con 0x7fa0940cbe70 2023-08-27T11:24:49.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa09953c700 1 --2- 172.21.15.45:0/4056992185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940ce9d0 0x7fa094070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:49.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa093fff700 1 --2- 172.21.15.45:0/4056992185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940cbe70 0x7fa0940ce490 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa0940b2950 tx=0x7fa080009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f252be6e63636f6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:49.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa0937fe700 1 -- 172.21.15.45:0/4056992185 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080011830 con 0x7fa0940cbe70 2023-08-27T11:24:49.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa0937fe700 1 -- 172.21.15.45:0/4056992185 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa080011990 con 0x7fa0940cbe70 2023-08-27T11:24:49.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.871+0000 7fa0937fe700 1 -- 172.21.15.45:0/4056992185 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080011d20 con 0x7fa0940cbe70 2023-08-27T11:24:49.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.872+0000 7fa09af9f700 1 -- 172.21.15.45:0/4056992185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940cbe70 msgr2=0x7fa0940ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.872+0000 7fa09af9f700 1 --2- 172.21.15.45:0/4056992185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940cbe70 0x7fa0940ce490 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa0940b2950 tx=0x7fa080009bd0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.872+0000 7fa09af9f700 1 -- 172.21.15.45:0/4056992185 shutdown_connections 2023-08-27T11:24:49.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.872+0000 7fa09af9f700 1 --2- 172.21.15.45:0/4056992185 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940ce9d0 0x7fa094070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.872+0000 7fa09af9f700 1 --2- 172.21.15.45:0/4056992185 >> v2:172.21.15.135:3300/0 conn(0x7fa09415a240 0x7fa0940cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.872+0000 7fa09af9f700 1 --2- 172.21.15.45:0/4056992185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940cbe70 0x7fa0940ce490 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.872+0000 7fa09af9f700 1 -- 172.21.15.45:0/4056992185 >> 172.21.15.45:0/4056992185 conn(0x7fa0940cfe50 msgr2=0x7fa0940d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:49.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.872+0000 7fa09af9f700 1 -- 172.21.15.45:0/4056992185 shutdown_connections 2023-08-27T11:24:49.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.872+0000 7fa09af9f700 1 -- 172.21.15.45:0/4056992185 wait complete. 2023-08-27T11:24:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.873+0000 7fa09af9f700 1 Processor -- start 2023-08-27T11:24:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.873+0000 7fa09af9f700 1 -- start start 2023-08-27T11:24:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa09af9f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940cbe70 0x7fa09414f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa09af9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940ce9d0 0x7fa0940ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa09af9f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa09415a240 0x7fa0940c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa09af9f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09414f9b0 con 0x7fa0940ce9d0 2023-08-27T11:24:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa09af9f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940ae3d0 con 0x7fa09415a240 2023-08-27T11:24:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa09af9f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940ae6b0 con 0x7fa0940cbe70 2023-08-27T11:24:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa093fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940ce9d0 0x7fa0940ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa093fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940ce9d0 0x7fa0940ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55770/0 (socket says 172.21.15.45:55770) 2023-08-27T11:24:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa093fff700 1 -- 172.21.15.45:0/2291870919 learned_addr learned my addr 172.21.15.45:0/2291870919 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa09953c700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa09415a240 0x7fa0940c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa098d3b700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940cbe70 0x7fa09414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa093fff700 1 -- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940cbe70 msgr2=0x7fa09414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa093fff700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940cbe70 0x7fa09414f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa093fff700 1 -- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa09415a240 msgr2=0x7fa0940c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:49.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa093fff700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa09415a240 0x7fa0940c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:49.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa093fff700 1 -- 172.21.15.45:0/2291870919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa080009550 con 0x7fa0940ce9d0 2023-08-27T11:24:49.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa09953c700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa09415a240 0x7fa0940c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:49.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.874+0000 7fa093fff700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940ce9d0 0x7fa0940ada80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa080000c00 tx=0x7fa080004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:49.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.875+0000 7fa0917fa700 1 -- 172.21.15.45:0/2291870919 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080004300 con 0x7fa0940ce9d0 2023-08-27T11:24:49.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.875+0000 7fa0917fa700 1 -- 172.21.15.45:0/2291870919 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08001f070 con 0x7fa0940ce9d0 2023-08-27T11:24:49.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.875+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0940ae990 con 0x7fa0940ce9d0 2023-08-27T11:24:49.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.875+0000 7fa0917fa700 1 -- 172.21.15.45:0/2291870919 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0800036a0 con 0x7fa0940ce9d0 2023-08-27T11:24:49.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.875+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0940aef50 con 0x7fa0940ce9d0 2023-08-27T11:24:49.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.876+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa094072810 con 0x7fa0940ce9d0 2023-08-27T11:24:49.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.876+0000 7fa0917fa700 1 -- 172.21.15.45:0/2291870919 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fa080021030 con 0x7fa0940ce9d0 2023-08-27T11:24:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.876+0000 7fa0917fa700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa078069f30 0x7fa07806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.877+0000 7fa0917fa700 1 -- 172.21.15.45:0/2291870919 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7653+0+0 (secure 0 0 0) 0x7fa080089a80 con 0x7fa0940ce9d0 2023-08-27T11:24:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.877+0000 7fa098d3b700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa078069f30 0x7fa07806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:49.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.877+0000 7fa098d3b700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa078069f30 0x7fa07806c3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa088009b40 tx=0x7fa088009210 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:49.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:49.881+0000 7fa0917fa700 1 -- 172.21.15.45:0/2291870919 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa080054e70 con 0x7fa0940ce9d0 2023-08-27T11:24:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.085+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0940734a0 con 0x7fa0940ce9d0 2023-08-27T11:24:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.087+0000 7fa0917fa700 1 -- 172.21.15.45:0/2291870919 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+20445 (secure 0 0 0) 0x7fa080058490 con 0x7fa0940ce9d0 2023-08-27T11:24:50.089 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:50.089 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":58,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:24:49.304673+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:13:35.610850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-27T11:13:36.282109+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":219,"snap_epoch":58,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2c","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:6833/584312721":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/3750381439":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/459759971":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/1050128524":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/2208777569":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/918212180":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/2898860171":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1692978501":"2023-08-28T11:13:52.047816+0000","172.21.15.135:6832/584312721":"2023-08-28T11:24:48.301631+0000","172.21.15.135:6832/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1902421107":"2023-08-28T11:24:48.301631+0000","172.21.15.135:6833/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1110669507":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/4257866035":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/2587284130":"2023-08-28T11:13:52.047816+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":4,"length":2},{"begin":8,"length":2},{"begin":12,"length":1},{"begin":14,"length":2},{"begin":18,"length":2},{"begin":23,"length":1},{"begin":25,"length":2},{"begin":29,"length":2},{"begin":33,"length":1},{"begin":35,"length":2},{"begin":39,"length":3},{"begin":44,"length":1},{"begin":46,"length":2},{"begin":50,"length":2},{"begin":54,"length":1},{"begin":56,"length":2},{"begin":60,"length":2},{"begin":64,"length":1},{"begin":66,"length":2},{"begin":70,"length":3},{"begin":75,"length":1},{"begin":77,"length":2},{"begin":81,"length":2},{"begin":85,"length":1},{"begin":87,"length":2},{"begin":92,"length":2},{"begin":96,"length":1},{"begin":98,"length":2},{"begin":102,"length":3},{"begin":106,"length":1},{"begin":108,"length":2},{"begin":165,"length":55}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:24:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa078069f30 msgr2=0x7fa07806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa078069f30 0x7fa07806c3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa088009b40 tx=0x7fa088009210 comp rx=0 tx=0).stop 2023-08-27T11:24:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940ce9d0 msgr2=0x7fa0940ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940ce9d0 0x7fa0940ada80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa080000c00 tx=0x7fa080004750 comp rx=0 tx=0).stop 2023-08-27T11:24:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 shutdown_connections 2023-08-27T11:24:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa078069f30 0x7fa07806c3f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa09415a240 0x7fa0940c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0940cbe70 0x7fa09414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 --2- 172.21.15.45:0/2291870919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0940ce9d0 0x7fa0940ada80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 >> 172.21.15.45:0/2291870919 conn(0x7fa0940cfe50 msgr2=0x7fa0940b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:50.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 shutdown_connections 2023-08-27T11:24:50.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.089+0000 7fa09af9f700 1 -- 172.21.15.45:0/2291870919 wait complete. 2023-08-27T11:24:50.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:24:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.228+0000 7fbe16ca2700 1 Processor -- start 2023-08-27T11:24:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.228+0000 7fbe16ca2700 1 -- start start 2023-08-27T11:24:50.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe16ca2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe10072080 0x7fbe10165100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe16ca2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe100b3220 con 0x7fbe100aea00 2023-08-27T11:24:50.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe16ca2700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe100b2c60 con 0x7fbe1015a220 2023-08-27T11:24:50.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe16ca2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe100b2f40 con 0x7fbe10072080 2023-08-27T11:24:50.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbe100aea00 0x7fbe10071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42552/0 (socket says 172.21.15.45:42552) 2023-08-27T11:24:50.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0ffff700 1 -- 172.21.15.45:0/1270466902 learned_addr learned my addr 172.21.15.45:0/1270466902 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:50.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535814625 0 0) 0x7fbe100b3220 con 0x7fbe100aea00 2023-08-27T11:24:50.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe1523f700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe10072080 0x7fbe10165100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe000035b0 con 0x7fbe100aea00 2023-08-27T11:24:50.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3158466745 0 0) 0x7fbe000035b0 con 0x7fbe100aea00 2023-08-27T11:24:50.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe10072080 msgr2=0x7fbe10165100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:50.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe10072080 0x7fbe10165100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 >> v1:172.21.15.135:6789/0 conn(0x7fbe1015a220 legacy=0x7fbe100cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe100cec70 con 0x7fbe100aea00 2023-08-27T11:24:50.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fbdfc001670 con 0x7fbe100aea00 2023-08-27T11:24:50.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 >> v1:172.21.15.45:6789/0 conn(0x7fbe100aea00 legacy=0x7fbe10071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe00003a30 0x7fbe00003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.229+0000 7fbe0f7fe700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe10072080 0x7fbe00004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe0f7fe700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00004d90 0x7fbe00005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe100b2f40 con 0x7fbe00003a30 2023-08-27T11:24:50.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0000ba40 con 0x7fbe00004d90 2023-08-27T11:24:50.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0000bc00 con 0x7fbe10072080 2023-08-27T11:24:50.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe14a3e700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe00003a30 0x7fbe00003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe0ffff700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe10072080 0x7fbe00004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe1523f700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00004d90 0x7fbe00005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe14a3e700 1 -- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe10072080 msgr2=0x7fbe00004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:50.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe14a3e700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe10072080 0x7fbe00004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe14a3e700 1 -- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00004d90 msgr2=0x7fbe00005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:50.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe14a3e700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00004d90 0x7fbe00005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe14a3e700 1 -- 172.21.15.45:0/1270466902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000035b0 con 0x7fbe00003a30 2023-08-27T11:24:50.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe14a3e700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe00003a30 0x7fbe00003e10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbe04001490 tx=0x7fbe00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f185595d8d4a7767 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:50.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe04011670 con 0x7fbe00003a30 2023-08-27T11:24:50.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe040117f0 con 0x7fbe00003a30 2023-08-27T11:24:50.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.230+0000 7fbe0f7fe700 1 -- 172.21.15.45:0/1270466902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe04011ba0 con 0x7fbe00003a30 2023-08-27T11:24:50.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.231+0000 7fbe16ca2700 1 -- 172.21.15.45:0/1270466902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe00003a30 msgr2=0x7fbe00003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.231+0000 7fbe16ca2700 1 --2- 172.21.15.45:0/1270466902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe00003a30 0x7fbe00003e10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbe04001490 tx=0x7fbe00005880 comp rx=0 tx=0).stop 2023-08-27T11:24:50.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.231+0000 7fbe1523f700 1 -- 172.21.15.45:0/1270466902 reap_dead start 2023-08-27T11:24:50.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.231+0000 7fbe16ca2700 1 -- 172.21.15.45:0/1270466902 shutdown_connections 2023-08-27T11:24:50.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.231+0000 7fbe16ca2700 1 -- 172.21.15.45:0/1270466902 >> 172.21.15.45:0/1270466902 conn(0x7fbe100b4650 msgr2=0x7fbe100b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:50.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.231+0000 7fbe16ca2700 1 -- 172.21.15.45:0/1270466902 shutdown_connections 2023-08-27T11:24:50.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.231+0000 7fbe16ca2700 1 -- 172.21.15.45:0/1270466902 wait complete. 2023-08-27T11:24:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.232+0000 7fbe16ca2700 1 Processor -- start 2023-08-27T11:24:50.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.233+0000 7fbe16ca2700 1 -- start start 2023-08-27T11:24:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe16ca2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00003a30 0x7fbe100ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe16ca2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe1006c9d0 0x7fbe100ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe16ca2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10098ba0 0x7fbe1015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe16ca2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1015b150 con 0x7fbe10098ba0 2023-08-27T11:24:50.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe16ca2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe100b1b30 con 0x7fbe00003a30 2023-08-27T11:24:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe16ca2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe100c5450 con 0x7fbe1006c9d0 2023-08-27T11:24:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe1523f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10098ba0 0x7fbe1015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe1523f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10098ba0 0x7fbe1015b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55792/0 (socket says 172.21.15.45:55792) 2023-08-27T11:24:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe1523f700 1 -- 172.21.15.45:0/148970716 learned_addr learned my addr 172.21.15.45:0/148970716 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe14a3e700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00003a30 0x7fbe100ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe0ffff700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe1006c9d0 0x7fbe100ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe1523f700 1 -- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe1006c9d0 msgr2=0x7fbe100ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe1523f700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe1006c9d0 0x7fbe100ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe1523f700 1 -- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00003a30 msgr2=0x7fbe100ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe1523f700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00003a30 0x7fbe100ac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe1523f700 1 -- 172.21.15.45:0/148970716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1015a820 con 0x7fbe10098ba0 2023-08-27T11:24:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe14a3e700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00003a30 0x7fbe100ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe1523f700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10098ba0 0x7fbe1015b5e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbe1006ff10 tx=0x7fbe08011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe0d7fa700 1 -- 172.21.15.45:0/148970716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800e040 con 0x7fbe10098ba0 2023-08-27T11:24:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe0d7fa700 1 -- 172.21.15.45:0/148970716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0800f040 con 0x7fbe10098ba0 2023-08-27T11:24:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1015ab00 con 0x7fbe10098ba0 2023-08-27T11:24:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe0d7fa700 1 -- 172.21.15.45:0/148970716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800fdb0 con 0x7fbe10098ba0 2023-08-27T11:24:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.234+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1015bb20 con 0x7fbe10098ba0 2023-08-27T11:24:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.235+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe100ccf10 con 0x7fbe10098ba0 2023-08-27T11:24:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.236+0000 7fbe0d7fa700 1 -- 172.21.15.45:0/148970716 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fbe08015430 con 0x7fbe10098ba0 2023-08-27T11:24:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.236+0000 7fbe0d7fa700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbdf4069f30 0x7fbdf406c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.236+0000 7fbe0d7fa700 1 -- 172.21.15.45:0/148970716 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7653+0+0 (secure 0 0 0) 0x7fbe08088d60 con 0x7fbe10098ba0 2023-08-27T11:24:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.236+0000 7fbe14a3e700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbdf4069f30 0x7fbdf406c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.237+0000 7fbe14a3e700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbdf4069f30 0x7fbdf406c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe10155ad0 tx=0x7fbe04013000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.241+0000 7fbe0d7fa700 1 -- 172.21.15.45:0/148970716 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbe08054080 con 0x7fbe10098ba0 2023-08-27T11:24:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.486+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbe10071ea0 con 0x7fbe10098ba0 2023-08-27T11:24:50.489 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-08-27T11:24:50.487+0000 7f87a1c48700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:24:50.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.491+0000 7fbe0d7fa700 1 -- 172.21.15.45:0/148970716 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbe080576a0 con 0x7fbe10098ba0 2023-08-27T11:24:50.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbdf4069f30 msgr2=0x7fbdf406c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbdf4069f30 0x7fbdf406c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe10155ad0 tx=0x7fbe04013000 comp rx=0 tx=0).stop 2023-08-27T11:24:50.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10098ba0 msgr2=0x7fbe1015b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10098ba0 0x7fbe1015b5e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbe1006ff10 tx=0x7fbe08011f80 comp rx=0 tx=0).stop 2023-08-27T11:24:50.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 shutdown_connections 2023-08-27T11:24:50.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbdf4069f30 0x7fbdf406c3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10098ba0 0x7fbe1015b5e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe00003a30 0x7fbe100ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 --2- 172.21.15.45:0/148970716 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe1006c9d0 0x7fbe100ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 >> 172.21.15.45:0/148970716 conn(0x7fbe100b4650 msgr2=0x7fbe1006d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:50.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 shutdown_connections 2023-08-27T11:24:50.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.494+0000 7fbe16ca2700 1 -- 172.21.15.45:0/148970716 wait complete. 2023-08-27T11:24:50.499 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:24:50.508 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:24:50.530 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-08-27T11:24:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.636+0000 7fdc313dc700 1 Processor -- start 2023-08-27T11:24:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.636+0000 7fdc313dc700 1 -- start start 2023-08-27T11:24:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc313dc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc2c15d2c0 0x7fdc2c0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc313dc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc313dc700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c15d6a0 con 0x7fdc2c0b8c20 2023-08-27T11:24:50.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc313dc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c0d2c90 con 0x7fdc2c15d2c0 2023-08-27T11:24:50.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc313dc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0d1bf0 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc2affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc2c15d2c0 0x7fdc2c0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc2affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdc2c15d2c0 0x7fdc2c0d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55808/0 (socket says 172.21.15.45:55808) 2023-08-27T11:24:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc2affd700 1 -- 172.21.15.45:0/3862625651 learned_addr learned my addr 172.21.15.45:0/3862625651 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc2b7fe700 1 --2- 172.21.15.45:0/3862625651 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc2affd700 1 -- 172.21.15.45:0/3862625651 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c0ce4b0 msgr2=0x7fdc2c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc2affd700 1 --2- 172.21.15.45:0/3862625651 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc2affd700 1 -- 172.21.15.45:0/3862625651 >> v1:172.21.15.135:6789/0 conn(0x7fdc2c0b8c20 legacy=0x7fdc2c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc2affd700 1 -- 172.21.15.45:0/3862625651 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0d1ed0 con 0x7fdc2c15d2c0 2023-08-27T11:24:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.637+0000 7fdc2affd700 1 --2- 172.21.15.45:0/3862625651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c15d2c0 0x7fdc2c0d13a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009920 tx=0x7fdc20009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaad6e6d95a6b06a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc29ffb700 1 -- 172.21.15.45:0/3862625651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20011830 con 0x7fdc2c15d2c0 2023-08-27T11:24:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc29ffb700 1 -- 172.21.15.45:0/3862625651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc20011990 con 0x7fdc2c15d2c0 2023-08-27T11:24:50.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc29ffb700 1 -- 172.21.15.45:0/3862625651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20011d20 con 0x7fdc2c15d2c0 2023-08-27T11:24:50.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc313dc700 1 -- 172.21.15.45:0/3862625651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c15d2c0 msgr2=0x7fdc2c0d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc313dc700 1 --2- 172.21.15.45:0/3862625651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c15d2c0 0x7fdc2c0d13a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009920 tx=0x7fdc20009c30 comp rx=0 tx=0).stop 2023-08-27T11:24:50.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc313dc700 1 -- 172.21.15.45:0/3862625651 shutdown_connections 2023-08-27T11:24:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc313dc700 1 --2- 172.21.15.45:0/3862625651 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc313dc700 1 --2- 172.21.15.45:0/3862625651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c15d2c0 0x7fdc2c0d13a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc313dc700 1 -- 172.21.15.45:0/3862625651 >> 172.21.15.45:0/3862625651 conn(0x7fdc2c0d2fd0 msgr2=0x7fdc2c0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc313dc700 1 -- 172.21.15.45:0/3862625651 shutdown_connections 2023-08-27T11:24:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.638+0000 7fdc313dc700 1 -- 172.21.15.45:0/3862625651 wait complete. 2023-08-27T11:24:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.639+0000 7fdc313dc700 1 Processor -- start 2023-08-27T11:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.639+0000 7fdc313dc700 1 -- start start 2023-08-27T11:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc313dc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc2c0b8c20 0x7fdc2c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc313dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc313dc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c15d2c0 0x7fdc2c1f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc313dc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b59e0 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc313dc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b4d00 con 0x7fdc2c0b8c20 2023-08-27T11:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc313dc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0cc8d0 con 0x7fdc2c15d2c0 2023-08-27T11:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc2a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc2a7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55816/0 (socket says 172.21.15.45:55816) 2023-08-27T11:24:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc2a7fc700 1 -- 172.21.15.45:0/1069043973 learned_addr learned my addr 172.21.15.45:0/1069043973 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc2affd700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc2c0b8c20 0x7fdc2c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.640+0000 7fdc2b7fe700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c15d2c0 0x7fdc2c1f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc2a7fc700 1 -- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c15d2c0 msgr2=0x7fdc2c1f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc2a7fc700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c15d2c0 0x7fdc2c1f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc2a7fc700 1 -- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc2c0b8c20 msgr2=0x7fdc2c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc2a7fc700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc2c0b8c20 0x7fdc2c0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc2a7fc700 1 -- 172.21.15.45:0/1069043973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc20009550 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc2a7fc700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b54a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c4e0 tx=0x7fdc1c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc137fe700 1 -- 172.21.15.45:0/1069043973 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0052d0 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc137fe700 1 -- 172.21.15.45:0/1069043973 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c005430 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c0ccbb0 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc137fe700 1 -- 172.21.15.45:0/1069043973 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0038c0 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.641+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c0b4f80 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.642+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c04e620 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.642+0000 7fdc137fe700 1 -- 172.21.15.45:0/1069043973 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fdc1c003a20 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.643+0000 7fdc137fe700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdc18069f30 0x7fdc1806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:50.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.643+0000 7fdc137fe700 1 -- 172.21.15.45:0/1069043973 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7695+0+0 (secure 0 0 0) 0x7fdc1c087aa0 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.643+0000 7fdc2affd700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdc18069f30 0x7fdc1806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:50.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.643+0000 7fdc2affd700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdc18069f30 0x7fdc1806c3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdc20014040 tx=0x7fdc20008030 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.647+0000 7fdc137fe700 1 -- 172.21.15.45:0/1069043973 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdc1c052e70 con 0x7fdc2c0ce4b0 2023-08-27T11:24:50.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:50.891+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdc2c15d6a0 con 0x7fdc2c0ce4b0 2023-08-27T11:24:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.311+0000 7fdc137fe700 1 -- 172.21.15.45:0/1069043973 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 17) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fdc1c053240 con 0x7fdc2c0ce4b0 2023-08-27T11:24:51.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.492+0000 7fdc137fe700 1 -- 172.21.15.45:0/1069043973 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdc1c056900 con 0x7fdc2c0ce4b0 2023-08-27T11:24:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.495+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdc18069f30 msgr2=0x7fdc1806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.495+0000 7fdc313dc700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdc18069f30 0x7fdc1806c3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdc20014040 tx=0x7fdc20008030 comp rx=0 tx=0).stop 2023-08-27T11:24:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.495+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c0ce4b0 msgr2=0x7fdc2c0b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:51.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.495+0000 7fdc313dc700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b54a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c4e0 tx=0x7fdc1c00c7f0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.496+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 shutdown_connections 2023-08-27T11:24:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.496+0000 7fdc313dc700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdc18069f30 0x7fdc1806c3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.496+0000 7fdc313dc700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdc2c15d2c0 0x7fdc2c1f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.496+0000 7fdc313dc700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc2c0b8c20 0x7fdc2c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.496+0000 7fdc313dc700 1 --2- 172.21.15.45:0/1069043973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc2c0ce4b0 0x7fdc2c0b54a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.496+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 >> 172.21.15.45:0/1069043973 conn(0x7fdc2c0d2fd0 msgr2=0x7fdc2c15b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:51.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.496+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 shutdown_connections 2023-08-27T11:24:51.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.496+0000 7fdc313dc700 1 -- 172.21.15.45:0/1069043973 wait complete. 2023-08-27T11:24:51.513 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:24:51.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.690+0000 7f18fce31700 1 Processor -- start 2023-08-27T11:24:51.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.690+0000 7f18fce31700 1 -- start start 2023-08-27T11:24:51.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18fce31700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f18f815e040 0x7f18f80d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:51.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18fce31700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18f815e420 0x7f18f815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:51.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18fce31700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815d210 0x7f18f815d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:51.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18fce31700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f815db80 con 0x7f18f815e420 2023-08-27T11:24:51.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18fce31700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f80738a0 con 0x7f18f815e040 2023-08-27T11:24:51.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18fce31700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8073e20 con 0x7f18f815d210 2023-08-27T11:24:51.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f5d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18f815e420 0x7f18f815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:51.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f5d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18f815e420 0x7f18f815e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55824/0 (socket says 172.21.15.45:55824) 2023-08-27T11:24:51.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f5d9b700 1 -- 172.21.15.45:0/1889447034 learned_addr learned my addr 172.21.15.45:0/1889447034 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:51.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f659c700 1 --2- 172.21.15.45:0/1889447034 >> v2:172.21.15.135:3300/0 conn(0x7f18f815e040 0x7f18f80d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:51.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f6d9d700 1 --2- 172.21.15.45:0/1889447034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815d210 0x7f18f815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:51.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f5d9b700 1 -- 172.21.15.45:0/1889447034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815d210 msgr2=0x7f18f815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:51.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f5d9b700 1 --2- 172.21.15.45:0/1889447034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815d210 0x7f18f815d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f5d9b700 1 -- 172.21.15.45:0/1889447034 >> v2:172.21.15.135:3300/0 conn(0x7f18f815e040 msgr2=0x7f18f80d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:51.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f5d9b700 1 --2- 172.21.15.45:0/1889447034 >> v2:172.21.15.135:3300/0 conn(0x7f18f815e040 0x7f18f80d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.691+0000 7f18f5d9b700 1 -- 172.21.15.45:0/1889447034 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8073b40 con 0x7f18f815e420 2023-08-27T11:24:51.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18f5d9b700 1 --2- 172.21.15.45:0/1889447034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e420 0x7f18f815e8b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f18f80744e0 tx=0x7f18e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe569b35d6b4102 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:51.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18f559a700 1 -- 172.21.15.45:0/1889447034 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800b870 con 0x7f18f815e420 2023-08-27T11:24:51.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18f559a700 1 -- 172.21.15.45:0/1889447034 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e800b9d0 con 0x7f18f815e420 2023-08-27T11:24:51.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18f559a700 1 -- 172.21.15.45:0/1889447034 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800bd60 con 0x7f18f815e420 2023-08-27T11:24:51.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18fce31700 1 -- 172.21.15.45:0/1889447034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e420 msgr2=0x7f18f815e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:51.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18fce31700 1 --2- 172.21.15.45:0/1889447034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e420 0x7f18f815e8b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f18f80744e0 tx=0x7f18e8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18fce31700 1 -- 172.21.15.45:0/1889447034 shutdown_connections 2023-08-27T11:24:51.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18fce31700 1 --2- 172.21.15.45:0/1889447034 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815d210 0x7f18f815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18fce31700 1 --2- 172.21.15.45:0/1889447034 >> v2:172.21.15.135:3300/0 conn(0x7f18f815e040 0x7f18f80d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18fce31700 1 --2- 172.21.15.45:0/1889447034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e420 0x7f18f815e8b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18fce31700 1 -- 172.21.15.45:0/1889447034 >> 172.21.15.45:0/1889447034 conn(0x7f18f80cf520 msgr2=0x7f18f80cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:51.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18fce31700 1 -- 172.21.15.45:0/1889447034 shutdown_connections 2023-08-27T11:24:51.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.692+0000 7f18fce31700 1 -- 172.21.15.45:0/1889447034 wait complete. 2023-08-27T11:24:51.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.694+0000 7f18fce31700 1 Processor -- start 2023-08-27T11:24:51.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.694+0000 7f18fce31700 1 -- start start 2023-08-27T11:24:51.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.695+0000 7f18fce31700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f815d210 0x7f18f80adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:51.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.695+0000 7f18fce31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e040 0x7f18f8152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:51.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.695+0000 7f18fce31700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815e420 0x7f18f8150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:51.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.695+0000 7f18fce31700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8153380 con 0x7f18f815e040 2023-08-27T11:24:51.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.695+0000 7f18fce31700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80c7200 con 0x7f18f815d210 2023-08-27T11:24:51.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.695+0000 7f18fce31700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8151090 con 0x7f18f815e420 2023-08-27T11:24:51.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.695+0000 7f18f5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e040 0x7f18f8152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:51.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.695+0000 7f18f5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e040 0x7f18f8152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55830/0 (socket says 172.21.15.45:55830) 2023-08-27T11:24:51.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.695+0000 7f18f5d9b700 1 -- 172.21.15.45:0/778222248 learned_addr learned my addr 172.21.15.45:0/778222248 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:51.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18f659c700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f815d210 0x7f18f80adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:51.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18f6d9d700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815e420 0x7f18f8150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:51.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18f5d9b700 1 -- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815e420 msgr2=0x7f18f8150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:51.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18f5d9b700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815e420 0x7f18f8150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18f5d9b700 1 -- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f815d210 msgr2=0x7f18f80adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:51.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18f5d9b700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f815d210 0x7f18f80adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:51.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18f5d9b700 1 -- 172.21.15.45:0/778222248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8009550 con 0x7f18f815e040 2023-08-27T11:24:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18f659c700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f815d210 0x7f18f80adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18f5d9b700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e040 0x7f18f8152e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f18e8000c00 tx=0x7f18e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18e6ffd700 1 -- 172.21.15.45:0/778222248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8004520 con 0x7f18f815e040 2023-08-27T11:24:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18e6ffd700 1 -- 172.21.15.45:0/778222248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e801e070 con 0x7f18f815e040 2023-08-27T11:24:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f80c65f0 con 0x7f18f815e040 2023-08-27T11:24:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18e6ffd700 1 -- 172.21.15.45:0/778222248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e80038c0 con 0x7f18f815e040 2023-08-27T11:24:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.696+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f80c6bb0 con 0x7f18f815e040 2023-08-27T11:24:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.697+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f80b54a0 con 0x7f18f815e040 2023-08-27T11:24:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.698+0000 7f18e6ffd700 1 -- 172.21.15.45:0/778222248 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f18e8003a20 con 0x7f18f815e040 2023-08-27T11:24:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.698+0000 7f18e6ffd700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f18d806a1a0 0x7f18d806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:51.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.698+0000 7f18e6ffd700 1 -- 172.21.15.45:0/778222248 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f18e8020070 con 0x7f18f815e040 2023-08-27T11:24:51.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.698+0000 7f18f659c700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f18d806a1a0 0x7f18d806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:51.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.699+0000 7f18f659c700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f18d806a1a0 0x7f18d806c660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009a00 tx=0x7f18ec008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:51.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.702+0000 7f18e6ffd700 1 -- 172.21.15.45:0/778222248 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18e80551e0 con 0x7f18f815e040 2023-08-27T11:24:51.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:51.900+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18f806da00 con 0x7f18f815e040 2023-08-27T11:24:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.501+0000 7f18e6ffd700 1 -- 172.21.15.45:0/778222248 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) v1 ==== 181+0+0 (secure 0 0 0) 0x7f18e8015020 con 0x7f18f815e040 2023-08-27T11:24:52.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.505+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f18d806a1a0 msgr2=0x7f18d806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:52.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.505+0000 7f18fce31700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f18d806a1a0 0x7f18d806c660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009a00 tx=0x7f18ec008040 comp rx=0 tx=0).stop 2023-08-27T11:24:52.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e040 msgr2=0x7f18f8152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:52.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e040 0x7f18f8152e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f18e8000c00 tx=0x7f18e8004750 comp rx=0 tx=0).stop 2023-08-27T11:24:52.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 shutdown_connections 2023-08-27T11:24:52.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f18d806a1a0 0x7f18d806c660 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f18f815e420 0x7f18f8150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f815d210 0x7f18f80adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 --2- 172.21.15.45:0/778222248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18f815e040 0x7f18f8152e40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 >> 172.21.15.45:0/778222248 conn(0x7f18f80cf520 msgr2=0x7f18f815c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 shutdown_connections 2023-08-27T11:24:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.506+0000 7f18fce31700 1 -- 172.21.15.45:0/778222248 wait complete. 2023-08-27T11:24:52.510 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:24:52.525 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:24:52.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.660+0000 7f9c03294700 1 Processor -- start 2023-08-27T11:24:52.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.660+0000 7f9c03294700 1 -- start start 2023-08-27T11:24:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.661+0000 7f9c03294700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9bfc0af200 0x7f9bfc0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.661+0000 7f9c03294700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0ce340 con 0x7f9bfc0ca620 2023-08-27T11:24:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.661+0000 7f9c03294700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0ceec0 con 0x7f9bfc0caa00 2023-08-27T11:24:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.661+0000 7f9c03294700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc15d7c0 con 0x7f9bfc0af200 2023-08-27T11:24:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.661+0000 7f9c01030700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9bfc0ca620 0x7f9bfc0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42612/0 (socket says 172.21.15.45:42612) 2023-08-27T11:24:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.661+0000 7f9c01030700 1 -- 172.21.15.45:0/3884375408 learned_addr learned my addr 172.21.15.45:0/3884375408 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.661+0000 7f9c0082f700 1 --2- 172.21.15.45:0/3884375408 >> v2:172.21.15.135:3300/0 conn(0x7f9bfc0af200 0x7f9bfc0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.661+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018451931 0 0) 0x7f9bfc0ce340 con 0x7f9bfc0ca620 2023-08-27T11:24:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.661+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be80035b0 con 0x7f9bfc0ca620 2023-08-27T11:24:52.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3833115404 0 0) 0x7f9be80035b0 con 0x7f9bfc0ca620 2023-08-27T11:24:52.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 >> v2:172.21.15.135:3300/0 conn(0x7f9bfc0af200 msgr2=0x7f9bfc0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:52.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 --2- 172.21.15.45:0/3884375408 >> v2:172.21.15.135:3300/0 conn(0x7f9bfc0af200 0x7f9bfc0cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 >> v1:172.21.15.135:6789/0 conn(0x7f9bfc0caa00 legacy=0x7f9bfc0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:52.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0cc570 con 0x7f9bfc0ca620 2023-08-27T11:24:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f9bf8001670 con 0x7f9bfc0ca620 2023-08-27T11:24:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 >> v1:172.21.15.45:6789/0 conn(0x7f9bfc0ca620 legacy=0x7f9bfc0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9be8003a00 0x7f9be8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be8004850 0x7f9be8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:52.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8005210 0x7f9be800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:52.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc15d7c0 con 0x7f9be8004850 2023-08-27T11:24:52.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be800c0b0 con 0x7f9be8005210 2023-08-27T11:24:52.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be800c270 con 0x7f9be8003a00 2023-08-27T11:24:52.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c0082f700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be8004850 0x7f9be8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:52.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c01030700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9be8003a00 0x7f9be8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:52.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c01831700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8005210 0x7f9be800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:52.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c0082f700 1 -- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9be8003a00 msgr2=0x7f9be8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:52.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c0082f700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9be8003a00 0x7f9be8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c0082f700 1 -- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8005210 msgr2=0x7f9be800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:52.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c0082f700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8005210 0x7f9be800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c0082f700 1 -- 172.21.15.45:0/3884375408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80035b0 con 0x7f9be8004850 2023-08-27T11:24:52.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c01030700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9be8003a00 0x7f9be8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:52.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c01831700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8005210 0x7f9be800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:52.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c01831700 1 -- 172.21.15.45:0/3884375408 reap_dead start 2023-08-27T11:24:52.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9c0082f700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be8004850 0x7f9be8004cd0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009b30 tx=0x7f9be8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a44add1fcf0d75 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00e6c0 con 0x7f9be8004850 2023-08-27T11:24:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec00e820 con 0x7f9be8004850 2023-08-27T11:24:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.662+0000 7f9bf3fff700 1 -- 172.21.15.45:0/3884375408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00eb90 con 0x7f9be8004850 2023-08-27T11:24:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.663+0000 7f9c03294700 1 -- 172.21.15.45:0/3884375408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be8004850 msgr2=0x7f9be8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.663+0000 7f9c03294700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be8004850 0x7f9be8004cd0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009b30 tx=0x7f9be8005880 comp rx=0 tx=0).stop 2023-08-27T11:24:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.663+0000 7f9c03294700 1 -- 172.21.15.45:0/3884375408 shutdown_connections 2023-08-27T11:24:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.663+0000 7f9c03294700 1 --2- 172.21.15.45:0/3884375408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be8004850 0x7f9be8004cd0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.663+0000 7f9c03294700 1 -- 172.21.15.45:0/3884375408 >> 172.21.15.45:0/3884375408 conn(0x7f9bfc0cc7e0 msgr2=0x7f9bfc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.663+0000 7f9c03294700 1 -- 172.21.15.45:0/3884375408 shutdown_connections 2023-08-27T11:24:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.663+0000 7f9c03294700 1 -- 172.21.15.45:0/3884375408 wait complete. 2023-08-27T11:24:52.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.664+0000 7f9c03294700 1 Processor -- start 2023-08-27T11:24:52.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.664+0000 7f9c03294700 1 -- start start 2023-08-27T11:24:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.665+0000 7f9c03294700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8004850 0x7f9bfc0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.665+0000 7f9c03294700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bfc156690 0x7f9bfc14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.665+0000 7f9c03294700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9bfc158e30 0x7f9bfc06df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.665+0000 7f9c03294700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c5750 con 0x7f9bfc156690 2023-08-27T11:24:52.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.665+0000 7f9c03294700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c7370 con 0x7f9be8004850 2023-08-27T11:24:52.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.665+0000 7f9c03294700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc14c910 con 0x7f9bfc158e30 2023-08-27T11:24:52.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.665+0000 7f9c0082f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bfc156690 0x7f9bfc14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:52.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.665+0000 7f9c0082f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bfc156690 0x7f9bfc14c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55850/0 (socket says 172.21.15.45:55850) 2023-08-27T11:24:52.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.665+0000 7f9c0082f700 1 -- 172.21.15.45:0/2908173507 learned_addr learned my addr 172.21.15.45:0/2908173507 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:52.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c01030700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8004850 0x7f9bfc0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:52.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c01831700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9bfc158e30 0x7f9bfc06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:52.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c0082f700 1 -- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9bfc158e30 msgr2=0x7f9bfc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:52.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c0082f700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9bfc158e30 0x7f9bfc06df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c0082f700 1 -- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8004850 msgr2=0x7f9bfc0c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:52.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c0082f700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8004850 0x7f9bfc0c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:52.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c0082f700 1 -- 172.21.15.45:0/2908173507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0c4b80 con 0x7f9bfc156690 2023-08-27T11:24:52.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c0082f700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bfc156690 0x7f9bfc14c3d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00efd0 tx=0x7f9bec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:52.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9bf1ffb700 1 -- 172.21.15.45:0/2908173507 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00e6c0 con 0x7f9bfc156690 2023-08-27T11:24:52.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9bf1ffb700 1 -- 172.21.15.45:0/2908173507 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec013e40 con 0x7f9bfc156690 2023-08-27T11:24:52.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9bf1ffb700 1 -- 172.21.15.45:0/2908173507 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec01c690 con 0x7f9bfc156690 2023-08-27T11:24:52.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc0c4e20 con 0x7f9bfc156690 2023-08-27T11:24:52.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.666+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc0c53e0 con 0x7f9bfc156690 2023-08-27T11:24:52.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.667+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc0c5570 con 0x7f9bfc156690 2023-08-27T11:24:52.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.668+0000 7f9bf1ffb700 1 -- 172.21.15.45:0/2908173507 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f9bec01c7f0 con 0x7f9bfc156690 2023-08-27T11:24:52.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.668+0000 7f9bf1ffb700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f9be006a170 0x7f9be006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:52.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.668+0000 7f9bf1ffb700 1 -- 172.21.15.45:0/2908173507 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7257+0+0 (secure 0 0 0) 0x7f9bec08a250 con 0x7f9bfc156690 2023-08-27T11:24:52.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.668+0000 7f9c01030700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f9be006a170 0x7f9be006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:52.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.669+0000 7f9c01030700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f9be006a170 0x7f9be006c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8001c90 tx=0x7f9bf8009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:52.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.673+0000 7f9bf1ffb700 1 -- 172.21.15.45:0/2908173507 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9bec055650 con 0x7f9bfc156690 2023-08-27T11:24:52.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:52.873+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9bfc0ceb20 con 0x7f9bfc156690 2023-08-27T11:24:53.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.005+0000 7f9bf1ffb700 1 -- 172.21.15.45:0/2908173507 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9bec017020 con 0x7f9bfc156690 2023-08-27T11:24:53.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f9be006a170 msgr2=0x7f9be006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f9be006a170 0x7f9be006c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8001c90 tx=0x7f9bf8009040 comp rx=0 tx=0).stop 2023-08-27T11:24:53.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bfc156690 msgr2=0x7f9bfc14c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bfc156690 0x7f9bfc14c3d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00efd0 tx=0x7f9bec0047c0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 shutdown_connections 2023-08-27T11:24:53.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f9be006a170 0x7f9be006c630 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9bfc158e30 0x7f9bfc06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be8004850 0x7f9bfc0c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 --2- 172.21.15.45:0/2908173507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bfc156690 0x7f9bfc14c3d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.008+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 >> 172.21.15.45:0/2908173507 conn(0x7f9bfc0cc7e0 msgr2=0x7f9bfc0ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.009+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 shutdown_connections 2023-08-27T11:24:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.009+0000 7f9c03294700 1 -- 172.21.15.45:0/2908173507 wait complete. 2023-08-27T11:24:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-08-27T11:24:53.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:24:53.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.150+0000 7ff193e1f700 1 Processor -- start 2023-08-27T11:24:53.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.151+0000 7ff193e1f700 1 -- start start 2023-08-27T11:24:53.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.151+0000 7ff193e1f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c15d340 0x7ff18c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.151+0000 7ff193e1f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff18c0cd880 0x7ff18c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.151+0000 7ff193e1f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18c06b160 con 0x7ff18c0cdd00 2023-08-27T11:24:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.151+0000 7ff193e1f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff18c072010 con 0x7ff18c0cd880 2023-08-27T11:24:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.151+0000 7ff193e1f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0af590 con 0x7ff18c15d340 2023-08-27T11:24:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.151+0000 7ff1913ba700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff18c0cd880 0x7ff18c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff1923bc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff18c0cdd00 0x7ff18c0b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42636/0 (socket says 172.21.15.45:42636) 2023-08-27T11:24:53.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff1913ba700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff18c0cd880 0x7ff18c06a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55856/0 (socket says 172.21.15.45:55856) 2023-08-27T11:24:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff1923bc700 1 -- 172.21.15.45:0/1008458328 learned_addr learned my addr 172.21.15.45:0/1008458328 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff191bbb700 1 --2- 172.21.15.45:0/1008458328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c15d340 0x7ff18c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff1913ba700 1 -- 172.21.15.45:0/1008458328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c15d340 msgr2=0x7ff18c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff1913ba700 1 --2- 172.21.15.45:0/1008458328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c15d340 0x7ff18c160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff1913ba700 1 -- 172.21.15.45:0/1008458328 >> v1:172.21.15.45:6789/0 conn(0x7ff18c0cdd00 legacy=0x7ff18c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff1913ba700 1 -- 172.21.15.45:0/1008458328 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c0af2b0 con 0x7ff18c0cd880 2023-08-27T11:24:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff190bb9700 1 -- 172.21.15.45:0/1008458328 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652331460 0 0) 0x7ff18c06b160 con 0x7ff18c0cdd00 2023-08-27T11:24:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff191bbb700 1 --2- 172.21.15.45:0/1008458328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c15d340 0x7ff18c160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff1913ba700 1 --2- 172.21.15.45:0/1008458328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 0x7ff18c06a280 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff188009840 tx=0x7ff188009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501355e18a8a906b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:53.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff190bb9700 1 -- 172.21.15.45:0/1008458328 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188011830 con 0x7ff18c0cd880 2023-08-27T11:24:53.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff190bb9700 1 -- 172.21.15.45:0/1008458328 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff188011990 con 0x7ff18c0cd880 2023-08-27T11:24:53.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.152+0000 7ff190bb9700 1 -- 172.21.15.45:0/1008458328 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188011d20 con 0x7ff18c0cd880 2023-08-27T11:24:53.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.153+0000 7ff193e1f700 1 -- 172.21.15.45:0/1008458328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 msgr2=0x7ff18c06a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.153+0000 7ff193e1f700 1 --2- 172.21.15.45:0/1008458328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 0x7ff18c06a280 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff188009840 tx=0x7ff188009b50 comp rx=0 tx=0).stop 2023-08-27T11:24:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.153+0000 7ff193e1f700 1 -- 172.21.15.45:0/1008458328 shutdown_connections 2023-08-27T11:24:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.153+0000 7ff193e1f700 1 --2- 172.21.15.45:0/1008458328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c15d340 0x7ff18c160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.153+0000 7ff193e1f700 1 --2- 172.21.15.45:0/1008458328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 0x7ff18c06a280 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.153+0000 7ff193e1f700 1 -- 172.21.15.45:0/1008458328 >> 172.21.15.45:0/1008458328 conn(0x7ff18c0cc7c0 msgr2=0x7ff18c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.153+0000 7ff193e1f700 1 -- 172.21.15.45:0/1008458328 shutdown_connections 2023-08-27T11:24:53.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.153+0000 7ff193e1f700 1 -- 172.21.15.45:0/1008458328 wait complete. 2023-08-27T11:24:53.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.154+0000 7ff193e1f700 1 Processor -- start 2023-08-27T11:24:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.154+0000 7ff193e1f700 1 -- start start 2023-08-27T11:24:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff193e1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 0x7ff18c0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff193e1f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c0cdd00 0x7ff18c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff193e1f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff18c15d340 0x7ff18c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff193e1f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0cc3e0 con 0x7ff18c0cd880 2023-08-27T11:24:53.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff193e1f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0ca110 con 0x7ff18c15d340 2023-08-27T11:24:53.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff193e1f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0ca3f0 con 0x7ff18c0cdd00 2023-08-27T11:24:53.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff191bbb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 0x7ff18c0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff191bbb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 0x7ff18c0c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55862/0 (socket says 172.21.15.45:55862) 2023-08-27T11:24:53.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff191bbb700 1 -- 172.21.15.45:0/103630266 learned_addr learned my addr 172.21.15.45:0/103630266 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:53.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff191bbb700 1 -- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c0cdd00 msgr2=0x7ff18c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff1913ba700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c0cdd00 0x7ff18c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff1923bc700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff18c15d340 0x7ff18c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff191bbb700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c0cdd00 0x7ff18c0cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff191bbb700 1 -- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff18c15d340 msgr2=0x7ff18c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff191bbb700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff18c15d340 0x7ff18c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff191bbb700 1 -- 172.21.15.45:0/103630266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1880094a0 con 0x7ff18c0cd880 2023-08-27T11:24:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff1913ba700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c0cdd00 0x7ff18c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff1923bc700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff18c15d340 0x7ff18c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.155+0000 7ff191bbb700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 0x7ff18c0c89f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00e810 tx=0x7ff17c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:53.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.156+0000 7ff1827fc700 1 -- 172.21.15.45:0/103630266 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c013610 con 0x7ff18c0cd880 2023-08-27T11:24:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.156+0000 7ff1827fc700 1 -- 172.21.15.45:0/103630266 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c013770 con 0x7ff18c0cd880 2023-08-27T11:24:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.156+0000 7ff1827fc700 1 -- 172.21.15.45:0/103630266 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c013b20 con 0x7ff18c0cd880 2023-08-27T11:24:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.156+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c0ca6d0 con 0x7ff18c0cd880 2023-08-27T11:24:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.156+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c0cac90 con 0x7ff18c0cd880 2023-08-27T11:24:53.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.157+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c0cb000 con 0x7ff18c0cd880 2023-08-27T11:24:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.157+0000 7ff1827fc700 1 -- 172.21.15.45:0/103630266 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) v1 ==== 88387+0+0 (secure 0 0 0) 0x7ff17c013c80 con 0x7ff18c0cd880 2023-08-27T11:24:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.157+0000 7ff1827fc700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff17006a170 0x7ff17006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.158+0000 7ff1827fc700 1 -- 172.21.15.45:0/103630266 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff17c088630 con 0x7ff18c0cd880 2023-08-27T11:24:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.158+0000 7ff1913ba700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff17006a170 0x7ff17006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.158+0000 7ff1913ba700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff17006a170 0x7ff17006c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff18c1f6450 tx=0x7ff188008030 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:53.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.162+0000 7ff1827fc700 1 -- 172.21.15.45:0/103630266 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff17c0539c0 con 0x7ff18c0cd880 2023-08-27T11:24:53.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.332+0000 7ff1827fc700 1 -- 172.21.15.45:0/103630266 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff17c053d90 con 0x7ff18c0cd880 2023-08-27T11:24:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.375+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff18c072690 con 0x7ff18c0cd880 2023-08-27T11:24:53.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.379+0000 7ff1827fc700 1 -- 172.21.15.45:0/103630266 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff17c057450 con 0x7ff18c0cd880 2023-08-27T11:24:53.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff17006a170 msgr2=0x7ff17006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff17006a170 0x7ff17006c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff18c1f6450 tx=0x7ff188008030 comp rx=0 tx=0).stop 2023-08-27T11:24:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 msgr2=0x7ff18c0c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 0x7ff18c0c89f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00e810 tx=0x7ff17c00ebd0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 shutdown_connections 2023-08-27T11:24:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff17006a170 0x7ff17006c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff18c0cd880 0x7ff18c0c89f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff18c15d340 0x7ff18c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 --2- 172.21.15.45:0/103630266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff18c0cdd00 0x7ff18c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 >> 172.21.15.45:0/103630266 conn(0x7ff18c0cc7c0 msgr2=0x7ff18c06f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 shutdown_connections 2023-08-27T11:24:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.381+0000 7ff193e1f700 1 -- 172.21.15.45:0/103630266 wait complete. 2023-08-27T11:24:53.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-27T11:24:53.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.522+0000 7fe7a72e6700 1 Processor -- start 2023-08-27T11:24:53.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.522+0000 7fe7a72e6700 1 -- start start 2023-08-27T11:24:53.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.522+0000 7fe7a72e6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015a700 0x7fe7a015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.522+0000 7fe7a72e6700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe7a015aae0 0x7fe7a015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a72e6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a00cb060 con 0x7fe7a006ef10 2023-08-27T11:24:53.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a72e6700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a006e320 con 0x7fe7a015aae0 2023-08-27T11:24:53.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a72e6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a015d7d0 con 0x7fe7a015a700 2023-08-27T11:24:53.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a5883700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe7a006ef10 0x7fe7a00cb4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42650/0 (socket says 172.21.15.45:42650) 2023-08-27T11:24:53.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a5883700 1 -- 172.21.15.45:0/2649050119 learned_addr learned my addr 172.21.15.45:0/2649050119 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:53.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554678419 0 0) 0x7fe7a00cb060 con 0x7fe7a006ef10 2023-08-27T11:24:53.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe78c0035b0 con 0x7fe7a006ef10 2023-08-27T11:24:53.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a5082700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015a700 0x7fe7a015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a4881700 1 --2- 172.21.15.45:0/2649050119 >> v2:172.21.15.135:3300/0 conn(0x7fe7a015aae0 0x7fe7a015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 750991121 0 0) 0x7fe78c0035b0 con 0x7fe7a006ef10 2023-08-27T11:24:53.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015a700 msgr2=0x7fe7a015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:53.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015a700 0x7fe7a015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 >> v2:172.21.15.135:3300/0 conn(0x7fe7a015aae0 msgr2=0x7fe7a015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:53.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 --2- 172.21.15.45:0/2649050119 >> v2:172.21.15.135:3300/0 conn(0x7fe7a015aae0 0x7fe7a015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00ca450 con 0x7fe7a006ef10 2023-08-27T11:24:53.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fe7980019f0 con 0x7fe7a006ef10 2023-08-27T11:24:53.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 >> v1:172.21.15.45:6789/0 conn(0x7fe7a006ef10 legacy=0x7fe7a00cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe78c003a00 0x7fe78c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe78c004850 0x7fe78c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c005210 0x7fe78c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a006e320 con 0x7fe78c005210 2023-08-27T11:24:53.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a015d7d0 con 0x7fe78c004850 2023-08-27T11:24:53.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c00c050 con 0x7fe78c003a00 2023-08-27T11:24:53.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a4881700 1 --2- 172.21.15.45:0/2649050119 >> v2:172.21.15.135:3300/0 conn(0x7fe7a015aae0 0x7fe7a015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:53.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a5082700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015a700 0x7fe7a015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:53.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.523+0000 7fe7a5883700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c005210 0x7fe78c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a5082700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe78c003a00 0x7fe78c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a4881700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe78c004850 0x7fe78c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a5883700 1 -- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe78c003a00 msgr2=0x7fe78c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:53.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a5883700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe78c003a00 0x7fe78c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a5883700 1 -- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe78c004850 msgr2=0x7fe78c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:53.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a5883700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe78c004850 0x7fe78c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a5883700 1 -- 172.21.15.45:0/2649050119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c0035b0 con 0x7fe78c005210 2023-08-27T11:24:53.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a5883700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c005210 0x7fe78c00ba10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe79800a440 tx=0x7fe78c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5300b37c5a0df33a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:53.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798010690 con 0x7fe78c005210 2023-08-27T11:24:53.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe798010810 con 0x7fe78c005210 2023-08-27T11:24:53.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe793fff700 1 -- 172.21.15.45:0/2649050119 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798010bc0 con 0x7fe78c005210 2023-08-27T11:24:53.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2649050119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c005210 msgr2=0x7fe78c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a72e6700 1 --2- 172.21.15.45:0/2649050119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c005210 0x7fe78c00ba10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe79800a440 tx=0x7fe78c005880 comp rx=0 tx=0).stop 2023-08-27T11:24:53.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a5883700 1 -- 172.21.15.45:0/2649050119 reap_dead start 2023-08-27T11:24:53.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2649050119 shutdown_connections 2023-08-27T11:24:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.524+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2649050119 >> 172.21.15.45:0/2649050119 conn(0x7fe7a00cc7e0 msgr2=0x7fe7a00ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.525+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2649050119 shutdown_connections 2023-08-27T11:24:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.525+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2649050119 wait complete. 2023-08-27T11:24:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.526+0000 7fe7a72e6700 1 Processor -- start 2023-08-27T11:24:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.526+0000 7fe7a72e6700 1 -- start start 2023-08-27T11:24:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a72e6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015b830 0x7fe7a0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a72e6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7a015bc10 0x7fe7a00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a72e6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7a00acdf0 0x7fe7a0151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a72e6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00ad270 con 0x7fe7a015bc10 2023-08-27T11:24:53.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a72e6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00b34a0 con 0x7fe7a00acdf0 2023-08-27T11:24:53.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a72e6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00ae0a0 con 0x7fe7a015b830 2023-08-27T11:24:53.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a4881700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7a015bc10 0x7fe7a00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a4881700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7a015bc10 0x7fe7a00ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55884/0 (socket says 172.21.15.45:55884) 2023-08-27T11:24:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a4881700 1 -- 172.21.15.45:0/2408903051 learned_addr learned my addr 172.21.15.45:0/2408903051 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a5883700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7a00acdf0 0x7fe7a0151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a5082700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015b830 0x7fe7a0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a4881700 1 -- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015b830 msgr2=0x7fe7a0154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a4881700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015b830 0x7fe7a0154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a4881700 1 -- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7a00acdf0 msgr2=0x7fe7a0151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a4881700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7a00acdf0 0x7fe7a0151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a4881700 1 -- 172.21.15.45:0/2408903051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00ad490 con 0x7fe7a015bc10 2023-08-27T11:24:53.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a4881700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7a015bc10 0x7fe7a00ac8b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe794010870 tx=0x7fe794010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:53.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe791ffb700 1 -- 172.21.15.45:0/2408903051 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794015880 con 0x7fe7a015bc10 2023-08-27T11:24:53.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe791ffb700 1 -- 172.21.15.45:0/2408903051 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79400c050 con 0x7fe7a015bc10 2023-08-27T11:24:53.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a00ad770 con 0x7fe7a015bc10 2023-08-27T11:24:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.527+0000 7fe791ffb700 1 -- 172.21.15.45:0/2408903051 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794015c20 con 0x7fe7a015bc10 2023-08-27T11:24:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.528+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a00add30 con 0x7fe7a015bc10 2023-08-27T11:24:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.529+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a00adef0 con 0x7fe7a015bc10 2023-08-27T11:24:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.529+0000 7fe791ffb700 1 -- 172.21.15.45:0/2408903051 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe794016420 con 0x7fe7a015bc10 2023-08-27T11:24:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.529+0000 7fe791ffb700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe78406a200 0x7fe78406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.529+0000 7fe791ffb700 1 -- 172.21.15.45:0/2408903051 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe794089280 con 0x7fe7a015bc10 2023-08-27T11:24:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.530+0000 7fe7a5082700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe78406a200 0x7fe78406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.530+0000 7fe7a5082700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe78406a200 0x7fe78406c6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00748b0 tx=0x7fe79c008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:53.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.534+0000 7fe791ffb700 1 -- 172.21.15.45:0/2408903051 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe7940543c0 con 0x7fe7a015bc10 2023-08-27T11:24:53.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.734+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fe7a00726d0 con 0x7fe7a015bc10 2023-08-27T11:24:53.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.738+0000 7fe791ffb700 1 -- 172.21.15.45:0/2408903051 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v14) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe7940579e0 con 0x7fe7a015bc10 2023-08-27T11:24:53.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe78406a200 msgr2=0x7fe78406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe78406a200 0x7fe78406c6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00748b0 tx=0x7fe79c008040 comp rx=0 tx=0).stop 2023-08-27T11:24:53.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7a015bc10 msgr2=0x7fe7a00ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7a015bc10 0x7fe7a00ac8b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe794010870 tx=0x7fe794010b80 comp rx=0 tx=0).stop 2023-08-27T11:24:53.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 shutdown_connections 2023-08-27T11:24:53.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe78406a200 0x7fe78406c6c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7a00acdf0 0x7fe7a0151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7a015b830 0x7fe7a0154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 --2- 172.21.15.45:0/2408903051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7a015bc10 0x7fe7a00ac8b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 >> 172.21.15.45:0/2408903051 conn(0x7fe7a00cc7e0 msgr2=0x7fe7a006a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 shutdown_connections 2023-08-27T11:24:53.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.740+0000 7fe7a72e6700 1 -- 172.21.15.45:0/2408903051 wait complete. 2023-08-27T11:24:53.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-27T11:24:53.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.880+0000 7f708daea700 1 Processor -- start 2023-08-27T11:24:53.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.880+0000 7f708daea700 1 -- start start 2023-08-27T11:24:53.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f708daea700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f708815d3a0 0x7f70880d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f708daea700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f708806c4b0 0x7f70880cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f708daea700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708815d780 con 0x7f70880d02c0 2023-08-27T11:24:53.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f708daea700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70880cfe30 con 0x7f708806c4b0 2023-08-27T11:24:53.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f708daea700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70880cf220 con 0x7f708815d3a0 2023-08-27T11:24:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f7087fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f708806c4b0 0x7f70880cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f7086ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f70880d02c0 0x7f708806bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42668/0 (socket says 172.21.15.45:42668) 2023-08-27T11:24:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f7086ffd700 1 -- 172.21.15.45:0/3817052887 learned_addr learned my addr 172.21.15.45:0/3817052887 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f70877fe700 1 --2- 172.21.15.45:0/3817052887 >> v2:172.21.15.135:3300/0 conn(0x7f708815d3a0 0x7f70880d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f7087fff700 1 -- 172.21.15.45:0/3817052887 >> v2:172.21.15.135:3300/0 conn(0x7f708815d3a0 msgr2=0x7f70880d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:53.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f7087fff700 1 --2- 172.21.15.45:0/3817052887 >> v2:172.21.15.135:3300/0 conn(0x7f708815d3a0 0x7f70880d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f7087fff700 1 -- 172.21.15.45:0/3817052887 >> v1:172.21.15.45:6789/0 conn(0x7f70880d02c0 legacy=0x7f708806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f70867fc700 1 -- 172.21.15.45:0/3817052887 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217426090 0 0) 0x7f7070001110 con 0x7f70880d02c0 2023-08-27T11:24:53.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f7087fff700 1 -- 172.21.15.45:0/3817052887 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880cf500 con 0x7f708806c4b0 2023-08-27T11:24:53.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.881+0000 7f7087fff700 1 --2- 172.21.15.45:0/3817052887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708806c4b0 0x7f70880cb490 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f70880701d0 tx=0x7f708000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c03f0b91d8ee24 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:53.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f70867fc700 1 -- 172.21.15.45:0/3817052887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708000e070 con 0x7f708806c4b0 2023-08-27T11:24:53.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f70867fc700 1 -- 172.21.15.45:0/3817052887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708000bbd0 con 0x7f708806c4b0 2023-08-27T11:24:53.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f70867fc700 1 -- 172.21.15.45:0/3817052887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080012430 con 0x7f708806c4b0 2023-08-27T11:24:53.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f708daea700 1 -- 172.21.15.45:0/3817052887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708806c4b0 msgr2=0x7f70880cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f708daea700 1 --2- 172.21.15.45:0/3817052887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708806c4b0 0x7f70880cb490 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f70880701d0 tx=0x7f708000b330 comp rx=0 tx=0).stop 2023-08-27T11:24:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f708daea700 1 -- 172.21.15.45:0/3817052887 shutdown_connections 2023-08-27T11:24:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f708daea700 1 --2- 172.21.15.45:0/3817052887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708806c4b0 0x7f70880cb490 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f708daea700 1 --2- 172.21.15.45:0/3817052887 >> v2:172.21.15.135:3300/0 conn(0x7f708815d3a0 0x7f70880d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f708daea700 1 -- 172.21.15.45:0/3817052887 >> 172.21.15.45:0/3817052887 conn(0x7f708809a1a0 msgr2=0x7f70880d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f708daea700 1 -- 172.21.15.45:0/3817052887 shutdown_connections 2023-08-27T11:24:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.882+0000 7f708daea700 1 -- 172.21.15.45:0/3817052887 wait complete. 2023-08-27T11:24:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.883+0000 7f708daea700 1 Processor -- start 2023-08-27T11:24:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f708daea700 1 -- start start 2023-08-27T11:24:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f708daea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f708806c4b0 0x7f7088155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f708daea700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70880d02c0 0x7f70880b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f708daea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708815d3a0 0x7f70880b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f708daea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70880b3a60 con 0x7f708815d3a0 2023-08-27T11:24:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f708daea700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088156700 con 0x7f70880d02c0 2023-08-27T11:24:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f708daea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70881561f0 con 0x7f708806c4b0 2023-08-27T11:24:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f7087fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708815d3a0 0x7f70880b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f7087fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708815d3a0 0x7f70880b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55900/0 (socket says 172.21.15.45:55900) 2023-08-27T11:24:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.884+0000 7f7087fff700 1 -- 172.21.15.45:0/4190025813 learned_addr learned my addr 172.21.15.45:0/4190025813 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f70877fe700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f708806c4b0 0x7f7088155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f7086ffd700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70880d02c0 0x7f70880b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f7087fff700 1 -- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f708806c4b0 msgr2=0x7f7088155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f7087fff700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f708806c4b0 0x7f7088155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f7087fff700 1 -- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70880d02c0 msgr2=0x7f70880b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f7087fff700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70880d02c0 0x7f70880b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f7087fff700 1 -- 172.21.15.45:0/4190025813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7080009a70 con 0x7f708815d3a0 2023-08-27T11:24:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f7087fff700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708815d3a0 0x7f70880b6510 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f708808b380 tx=0x7f7080004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f707ffff700 1 -- 172.21.15.45:0/4190025813 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080012430 con 0x7f708815d3a0 2023-08-27T11:24:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f707ffff700 1 -- 172.21.15.45:0/4190025813 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7080020070 con 0x7f708815d3a0 2023-08-27T11:24:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f707ffff700 1 -- 172.21.15.45:0/4190025813 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080004280 con 0x7f708815d3a0 2023-08-27T11:24:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70881e7bb0 con 0x7f708815d3a0 2023-08-27T11:24:53.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.885+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70881e8170 con 0x7f708815d3a0 2023-08-27T11:24:53.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.886+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70880b3bd0 con 0x7f708815d3a0 2023-08-27T11:24:53.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.887+0000 7f707ffff700 1 -- 172.21.15.45:0/4190025813 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f70800043e0 con 0x7f708815d3a0 2023-08-27T11:24:53.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.887+0000 7f707ffff700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f706806a200 0x7f706806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:53.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.887+0000 7f707ffff700 1 -- 172.21.15.45:0/4190025813 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7080022070 con 0x7f708815d3a0 2023-08-27T11:24:53.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.887+0000 7f70877fe700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f706806a200 0x7f706806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:53.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.888+0000 7f70877fe700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f706806a200 0x7f706806c6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70881f6300 tx=0x7f7078008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:53.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:53.891+0000 7f707ffff700 1 -- 172.21.15.45:0/4190025813 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f70880b3bd0 con 0x7f708815d3a0 2023-08-27T11:24:54.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.093+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f70880b3bd0 con 0x7f708815d3a0 2023-08-27T11:24:54.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.098+0000 7f707ffff700 1 -- 172.21.15.45:0/4190025813 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v15) v1 ==== 115+0+0 (secure 0 0 0) 0x7f70880b3bd0 con 0x7f708815d3a0 2023-08-27T11:24:54.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.100+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f706806a200 msgr2=0x7f706806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.100+0000 7f708daea700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f706806a200 0x7f706806c6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70881f6300 tx=0x7f7078008040 comp rx=0 tx=0).stop 2023-08-27T11:24:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.100+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708815d3a0 msgr2=0x7f70880b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.100+0000 7f708daea700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708815d3a0 0x7f70880b6510 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f708808b380 tx=0x7f7080004ad0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.100+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 shutdown_connections 2023-08-27T11:24:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.100+0000 7f708daea700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f706806a200 0x7f706806c6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.100+0000 7f708daea700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f708815d3a0 0x7f70880b6510 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.100+0000 7f708daea700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f708806c4b0 0x7f7088155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.101+0000 7f708daea700 1 --2- 172.21.15.45:0/4190025813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70880d02c0 0x7f70880b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.101+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 >> 172.21.15.45:0/4190025813 conn(0x7f708809a1a0 msgr2=0x7f70880ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.101+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 shutdown_connections 2023-08-27T11:24:54.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.101+0000 7f708daea700 1 -- 172.21.15.45:0/4190025813 wait complete. 2023-08-27T11:24:54.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-08-27T11:24:54.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.239+0000 7fabf8612700 1 Processor -- start 2023-08-27T11:24:54.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.239+0000 7fabf8612700 1 -- start start 2023-08-27T11:24:54.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf8612700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf0155660 0x7fabf006e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:54.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf8612700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf00cbf00 con 0x7fabf0070e80 2023-08-27T11:24:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf8612700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf00cc180 con 0x7fabf006a990 2023-08-27T11:24:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf8612700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00caa60 con 0x7fabf0155660 2023-08-27T11:24:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf5bad700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fabf0070e80 0x7fabf006a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42682/0 (socket says 172.21.15.45:42682) 2023-08-27T11:24:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf5bad700 1 -- 172.21.15.45:0/2209371176 learned_addr learned my addr 172.21.15.45:0/2209371176 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf63ae700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf0155660 0x7fabf006e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349891719 0 0) 0x7fabf00cbf00 con 0x7fabf0070e80 2023-08-27T11:24:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabdc0035b0 con 0x7fabf0070e80 2023-08-27T11:24:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1354415633 0 0) 0x7fabdc0035b0 con 0x7fabf0070e80 2023-08-27T11:24:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf0155660 msgr2=0x7fabf006e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf0155660 0x7fabf006e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 >> v1:172.21.15.135:6789/0 conn(0x7fabf006a990 legacy=0x7fabf006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf00ca4a0 con 0x7fabf0070e80 2023-08-27T11:24:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fabe0001670 con 0x7fabf0070e80 2023-08-27T11:24:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 >> v1:172.21.15.45:6789/0 conn(0x7fabf0070e80 legacy=0x7fabf006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.240+0000 7fabf53ac700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf0155660 0x7fabdc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:54.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf53ac700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabdc003fb0 0x7fabdc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:54.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf53ac700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabdc004d90 0x7fabdc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:54.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00caa60 con 0x7fabdc004d90 2023-08-27T11:24:54.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00bba0 con 0x7fabdc003fb0 2023-08-27T11:24:54.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00bd60 con 0x7fabf0155660 2023-08-27T11:24:54.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf6baf700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabdc004d90 0x7fabdc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:54.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf63ae700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf0155660 0x7fabdc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:54.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf5bad700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabdc003fb0 0x7fabdc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:54.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf6baf700 1 -- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf0155660 msgr2=0x7fabdc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:54.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf6baf700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabf0155660 0x7fabdc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf6baf700 1 -- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabdc003fb0 msgr2=0x7fabdc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:54.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf6baf700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabdc003fb0 0x7fabdc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf6baf700 1 -- 172.21.15.45:0/2209371176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0035b0 con 0x7fabdc004d90 2023-08-27T11:24:54.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf6baf700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabdc004d90 0x7fabdc005210 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c190 tx=0x7fabdc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56227f388d140ec8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:54.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8001db0 con 0x7fabdc004d90 2023-08-27T11:24:54.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe800ccd0 con 0x7fabdc004d90 2023-08-27T11:24:54.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.241+0000 7fabf53ac700 1 -- 172.21.15.45:0/2209371176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8003c40 con 0x7fabdc004d90 2023-08-27T11:24:54.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.242+0000 7fabf8612700 1 -- 172.21.15.45:0/2209371176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabdc004d90 msgr2=0x7fabdc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:54.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.242+0000 7fabf8612700 1 --2- 172.21.15.45:0/2209371176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabdc004d90 0x7fabdc005210 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c190 tx=0x7fabdc005880 comp rx=0 tx=0).stop 2023-08-27T11:24:54.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.242+0000 7fabf6baf700 1 -- 172.21.15.45:0/2209371176 reap_dead start 2023-08-27T11:24:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.242+0000 7fabf8612700 1 -- 172.21.15.45:0/2209371176 shutdown_connections 2023-08-27T11:24:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.242+0000 7fabf8612700 1 -- 172.21.15.45:0/2209371176 >> 172.21.15.45:0/2209371176 conn(0x7fabf00cfe50 msgr2=0x7fabf00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.242+0000 7fabf8612700 1 -- 172.21.15.45:0/2209371176 shutdown_connections 2023-08-27T11:24:54.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.242+0000 7fabf8612700 1 -- 172.21.15.45:0/2209371176 wait complete. 2023-08-27T11:24:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.243+0000 7fabf8612700 1 Processor -- start 2023-08-27T11:24:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.243+0000 7fabf8612700 1 -- start start 2023-08-27T11:24:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf8612700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabdc004d90 0x7fabf014c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:54.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf8612700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0152e10 0x7fabf0153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf8612700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf0156f20 0x7fabf01573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf8612700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00a89a0 con 0x7fabf0152e10 2023-08-27T11:24:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf8612700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf01537d0 con 0x7fabf0156f20 2023-08-27T11:24:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf8612700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf01588f0 con 0x7fabdc004d90 2023-08-27T11:24:54.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf5bad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0152e10 0x7fabf0153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf5bad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0152e10 0x7fabf0153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55918/0 (socket says 172.21.15.45:55918) 2023-08-27T11:24:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf5bad700 1 -- 172.21.15.45:0/4248240038 learned_addr learned my addr 172.21.15.45:0/4248240038 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf6baf700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf0156f20 0x7fabf01573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf63ae700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabdc004d90 0x7fabf014c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:54.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf5bad700 1 -- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabdc004d90 msgr2=0x7fabf014c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf5bad700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabdc004d90 0x7fabf014c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf5bad700 1 -- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf0156f20 msgr2=0x7fabf01573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf5bad700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf0156f20 0x7fabf01573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf5bad700 1 -- 172.21.15.45:0/4248240038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf0157ce0 con 0x7fabf0152e10 2023-08-27T11:24:54.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf6baf700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf0156f20 0x7fabf01573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:54.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.244+0000 7fabf5bad700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0152e10 0x7fabf0153290 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fabe00072f0 tx=0x7fabe00076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:54.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.245+0000 7fabe6ffd700 1 -- 172.21.15.45:0/4248240038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe00077d0 con 0x7fabf0152e10 2023-08-27T11:24:54.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.245+0000 7fabe6ffd700 1 -- 172.21.15.45:0/4248240038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe000f870 con 0x7fabf0152e10 2023-08-27T11:24:54.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.245+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf0157fc0 con 0x7fabf0152e10 2023-08-27T11:24:54.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.245+0000 7fabe6ffd700 1 -- 172.21.15.45:0/4248240038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe000fc20 con 0x7fabf0152e10 2023-08-27T11:24:54.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.245+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf0158580 con 0x7fabf0152e10 2023-08-27T11:24:54.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.246+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf0158740 con 0x7fabf0152e10 2023-08-27T11:24:54.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.246+0000 7fabe6ffd700 1 -- 172.21.15.45:0/4248240038 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fabe000fd80 con 0x7fabf0152e10 2023-08-27T11:24:54.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.247+0000 7fabe6ffd700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fabd406a200 0x7fabd406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:54.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.247+0000 7fabe6ffd700 1 -- 172.21.15.45:0/4248240038 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fabe0088b60 con 0x7fabf0152e10 2023-08-27T11:24:54.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.247+0000 7fabf63ae700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fabd406a200 0x7fabd406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:54.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.247+0000 7fabf63ae700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fabd406a200 0x7fabd406c6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fabec009ba0 tx=0x7fabec008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:54.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.251+0000 7fabe6ffd700 1 -- 172.21.15.45:0/4248240038 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fabe0053e70 con 0x7fabf0152e10 2023-08-27T11:24:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:54.538+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fabf0072810 con 0x7fabf0152e10 2023-08-27T11:24:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.506+0000 7fabe6ffd700 1 -- 172.21.15.45:0/4248240038 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v472) v1 ==== 158+0+0 (secure 0 0 0) 0x7fabe0057490 con 0x7fabf0152e10 2023-08-27T11:24:55.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fabd406a200 msgr2=0x7fabd406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:55.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fabd406a200 0x7fabd406c6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fabec009ba0 tx=0x7fabec008040 comp rx=0 tx=0).stop 2023-08-27T11:24:55.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0152e10 msgr2=0x7fabf0153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:55.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0152e10 0x7fabf0153290 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fabe00072f0 tx=0x7fabe00076b0 comp rx=0 tx=0).stop 2023-08-27T11:24:55.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 shutdown_connections 2023-08-27T11:24:55.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fabd406a200 0x7fabd406c6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:55.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabf0156f20 0x7fabf01573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:55.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabdc004d90 0x7fabf014c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 --2- 172.21.15.45:0/4248240038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0152e10 0x7fabf0153290 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 >> 172.21.15.45:0/4248240038 conn(0x7fabf00cfe50 msgr2=0x7fabf006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 shutdown_connections 2023-08-27T11:24:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.509+0000 7fabf8612700 1 -- 172.21.15.45:0/4248240038 wait complete. 2023-08-27T11:24:55.525 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-08-27T11:24:55.526 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-08-27T11:24:55.526 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-27T11:24:55.527 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-08-27T11:24:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.658+0000 7f5d9858c700 1 Processor -- start 2023-08-27T11:24:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.658+0000 7f5d9858c700 1 -- start start 2023-08-27T11:24:55.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d9858c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5d9015e990 0x7f5d900ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:55.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d9858c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d900ccb80 con 0x7f5d900adad0 2023-08-27T11:24:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d9858c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d900cce60 con 0x7f5d900cc650 2023-08-27T11:24:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d9858c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d900cd600 con 0x7f5d9015e990 2023-08-27T11:24:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d96328700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5d9015e990 0x7f5d900ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d96b29700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5d900adad0 0x7f5d900cf540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42702/0 (socket says 172.21.15.45:42702) 2023-08-27T11:24:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d96b29700 1 -- 172.21.15.45:0/2537852951 learned_addr learned my addr 172.21.15.45:0/2537852951 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d96328700 1 -- 172.21.15.45:0/2537852951 >> v1:172.21.15.135:6789/0 conn(0x7f5d900cc650 legacy=0x7f5d900ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d96328700 1 -- 172.21.15.45:0/2537852951 >> v1:172.21.15.45:6789/0 conn(0x7f5d900adad0 legacy=0x7f5d900cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d96328700 1 -- 172.21.15.45:0/2537852951 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d900cd8e0 con 0x7f5d9015e990 2023-08-27T11:24:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d95326700 1 -- 172.21.15.45:0/2537852951 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712303197 0 0) 0x7f5d900ccb80 con 0x7f5d900adad0 2023-08-27T11:24:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.659+0000 7f5d96328700 1 --2- 172.21.15.45:0/2537852951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d9015e990 0x7f5d900ced40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009840 tx=0x7f5d8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a3140952666b3c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d95326700 1 -- 172.21.15.45:0/2537852951 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c004230 con 0x7f5d9015e990 2023-08-27T11:24:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d95326700 1 -- 172.21.15.45:0/2537852951 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c004390 con 0x7f5d9015e990 2023-08-27T11:24:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d95326700 1 -- 172.21.15.45:0/2537852951 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0036a0 con 0x7f5d9015e990 2023-08-27T11:24:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d9858c700 1 -- 172.21.15.45:0/2537852951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d9015e990 msgr2=0x7f5d900ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d9858c700 1 --2- 172.21.15.45:0/2537852951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d9015e990 0x7f5d900ced40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009840 tx=0x7f5d8c009b50 comp rx=0 tx=0).stop 2023-08-27T11:24:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d9858c700 1 -- 172.21.15.45:0/2537852951 shutdown_connections 2023-08-27T11:24:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d9858c700 1 --2- 172.21.15.45:0/2537852951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d9015e990 0x7f5d900ced40 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d9858c700 1 -- 172.21.15.45:0/2537852951 >> 172.21.15.45:0/2537852951 conn(0x7f5d900d0f60 msgr2=0x7f5d900d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d9858c700 1 -- 172.21.15.45:0/2537852951 shutdown_connections 2023-08-27T11:24:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.660+0000 7f5d9858c700 1 -- 172.21.15.45:0/2537852951 wait complete. 2023-08-27T11:24:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d9858c700 1 Processor -- start 2023-08-27T11:24:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d9858c700 1 -- start start 2023-08-27T11:24:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d9858c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d900adad0 0x7f5d901f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d9858c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d900cc650 0x7f5d900aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d9858c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d9015e990 0x7f5d900af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d9858c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d90158c50 con 0x7f5d900adad0 2023-08-27T11:24:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d9858c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d90155a20 con 0x7f5d900cc650 2023-08-27T11:24:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d9858c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d901580d0 con 0x7f5d9015e990 2023-08-27T11:24:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d96328700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d900adad0 0x7f5d901f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d96328700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d900adad0 0x7f5d901f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55930/0 (socket says 172.21.15.45:55930) 2023-08-27T11:24:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.662+0000 7f5d96328700 1 -- 172.21.15.45:0/2004075976 learned_addr learned my addr 172.21.15.45:0/2004075976 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d95b27700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d900cc650 0x7f5d900aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d96b29700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d9015e990 0x7f5d900af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d96328700 1 -- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d9015e990 msgr2=0x7f5d900af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d96328700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d9015e990 0x7f5d900af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d96328700 1 -- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d900cc650 msgr2=0x7f5d900aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d96328700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d900cc650 0x7f5d900aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d96328700 1 -- 172.21.15.45:0/2004075976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c0094a0 con 0x7f5d900adad0 2023-08-27T11:24:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d95b27700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d900cc650 0x7f5d900aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d96b29700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d9015e990 0x7f5d900af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d96328700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d900adad0 0x7f5d901f2cf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0090b0 tx=0x7f5d8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d86ffd700 1 -- 172.21.15.45:0/2004075976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c011830 con 0x7f5d900adad0 2023-08-27T11:24:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d86ffd700 1 -- 172.21.15.45:0/2004075976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c01e070 con 0x7f5d900adad0 2023-08-27T11:24:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d901583b0 con 0x7f5d900adad0 2023-08-27T11:24:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d86ffd700 1 -- 172.21.15.45:0/2004075976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c011bb0 con 0x7f5d900adad0 2023-08-27T11:24:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.663+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d90158970 con 0x7f5d900adad0 2023-08-27T11:24:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.664+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d900b3e70 con 0x7f5d900adad0 2023-08-27T11:24:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.665+0000 7f5d86ffd700 1 -- 172.21.15.45:0/2004075976 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5d8c011d10 con 0x7f5d900adad0 2023-08-27T11:24:55.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.665+0000 7f5d86ffd700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f5d7406a200 0x7f5d7406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:55.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.665+0000 7f5d86ffd700 1 -- 172.21.15.45:0/2004075976 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5d8c089d10 con 0x7f5d900adad0 2023-08-27T11:24:55.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.665+0000 7f5d95b27700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f5d7406a200 0x7f5d7406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:55.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.666+0000 7f5d95b27700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f5d7406a200 0x7f5d7406c6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5d80006fd0 tx=0x7f5d80008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:55.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.669+0000 7f5d86ffd700 1 -- 172.21.15.45:0/2004075976 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5d8c055090 con 0x7f5d900adad0 2023-08-27T11:24:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:55.952+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f5d900cfc50 con 0x7f5d900adad0 2023-08-27T11:24:56.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.511+0000 7f5d86ffd700 1 -- 172.21.15.45:0/2004075976 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v473) v1 ==== 157+0+0 (secure 0 0 0) 0x7f5d8c0586b0 con 0x7f5d900adad0 2023-08-27T11:24:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f5d7406a200 msgr2=0x7f5d7406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f5d7406a200 0x7f5d7406c6c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5d80006fd0 tx=0x7f5d80008040 comp rx=0 tx=0).stop 2023-08-27T11:24:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d900adad0 msgr2=0x7f5d901f2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d900adad0 0x7f5d901f2cf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0090b0 tx=0x7f5d8c004750 comp rx=0 tx=0).stop 2023-08-27T11:24:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 shutdown_connections 2023-08-27T11:24:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f5d7406a200 0x7f5d7406c6c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d900adad0 0x7f5d901f2cf0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d9015e990 0x7f5d900af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 --2- 172.21.15.45:0/2004075976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d900cc650 0x7f5d900aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 >> 172.21.15.45:0/2004075976 conn(0x7f5d900d0f60 msgr2=0x7f5d900cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:56.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 shutdown_connections 2023-08-27T11:24:56.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.514+0000 7f5d9858c700 1 -- 172.21.15.45:0/2004075976 wait complete. 2023-08-27T11:24:56.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:24:56.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.661+0000 7fc729f12700 1 Processor -- start 2023-08-27T11:24:56.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.661+0000 7fc729f12700 1 -- start start 2023-08-27T11:24:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.662+0000 7fc729f12700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc7240cbe70 0x7fc7240cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.662+0000 7fc729f12700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72406d690 con 0x7fc72415a240 2023-08-27T11:24:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.662+0000 7fc729f12700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240cc440 con 0x7fc7240ceba0 2023-08-27T11:24:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.662+0000 7fc729f12700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72406d040 con 0x7fc7240cbe70 2023-08-27T11:24:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.662+0000 7fc7237fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc72415a240 0x7fc7240ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42714/0 (socket says 172.21.15.45:42714) 2023-08-27T11:24:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.662+0000 7fc7237fe700 1 -- 172.21.15.45:0/2024444702 learned_addr learned my addr 172.21.15.45:0/2024444702 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.662+0000 7fc723fff700 1 --2- 172.21.15.45:0/2024444702 >> v2:172.21.15.135:3300/0 conn(0x7fc7240cbe70 0x7fc7240cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.662+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7197532 0 0) 0x7fc72406d690 con 0x7fc72415a240 2023-08-27T11:24:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.662+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7100035b0 con 0x7fc72415a240 2023-08-27T11:24:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2116074292 0 0) 0x7fc7100035b0 con 0x7fc72415a240 2023-08-27T11:24:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 >> v2:172.21.15.135:3300/0 conn(0x7fc7240cbe70 msgr2=0x7fc7240cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 --2- 172.21.15.45:0/2024444702 >> v2:172.21.15.135:3300/0 conn(0x7fc7240cbe70 0x7fc7240cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 >> v1:172.21.15.135:6789/0 conn(0x7fc7240ceba0 legacy=0x7fc72406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72406d320 con 0x7fc72415a240 2023-08-27T11:24:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fc714001670 con 0x7fc72415a240 2023-08-27T11:24:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 >> v1:172.21.15.45:6789/0 conn(0x7fc72415a240 legacy=0x7fc7240ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc710003a00 0x7fc710003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:56.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc710004850 0x7fc710004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:56.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc71000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:56.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72406d040 con 0x7fc710005210 2023-08-27T11:24:56.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71000c0b0 con 0x7fc710003a00 2023-08-27T11:24:56.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71000c270 con 0x7fc710004850 2023-08-27T11:24:56.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc723fff700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc71000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:56.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc7237fe700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc710003a00 0x7fc710003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:56.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc722ffd700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc710004850 0x7fc710004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:56.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc723fff700 1 -- 172.21.15.45:0/2024444702 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc710004850 msgr2=0x7fc710004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:56.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc723fff700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc710004850 0x7fc710004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc723fff700 1 -- 172.21.15.45:0/2024444702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc710003a00 msgr2=0x7fc710003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:56.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc723fff700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc710003a00 0x7fc710003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc723fff700 1 -- 172.21.15.45:0/2024444702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7100035b0 con 0x7fc710005210 2023-08-27T11:24:56.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc723fff700 1 -- 172.21.15.45:0/2024444702 reap_dead start 2023-08-27T11:24:56.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.663+0000 7fc723fff700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc71000ba10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009fc0 tx=0x7fc710005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43566e42b19d411 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:56.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c003a20 con 0x7fc710005210 2023-08-27T11:24:56.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c00f040 con 0x7fc710005210 2023-08-27T11:24:56.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc7227fc700 1 -- 172.21.15.45:0/2024444702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c003de0 con 0x7fc710005210 2023-08-27T11:24:56.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc729f12700 1 -- 172.21.15.45:0/2024444702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 msgr2=0x7fc71000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:56.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc729f12700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc71000ba10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009fc0 tx=0x7fc710005880 comp rx=0 tx=0).stop 2023-08-27T11:24:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc729f12700 1 -- 172.21.15.45:0/2024444702 shutdown_connections 2023-08-27T11:24:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc729f12700 1 --2- 172.21.15.45:0/2024444702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc71000ba10 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc729f12700 1 -- 172.21.15.45:0/2024444702 >> 172.21.15.45:0/2024444702 conn(0x7fc7240cfe50 msgr2=0x7fc7240d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc729f12700 1 -- 172.21.15.45:0/2024444702 shutdown_connections 2023-08-27T11:24:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.664+0000 7fc729f12700 1 -- 172.21.15.45:0/2024444702 wait complete. 2023-08-27T11:24:56.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.665+0000 7fc729f12700 1 Processor -- start 2023-08-27T11:24:56.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.666+0000 7fc729f12700 1 -- start start 2023-08-27T11:24:56.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.666+0000 7fc729f12700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc7241ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:56.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.666+0000 7fc729f12700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc72415b030 0x7fc72415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:56.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.666+0000 7fc729f12700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc72415b4b0 0x7fc72415c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:56.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.666+0000 7fc729f12700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72414bd80 con 0x7fc710005210 2023-08-27T11:24:56.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.666+0000 7fc729f12700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72415b9f0 con 0x7fc72415b030 2023-08-27T11:24:56.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.666+0000 7fc729f12700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72415bcd0 con 0x7fc72415b4b0 2023-08-27T11:24:56.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.666+0000 7fc7237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc7241ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:56.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc7237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc7241ef1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55950/0 (socket says 172.21.15.45:55950) 2023-08-27T11:24:56.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc7237fe700 1 -- 172.21.15.45:0/1245412649 learned_addr learned my addr 172.21.15.45:0/1245412649 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:56.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc722ffd700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc72415b030 0x7fc72415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc723fff700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc72415b4b0 0x7fc72415c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc7237fe700 1 -- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc72415b4b0 msgr2=0x7fc72415c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc7237fe700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc72415b4b0 0x7fc72415c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc7237fe700 1 -- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc72415b030 msgr2=0x7fc72415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc7237fe700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc72415b030 0x7fc72415c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc7237fe700 1 -- 172.21.15.45:0/1245412649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240c3a20 con 0x7fc710005210 2023-08-27T11:24:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc722ffd700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc72415b030 0x7fc72415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc7237fe700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc7241ef1b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc714004aa0 tx=0x7fc714004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc71bfff700 1 -- 172.21.15.45:0/1245412649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714011a70 con 0x7fc710005210 2023-08-27T11:24:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc71bfff700 1 -- 172.21.15.45:0/1245412649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc714011bd0 con 0x7fc710005210 2023-08-27T11:24:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7240c3d00 con 0x7fc710005210 2023-08-27T11:24:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc71bfff700 1 -- 172.21.15.45:0/1245412649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714014370 con 0x7fc710005210 2023-08-27T11:24:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.667+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72414c050 con 0x7fc710005210 2023-08-27T11:24:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.668+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc724072810 con 0x7fc710005210 2023-08-27T11:24:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.669+0000 7fc71bfff700 1 -- 172.21.15.45:0/1245412649 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc7140144d0 con 0x7fc710005210 2023-08-27T11:24:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.669+0000 7fc71bfff700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc70406a200 0x7fc70406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.669+0000 7fc71bfff700 1 -- 172.21.15.45:0/1245412649 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc71408aa40 con 0x7fc710005210 2023-08-27T11:24:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.669+0000 7fc722ffd700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc70406a200 0x7fc70406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.670+0000 7fc722ffd700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc70406a200 0x7fc70406c6c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000c00 tx=0x7fc70c009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.673+0000 7fc71bfff700 1 -- 172.21.15.45:0/1245412649 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc71408ce60 con 0x7fc710005210 2023-08-27T11:24:56.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.876+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7240734a0 con 0x7fc710005210 2023-08-27T11:24:56.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.877+0000 7fc71bfff700 1 -- 172.21.15.45:0/1245412649 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16774 (secure 0 0 0) 0x7fc714055d30 con 0x7fc710005210 2023-08-27T11:24:56.879 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:56.879 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":61,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:24:53.001841+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/2587284130":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/1110669507":"2023-08-28T11:13:52.047816+0000","172.21.15.135:6833/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:6832/4120550306":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/4257866035":"2023-08-28T11:24:48.301631+0000","172.21.15.135:6832/584312721":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/1692978501":"2023-08-28T11:13:52.047816+0000","172.21.15.45:6835/2587151358":"2023-08-28T11:24:50.488702+0000","172.21.15.135:0/2898860171":"2023-08-28T11:13:52.047816+0000","172.21.15.135:0/918212180":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/2208777569":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/1050128524":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/1902421107":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/459759971":"2023-08-28T11:24:48.301631+0000","172.21.15.135:0/3750381439":"2023-08-28T11:24:48.301631+0000","172.21.15.135:6833/584312721":"2023-08-28T11:24:48.301631+0000","172.21.15.45:6833/2587151358":"2023-08-28T11:24:50.488702+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":4,"length":2},{"begin":8,"length":2},{"begin":12,"length":1},{"begin":14,"length":2},{"begin":18,"length":2},{"begin":23,"length":1},{"begin":25,"length":2},{"begin":29,"length":2},{"begin":33,"length":1},{"begin":35,"length":2},{"begin":39,"length":3},{"begin":44,"length":1},{"begin":46,"length":2},{"begin":50,"length":2},{"begin":54,"length":1},{"begin":56,"length":2},{"begin":60,"length":2},{"begin":64,"length":1},{"begin":66,"length":2},{"begin":70,"length":3},{"begin":75,"length":1},{"begin":77,"length":2},{"begin":81,"length":2},{"begin":85,"length":1},{"begin":87,"length":2},{"begin":92,"length":2},{"begin":96,"length":1},{"begin":98,"length":2},{"begin":102,"length":3},{"begin":108,"length":2},{"begin":165,"length":54}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:24:56.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc70406a200 msgr2=0x7fc70406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:56.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc70406a200 0x7fc70406c6c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000c00 tx=0x7fc70c009040 comp rx=0 tx=0).stop 2023-08-27T11:24:56.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 msgr2=0x7fc7241ef1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:56.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc7241ef1b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc714004aa0 tx=0x7fc714004e60 comp rx=0 tx=0).stop 2023-08-27T11:24:56.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 shutdown_connections 2023-08-27T11:24:56.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc70406a200 0x7fc70406c6c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc710005210 0x7fc7241ef1b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc72415b4b0 0x7fc72415c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 --2- 172.21.15.45:0/1245412649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc72415b030 0x7fc72415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 >> 172.21.15.45:0/1245412649 conn(0x7fc7240cfe50 msgr2=0x7fc7240cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 shutdown_connections 2023-08-27T11:24:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:56.879+0000 7fc729f12700 1 -- 172.21.15.45:0/1245412649 wait complete. 2023-08-27T11:24:56.893 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:24:56.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:24:57.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.021+0000 7f9444994700 1 Processor -- start 2023-08-27T11:24:57.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.021+0000 7f9444994700 1 -- start start 2023-08-27T11:24:57.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f9444994700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94400cd540 0x7f94400cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f9444994700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94400cf240 0x7f94400cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f9444994700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400cb9b0 con 0x7f944015b2e0 2023-08-27T11:24:57.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f9444994700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94400cfc10 con 0x7f94400cf240 2023-08-27T11:24:57.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f9444994700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400ce2e0 con 0x7f94400cd540 2023-08-27T11:24:57.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94400cf240 0x7f94400cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94400cf240 0x7f94400cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55962/0 (socket says 172.21.15.45:55962) 2023-08-27T11:24:57.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943ed9d700 1 -- 172.21.15.45:0/3257001838 learned_addr learned my addr 172.21.15.45:0/3257001838 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:57.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943dd9b700 1 --2- 172.21.15.45:0/3257001838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94400cd540 0x7f94400cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943ed9d700 1 -- 172.21.15.45:0/3257001838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94400cd540 msgr2=0x7f94400cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:57.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943ed9d700 1 --2- 172.21.15.45:0/3257001838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94400cd540 0x7f94400cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943ed9d700 1 -- 172.21.15.45:0/3257001838 >> v1:172.21.15.135:6789/0 conn(0x7f944015b2e0 legacy=0x7f94400cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:57.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943ed9d700 1 -- 172.21.15.45:0/3257001838 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400cc230 con 0x7f94400cf240 2023-08-27T11:24:57.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943dd9b700 1 --2- 172.21.15.45:0/3257001838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94400cd540 0x7f94400cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:57.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943ed9d700 1 --2- 172.21.15.45:0/3257001838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 0x7f94400cf6d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9438009e50 tx=0x7f943800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4467fdf2fd040e02 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:57.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.022+0000 7f943d59a700 1 -- 172.21.15.45:0/3257001838 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943800e070 con 0x7f94400cf240 2023-08-27T11:24:57.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f943d59a700 1 -- 172.21.15.45:0/3257001838 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438003a20 con 0x7f94400cf240 2023-08-27T11:24:57.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f943d59a700 1 -- 172.21.15.45:0/3257001838 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438003dd0 con 0x7f94400cf240 2023-08-27T11:24:57.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f9444994700 1 -- 172.21.15.45:0/3257001838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 msgr2=0x7f94400cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f9444994700 1 --2- 172.21.15.45:0/3257001838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 0x7f94400cf6d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9438009e50 tx=0x7f943800b330 comp rx=0 tx=0).stop 2023-08-27T11:24:57.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f9444994700 1 -- 172.21.15.45:0/3257001838 shutdown_connections 2023-08-27T11:24:57.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f9444994700 1 --2- 172.21.15.45:0/3257001838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 0x7f94400cf6d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f9444994700 1 --2- 172.21.15.45:0/3257001838 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94400cd540 0x7f94400cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f9444994700 1 -- 172.21.15.45:0/3257001838 >> 172.21.15.45:0/3257001838 conn(0x7f944009a1a0 msgr2=0x7f94400d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f9444994700 1 -- 172.21.15.45:0/3257001838 shutdown_connections 2023-08-27T11:24:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.023+0000 7f9444994700 1 -- 172.21.15.45:0/3257001838 wait complete. 2023-08-27T11:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f9444994700 1 Processor -- start 2023-08-27T11:24:57.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f9444994700 1 -- start start 2023-08-27T11:24:57.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f9444994700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94400cd540 0x7f9440152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f9444994700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 0x7f94400c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f9444994700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944015b2e0 0x7f944006e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f9444994700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400b1810 con 0x7f94400cf240 2023-08-27T11:24:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f9444994700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400c7240 con 0x7f94400cd540 2023-08-27T11:24:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f9444994700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400c7520 con 0x7f944015b2e0 2023-08-27T11:24:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f943dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 0x7f94400c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f943dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 0x7f94400c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55966/0 (socket says 172.21.15.45:55966) 2023-08-27T11:24:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.025+0000 7f943dd9b700 1 -- 172.21.15.45:0/4085734023 learned_addr learned my addr 172.21.15.45:0/4085734023 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f943ed9d700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944015b2e0 0x7f944006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f943e59c700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94400cd540 0x7f9440152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f943dd9b700 1 -- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944015b2e0 msgr2=0x7f944006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f943dd9b700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944015b2e0 0x7f944006e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f943dd9b700 1 -- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94400cd540 msgr2=0x7f9440152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f943dd9b700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94400cd540 0x7f9440152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f943dd9b700 1 -- 172.21.15.45:0/4085734023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9438009b30 con 0x7f94400cf240 2023-08-27T11:24:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f943e59c700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94400cd540 0x7f9440152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f943dd9b700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 0x7f94400c68f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f943400c7f0 tx=0x7f943400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f942effd700 1 -- 172.21.15.45:0/4085734023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434011800 con 0x7f94400cf240 2023-08-27T11:24:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f942effd700 1 -- 172.21.15.45:0/4085734023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434011960 con 0x7f94400cf240 2023-08-27T11:24:57.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94400c7800 con 0x7f94400cf240 2023-08-27T11:24:57.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f942effd700 1 -- 172.21.15.45:0/4085734023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434011d10 con 0x7f94400cf240 2023-08-27T11:24:57.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.026+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94401526d0 con 0x7f94400cf240 2023-08-27T11:24:57.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.027+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94400b1980 con 0x7f94400cf240 2023-08-27T11:24:57.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.028+0000 7f942effd700 1 -- 172.21.15.45:0/4085734023 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9434018360 con 0x7f94400cf240 2023-08-27T11:24:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.028+0000 7f942effd700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f942006a200 0x7f942006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.028+0000 7f942effd700 1 -- 172.21.15.45:0/4085734023 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9434088a00 con 0x7f94400cf240 2023-08-27T11:24:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.028+0000 7f943e59c700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f942006a200 0x7f942006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.029+0000 7f943e59c700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f942006a200 0x7f942006c6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f94400c9fd0 tx=0x7f9430009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.033+0000 7f942effd700 1 -- 172.21.15.45:0/4085734023 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f94400b1980 con 0x7f94400cf240 2023-08-27T11:24:57.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.235+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94400b1950 con 0x7f94400cf240 2023-08-27T11:24:57.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.235+0000 7f942effd700 1 -- 172.21.15.45:0/4085734023 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9434057700 con 0x7f94400cf240 2023-08-27T11:24:57.238 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:24:57.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.237+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f942006a200 msgr2=0x7f942006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.237+0000 7f9444994700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f942006a200 0x7f942006c6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f94400c9fd0 tx=0x7f9430009040 comp rx=0 tx=0).stop 2023-08-27T11:24:57.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.237+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 msgr2=0x7f94400c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.237+0000 7f9444994700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 0x7f94400c68f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f943400c7f0 tx=0x7f943400cbb0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.237+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 shutdown_connections 2023-08-27T11:24:57.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.237+0000 7f9444994700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f942006a200 0x7f942006c6c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.237+0000 7f9444994700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f944015b2e0 0x7f944006e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.237+0000 7f9444994700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94400cd540 0x7f9440152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.237+0000 7f9444994700 1 --2- 172.21.15.45:0/4085734023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94400cf240 0x7f94400c68f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.238+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 >> 172.21.15.45:0/4085734023 conn(0x7f944009a1a0 msgr2=0x7f94400ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:57.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.238+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 shutdown_connections 2023-08-27T11:24:57.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.238+0000 7f9444994700 1 -- 172.21.15.45:0/4085734023 wait complete. 2023-08-27T11:24:57.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:24:57.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.378+0000 7f1a20d08700 1 Processor -- start 2023-08-27T11:24:57.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.378+0000 7f1a20d08700 1 -- start start 2023-08-27T11:24:57.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a20d08700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1a1c0af200 0x7f1a1c0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a20d08700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1a1c0caa00 0x7f1a1c0acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a20d08700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1c0ce340 con 0x7f1a1c0ca620 2023-08-27T11:24:57.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a20d08700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1c0ceec0 con 0x7f1a1c0caa00 2023-08-27T11:24:57.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a20d08700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1c15d7c0 con 0x7f1a1c0af200 2023-08-27T11:24:57.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1a1c0caa00 0x7f1a1c0acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1a59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1a1c0ca620 0x7f1a1c0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42748/0 (socket says 172.21.15.45:42748) 2023-08-27T11:24:57.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1a59c700 1 -- 172.21.15.45:0/1118660248 learned_addr learned my addr 172.21.15.45:0/1118660248 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:57.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a19d9b700 1 --2- 172.21.15.45:0/1118660248 >> v2:172.21.15.135:3300/0 conn(0x7f1a1c0af200 0x7f1a1c0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1ad9d700 1 -- 172.21.15.45:0/1118660248 >> v2:172.21.15.135:3300/0 conn(0x7f1a1c0af200 msgr2=0x7f1a1c0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:57.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1ad9d700 1 --2- 172.21.15.45:0/1118660248 >> v2:172.21.15.135:3300/0 conn(0x7f1a1c0af200 0x7f1a1c0cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1ad9d700 1 -- 172.21.15.45:0/1118660248 >> v1:172.21.15.45:6789/0 conn(0x7f1a1c0ca620 legacy=0x7f1a1c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1ad9d700 1 -- 172.21.15.45:0/1118660248 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c0cc570 con 0x7f1a1c0caa00 2023-08-27T11:24:57.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1ad9d700 1 --2- 172.21.15.45:0/1118660248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0caa00 0x7f1a1c0acfc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c06f860 tx=0x7f1a1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e228824e1d86b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:57.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1959a700 1 -- 172.21.15.45:0/1118660248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400e070 con 0x7f1a1c0caa00 2023-08-27T11:24:57.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.379+0000 7f1a1959a700 1 -- 172.21.15.45:0/1118660248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1400bbd0 con 0x7f1a1c0caa00 2023-08-27T11:24:57.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.380+0000 7f1a1959a700 1 -- 172.21.15.45:0/1118660248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14012430 con 0x7f1a1c0caa00 2023-08-27T11:24:57.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.380+0000 7f1a20d08700 1 -- 172.21.15.45:0/1118660248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0caa00 msgr2=0x7f1a1c0acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.380+0000 7f1a20d08700 1 --2- 172.21.15.45:0/1118660248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0caa00 0x7f1a1c0acfc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c06f860 tx=0x7f1a1400b330 comp rx=0 tx=0).stop 2023-08-27T11:24:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.380+0000 7f1a20d08700 1 -- 172.21.15.45:0/1118660248 shutdown_connections 2023-08-27T11:24:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.380+0000 7f1a20d08700 1 --2- 172.21.15.45:0/1118660248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0caa00 0x7f1a1c0acfc0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.380+0000 7f1a20d08700 1 --2- 172.21.15.45:0/1118660248 >> v2:172.21.15.135:3300/0 conn(0x7f1a1c0af200 0x7f1a1c0cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.380+0000 7f1a20d08700 1 -- 172.21.15.45:0/1118660248 >> 172.21.15.45:0/1118660248 conn(0x7f1a1c0cc7e0 msgr2=0x7f1a1c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.380+0000 7f1a20d08700 1 -- 172.21.15.45:0/1118660248 shutdown_connections 2023-08-27T11:24:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.380+0000 7f1a20d08700 1 -- 172.21.15.45:0/1118660248 wait complete. 2023-08-27T11:24:57.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.381+0000 7f1a20d08700 1 Processor -- start 2023-08-27T11:24:57.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a20d08700 1 -- start start 2023-08-27T11:24:57.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a20d08700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0af200 0x7f1a1c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a20d08700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c0ca620 0x7f1a1c0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a20d08700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1c0caa00 0x7f1a1c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a20d08700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c1f2ee0 con 0x7f1a1c0af200 2023-08-27T11:24:57.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a20d08700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0b3d50 con 0x7f1a1c0ca620 2023-08-27T11:24:57.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a20d08700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0b4010 con 0x7f1a1c0caa00 2023-08-27T11:24:57.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a1a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0af200 0x7f1a1c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a1a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0af200 0x7f1a1c157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55978/0 (socket says 172.21.15.45:55978) 2023-08-27T11:24:57.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a1a59c700 1 -- 172.21.15.45:0/3876545643 learned_addr learned my addr 172.21.15.45:0/3876545643 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:57.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a1ad9d700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1c0caa00 0x7f1a1c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.382+0000 7f1a19d9b700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c0ca620 0x7f1a1c0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a1a59c700 1 -- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1c0caa00 msgr2=0x7f1a1c156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a1a59c700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1c0caa00 0x7f1a1c156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a1a59c700 1 -- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c0ca620 msgr2=0x7f1a1c0b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a1a59c700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c0ca620 0x7f1a1c0b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a1a59c700 1 -- 172.21.15.45:0/3876545643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14009a70 con 0x7f1a1c0af200 2023-08-27T11:24:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a19d9b700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c0ca620 0x7f1a1c0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a1a59c700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0af200 0x7f1a1c157890 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000c520 tx=0x7f1a1000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:57.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a0affd700 1 -- 172.21.15.45:0/3876545643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10005d70 con 0x7f1a1c0af200 2023-08-27T11:24:57.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a0affd700 1 -- 172.21.15.45:0/3876545643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a100070d0 con 0x7f1a1c0af200 2023-08-27T11:24:57.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c0b42f0 con 0x7f1a1c0af200 2023-08-27T11:24:57.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a0affd700 1 -- 172.21.15.45:0/3876545643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10007440 con 0x7f1a1c0af200 2023-08-27T11:24:57.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.383+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1c0b48b0 con 0x7f1a1c0af200 2023-08-27T11:24:57.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.384+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1c0cbce0 con 0x7f1a1c0af200 2023-08-27T11:24:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.384+0000 7f1a0affd700 1 -- 172.21.15.45:0/3876545643 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1a100075a0 con 0x7f1a1c0af200 2023-08-27T11:24:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.385+0000 7f1a0affd700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f19fc06a200 0x7f19fc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.385+0000 7f1a0affd700 1 -- 172.21.15.45:0/3876545643 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1a10087ca0 con 0x7f1a1c0af200 2023-08-27T11:24:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.385+0000 7f1a19d9b700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f19fc06a200 0x7f19fc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.386+0000 7f1a19d9b700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f19fc06a200 0x7f19fc06c6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0ade20 tx=0x7f1a0c008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.389+0000 7f1a0affd700 1 -- 172.21.15.45:0/3876545643 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1a100530d0 con 0x7f1a1c0af200 2023-08-27T11:24:57.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.588+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1a1c0734a0 con 0x7f1a1c0af200 2023-08-27T11:24:57.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.589+0000 7f1a0affd700 1 -- 172.21.15.45:0/3876545643 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1a100566f0 con 0x7f1a1c0af200 2023-08-27T11:24:57.591 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:24:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f19fc06a200 msgr2=0x7f19fc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f19fc06a200 0x7f19fc06c6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0ade20 tx=0x7f1a0c008040 comp rx=0 tx=0).stop 2023-08-27T11:24:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0af200 msgr2=0x7f1a1c157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0af200 0x7f1a1c157890 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000c520 tx=0x7f1a1000c830 comp rx=0 tx=0).stop 2023-08-27T11:24:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 shutdown_connections 2023-08-27T11:24:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f19fc06a200 0x7f19fc06c6c0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0ade20 tx=0x7f1a0c008040 comp rx=0 tx=0).stop 2023-08-27T11:24:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1c0af200 0x7f1a1c157890 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1a1c0caa00 0x7f1a1c156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 --2- 172.21.15.45:0/3876545643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c0ca620 0x7f1a1c0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 >> 172.21.15.45:0/3876545643 conn(0x7f1a1c0cc7e0 msgr2=0x7f1a1c0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 shutdown_connections 2023-08-27T11:24:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.591+0000 7f1a20d08700 1 -- 172.21.15.45:0/3876545643 wait complete. 2023-08-27T11:24:57.605 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:24:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.732+0000 7f28a06a8700 1 Processor -- start 2023-08-27T11:24:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.732+0000 7f28a06a8700 1 -- start start 2023-08-27T11:24:57.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f28a06a8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 0x7f28980ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f28a06a8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28980d0520 0x7f289815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f28a06a8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289815ec50 con 0x7f28980cb5c0 2023-08-27T11:24:57.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f28a06a8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289815c620 con 0x7f28980d0520 2023-08-27T11:24:57.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f28a06a8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289815eea0 con 0x7f28980b65d0 2023-08-27T11:24:57.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289ec45700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28980d0520 0x7f289815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289ec45700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28980d0520 0x7f289815c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55990/0 (socket says 172.21.15.45:55990) 2023-08-27T11:24:57.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289ec45700 1 -- 172.21.15.45:0/3829780877 learned_addr learned my addr 172.21.15.45:0/3829780877 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:57.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289e444700 1 --2- 172.21.15.45:0/3829780877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 0x7f28980ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289ec45700 1 -- 172.21.15.45:0/3829780877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 msgr2=0x7f28980ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:57.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289ec45700 1 --2- 172.21.15.45:0/3829780877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 0x7f28980ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289ec45700 1 -- 172.21.15.45:0/3829780877 >> v1:172.21.15.135:6789/0 conn(0x7f28980cb5c0 legacy=0x7f28980cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:57.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289ec45700 1 -- 172.21.15.45:0/3829780877 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289815f180 con 0x7f28980d0520 2023-08-27T11:24:57.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289ec45700 1 --2- 172.21.15.45:0/3829780877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980d0520 0x7f289815c0e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2890009e50 tx=0x7f289000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4c947b141e7bdb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:57.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289d442700 1 -- 172.21.15.45:0/3829780877 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289000e070 con 0x7f28980d0520 2023-08-27T11:24:57.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289d442700 1 -- 172.21.15.45:0/3829780877 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890003a20 con 0x7f28980d0520 2023-08-27T11:24:57.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.733+0000 7f289d442700 1 -- 172.21.15.45:0/3829780877 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890003dd0 con 0x7f28980d0520 2023-08-27T11:24:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.734+0000 7f28a06a8700 1 -- 172.21.15.45:0/3829780877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980d0520 msgr2=0x7f289815c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.734+0000 7f28a06a8700 1 --2- 172.21.15.45:0/3829780877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980d0520 0x7f289815c0e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2890009e50 tx=0x7f289000b330 comp rx=0 tx=0).stop 2023-08-27T11:24:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.734+0000 7f28a06a8700 1 -- 172.21.15.45:0/3829780877 shutdown_connections 2023-08-27T11:24:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.734+0000 7f28a06a8700 1 --2- 172.21.15.45:0/3829780877 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 0x7f28980ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.734+0000 7f28a06a8700 1 --2- 172.21.15.45:0/3829780877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980d0520 0x7f289815c0e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.734+0000 7f28a06a8700 1 -- 172.21.15.45:0/3829780877 >> 172.21.15.45:0/3829780877 conn(0x7f289806f960 msgr2=0x7f289806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:57.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.734+0000 7f28a06a8700 1 -- 172.21.15.45:0/3829780877 shutdown_connections 2023-08-27T11:24:57.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.734+0000 7f28a06a8700 1 -- 172.21.15.45:0/3829780877 wait complete. 2023-08-27T11:24:57.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.735+0000 7f28a06a8700 1 Processor -- start 2023-08-27T11:24:57.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.735+0000 7f28a06a8700 1 -- start start 2023-08-27T11:24:57.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f28a06a8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 0x7f28981f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f28a06a8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980cb5c0 0x7f2898153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f28a06a8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28980d0520 0x7f2898154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f28a06a8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980c82e0 con 0x7f28980cb5c0 2023-08-27T11:24:57.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f28a06a8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289814abe0 con 0x7f28980d0520 2023-08-27T11:24:57.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f28a06a8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28980c7f40 con 0x7f28980b65d0 2023-08-27T11:24:57.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289dc43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980cb5c0 0x7f2898153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289dc43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980cb5c0 0x7f2898153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55994/0 (socket says 172.21.15.45:55994) 2023-08-27T11:24:57.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289dc43700 1 -- 172.21.15.45:0/981656315 learned_addr learned my addr 172.21.15.45:0/981656315 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:57.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289e444700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 0x7f28981f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289ec45700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28980d0520 0x7f2898154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289dc43700 1 -- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 msgr2=0x7f28981f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289dc43700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 0x7f28981f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289dc43700 1 -- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28980d0520 msgr2=0x7f2898154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289dc43700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28980d0520 0x7f2898154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289dc43700 1 -- 172.21.15.45:0/981656315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890009b30 con 0x7f28980cb5c0 2023-08-27T11:24:57.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289ec45700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28980d0520 0x7f2898154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.736+0000 7f289e444700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 0x7f28981f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:24:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.737+0000 7f289dc43700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980cb5c0 0x7f2898153dd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f288800c500 tx=0x7f288800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.737+0000 7f288effd700 1 -- 172.21.15.45:0/981656315 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888011790 con 0x7f28980cb5c0 2023-08-27T11:24:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.737+0000 7f288effd700 1 -- 172.21.15.45:0/981656315 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28880118f0 con 0x7f28980cb5c0 2023-08-27T11:24:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.737+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28981f34a0 con 0x7f28980cb5c0 2023-08-27T11:24:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.737+0000 7f288effd700 1 -- 172.21.15.45:0/981656315 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888011ca0 con 0x7f28980cb5c0 2023-08-27T11:24:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.737+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28981f3a60 con 0x7f28980cb5c0 2023-08-27T11:24:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.738+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289806f4a0 con 0x7f28980cb5c0 2023-08-27T11:24:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.738+0000 7f288effd700 1 -- 172.21.15.45:0/981656315 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f288801c460 con 0x7f28980cb5c0 2023-08-27T11:24:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.739+0000 7f288effd700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f287c06a200 0x7f287c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.739+0000 7f288effd700 1 -- 172.21.15.45:0/981656315 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2888088750 con 0x7f28980cb5c0 2023-08-27T11:24:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.739+0000 7f289e444700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f287c06a200 0x7f287c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.740+0000 7f289e444700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f287c06a200 0x7f287c06c6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f289400aa50 tx=0x7f2894008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.743+0000 7f288effd700 1 -- 172.21.15.45:0/981656315 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2888053ad0 con 0x7f28980cb5c0 2023-08-27T11:24:57.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.943+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f28980734a0 con 0x7f28980cb5c0 2023-08-27T11:24:57.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.947+0000 7f288effd700 1 -- 172.21.15.45:0/981656315 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f28880570f0 con 0x7f28980cb5c0 2023-08-27T11:24:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f287c06a200 msgr2=0x7f287c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f287c06a200 0x7f287c06c6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f289400aa50 tx=0x7f2894008040 comp rx=0 tx=0).stop 2023-08-27T11:24:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980cb5c0 msgr2=0x7f2898153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:57.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980cb5c0 0x7f2898153dd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f288800c500 tx=0x7f288800c8c0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 shutdown_connections 2023-08-27T11:24:57.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f287c06a200 0x7f287c06c6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28980d0520 0x7f2898154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28980b65d0 0x7f28981f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 --2- 172.21.15.45:0/981656315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28980cb5c0 0x7f2898153dd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:57.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 >> 172.21.15.45:0/981656315 conn(0x7f289806f960 msgr2=0x7f28980cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:57.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 shutdown_connections 2023-08-27T11:24:57.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:57.949+0000 7f28a06a8700 1 -- 172.21.15.45:0/981656315 wait complete. 2023-08-27T11:24:57.963 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:24:57.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:24:58.008 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:24:58.059 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:24:58.059 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:24:58.060 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:24:58.078 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:24:58.078 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.097 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:24:58.098 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:24:58.098 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.134 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:24:58.135 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:24:58.135 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:24:58.156 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:24:58.156 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:24:58.157 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:24:58.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.297+0000 7f579e74f700 1 Processor -- start 2023-08-27T11:24:58.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.297+0000 7f579e74f700 1 -- start start 2023-08-27T11:24:58.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579e74f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57980ceba0 0x7f57980cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:58.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579e74f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f57980cbe70 0x7f579806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:58.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579e74f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579806d690 con 0x7f579815a240 2023-08-27T11:24:58.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579e74f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57980cc440 con 0x7f57980cbe70 2023-08-27T11:24:58.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579e74f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579806d040 con 0x7f57980ceba0 2023-08-27T11:24:58.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f5797fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f579815a240 0x7f57980ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42776/0 (socket says 172.21.15.45:42776) 2023-08-27T11:24:58.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f5797fff700 1 -- 172.21.15.45:0/2258408989 learned_addr learned my addr 172.21.15.45:0/2258408989 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:24:58.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579ccec700 1 --2- 172.21.15.45:0/2258408989 >> v2:172.21.15.135:3300/0 conn(0x7f57980cbe70 0x7f579806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:58.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f57977fe700 1 --2- 172.21.15.45:0/2258408989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57980ceba0 0x7f57980cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:58.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f5796ffd700 1 -- 172.21.15.45:0/2258408989 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899651910 0 0) 0x7f579806d690 con 0x7f579815a240 2023-08-27T11:24:58.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f5796ffd700 1 -- 172.21.15.45:0/2258408989 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57800035b0 con 0x7f579815a240 2023-08-27T11:24:58.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579ccec700 1 -- 172.21.15.45:0/2258408989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57980ceba0 msgr2=0x7f57980cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:24:58.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579ccec700 1 --2- 172.21.15.45:0/2258408989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57980ceba0 0x7f57980cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:58.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579ccec700 1 -- 172.21.15.45:0/2258408989 >> v1:172.21.15.45:6789/0 conn(0x7f579815a240 legacy=0x7f57980ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:58.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f5796ffd700 1 -- 172.21.15.45:0/2258408989 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3138760429 0 0) 0x7f57800035b0 con 0x7f579815a240 2023-08-27T11:24:58.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f579ccec700 1 -- 172.21.15.45:0/2258408989 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579806d320 con 0x7f57980cbe70 2023-08-27T11:24:58.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.298+0000 7f57977fe700 1 --2- 172.21.15.45:0/2258408989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57980ceba0 0x7f57980cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:24:58.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f579ccec700 1 --2- 172.21.15.45:0/2258408989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980cbe70 0x7f579806db20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5790009d90 tx=0x7f579000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43366224c1f25797 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:58.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f5796ffd700 1 -- 172.21.15.45:0/2258408989 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000e070 con 0x7f57980cbe70 2023-08-27T11:24:58.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f5796ffd700 1 -- 172.21.15.45:0/2258408989 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5790003a20 con 0x7f57980cbe70 2023-08-27T11:24:58.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f5796ffd700 1 -- 172.21.15.45:0/2258408989 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c001670 con 0x7f57980cbe70 2023-08-27T11:24:58.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f579e74f700 1 -- 172.21.15.45:0/2258408989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980cbe70 msgr2=0x7f579806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:58.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f579e74f700 1 --2- 172.21.15.45:0/2258408989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980cbe70 0x7f579806db20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5790009d90 tx=0x7f579000b330 comp rx=0 tx=0).stop 2023-08-27T11:24:58.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f579e74f700 1 -- 172.21.15.45:0/2258408989 shutdown_connections 2023-08-27T11:24:58.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f579e74f700 1 --2- 172.21.15.45:0/2258408989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980cbe70 0x7f579806db20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:58.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f579e74f700 1 --2- 172.21.15.45:0/2258408989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57980ceba0 0x7f57980cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:58.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f579e74f700 1 -- 172.21.15.45:0/2258408989 >> 172.21.15.45:0/2258408989 conn(0x7f57980cfe50 msgr2=0x7f57980d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:58.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.299+0000 7f579e74f700 1 -- 172.21.15.45:0/2258408989 shutdown_connections 2023-08-27T11:24:58.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.300+0000 7f579e74f700 1 -- 172.21.15.45:0/2258408989 wait complete. 2023-08-27T11:24:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.301+0000 7f579e74f700 1 Processor -- start 2023-08-27T11:24:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.301+0000 7f579e74f700 1 -- start start 2023-08-27T11:24:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f579e74f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57980cbe70 0x7f5798154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:58.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f579e74f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980ceba0 0x7f57980cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:58.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f579e74f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f579815a240 0x7f57980cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:58.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f579e74f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980c9290 con 0x7f57980cbe70 2023-08-27T11:24:58.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f579e74f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980c9570 con 0x7f57980ceba0 2023-08-27T11:24:58.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f579e74f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57980c9850 con 0x7f579815a240 2023-08-27T11:24:58.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5797fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57980cbe70 0x7f5798154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:58.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5797fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57980cbe70 0x7f5798154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56006/0 (socket says 172.21.15.45:56006) 2023-08-27T11:24:58.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5797fff700 1 -- 172.21.15.45:0/1057135330 learned_addr learned my addr 172.21.15.45:0/1057135330 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:24:58.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f57977fe700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980ceba0 0x7f57980cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:58.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f579ccec700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f579815a240 0x7f57980cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:58.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5797fff700 1 -- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f579815a240 msgr2=0x7f57980cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:58.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5797fff700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f579815a240 0x7f57980cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:58.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5797fff700 1 -- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980ceba0 msgr2=0x7f57980cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:58.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5797fff700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980ceba0 0x7f57980cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:58.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5797fff700 1 -- 172.21.15.45:0/1057135330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57880094a0 con 0x7f57980cbe70 2023-08-27T11:24:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f579ccec700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f579815a240 0x7f57980cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f57977fe700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980ceba0 0x7f57980cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:24:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5797fff700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57980cbe70 0x7f5798154f60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c750 tx=0x7f578c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5794ff9700 1 -- 172.21.15.45:0/1057135330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c004c80 con 0x7f57980cbe70 2023-08-27T11:24:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5794ff9700 1 -- 172.21.15.45:0/1057135330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c004de0 con 0x7f57980cbe70 2023-08-27T11:24:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5790009a70 con 0x7f57980cbe70 2023-08-27T11:24:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.302+0000 7f5794ff9700 1 -- 172.21.15.45:0/1057135330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0072f0 con 0x7f57980cbe70 2023-08-27T11:24:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.303+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57981554a0 con 0x7f57980cbe70 2023-08-27T11:24:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.303+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5798072810 con 0x7f57980cbe70 2023-08-27T11:24:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.304+0000 7f5794ff9700 1 -- 172.21.15.45:0/1057135330 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f578c007450 con 0x7f57980cbe70 2023-08-27T11:24:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.304+0000 7f5794ff9700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f578406a200 0x7f578406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:24:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.304+0000 7f5794ff9700 1 -- 172.21.15.45:0/1057135330 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f578c088760 con 0x7f57980cbe70 2023-08-27T11:24:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.305+0000 7f57977fe700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f578406a200 0x7f578406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:24:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.305+0000 7f57977fe700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f578406a200 0x7f578406c6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57980a8850 tx=0x7f5788008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:24:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.309+0000 7f5794ff9700 1 -- 172.21.15.45:0/1057135330 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f578c053ae0 con 0x7f57980cbe70 2023-08-27T11:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.554+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57980734a0 con 0x7f57980cbe70 2023-08-27T11:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.554+0000 7f5794ff9700 1 -- 172.21.15.45:0/1057135330 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f578c007700 con 0x7f57980cbe70 2023-08-27T11:24:58.556 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:24:58.557 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1324661970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1324661970},{"type":"v1","addr":"172.21.15.135:6836","nonce":1324661970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2014390806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2014390806},{"type":"v1","addr":"172.21.15.135:6837","nonce":2014390806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2103056681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2103056681},{"type":"v1","addr":"172.21.15.45:6834","nonce":2103056681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1890307674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1890307674},{"type":"v1","addr":"172.21.15.45:6835","nonce":1890307674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2023-08-27T11:24:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f578406a200 msgr2=0x7f578406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f578406a200 0x7f578406c6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57980a8850 tx=0x7f5788008040 comp rx=0 tx=0).stop 2023-08-27T11:24:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57980cbe70 msgr2=0x7f5798154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:24:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57980cbe70 0x7f5798154f60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c750 tx=0x7f578c00cb10 comp rx=0 tx=0).stop 2023-08-27T11:24:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 shutdown_connections 2023-08-27T11:24:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f578406a200 0x7f578406c6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57980cbe70 0x7f5798154f60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f579815a240 0x7f57980cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 --2- 172.21.15.45:0/1057135330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57980ceba0 0x7f57980cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 >> 172.21.15.45:0/1057135330 conn(0x7f57980cfe50 msgr2=0x7f579806e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 shutdown_connections 2023-08-27T11:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:24:58.556+0000 7f579e74f700 1 -- 172.21.15.45:0/1057135330 wait complete. 2023-08-27T11:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2023-08-27T11:24:58.570 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:24:58.570 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:24:58.571 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:24:58.571 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:24:58.571 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:24:58.572 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:24:58.572 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:24:58.572 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:24:58.572 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:24:58.573 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:24:58.573 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:24:58.573 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:24:58.574 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:24:58.574 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:24:58.574 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:24:58.574 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:24:58.580 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:24:58.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:24:58.582 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:24:58.582 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:24:58.588 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:24:58.589 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:24:58.624 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:24:58.625 DEBUG:teuthology.parallel:result is None 2023-08-27T11:24:58.626 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:24:58.626 DEBUG:teuthology.parallel:result is None 2023-08-27T11:24:58.631 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:24:58.631 DEBUG:teuthology.parallel:result is None 2023-08-27T11:24:58.704 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-08-27T11:24:58.705 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-08-27T11:24:58.717 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2023-08-27T11:25:02.632 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-08-27T11:25:04.588 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:25:04.589 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:25:04.592 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:25:04.592 DEBUG:teuthology.parallel:result is None 2023-08-27T11:25:04.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:25:04.680 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2023-08-27T11:25:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.732+0000 7f6b343d0700 1 Processor -- start 2023-08-27T11:25:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.732+0000 7f6b343d0700 1 -- start start 2023-08-27T11:25:04.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.732+0000 7f6b343d0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c0ca420 0x7f6b2c071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:04.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.732+0000 7f6b343d0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6b2c15ed10 0x7f6b2c15f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:04.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.732+0000 7f6b343d0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c15f680 con 0x7f6b2c144a60 2023-08-27T11:25:04.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.732+0000 7f6b343d0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0cd9a0 con 0x7f6b2c15ed10 2023-08-27T11:25:04.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.732+0000 7f6b343d0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0cdc60 con 0x7f6b2c0ca420 2023-08-27T11:25:04.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3196b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c0ca420 0x7f6b2c071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:04.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3296d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6b2c15ed10 0x7f6b2c15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3196b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c0ca420 0x7f6b2c071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:53954/0 (socket says 172.21.15.45:53954) 2023-08-27T11:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3196b700 1 -- 172.21.15.45:0/1809742568 learned_addr learned my addr 172.21.15.45:0/1809742568 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3296d700 1 -- 172.21.15.45:0/1809742568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c0ca420 msgr2=0x7f6b2c071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3296d700 1 --2- 172.21.15.45:0/1809742568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c0ca420 0x7f6b2c071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:04.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3296d700 1 -- 172.21.15.45:0/1809742568 >> v1:172.21.15.135:6789/0 conn(0x7f6b2c144a60 legacy=0x7f6b2c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3296d700 1 -- 172.21.15.45:0/1809742568 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0cdf40 con 0x7f6b2c15ed10 2023-08-27T11:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3116a700 1 -- 172.21.15.45:0/1809742568 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930288155 0 0) 0x7f6b2c15f680 con 0x7f6b2c144a60 2023-08-27T11:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.733+0000 7f6b3296d700 1 --2- 172.21.15.45:0/1809742568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2c15ed10 0x7f6b2c15f140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009d90 tx=0x7f6b2400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b037daed853a16b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b3116a700 1 -- 172.21.15.45:0/1809742568 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400e070 con 0x7f6b2c15ed10 2023-08-27T11:25:04.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b3116a700 1 -- 172.21.15.45:0/1809742568 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400bb20 con 0x7f6b2c15ed10 2023-08-27T11:25:04.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b3116a700 1 -- 172.21.15.45:0/1809742568 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24012430 con 0x7f6b2c15ed10 2023-08-27T11:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b343d0700 1 -- 172.21.15.45:0/1809742568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2c15ed10 msgr2=0x7f6b2c15f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b343d0700 1 --2- 172.21.15.45:0/1809742568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2c15ed10 0x7f6b2c15f140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009d90 tx=0x7f6b2400b330 comp rx=0 tx=0).stop 2023-08-27T11:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b343d0700 1 -- 172.21.15.45:0/1809742568 shutdown_connections 2023-08-27T11:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b343d0700 1 --2- 172.21.15.45:0/1809742568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2c15ed10 0x7f6b2c15f140 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b343d0700 1 --2- 172.21.15.45:0/1809742568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c0ca420 0x7f6b2c071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b343d0700 1 -- 172.21.15.45:0/1809742568 >> 172.21.15.45:0/1809742568 conn(0x7f6b2c0cfdd0 msgr2=0x7f6b2c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b343d0700 1 -- 172.21.15.45:0/1809742568 shutdown_connections 2023-08-27T11:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.734+0000 7f6b343d0700 1 -- 172.21.15.45:0/1809742568 wait complete. 2023-08-27T11:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.736+0000 7f6b343d0700 1 Processor -- start 2023-08-27T11:25:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.736+0000 7f6b343d0700 1 -- start start 2023-08-27T11:25:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.736+0000 7f6b343d0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2c0ca420 0x7f6b2c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.736+0000 7f6b343d0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c144a60 0x7f6b2c0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.736+0000 7f6b343d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c15ed10 0x7f6b2c1f0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.736+0000 7f6b343d0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c13d2a0 con 0x7f6b2c15ed10 2023-08-27T11:25:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.736+0000 7f6b343d0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0b3010 con 0x7f6b2c0ca420 2023-08-27T11:25:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.736+0000 7f6b343d0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c15bb40 con 0x7f6b2c144a60 2023-08-27T11:25:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.736+0000 7f6b3296d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c15ed10 0x7f6b2c1f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3296d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c15ed10 0x7f6b2c1f0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56052/0 (socket says 172.21.15.45:56052) 2023-08-27T11:25:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3296d700 1 -- 172.21.15.45:0/1872709447 learned_addr learned my addr 172.21.15.45:0/1872709447 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3196b700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c144a60 0x7f6b2c0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:04.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3216c700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2c0ca420 0x7f6b2c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:04.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3296d700 1 -- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c144a60 msgr2=0x7f6b2c0b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:04.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3296d700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c144a60 0x7f6b2c0b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:04.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3296d700 1 -- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2c0ca420 msgr2=0x7f6b2c0b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:04.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3296d700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2c0ca420 0x7f6b2c0b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:04.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3296d700 1 -- 172.21.15.45:0/1872709447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b24009a70 con 0x7f6b2c15ed10 2023-08-27T11:25:04.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b3296d700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c15ed10 0x7f6b2c1f0020 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c072a20 tx=0x7f6b24004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:04.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b1effd700 1 -- 172.21.15.45:0/1872709447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24012430 con 0x7f6b2c15ed10 2023-08-27T11:25:04.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b1effd700 1 -- 172.21.15.45:0/1872709447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24020070 con 0x7f6b2c15ed10 2023-08-27T11:25:04.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c1f0560 con 0x7f6b2c15ed10 2023-08-27T11:25:04.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b1effd700 1 -- 172.21.15.45:0/1872709447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24004280 con 0x7f6b2c15ed10 2023-08-27T11:25:04.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.737+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c0b5900 con 0x7f6b2c15ed10 2023-08-27T11:25:04.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.738+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c15bcb0 con 0x7f6b2c15ed10 2023-08-27T11:25:04.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.739+0000 7f6b1effd700 1 -- 172.21.15.45:0/1872709447 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6b240043e0 con 0x7f6b2c15ed10 2023-08-27T11:25:04.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.739+0000 7f6b1effd700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f6b1006a200 0x7f6b1006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:04.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.739+0000 7f6b1effd700 1 -- 172.21.15.45:0/1872709447 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f6b24022070 con 0x7f6b2c15ed10 2023-08-27T11:25:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.739+0000 7f6b3216c700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f6b1006a200 0x7f6b1006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.740+0000 7f6b3216c700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f6b1006a200 0x7f6b1006c6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b28000f60 tx=0x7f6b28009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.744+0000 7f6b1effd700 1 -- 172.21.15.45:0/1872709447 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6b24055ed0 con 0x7f6b2c15ed10 2023-08-27T11:25:04.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:04.959+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6b2c0727c0 con 0x7f6b2c15ed10 2023-08-27T11:25:06.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.011+0000 7f6b1effd700 1 -- 172.21.15.45:0/1872709447 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6b240594f0 con 0x7f6b2c15ed10 2023-08-27T11:25:06.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f6b1006a200 msgr2=0x7f6b1006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f6b1006a200 0x7f6b1006c6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b28000f60 tx=0x7f6b28009040 comp rx=0 tx=0).stop 2023-08-27T11:25:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c15ed10 msgr2=0x7f6b2c1f0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c15ed10 0x7f6b2c1f0020 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c072a20 tx=0x7f6b24004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 shutdown_connections 2023-08-27T11:25:06.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f6b1006a200 0x7f6b1006c6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c15ed10 0x7f6b2c1f0020 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b2c0ca420 0x7f6b2c0b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 --2- 172.21.15.45:0/1872709447 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6b2c144a60 0x7f6b2c0b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 >> 172.21.15.45:0/1872709447 conn(0x7f6b2c0cfdd0 msgr2=0x7f6b2c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 shutdown_connections 2023-08-27T11:25:06.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.014+0000 7f6b343d0700 1 -- 172.21.15.45:0/1872709447 wait complete. 2023-08-27T11:25:06.018 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-08-27T11:25:06.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:25:06.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.162+0000 7f161f16a700 1 Processor -- start 2023-08-27T11:25:06.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.163+0000 7f161f16a700 1 -- start start 2023-08-27T11:25:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.163+0000 7f161f16a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 0x7f16181619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.163+0000 7f161f16a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16180af040 0x7f16180cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.163+0000 7f161f16a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f161806e3a0 0x7f161806e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.163+0000 7f161f16a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16180cdf90 con 0x7f16180af040 2023-08-27T11:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.163+0000 7f161f16a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161806ed70 con 0x7f161806e3a0 2023-08-27T11:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.163+0000 7f161f16a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1618161550 con 0x7f161815d3d0 2023-08-27T11:25:06.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.163+0000 7f1617fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16180af040 0x7f16180cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f1617fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16180af040 0x7f16180cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56056/0 (socket says 172.21.15.45:56056) 2023-08-27T11:25:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f1617fff700 1 -- 172.21.15.45:0/1121933096 learned_addr learned my addr 172.21.15.45:0/1121933096 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f161cf06700 1 --2- 172.21.15.45:0/1121933096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 0x7f16181619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f161d707700 1 --2- 172.21.15.45:0/1121933096 >> v2:172.21.15.135:3300/0 conn(0x7f161806e3a0 0x7f161806e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f1617fff700 1 -- 172.21.15.45:0/1121933096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 msgr2=0x7f16181619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f1617fff700 1 --2- 172.21.15.45:0/1121933096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 0x7f16181619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f1617fff700 1 -- 172.21.15.45:0/1121933096 >> v2:172.21.15.135:3300/0 conn(0x7f161806e3a0 msgr2=0x7f161806e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f1617fff700 1 --2- 172.21.15.45:0/1121933096 >> v2:172.21.15.135:3300/0 conn(0x7f161806e3a0 0x7f161806e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f1617fff700 1 -- 172.21.15.45:0/1121933096 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1618161f20 con 0x7f16180af040 2023-08-27T11:25:06.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f1617fff700 1 --2- 172.21.15.45:0/1121933096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 0x7f16180cf420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f161806e0b0 tx=0x7f1604009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a789a92ddf6444c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f16177fe700 1 -- 172.21.15.45:0/1121933096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160400b870 con 0x7f16180af040 2023-08-27T11:25:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f16177fe700 1 -- 172.21.15.45:0/1121933096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160400b9d0 con 0x7f16180af040 2023-08-27T11:25:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.164+0000 7f16177fe700 1 -- 172.21.15.45:0/1121933096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160400bd60 con 0x7f16180af040 2023-08-27T11:25:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.165+0000 7f161f16a700 1 -- 172.21.15.45:0/1121933096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 msgr2=0x7f16180cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.165+0000 7f161f16a700 1 --2- 172.21.15.45:0/1121933096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 0x7f16180cf420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f161806e0b0 tx=0x7f1604009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.165+0000 7f161f16a700 1 -- 172.21.15.45:0/1121933096 shutdown_connections 2023-08-27T11:25:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.165+0000 7f161f16a700 1 --2- 172.21.15.45:0/1121933096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 0x7f16181619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.165+0000 7f161f16a700 1 --2- 172.21.15.45:0/1121933096 >> v2:172.21.15.135:3300/0 conn(0x7f161806e3a0 0x7f161806e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.165+0000 7f161f16a700 1 --2- 172.21.15.45:0/1121933096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 0x7f16180cf420 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.165+0000 7f161f16a700 1 -- 172.21.15.45:0/1121933096 >> 172.21.15.45:0/1121933096 conn(0x7f16180cfd60 msgr2=0x7f16180d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.165+0000 7f161f16a700 1 -- 172.21.15.45:0/1121933096 shutdown_connections 2023-08-27T11:25:06.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.165+0000 7f161f16a700 1 -- 172.21.15.45:0/1121933096 wait complete. 2023-08-27T11:25:06.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.166+0000 7f161f16a700 1 Processor -- start 2023-08-27T11:25:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.166+0000 7f161f16a700 1 -- start start 2023-08-27T11:25:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f161f16a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f161806e3a0 0x7f16180c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f161f16a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 0x7f16181f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f161f16a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 0x7f161815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f161f16a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618150ce0 con 0x7f16180af040 2023-08-27T11:25:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f161f16a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16180c8440 con 0x7f161806e3a0 2023-08-27T11:25:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f161f16a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16180c8720 con 0x7f161815d3d0 2023-08-27T11:25:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f1617fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 0x7f16181f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f1617fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 0x7f16181f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56064/0 (socket says 172.21.15.45:56064) 2023-08-27T11:25:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f1617fff700 1 -- 172.21.15.45:0/2518338198 learned_addr learned my addr 172.21.15.45:0/2518338198 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f161d707700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 0x7f161815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f161cf06700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f161806e3a0 0x7f16180c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f1617fff700 1 -- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 msgr2=0x7f161815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f1617fff700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 0x7f161815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f1617fff700 1 -- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f161806e3a0 msgr2=0x7f16180c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f1617fff700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f161806e3a0 0x7f16180c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f1617fff700 1 -- 172.21.15.45:0/2518338198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1604009550 con 0x7f16180af040 2023-08-27T11:25:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.167+0000 7f1617fff700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 0x7f16181f28e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1604000c00 tx=0x7f1604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.168+0000 7f16157fa700 1 -- 172.21.15.45:0/2518338198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604004520 con 0x7f16180af040 2023-08-27T11:25:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.168+0000 7f16157fa700 1 -- 172.21.15.45:0/2518338198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160401e070 con 0x7f16180af040 2023-08-27T11:25:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.168+0000 7f16157fa700 1 -- 172.21.15.45:0/2518338198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16040038c0 con 0x7f16180af040 2023-08-27T11:25:06.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.168+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16180c8a00 con 0x7f16180af040 2023-08-27T11:25:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.168+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16180c8fc0 con 0x7f16180af040 2023-08-27T11:25:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.169+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1618072780 con 0x7f16180af040 2023-08-27T11:25:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.169+0000 7f16157fa700 1 -- 172.21.15.45:0/2518338198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1604003a20 con 0x7f16180af040 2023-08-27T11:25:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.170+0000 7f16157fa700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f15fc06a1d0 0x7f15fc06c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.170+0000 7f16157fa700 1 -- 172.21.15.45:0/2518338198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f1604020070 con 0x7f16180af040 2023-08-27T11:25:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.170+0000 7f161cf06700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f15fc06a1d0 0x7f15fc06c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.170+0000 7f161cf06700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f15fc06a1d0 0x7f15fc06c690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f160c009930 tx=0x7f160c008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.174+0000 7f16157fa700 1 -- 172.21.15.45:0/2518338198 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f16040552b0 con 0x7f16180af040 2023-08-27T11:25:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.454+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f16180734a0 con 0x7f16180af040 2023-08-27T11:25:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.454+0000 7f16157fa700 1 -- 172.21.15.45:0/2518338198 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1604015090 con 0x7f16180af040 2023-08-27T11:25:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:06.456 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:25:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-08-27T11:25:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-08-27T11:25:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tknzGlBhAABVGsUvHGyOdTgH9miKO0Cg==", 2023-08-27T11:25:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:25:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:25:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:25:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-08-27T11:25:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkEWbPExAAgt/9BpPD1vK9QoG8l3HeFw==", 2023-08-27T11:25:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:25:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:25:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:25:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-08-27T11:25:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk6qRCDhAAGaj+NY32IDjdRcuSNbASpw==", 2023-08-27T11:25:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:25:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:25:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:25:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.463 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-08-27T11:25:06.463 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkE9NxGxAAnWum05rSbYQSLZ9IjjqDpQ==", 2023-08-27T11:25:06.464 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.464 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:25:06.465 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:25:06.465 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:25:06.465 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.466 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-08-27T11:25:06.466 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBML+tkOzvBBBAA196QdZiLzcsm5hLBGDl0Qw==", 2023-08-27T11:25:06.466 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.467 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:06.467 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:25:06.467 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.468 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-08-27T11:25:06.468 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBPL+tkIXaxBBAAfHwZ2SsmOENhnNxZsG230Q==", 2023-08-27T11:25:06.468 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.468 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:06.469 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:25:06.469 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.470 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-08-27T11:25:06.470 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBSL+tkpzphBBAA37UzwlmuLdSWDIrkBu/8Rw==", 2023-08-27T11:25:06.470 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.470 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:06.470 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:25:06.471 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.471 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.472 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-08-27T11:25:06.472 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVL+tkt63hAxAAJReGReJHqyggSbdFEhm3+A==", 2023-08-27T11:25:06.473 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.473 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:25:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-08-27T11:25:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBbL+tkTOFXBxAAZ1I44e50IrNzgj7mjKyuGA==", 2023-08-27T11:25:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.476 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:25:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.477 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-08-27T11:25:06.478 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBeL+tkdezFBhAANof2C80WiBZ6hWm3oB3kfg==", 2023-08-27T11:25:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:25:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-08-27T11:25:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBhL+tk7auXBhAAJPwKwTVIUzao5D3t4iXYcw==", 2023-08-27T11:25:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.481 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:25:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-08-27T11:25:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBkL+tkY+ztBRAAvM2oxELLN8WV0i0spqfY6Q==", 2023-08-27T11:25:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:25:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:25:06.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-08-27T11:25:06.484 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ==", 2023-08-27T11:25:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:25:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2023-08-27T11:25:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:25:06.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.486 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-08-27T11:25:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkMCDoJhAAevee0PlTshlWJcPzX8fpYg==", 2023-08-27T11:25:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.487 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:25:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:25:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:25:06.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:25:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-08-27T11:25:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk1xh+IxAAswoUpd+D70hUwfeKI2gc0w==", 2023-08-27T11:25:06.490 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:25:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:25:06.491 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:25:06.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:25:06.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.493 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-08-27T11:25:06.494 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkGWuaLBAAYM2t+TMEs59u635jL9f9kg==", 2023-08-27T11:25:06.494 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.494 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:25:06.494 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:25:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:25:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:25:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-08-27T11:25:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkkw7pGRAAVim28nKfumQg/LcBfJpivA==", 2023-08-27T11:25:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.496 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:25:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:25:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:25:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-08-27T11:25:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkNxfXAhAA/K5r8Mx/C7nRitNuzXXvKg==", 2023-08-27T11:25:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.498 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:25:06.499 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:25:06.499 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:06.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:06.499 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-08-27T11:25:06.500 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkW6B8NRAAF2hsNVRi3Ji4M8anuYwpDg==", 2023-08-27T11:25:06.500 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:25:06.500 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:25:06.500 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:25:06.500 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:25:06.501 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.501 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:06.501 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-08-27T11:25:06.501 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:25:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f15fc06a1d0 msgr2=0x7f15fc06c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f15fc06a1d0 0x7f15fc06c690 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f160c009930 tx=0x7f160c008040 comp rx=0 tx=0).stop 2023-08-27T11:25:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 msgr2=0x7f16181f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 0x7f16181f28e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1604000c00 tx=0x7f1604004750 comp rx=0 tx=0).stop 2023-08-27T11:25:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 shutdown_connections 2023-08-27T11:25:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f15fc06a1d0 0x7f15fc06c690 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f161815d3d0 0x7f161815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f161806e3a0 0x7f16180c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 --2- 172.21.15.45:0/2518338198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16180af040 0x7f16181f28e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 >> 172.21.15.45:0/2518338198 conn(0x7f16180cfd60 msgr2=0x7f16180ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 shutdown_connections 2023-08-27T11:25:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.457+0000 7f161f16a700 1 -- 172.21.15.45:0/2518338198 wait complete. 2023-08-27T11:25:06.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:06.513 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:25:06.551 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:25:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.636+0000 7fc74b4f3700 1 Processor -- start 2023-08-27T11:25:06.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.636+0000 7fc74b4f3700 1 -- start start 2023-08-27T11:25:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74b4f3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440ca620 0x7fc7440cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74b4f3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc7440af200 0x7fc7440ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74b4f3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7440ce340 con 0x7fc7440caa00 2023-08-27T11:25:06.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74b4f3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7440ceec0 con 0x7fc7440af200 2023-08-27T11:25:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74b4f3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74415d7c0 con 0x7fc7440ca620 2023-08-27T11:25:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc749a90700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc7440caa00 0x7fc7440acdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42850/0 (socket says 172.21.15.45:42850) 2023-08-27T11:25:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc749a90700 1 -- 172.21.15.45:0/2861638812 learned_addr learned my addr 172.21.15.45:0/2861638812 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc748a8e700 1 --2- 172.21.15.45:0/2861638812 >> v2:172.21.15.135:3300/0 conn(0x7fc7440af200 0x7fc7440ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74928f700 1 --2- 172.21.15.45:0/2861638812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440ca620 0x7fc7440cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc73bfff700 1 -- 172.21.15.45:0/2861638812 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562697347 0 0) 0x7fc7440ce340 con 0x7fc7440caa00 2023-08-27T11:25:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc73bfff700 1 -- 172.21.15.45:0/2861638812 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7300035b0 con 0x7fc7440caa00 2023-08-27T11:25:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74928f700 1 -- 172.21.15.45:0/2861638812 >> v2:172.21.15.135:3300/0 conn(0x7fc7440af200 msgr2=0x7fc7440ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74928f700 1 --2- 172.21.15.45:0/2861638812 >> v2:172.21.15.135:3300/0 conn(0x7fc7440af200 0x7fc7440ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74928f700 1 -- 172.21.15.45:0/2861638812 >> v1:172.21.15.45:6789/0 conn(0x7fc7440caa00 legacy=0x7fc7440acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc74928f700 1 -- 172.21.15.45:0/2861638812 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440cc570 con 0x7fc7440ca620 2023-08-27T11:25:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.637+0000 7fc748a8e700 1 --2- 172.21.15.45:0/2861638812 >> v2:172.21.15.135:3300/0 conn(0x7fc7440af200 0x7fc7440ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc74928f700 1 --2- 172.21.15.45:0/2861638812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440ca620 0x7fc7440cf350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc740009840 tx=0x7fc740009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b56d0b5f3fd3ac server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc73bfff700 1 -- 172.21.15.45:0/2861638812 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc740011830 con 0x7fc7440ca620 2023-08-27T11:25:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc73bfff700 1 -- 172.21.15.45:0/2861638812 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc740011990 con 0x7fc7440ca620 2023-08-27T11:25:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc73bfff700 1 -- 172.21.15.45:0/2861638812 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc740011d20 con 0x7fc7440ca620 2023-08-27T11:25:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2861638812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440ca620 msgr2=0x7fc7440cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc74b4f3700 1 --2- 172.21.15.45:0/2861638812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440ca620 0x7fc7440cf350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc740009840 tx=0x7fc740009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2861638812 shutdown_connections 2023-08-27T11:25:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc74b4f3700 1 --2- 172.21.15.45:0/2861638812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440ca620 0x7fc7440cf350 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc74b4f3700 1 --2- 172.21.15.45:0/2861638812 >> v2:172.21.15.135:3300/0 conn(0x7fc7440af200 0x7fc7440ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.638+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2861638812 >> 172.21.15.45:0/2861638812 conn(0x7fc7440cc7e0 msgr2=0x7fc7440ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.639+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2861638812 shutdown_connections 2023-08-27T11:25:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.639+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2861638812 wait complete. 2023-08-27T11:25:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.640+0000 7fc74b4f3700 1 Processor -- start 2023-08-27T11:25:06.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.640+0000 7fc74b4f3700 1 -- start start 2023-08-27T11:25:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc74b4f3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440af200 0x7fc7441f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc74b4f3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7440ca620 0x7fc74415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc74b4f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7440caa00 0x7fc7440c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc74b4f3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440c38e0 con 0x7fc7440caa00 2023-08-27T11:25:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc74b4f3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440c3b80 con 0x7fc7440ca620 2023-08-27T11:25:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc74b4f3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440c3e60 con 0x7fc7440af200 2023-08-27T11:25:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc749a90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7440caa00 0x7fc7440c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc749a90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7440caa00 0x7fc7440c33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56080/0 (socket says 172.21.15.45:56080) 2023-08-27T11:25:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc749a90700 1 -- 172.21.15.45:0/2749329161 learned_addr learned my addr 172.21.15.45:0/2749329161 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc74928f700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440af200 0x7fc7441f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.641+0000 7fc748a8e700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7440ca620 0x7fc74415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc749a90700 1 -- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440af200 msgr2=0x7fc7441f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc749a90700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440af200 0x7fc7441f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc749a90700 1 -- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7440ca620 msgr2=0x7fc74415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc749a90700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7440ca620 0x7fc74415a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc749a90700 1 -- 172.21.15.45:0/2749329161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7340094a0 con 0x7fc7440caa00 2023-08-27T11:25:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc749a90700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7440caa00 0x7fc7440c33a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00caa0 tx=0x7fc73c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc739ffb700 1 -- 172.21.15.45:0/2749329161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c00a9e0 con 0x7fc7440caa00 2023-08-27T11:25:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc739ffb700 1 -- 172.21.15.45:0/2749329161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c00ab40 con 0x7fc7440caa00 2023-08-27T11:25:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7400094a0 con 0x7fc7440caa00 2023-08-27T11:25:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc739ffb700 1 -- 172.21.15.45:0/2749329161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c012400 con 0x7fc7440caa00 2023-08-27T11:25:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.642+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7440c4420 con 0x7fc7440caa00 2023-08-27T11:25:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.643+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7440c6490 con 0x7fc7440caa00 2023-08-27T11:25:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.644+0000 7fc739ffb700 1 -- 172.21.15.45:0/2749329161 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc73c01b3b0 con 0x7fc7440caa00 2023-08-27T11:25:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.644+0000 7fc739ffb700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc72806a200 0x7fc72806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:06.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.644+0000 7fc739ffb700 1 -- 172.21.15.45:0/2749329161 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fc73c0880e0 con 0x7fc7440caa00 2023-08-27T11:25:06.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.644+0000 7fc74928f700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc72806a200 0x7fc72806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:06.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.645+0000 7fc74928f700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc72806a200 0x7fc72806c6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc740000c00 tx=0x7fc740008030 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:06.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.648+0000 7fc739ffb700 1 -- 172.21.15.45:0/2749329161 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc73c053960 con 0x7fc7440caa00 2023-08-27T11:25:06.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.850+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7440cbce0 con 0x7fc7440caa00 2023-08-27T11:25:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.851+0000 7fc739ffb700 1 -- 172.21.15.45:0/2749329161 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+14926 (secure 0 0 0) 0x7fc73c056f80 con 0x7fc7440caa00 2023-08-27T11:25:06.852 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:06.853 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":66,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:25:06.008555+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.853+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc72806a200 msgr2=0x7fc72806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.853+0000 7fc74b4f3700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc72806a200 0x7fc72806c6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc740000c00 tx=0x7fc740008030 comp rx=0 tx=0).stop 2023-08-27T11:25:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.853+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7440caa00 msgr2=0x7fc7440c33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.853+0000 7fc74b4f3700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7440caa00 0x7fc7440c33a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00caa0 tx=0x7fc73c00cdb0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.854+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 shutdown_connections 2023-08-27T11:25:06.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.854+0000 7fc74b4f3700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc72806a200 0x7fc72806c6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.854+0000 7fc74b4f3700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7440caa00 0x7fc7440c33a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.854+0000 7fc74b4f3700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc7440af200 0x7fc7441f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.854+0000 7fc74b4f3700 1 --2- 172.21.15.45:0/2749329161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7440ca620 0x7fc74415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.854+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 >> 172.21.15.45:0/2749329161 conn(0x7fc7440cc7e0 msgr2=0x7fc74406ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.854+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 shutdown_connections 2023-08-27T11:25:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:06.854+0000 7fc74b4f3700 1 -- 172.21.15.45:0/2749329161 wait complete. 2023-08-27T11:25:06.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:25:06.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:25:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.034+0000 7f743d9dc700 1 Processor -- start 2023-08-27T11:25:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.034+0000 7f743d9dc700 1 -- start start 2023-08-27T11:25:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.035+0000 7f743d9dc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f743815d0a0 0x7f743806ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.035+0000 7f743d9dc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74380d0a50 0x7f74380cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.035+0000 7f743d9dc700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7438161870 con 0x7f743815d480 2023-08-27T11:25:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.035+0000 7f743d9dc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743806d3f0 con 0x7f74380d0a50 2023-08-27T11:25:07.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.035+0000 7f743d9dc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7438161da0 con 0x7f743815d0a0 2023-08-27T11:25:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.035+0000 7f74367fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74380d0a50 0x7f74380cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.035+0000 7f74367fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74380d0a50 0x7f74380cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56086/0 (socket says 172.21.15.45:56086) 2023-08-27T11:25:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.035+0000 7f74367fc700 1 -- 172.21.15.45:0/2662957461 learned_addr learned my addr 172.21.15.45:0/2662957461 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.035+0000 7f7436ffd700 1 --2- 172.21.15.45:0/2662957461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f743815d0a0 0x7f743806ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f74367fc700 1 -- 172.21.15.45:0/2662957461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f743815d0a0 msgr2=0x7f743806ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f74367fc700 1 --2- 172.21.15.45:0/2662957461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f743815d0a0 0x7f743806ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f74367fc700 1 -- 172.21.15.45:0/2662957461 >> v1:172.21.15.135:6789/0 conn(0x7f743815d480 legacy=0x7f74380b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f74367fc700 1 -- 172.21.15.45:0/2662957461 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438161ac0 con 0x7f74380d0a50 2023-08-27T11:25:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f74367fc700 1 --2- 172.21.15.45:0/2662957461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74380d0a50 0x7f74380cd880 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f74280098f0 tx=0x7f7428009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9ef2efb0447108 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f7435ffb700 1 -- 172.21.15.45:0/2662957461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742800b8a0 con 0x7f74380d0a50 2023-08-27T11:25:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f7435ffb700 1 -- 172.21.15.45:0/2662957461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742800ba00 con 0x7f74380d0a50 2023-08-27T11:25:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f7435ffb700 1 -- 172.21.15.45:0/2662957461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742800bd90 con 0x7f74380d0a50 2023-08-27T11:25:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f743d9dc700 1 -- 172.21.15.45:0/2662957461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74380d0a50 msgr2=0x7f74380cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f743d9dc700 1 --2- 172.21.15.45:0/2662957461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74380d0a50 0x7f74380cd880 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f74280098f0 tx=0x7f7428009c00 comp rx=0 tx=0).stop 2023-08-27T11:25:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f743d9dc700 1 -- 172.21.15.45:0/2662957461 shutdown_connections 2023-08-27T11:25:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f743d9dc700 1 --2- 172.21.15.45:0/2662957461 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f743815d0a0 0x7f743806ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f743d9dc700 1 --2- 172.21.15.45:0/2662957461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74380d0a50 0x7f74380cd880 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.036+0000 7f743d9dc700 1 -- 172.21.15.45:0/2662957461 >> 172.21.15.45:0/2662957461 conn(0x7f743806f960 msgr2=0x7f743806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.037+0000 7f743d9dc700 1 -- 172.21.15.45:0/2662957461 shutdown_connections 2023-08-27T11:25:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.037+0000 7f743d9dc700 1 -- 172.21.15.45:0/2662957461 wait complete. 2023-08-27T11:25:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.038+0000 7f743d9dc700 1 Processor -- start 2023-08-27T11:25:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.038+0000 7f743d9dc700 1 -- start start 2023-08-27T11:25:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f743d9dc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74380d0a50 0x7f74381f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f743d9dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f743815d0a0 0x7f74380b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f743d9dc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f743815d480 0x7f74380ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f743d9dc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74380b6830 con 0x7f743815d0a0 2023-08-27T11:25:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f743d9dc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74380ad1d0 con 0x7f743815d480 2023-08-27T11:25:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f743d9dc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380ad4b0 con 0x7f74380d0a50 2023-08-27T11:25:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74367fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f743815d0a0 0x7f74380b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74367fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f743815d0a0 0x7f74380b62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56092/0 (socket says 172.21.15.45:56092) 2023-08-27T11:25:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74367fc700 1 -- 172.21.15.45:0/1620318095 learned_addr learned my addr 172.21.15.45:0/1620318095 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f7436ffd700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74380d0a50 0x7f74381f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74377fe700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f743815d480 0x7f74380ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74367fc700 1 -- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74380d0a50 msgr2=0x7f74381f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74367fc700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74380d0a50 0x7f74381f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74367fc700 1 -- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f743815d480 msgr2=0x7f74380ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74367fc700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f743815d480 0x7f74380ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74367fc700 1 -- 172.21.15.45:0/1620318095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428009550 con 0x7f743815d0a0 2023-08-27T11:25:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f7436ffd700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74380d0a50 0x7f74381f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f74367fc700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f743815d0a0 0x7f74380b62f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7428009c40 tx=0x7f7428004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f742ffff700 1 -- 172.21.15.45:0/1620318095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280036a0 con 0x7f743815d0a0 2023-08-27T11:25:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f742ffff700 1 -- 172.21.15.45:0/1620318095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742801e070 con 0x7f743815d0a0 2023-08-27T11:25:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.039+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74380ad790 con 0x7f743815d0a0 2023-08-27T11:25:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.040+0000 7f742ffff700 1 -- 172.21.15.45:0/1620318095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428003a60 con 0x7f743815d0a0 2023-08-27T11:25:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.040+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74380aec00 con 0x7f743815d0a0 2023-08-27T11:25:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.040+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74381f5f60 con 0x7f743815d0a0 2023-08-27T11:25:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.041+0000 7f742ffff700 1 -- 172.21.15.45:0/1620318095 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7428003bc0 con 0x7f743815d0a0 2023-08-27T11:25:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.042+0000 7f742ffff700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f741806a200 0x7f741806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.042+0000 7f742ffff700 1 -- 172.21.15.45:0/1620318095 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f7428020070 con 0x7f743815d0a0 2023-08-27T11:25:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.042+0000 7f7436ffd700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f741806a200 0x7f741806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.043+0000 7f7436ffd700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f741806a200 0x7f741806c6c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f74200099f0 tx=0x7f7420008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.046+0000 7f742ffff700 1 -- 172.21.15.45:0/1620318095 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f74280554f0 con 0x7f743815d0a0 2023-08-27T11:25:07.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.244+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74381f6110 con 0x7f743815d0a0 2023-08-27T11:25:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.244+0000 7f742ffff700 1 -- 172.21.15.45:0/1620318095 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74381f6110 con 0x7f743815d0a0 2023-08-27T11:25:07.246 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:25:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f741806a200 msgr2=0x7f741806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f741806a200 0x7f741806c6c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f74200099f0 tx=0x7f7420008040 comp rx=0 tx=0).stop 2023-08-27T11:25:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f743815d0a0 msgr2=0x7f74380b62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f743815d0a0 0x7f74380b62f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7428009c40 tx=0x7f7428004750 comp rx=0 tx=0).stop 2023-08-27T11:25:07.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 shutdown_connections 2023-08-27T11:25:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f741806a200 0x7f741806c6c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f743815d480 0x7f74380ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f74380d0a50 0x7f74381f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 --2- 172.21.15.45:0/1620318095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f743815d0a0 0x7f74380b62f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.246+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 >> 172.21.15.45:0/1620318095 conn(0x7f743806f960 msgr2=0x7f74381602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.247+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 shutdown_connections 2023-08-27T11:25:07.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.247+0000 7f743d9dc700 1 -- 172.21.15.45:0/1620318095 wait complete. 2023-08-27T11:25:07.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:25:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.386+0000 7f435a7ed700 1 Processor -- start 2023-08-27T11:25:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.386+0000 7f435a7ed700 1 -- start start 2023-08-27T11:25:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.387+0000 7f435a7ed700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f435415d340 0x7f4354160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.387+0000 7f435a7ed700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435406b160 con 0x7f435406a990 2023-08-27T11:25:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.387+0000 7f435a7ed700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43540af590 con 0x7f43540cd880 2023-08-27T11:25:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.387+0000 7f435a7ed700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4354072010 con 0x7f435415d340 2023-08-27T11:25:07.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.387+0000 7f4353fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f435415d340 0x7f4354160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.387+0000 7f4358d8a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f435406a990 0x7f43540b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42878/0 (socket says 172.21.15.45:42878) 2023-08-27T11:25:07.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.387+0000 7f4358d8a700 1 -- 172.21.15.45:0/2876956459 learned_addr learned my addr 172.21.15.45:0/2876956459 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:07.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f4353fff700 1 -- 172.21.15.45:0/2876956459 >> v1:172.21.15.135:6789/0 conn(0x7f43540cd880 legacy=0x7f435406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:07.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f4353fff700 1 -- 172.21.15.45:0/2876956459 >> v1:172.21.15.45:6789/0 conn(0x7f435406a990 legacy=0x7f43540b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f4353fff700 1 -- 172.21.15.45:0/2876956459 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540af2b0 con 0x7f435415d340 2023-08-27T11:25:07.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f4352ffd700 1 -- 172.21.15.45:0/2876956459 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395270198 0 0) 0x7f435406b160 con 0x7f435406a990 2023-08-27T11:25:07.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f4353fff700 1 --2- 172.21.15.45:0/2876956459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435415d340 0x7f4354160d00 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4344009840 tx=0x7f4344009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c04e4399a340529 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:07.394 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f4352ffd700 1 -- 172.21.15.45:0/2876956459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344004230 con 0x7f435415d340 2023-08-27T11:25:07.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f4352ffd700 1 -- 172.21.15.45:0/2876956459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4344004390 con 0x7f435415d340 2023-08-27T11:25:07.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f4352ffd700 1 -- 172.21.15.45:0/2876956459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440036a0 con 0x7f435415d340 2023-08-27T11:25:07.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f435a7ed700 1 -- 172.21.15.45:0/2876956459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435415d340 msgr2=0x7f4354160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f435a7ed700 1 --2- 172.21.15.45:0/2876956459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435415d340 0x7f4354160d00 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4344009840 tx=0x7f4344009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:07.395 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f435a7ed700 1 -- 172.21.15.45:0/2876956459 shutdown_connections 2023-08-27T11:25:07.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f435a7ed700 1 --2- 172.21.15.45:0/2876956459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f435415d340 0x7f4354160d00 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.388+0000 7f435a7ed700 1 -- 172.21.15.45:0/2876956459 >> 172.21.15.45:0/2876956459 conn(0x7f43540cc7c0 msgr2=0x7f43540ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:07.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.389+0000 7f435a7ed700 1 -- 172.21.15.45:0/2876956459 shutdown_connections 2023-08-27T11:25:07.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.389+0000 7f435a7ed700 1 -- 172.21.15.45:0/2876956459 wait complete. 2023-08-27T11:25:07.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.390+0000 7f435a7ed700 1 Processor -- start 2023-08-27T11:25:07.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.390+0000 7f435a7ed700 1 -- start start 2023-08-27T11:25:07.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f435a7ed700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f435406a990 0x7f43541516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f435a7ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43540cd880 0x7f43541e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f435a7ed700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f435415d340 0x7f43540ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f435a7ed700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4354151bf0 con 0x7f43540cd880 2023-08-27T11:25:07.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f435a7ed700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c7940 con 0x7f435415d340 2023-08-27T11:25:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f435a7ed700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43541e7c10 con 0x7f435406a990 2023-08-27T11:25:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f43537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43540cd880 0x7f43541e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f43537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43540cd880 0x7f43541e6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56106/0 (socket says 172.21.15.45:56106) 2023-08-27T11:25:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f43537fe700 1 -- 172.21.15.45:0/1049806638 learned_addr learned my addr 172.21.15.45:0/1049806638 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f4358d8a700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f435415d340 0x7f43540ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f4353fff700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f435406a990 0x7f43541516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f43537fe700 1 -- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f435406a990 msgr2=0x7f43541516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f43537fe700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f435406a990 0x7f43541516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f43537fe700 1 -- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f435415d340 msgr2=0x7f43540ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f43537fe700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f435415d340 0x7f43540ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f43537fe700 1 -- 172.21.15.45:0/1049806638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43440094a0 con 0x7f43540cd880 2023-08-27T11:25:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f43537fe700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43540cd880 0x7f43541e6b40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f433c00b4a0 tx=0x7f433c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f4350ff9700 1 -- 172.21.15.45:0/1049806638 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011830 con 0x7f43540cd880 2023-08-27T11:25:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f4350ff9700 1 -- 172.21.15.45:0/1049806638 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c011990 con 0x7f43540cd880 2023-08-27T11:25:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43541e7370 con 0x7f43540cd880 2023-08-27T11:25:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.391+0000 7f4350ff9700 1 -- 172.21.15.45:0/1049806638 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011d40 con 0x7f43540cd880 2023-08-27T11:25:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.392+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43541e7930 con 0x7f43540cd880 2023-08-27T11:25:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.393+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4354072690 con 0x7f43540cd880 2023-08-27T11:25:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.393+0000 7f4350ff9700 1 -- 172.21.15.45:0/1049806638 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f433c01a400 con 0x7f43540cd880 2023-08-27T11:25:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.393+0000 7f4350ff9700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f433406a200 0x7f433406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.393+0000 7f4350ff9700 1 -- 172.21.15.45:0/1049806638 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f433c0882b0 con 0x7f43540cd880 2023-08-27T11:25:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.394+0000 7f4353fff700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f433406a200 0x7f433406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.394+0000 7f4353fff700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f433406a200 0x7f433406c6c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f43440090b0 tx=0x7f4344008030 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.398+0000 7f4350ff9700 1 -- 172.21.15.45:0/1049806638 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f433c053a60 con 0x7f43540cd880 2023-08-27T11:25:07.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.594+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f43540734a0 con 0x7f43540cd880 2023-08-27T11:25:07.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.595+0000 7f4350ff9700 1 -- 172.21.15.45:0/1049806638 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7f433c057080 con 0x7f43540cd880 2023-08-27T11:25:07.596 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:25:07.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f433406a200 msgr2=0x7f433406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f433406a200 0x7f433406c6c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f43440090b0 tx=0x7f4344008030 comp rx=0 tx=0).stop 2023-08-27T11:25:07.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43540cd880 msgr2=0x7f43541e6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43540cd880 0x7f43541e6b40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f433c00b4a0 tx=0x7f433c00b860 comp rx=0 tx=0).stop 2023-08-27T11:25:07.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 shutdown_connections 2023-08-27T11:25:07.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f433406a200 0x7f433406c6c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f435415d340 0x7f43540ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f435406a990 0x7f43541516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 --2- 172.21.15.45:0/1049806638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43540cd880 0x7f43541e6b40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 >> 172.21.15.45:0/1049806638 conn(0x7f43540cc7c0 msgr2=0x7f435415baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:07.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 shutdown_connections 2023-08-27T11:25:07.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.597+0000 7f435a7ed700 1 -- 172.21.15.45:0/1049806638 wait complete. 2023-08-27T11:25:07.610 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:25:07.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.734+0000 7fbe91e34700 1 Processor -- start 2023-08-27T11:25:07.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.734+0000 7fbe91e34700 1 -- start start 2023-08-27T11:25:07.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.735+0000 7fbe91e34700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0cc650 0x7fbe8c0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.735+0000 7fbe91e34700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c0ccb80 con 0x7fbe8c15e990 2023-08-27T11:25:07.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.735+0000 7fbe91e34700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c0cce60 con 0x7fbe8c0adad0 2023-08-27T11:25:07.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.735+0000 7fbe91e34700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0cd600 con 0x7fbe8c0cc650 2023-08-27T11:25:07.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.735+0000 7fbe8b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbe8c15e990 0x7fbe8c0ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42888/0 (socket says 172.21.15.45:42888) 2023-08-27T11:25:07.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.735+0000 7fbe8b7fe700 1 -- 172.21.15.45:0/3688215857 learned_addr learned my addr 172.21.15.45:0/3688215857 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:07.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.735+0000 7fbe8affd700 1 --2- 172.21.15.45:0/3688215857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0cc650 0x7fbe8c0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8a7fc700 1 -- 172.21.15.45:0/3688215857 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895732243 0 0) 0x7fbe8c0ccb80 con 0x7fbe8c15e990 2023-08-27T11:25:07.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8a7fc700 1 -- 172.21.15.45:0/3688215857 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe780035b0 con 0x7fbe8c15e990 2023-08-27T11:25:07.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8a7fc700 1 -- 172.21.15.45:0/3688215857 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880705027 0 0) 0x7fbe8c0cce60 con 0x7fbe8c0adad0 2023-08-27T11:25:07.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8a7fc700 1 -- 172.21.15.45:0/3688215857 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe8c0ccb80 con 0x7fbe8c0adad0 2023-08-27T11:25:07.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8affd700 1 -- 172.21.15.45:0/3688215857 >> v1:172.21.15.135:6789/0 conn(0x7fbe8c0adad0 legacy=0x7fbe8c0cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8affd700 1 -- 172.21.15.45:0/3688215857 >> v1:172.21.15.45:6789/0 conn(0x7fbe8c15e990 legacy=0x7fbe8c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8affd700 1 -- 172.21.15.45:0/3688215857 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c0cd8e0 con 0x7fbe8c0cc650 2023-08-27T11:25:07.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8affd700 1 --2- 172.21.15.45:0/3688215857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0cc650 0x7fbe8c0ced40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009870 tx=0x7fbe74009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d221f538213615 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:07.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8a7fc700 1 -- 172.21.15.45:0/3688215857 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74011830 con 0x7fbe8c0cc650 2023-08-27T11:25:07.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8a7fc700 1 -- 172.21.15.45:0/3688215857 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe74011990 con 0x7fbe8c0cc650 2023-08-27T11:25:07.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.736+0000 7fbe8a7fc700 1 -- 172.21.15.45:0/3688215857 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74011d20 con 0x7fbe8c0cc650 2023-08-27T11:25:07.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.737+0000 7fbe91e34700 1 -- 172.21.15.45:0/3688215857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0cc650 msgr2=0x7fbe8c0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.737+0000 7fbe91e34700 1 --2- 172.21.15.45:0/3688215857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0cc650 0x7fbe8c0ced40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009870 tx=0x7fbe74009b80 comp rx=0 tx=0).stop 2023-08-27T11:25:07.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.737+0000 7fbe91e34700 1 -- 172.21.15.45:0/3688215857 shutdown_connections 2023-08-27T11:25:07.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.737+0000 7fbe91e34700 1 --2- 172.21.15.45:0/3688215857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0cc650 0x7fbe8c0ced40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.737+0000 7fbe91e34700 1 -- 172.21.15.45:0/3688215857 >> 172.21.15.45:0/3688215857 conn(0x7fbe8c0d0f60 msgr2=0x7fbe8c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:07.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.737+0000 7fbe91e34700 1 -- 172.21.15.45:0/3688215857 shutdown_connections 2023-08-27T11:25:07.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.737+0000 7fbe91e34700 1 -- 172.21.15.45:0/3688215857 wait complete. 2023-08-27T11:25:07.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.738+0000 7fbe91e34700 1 Processor -- start 2023-08-27T11:25:07.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.738+0000 7fbe91e34700 1 -- start start 2023-08-27T11:25:07.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe91e34700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0adad0 0x7fbe8c0cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe91e34700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe8c0cc650 0x7fbe8c15c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe91e34700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe8c15e990 0x7fbe8c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe91e34700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0b44b0 con 0x7fbe8c0cc650 2023-08-27T11:25:07.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe91e34700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0b6910 con 0x7fbe8c15e990 2023-08-27T11:25:07.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe91e34700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0b3b80 con 0x7fbe8c0adad0 2023-08-27T11:25:07.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe8c0cc650 0x7fbe8c15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe8c0cc650 0x7fbe8c15c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56120/0 (socket says 172.21.15.45:56120) 2023-08-27T11:25:07.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8affd700 1 -- 172.21.15.45:0/1902781604 learned_addr learned my addr 172.21.15.45:0/1902781604 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:07.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8b7fe700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0adad0 0x7fbe8c0cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8bfff700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe8c15e990 0x7fbe8c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8affd700 1 -- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0adad0 msgr2=0x7fbe8c0cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8affd700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0adad0 0x7fbe8c0cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8affd700 1 -- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe8c15e990 msgr2=0x7fbe8c1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8affd700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe8c15e990 0x7fbe8c1578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.739+0000 7fbe8affd700 1 -- 172.21.15.45:0/1902781604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe740094a0 con 0x7fbe8c0cc650 2023-08-27T11:25:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.740+0000 7fbe8affd700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe8c0cc650 0x7fbe8c15c520 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbe740090b0 tx=0x7fbe74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.740+0000 7fbe83fff700 1 -- 172.21.15.45:0/1902781604 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7401a6e0 con 0x7fbe8c0cc650 2023-08-27T11:25:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.740+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8c0b3e60 con 0x7fbe8c0cc650 2023-08-27T11:25:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.740+0000 7fbe83fff700 1 -- 172.21.15.45:0/1902781604 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7401a840 con 0x7fbe8c0cc650 2023-08-27T11:25:07.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.740+0000 7fbe83fff700 1 -- 172.21.15.45:0/1902781604 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7401abb0 con 0x7fbe8c0cc650 2023-08-27T11:25:07.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.740+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8c09ad50 con 0x7fbe8c0cc650 2023-08-27T11:25:07.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.741+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8c0cd340 con 0x7fbe8c0cc650 2023-08-27T11:25:07.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.741+0000 7fbe83fff700 1 -- 172.21.15.45:0/1902781604 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbe74022070 con 0x7fbe8c0cc650 2023-08-27T11:25:07.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.742+0000 7fbe83fff700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbe6c06a200 0x7fbe6c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:07.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.742+0000 7fbe83fff700 1 -- 172.21.15.45:0/1902781604 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fbe740894c0 con 0x7fbe8c0cc650 2023-08-27T11:25:07.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.742+0000 7fbe8b7fe700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbe6c06a200 0x7fbe6c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:07.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.742+0000 7fbe8b7fe700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbe6c06a200 0x7fbe6c06c6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007fd0 tx=0x7fbe7c007e00 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.746+0000 7fbe83fff700 1 -- 172.21.15.45:0/1902781604 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbe74054e30 con 0x7fbe8c0cc650 2023-08-27T11:25:07.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.948+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbe8c0b3cc0 con 0x7fbe8c0cc650 2023-08-27T11:25:07.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.948+0000 7fbe83fff700 1 -- 172.21.15.45:0/1902781604 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbe74015090 con 0x7fbe8c0cc650 2023-08-27T11:25:07.950 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:07.950 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:25:07.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:07.950 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-08-27T11:25:07.950 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-08-27T11:25:07.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:25:07.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:07.951 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-08-27T11:25:07.951 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-08-27T11:25:07.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:07.952 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:25:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbe6c06a200 msgr2=0x7fbe6c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbe6c06a200 0x7fbe6c06c6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007fd0 tx=0x7fbe7c007e00 comp rx=0 tx=0).stop 2023-08-27T11:25:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe8c0cc650 msgr2=0x7fbe8c15c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe8c0cc650 0x7fbe8c15c520 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbe740090b0 tx=0x7fbe74004750 comp rx=0 tx=0).stop 2023-08-27T11:25:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 shutdown_connections 2023-08-27T11:25:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbe6c06a200 0x7fbe6c06c6c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe8c15e990 0x7fbe8c1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe8c0adad0 0x7fbe8c0cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 --2- 172.21.15.45:0/1902781604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe8c0cc650 0x7fbe8c15c520 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.950+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 >> 172.21.15.45:0/1902781604 conn(0x7fbe8c0d0f60 msgr2=0x7fbe8c06e350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.951+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 shutdown_connections 2023-08-27T11:25:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:07.951+0000 7fbe91e34700 1 -- 172.21.15.45:0/1902781604 wait complete. 2023-08-27T11:25:07.963 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:25:07.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:25:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.092+0000 7f3a799b2700 1 Processor -- start 2023-08-27T11:25:08.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.092+0000 7f3a799b2700 1 -- start start 2023-08-27T11:25:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.093+0000 7f3a799b2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7406a990 0x7f3a740b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.093+0000 7f3a799b2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7406b160 con 0x7f3a74160d00 2023-08-27T11:25:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.093+0000 7f3a799b2700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a740cdf90 con 0x7f3a7415d340 2023-08-27T11:25:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.093+0000 7f3a799b2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a74072010 con 0x7f3a7406a990 2023-08-27T11:25:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.093+0000 7f3a727fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3a74160d00 0x7f3a7406a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42904/0 (socket says 172.21.15.45:42904) 2023-08-27T11:25:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.093+0000 7f3a727fc700 1 -- 172.21.15.45:0/89831388 learned_addr learned my addr 172.21.15.45:0/89831388 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.093+0000 7f3a737fe700 1 --2- 172.21.15.45:0/89831388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7406a990 0x7f3a740b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.093+0000 7f3a71ffb700 1 -- 172.21.15.45:0/89831388 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776526752 0 0) 0x7f3a7406b160 con 0x7f3a74160d00 2023-08-27T11:25:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.094+0000 7f3a71ffb700 1 -- 172.21.15.45:0/89831388 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a5c0035b0 con 0x7f3a74160d00 2023-08-27T11:25:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.094+0000 7f3a71ffb700 1 -- 172.21.15.45:0/89831388 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866744340 0 0) 0x7f3a740cdf90 con 0x7f3a7415d340 2023-08-27T11:25:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.094+0000 7f3a737fe700 1 -- 172.21.15.45:0/89831388 >> v1:172.21.15.135:6789/0 conn(0x7f3a7415d340 legacy=0x7f3a740cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.094+0000 7f3a737fe700 1 -- 172.21.15.45:0/89831388 >> v1:172.21.15.45:6789/0 conn(0x7f3a74160d00 legacy=0x7f3a7406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.094+0000 7f3a737fe700 1 -- 172.21.15.45:0/89831388 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740af590 con 0x7f3a7406a990 2023-08-27T11:25:08.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.094+0000 7f3a737fe700 1 --2- 172.21.15.45:0/89831388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7406a990 0x7f3a740b18a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009dc0 tx=0x7f3a6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed16d9c13a33331 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.094+0000 7f3a71ffb700 1 -- 172.21.15.45:0/89831388 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00e070 con 0x7f3a7406a990 2023-08-27T11:25:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.094+0000 7f3a71ffb700 1 -- 172.21.15.45:0/89831388 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c003a20 con 0x7f3a7406a990 2023-08-27T11:25:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.094+0000 7f3a71ffb700 1 -- 172.21.15.45:0/89831388 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64001670 con 0x7f3a7406a990 2023-08-27T11:25:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.095+0000 7f3a799b2700 1 -- 172.21.15.45:0/89831388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7406a990 msgr2=0x7f3a740b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.095+0000 7f3a799b2700 1 --2- 172.21.15.45:0/89831388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7406a990 0x7f3a740b18a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009dc0 tx=0x7f3a6c00b330 comp rx=0 tx=0).stop 2023-08-27T11:25:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.095+0000 7f3a799b2700 1 -- 172.21.15.45:0/89831388 shutdown_connections 2023-08-27T11:25:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.095+0000 7f3a799b2700 1 --2- 172.21.15.45:0/89831388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7406a990 0x7f3a740b18a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.095+0000 7f3a799b2700 1 -- 172.21.15.45:0/89831388 >> 172.21.15.45:0/89831388 conn(0x7f3a740cc7c0 msgr2=0x7f3a740ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.095+0000 7f3a799b2700 1 -- 172.21.15.45:0/89831388 shutdown_connections 2023-08-27T11:25:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.095+0000 7f3a799b2700 1 -- 172.21.15.45:0/89831388 wait complete. 2023-08-27T11:25:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.096+0000 7f3a799b2700 1 Processor -- start 2023-08-27T11:25:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.096+0000 7f3a799b2700 1 -- start start 2023-08-27T11:25:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a799b2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a7406a990 0x7f3a740c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a799b2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7415d340 0x7f3a740ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a799b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a74160d00 0x7f3a7415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a799b2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740ae6e0 con 0x7f3a74160d00 2023-08-27T11:25:08.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a799b2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740ad800 con 0x7f3a7406a990 2023-08-27T11:25:08.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a799b2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740ada60 con 0x7f3a7415d340 2023-08-27T11:25:08.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a737fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a74160d00 0x7f3a7415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:08.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a737fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a74160d00 0x7f3a7415fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56136/0 (socket says 172.21.15.45:56136) 2023-08-27T11:25:08.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a737fe700 1 -- 172.21.15.45:0/2255067977 learned_addr learned my addr 172.21.15.45:0/2255067977 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a727fc700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7415d340 0x7f3a740ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a72ffd700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a7406a990 0x7f3a740c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a737fe700 1 -- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7415d340 msgr2=0x7f3a740ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a737fe700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7415d340 0x7f3a740ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a737fe700 1 -- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a7406a990 msgr2=0x7f3a740c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:08.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a737fe700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a7406a990 0x7f3a740c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:08.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a737fe700 1 -- 172.21.15.45:0/2255067977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c009a70 con 0x7f3a74160d00 2023-08-27T11:25:08.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.097+0000 7f3a727fc700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7415d340 0x7f3a740ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:08.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.098+0000 7f3a737fe700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a74160d00 0x7f3a7415fea0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c015040 tx=0x7f3a6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:08.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.098+0000 7f3a5bfff700 1 -- 172.21.15.45:0/2255067977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c004670 con 0x7f3a74160d00 2023-08-27T11:25:08.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.098+0000 7f3a5bfff700 1 -- 172.21.15.45:0/2255067977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c0047d0 con 0x7f3a74160d00 2023-08-27T11:25:08.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.098+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a740add40 con 0x7f3a74160d00 2023-08-27T11:25:08.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.098+0000 7f3a5bfff700 1 -- 172.21.15.45:0/2255067977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c012690 con 0x7f3a74160d00 2023-08-27T11:25:08.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.098+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a740ae300 con 0x7f3a74160d00 2023-08-27T11:25:08.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.099+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a74072690 con 0x7f3a74160d00 2023-08-27T11:25:08.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.099+0000 7f3a5bfff700 1 -- 172.21.15.45:0/2255067977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3a6c0127f0 con 0x7f3a74160d00 2023-08-27T11:25:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.100+0000 7f3a5bfff700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f3a6006a200 0x7f3a6006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.100+0000 7f3a5bfff700 1 -- 172.21.15.45:0/2255067977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f3a6c08a020 con 0x7f3a74160d00 2023-08-27T11:25:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.100+0000 7f3a72ffd700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f3a6006a200 0x7f3a6006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.100+0000 7f3a72ffd700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f3a6006a200 0x7f3a6006c6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a68000f60 tx=0x7f3a68009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:08.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.104+0000 7f3a5bfff700 1 -- 172.21.15.45:0/2255067977 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3a6c055760 con 0x7f3a74160d00 2023-08-27T11:25:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:08.306+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3a740734a0 con 0x7f3a74160d00 2023-08-27T11:25:09.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.040+0000 7f3a5bfff700 1 -- 172.21.15.45:0/2255067977 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3a6c058d80 con 0x7f3a74160d00 2023-08-27T11:25:09.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f3a6006a200 msgr2=0x7f3a6006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:09.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f3a6006a200 0x7f3a6006c6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a68000f60 tx=0x7f3a68009040 comp rx=0 tx=0).stop 2023-08-27T11:25:09.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a74160d00 msgr2=0x7f3a7415fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:09.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a74160d00 0x7f3a7415fea0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c015040 tx=0x7f3a6c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 shutdown_connections 2023-08-27T11:25:09.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f3a6006a200 0x7f3a6006c6c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a74160d00 0x7f3a7415fea0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a7406a990 0x7f3a740c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 --2- 172.21.15.45:0/2255067977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3a7415d340 0x7f3a740ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.045+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 >> 172.21.15.45:0/2255067977 conn(0x7f3a740cc7c0 msgr2=0x7f3a740cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.048+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 shutdown_connections 2023-08-27T11:25:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.048+0000 7f3a799b2700 1 -- 172.21.15.45:0/2255067977 wait complete. 2023-08-27T11:25:09.050 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-08-27T11:25:09.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:25:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.203+0000 7f80f9552700 1 Processor -- start 2023-08-27T11:25:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.203+0000 7f80f9552700 1 -- start start 2023-08-27T11:25:09.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f9552700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80f40cf990 0x7f80f40d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:09.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f9552700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f80f40cfd70 0x7f80f40cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:09.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f9552700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40ce290 0x7f80f415ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f9552700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f40d1320 con 0x7f80f40cf990 2023-08-27T11:25:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f9552700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f40d1570 con 0x7f80f40cfd70 2023-08-27T11:25:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f9552700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40d1850 con 0x7f80f40ce290 2023-08-27T11:25:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f2ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80f40cf990 0x7f80f40d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:09.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f2ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80f40cf990 0x7f80f40d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56140/0 (socket says 172.21.15.45:56140) 2023-08-27T11:25:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f2ffd700 1 -- 172.21.15.45:0/242037172 learned_addr learned my addr 172.21.15.45:0/242037172 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f37fe700 1 --2- 172.21.15.45:0/242037172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40ce290 0x7f80f415ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f27fc700 1 --2- 172.21.15.45:0/242037172 >> v2:172.21.15.135:3300/0 conn(0x7f80f40cfd70 0x7f80f40cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f2ffd700 1 -- 172.21.15.45:0/242037172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40ce290 msgr2=0x7f80f415ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f2ffd700 1 --2- 172.21.15.45:0/242037172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40ce290 0x7f80f415ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f2ffd700 1 -- 172.21.15.45:0/242037172 >> v2:172.21.15.135:3300/0 conn(0x7f80f40cfd70 msgr2=0x7f80f40cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:09.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f2ffd700 1 --2- 172.21.15.45:0/242037172 >> v2:172.21.15.135:3300/0 conn(0x7f80f40cfd70 0x7f80f40cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f2ffd700 1 -- 172.21.15.45:0/242037172 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f40d1b30 con 0x7f80f40cf990 2023-08-27T11:25:09.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.204+0000 7f80f27fc700 1 --2- 172.21.15.45:0/242037172 >> v2:172.21.15.135:3300/0 conn(0x7f80f40cfd70 0x7f80f40cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:09.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f2ffd700 1 --2- 172.21.15.45:0/242037172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40cf990 0x7f80f40d2330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f80f4072960 tx=0x7f80e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a324dc15055e18d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:09.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f1ffb700 1 -- 172.21.15.45:0/242037172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e8011830 con 0x7f80f40cf990 2023-08-27T11:25:09.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f1ffb700 1 -- 172.21.15.45:0/242037172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e8011990 con 0x7f80f40cf990 2023-08-27T11:25:09.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f1ffb700 1 -- 172.21.15.45:0/242037172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e8011d20 con 0x7f80f40cf990 2023-08-27T11:25:09.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f9552700 1 -- 172.21.15.45:0/242037172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40cf990 msgr2=0x7f80f40d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:09.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f9552700 1 --2- 172.21.15.45:0/242037172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40cf990 0x7f80f40d2330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f80f4072960 tx=0x7f80e8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f9552700 1 -- 172.21.15.45:0/242037172 shutdown_connections 2023-08-27T11:25:09.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f9552700 1 --2- 172.21.15.45:0/242037172 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40ce290 0x7f80f415ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f9552700 1 --2- 172.21.15.45:0/242037172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40cf990 0x7f80f40d2330 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f9552700 1 --2- 172.21.15.45:0/242037172 >> v2:172.21.15.135:3300/0 conn(0x7f80f40cfd70 0x7f80f40cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f9552700 1 -- 172.21.15.45:0/242037172 >> 172.21.15.45:0/242037172 conn(0x7f80f406ab20 msgr2=0x7f80f406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:09.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f9552700 1 -- 172.21.15.45:0/242037172 shutdown_connections 2023-08-27T11:25:09.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.205+0000 7f80f9552700 1 -- 172.21.15.45:0/242037172 wait complete. 2023-08-27T11:25:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.207+0000 7f80f9552700 1 Processor -- start 2023-08-27T11:25:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.207+0000 7f80f9552700 1 -- start start 2023-08-27T11:25:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f9552700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40ce290 0x7f80f40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f9552700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f40cf990 0x7f80f40b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f9552700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40cfd70 0x7f80f40b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f9552700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40cbe10 con 0x7f80f40ce290 2023-08-27T11:25:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f9552700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40cafa0 con 0x7f80f40cf990 2023-08-27T11:25:09.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f9552700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40cb200 con 0x7f80f40cfd70 2023-08-27T11:25:09.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40ce290 0x7f80f40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:09.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40ce290 0x7f80f40caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56146/0 (socket says 172.21.15.45:56146) 2023-08-27T11:25:09.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f2ffd700 1 -- 172.21.15.45:0/3337636415 learned_addr learned my addr 172.21.15.45:0/3337636415 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:09.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f27fc700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f40cf990 0x7f80f40b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.208+0000 7f80f37fe700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40cfd70 0x7f80f40b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80f2ffd700 1 -- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40cfd70 msgr2=0x7f80f40b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80f2ffd700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40cfd70 0x7f80f40b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80f2ffd700 1 -- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f40cf990 msgr2=0x7f80f40b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80f2ffd700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f40cf990 0x7f80f40b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:09.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80f2ffd700 1 -- 172.21.15.45:0/3337636415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e8009550 con 0x7f80f40ce290 2023-08-27T11:25:09.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80f37fe700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40cfd70 0x7f80f40b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:09.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80f2ffd700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40ce290 0x7f80f40caa60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f80e8014040 tx=0x7f80e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80db7fe700 1 -- 172.21.15.45:0/3337636415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e80036a0 con 0x7f80f40ce290 2023-08-27T11:25:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80db7fe700 1 -- 172.21.15.45:0/3337636415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e801e070 con 0x7f80f40ce290 2023-08-27T11:25:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f40cb4e0 con 0x7f80f40ce290 2023-08-27T11:25:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80db7fe700 1 -- 172.21.15.45:0/3337636415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e8003a20 con 0x7f80f40ce290 2023-08-27T11:25:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.209+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f40cbaa0 con 0x7f80f40ce290 2023-08-27T11:25:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.210+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f404eb90 con 0x7f80f40ce290 2023-08-27T11:25:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.210+0000 7f80db7fe700 1 -- 172.21.15.45:0/3337636415 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f80e8003b80 con 0x7f80f40ce290 2023-08-27T11:25:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.211+0000 7f80db7fe700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f80e006a200 0x7f80e006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.211+0000 7f80db7fe700 1 -- 172.21.15.45:0/3337636415 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f80e8089500 con 0x7f80f40ce290 2023-08-27T11:25:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.211+0000 7f80f27fc700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f80e006a200 0x7f80e006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.212+0000 7f80f27fc700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f80e006a200 0x7f80e006c6c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80f40b8140 tx=0x7f80e4008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.215+0000 7f80db7fe700 1 -- 172.21.15.45:0/3337636415 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f80e8054be0 con 0x7f80f40ce290 2023-08-27T11:25:09.346 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-08-27T11:25:09.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:09.415+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f80f404e620 con 0x7f80f40ce290 2023-08-27T11:25:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.044+0000 7f80db7fe700 1 -- 172.21.15.45:0/3337636415 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) v1 ==== 170+0+0 (secure 0 0 0) 0x7f80e8058200 con 0x7f80f40ce290 2023-08-27T11:25:10.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.051+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f80e006a200 msgr2=0x7f80e006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:10.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.051+0000 7f80f9552700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f80e006a200 0x7f80e006c6c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80f40b8140 tx=0x7f80e4008040 comp rx=0 tx=0).stop 2023-08-27T11:25:10.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.051+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40ce290 msgr2=0x7f80f40caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:10.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.051+0000 7f80f9552700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40ce290 0x7f80f40caa60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f80e8014040 tx=0x7f80e8004750 comp rx=0 tx=0).stop 2023-08-27T11:25:10.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.053+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 shutdown_connections 2023-08-27T11:25:10.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.053+0000 7f80f9552700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f80e006a200 0x7f80e006c6c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:10.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.053+0000 7f80f9552700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80f40ce290 0x7f80f40caa60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:10.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.053+0000 7f80f9552700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f80f40cfd70 0x7f80f40b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:10.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.053+0000 7f80f9552700 1 --2- 172.21.15.45:0/3337636415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f40cf990 0x7f80f40b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:10.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.053+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 >> 172.21.15.45:0/3337636415 conn(0x7f80f406ab20 msgr2=0x7f80f41578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.055+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 shutdown_connections 2023-08-27T11:25:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.056+0000 7f80f9552700 1 -- 172.21.15.45:0/3337636415 wait complete. 2023-08-27T11:25:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-08-27T11:25:10.080 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:25:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.214+0000 7fc10232e700 1 Processor -- start 2023-08-27T11:25:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.214+0000 7fc10232e700 1 -- start start 2023-08-27T11:25:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc10232e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0fc15d2c0 0x7fc0fc0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc10232e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0fc0ce4b0 0x7fc0fc0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc10232e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0fc15d6a0 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc10232e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc0d2c90 con 0x7fc0fc0ce4b0 2023-08-27T11:25:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc10232e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0d1bf0 con 0x7fc0fc15d2c0 2023-08-27T11:25:10.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc1008cb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0fc0ce4b0 0x7fc0fc0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc0fb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc0fc0b8c20 0x7fc0fc0cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42932/0 (socket says 172.21.15.45:42932) 2023-08-27T11:25:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc0fb7fe700 1 -- 172.21.15.45:0/4130400047 learned_addr learned my addr 172.21.15.45:0/4130400047 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc1008cb700 1 -- 172.21.15.45:0/4130400047 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0fc15d2c0 msgr2=0x7fc0fc0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc1008cb700 1 --2- 172.21.15.45:0/4130400047 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0fc15d2c0 0x7fc0fc0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc1008cb700 1 -- 172.21.15.45:0/4130400047 >> v1:172.21.15.45:6789/0 conn(0x7fc0fc0b8c20 legacy=0x7fc0fc0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:10.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc1008cb700 1 -- 172.21.15.45:0/4130400047 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc0d1ed0 con 0x7fc0fc0ce4b0 2023-08-27T11:25:10.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc1008cb700 1 --2- 172.21.15.45:0/4130400047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0ce4b0 0x7fc0fc0b6100 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009d90 tx=0x7fc0f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b4ef957eea2a56 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:10.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc0faffd700 1 -- 172.21.15.45:0/4130400047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f400e070 con 0x7fc0fc0ce4b0 2023-08-27T11:25:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc0faffd700 1 -- 172.21.15.45:0/4130400047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f400bbd0 con 0x7fc0fc0ce4b0 2023-08-27T11:25:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.215+0000 7fc0faffd700 1 -- 172.21.15.45:0/4130400047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4012430 con 0x7fc0fc0ce4b0 2023-08-27T11:25:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.216+0000 7fc10232e700 1 -- 172.21.15.45:0/4130400047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0ce4b0 msgr2=0x7fc0fc0b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.216+0000 7fc10232e700 1 --2- 172.21.15.45:0/4130400047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0ce4b0 0x7fc0fc0b6100 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009d90 tx=0x7fc0f400b330 comp rx=0 tx=0).stop 2023-08-27T11:25:10.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.216+0000 7fc10232e700 1 -- 172.21.15.45:0/4130400047 shutdown_connections 2023-08-27T11:25:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.216+0000 7fc10232e700 1 --2- 172.21.15.45:0/4130400047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0ce4b0 0x7fc0fc0b6100 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.216+0000 7fc10232e700 1 --2- 172.21.15.45:0/4130400047 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0fc15d2c0 0x7fc0fc0d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.216+0000 7fc10232e700 1 -- 172.21.15.45:0/4130400047 >> 172.21.15.45:0/4130400047 conn(0x7fc0fc0d2fd0 msgr2=0x7fc0fc0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:10.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.216+0000 7fc10232e700 1 -- 172.21.15.45:0/4130400047 shutdown_connections 2023-08-27T11:25:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.216+0000 7fc10232e700 1 -- 172.21.15.45:0/4130400047 wait complete. 2023-08-27T11:25:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.217+0000 7fc10232e700 1 Processor -- start 2023-08-27T11:25:10.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.217+0000 7fc10232e700 1 -- start start 2023-08-27T11:25:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc10232e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0b8c20 0x7fc0fc156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc10232e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0fc0ce4b0 0x7fc0fc0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc10232e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0fc15d2c0 0x7fc0fc15fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc10232e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0cb4e0 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc10232e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc156790 con 0x7fc0fc15d2c0 2023-08-27T11:25:10.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc10232e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0cc300 con 0x7fc0fc0ce4b0 2023-08-27T11:25:10.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc0fbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0b8c20 0x7fc0fc156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:10.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc0fbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0b8c20 0x7fc0fc156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56160/0 (socket says 172.21.15.45:56160) 2023-08-27T11:25:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc0fbfff700 1 -- 172.21.15.45:0/2187379882 learned_addr learned my addr 172.21.15.45:0/2187379882 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc0fb7fe700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0fc0ce4b0 0x7fc0fc0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.218+0000 7fc1008cb700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0fc15d2c0 0x7fc0fc15fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc0fbfff700 1 -- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0fc0ce4b0 msgr2=0x7fc0fc0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc0fbfff700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0fc0ce4b0 0x7fc0fc0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc0fbfff700 1 -- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0fc15d2c0 msgr2=0x7fc0fc15fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc0fbfff700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0fc15d2c0 0x7fc0fc15fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc0fbfff700 1 -- 172.21.15.45:0/2187379882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4009a70 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc0fbfff700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0b8c20 0x7fc0fc156250 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b4d0 tx=0x7fc0ec00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc0f8ff9700 1 -- 172.21.15.45:0/2187379882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00bea0 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc0f8ff9700 1 -- 172.21.15.45:0/2187379882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec011830 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0fc0cb6f0 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc0f8ff9700 1 -- 172.21.15.45:0/2187379882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec011be0 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.219+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc0cbcb0 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.220+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc04e620 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.221+0000 7fc0f8ff9700 1 -- 172.21.15.45:0/2187379882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc0ec011d40 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.221+0000 7fc0f8ff9700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc0e806a200 0x7fc0e806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.221+0000 7fc0f8ff9700 1 -- 172.21.15.45:0/2187379882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fc0ec088bc0 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.221+0000 7fc0fb7fe700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc0e806a200 0x7fc0e806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.222+0000 7fc0fb7fe700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc0e806a200 0x7fc0e806c6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc15f140 tx=0x7fc0f0009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:10.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.225+0000 7fc0f8ff9700 1 -- 172.21.15.45:0/2187379882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc0ec054120 con 0x7fc0fc0b8c20 2023-08-27T11:25:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:10.471+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc0fc0cda70 con 0x7fc0fc0b8c20 2023-08-27T11:25:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.075+0000 7fc0f8ff9700 1 -- 172.21.15.45:0/2187379882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc0ec0150c0 con 0x7fc0fc0b8c20 2023-08-27T11:25:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.086+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc0e806a200 msgr2=0x7fc0e806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.086+0000 7fc10232e700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc0e806a200 0x7fc0e806c6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc15f140 tx=0x7fc0f0009040 comp rx=0 tx=0).stop 2023-08-27T11:25:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.086+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0b8c20 msgr2=0x7fc0fc156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.086+0000 7fc10232e700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0b8c20 0x7fc0fc156250 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b4d0 tx=0x7fc0ec00b7e0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.090+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 shutdown_connections 2023-08-27T11:25:11.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.090+0000 7fc10232e700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fc0e806a200 0x7fc0e806c6c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.090+0000 7fc10232e700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0fc0b8c20 0x7fc0fc156250 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.090+0000 7fc10232e700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0fc15d2c0 0x7fc0fc15fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.090+0000 7fc10232e700 1 --2- 172.21.15.45:0/2187379882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc0fc0ce4b0 0x7fc0fc0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.090+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 >> 172.21.15.45:0/2187379882 conn(0x7fc0fc0d2fd0 msgr2=0x7fc0fc0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.091+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 shutdown_connections 2023-08-27T11:25:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.091+0000 7fc10232e700 1 -- 172.21.15.45:0/2187379882 wait complete. 2023-08-27T11:25:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 8 and data pool 9 2023-08-27T11:25:11.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:11.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.237+0000 7f4c253a0700 1 Processor -- start 2023-08-27T11:25:11.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.237+0000 7f4c253a0700 1 -- start start 2023-08-27T11:25:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.238+0000 7f4c253a0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4c2015e400 0x7f4c200cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.238+0000 7f4c253a0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2006dcd0 con 0x7f4c20159e60 2023-08-27T11:25:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.238+0000 7f4c253a0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2015e970 con 0x7f4c200cba50 2023-08-27T11:25:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.238+0000 7f4c253a0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2006e830 con 0x7f4c2015e400 2023-08-27T11:25:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.238+0000 7f4c1ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4c20159e60 0x7f4c200cd8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42956/0 (socket says 172.21.15.45:42956) 2023-08-27T11:25:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.238+0000 7f4c1ffff700 1 -- 172.21.15.45:0/3708919478 learned_addr learned my addr 172.21.15.45:0/3708919478 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.238+0000 7f4c1f7fe700 1 --2- 172.21.15.45:0/3708919478 >> v2:172.21.15.135:3300/0 conn(0x7f4c2015e400 0x7f4c200cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.239+0000 7f4c1effd700 1 -- 172.21.15.45:0/3708919478 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642766588 0 0) 0x7f4c2006dcd0 con 0x7f4c20159e60 2023-08-27T11:25:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.239+0000 7f4c1effd700 1 -- 172.21.15.45:0/3708919478 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c080035b0 con 0x7f4c20159e60 2023-08-27T11:25:11.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.239+0000 7f4c1f7fe700 1 -- 172.21.15.45:0/3708919478 >> v1:172.21.15.135:6789/0 conn(0x7f4c200cba50 legacy=0x7f4c2006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.239+0000 7f4c1f7fe700 1 -- 172.21.15.45:0/3708919478 >> v1:172.21.15.45:6789/0 conn(0x7f4c20159e60 legacy=0x7f4c200cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.239+0000 7f4c1f7fe700 1 -- 172.21.15.45:0/3708919478 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200cd550 con 0x7f4c2015e400 2023-08-27T11:25:11.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.239+0000 7f4c1f7fe700 1 --2- 172.21.15.45:0/3708919478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 0x7f4c200cf620 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009840 tx=0x7f4c14009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cf5d0bcc8df7d67 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:11.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.239+0000 7f4c1effd700 1 -- 172.21.15.45:0/3708919478 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14011830 con 0x7f4c2015e400 2023-08-27T11:25:11.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.239+0000 7f4c1effd700 1 -- 172.21.15.45:0/3708919478 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c14011990 con 0x7f4c2015e400 2023-08-27T11:25:11.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.239+0000 7f4c1effd700 1 -- 172.21.15.45:0/3708919478 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14011d20 con 0x7f4c2015e400 2023-08-27T11:25:11.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.240+0000 7f4c253a0700 1 -- 172.21.15.45:0/3708919478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 msgr2=0x7f4c200cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.240+0000 7f4c253a0700 1 --2- 172.21.15.45:0/3708919478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 0x7f4c200cf620 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009840 tx=0x7f4c14009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:11.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.240+0000 7f4c253a0700 1 -- 172.21.15.45:0/3708919478 shutdown_connections 2023-08-27T11:25:11.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.240+0000 7f4c253a0700 1 --2- 172.21.15.45:0/3708919478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 0x7f4c200cf620 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.240+0000 7f4c253a0700 1 -- 172.21.15.45:0/3708919478 >> 172.21.15.45:0/3708919478 conn(0x7f4c200cc780 msgr2=0x7f4c200ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.240+0000 7f4c253a0700 1 -- 172.21.15.45:0/3708919478 shutdown_connections 2023-08-27T11:25:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.240+0000 7f4c253a0700 1 -- 172.21.15.45:0/3708919478 wait complete. 2023-08-27T11:25:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.241+0000 7f4c253a0700 1 Processor -- start 2023-08-27T11:25:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.241+0000 7f4c253a0700 1 -- start start 2023-08-27T11:25:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c253a0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c200cba50 0x7f4c20155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c253a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20159e60 0x7f4c2015b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c253a0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 0x7f4c2006a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c253a0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200ad380 con 0x7f4c20159e60 2023-08-27T11:25:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c253a0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2015bc20 con 0x7f4c2015e400 2023-08-27T11:25:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c253a0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200ad7a0 con 0x7f4c200cba50 2023-08-27T11:25:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c1f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20159e60 0x7f4c2015b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c1f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20159e60 0x7f4c2015b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56188/0 (socket says 172.21.15.45:56188) 2023-08-27T11:25:11.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c1f7fe700 1 -- 172.21.15.45:0/3989951417 learned_addr learned my addr 172.21.15.45:0/3989951417 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:11.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c1ffff700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c200cba50 0x7f4c20155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c24b9f700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 0x7f4c2006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c1f7fe700 1 -- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c200cba50 msgr2=0x7f4c20155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c1f7fe700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c200cba50 0x7f4c20155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c1f7fe700 1 -- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 msgr2=0x7f4c2006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c1f7fe700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 0x7f4c2006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c1f7fe700 1 -- 172.21.15.45:0/3989951417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c140094a0 con 0x7f4c20159e60 2023-08-27T11:25:11.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.242+0000 7f4c24b9f700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 0x7f4c2006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.243+0000 7f4c1f7fe700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20159e60 0x7f4c2015b160 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009fd0 tx=0x7f4c14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.243+0000 7f4c1cff9700 1 -- 172.21.15.45:0/3989951417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c140038a0 con 0x7f4c20159e60 2023-08-27T11:25:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.243+0000 7f4c1cff9700 1 -- 172.21.15.45:0/3989951417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1401f070 con 0x7f4c20159e60 2023-08-27T11:25:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.243+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c200ac9e0 con 0x7f4c20159e60 2023-08-27T11:25:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.243+0000 7f4c1cff9700 1 -- 172.21.15.45:0/3989951417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14003c40 con 0x7f4c20159e60 2023-08-27T11:25:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.243+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c200acfa0 con 0x7f4c20159e60 2023-08-27T11:25:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.244+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c200ad4c0 con 0x7f4c20159e60 2023-08-27T11:25:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.244+0000 7f4c1cff9700 1 -- 172.21.15.45:0/3989951417 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4c14026480 con 0x7f4c20159e60 2023-08-27T11:25:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.244+0000 7f4c1cff9700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f4c0c06a200 0x7f4c0c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.245+0000 7f4c1cff9700 1 -- 172.21.15.45:0/3989951417 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4c14021070 con 0x7f4c20159e60 2023-08-27T11:25:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.245+0000 7f4c1ffff700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f4c0c06a200 0x7f4c0c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.245+0000 7f4c1ffff700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f4c0c06a200 0x7f4c0c06c6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c100011e0 tx=0x7f4c10009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:11.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.249+0000 7f4c1cff9700 1 -- 172.21.15.45:0/3989951417 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4c14055620 con 0x7f4c20159e60 2023-08-27T11:25:11.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.450+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c200726a0 con 0x7f4c20159e60 2023-08-27T11:25:11.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.451+0000 7f4c1cff9700 1 -- 172.21.15.45:0/3989951417 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+18647 (secure 0 0 0) 0x7f4c14015090 con 0x7f4c20159e60 2023-08-27T11:25:11.452 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:11.453 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":69,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:25:10.472570+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:08.337329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:09.446448+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f4c0c06a200 msgr2=0x7f4c0c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f4c0c06a200 0x7f4c0c06c6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c100011e0 tx=0x7f4c10009040 comp rx=0 tx=0).stop 2023-08-27T11:25:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20159e60 msgr2=0x7f4c2015b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20159e60 0x7f4c2015b160 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009fd0 tx=0x7f4c14004750 comp rx=0 tx=0).stop 2023-08-27T11:25:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 shutdown_connections 2023-08-27T11:25:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f4c0c06a200 0x7f4c0c06c6c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c2015e400 0x7f4c2006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4c200cba50 0x7f4c20155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 --2- 172.21.15.45:0/3989951417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c20159e60 0x7f4c2015b160 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 >> 172.21.15.45:0/3989951417 conn(0x7f4c200cc780 msgr2=0x7f4c200c4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:11.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 shutdown_connections 2023-08-27T11:25:11.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.453+0000 7f4c253a0700 1 -- 172.21.15.45:0/3989951417 wait complete. 2023-08-27T11:25:11.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.592+0000 7f2b58f74700 1 Processor -- start 2023-08-27T11:25:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.592+0000 7f2b58f74700 1 -- start start 2023-08-27T11:25:11.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b58f74700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b54070e80 0x7f2b5406e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b58f74700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2b5406a990 0x7f2b5415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b58f74700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b540cbf00 con 0x7f2b54155660 2023-08-27T11:25:11.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b58f74700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5415c760 con 0x7f2b54070e80 2023-08-27T11:25:11.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b58f74700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b540cc180 con 0x7f2b5406a990 2023-08-27T11:25:11.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b51d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b54070e80 0x7f2b5406e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b5259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2b54155660 0x7f2b5406a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42970/0 (socket says 172.21.15.45:42970) 2023-08-27T11:25:11.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b5259c700 1 -- 172.21.15.45:0/2344992984 learned_addr learned my addr 172.21.15.45:0/2344992984 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b52d9d700 1 --2- 172.21.15.45:0/2344992984 >> v2:172.21.15.135:3300/0 conn(0x7f2b5406a990 0x7f2b5415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b51d9b700 1 -- 172.21.15.45:0/2344992984 >> v2:172.21.15.135:3300/0 conn(0x7f2b5406a990 msgr2=0x7f2b5415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b51d9b700 1 --2- 172.21.15.45:0/2344992984 >> v2:172.21.15.135:3300/0 conn(0x7f2b5406a990 0x7f2b5415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b51d9b700 1 -- 172.21.15.45:0/2344992984 >> v1:172.21.15.45:6789/0 conn(0x7f2b54155660 legacy=0x7f2b5406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b5159a700 1 -- 172.21.15.45:0/2344992984 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59910862 0 0) 0x7f2b540cbf00 con 0x7f2b54155660 2023-08-27T11:25:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b51d9b700 1 -- 172.21.15.45:0/2344992984 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540caa60 con 0x7f2b54070e80 2023-08-27T11:25:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b52d9d700 1 --2- 172.21.15.45:0/2344992984 >> v2:172.21.15.135:3300/0 conn(0x7f2b5406a990 0x7f2b5415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.593+0000 7f2b51d9b700 1 --2- 172.21.15.45:0/2344992984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b54070e80 0x7f2b5406e850 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2b540cba40 tx=0x7f2b48009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c87cd4afd725a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b5159a700 1 -- 172.21.15.45:0/2344992984 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48011830 con 0x7f2b54070e80 2023-08-27T11:25:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b5159a700 1 -- 172.21.15.45:0/2344992984 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48011990 con 0x7f2b54070e80 2023-08-27T11:25:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b5159a700 1 -- 172.21.15.45:0/2344992984 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48011d20 con 0x7f2b54070e80 2023-08-27T11:25:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b58f74700 1 -- 172.21.15.45:0/2344992984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b54070e80 msgr2=0x7f2b5406e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b58f74700 1 --2- 172.21.15.45:0/2344992984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b54070e80 0x7f2b5406e850 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2b540cba40 tx=0x7f2b48009b20 comp rx=0 tx=0).stop 2023-08-27T11:25:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b58f74700 1 -- 172.21.15.45:0/2344992984 shutdown_connections 2023-08-27T11:25:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b58f74700 1 --2- 172.21.15.45:0/2344992984 >> v2:172.21.15.135:3300/0 conn(0x7f2b5406a990 0x7f2b5415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b58f74700 1 --2- 172.21.15.45:0/2344992984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b54070e80 0x7f2b5406e850 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b58f74700 1 -- 172.21.15.45:0/2344992984 >> 172.21.15.45:0/2344992984 conn(0x7f2b540cfe50 msgr2=0x7f2b540d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b58f74700 1 -- 172.21.15.45:0/2344992984 shutdown_connections 2023-08-27T11:25:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.594+0000 7f2b58f74700 1 -- 172.21.15.45:0/2344992984 wait complete. 2023-08-27T11:25:11.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.596+0000 7f2b58f74700 1 Processor -- start 2023-08-27T11:25:11.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.596+0000 7f2b58f74700 1 -- start start 2023-08-27T11:25:11.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.596+0000 7f2b58f74700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b5406a990 0x7f2b54144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.596+0000 7f2b58f74700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b54070e80 0x7f2b541434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.596+0000 7f2b58f74700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b54155660 0x7f2b54145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.596+0000 7f2b58f74700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b54143e60 con 0x7f2b5406a990 2023-08-27T11:25:11.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.596+0000 7f2b58f74700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b54144140 con 0x7f2b54070e80 2023-08-27T11:25:11.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.596+0000 7f2b58f74700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b54144420 con 0x7f2b54155660 2023-08-27T11:25:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b5259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b5406a990 0x7f2b54144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b5259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b5406a990 0x7f2b54144f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56202/0 (socket says 172.21.15.45:56202) 2023-08-27T11:25:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b5259c700 1 -- 172.21.15.45:0/4195316391 learned_addr learned my addr 172.21.15.45:0/4195316391 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b51d9b700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b54070e80 0x7f2b541434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b52d9d700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b54155660 0x7f2b54145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b5259c700 1 -- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b54155660 msgr2=0x7f2b54145c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b5259c700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b54155660 0x7f2b54145c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b5259c700 1 -- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b54070e80 msgr2=0x7f2b541434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b5259c700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b54070e80 0x7f2b541434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b5259c700 1 -- 172.21.15.45:0/4195316391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480094a0 con 0x7f2b5406a990 2023-08-27T11:25:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b5259c700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b5406a990 0x7f2b54144f00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c550 tx=0x7f2b4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b42ffd700 1 -- 172.21.15.45:0/4195316391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44004c80 con 0x7f2b5406a990 2023-08-27T11:25:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b42ffd700 1 -- 172.21.15.45:0/4195316391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b44004e00 con 0x7f2b5406a990 2023-08-27T11:25:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b42ffd700 1 -- 172.21.15.45:0/4195316391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44007310 con 0x7f2b5406a990 2023-08-27T11:25:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b54144700 con 0x7f2b5406a990 2023-08-27T11:25:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.597+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b54143a30 con 0x7f2b5406a990 2023-08-27T11:25:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.598+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b54143c20 con 0x7f2b5406a990 2023-08-27T11:25:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.599+0000 7f2b42ffd700 1 -- 172.21.15.45:0/4195316391 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2b44007470 con 0x7f2b5406a990 2023-08-27T11:25:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.599+0000 7f2b42ffd700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f2b3406a200 0x7f2b3406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.599+0000 7f2b42ffd700 1 -- 172.21.15.45:0/4195316391 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2b44088530 con 0x7f2b5406a990 2023-08-27T11:25:11.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.599+0000 7f2b51d9b700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f2b3406a200 0x7f2b3406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.600+0000 7f2b51d9b700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f2b3406a200 0x7f2b3406c6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2b540cba40 tx=0x7f2b48008030 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:11.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.603+0000 7f2b42ffd700 1 -- 172.21.15.45:0/4195316391 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2b54143c20 con 0x7f2b5406a990 2023-08-27T11:25:11.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.804+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b54143c20 con 0x7f2b5406a990 2023-08-27T11:25:11.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.805+0000 7f2b42ffd700 1 -- 172.21.15.45:0/4195316391 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+18647 (secure 0 0 0) 0x7f2b54143c20 con 0x7f2b5406a990 2023-08-27T11:25:11.806 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:11.807 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":69,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:25:10.472570+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:08.337329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:09.446448+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f2b3406a200 msgr2=0x7f2b3406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f2b3406a200 0x7f2b3406c6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2b540cba40 tx=0x7f2b48008030 comp rx=0 tx=0).stop 2023-08-27T11:25:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b5406a990 msgr2=0x7f2b54144f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b5406a990 0x7f2b54144f00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400c550 tx=0x7f2b4400c910 comp rx=0 tx=0).stop 2023-08-27T11:25:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 shutdown_connections 2023-08-27T11:25:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f2b3406a200 0x7f2b3406c6c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b5406a990 0x7f2b54144f00 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2b54155660 0x7f2b54145c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 --2- 172.21.15.45:0/4195316391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b54070e80 0x7f2b541434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 >> 172.21.15.45:0/4195316391 conn(0x7f2b540cfe50 msgr2=0x7f2b54151ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 shutdown_connections 2023-08-27T11:25:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.807+0000 7f2b58f74700 1 -- 172.21.15.45:0/4195316391 wait complete. 2023-08-27T11:25:11.821 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:25:11.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.948+0000 7f5108d1a700 1 Processor -- start 2023-08-27T11:25:11.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.948+0000 7f5108d1a700 1 -- start start 2023-08-27T11:25:11.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5108d1a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f510415af30 0x7f51040cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5108d1a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51040cf960 0x7f51040cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5108d1a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f510415b310 0x7f51040cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5108d1a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510406f9c0 con 0x7f51040cf960 2023-08-27T11:25:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5108d1a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51040cb8b0 con 0x7f510415af30 2023-08-27T11:25:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5108d1a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51040cecd0 con 0x7f510415b310 2023-08-27T11:25:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5101d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51040cf960 0x7f51040cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5101d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51040cf960 0x7f51040cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56208/0 (socket says 172.21.15.45:56208) 2023-08-27T11:25:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5101d9b700 1 -- 172.21.15.45:0/4023632351 learned_addr learned my addr 172.21.15.45:0/4023632351 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f510259c700 1 --2- 172.21.15.45:0/4023632351 >> v2:172.21.15.135:3300/0 conn(0x7f510415af30 0x7f51040cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5102d9d700 1 --2- 172.21.15.45:0/4023632351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f510415b310 0x7f51040cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5101d9b700 1 -- 172.21.15.45:0/4023632351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f510415b310 msgr2=0x7f51040cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5101d9b700 1 --2- 172.21.15.45:0/4023632351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f510415b310 0x7f51040cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5101d9b700 1 -- 172.21.15.45:0/4023632351 >> v2:172.21.15.135:3300/0 conn(0x7f510415af30 msgr2=0x7f51040cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5101d9b700 1 --2- 172.21.15.45:0/4023632351 >> v2:172.21.15.135:3300/0 conn(0x7f510415af30 0x7f51040cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5101d9b700 1 -- 172.21.15.45:0/4023632351 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51040cbae0 con 0x7f51040cf960 2023-08-27T11:25:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f510259c700 1 --2- 172.21.15.45:0/4023632351 >> v2:172.21.15.135:3300/0 conn(0x7f510415af30 0x7f51040cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.949+0000 7f5102d9d700 1 --2- 172.21.15.45:0/4023632351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f510415b310 0x7f51040cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5101d9b700 1 --2- 172.21.15.45:0/4023632351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51040cf960 0x7f51040cd770 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f510409a8d0 tx=0x7f50f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7753d23e62fd15e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f510159a700 1 -- 172.21.15.45:0/4023632351 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f400b870 con 0x7f51040cf960 2023-08-27T11:25:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f510159a700 1 -- 172.21.15.45:0/4023632351 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f400b9d0 con 0x7f51040cf960 2023-08-27T11:25:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f510159a700 1 -- 172.21.15.45:0/4023632351 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f400bd60 con 0x7f51040cf960 2023-08-27T11:25:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5108d1a700 1 -- 172.21.15.45:0/4023632351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51040cf960 msgr2=0x7f51040cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5108d1a700 1 --2- 172.21.15.45:0/4023632351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51040cf960 0x7f51040cd770 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f510409a8d0 tx=0x7f50f4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5108d1a700 1 -- 172.21.15.45:0/4023632351 shutdown_connections 2023-08-27T11:25:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5108d1a700 1 --2- 172.21.15.45:0/4023632351 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f510415b310 0x7f51040cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5108d1a700 1 --2- 172.21.15.45:0/4023632351 >> v2:172.21.15.135:3300/0 conn(0x7f510415af30 0x7f51040cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5108d1a700 1 --2- 172.21.15.45:0/4023632351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51040cf960 0x7f51040cd770 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5108d1a700 1 -- 172.21.15.45:0/4023632351 >> 172.21.15.45:0/4023632351 conn(0x7f510409a1a0 msgr2=0x7f51040d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5108d1a700 1 -- 172.21.15.45:0/4023632351 shutdown_connections 2023-08-27T11:25:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.950+0000 7f5108d1a700 1 -- 172.21.15.45:0/4023632351 wait complete. 2023-08-27T11:25:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.952+0000 7f5108d1a700 1 Processor -- start 2023-08-27T11:25:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.952+0000 7f5108d1a700 1 -- start start 2023-08-27T11:25:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.952+0000 7f5108d1a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51040cf960 0x7f510415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.952+0000 7f5108d1a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510415af30 0x7f51041f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5108d1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415b310 0x7f510414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5108d1a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510409cbd0 con 0x7f510415b310 2023-08-27T11:25:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5108d1a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510409bf50 con 0x7f510415af30 2023-08-27T11:25:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5108d1a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510409c230 con 0x7f51040cf960 2023-08-27T11:25:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5102d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415b310 0x7f510414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5102d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415b310 0x7f510414fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56218/0 (socket says 172.21.15.45:56218) 2023-08-27T11:25:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5102d9d700 1 -- 172.21.15.45:0/2627434977 learned_addr learned my addr 172.21.15.45:0/2627434977 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5101d9b700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510415af30 0x7f51041f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f510259c700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51040cf960 0x7f510415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5102d9d700 1 -- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51040cf960 msgr2=0x7f510415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5102d9d700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51040cf960 0x7f510415e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5102d9d700 1 -- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510415af30 msgr2=0x7f51041f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5102d9d700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510415af30 0x7f51041f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5102d9d700 1 -- 172.21.15.45:0/2627434977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f4009550 con 0x7f510415b310 2023-08-27T11:25:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5102d9d700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415b310 0x7f510414fda0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009f70 tx=0x7f50fc00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f50f2ffd700 1 -- 172.21.15.45:0/2627434977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003a20 con 0x7f510415b310 2023-08-27T11:25:11.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f50f2ffd700 1 -- 172.21.15.45:0/2627434977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc00f040 con 0x7f510415b310 2023-08-27T11:25:11.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f50f2ffd700 1 -- 172.21.15.45:0/2627434977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003de0 con 0x7f510415b310 2023-08-27T11:25:11.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510409c510 con 0x7f510415b310 2023-08-27T11:25:11.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.953+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51041f4ed0 con 0x7f510415b310 2023-08-27T11:25:11.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.954+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51041f50c0 con 0x7f510415b310 2023-08-27T11:25:11.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.955+0000 7f50f2ffd700 1 -- 172.21.15.45:0/2627434977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f50fc004030 con 0x7f510415b310 2023-08-27T11:25:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.955+0000 7f50f2ffd700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f50e406a200 0x7f50e406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.955+0000 7f50f2ffd700 1 -- 172.21.15.45:0/2627434977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f50fc088ad0 con 0x7f510415b310 2023-08-27T11:25:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.955+0000 7f510259c700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f50e406a200 0x7f50e406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.956+0000 7f510259c700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f50e406a200 0x7f50e406c6c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009930 tx=0x7f50f8008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:11.960+0000 7f50f2ffd700 1 -- 172.21.15.45:0/2627434977 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f50fc053e30 con 0x7f510415b310 2023-08-27T11:25:12.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:12.210+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f510409cd10 con 0x7f510415b310 2023-08-27T11:25:13.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.082+0000 7f50f2ffd700 1 -- 172.21.15.45:0/2627434977 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f50fc057450 con 0x7f510415b310 2023-08-27T11:25:13.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.084+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f50e406a200 msgr2=0x7f50e406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:13.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.084+0000 7f5108d1a700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f50e406a200 0x7f50e406c6c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009930 tx=0x7f50f8008040 comp rx=0 tx=0).stop 2023-08-27T11:25:13.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.084+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415b310 msgr2=0x7f510414fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:13.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.084+0000 7f5108d1a700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415b310 0x7f510414fda0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009f70 tx=0x7f50fc00d460 comp rx=0 tx=0).stop 2023-08-27T11:25:13.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.085+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 shutdown_connections 2023-08-27T11:25:13.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.085+0000 7f5108d1a700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f50e406a200 0x7f50e406c6c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.085+0000 7f5108d1a700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415b310 0x7f510414fda0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.085+0000 7f5108d1a700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f51040cf960 0x7f510415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.085+0000 7f5108d1a700 1 --2- 172.21.15.45:0/2627434977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510415af30 0x7f51041f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.085+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 >> 172.21.15.45:0/2627434977 conn(0x7f510409a1a0 msgr2=0x7f51040ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:13.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.085+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 shutdown_connections 2023-08-27T11:25:13.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.085+0000 7f5108d1a700 1 -- 172.21.15.45:0/2627434977 wait complete. 2023-08-27T11:25:13.101 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:25:13.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:25:13.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.236+0000 7fa1ed7dc700 1 Processor -- start 2023-08-27T11:25:13.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.236+0000 7fa1ed7dc700 1 -- start start 2023-08-27T11:25:13.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1ed7dc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1e815d2c0 0x7fa1e80d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:13.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1ed7dc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e80b8c20 0x7fa1e80cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:13.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1ed7dc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa1e80ce2e0 0x7fa1e806c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:13.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1ed7dc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e80ce770 con 0x7fa1e815d2c0 2023-08-27T11:25:13.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1ed7dc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e815d6a0 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1ed7dc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80d2c90 con 0x7fa1e80b8c20 2023-08-27T11:25:13.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1e815d2c0 0x7fa1e80d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:13.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1e815d2c0 0x7fa1e80d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56222/0 (socket says 172.21.15.45:56222) 2023-08-27T11:25:13.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e6ffd700 1 -- 172.21.15.45:0/1549206038 learned_addr learned my addr 172.21.15.45:0/1549206038 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e67fc700 1 --2- 172.21.15.45:0/1549206038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e80b8c20 0x7fa1e80cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e6ffd700 1 -- 172.21.15.45:0/1549206038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e80b8c20 msgr2=0x7fa1e80cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e77fe700 1 --2- 172.21.15.45:0/1549206038 >> v2:172.21.15.135:3300/0 conn(0x7fa1e80ce2e0 0x7fa1e806c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e6ffd700 1 --2- 172.21.15.45:0/1549206038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e80b8c20 0x7fa1e80cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e6ffd700 1 -- 172.21.15.45:0/1549206038 >> v2:172.21.15.135:3300/0 conn(0x7fa1e80ce2e0 msgr2=0x7fa1e806c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:13.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e6ffd700 1 --2- 172.21.15.45:0/1549206038 >> v2:172.21.15.135:3300/0 conn(0x7fa1e80ce2e0 0x7fa1e806c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e6ffd700 1 -- 172.21.15.45:0/1549206038 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e80d1bf0 con 0x7fa1e815d2c0 2023-08-27T11:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e77fe700 1 --2- 172.21.15.45:0/1549206038 >> v2:172.21.15.135:3300/0 conn(0x7fa1e80ce2e0 0x7fa1e806c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e67fc700 1 --2- 172.21.15.45:0/1549206038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e80b8c20 0x7fa1e80cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e6ffd700 1 --2- 172.21.15.45:0/1549206038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e815d2c0 0x7fa1e80d13a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80b5ba0 tx=0x7fa1dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20b5f9d493b0671 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:13.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e5ffb700 1 -- 172.21.15.45:0/1549206038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc00b870 con 0x7fa1e815d2c0 2023-08-27T11:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e5ffb700 1 -- 172.21.15.45:0/1549206038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc00b9d0 con 0x7fa1e815d2c0 2023-08-27T11:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.237+0000 7fa1e5ffb700 1 -- 172.21.15.45:0/1549206038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc00bd60 con 0x7fa1e815d2c0 2023-08-27T11:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.238+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/1549206038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e815d2c0 msgr2=0x7fa1e80d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.238+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/1549206038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e815d2c0 0x7fa1e80d13a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80b5ba0 tx=0x7fa1dc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.238+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/1549206038 shutdown_connections 2023-08-27T11:25:13.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.238+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/1549206038 >> v2:172.21.15.135:3300/0 conn(0x7fa1e80ce2e0 0x7fa1e806c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.238+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/1549206038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e815d2c0 0x7fa1e80d13a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.238+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/1549206038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e80b8c20 0x7fa1e80cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.238+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/1549206038 >> 172.21.15.45:0/1549206038 conn(0x7fa1e80d2fd0 msgr2=0x7fa1e80d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:13.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.238+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/1549206038 shutdown_connections 2023-08-27T11:25:13.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.238+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/1549206038 wait complete. 2023-08-27T11:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.239+0000 7fa1ed7dc700 1 Processor -- start 2023-08-27T11:25:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.239+0000 7fa1ed7dc700 1 -- start start 2023-08-27T11:25:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1ed7dc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1e80b8c20 0x7fa1e806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1ed7dc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e80ce2e0 0x7fa1e8151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1ed7dc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e815d2c0 0x7fa1e80cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1ed7dc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e815f0c0 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1ed7dc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e806d0e0 con 0x7fa1e80b8c20 2023-08-27T11:25:13.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1ed7dc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8151a70 con 0x7fa1e815d2c0 2023-08-27T11:25:13.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1e67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e80ce2e0 0x7fa1e8151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:13.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1e67fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e80ce2e0 0x7fa1e8151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56230/0 (socket says 172.21.15.45:56230) 2023-08-27T11:25:13.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1e67fc700 1 -- 172.21.15.45:0/3693327585 learned_addr learned my addr 172.21.15.45:0/3693327585 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1e6ffd700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1e80b8c20 0x7fa1e806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.240+0000 7fa1e77fe700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e815d2c0 0x7fa1e80cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1e67fc700 1 -- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e815d2c0 msgr2=0x7fa1e80cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:13.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1e67fc700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e815d2c0 0x7fa1e80cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1e67fc700 1 -- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1e80b8c20 msgr2=0x7fa1e806cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1e67fc700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1e80b8c20 0x7fa1e806cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1e67fc700 1 -- 172.21.15.45:0/3693327585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc009550 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1e77fe700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e815d2c0 0x7fa1e80cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:13.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1e67fc700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e80ce2e0 0x7fa1e8151530 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c740 tx=0x7fa1d800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:13.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1cf7fe700 1 -- 172.21.15.45:0/3693327585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8011800 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1cf7fe700 1 -- 172.21.15.45:0/3693327585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8011960 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e815e1f0 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1cf7fe700 1 -- 172.21.15.45:0/3693327585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8011cd0 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.241+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e815e720 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.242+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e804e620 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.242+0000 7fa1cf7fe700 1 -- 172.21.15.45:0/3693327585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa1d8021350 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.243+0000 7fa1cf7fe700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa1d406a200 0x7fa1d406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:13.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.243+0000 7fa1cf7fe700 1 -- 172.21.15.45:0/3693327585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fa1d8088750 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.243+0000 7fa1e6ffd700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa1d406a200 0x7fa1d406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:13.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.244+0000 7fa1e6ffd700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa1d406a200 0x7fa1d406c6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc014040 tx=0x7fa1dc008030 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:13.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.248+0000 7fa1cf7fe700 1 -- 172.21.15.45:0/3693327585 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa1d80575d0 con 0x7fa1e80ce2e0 2023-08-27T11:25:13.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:13.494+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa1e80c68a0 con 0x7fa1e80ce2e0 2023-08-27T11:25:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.083+0000 7fa1cf7fe700 1 -- 172.21.15.45:0/3693327585 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa1d8057160 con 0x7fa1e80ce2e0 2023-08-27T11:25:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa1d406a200 msgr2=0x7fa1d406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa1d406a200 0x7fa1d406c6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc014040 tx=0x7fa1dc008030 comp rx=0 tx=0).stop 2023-08-27T11:25:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e80ce2e0 msgr2=0x7fa1e8151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e80ce2e0 0x7fa1e8151530 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c740 tx=0x7fa1d800ca50 comp rx=0 tx=0).stop 2023-08-27T11:25:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 shutdown_connections 2023-08-27T11:25:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fa1d406a200 0x7fa1d406c6c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1e815d2c0 0x7fa1e80cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1e80b8c20 0x7fa1e806cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 --2- 172.21.15.45:0/3693327585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e80ce2e0 0x7fa1e8151530 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 >> 172.21.15.45:0/3693327585 conn(0x7fa1e80d2fd0 msgr2=0x7fa1e80c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:14.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 shutdown_connections 2023-08-27T11:25:14.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.086+0000 7fa1ed7dc700 1 -- 172.21.15.45:0/3693327585 wait complete. 2023-08-27T11:25:14.103 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:25:14.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:25:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.235+0000 7f0c65380700 1 Processor -- start 2023-08-27T11:25:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.235+0000 7f0c65380700 1 -- start start 2023-08-27T11:25:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.236+0000 7f0c65380700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c600b8c20 0x7f0c600cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.236+0000 7f0c65380700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6015d6a0 con 0x7f0c6015d2c0 2023-08-27T11:25:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.236+0000 7f0c65380700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c600d2c90 con 0x7f0c600ce2e0 2023-08-27T11:25:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.236+0000 7f0c65380700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600ce770 con 0x7f0c600b8c20 2023-08-27T11:25:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.236+0000 7f0c5effd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0c6015d2c0 0x7f0c600d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43012/0 (socket says 172.21.15.45:43012) 2023-08-27T11:25:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.236+0000 7f0c5effd700 1 -- 172.21.15.45:0/4128491729 learned_addr learned my addr 172.21.15.45:0/4128491729 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.236+0000 7f0c5e7fc700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c600b8c20 0x7f0c600cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.236+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398037376 0 0) 0x7f0c54001110 con 0x7f0c6015d2c0 2023-08-27T11:25:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.236+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c480035b0 con 0x7f0c6015d2c0 2023-08-27T11:25:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1942651422 0 0) 0x7f0c480035b0 con 0x7f0c6015d2c0 2023-08-27T11:25:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c600b8c20 msgr2=0x7f0c600cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c600b8c20 0x7f0c600cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 >> v1:172.21.15.135:6789/0 conn(0x7f0c600ce2e0 legacy=0x7f0c600b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c600d1bf0 con 0x7f0c6015d2c0 2023-08-27T11:25:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f0c54001800 con 0x7f0c6015d2c0 2023-08-27T11:25:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 >> v1:172.21.15.45:6789/0 conn(0x7f0c6015d2c0 legacy=0x7f0c600d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c48003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c600b8c20 0x7f0c48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c48004d90 0x7f0c48005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600ce770 con 0x7f0c48003a30 2023-08-27T11:25:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4800baf0 con 0x7f0c48004d90 2023-08-27T11:25:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4800bcb0 con 0x7f0c600b8c20 2023-08-27T11:25:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5effd700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c48003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5e7fc700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c600b8c20 0x7f0c48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5f7fe700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c48004d90 0x7f0c48005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5effd700 1 -- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c600b8c20 msgr2=0x7f0c48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:14.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5effd700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c600b8c20 0x7f0c48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5effd700 1 -- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c48004d90 msgr2=0x7f0c48005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:14.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5effd700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c48004d90 0x7f0c48005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5effd700 1 -- 172.21.15.45:0/4128491729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c480035b0 con 0x7f0c48003a30 2023-08-27T11:25:14.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5effd700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c48003e50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400c210 tx=0x7f0c48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c664fcef086c43 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:14.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54007db0 con 0x7f0c48003a30 2023-08-27T11:25:14.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5400cdd0 con 0x7f0c48003a30 2023-08-27T11:25:14.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.237+0000 7f0c5dffb700 1 -- 172.21.15.45:0/4128491729 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54005ba0 con 0x7f0c48003a30 2023-08-27T11:25:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.238+0000 7f0c65380700 1 -- 172.21.15.45:0/4128491729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 msgr2=0x7f0c48003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.238+0000 7f0c65380700 1 --2- 172.21.15.45:0/4128491729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c48003e50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400c210 tx=0x7f0c48005880 comp rx=0 tx=0).stop 2023-08-27T11:25:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.238+0000 7f0c5f7fe700 1 -- 172.21.15.45:0/4128491729 reap_dead start 2023-08-27T11:25:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.238+0000 7f0c65380700 1 -- 172.21.15.45:0/4128491729 shutdown_connections 2023-08-27T11:25:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.238+0000 7f0c65380700 1 -- 172.21.15.45:0/4128491729 >> 172.21.15.45:0/4128491729 conn(0x7f0c600d2fd0 msgr2=0x7f0c600d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:14.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.238+0000 7f0c65380700 1 -- 172.21.15.45:0/4128491729 shutdown_connections 2023-08-27T11:25:14.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.238+0000 7f0c65380700 1 -- 172.21.15.45:0/4128491729 wait complete. 2023-08-27T11:25:14.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.239+0000 7f0c65380700 1 Processor -- start 2023-08-27T11:25:14.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.239+0000 7f0c65380700 1 -- start start 2023-08-27T11:25:14.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c65380700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c600cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c65380700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c601e5790 0x7f0c601e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c65380700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c6014c1d0 0x7f0c600c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c65380700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601e6980 con 0x7f0c48003a30 2023-08-27T11:25:14.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c65380700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600cc780 con 0x7f0c6014c1d0 2023-08-27T11:25:14.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c65380700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601e5d00 con 0x7f0c601e5790 2023-08-27T11:25:14.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c600cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c5effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c600cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56248/0 (socket says 172.21.15.45:56248) 2023-08-27T11:25:14.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c5effd700 1 -- 172.21.15.45:0/1629760270 learned_addr learned my addr 172.21.15.45:0/1629760270 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:14.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c5e7fc700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c601e5790 0x7f0c601e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.240+0000 7f0c5f7fe700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c6014c1d0 0x7f0c600c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c5effd700 1 -- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c601e5790 msgr2=0x7f0c601e6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c5effd700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c601e5790 0x7f0c601e6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c5effd700 1 -- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c6014c1d0 msgr2=0x7f0c600c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c5effd700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c6014c1d0 0x7f0c600c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c5effd700 1 -- 172.21.15.45:0/1629760270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c601e5fe0 con 0x7f0c48003a30 2023-08-27T11:25:14.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c5e7fc700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c601e5790 0x7f0c601e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:14.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c5f7fe700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c6014c1d0 0x7f0c600c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:14.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c5effd700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c600cc240 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800bac0 tx=0x7f0c54001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:14.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c477fe700 1 -- 172.21.15.45:0/1629760270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54004a70 con 0x7f0c48003a30 2023-08-27T11:25:14.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c477fe700 1 -- 172.21.15.45:0/1629760270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54004bd0 con 0x7f0c48003a30 2023-08-27T11:25:14.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c601e62c0 con 0x7f0c48003a30 2023-08-27T11:25:14.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c477fe700 1 -- 172.21.15.45:0/1629760270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c540017d0 con 0x7f0c48003a30 2023-08-27T11:25:14.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.241+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c601e72e0 con 0x7f0c48003a30 2023-08-27T11:25:14.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.242+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c601e6ac0 con 0x7f0c48003a30 2023-08-27T11:25:14.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.243+0000 7f0c477fe700 1 -- 172.21.15.45:0/1629760270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0c54019070 con 0x7f0c48003a30 2023-08-27T11:25:14.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.243+0000 7f0c477fe700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f0c4c06a200 0x7f0c4c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.243+0000 7f0c477fe700 1 -- 172.21.15.45:0/1629760270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0c54089e30 con 0x7f0c48003a30 2023-08-27T11:25:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.243+0000 7f0c5e7fc700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f0c4c06a200 0x7f0c4c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.244+0000 7f0c5e7fc700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f0c4c06a200 0x7f0c4c06c6c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c601f6220 tx=0x7f0c50008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.247+0000 7f0c477fe700 1 -- 172.21.15.45:0/1629760270 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0c54055310 con 0x7f0c48003a30 2023-08-27T11:25:14.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.494+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c6015d6a0 con 0x7f0c48003a30 2023-08-27T11:25:14.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.494+0000 7f0c477fe700 1 -- 172.21.15.45:0/1629760270 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4872 (secure 0 0 0) 0x7f0c6015d6a0 con 0x7f0c48003a30 2023-08-27T11:25:14.496 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:14.497 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/72415568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":72415568},{"type":"v1","addr":"172.21.15.135:6837","nonce":72415568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2554143619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2554143619},{"type":"v1","addr":"172.21.15.135:6835","nonce":2554143619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6306,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/990560009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":990560009},{"type":"v1","addr":"172.21.15.45:6834","nonce":990560009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:10.472982+0000","modified":"2023-08-27T11:25:14.080477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6309":{"gid":6309,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6835/2659160949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2659160949},{"type":"v1","addr":"172.21.15.45:6835","nonce":2659160949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-27T11:25:14.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f0c4c06a200 msgr2=0x7f0c4c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f0c4c06a200 0x7f0c4c06c6c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c601f6220 tx=0x7f0c50008040 comp rx=0 tx=0).stop 2023-08-27T11:25:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 msgr2=0x7f0c600cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c600cc240 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800bac0 tx=0x7f0c54001b90 comp rx=0 tx=0).stop 2023-08-27T11:25:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 shutdown_connections 2023-08-27T11:25:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f0c4c06a200 0x7f0c4c06c6c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48003a30 0x7f0c600cc240 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c6014c1d0 0x7f0c600c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 --2- 172.21.15.45:0/1629760270 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0c601e5790 0x7f0c601e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.496+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 >> 172.21.15.45:0/1629760270 conn(0x7f0c600d2fd0 msgr2=0x7f0c6015f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:14.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.497+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 shutdown_connections 2023-08-27T11:25:14.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.497+0000 7f0c65380700 1 -- 172.21.15.45:0/1629760270 wait complete. 2023-08-27T11:25:14.502 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-08-27T11:25:14.510 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:25:14.526 INFO:tasks.ceph.mds.c.smithi045.stderr:2023-08-27T11:25:14.525+0000 7f77f3c04700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:25:14.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.636+0000 7ff7635d3700 1 Processor -- start 2023-08-27T11:25:14.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.636+0000 7ff7635d3700 1 -- start start 2023-08-27T11:25:14.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.637+0000 7ff7635d3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff75c0b2140 0x7ff75c0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.637+0000 7ff7635d3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0ce1c0 con 0x7ff75c15d830 2023-08-27T11:25:14.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.637+0000 7ff7635d3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0cf290 con 0x7ff75c15d450 2023-08-27T11:25:14.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.637+0000 7ff7635d3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c0ce440 con 0x7ff75c0b2140 2023-08-27T11:25:14.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.637+0000 7ff760b6e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff75c0b2140 0x7ff75c0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.637+0000 7ff761b70700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff75c15d830 0x7ff75c161250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43034/0 (socket says 172.21.15.45:43034) 2023-08-27T11:25:14.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.637+0000 7ff761b70700 1 -- 172.21.15.45:0/1357996181 learned_addr learned my addr 172.21.15.45:0/1357996181 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:14.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff760b6e700 1 -- 172.21.15.45:0/1357996181 >> v1:172.21.15.135:6789/0 conn(0x7ff75c15d450 legacy=0x7ff75c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:14.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff760b6e700 1 -- 172.21.15.45:0/1357996181 >> v1:172.21.15.45:6789/0 conn(0x7ff75c15d830 legacy=0x7ff75c161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff760b6e700 1 -- 172.21.15.45:0/1357996181 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0ce720 con 0x7ff75c0b2140 2023-08-27T11:25:14.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff753fff700 1 -- 172.21.15.45:0/1357996181 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634311643 0 0) 0x7ff75c0ce1c0 con 0x7ff75c15d830 2023-08-27T11:25:14.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff760b6e700 1 --2- 172.21.15.45:0/1357996181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 0x7ff75c0cfd30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff758009840 tx=0x7ff758009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c679be49aec126 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:14.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff753fff700 1 -- 172.21.15.45:0/1357996181 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758004230 con 0x7ff75c0b2140 2023-08-27T11:25:14.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff753fff700 1 -- 172.21.15.45:0/1357996181 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758004390 con 0x7ff75c0b2140 2023-08-27T11:25:14.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff753fff700 1 -- 172.21.15.45:0/1357996181 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580036a0 con 0x7ff75c0b2140 2023-08-27T11:25:14.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff7635d3700 1 -- 172.21.15.45:0/1357996181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 msgr2=0x7ff75c0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff7635d3700 1 --2- 172.21.15.45:0/1357996181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 0x7ff75c0cfd30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff758009840 tx=0x7ff758009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:14.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff7635d3700 1 -- 172.21.15.45:0/1357996181 shutdown_connections 2023-08-27T11:25:14.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff7635d3700 1 --2- 172.21.15.45:0/1357996181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 0x7ff75c0cfd30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.638+0000 7ff7635d3700 1 -- 172.21.15.45:0/1357996181 >> 172.21.15.45:0/1357996181 conn(0x7ff75c0d0f60 msgr2=0x7ff75c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:14.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.639+0000 7ff7635d3700 1 -- 172.21.15.45:0/1357996181 shutdown_connections 2023-08-27T11:25:14.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.639+0000 7ff7635d3700 1 -- 172.21.15.45:0/1357996181 wait complete. 2023-08-27T11:25:14.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.640+0000 7ff7635d3700 1 Processor -- start 2023-08-27T11:25:14.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.640+0000 7ff7635d3700 1 -- start start 2023-08-27T11:25:14.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff7635d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 0x7ff75c158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff7635d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff75c15d450 0x7ff75c1e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff7635d3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75c15d830 0x7ff75c1e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff7635d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1e34a0 con 0x7ff75c0b2140 2023-08-27T11:25:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff7635d3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1e3780 con 0x7ff75c15d830 2023-08-27T11:25:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff7635d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1e3a60 con 0x7ff75c15d450 2023-08-27T11:25:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff76136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 0x7ff75c158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff76136f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 0x7ff75c158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56260/0 (socket says 172.21.15.45:56260) 2023-08-27T11:25:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff76136f700 1 -- 172.21.15.45:0/2852022438 learned_addr learned my addr 172.21.15.45:0/2852022438 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff761b70700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75c15d830 0x7ff75c1e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff760b6e700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff75c15d450 0x7ff75c1e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff76136f700 1 -- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff75c15d450 msgr2=0x7ff75c1e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff76136f700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff75c15d450 0x7ff75c1e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff76136f700 1 -- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75c15d830 msgr2=0x7ff75c1e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff76136f700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75c15d830 0x7ff75c1e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff76136f700 1 -- 172.21.15.45:0/2852022438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7580094a0 con 0x7ff75c0b2140 2023-08-27T11:25:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff761b70700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75c15d830 0x7ff75c1e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff760b6e700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff75c15d450 0x7ff75c1e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff76136f700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 0x7ff75c158b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00c4d0 tx=0x7ff74c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:14.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff751ffb700 1 -- 172.21.15.45:0/2852022438 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c011790 con 0x7ff75c0b2140 2023-08-27T11:25:14.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff751ffb700 1 -- 172.21.15.45:0/2852022438 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c0118f0 con 0x7ff75c0b2140 2023-08-27T11:25:14.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c1e3d40 con 0x7ff75c0b2140 2023-08-27T11:25:14.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.641+0000 7ff751ffb700 1 -- 172.21.15.45:0/2852022438 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c011ca0 con 0x7ff75c0b2140 2023-08-27T11:25:14.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.642+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c1e76f0 con 0x7ff75c0b2140 2023-08-27T11:25:14.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.643+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c1e3f80 con 0x7ff75c0b2140 2023-08-27T11:25:14.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.643+0000 7ff751ffb700 1 -- 172.21.15.45:0/2852022438 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff74c01c460 con 0x7ff75c0b2140 2023-08-27T11:25:14.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.643+0000 7ff751ffb700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff74006a200 0x7ff74006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:14.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.643+0000 7ff751ffb700 1 -- 172.21.15.45:0/2852022438 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff74c0885f0 con 0x7ff75c0b2140 2023-08-27T11:25:14.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.643+0000 7ff760b6e700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff74006a200 0x7ff74006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:14.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.644+0000 7ff760b6e700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff74006a200 0x7ff74006c6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff75c1e54e0 tx=0x7ff758008030 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:14.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.648+0000 7ff751ffb700 1 -- 172.21.15.45:0/2852022438 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff74c053af0 con 0x7ff75c0b2140 2023-08-27T11:25:14.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.851+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff75c09ee50 con 0x7ff75c0b2140 2023-08-27T11:25:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.852+0000 7ff751ffb700 1 -- 172.21.15.45:0/2852022438 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+18634 (secure 0 0 0) 0x7ff74c057110 con 0x7ff75c0b2140 2023-08-27T11:25:14.854 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:14.854 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":70,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:25:12.066925+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:08.337329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:09.446448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:25:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff74006a200 msgr2=0x7ff74006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff74006a200 0x7ff74006c6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff75c1e54e0 tx=0x7ff758008030 comp rx=0 tx=0).stop 2023-08-27T11:25:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 msgr2=0x7ff75c158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 0x7ff75c158b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00c4d0 tx=0x7ff74c00c890 comp rx=0 tx=0).stop 2023-08-27T11:25:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 shutdown_connections 2023-08-27T11:25:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7ff74006a200 0x7ff74006c6c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff75c0b2140 0x7ff75c158b50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75c15d830 0x7ff75c1e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 --2- 172.21.15.45:0/2852022438 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff75c15d450 0x7ff75c1e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:14.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.854+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 >> 172.21.15.45:0/2852022438 conn(0x7ff75c0d0f60 msgr2=0x7ff75c15c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:14.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.855+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 shutdown_connections 2023-08-27T11:25:14.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:14.855+0000 7ff7635d3700 1 -- 172.21.15.45:0/2852022438 wait complete. 2023-08-27T11:25:14.868 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:25:14.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:15.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.032+0000 7fe1c05ae700 1 Processor -- start 2023-08-27T11:25:15.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.032+0000 7fe1c05ae700 1 -- start start 2023-08-27T11:25:15.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1c05ae700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1b80caa00 0x7fe1b80cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1c05ae700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80ce340 con 0x7fe1b80af200 2023-08-27T11:25:15.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1c05ae700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80ceec0 con 0x7fe1b80ca620 2023-08-27T11:25:15.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1c05ae700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b815d7c0 con 0x7fe1b80caa00 2023-08-27T11:25:15.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1beb4b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1b80caa00 0x7fe1b80cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1beb4b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1b80caa00 0x7fe1b80cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56272/0 (socket says 172.21.15.45:56272) 2023-08-27T11:25:15.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1bdb49700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe1b80af200 0x7fe1b80acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43046/0 (socket says 172.21.15.45:43046) 2023-08-27T11:25:15.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1beb4b700 1 -- 172.21.15.45:0/571699311 learned_addr learned my addr 172.21.15.45:0/571699311 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:15.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1beb4b700 1 -- 172.21.15.45:0/571699311 >> v1:172.21.15.135:6789/0 conn(0x7fe1b80ca620 legacy=0x7fe1b80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1beb4b700 1 -- 172.21.15.45:0/571699311 >> v1:172.21.15.45:6789/0 conn(0x7fe1b80af200 legacy=0x7fe1b80acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1beb4b700 1 -- 172.21.15.45:0/571699311 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80cc570 con 0x7fe1b80caa00 2023-08-27T11:25:15.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1bd348700 1 -- 172.21.15.45:0/571699311 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674910694 0 0) 0x7fe1b80ce340 con 0x7fe1b80af200 2023-08-27T11:25:15.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.033+0000 7fe1beb4b700 1 --2- 172.21.15.45:0/571699311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80caa00 0x7fe1b80cf350 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009e50 tx=0x7fe1b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c5902405e17f18 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1bd348700 1 -- 172.21.15.45:0/571699311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000e070 con 0x7fe1b80caa00 2023-08-27T11:25:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1bd348700 1 -- 172.21.15.45:0/571699311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b000b9c0 con 0x7fe1b80caa00 2023-08-27T11:25:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1bd348700 1 -- 172.21.15.45:0/571699311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000bd70 con 0x7fe1b80caa00 2023-08-27T11:25:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1c05ae700 1 -- 172.21.15.45:0/571699311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80caa00 msgr2=0x7fe1b80cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1c05ae700 1 --2- 172.21.15.45:0/571699311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80caa00 0x7fe1b80cf350 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009e50 tx=0x7fe1b000b330 comp rx=0 tx=0).stop 2023-08-27T11:25:15.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1c05ae700 1 -- 172.21.15.45:0/571699311 shutdown_connections 2023-08-27T11:25:15.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1c05ae700 1 --2- 172.21.15.45:0/571699311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80caa00 0x7fe1b80cf350 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1c05ae700 1 -- 172.21.15.45:0/571699311 >> 172.21.15.45:0/571699311 conn(0x7fe1b80cc7e0 msgr2=0x7fe1b80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1c05ae700 1 -- 172.21.15.45:0/571699311 shutdown_connections 2023-08-27T11:25:15.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.034+0000 7fe1c05ae700 1 -- 172.21.15.45:0/571699311 wait complete. 2023-08-27T11:25:15.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.035+0000 7fe1c05ae700 1 Processor -- start 2023-08-27T11:25:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.035+0000 7fe1c05ae700 1 -- start start 2023-08-27T11:25:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1c05ae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80af200 0x7fe1b814a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1c05ae700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1b80ca620 0x7fe1b8159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1c05ae700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1b80caa00 0x7fe1b815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1c05ae700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b814ab10 con 0x7fe1b80af200 2023-08-27T11:25:15.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1c05ae700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80b5930 con 0x7fe1b80ca620 2023-08-27T11:25:15.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1c05ae700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80c6ed0 con 0x7fe1b80caa00 2023-08-27T11:25:15.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1be34a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80af200 0x7fe1b814a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1be34a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80af200 0x7fe1b814a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56274/0 (socket says 172.21.15.45:56274) 2023-08-27T11:25:15.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1be34a700 1 -- 172.21.15.45:0/1760974262 learned_addr learned my addr 172.21.15.45:0/1760974262 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:15.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1beb4b700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1b80caa00 0x7fe1b815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.036+0000 7fe1bdb49700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1b80ca620 0x7fe1b8159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1be34a700 1 -- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1b80caa00 msgr2=0x7fe1b815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1be34a700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1b80caa00 0x7fe1b815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1be34a700 1 -- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1b80ca620 msgr2=0x7fe1b8159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1be34a700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1b80ca620 0x7fe1b8159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1be34a700 1 -- 172.21.15.45:0/1760974262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b0009b30 con 0x7fe1b80af200 2023-08-27T11:25:15.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1bdb49700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1b80ca620 0x7fe1b8159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1be34a700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80af200 0x7fe1b814a5d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400c4e0 tx=0x7fe1b400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1aeffd700 1 -- 172.21.15.45:0/1760974262 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40052d0 con 0x7fe1b80af200 2023-08-27T11:25:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1aeffd700 1 -- 172.21.15.45:0/1760974262 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b4005430 con 0x7fe1b80af200 2023-08-27T11:25:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1aeffd700 1 -- 172.21.15.45:0/1760974262 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40038c0 con 0x7fe1b80af200 2023-08-27T11:25:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b80c71b0 con 0x7fe1b80af200 2023-08-27T11:25:15.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.037+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b80c7770 con 0x7fe1b80af200 2023-08-27T11:25:15.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.038+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b80734a0 con 0x7fe1b80af200 2023-08-27T11:25:15.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.038+0000 7fe1aeffd700 1 -- 172.21.15.45:0/1760974262 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe1b4003a20 con 0x7fe1b80af200 2023-08-27T11:25:15.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.039+0000 7fe1aeffd700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe19c06a200 0x7fe19c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.039+0000 7fe1aeffd700 1 -- 172.21.15.45:0/1760974262 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe1b4087c40 con 0x7fe1b80af200 2023-08-27T11:25:15.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.039+0000 7fe1bdb49700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe19c06a200 0x7fe19c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.039+0000 7fe1bdb49700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe19c06a200 0x7fe19c06c6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80cf710 tx=0x7fe1a8007ce0 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:15.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.043+0000 7fe1aeffd700 1 -- 172.21.15.45:0/1760974262 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe1b40530b0 con 0x7fe1b80af200 2023-08-27T11:25:15.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.241+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 --> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1b80ce590 con 0x7fe19c06a200 2023-08-27T11:25:15.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.250+0000 7fe1aeffd700 1 -- 172.21.15.45:0/1760974262 <== mgr.5402 v2:172.21.15.135:6832/3181978685 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299372 (secure 0 0 0) 0x7fe1a8005b70 con 0x7fe19c06a200 2023-08-27T11:25:15.255 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:15.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe19c06a200 msgr2=0x7fe19c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe19c06a200 0x7fe19c06c6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80cf710 tx=0x7fe1a8007ce0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80af200 msgr2=0x7fe1b814a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80af200 0x7fe1b814a5d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400c4e0 tx=0x7fe1b400c7f0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 shutdown_connections 2023-08-27T11:25:15.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe19c06a200 0x7fe19c06c6c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1b80af200 0x7fe1b814a5d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1b80caa00 0x7fe1b815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 --2- 172.21.15.45:0/1760974262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1b80ca620 0x7fe1b8159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 >> 172.21.15.45:0/1760974262 conn(0x7fe1b80cc7e0 msgr2=0x7fe1b80af740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:15.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 shutdown_connections 2023-08-27T11:25:15.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.253+0000 7fe1c05ae700 1 -- 172.21.15.45:0/1760974262 wait complete. 2023-08-27T11:25:15.260 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:25:15.266 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-08-27T11:25:14.322104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265144,"kb_used_data":7372,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754709576,"statfs":{"total":773094113280,"available":772822605824,"internally_reserved":0,"allocated":7548928,"data_stored":4428794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.279241"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045823+0000","last_change":"2023-08-27T11:25:09.045823+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045823+0000","last_undegraded":"2023-08-27T11:25:09.045823+0000","last_fullsized":"2023-08-27T11:25:09.045823+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044948+0000","last_change":"2023-08-27T11:25:09.044948+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044948+0000","last_undegraded":"2023-08-27T11:25:09.044948+0000","last_fullsized":"2023-08-27T11:25:09.044948+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.046665+0000","last_change":"2023-08-27T11:25:09.046665+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.046665+0000","last_undegraded":"2023-08-27T11:25:09.046665+0000","last_fullsized":"2023-08-27T11:25:09.046665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045158+0000","last_change":"2023-08-27T11:25:09.045158+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045158+0000","last_undegraded":"2023-08-27T11:25:09.045158+0000","last_fullsized":"2023-08-27T11:25:09.045158+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045826+0000","last_change":"2023-08-27T11:25:09.045826+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045826+0000","last_undegraded":"2023-08-27T11:25:09.045826+0000","last_fullsized":"2023-08-27T11:25:09.045826+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.046796+0000","last_change":"2023-08-27T11:25:09.046796+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.046796+0000","last_undegraded":"2023-08-27T11:25:09.046796+0000","last_fullsized":"2023-08-27T11:25:09.046796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045408+0000","last_change":"2023-08-27T11:25:09.045408+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045408+0000","last_undegraded":"2023-08-27T11:25:09.045408+0000","last_fullsized":"2023-08-27T11:25:09.045408+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045965+0000","last_change":"2023-08-27T11:25:09.045965+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045965+0000","last_undegraded":"2023-08-27T11:25:09.045965+0000","last_fullsized":"2023-08-27T11:25:09.045965+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.048008+0000","last_change":"2023-08-27T11:25:09.048008+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.048008+0000","last_undegraded":"2023-08-27T11:25:09.048008+0000","last_fullsized":"2023-08-27T11:25:09.048008+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044403+0000","last_change":"2023-08-27T11:25:09.044403+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044403+0000","last_undegraded":"2023-08-27T11:25:09.044403+0000","last_fullsized":"2023-08-27T11:25:09.044403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.048244+0000","last_change":"2023-08-27T11:25:09.048244+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.048244+0000","last_undegraded":"2023-08-27T11:25:09.048244+0000","last_fullsized":"2023-08-27T11:25:09.048244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047418+0000","last_change":"2023-08-27T11:25:09.047418+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047418+0000","last_undegraded":"2023-08-27T11:25:09.047418+0000","last_fullsized":"2023-08-27T11:25:09.047418+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044870+0000","last_change":"2023-08-27T11:25:09.044870+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044870+0000","last_undegraded":"2023-08-27T11:25:09.044870+0000","last_fullsized":"2023-08-27T11:25:09.044870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.051997+0000","last_change":"2023-08-27T11:25:09.051997+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.051997+0000","last_undegraded":"2023-08-27T11:25:09.051997+0000","last_fullsized":"2023-08-27T11:25:09.051997+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044564+0000","last_change":"2023-08-27T11:25:09.044564+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044564+0000","last_undegraded":"2023-08-27T11:25:09.044564+0000","last_fullsized":"2023-08-27T11:25:09.044564+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047863+0000","last_change":"2023-08-27T11:25:09.047863+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047863+0000","last_undegraded":"2023-08-27T11:25:09.047863+0000","last_fullsized":"2023-08-27T11:25:09.047863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042153+0000","last_change":"2023-08-27T11:25:09.042153+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042153+0000","last_undegraded":"2023-08-27T11:25:09.042153+0000","last_fullsized":"2023-08-27T11:25:09.042153+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047304+0000","last_change":"2023-08-27T11:25:09.047304+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047304+0000","last_undegraded":"2023-08-27T11:25:09.047304+0000","last_fullsized":"2023-08-27T11:25:09.047304+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.052065+0000","last_change":"2023-08-27T11:25:09.052065+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.052065+0000","last_undegraded":"2023-08-27T11:25:09.052065+0000","last_fullsized":"2023-08-27T11:25:09.052065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044678+0000","last_change":"2023-08-27T11:25:09.044678+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044678+0000","last_undegraded":"2023-08-27T11:25:09.044678+0000","last_fullsized":"2023-08-27T11:25:09.044678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042196+0000","last_change":"2023-08-27T11:25:09.042196+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042196+0000","last_undegraded":"2023-08-27T11:25:09.042196+0000","last_fullsized":"2023-08-27T11:25:09.042196+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045417+0000","last_change":"2023-08-27T11:25:09.045417+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045417+0000","last_undegraded":"2023-08-27T11:25:09.045417+0000","last_fullsized":"2023-08-27T11:25:09.045417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042127+0000","last_change":"2023-08-27T11:25:09.042127+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042127+0000","last_undegraded":"2023-08-27T11:25:09.042127+0000","last_fullsized":"2023-08-27T11:25:09.042127+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044403+0000","last_change":"2023-08-27T11:25:09.044403+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044403+0000","last_undegraded":"2023-08-27T11:25:09.044403+0000","last_fullsized":"2023-08-27T11:25:09.044403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047735+0000","last_change":"2023-08-27T11:25:09.047735+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047735+0000","last_undegraded":"2023-08-27T11:25:09.047735+0000","last_fullsized":"2023-08-27T11:25:09.047735+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.040134+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:25:09.040134+0000","last_peered":"2023-08-27T11:25:09.040134+0000","last_clean":"2023-08-27T11:25:09.040134+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:25:09.040134+0000","last_undegraded":"2023-08-27T11:25:09.040134+0000","last_fullsized":"2023-08-27T11:25:09.040134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:32:58.034773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045725+0000","last_change":"2023-08-27T11:25:09.045725+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045725+0000","last_undegraded":"2023-08-27T11:25:09.045725+0000","last_fullsized":"2023-08-27T11:25:09.045725+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.046326+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:25:09.046326+0000","last_peered":"2023-08-27T11:25:09.046326+0000","last_clean":"2023-08-27T11:25:09.046326+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:25:09.046326+0000","last_undegraded":"2023-08-27T11:25:09.046326+0000","last_fullsized":"2023-08-27T11:25:09.046326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:04:56.999493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044379+0000","last_change":"2023-08-27T11:25:09.044379+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044379+0000","last_undegraded":"2023-08-27T11:25:09.044379+0000","last_fullsized":"2023-08-27T11:25:09.044379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.045788+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:25:09.045788+0000","last_peered":"2023-08-27T11:25:09.045788+0000","last_clean":"2023-08-27T11:25:09.045788+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:25:09.045788+0000","last_undegraded":"2023-08-27T11:25:09.045788+0000","last_fullsized":"2023-08-27T11:25:09.045788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:38.184300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.051943+0000","last_change":"2023-08-27T11:25:09.051943+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.051943+0000","last_undegraded":"2023-08-27T11:25:09.051943+0000","last_fullsized":"2023-08-27T11:25:09.051943+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.046525+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:25:09.046525+0000","last_peered":"2023-08-27T11:25:09.046525+0000","last_clean":"2023-08-27T11:25:09.046525+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:25:09.046525+0000","last_undegraded":"2023-08-27T11:25:09.046525+0000","last_fullsized":"2023-08-27T11:25:09.046525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:38:17.153603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":341,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.049404+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:25:09.049404+0000","last_peered":"2023-08-27T11:25:09.049404+0000","last_clean":"2023-08-27T11:25:09.049404+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:25:09.049404+0000","last_undegraded":"2023-08-27T11:25:09.049404+0000","last_fullsized":"2023-08-27T11:25:09.049404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:42:09.976948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042667+0000","last_change":"2023-08-27T11:25:09.042667+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042667+0000","last_undegraded":"2023-08-27T11:25:09.042667+0000","last_fullsized":"2023-08-27T11:25:09.042667+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.049543+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:25:09.049543+0000","last_peered":"2023-08-27T11:25:09.049543+0000","last_clean":"2023-08-27T11:25:09.049543+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:25:09.049543+0000","last_undegraded":"2023-08-27T11:25:09.049543+0000","last_fullsized":"2023-08-27T11:25:09.049543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:08:36.959604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044316+0000","last_change":"2023-08-27T11:25:09.044316+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044316+0000","last_undegraded":"2023-08-27T11:25:09.044316+0000","last_fullsized":"2023-08-27T11:25:09.044316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.049483+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:25:09.049483+0000","last_peered":"2023-08-27T11:25:09.049483+0000","last_clean":"2023-08-27T11:25:09.049483+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:25:09.049483+0000","last_undegraded":"2023-08-27T11:25:09.049483+0000","last_fullsized":"2023-08-27T11:25:09.049483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:17.326927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045853+0000","last_change":"2023-08-27T11:25:09.045853+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045853+0000","last_undegraded":"2023-08-27T11:25:09.045853+0000","last_fullsized":"2023-08-27T11:25:09.045853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.040147+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:25:09.040147+0000","last_peered":"2023-08-27T11:25:09.040147+0000","last_clean":"2023-08-27T11:25:09.040147+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:25:09.040147+0000","last_undegraded":"2023-08-27T11:25:09.040147+0000","last_fullsized":"2023-08-27T11:25:09.040147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:37.914648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044553+0000","last_change":"2023-08-27T11:25:09.044553+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044553+0000","last_undegraded":"2023-08-27T11:25:09.044553+0000","last_fullsized":"2023-08-27T11:25:09.044553+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.052586+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:25:09.052586+0000","last_peered":"2023-08-27T11:25:09.052586+0000","last_clean":"2023-08-27T11:25:09.052586+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:25:09.052586+0000","last_undegraded":"2023-08-27T11:25:09.052586+0000","last_fullsized":"2023-08-27T11:25:09.052586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:55.622304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":993925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.875,"15min":0.875},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.55900000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.33700000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.314},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.48299999999999998,"5min":0.35199999999999998,"15min":0.252},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.439,"5min":0.439,"15min":0.29099999999999998},"max":{"1min":0.79100000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48399999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65600000000000003,"15min":0.61599999999999999},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65700000000000003,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.39900000000000002},"max":{"1min":0.82199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55600000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.26500000000000001,"15min":0.23499999999999999},"max":{"1min":0.65400000000000003,"5min":0.67500000000000004,"15min":0.72499999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.52900000000000003},"min":{"1min":0.39100000000000001,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.57399999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.74199999999999999,"15min":0.84699999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.30099999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.90100000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.626}]}]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.59699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.377,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":1.3540000000000001},"last":0.79900000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":3.3780000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.81399999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.71199999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":9.3770000000000007},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.746,"5min":0.82799999999999996,"15min":1.325},"last":0.74199999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60399999999999998,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34100000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.84899999999999998,"15min":1.395},"last":0.75800000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.25900000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34300000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68500000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54600000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.5},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.82099999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.32900000000000001},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.65500000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72299999999999998,"5min":0.76100000000000001,"15min":0.88900000000000001},"last":0.621},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.34899999999999998},"max":{"1min":0.76400000000000001,"5min":0.77400000000000002,"15min":0.86299999999999999},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":802816,"data_stored":410091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.20399999999999999},"max":{"1min":0.52700000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.60699999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.52700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":0.68100000000000005,"5min":0.68300000000000005,"15min":0.78600000000000003},"last":0.45000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.32500000000000001,"5min":0.311,"15min":0.27600000000000002},"max":{"1min":0.61899999999999999,"5min":0.70399999999999996,"15min":0.77100000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45700000000000002,"15min":0.434},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.23400000000000001},"max":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.80000000000000004},"last":0.48999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.51700000000000002,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.97999999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.94199999999999995,"15min":2.9380000000000002},"last":0.749}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.42099999999999999},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.627},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97999999999999998,"15min":2.9740000000000002},"last":0.72499999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.54700000000000004,"5min":0.47399999999999998,"15min":0.42499999999999999},"max":{"1min":0.85499999999999998,"5min":1.002,"15min":1.139},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.876,"5min":0.91600000000000004,"15min":1.0169999999999999},"last":0.77400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":1000459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41999999999999998,"15min":0.40600000000000003},"min":{"1min":0.28199999999999997,"5min":0.27200000000000002,"15min":0.23699999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.90200000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.47499999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":1.2889999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42899999999999999,"15min":0.41399999999999998},"min":{"1min":0.317,"5min":0.313,"15min":0.22900000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":1.3340000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.505},"min":{"1min":0.30199999999999999,"5min":0.29099999999999998,"15min":0.255},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":8.1349999999999998},"last":0.38200000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.56399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.23899999999999999},"max":{"1min":0.56899999999999995,"5min":0.69899999999999995,"15min":11.026999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.29699999999999999},"max":{"1min":0.63800000000000001,"5min":0.73299999999999998,"15min":11.926},"last":0.45400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66500000000000004,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":1.0069999999999999,"15min":10.964},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":1.3640000000000001},"last":0.85099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.73699999999999999},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":8.0779999999999994},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70199999999999996,"15min":0.81000000000000005},"min":{"1min":0.53300000000000003,"5min":0.42799999999999999,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1,"15min":10.922000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.35699999999999998},"max":{"1min":0.82499999999999996,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.51400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":8.0380000000000003},"last":0.873}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.78500000000000003},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":8.1069999999999993},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":1.036,"15min":10.996},"last":0.66900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575010,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":410091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38500000000000001,"15min":0.372},"min":{"1min":0.25800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.65400000000000003},"last":0.505},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.437,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.20799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.753},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46000000000000002,"15min":0.443},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.24099999999999999},"max":{"1min":0.71299999999999997,"5min":0.71699999999999997,"15min":0.77800000000000002},"last":0.434}]},{"osd":3,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48099999999999998,"15min":0.46200000000000002},"min":{"1min":0.34599999999999997,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.29299999999999998},"max":{"1min":0.61899999999999999,"5min":0.65300000000000002,"15min":0.69099999999999995},"last":0.376}]},{"osd":4,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76500000000000001,"5min":1.163,"15min":1.163},"last":0.622},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76800000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0660000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95199999999999996,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68100000000000005,"15min":0.68000000000000005},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.1319999999999999,"15min":1.2050000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0189999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.499,"15min":0.40100000000000002},"max":{"1min":1.0409999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45700000000000002,"15min":0.42299999999999999},"max":{"1min":0.96999999999999997,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40999999999999998,"15min":0.52300000000000002},"min":{"1min":0.26800000000000002,"5min":0.23899999999999999,"15min":0.22},"max":{"1min":0.53800000000000003,"5min":0.63300000000000001,"15min":11.885999999999999},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44500000000000001,"15min":0.40000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.224},"max":{"1min":0.59899999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33400000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.32500000000000001,"5min":0.29299999999999998,"15min":0.255},"max":{"1min":0.57399999999999995,"5min":0.67900000000000005,"15min":6.0830000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.503,"15min":0.60599999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.245},"max":{"1min":0.68899999999999995,"5min":0.80700000000000005,"15min":12.050000000000001},"last":0.371}]},{"osd":2,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.504,"15min":0.45800000000000002},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.27100000000000002},"max":{"1min":0.66700000000000004,"5min":0.93100000000000005,"15min":1.25},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54200000000000004,"15min":0.63800000000000001},"min":{"1min":0.376,"5min":0.374,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.83999999999999997,"15min":11.939},"last":0.31}]},{"osd":4,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.76900000000000002},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.376},"max":{"1min":0.86099999999999999,"5min":1.0329999999999999,"15min":11.007},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70199999999999996,"15min":0.65400000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":0.91000000000000003,"5min":1.113,"15min":1.2929999999999999},"last":0.503}]},{"osd":5,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70499999999999996,"15min":0.81200000000000006},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.436},"max":{"1min":0.92500000000000004,"5min":1.0800000000000001,"15min":11.981},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.79300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.218,"15min":11.09},"last":0.63500000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72799999999999998,"15min":0.91400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.38200000000000001},"max":{"1min":0.88900000000000001,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75800000000000001,"15min":0.85499999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.47799999999999998},"max":{"1min":0.95899999999999996,"5min":1.0389999999999999,"15min":12.016},"last":0.58199999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.80900000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.41599999999999998},"max":{"1min":0.91100000000000003,"5min":1.006,"15min":11.054},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.69299999999999995},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":1.0640000000000001,"15min":1.361},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.23499999999999999},"max":{"1min":0.71099999999999997,"5min":0.81799999999999995,"15min":0.89800000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.60399999999999998},"min":{"1min":0.501,"5min":0.40699999999999997,"15min":0.255},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.873},"last":0.57699999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.622,"15min":0.59699999999999998},"min":{"1min":0.51800000000000002,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82999999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.747,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58999999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.312},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.629},"min":{"1min":0.52100000000000002,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.745}]},{"osd":3,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.503,"5min":0.40200000000000002,"15min":0.30099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.62},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.49399999999999999,"5min":0.371,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.88800000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.437},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.497,"15min":0.51600000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.26200000000000001},"max":{"1min":0.72299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45600000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.27800000000000002},"max":{"1min":0.72199999999999998,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.93899999999999995},"last":0.503}]},{"osd":7,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.874,"15min":0.878},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.48299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36116,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335724,"statfs":{"total":96636764160,"available":96599781376,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.93300000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.26200000000000001},"max":{"1min":0.749,"5min":0.84999999999999998,"15min":0.88100000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.248},"max":{"1min":0.80400000000000005,"5min":0.81899999999999995,"15min":0.96999999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":0.77400000000000002,"15min":0.82299999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.33400000000000002,"15min":0.32800000000000001},"max":{"1min":0.75,"5min":0.87,"15min":0.87},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":1.0109999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.80600000000000005,"15min":0.94199999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.34799999999999998,"15min":0.34000000000000002},"max":{"1min":0.79400000000000004,"5min":0.79500000000000004,"15min":0.83799999999999997},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71299999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.372}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47999999999999998,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.89000000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.379,"15min":0.379},"max":{"1min":0.75,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.88100000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.84999999999999998},"last":0.497}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:15.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:15.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.430+0000 7f73f03e8700 1 Processor -- start 2023-08-27T11:25:15.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.431+0000 7f73f03e8700 1 -- start start 2023-08-27T11:25:15.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.431+0000 7f73f03e8700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f73e815a240 0x7f73e80cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.431+0000 7f73f03e8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73e80cbe70 0x7f73e80ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73f03e8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e806d690 con 0x7f73e80ce9d0 2023-08-27T11:25:15.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73f03e8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e80cede0 con 0x7f73e80cbe70 2023-08-27T11:25:15.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73f03e8700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e80cc440 con 0x7f73e815a240 2023-08-27T11:25:15.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed983700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73e80cbe70 0x7f73e80ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ee985700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f73e80ce9d0 0x7f73e8070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43062/0 (socket says 172.21.15.45:43062) 2023-08-27T11:25:15.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ee985700 1 -- 172.21.15.45:0/2282560054 learned_addr learned my addr 172.21.15.45:0/2282560054 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:15.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ee184700 1 --2- 172.21.15.45:0/2282560054 >> v2:172.21.15.135:3300/0 conn(0x7f73e815a240 0x7f73e80cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed983700 1 -- 172.21.15.45:0/2282560054 >> v2:172.21.15.135:3300/0 conn(0x7f73e815a240 msgr2=0x7f73e80cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:15.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed983700 1 --2- 172.21.15.45:0/2282560054 >> v2:172.21.15.135:3300/0 conn(0x7f73e815a240 0x7f73e80cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed983700 1 -- 172.21.15.45:0/2282560054 >> v1:172.21.15.45:6789/0 conn(0x7f73e80ce9d0 legacy=0x7f73e8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed182700 1 -- 172.21.15.45:0/2282560054 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394226618 0 0) 0x7f73e806d690 con 0x7f73e80ce9d0 2023-08-27T11:25:15.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed983700 1 -- 172.21.15.45:0/2282560054 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e806d040 con 0x7f73e80cbe70 2023-08-27T11:25:15.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ee184700 1 --2- 172.21.15.45:0/2282560054 >> v2:172.21.15.135:3300/0 conn(0x7f73e815a240 0x7f73e80cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:15.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed983700 1 --2- 172.21.15.45:0/2282560054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80cbe70 0x7f73e80ce490 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b2950 tx=0x7f73e4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa701204bda19778 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:15.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed182700 1 -- 172.21.15.45:0/2282560054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4011830 con 0x7f73e80cbe70 2023-08-27T11:25:15.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed182700 1 -- 172.21.15.45:0/2282560054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e4011990 con 0x7f73e80cbe70 2023-08-27T11:25:15.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.432+0000 7f73ed182700 1 -- 172.21.15.45:0/2282560054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4011d20 con 0x7f73e80cbe70 2023-08-27T11:25:15.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.433+0000 7f73f03e8700 1 -- 172.21.15.45:0/2282560054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80cbe70 msgr2=0x7f73e80ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.433+0000 7f73f03e8700 1 --2- 172.21.15.45:0/2282560054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80cbe70 0x7f73e80ce490 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f73e80b2950 tx=0x7f73e4009b20 comp rx=0 tx=0).stop 2023-08-27T11:25:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.433+0000 7f73f03e8700 1 -- 172.21.15.45:0/2282560054 shutdown_connections 2023-08-27T11:25:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.433+0000 7f73f03e8700 1 --2- 172.21.15.45:0/2282560054 >> v2:172.21.15.135:3300/0 conn(0x7f73e815a240 0x7f73e80cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.433+0000 7f73f03e8700 1 --2- 172.21.15.45:0/2282560054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80cbe70 0x7f73e80ce490 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.433+0000 7f73f03e8700 1 -- 172.21.15.45:0/2282560054 >> 172.21.15.45:0/2282560054 conn(0x7f73e80cfe50 msgr2=0x7f73e80d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.433+0000 7f73f03e8700 1 -- 172.21.15.45:0/2282560054 shutdown_connections 2023-08-27T11:25:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.433+0000 7f73f03e8700 1 -- 172.21.15.45:0/2282560054 wait complete. 2023-08-27T11:25:15.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.434+0000 7f73f03e8700 1 Processor -- start 2023-08-27T11:25:15.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.434+0000 7f73f03e8700 1 -- start start 2023-08-27T11:25:15.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73f03e8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73e80cbe70 0x7f73e80c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73f03e8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80ce9d0 0x7f73e814de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73f03e8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73e815a240 0x7f73e814c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73f03e8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8142a70 con 0x7f73e80ce9d0 2023-08-27T11:25:15.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73f03e8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80c8ae0 con 0x7f73e80cbe70 2023-08-27T11:25:15.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73f03e8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80c8dc0 con 0x7f73e815a240 2023-08-27T11:25:15.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73ed983700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80ce9d0 0x7f73e814de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73ed983700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80ce9d0 0x7f73e814de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56290/0 (socket says 172.21.15.45:56290) 2023-08-27T11:25:15.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73ed983700 1 -- 172.21.15.45:0/3218112723 learned_addr learned my addr 172.21.15.45:0/3218112723 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:15.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73ee985700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73e815a240 0x7f73e814c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73ee184700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73e80cbe70 0x7f73e80c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73ed983700 1 -- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73e815a240 msgr2=0x7f73e814c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.435+0000 7f73ed983700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73e815a240 0x7f73e814c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73ed983700 1 -- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73e80cbe70 msgr2=0x7f73e80c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73ed983700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73e80cbe70 0x7f73e80c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73ed983700 1 -- 172.21.15.45:0/3218112723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e40094a0 con 0x7f73e80ce9d0 2023-08-27T11:25:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73ee985700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73e815a240 0x7f73e814c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73ee184700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73e80cbe70 0x7f73e80c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73ed983700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80ce9d0 0x7f73e814de80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f73e40090b0 tx=0x7f73e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73deffd700 1 -- 172.21.15.45:0/3218112723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e40038a0 con 0x7f73e80ce9d0 2023-08-27T11:25:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73deffd700 1 -- 172.21.15.45:0/3218112723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e401f070 con 0x7f73e80ce9d0 2023-08-27T11:25:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73deffd700 1 -- 172.21.15.45:0/3218112723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4003c20 con 0x7f73e80ce9d0 2023-08-27T11:25:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e80c90a0 con 0x7f73e80ce9d0 2023-08-27T11:25:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.436+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e80c9660 con 0x7f73e80ce9d0 2023-08-27T11:25:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.437+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e8072810 con 0x7f73e80ce9d0 2023-08-27T11:25:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.437+0000 7f73deffd700 1 -- 172.21.15.45:0/3218112723 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f73e401a430 con 0x7f73e80ce9d0 2023-08-27T11:25:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.438+0000 7f73deffd700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f73cc06a200 0x7f73cc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.438+0000 7f73deffd700 1 -- 172.21.15.45:0/3218112723 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f73e4021070 con 0x7f73e80ce9d0 2023-08-27T11:25:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.438+0000 7f73ee184700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f73cc06a200 0x7f73cc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.439+0000 7f73ee184700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f73cc06a200 0x7f73cc06c6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009960 tx=0x7f73d8008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:15.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.442+0000 7f73deffd700 1 -- 172.21.15.45:0/3218112723 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f73e40bc990 con 0x7f73e80ce9d0 2023-08-27T11:25:15.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.638+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 --> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73e80ce1a0 con 0x7f73cc06a200 2023-08-27T11:25:15.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.647+0000 7f73deffd700 1 -- 172.21.15.45:0/3218112723 <== mgr.5402 v2:172.21.15.135:6832/3181978685 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299372 (secure 0 0 0) 0x7f73d8009570 con 0x7f73cc06a200 2023-08-27T11:25:15.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f73cc06a200 msgr2=0x7f73cc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f73cc06a200 0x7f73cc06c6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009960 tx=0x7f73d8008040 comp rx=0 tx=0).stop 2023-08-27T11:25:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80ce9d0 msgr2=0x7f73e814de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80ce9d0 0x7f73e814de80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f73e40090b0 tx=0x7f73e4004750 comp rx=0 tx=0).stop 2023-08-27T11:25:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 shutdown_connections 2023-08-27T11:25:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f73cc06a200 0x7f73cc06c6c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73e815a240 0x7f73e814c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73e80cbe70 0x7f73e80c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 --2- 172.21.15.45:0/3218112723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73e80ce9d0 0x7f73e814de80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.650+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 >> 172.21.15.45:0/3218112723 conn(0x7f73e80cfe50 msgr2=0x7f73e80c2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.651+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 shutdown_connections 2023-08-27T11:25:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.651+0000 7f73f03e8700 1 -- 172.21.15.45:0/3218112723 wait complete. 2023-08-27T11:25:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:25:15.664 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-08-27T11:25:14.322104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265144,"kb_used_data":7372,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754709576,"statfs":{"total":773094113280,"available":772822605824,"internally_reserved":0,"allocated":7548928,"data_stored":4428794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.279241"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045823+0000","last_change":"2023-08-27T11:25:09.045823+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045823+0000","last_undegraded":"2023-08-27T11:25:09.045823+0000","last_fullsized":"2023-08-27T11:25:09.045823+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044948+0000","last_change":"2023-08-27T11:25:09.044948+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044948+0000","last_undegraded":"2023-08-27T11:25:09.044948+0000","last_fullsized":"2023-08-27T11:25:09.044948+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.046665+0000","last_change":"2023-08-27T11:25:09.046665+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.046665+0000","last_undegraded":"2023-08-27T11:25:09.046665+0000","last_fullsized":"2023-08-27T11:25:09.046665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045158+0000","last_change":"2023-08-27T11:25:09.045158+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045158+0000","last_undegraded":"2023-08-27T11:25:09.045158+0000","last_fullsized":"2023-08-27T11:25:09.045158+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045826+0000","last_change":"2023-08-27T11:25:09.045826+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045826+0000","last_undegraded":"2023-08-27T11:25:09.045826+0000","last_fullsized":"2023-08-27T11:25:09.045826+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.046796+0000","last_change":"2023-08-27T11:25:09.046796+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.046796+0000","last_undegraded":"2023-08-27T11:25:09.046796+0000","last_fullsized":"2023-08-27T11:25:09.046796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045408+0000","last_change":"2023-08-27T11:25:09.045408+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045408+0000","last_undegraded":"2023-08-27T11:25:09.045408+0000","last_fullsized":"2023-08-27T11:25:09.045408+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045965+0000","last_change":"2023-08-27T11:25:09.045965+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045965+0000","last_undegraded":"2023-08-27T11:25:09.045965+0000","last_fullsized":"2023-08-27T11:25:09.045965+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.048008+0000","last_change":"2023-08-27T11:25:09.048008+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.048008+0000","last_undegraded":"2023-08-27T11:25:09.048008+0000","last_fullsized":"2023-08-27T11:25:09.048008+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044403+0000","last_change":"2023-08-27T11:25:09.044403+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044403+0000","last_undegraded":"2023-08-27T11:25:09.044403+0000","last_fullsized":"2023-08-27T11:25:09.044403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.048244+0000","last_change":"2023-08-27T11:25:09.048244+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.048244+0000","last_undegraded":"2023-08-27T11:25:09.048244+0000","last_fullsized":"2023-08-27T11:25:09.048244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047418+0000","last_change":"2023-08-27T11:25:09.047418+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047418+0000","last_undegraded":"2023-08-27T11:25:09.047418+0000","last_fullsized":"2023-08-27T11:25:09.047418+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044870+0000","last_change":"2023-08-27T11:25:09.044870+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044870+0000","last_undegraded":"2023-08-27T11:25:09.044870+0000","last_fullsized":"2023-08-27T11:25:09.044870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.051997+0000","last_change":"2023-08-27T11:25:09.051997+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.051997+0000","last_undegraded":"2023-08-27T11:25:09.051997+0000","last_fullsized":"2023-08-27T11:25:09.051997+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044564+0000","last_change":"2023-08-27T11:25:09.044564+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044564+0000","last_undegraded":"2023-08-27T11:25:09.044564+0000","last_fullsized":"2023-08-27T11:25:09.044564+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047863+0000","last_change":"2023-08-27T11:25:09.047863+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047863+0000","last_undegraded":"2023-08-27T11:25:09.047863+0000","last_fullsized":"2023-08-27T11:25:09.047863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042153+0000","last_change":"2023-08-27T11:25:09.042153+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042153+0000","last_undegraded":"2023-08-27T11:25:09.042153+0000","last_fullsized":"2023-08-27T11:25:09.042153+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047304+0000","last_change":"2023-08-27T11:25:09.047304+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047304+0000","last_undegraded":"2023-08-27T11:25:09.047304+0000","last_fullsized":"2023-08-27T11:25:09.047304+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.052065+0000","last_change":"2023-08-27T11:25:09.052065+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.052065+0000","last_undegraded":"2023-08-27T11:25:09.052065+0000","last_fullsized":"2023-08-27T11:25:09.052065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044678+0000","last_change":"2023-08-27T11:25:09.044678+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044678+0000","last_undegraded":"2023-08-27T11:25:09.044678+0000","last_fullsized":"2023-08-27T11:25:09.044678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042196+0000","last_change":"2023-08-27T11:25:09.042196+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042196+0000","last_undegraded":"2023-08-27T11:25:09.042196+0000","last_fullsized":"2023-08-27T11:25:09.042196+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045417+0000","last_change":"2023-08-27T11:25:09.045417+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045417+0000","last_undegraded":"2023-08-27T11:25:09.045417+0000","last_fullsized":"2023-08-27T11:25:09.045417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042127+0000","last_change":"2023-08-27T11:25:09.042127+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042127+0000","last_undegraded":"2023-08-27T11:25:09.042127+0000","last_fullsized":"2023-08-27T11:25:09.042127+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044403+0000","last_change":"2023-08-27T11:25:09.044403+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044403+0000","last_undegraded":"2023-08-27T11:25:09.044403+0000","last_fullsized":"2023-08-27T11:25:09.044403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047735+0000","last_change":"2023-08-27T11:25:09.047735+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047735+0000","last_undegraded":"2023-08-27T11:25:09.047735+0000","last_fullsized":"2023-08-27T11:25:09.047735+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.040134+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:25:09.040134+0000","last_peered":"2023-08-27T11:25:09.040134+0000","last_clean":"2023-08-27T11:25:09.040134+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:25:09.040134+0000","last_undegraded":"2023-08-27T11:25:09.040134+0000","last_fullsized":"2023-08-27T11:25:09.040134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:32:58.034773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045725+0000","last_change":"2023-08-27T11:25:09.045725+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045725+0000","last_undegraded":"2023-08-27T11:25:09.045725+0000","last_fullsized":"2023-08-27T11:25:09.045725+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.046326+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:25:09.046326+0000","last_peered":"2023-08-27T11:25:09.046326+0000","last_clean":"2023-08-27T11:25:09.046326+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:25:09.046326+0000","last_undegraded":"2023-08-27T11:25:09.046326+0000","last_fullsized":"2023-08-27T11:25:09.046326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:04:56.999493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044379+0000","last_change":"2023-08-27T11:25:09.044379+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044379+0000","last_undegraded":"2023-08-27T11:25:09.044379+0000","last_fullsized":"2023-08-27T11:25:09.044379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.045788+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:25:09.045788+0000","last_peered":"2023-08-27T11:25:09.045788+0000","last_clean":"2023-08-27T11:25:09.045788+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:25:09.045788+0000","last_undegraded":"2023-08-27T11:25:09.045788+0000","last_fullsized":"2023-08-27T11:25:09.045788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:38.184300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.051943+0000","last_change":"2023-08-27T11:25:09.051943+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.051943+0000","last_undegraded":"2023-08-27T11:25:09.051943+0000","last_fullsized":"2023-08-27T11:25:09.051943+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.046525+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:25:09.046525+0000","last_peered":"2023-08-27T11:25:09.046525+0000","last_clean":"2023-08-27T11:25:09.046525+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:25:09.046525+0000","last_undegraded":"2023-08-27T11:25:09.046525+0000","last_fullsized":"2023-08-27T11:25:09.046525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:38:17.153603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":341,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.049404+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:25:09.049404+0000","last_peered":"2023-08-27T11:25:09.049404+0000","last_clean":"2023-08-27T11:25:09.049404+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:25:09.049404+0000","last_undegraded":"2023-08-27T11:25:09.049404+0000","last_fullsized":"2023-08-27T11:25:09.049404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:42:09.976948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042667+0000","last_change":"2023-08-27T11:25:09.042667+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042667+0000","last_undegraded":"2023-08-27T11:25:09.042667+0000","last_fullsized":"2023-08-27T11:25:09.042667+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.049543+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:25:09.049543+0000","last_peered":"2023-08-27T11:25:09.049543+0000","last_clean":"2023-08-27T11:25:09.049543+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:25:09.049543+0000","last_undegraded":"2023-08-27T11:25:09.049543+0000","last_fullsized":"2023-08-27T11:25:09.049543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:08:36.959604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044316+0000","last_change":"2023-08-27T11:25:09.044316+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044316+0000","last_undegraded":"2023-08-27T11:25:09.044316+0000","last_fullsized":"2023-08-27T11:25:09.044316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.049483+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:25:09.049483+0000","last_peered":"2023-08-27T11:25:09.049483+0000","last_clean":"2023-08-27T11:25:09.049483+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:25:09.049483+0000","last_undegraded":"2023-08-27T11:25:09.049483+0000","last_fullsized":"2023-08-27T11:25:09.049483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:17.326927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045853+0000","last_change":"2023-08-27T11:25:09.045853+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045853+0000","last_undegraded":"2023-08-27T11:25:09.045853+0000","last_fullsized":"2023-08-27T11:25:09.045853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.040147+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:25:09.040147+0000","last_peered":"2023-08-27T11:25:09.040147+0000","last_clean":"2023-08-27T11:25:09.040147+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:25:09.040147+0000","last_undegraded":"2023-08-27T11:25:09.040147+0000","last_fullsized":"2023-08-27T11:25:09.040147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:37.914648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044553+0000","last_change":"2023-08-27T11:25:09.044553+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044553+0000","last_undegraded":"2023-08-27T11:25:09.044553+0000","last_fullsized":"2023-08-27T11:25:09.044553+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.052586+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:25:09.052586+0000","last_peered":"2023-08-27T11:25:09.052586+0000","last_clean":"2023-08-27T11:25:09.052586+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:25:09.052586+0000","last_undegraded":"2023-08-27T11:25:09.052586+0000","last_fullsized":"2023-08-27T11:25:09.052586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:55.622304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":993925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.875,"15min":0.875},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.55900000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.33700000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.314},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.48299999999999998,"5min":0.35199999999999998,"15min":0.252},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.439,"5min":0.439,"15min":0.29099999999999998},"max":{"1min":0.79100000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48399999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65600000000000003,"15min":0.61599999999999999},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65700000000000003,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.39900000000000002},"max":{"1min":0.82199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55600000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.26500000000000001,"15min":0.23499999999999999},"max":{"1min":0.65400000000000003,"5min":0.67500000000000004,"15min":0.72499999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.52900000000000003},"min":{"1min":0.39100000000000001,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.57399999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.74199999999999999,"15min":0.84699999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.30099999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.90100000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.626}]}]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.59699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.377,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":1.3540000000000001},"last":0.79900000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":3.3780000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.81399999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.71199999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":9.3770000000000007},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.746,"5min":0.82799999999999996,"15min":1.325},"last":0.74199999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60399999999999998,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34100000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.84899999999999998,"15min":1.395},"last":0.75800000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.25900000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34300000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68500000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54600000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.5},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.82099999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.32900000000000001},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.65500000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72299999999999998,"5min":0.76100000000000001,"15min":0.88900000000000001},"last":0.621},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.34899999999999998},"max":{"1min":0.76400000000000001,"5min":0.77400000000000002,"15min":0.86299999999999999},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":802816,"data_stored":410091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.20399999999999999},"max":{"1min":0.52700000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.60699999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.52700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":0.68100000000000005,"5min":0.68300000000000005,"15min":0.78600000000000003},"last":0.45000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.32500000000000001,"5min":0.311,"15min":0.27600000000000002},"max":{"1min":0.61899999999999999,"5min":0.70399999999999996,"15min":0.77100000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45700000000000002,"15min":0.434},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.23400000000000001},"max":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.80000000000000004},"last":0.48999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.51700000000000002,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.97999999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.94199999999999995,"15min":2.9380000000000002},"last":0.749}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.42099999999999999},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.627},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97999999999999998,"15min":2.9740000000000002},"last":0.72499999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.54700000000000004,"5min":0.47399999999999998,"15min":0.42499999999999999},"max":{"1min":0.85499999999999998,"5min":1.002,"15min":1.139},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.876,"5min":0.91600000000000004,"15min":1.0169999999999999},"last":0.77400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":1000459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41999999999999998,"15min":0.40600000000000003},"min":{"1min":0.28199999999999997,"5min":0.27200000000000002,"15min":0.23699999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.90200000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.47499999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":1.2889999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42899999999999999,"15min":0.41399999999999998},"min":{"1min":0.317,"5min":0.313,"15min":0.22900000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":1.3340000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.505},"min":{"1min":0.30199999999999999,"5min":0.29099999999999998,"15min":0.255},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":8.1349999999999998},"last":0.38200000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.56399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.23899999999999999},"max":{"1min":0.56899999999999995,"5min":0.69899999999999995,"15min":11.026999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.29699999999999999},"max":{"1min":0.63800000000000001,"5min":0.73299999999999998,"15min":11.926},"last":0.45400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66500000000000004,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":1.0069999999999999,"15min":10.964},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":1.3640000000000001},"last":0.85099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.73699999999999999},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":8.0779999999999994},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70199999999999996,"15min":0.81000000000000005},"min":{"1min":0.53300000000000003,"5min":0.42799999999999999,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1,"15min":10.922000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.35699999999999998},"max":{"1min":0.82499999999999996,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.51400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":8.0380000000000003},"last":0.873}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.78500000000000003},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":8.1069999999999993},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":1.036,"15min":10.996},"last":0.66900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575010,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":410091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38500000000000001,"15min":0.372},"min":{"1min":0.25800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.65400000000000003},"last":0.505},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.437,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.20799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.753},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46000000000000002,"15min":0.443},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.24099999999999999},"max":{"1min":0.71299999999999997,"5min":0.71699999999999997,"15min":0.77800000000000002},"last":0.434}]},{"osd":3,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48099999999999998,"15min":0.46200000000000002},"min":{"1min":0.34599999999999997,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.29299999999999998},"max":{"1min":0.61899999999999999,"5min":0.65300000000000002,"15min":0.69099999999999995},"last":0.376}]},{"osd":4,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76500000000000001,"5min":1.163,"15min":1.163},"last":0.622},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76800000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0660000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95199999999999996,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68100000000000005,"15min":0.68000000000000005},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.1319999999999999,"15min":1.2050000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0189999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.499,"15min":0.40100000000000002},"max":{"1min":1.0409999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45700000000000002,"15min":0.42299999999999999},"max":{"1min":0.96999999999999997,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40999999999999998,"15min":0.52300000000000002},"min":{"1min":0.26800000000000002,"5min":0.23899999999999999,"15min":0.22},"max":{"1min":0.53800000000000003,"5min":0.63300000000000001,"15min":11.885999999999999},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44500000000000001,"15min":0.40000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.224},"max":{"1min":0.59899999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33400000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.32500000000000001,"5min":0.29299999999999998,"15min":0.255},"max":{"1min":0.57399999999999995,"5min":0.67900000000000005,"15min":6.0830000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.503,"15min":0.60599999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.245},"max":{"1min":0.68899999999999995,"5min":0.80700000000000005,"15min":12.050000000000001},"last":0.371}]},{"osd":2,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.504,"15min":0.45800000000000002},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.27100000000000002},"max":{"1min":0.66700000000000004,"5min":0.93100000000000005,"15min":1.25},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54200000000000004,"15min":0.63800000000000001},"min":{"1min":0.376,"5min":0.374,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.83999999999999997,"15min":11.939},"last":0.31}]},{"osd":4,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.76900000000000002},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.376},"max":{"1min":0.86099999999999999,"5min":1.0329999999999999,"15min":11.007},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70199999999999996,"15min":0.65400000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":0.91000000000000003,"5min":1.113,"15min":1.2929999999999999},"last":0.503}]},{"osd":5,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70499999999999996,"15min":0.81200000000000006},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.436},"max":{"1min":0.92500000000000004,"5min":1.0800000000000001,"15min":11.981},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.79300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.218,"15min":11.09},"last":0.63500000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72799999999999998,"15min":0.91400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.38200000000000001},"max":{"1min":0.88900000000000001,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75800000000000001,"15min":0.85499999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.47799999999999998},"max":{"1min":0.95899999999999996,"5min":1.0389999999999999,"15min":12.016},"last":0.58199999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.80900000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.41599999999999998},"max":{"1min":0.91100000000000003,"5min":1.006,"15min":11.054},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.69299999999999995},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":1.0640000000000001,"15min":1.361},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.23499999999999999},"max":{"1min":0.71099999999999997,"5min":0.81799999999999995,"15min":0.89800000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.60399999999999998},"min":{"1min":0.501,"5min":0.40699999999999997,"15min":0.255},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.873},"last":0.57699999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.622,"15min":0.59699999999999998},"min":{"1min":0.51800000000000002,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82999999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.747,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58999999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.312},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.629},"min":{"1min":0.52100000000000002,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.745}]},{"osd":3,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.503,"5min":0.40200000000000002,"15min":0.30099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.62},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.49399999999999999,"5min":0.371,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.88800000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.437},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.497,"15min":0.51600000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.26200000000000001},"max":{"1min":0.72299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45600000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.27800000000000002},"max":{"1min":0.72199999999999998,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.93899999999999995},"last":0.503}]},{"osd":7,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.874,"15min":0.878},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.48299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36116,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335724,"statfs":{"total":96636764160,"available":96599781376,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.93300000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.26200000000000001},"max":{"1min":0.749,"5min":0.84999999999999998,"15min":0.88100000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.248},"max":{"1min":0.80400000000000005,"5min":0.81899999999999995,"15min":0.96999999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":0.77400000000000002,"15min":0.82299999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.33400000000000002,"15min":0.32800000000000001},"max":{"1min":0.75,"5min":0.87,"15min":0.87},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":1.0109999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.80600000000000005,"15min":0.94199999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.34799999999999998,"15min":0.34000000000000002},"max":{"1min":0.79400000000000004,"5min":0.79500000000000004,"15min":0.83799999999999997},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71299999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.372}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47999999999999998,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.89000000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.379,"15min":0.379},"max":{"1min":0.75,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.88100000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.84999999999999998},"last":0.497}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:15.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.828+0000 7f55e0605700 1 Processor -- start 2023-08-27T11:25:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.828+0000 7f55e0605700 1 -- start start 2023-08-27T11:25:15.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55e0605700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55d815dbc0 0x7f55d815a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55e0605700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d80caef0 con 0x7f55d8070e40 2023-08-27T11:25:15.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55e0605700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d80ce000 con 0x7f55d815a750 2023-08-27T11:25:15.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55e0605700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d80cac50 con 0x7f55d815dbc0 2023-08-27T11:25:15.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55de3a1700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f55d8070e40 0x7f55d8071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43072/0 (socket says 172.21.15.45:43072) 2023-08-27T11:25:15.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55de3a1700 1 -- 172.21.15.45:0/1472115498 learned_addr learned my addr 172.21.15.45:0/1472115498 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:15.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805473126 0 0) 0x7f55d80caef0 con 0x7f55d8070e40 2023-08-27T11:25:15.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c40035b0 con 0x7f55d8070e40 2023-08-27T11:25:15.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55ddba0700 1 --2- 172.21.15.45:0/1472115498 >> v2:172.21.15.135:3300/0 conn(0x7f55d815dbc0 0x7f55d815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2033723711 0 0) 0x7f55c40035b0 con 0x7f55d8070e40 2023-08-27T11:25:15.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 >> v2:172.21.15.135:3300/0 conn(0x7f55d815dbc0 msgr2=0x7f55d815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:15.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 --2- 172.21.15.45:0/1472115498 >> v2:172.21.15.135:3300/0 conn(0x7f55d815dbc0 0x7f55d815a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 >> v1:172.21.15.135:6789/0 conn(0x7f55d815a750 legacy=0x7f55d80cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d815b5c0 con 0x7f55d8070e40 2023-08-27T11:25:15.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f55d4001670 con 0x7f55d8070e40 2023-08-27T11:25:15.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 >> v1:172.21.15.45:6789/0 conn(0x7f55d8070e40 legacy=0x7f55d8071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4003a30 0x7f55c4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c4004850 0x7f55c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55c4005260 0x7f55c400dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d80cac50 con 0x7f55c4003a30 2023-08-27T11:25:15.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c400e3a0 con 0x7f55c4004850 2023-08-27T11:25:15.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c400e560 con 0x7f55c4005260 2023-08-27T11:25:15.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.829+0000 7f55de3a1700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4003a30 0x7f55c4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55ddba0700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c4004850 0x7f55c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55deba2700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55c4005260 0x7f55c400dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55de3a1700 1 -- 172.21.15.45:0/1472115498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55c4005260 msgr2=0x7f55c400dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:15.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55de3a1700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55c4005260 0x7f55c400dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55de3a1700 1 -- 172.21.15.45:0/1472115498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c4004850 msgr2=0x7f55c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:15.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55de3a1700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c4004850 0x7f55c4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55de3a1700 1 -- 172.21.15.45:0/1472115498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c40035b0 con 0x7f55c4003a30 2023-08-27T11:25:15.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55deba2700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55c4005260 0x7f55c400dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:15.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55deba2700 1 -- 172.21.15.45:0/1472115498 reap_dead start 2023-08-27T11:25:15.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55de3a1700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4003a30 0x7f55c4003e50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c210 tx=0x7f55c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3026abafb534fdd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:15.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4007c10 con 0x7f55c4003a30 2023-08-27T11:25:15.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d400cdd0 con 0x7f55c4003a30 2023-08-27T11:25:15.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55dd39f700 1 -- 172.21.15.45:0/1472115498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4005ba0 con 0x7f55c4003a30 2023-08-27T11:25:15.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55e0605700 1 -- 172.21.15.45:0/1472115498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4003a30 msgr2=0x7f55c4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.830+0000 7f55e0605700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4003a30 0x7f55c4003e50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c210 tx=0x7f55c4005880 comp rx=0 tx=0).stop 2023-08-27T11:25:15.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.831+0000 7f55e0605700 1 -- 172.21.15.45:0/1472115498 shutdown_connections 2023-08-27T11:25:15.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.831+0000 7f55e0605700 1 --2- 172.21.15.45:0/1472115498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4003a30 0x7f55c4003e50 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.831+0000 7f55e0605700 1 -- 172.21.15.45:0/1472115498 >> 172.21.15.45:0/1472115498 conn(0x7f55d80cfd60 msgr2=0x7f55d80d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:15.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.831+0000 7f55e0605700 1 -- 172.21.15.45:0/1472115498 shutdown_connections 2023-08-27T11:25:15.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.831+0000 7f55e0605700 1 -- 172.21.15.45:0/1472115498 wait complete. 2023-08-27T11:25:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.832+0000 7f55e0605700 1 Processor -- start 2023-08-27T11:25:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.832+0000 7f55e0605700 1 -- start start 2023-08-27T11:25:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.832+0000 7f55e0605700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c4003a30 0x7f55d80c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55e0605700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55d80c4000 0x7f55d814f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55e0605700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d80b4030 0x7f55d80b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55e0605700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d80c4cb0 con 0x7f55d80b4030 2023-08-27T11:25:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55e0605700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d80b49f0 con 0x7f55c4003a30 2023-08-27T11:25:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55e0605700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d80c7e90 con 0x7f55d80c4000 2023-08-27T11:25:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55deba2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d80b4030 0x7f55d80b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55deba2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d80b4030 0x7f55d80b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56312/0 (socket says 172.21.15.45:56312) 2023-08-27T11:25:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55deba2700 1 -- 172.21.15.45:0/3427635765 learned_addr learned my addr 172.21.15.45:0/3427635765 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55ddba0700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55d80c4000 0x7f55d814f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55de3a1700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c4003a30 0x7f55d80c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55deba2700 1 -- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55d80c4000 msgr2=0x7f55d814f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55deba2700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55d80c4000 0x7f55d814f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55deba2700 1 -- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c4003a30 msgr2=0x7f55d80c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55deba2700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c4003a30 0x7f55d80c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55deba2700 1 -- 172.21.15.45:0/3427635765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d80c8170 con 0x7f55d80b4030 2023-08-27T11:25:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55deba2700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d80b4030 0x7f55d80b44b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f55d8072b20 tx=0x7f55d000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55ceffd700 1 -- 172.21.15.45:0/3427635765 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0003c10 con 0x7f55d80b4030 2023-08-27T11:25:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55ceffd700 1 -- 172.21.15.45:0/3427635765 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d0003d90 con 0x7f55d80b4030 2023-08-27T11:25:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55ceffd700 1 -- 172.21.15.45:0/3427635765 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0005310 con 0x7f55d80b4030 2023-08-27T11:25:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d80c8450 con 0x7f55d80b4030 2023-08-27T11:25:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.833+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d814f9e0 con 0x7f55d80b4030 2023-08-27T11:25:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.834+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d8072180 con 0x7f55d80b4030 2023-08-27T11:25:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.835+0000 7f55ceffd700 1 -- 172.21.15.45:0/3427635765 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f55d0005470 con 0x7f55d80b4030 2023-08-27T11:25:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.835+0000 7f55ceffd700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f55bc06a200 0x7f55bc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.835+0000 7f55ceffd700 1 -- 172.21.15.45:0/3427635765 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f55d0088880 con 0x7f55d80b4030 2023-08-27T11:25:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.836+0000 7f55de3a1700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f55bc06a200 0x7f55bc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.836+0000 7f55de3a1700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f55bc06a200 0x7f55bc06c6c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55d4000c00 tx=0x7f55d4007000 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:15.840+0000 7f55ceffd700 1 -- 172.21.15.45:0/3427635765 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f55d0053cf0 con 0x7f55d80b4030 2023-08-27T11:25:16.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.037+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 --> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55d815b000 con 0x7f55bc06a200 2023-08-27T11:25:16.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.045+0000 7f55ceffd700 1 -- 172.21.15.45:0/3427635765 <== mgr.5402 v2:172.21.15.135:6832/3181978685 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299372 (secure 0 0 0) 0x7f55d400cc30 con 0x7f55bc06a200 2023-08-27T11:25:16.050 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:16.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.048+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f55bc06a200 msgr2=0x7f55bc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.048+0000 7f55e0605700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f55bc06a200 0x7f55bc06c6c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55d4000c00 tx=0x7f55d4007000 comp rx=0 tx=0).stop 2023-08-27T11:25:16.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.048+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d80b4030 msgr2=0x7f55d80b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.048+0000 7f55e0605700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d80b4030 0x7f55d80b44b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f55d8072b20 tx=0x7f55d000ee50 comp rx=0 tx=0).stop 2023-08-27T11:25:16.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.049+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 shutdown_connections 2023-08-27T11:25:16.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.049+0000 7f55e0605700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f55bc06a200 0x7f55bc06c6c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.049+0000 7f55e0605700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55d80b4030 0x7f55d80b44b0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.049+0000 7f55e0605700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c4003a30 0x7f55d80c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.049+0000 7f55e0605700 1 --2- 172.21.15.45:0/3427635765 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f55d80c4000 0x7f55d814f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.049+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 >> 172.21.15.45:0/3427635765 conn(0x7f55d80cfd60 msgr2=0x7f55d80d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:16.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.049+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 shutdown_connections 2023-08-27T11:25:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.049+0000 7f55e0605700 1 -- 172.21.15.45:0/3427635765 wait complete. 2023-08-27T11:25:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:25:16.062 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-08-27T11:25:14.322104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265144,"kb_used_data":7372,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754709576,"statfs":{"total":773094113280,"available":772822605824,"internally_reserved":0,"allocated":7548928,"data_stored":4428794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.279241"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045823+0000","last_change":"2023-08-27T11:25:09.045823+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045823+0000","last_undegraded":"2023-08-27T11:25:09.045823+0000","last_fullsized":"2023-08-27T11:25:09.045823+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044948+0000","last_change":"2023-08-27T11:25:09.044948+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044948+0000","last_undegraded":"2023-08-27T11:25:09.044948+0000","last_fullsized":"2023-08-27T11:25:09.044948+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.046665+0000","last_change":"2023-08-27T11:25:09.046665+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.046665+0000","last_undegraded":"2023-08-27T11:25:09.046665+0000","last_fullsized":"2023-08-27T11:25:09.046665+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045158+0000","last_change":"2023-08-27T11:25:09.045158+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045158+0000","last_undegraded":"2023-08-27T11:25:09.045158+0000","last_fullsized":"2023-08-27T11:25:09.045158+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045826+0000","last_change":"2023-08-27T11:25:09.045826+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045826+0000","last_undegraded":"2023-08-27T11:25:09.045826+0000","last_fullsized":"2023-08-27T11:25:09.045826+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.046796+0000","last_change":"2023-08-27T11:25:09.046796+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.046796+0000","last_undegraded":"2023-08-27T11:25:09.046796+0000","last_fullsized":"2023-08-27T11:25:09.046796+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045408+0000","last_change":"2023-08-27T11:25:09.045408+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045408+0000","last_undegraded":"2023-08-27T11:25:09.045408+0000","last_fullsized":"2023-08-27T11:25:09.045408+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045965+0000","last_change":"2023-08-27T11:25:09.045965+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045965+0000","last_undegraded":"2023-08-27T11:25:09.045965+0000","last_fullsized":"2023-08-27T11:25:09.045965+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.048008+0000","last_change":"2023-08-27T11:25:09.048008+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.048008+0000","last_undegraded":"2023-08-27T11:25:09.048008+0000","last_fullsized":"2023-08-27T11:25:09.048008+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044403+0000","last_change":"2023-08-27T11:25:09.044403+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044403+0000","last_undegraded":"2023-08-27T11:25:09.044403+0000","last_fullsized":"2023-08-27T11:25:09.044403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.048244+0000","last_change":"2023-08-27T11:25:09.048244+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.048244+0000","last_undegraded":"2023-08-27T11:25:09.048244+0000","last_fullsized":"2023-08-27T11:25:09.048244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047418+0000","last_change":"2023-08-27T11:25:09.047418+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047418+0000","last_undegraded":"2023-08-27T11:25:09.047418+0000","last_fullsized":"2023-08-27T11:25:09.047418+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044870+0000","last_change":"2023-08-27T11:25:09.044870+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044870+0000","last_undegraded":"2023-08-27T11:25:09.044870+0000","last_fullsized":"2023-08-27T11:25:09.044870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.051997+0000","last_change":"2023-08-27T11:25:09.051997+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.051997+0000","last_undegraded":"2023-08-27T11:25:09.051997+0000","last_fullsized":"2023-08-27T11:25:09.051997+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044564+0000","last_change":"2023-08-27T11:25:09.044564+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044564+0000","last_undegraded":"2023-08-27T11:25:09.044564+0000","last_fullsized":"2023-08-27T11:25:09.044564+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047863+0000","last_change":"2023-08-27T11:25:09.047863+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047863+0000","last_undegraded":"2023-08-27T11:25:09.047863+0000","last_fullsized":"2023-08-27T11:25:09.047863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042153+0000","last_change":"2023-08-27T11:25:09.042153+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042153+0000","last_undegraded":"2023-08-27T11:25:09.042153+0000","last_fullsized":"2023-08-27T11:25:09.042153+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047304+0000","last_change":"2023-08-27T11:25:09.047304+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047304+0000","last_undegraded":"2023-08-27T11:25:09.047304+0000","last_fullsized":"2023-08-27T11:25:09.047304+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.052065+0000","last_change":"2023-08-27T11:25:09.052065+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.052065+0000","last_undegraded":"2023-08-27T11:25:09.052065+0000","last_fullsized":"2023-08-27T11:25:09.052065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044678+0000","last_change":"2023-08-27T11:25:09.044678+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044678+0000","last_undegraded":"2023-08-27T11:25:09.044678+0000","last_fullsized":"2023-08-27T11:25:09.044678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042196+0000","last_change":"2023-08-27T11:25:09.042196+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042196+0000","last_undegraded":"2023-08-27T11:25:09.042196+0000","last_fullsized":"2023-08-27T11:25:09.042196+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045417+0000","last_change":"2023-08-27T11:25:09.045417+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045417+0000","last_undegraded":"2023-08-27T11:25:09.045417+0000","last_fullsized":"2023-08-27T11:25:09.045417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042127+0000","last_change":"2023-08-27T11:25:09.042127+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042127+0000","last_undegraded":"2023-08-27T11:25:09.042127+0000","last_fullsized":"2023-08-27T11:25:09.042127+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044403+0000","last_change":"2023-08-27T11:25:09.044403+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044403+0000","last_undegraded":"2023-08-27T11:25:09.044403+0000","last_fullsized":"2023-08-27T11:25:09.044403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.047735+0000","last_change":"2023-08-27T11:25:09.047735+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.047735+0000","last_undegraded":"2023-08-27T11:25:09.047735+0000","last_fullsized":"2023-08-27T11:25:09.047735+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.040134+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:25:09.040134+0000","last_peered":"2023-08-27T11:25:09.040134+0000","last_clean":"2023-08-27T11:25:09.040134+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:25:09.040134+0000","last_undegraded":"2023-08-27T11:25:09.040134+0000","last_fullsized":"2023-08-27T11:25:09.040134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:32:58.034773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045725+0000","last_change":"2023-08-27T11:25:09.045725+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045725+0000","last_undegraded":"2023-08-27T11:25:09.045725+0000","last_fullsized":"2023-08-27T11:25:09.045725+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.046326+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:25:09.046326+0000","last_peered":"2023-08-27T11:25:09.046326+0000","last_clean":"2023-08-27T11:25:09.046326+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:25:09.046326+0000","last_undegraded":"2023-08-27T11:25:09.046326+0000","last_fullsized":"2023-08-27T11:25:09.046326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:04:56.999493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044379+0000","last_change":"2023-08-27T11:25:09.044379+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044379+0000","last_undegraded":"2023-08-27T11:25:09.044379+0000","last_fullsized":"2023-08-27T11:25:09.044379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.045788+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:25:09.045788+0000","last_peered":"2023-08-27T11:25:09.045788+0000","last_clean":"2023-08-27T11:25:09.045788+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:25:09.045788+0000","last_undegraded":"2023-08-27T11:25:09.045788+0000","last_fullsized":"2023-08-27T11:25:09.045788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:12:38.184300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.051943+0000","last_change":"2023-08-27T11:25:09.051943+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.051943+0000","last_undegraded":"2023-08-27T11:25:09.051943+0000","last_fullsized":"2023-08-27T11:25:09.051943+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.046525+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:25:09.046525+0000","last_peered":"2023-08-27T11:25:09.046525+0000","last_clean":"2023-08-27T11:25:09.046525+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:25:09.046525+0000","last_undegraded":"2023-08-27T11:25:09.046525+0000","last_fullsized":"2023-08-27T11:25:09.046525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:38:17.153603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":341,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.049404+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:25:09.049404+0000","last_peered":"2023-08-27T11:25:09.049404+0000","last_clean":"2023-08-27T11:25:09.049404+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:25:09.049404+0000","last_undegraded":"2023-08-27T11:25:09.049404+0000","last_fullsized":"2023-08-27T11:25:09.049404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:42:09.976948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.042667+0000","last_change":"2023-08-27T11:25:09.042667+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.042667+0000","last_undegraded":"2023-08-27T11:25:09.042667+0000","last_fullsized":"2023-08-27T11:25:09.042667+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.049543+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:25:09.049543+0000","last_peered":"2023-08-27T11:25:09.049543+0000","last_clean":"2023-08-27T11:25:09.049543+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:25:09.049543+0000","last_undegraded":"2023-08-27T11:25:09.049543+0000","last_fullsized":"2023-08-27T11:25:09.049543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:08:36.959604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044316+0000","last_change":"2023-08-27T11:25:09.044316+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044316+0000","last_undegraded":"2023-08-27T11:25:09.044316+0000","last_fullsized":"2023-08-27T11:25:09.044316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.049483+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:25:09.049483+0000","last_peered":"2023-08-27T11:25:09.049483+0000","last_clean":"2023-08-27T11:25:09.049483+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:25:09.049483+0000","last_undegraded":"2023-08-27T11:25:09.049483+0000","last_fullsized":"2023-08-27T11:25:09.049483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:17.326927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.045853+0000","last_change":"2023-08-27T11:25:09.045853+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.045853+0000","last_undegraded":"2023-08-27T11:25:09.045853+0000","last_fullsized":"2023-08-27T11:25:09.045853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.040147+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:25:09.040147+0000","last_peered":"2023-08-27T11:25:09.040147+0000","last_clean":"2023-08-27T11:25:09.040147+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:25:09.040147+0000","last_undegraded":"2023-08-27T11:25:09.040147+0000","last_fullsized":"2023-08-27T11:25:09.040147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:33:37.914648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-27T11:25:10.038390+0000","last_change":"2023-08-27T11:25:10.038390+0000","last_active":"2023-08-27T11:25:10.038390+0000","last_peered":"2023-08-27T11:25:10.038390+0000","last_clean":"2023-08-27T11:25:10.038390+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:10.038390+0000","last_undegraded":"2023-08-27T11:25:10.038390+0000","last_fullsized":"2023-08-27T11:25:10.038390+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2023-08-27T11:25:09.044553+0000","last_change":"2023-08-27T11:25:09.044553+0000","last_active":"2023-08-27T11:25:09.029612+0000","last_peered":"2023-08-27T11:25:09.029612+0000","last_clean":"2023-08-27T11:25:09.029612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:25:09.044553+0000","last_undegraded":"2023-08-27T11:25:09.044553+0000","last_fullsized":"2023-08-27T11:25:09.044553+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-08-27T11:25:09.052586+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:25:09.052586+0000","last_peered":"2023-08-27T11:25:09.052586+0000","last_clean":"2023-08-27T11:25:09.052586+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:25:09.052586+0000","last_undegraded":"2023-08-27T11:25:09.052586+0000","last_fullsized":"2023-08-27T11:25:09.052586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:46:55.622304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":993925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.875,"15min":0.875},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.55900000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.33700000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.314},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.48299999999999998,"5min":0.35199999999999998,"15min":0.252},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.439,"5min":0.439,"15min":0.29099999999999998},"max":{"1min":0.79100000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48399999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65600000000000003,"15min":0.61599999999999999},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65700000000000003,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.39900000000000002},"max":{"1min":0.82199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55600000000000005}]},{"osd":4,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.26500000000000001,"15min":0.23499999999999999},"max":{"1min":0.65400000000000003,"5min":0.67500000000000004,"15min":0.72499999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.52900000000000003},"min":{"1min":0.39100000000000001,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.57399999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.74199999999999999,"15min":0.84699999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.30099999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.90100000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.626}]}]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.59699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.377,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":1.3540000000000001},"last":0.79900000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":3.3780000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.81399999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.71199999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":9.3770000000000007},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.746,"5min":0.82799999999999996,"15min":1.325},"last":0.74199999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60399999999999998,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34100000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.84899999999999998,"15min":1.395},"last":0.75800000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.25900000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34300000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68500000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54600000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.5},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.82099999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.32900000000000001},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.65500000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72299999999999998,"5min":0.76100000000000001,"15min":0.88900000000000001},"last":0.621},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.34899999999999998},"max":{"1min":0.76400000000000001,"5min":0.77400000000000002,"15min":0.86299999999999999},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":802816,"data_stored":410091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.20399999999999999},"max":{"1min":0.52700000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.60699999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.52700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":0.68100000000000005,"5min":0.68300000000000005,"15min":0.78600000000000003},"last":0.45000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.32500000000000001,"5min":0.311,"15min":0.27600000000000002},"max":{"1min":0.61899999999999999,"5min":0.70399999999999996,"15min":0.77100000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45700000000000002,"15min":0.434},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.23400000000000001},"max":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.80000000000000004},"last":0.48999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.51700000000000002,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.97999999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.94199999999999995,"15min":2.9380000000000002},"last":0.749}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.42099999999999999},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.627},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97999999999999998,"15min":2.9740000000000002},"last":0.72499999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.54700000000000004,"5min":0.47399999999999998,"15min":0.42499999999999999},"max":{"1min":0.85499999999999998,"5min":1.002,"15min":1.139},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.876,"5min":0.91600000000000004,"15min":1.0169999999999999},"last":0.77400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":1000459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41999999999999998,"15min":0.40600000000000003},"min":{"1min":0.28199999999999997,"5min":0.27200000000000002,"15min":0.23699999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.90200000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.47499999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":1.2889999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42899999999999999,"15min":0.41399999999999998},"min":{"1min":0.317,"5min":0.313,"15min":0.22900000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":1.3340000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.505},"min":{"1min":0.30199999999999999,"5min":0.29099999999999998,"15min":0.255},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":8.1349999999999998},"last":0.38200000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.56399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.23899999999999999},"max":{"1min":0.56899999999999995,"5min":0.69899999999999995,"15min":11.026999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.29699999999999999},"max":{"1min":0.63800000000000001,"5min":0.73299999999999998,"15min":11.926},"last":0.45400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66500000000000004,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":1.0069999999999999,"15min":10.964},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":1.3640000000000001},"last":0.85099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.73699999999999999},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":8.0779999999999994},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70199999999999996,"15min":0.81000000000000005},"min":{"1min":0.53300000000000003,"5min":0.42799999999999999,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1,"15min":10.922000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.35699999999999998},"max":{"1min":0.82499999999999996,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.51400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":8.0380000000000003},"last":0.873}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.78500000000000003},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":8.1069999999999993},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":1.036,"15min":10.996},"last":0.66900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575010,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":410091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38500000000000001,"15min":0.372},"min":{"1min":0.25800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.65400000000000003},"last":0.505},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.437,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.20799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.753},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46000000000000002,"15min":0.443},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.24099999999999999},"max":{"1min":0.71299999999999997,"5min":0.71699999999999997,"15min":0.77800000000000002},"last":0.434}]},{"osd":3,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48099999999999998,"15min":0.46200000000000002},"min":{"1min":0.34599999999999997,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.29299999999999998},"max":{"1min":0.61899999999999999,"5min":0.65300000000000002,"15min":0.69099999999999995},"last":0.376}]},{"osd":4,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76500000000000001,"5min":1.163,"15min":1.163},"last":0.622},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76800000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0660000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95199999999999996,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68100000000000005,"15min":0.68000000000000005},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.1319999999999999,"15min":1.2050000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0189999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.499,"15min":0.40100000000000002},"max":{"1min":1.0409999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45700000000000002,"15min":0.42299999999999999},"max":{"1min":0.96999999999999997,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40999999999999998,"15min":0.52300000000000002},"min":{"1min":0.26800000000000002,"5min":0.23899999999999999,"15min":0.22},"max":{"1min":0.53800000000000003,"5min":0.63300000000000001,"15min":11.885999999999999},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44500000000000001,"15min":0.40000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.224},"max":{"1min":0.59899999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33400000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.32500000000000001,"5min":0.29299999999999998,"15min":0.255},"max":{"1min":0.57399999999999995,"5min":0.67900000000000005,"15min":6.0830000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.503,"15min":0.60599999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.245},"max":{"1min":0.68899999999999995,"5min":0.80700000000000005,"15min":12.050000000000001},"last":0.371}]},{"osd":2,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.504,"15min":0.45800000000000002},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.27100000000000002},"max":{"1min":0.66700000000000004,"5min":0.93100000000000005,"15min":1.25},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54200000000000004,"15min":0.63800000000000001},"min":{"1min":0.376,"5min":0.374,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.83999999999999997,"15min":11.939},"last":0.31}]},{"osd":4,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.76900000000000002},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.376},"max":{"1min":0.86099999999999999,"5min":1.0329999999999999,"15min":11.007},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70199999999999996,"15min":0.65400000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":0.91000000000000003,"5min":1.113,"15min":1.2929999999999999},"last":0.503}]},{"osd":5,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70499999999999996,"15min":0.81200000000000006},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.436},"max":{"1min":0.92500000000000004,"5min":1.0800000000000001,"15min":11.981},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.79300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.218,"15min":11.09},"last":0.63500000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72799999999999998,"15min":0.91400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.38200000000000001},"max":{"1min":0.88900000000000001,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75800000000000001,"15min":0.85499999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.47799999999999998},"max":{"1min":0.95899999999999996,"5min":1.0389999999999999,"15min":12.016},"last":0.58199999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.80900000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.41599999999999998},"max":{"1min":0.91100000000000003,"5min":1.006,"15min":11.054},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.69299999999999995},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":1.0640000000000001,"15min":1.361},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.23499999999999999},"max":{"1min":0.71099999999999997,"5min":0.81799999999999995,"15min":0.89800000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.60399999999999998},"min":{"1min":0.501,"5min":0.40699999999999997,"15min":0.255},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.873},"last":0.57699999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.622,"15min":0.59699999999999998},"min":{"1min":0.51800000000000002,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82999999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.747,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58999999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.312},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.629},"min":{"1min":0.52100000000000002,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.745}]},{"osd":3,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.503,"5min":0.40200000000000002,"15min":0.30099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.62},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.49399999999999999,"5min":0.371,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.88800000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.437},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.497,"15min":0.51600000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.26200000000000001},"max":{"1min":0.72299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45600000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.27800000000000002},"max":{"1min":0.72199999999999998,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.93899999999999995},"last":0.503}]},{"osd":7,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.874,"15min":0.878},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.48299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36116,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335724,"statfs":{"total":96636764160,"available":96599781376,"internally_reserved":0,"allocated":790528,"data_stored":403557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.93300000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.26200000000000001},"max":{"1min":0.749,"5min":0.84999999999999998,"15min":0.88100000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.248},"max":{"1min":0.80400000000000005,"5min":0.81899999999999995,"15min":0.96999999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":0.77400000000000002,"15min":0.82299999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.33400000000000002,"15min":0.32800000000000001},"max":{"1min":0.75,"5min":0.87,"15min":0.87},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":1.0109999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.80600000000000005,"15min":0.94199999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.34799999999999998,"15min":0.34000000000000002},"max":{"1min":0.79400000000000004,"5min":0.79500000000000004,"15min":0.83799999999999997},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71299999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.372}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47999999999999998,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.89000000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.379,"15min":0.379},"max":{"1min":0.75,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.88100000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.84999999999999998},"last":0.497}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:16.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2023-08-27T11:25:16.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2023-08-27T11:25:16.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2023-08-27T11:25:16.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-08-27T11:25:16.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2023-08-27T11:25:16.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2023-08-27T11:25:16.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-08-27T11:25:16.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2023-08-27T11:25:16.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-08-27T11:25:16.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-08-27T11:25:16.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-08-27T11:25:16.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-08-27T11:25:16.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-08-27T11:25:16.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-08-27T11:25:16.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-08-27T11:25:16.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2023-08-27T11:25:16.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2023-08-27T11:25:16.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2023-08-27T11:25:16.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2023-08-27T11:25:16.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2023-08-27T11:25:16.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2023-08-27T11:25:16.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2023-08-27T11:25:16.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2023-08-27T11:25:16.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-08-27T11:25:16.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2023-08-27T11:25:16.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-08-27T11:25:16.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2023-08-27T11:25:16.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2023-08-27T11:25:16.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2023-08-27T11:25:16.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2023-08-27T11:25:16.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2023-08-27T11:25:16.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2023-08-27T11:25:16.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-08-27T11:25:16.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-08-27T11:25:16.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.045823+0000', 'last_change': '2023-08-27T11:25:09.045823+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.045823+0000', 'last_undegraded': '2023-08-27T11:25:09.045823+0000', 'last_fullsized': '2023-08-27T11:25:09.045823+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:16.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-08-27T11:25:16.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-08-27T11:25:16.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.044948+0000', 'last_change': '2023-08-27T11:25:09.044948+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.044948+0000', 'last_undegraded': '2023-08-27T11:25:09.044948+0000', 'last_fullsized': '2023-08-27T11:25:09.044948+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:16.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-08-27T11:25:16.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-08-27T11:25:16.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.046665+0000', 'last_change': '2023-08-27T11:25:09.046665+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.046665+0000', 'last_undegraded': '2023-08-27T11:25:09.046665+0000', 'last_fullsized': '2023-08-27T11:25:09.046665+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:16.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-08-27T11:25:16.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-08-27T11:25:16.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.045158+0000', 'last_change': '2023-08-27T11:25:09.045158+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.045158+0000', 'last_undegraded': '2023-08-27T11:25:09.045158+0000', 'last_fullsized': '2023-08-27T11:25:09.045158+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:16.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-08-27T11:25:16.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-08-27T11:25:16.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.045826+0000', 'last_change': '2023-08-27T11:25:09.045826+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.045826+0000', 'last_undegraded': '2023-08-27T11:25:09.045826+0000', 'last_fullsized': '2023-08-27T11:25:09.045826+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:16.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-08-27T11:25:16.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-08-27T11:25:16.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.046796+0000', 'last_change': '2023-08-27T11:25:09.046796+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.046796+0000', 'last_undegraded': '2023-08-27T11:25:09.046796+0000', 'last_fullsized': '2023-08-27T11:25:09.046796+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:16.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-08-27T11:25:16.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-08-27T11:25:16.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.045408+0000', 'last_change': '2023-08-27T11:25:09.045408+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.045408+0000', 'last_undegraded': '2023-08-27T11:25:09.045408+0000', 'last_fullsized': '2023-08-27T11:25:09.045408+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:16.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-08-27T11:25:16.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-08-27T11:25:16.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.045965+0000', 'last_change': '2023-08-27T11:25:09.045965+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.045965+0000', 'last_undegraded': '2023-08-27T11:25:09.045965+0000', 'last_fullsized': '2023-08-27T11:25:09.045965+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:16.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-08-27T11:25:16.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-08-27T11:25:16.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.048008+0000', 'last_change': '2023-08-27T11:25:09.048008+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.048008+0000', 'last_undegraded': '2023-08-27T11:25:09.048008+0000', 'last_fullsized': '2023-08-27T11:25:09.048008+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:25:16.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-08-27T11:25:16.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-08-27T11:25:16.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.044403+0000', 'last_change': '2023-08-27T11:25:09.044403+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.044403+0000', 'last_undegraded': '2023-08-27T11:25:09.044403+0000', 'last_fullsized': '2023-08-27T11:25:09.044403+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:16.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-08-27T11:25:16.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-08-27T11:25:16.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.048244+0000', 'last_change': '2023-08-27T11:25:09.048244+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.048244+0000', 'last_undegraded': '2023-08-27T11:25:09.048244+0000', 'last_fullsized': '2023-08-27T11:25:09.048244+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:25:16.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-08-27T11:25:16.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-08-27T11:25:16.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.047418+0000', 'last_change': '2023-08-27T11:25:09.047418+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.047418+0000', 'last_undegraded': '2023-08-27T11:25:09.047418+0000', 'last_fullsized': '2023-08-27T11:25:09.047418+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:16.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-08-27T11:25:16.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-08-27T11:25:16.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.044870+0000', 'last_change': '2023-08-27T11:25:09.044870+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.044870+0000', 'last_undegraded': '2023-08-27T11:25:09.044870+0000', 'last_fullsized': '2023-08-27T11:25:09.044870+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:16.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-08-27T11:25:16.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-08-27T11:25:16.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.051997+0000', 'last_change': '2023-08-27T11:25:09.051997+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.051997+0000', 'last_undegraded': '2023-08-27T11:25:09.051997+0000', 'last_fullsized': '2023-08-27T11:25:09.051997+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:25:16.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-08-27T11:25:16.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-08-27T11:25:16.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.044564+0000', 'last_change': '2023-08-27T11:25:09.044564+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.044564+0000', 'last_undegraded': '2023-08-27T11:25:09.044564+0000', 'last_fullsized': '2023-08-27T11:25:09.044564+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:16.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-08-27T11:25:16.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-08-27T11:25:16.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.047863+0000', 'last_change': '2023-08-27T11:25:09.047863+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.047863+0000', 'last_undegraded': '2023-08-27T11:25:09.047863+0000', 'last_fullsized': '2023-08-27T11:25:09.047863+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:25:16.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-08-27T11:25:16.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-08-27T11:25:16.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.042153+0000', 'last_change': '2023-08-27T11:25:09.042153+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.042153+0000', 'last_undegraded': '2023-08-27T11:25:09.042153+0000', 'last_fullsized': '2023-08-27T11:25:09.042153+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:16.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-08-27T11:25:16.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-08-27T11:25:16.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.047304+0000', 'last_change': '2023-08-27T11:25:09.047304+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.047304+0000', 'last_undegraded': '2023-08-27T11:25:09.047304+0000', 'last_fullsized': '2023-08-27T11:25:09.047304+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:16.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-08-27T11:25:16.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-08-27T11:25:16.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.052065+0000', 'last_change': '2023-08-27T11:25:09.052065+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.052065+0000', 'last_undegraded': '2023-08-27T11:25:09.052065+0000', 'last_fullsized': '2023-08-27T11:25:09.052065+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:25:16.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-08-27T11:25:16.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-08-27T11:25:16.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.044678+0000', 'last_change': '2023-08-27T11:25:09.044678+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.044678+0000', 'last_undegraded': '2023-08-27T11:25:09.044678+0000', 'last_fullsized': '2023-08-27T11:25:09.044678+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:16.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-08-27T11:25:16.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-08-27T11:25:16.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.042196+0000', 'last_change': '2023-08-27T11:25:09.042196+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.042196+0000', 'last_undegraded': '2023-08-27T11:25:09.042196+0000', 'last_fullsized': '2023-08-27T11:25:09.042196+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:16.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-08-27T11:25:16.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-08-27T11:25:16.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.045417+0000', 'last_change': '2023-08-27T11:25:09.045417+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.045417+0000', 'last_undegraded': '2023-08-27T11:25:09.045417+0000', 'last_fullsized': '2023-08-27T11:25:09.045417+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:25:16.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-08-27T11:25:16.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-08-27T11:25:16.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.042127+0000', 'last_change': '2023-08-27T11:25:09.042127+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.042127+0000', 'last_undegraded': '2023-08-27T11:25:09.042127+0000', 'last_fullsized': '2023-08-27T11:25:09.042127+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:16.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-08-27T11:25:16.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-08-27T11:25:16.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.044403+0000', 'last_change': '2023-08-27T11:25:09.044403+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.044403+0000', 'last_undegraded': '2023-08-27T11:25:09.044403+0000', 'last_fullsized': '2023-08-27T11:25:09.044403+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:16.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-08-27T11:25:16.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-08-27T11:25:16.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.047735+0000', 'last_change': '2023-08-27T11:25:09.047735+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.047735+0000', 'last_undegraded': '2023-08-27T11:25:09.047735+0000', 'last_fullsized': '2023-08-27T11:25:09.047735+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:16.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-08-27T11:25:16.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-08-27T11:25:16.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.045725+0000', 'last_change': '2023-08-27T11:25:09.045725+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.045725+0000', 'last_undegraded': '2023-08-27T11:25:09.045725+0000', 'last_fullsized': '2023-08-27T11:25:09.045725+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:16.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-08-27T11:25:16.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-08-27T11:25:16.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.044379+0000', 'last_change': '2023-08-27T11:25:09.044379+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.044379+0000', 'last_undegraded': '2023-08-27T11:25:09.044379+0000', 'last_fullsized': '2023-08-27T11:25:09.044379+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:16.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-08-27T11:25:16.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-08-27T11:25:16.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.051943+0000', 'last_change': '2023-08-27T11:25:09.051943+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.051943+0000', 'last_undegraded': '2023-08-27T11:25:09.051943+0000', 'last_fullsized': '2023-08-27T11:25:09.051943+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:25:16.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-08-27T11:25:16.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-08-27T11:25:16.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.042667+0000', 'last_change': '2023-08-27T11:25:09.042667+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.042667+0000', 'last_undegraded': '2023-08-27T11:25:09.042667+0000', 'last_fullsized': '2023-08-27T11:25:09.042667+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:25:16.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-08-27T11:25:16.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-08-27T11:25:16.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.044316+0000', 'last_change': '2023-08-27T11:25:09.044316+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.044316+0000', 'last_undegraded': '2023-08-27T11:25:09.044316+0000', 'last_fullsized': '2023-08-27T11:25:09.044316+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:25:16.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-08-27T11:25:16.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-08-27T11:25:16.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.045853+0000', 'last_change': '2023-08-27T11:25:09.045853+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.045853+0000', 'last_undegraded': '2023-08-27T11:25:09.045853+0000', 'last_fullsized': '2023-08-27T11:25:09.045853+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:25:16.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-08-27T11:25:16.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-27T11:25:10.038390+0000', 'last_change': '2023-08-27T11:25:10.038390+0000', 'last_active': '2023-08-27T11:25:10.038390+0000', 'last_peered': '2023-08-27T11:25:10.038390+0000', 'last_clean': '2023-08-27T11:25:10.038390+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:10.038390+0000', 'last_undegraded': '2023-08-27T11:25:10.038390+0000', 'last_fullsized': '2023-08-27T11:25:10.038390+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:10.038390+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-27T11:25:16.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-08-27T11:25:16.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:25:09.044553+0000', 'last_change': '2023-08-27T11:25:09.044553+0000', 'last_active': '2023-08-27T11:25:09.029612+0000', 'last_peered': '2023-08-27T11:25:09.029612+0000', 'last_clean': '2023-08-27T11:25:09.029612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:25:09.044553+0000', 'last_undegraded': '2023-08-27T11:25:09.044553+0000', 'last_fullsized': '2023-08-27T11:25:09.044553+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'last_clean_scrub_stamp': '2023-08-27T11:25:09.029612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:25:16.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-27T11:25:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.273+0000 7f1313d59700 1 Processor -- start 2023-08-27T11:25:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.273+0000 7f1313d59700 1 -- start start 2023-08-27T11:25:16.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f1313d59700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c15d3d0 0x7f130c1619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f1313d59700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f130c0af040 0x7f130c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f1313d59700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130c161550 con 0x7f130c06e3a0 2023-08-27T11:25:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f1313d59700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130c0cdf90 con 0x7f130c0af040 2023-08-27T11:25:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f1313d59700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c161f20 con 0x7f130c15d3d0 2023-08-27T11:25:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f13112f4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f130c0af040 0x7f130c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f13112f4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f130c0af040 0x7f130c0cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56318/0 (socket says 172.21.15.45:56318) 2023-08-27T11:25:16.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f13112f4700 1 -- 172.21.15.45:0/4033355224 learned_addr learned my addr 172.21.15.45:0/4033355224 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:16.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f1311af5700 1 --2- 172.21.15.45:0/4033355224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c15d3d0 0x7f130c1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f13112f4700 1 -- 172.21.15.45:0/4033355224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c15d3d0 msgr2=0x7f130c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f13112f4700 1 --2- 172.21.15.45:0/4033355224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c15d3d0 0x7f130c1619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f13112f4700 1 -- 172.21.15.45:0/4033355224 >> v1:172.21.15.135:6789/0 conn(0x7f130c06e3a0 legacy=0x7f130c070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.274+0000 7f13112f4700 1 -- 172.21.15.45:0/4033355224 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c0cd9d0 con 0x7f130c0af040 2023-08-27T11:25:16.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f13112f4700 1 --2- 172.21.15.45:0/4033355224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c0af040 0x7f130c0cf420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f13080098f0 tx=0x7f1308009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac840fe272503d1f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1310af3700 1 -- 172.21.15.45:0/4033355224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308011830 con 0x7f130c0af040 2023-08-27T11:25:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1310af3700 1 -- 172.21.15.45:0/4033355224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1308011990 con 0x7f130c0af040 2023-08-27T11:25:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1310af3700 1 -- 172.21.15.45:0/4033355224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308011d20 con 0x7f130c0af040 2023-08-27T11:25:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1313d59700 1 -- 172.21.15.45:0/4033355224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c0af040 msgr2=0x7f130c0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1313d59700 1 --2- 172.21.15.45:0/4033355224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c0af040 0x7f130c0cf420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f13080098f0 tx=0x7f1308009c00 comp rx=0 tx=0).stop 2023-08-27T11:25:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1313d59700 1 -- 172.21.15.45:0/4033355224 shutdown_connections 2023-08-27T11:25:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1313d59700 1 --2- 172.21.15.45:0/4033355224 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c15d3d0 0x7f130c1619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1313d59700 1 --2- 172.21.15.45:0/4033355224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c0af040 0x7f130c0cf420 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1313d59700 1 -- 172.21.15.45:0/4033355224 >> 172.21.15.45:0/4033355224 conn(0x7f130c0cfd60 msgr2=0x7f130c0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1313d59700 1 -- 172.21.15.45:0/4033355224 shutdown_connections 2023-08-27T11:25:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.275+0000 7f1313d59700 1 -- 172.21.15.45:0/4033355224 wait complete. 2023-08-27T11:25:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1313d59700 1 Processor -- start 2023-08-27T11:25:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1313d59700 1 -- start start 2023-08-27T11:25:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1313d59700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c06e3a0 0x7f130c1f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1313d59700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c0af040 0x7f130c0b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1313d59700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c15d3d0 0x7f130c160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1313d59700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0b4490 con 0x7f130c06e3a0 2023-08-27T11:25:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1313d59700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0b2ec0 con 0x7f130c15d3d0 2023-08-27T11:25:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1313d59700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c1f3270 con 0x7f130c0af040 2023-08-27T11:25:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1311af5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c06e3a0 0x7f130c1f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.277+0000 7f1311af5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c06e3a0 0x7f130c1f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56322/0 (socket says 172.21.15.45:56322) 2023-08-27T11:25:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f1311af5700 1 -- 172.21.15.45:0/2663868454 learned_addr learned my addr 172.21.15.45:0/2663868454 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f13112f4700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c0af040 0x7f130c0b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f13122f6700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c15d3d0 0x7f130c160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f1311af5700 1 -- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c0af040 msgr2=0x7f130c0b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f1311af5700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c0af040 0x7f130c0b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f1311af5700 1 -- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c15d3d0 msgr2=0x7f130c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f1311af5700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c15d3d0 0x7f130c160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f1311af5700 1 -- 172.21.15.45:0/2663868454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308009550 con 0x7f130c06e3a0 2023-08-27T11:25:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f1311af5700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c06e3a0 0x7f130c1f6b10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00e9b0 tx=0x7f12fc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f13027fc700 1 -- 172.21.15.45:0/2663868454 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc009710 con 0x7f130c06e3a0 2023-08-27T11:25:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f13027fc700 1 -- 172.21.15.45:0/2663868454 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc00ee50 con 0x7f130c06e3a0 2023-08-27T11:25:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f13027fc700 1 -- 172.21.15.45:0/2663868454 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc00ca50 con 0x7f130c06e3a0 2023-08-27T11:25:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c1f3550 con 0x7f130c06e3a0 2023-08-27T11:25:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.278+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c0b46e0 con 0x7f130c06e3a0 2023-08-27T11:25:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.279+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c0cf900 con 0x7f130c06e3a0 2023-08-27T11:25:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.280+0000 7f13027fc700 1 -- 172.21.15.45:0/2663868454 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f12fc00cbb0 con 0x7f130c06e3a0 2023-08-27T11:25:16.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.280+0000 7f13027fc700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f12f006a200 0x7f12f006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.280+0000 7f13027fc700 1 -- 172.21.15.45:0/2663868454 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f12fc088770 con 0x7f130c06e3a0 2023-08-27T11:25:16.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.280+0000 7f13112f4700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f12f006a200 0x7f12f006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.281+0000 7f13112f4700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f12f006a200 0x7f12f006c6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1308014040 tx=0x7f1308008030 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.285+0000 7f13027fc700 1 -- 172.21.15.45:0/2663868454 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f12fc057700 con 0x7f130c06e3a0 2023-08-27T11:25:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.566+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f130c0b4a50 con 0x7f130c06e3a0 2023-08-27T11:25:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.566+0000 7f13027fc700 1 -- 172.21.15.45:0/2663868454 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1109 (secure 0 0 0) 0x7f12fc00ce60 con 0x7f130c06e3a0 2023-08-27T11:25:16.568 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:16.568 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":813,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":70,"num_osds":8,"num_up_osds":8,"osd_up_since":1693134714,"num_in_osds":8,"osd_in_since":1693134705,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":271507456,"bytes_avail":772822605824,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":6309}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-08-27T11:14:54.080694+0000","services":{}},"progress_events":{}} 2023-08-27T11:25:16.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f12f006a200 msgr2=0x7f12f006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f12f006a200 0x7f12f006c6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1308014040 tx=0x7f1308008030 comp rx=0 tx=0).stop 2023-08-27T11:25:16.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c06e3a0 msgr2=0x7f130c1f6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c06e3a0 0x7f130c1f6b10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00e9b0 tx=0x7f12fc00ecc0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 shutdown_connections 2023-08-27T11:25:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f12f006a200 0x7f12f006c6c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f130c06e3a0 0x7f130c1f6b10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c15d3d0 0x7f130c160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 --2- 172.21.15.45:0/2663868454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130c0af040 0x7f130c0b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 >> 172.21.15.45:0/2663868454 conn(0x7f130c0cfd60 msgr2=0x7f130c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 shutdown_connections 2023-08-27T11:25:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.569+0000 7f1313d59700 1 -- 172.21.15.45:0/2663868454 wait complete. 2023-08-27T11:25:16.582 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-27T11:25:16.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.707+0000 7f45b885f700 1 Processor -- start 2023-08-27T11:25:16.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.707+0000 7f45b885f700 1 -- start start 2023-08-27T11:25:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.708+0000 7f45b885f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f45b415a240 0x7f45b40cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b885f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b40cbe70 0x7f45b40ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b885f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b406d690 con 0x7f45b40ce9d0 2023-08-27T11:25:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b885f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b40cee20 con 0x7f45b415a240 2023-08-27T11:25:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b885f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40cc440 con 0x7f45b40cbe70 2023-08-27T11:25:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b2d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45b40ce9d0 0x7f45b4070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43110/0 (socket says 172.21.15.45:43110) 2023-08-27T11:25:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b2d9d700 1 -- 172.21.15.45:0/4229566693 learned_addr learned my addr 172.21.15.45:0/4229566693 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397219787 0 0) 0x7f45b406d690 con 0x7f45b40ce9d0 2023-08-27T11:25:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b1d9b700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b40cbe70 0x7f45b40ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b259c700 1 --2- 172.21.15.45:0/4229566693 >> v2:172.21.15.135:3300/0 conn(0x7f45b415a240 0x7f45b40cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f459c0035b0 con 0x7f45b40ce9d0 2023-08-27T11:25:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4206077486 0 0) 0x7f459c0035b0 con 0x7f45b40ce9d0 2023-08-27T11:25:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b40cbe70 msgr2=0x7f45b40ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b40cbe70 0x7f45b40ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 >> v2:172.21.15.135:3300/0 conn(0x7f45b415a240 msgr2=0x7f45b40cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 --2- 172.21.15.45:0/4229566693 >> v2:172.21.15.135:3300/0 conn(0x7f45b415a240 0x7f45b40cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b406d040 con 0x7f45b40ce9d0 2023-08-27T11:25:16.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f45ac0019f0 con 0x7f45b40ce9d0 2023-08-27T11:25:16.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 >> v1:172.21.15.45:6789/0 conn(0x7f45b40ce9d0 legacy=0x7f45b4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f459c003a00 0x7f459c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b40cbe70 0x7f459c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f459c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40cee20 con 0x7f459c004d90 2023-08-27T11:25:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40cc440 con 0x7f459c003a00 2023-08-27T11:25:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459c00ba10 con 0x7f45b40cbe70 2023-08-27T11:25:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b2d9d700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f459c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.709+0000 7f45b259c700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f459c003a00 0x7f459c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b1d9b700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b40cbe70 0x7f459c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b2d9d700 1 -- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b40cbe70 msgr2=0x7f459c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b2d9d700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b40cbe70 0x7f459c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b2d9d700 1 -- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f459c003a00 msgr2=0x7f459c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:16.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b2d9d700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f459c003a00 0x7f459c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b2d9d700 1 -- 172.21.15.45:0/4229566693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c0035b0 con 0x7f459c004d90 2023-08-27T11:25:16.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b2d9d700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f459c005210 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00a860 tx=0x7f459c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f947e8dddaf6bc02 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac012600 con 0x7f459c004d90 2023-08-27T11:25:16.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac012780 con 0x7f459c004d90 2023-08-27T11:25:16.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.710+0000 7f45b159a700 1 -- 172.21.15.45:0/4229566693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac012b30 con 0x7f459c004d90 2023-08-27T11:25:16.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.711+0000 7f45b885f700 1 -- 172.21.15.45:0/4229566693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 msgr2=0x7f459c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.711+0000 7f45b885f700 1 --2- 172.21.15.45:0/4229566693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f459c005210 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00a860 tx=0x7f459c005880 comp rx=0 tx=0).stop 2023-08-27T11:25:16.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.711+0000 7f45b2d9d700 1 -- 172.21.15.45:0/4229566693 reap_dead start 2023-08-27T11:25:16.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.711+0000 7f45b885f700 1 -- 172.21.15.45:0/4229566693 shutdown_connections 2023-08-27T11:25:16.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.711+0000 7f45b885f700 1 -- 172.21.15.45:0/4229566693 >> 172.21.15.45:0/4229566693 conn(0x7f45b40cfe50 msgr2=0x7f45b40d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:16.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.711+0000 7f45b885f700 1 -- 172.21.15.45:0/4229566693 shutdown_connections 2023-08-27T11:25:16.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.711+0000 7f45b885f700 1 -- 172.21.15.45:0/4229566693 wait complete. 2023-08-27T11:25:16.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.712+0000 7f45b885f700 1 Processor -- start 2023-08-27T11:25:16.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.712+0000 7f45b885f700 1 -- start start 2023-08-27T11:25:16.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b885f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f45b414d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b885f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b4150190 0x7f45b415c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b885f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45b415ce60 0x7f45b40c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b885f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40c81f0 con 0x7f459c004d90 2023-08-27T11:25:16.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b885f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40c84d0 con 0x7f45b415ce60 2023-08-27T11:25:16.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b885f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40c87b0 con 0x7f45b4150190 2023-08-27T11:25:16.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f45b414d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f45b414d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56342/0 (socket says 172.21.15.45:56342) 2023-08-27T11:25:16.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b259c700 1 -- 172.21.15.45:0/3216903983 learned_addr learned my addr 172.21.15.45:0/3216903983 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:16.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b1d9b700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b4150190 0x7f45b415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b2d9d700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45b415ce60 0x7f45b40c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b259c700 1 -- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b4150190 msgr2=0x7f45b415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b259c700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b4150190 0x7f45b415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b259c700 1 -- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45b415ce60 msgr2=0x7f45b40c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b259c700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45b415ce60 0x7f45b40c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b259c700 1 -- 172.21.15.45:0/3216903983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b415c120 con 0x7f459c004d90 2023-08-27T11:25:16.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b1d9b700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b4150190 0x7f45b415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:16.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.713+0000 7f45b259c700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f45b414d570 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f45a4011bb0 tx=0x7f45a4009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.714+0000 7f45a2ffd700 1 -- 172.21.15.45:0/3216903983 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4016070 con 0x7f459c004d90 2023-08-27T11:25:16.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.714+0000 7f45a2ffd700 1 -- 172.21.15.45:0/3216903983 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a400eaa0 con 0x7f459c004d90 2023-08-27T11:25:16.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.714+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b415c400 con 0x7f459c004d90 2023-08-27T11:25:16.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.714+0000 7f45a2ffd700 1 -- 172.21.15.45:0/3216903983 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400ec50 con 0x7f459c004d90 2023-08-27T11:25:16.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.714+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b40c5810 con 0x7f459c004d90 2023-08-27T11:25:16.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.715+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b4067630 con 0x7f459c004d90 2023-08-27T11:25:16.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.715+0000 7f45a2ffd700 1 -- 172.21.15.45:0/3216903983 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f45a401a410 con 0x7f459c004d90 2023-08-27T11:25:16.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.715+0000 7f45a2ffd700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f459406a200 0x7f459406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:16.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.716+0000 7f45a2ffd700 1 -- 172.21.15.45:0/3216903983 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f45a408a210 con 0x7f459c004d90 2023-08-27T11:25:16.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.716+0000 7f45b1d9b700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f459406a200 0x7f459406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:16.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.716+0000 7f45b1d9b700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f459406a200 0x7f459406c6c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45b40c30f0 tx=0x7f45a8008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:16.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.720+0000 7f45a2ffd700 1 -- 172.21.15.45:0/3216903983 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f45a4059280 con 0x7f459c004d90 2023-08-27T11:25:16.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.916+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 --> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45b40ce1a0 con 0x7f459406a200 2023-08-27T11:25:16.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.925+0000 7f45a2ffd700 1 -- 172.21.15.45:0/3216903983 <== mgr.5402 v2:172.21.15.135:6832/3181978685 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311619 (secure 0 0 0) 0x7f45a8014950 con 0x7f459406a200 2023-08-27T11:25:16.929 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:16.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f459406a200 msgr2=0x7f459406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f459406a200 0x7f459406c6c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45b40c30f0 tx=0x7f45a8008040 comp rx=0 tx=0).stop 2023-08-27T11:25:16.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 msgr2=0x7f45b414d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:16.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f45b414d570 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f45a4011bb0 tx=0x7f45a4009f70 comp rx=0 tx=0).stop 2023-08-27T11:25:16.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 shutdown_connections 2023-08-27T11:25:16.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f459406a200 0x7f459406c6c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f459c004d90 0x7f45b414d570 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45b415ce60 0x7f45b40c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 --2- 172.21.15.45:0/3216903983 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45b4150190 0x7f45b415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:16.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 >> 172.21.15.45:0/3216903983 conn(0x7f45b40cfe50 msgr2=0x7f45b4070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:16.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 shutdown_connections 2023-08-27T11:25:16.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:16.928+0000 7f45b885f700 1 -- 172.21.15.45:0/3216903983 wait complete. 2023-08-27T11:25:16.935 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:25:16.941 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-08-27T11:25:16.322518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":244,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":192,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265568,"kb_used_data":7784,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754709152,"statfs":{"total":773094113280,"available":772822171648,"internally_reserved":0,"allocated":7970816,"data_stored":4638228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.278906"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651999+0000","last_change":"2023-08-27T11:25:11.089737+0000","last_active":"2023-08-27T11:25:12.651999+0000","last_peered":"2023-08-27T11:25:12.651999+0000","last_clean":"2023-08-27T11:25:12.651999+0000","last_became_active":"2023-08-27T11:25:11.088793+0000","last_became_peered":"2023-08-27T11:25:11.088793+0000","last_unstale":"2023-08-27T11:25:12.651999+0000","last_undegraded":"2023-08-27T11:25:12.651999+0000","last_fullsized":"2023-08-27T11:25:12.651999+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:24:46.661058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151313+0000","last_change":"2023-08-27T11:25:11.083482+0000","last_active":"2023-08-27T11:25:12.151313+0000","last_peered":"2023-08-27T11:25:12.151313+0000","last_clean":"2023-08-27T11:25:12.151313+0000","last_became_active":"2023-08-27T11:25:11.083108+0000","last_became_peered":"2023-08-27T11:25:11.083108+0000","last_unstale":"2023-08-27T11:25:12.151313+0000","last_undegraded":"2023-08-27T11:25:12.151313+0000","last_fullsized":"2023-08-27T11:25:12.151313+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:42.181255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652740+0000","last_change":"2023-08-27T11:25:11.074228+0000","last_active":"2023-08-27T11:25:12.652740+0000","last_peered":"2023-08-27T11:25:12.652740+0000","last_clean":"2023-08-27T11:25:12.652740+0000","last_became_active":"2023-08-27T11:25:11.073701+0000","last_became_peered":"2023-08-27T11:25:11.073701+0000","last_unstale":"2023-08-27T11:25:12.652740+0000","last_undegraded":"2023-08-27T11:25:12.652740+0000","last_fullsized":"2023-08-27T11:25:12.652740+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:19:50.055671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076532+0000","last_change":"2023-08-27T11:25:11.069621+0000","last_active":"2023-08-27T11:25:12.076532+0000","last_peered":"2023-08-27T11:25:12.076532+0000","last_clean":"2023-08-27T11:25:12.076532+0000","last_became_active":"2023-08-27T11:25:11.069346+0000","last_became_peered":"2023-08-27T11:25:11.069346+0000","last_unstale":"2023-08-27T11:25:12.076532+0000","last_undegraded":"2023-08-27T11:25:12.076532+0000","last_fullsized":"2023-08-27T11:25:12.076532+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:45.624206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651752+0000","last_change":"2023-08-27T11:25:11.065611+0000","last_active":"2023-08-27T11:25:12.651752+0000","last_peered":"2023-08-27T11:25:12.651752+0000","last_clean":"2023-08-27T11:25:12.651752+0000","last_became_active":"2023-08-27T11:25:11.065315+0000","last_became_peered":"2023-08-27T11:25:11.065315+0000","last_unstale":"2023-08-27T11:25:12.651752+0000","last_undegraded":"2023-08-27T11:25:12.651752+0000","last_fullsized":"2023-08-27T11:25:12.651752+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:53.418624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073327+0000","last_change":"2023-08-27T11:25:11.075312+0000","last_active":"2023-08-27T11:25:12.073327+0000","last_peered":"2023-08-27T11:25:12.073327+0000","last_clean":"2023-08-27T11:25:12.073327+0000","last_became_active":"2023-08-27T11:25:11.075019+0000","last_became_peered":"2023-08-27T11:25:11.075019+0000","last_unstale":"2023-08-27T11:25:12.073327+0000","last_undegraded":"2023-08-27T11:25:12.073327+0000","last_fullsized":"2023-08-27T11:25:12.073327+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:14:28.271506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075021+0000","last_change":"2023-08-27T11:25:11.066375+0000","last_active":"2023-08-27T11:25:12.075021+0000","last_peered":"2023-08-27T11:25:12.075021+0000","last_clean":"2023-08-27T11:25:12.075021+0000","last_became_active":"2023-08-27T11:25:11.066087+0000","last_became_peered":"2023-08-27T11:25:11.066087+0000","last_unstale":"2023-08-27T11:25:12.075021+0000","last_undegraded":"2023-08-27T11:25:12.075021+0000","last_fullsized":"2023-08-27T11:25:12.075021+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:57:52.752858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151448+0000","last_change":"2023-08-27T11:25:11.070672+0000","last_active":"2023-08-27T11:25:12.151448+0000","last_peered":"2023-08-27T11:25:12.151448+0000","last_clean":"2023-08-27T11:25:12.151448+0000","last_became_active":"2023-08-27T11:25:11.070436+0000","last_became_peered":"2023-08-27T11:25:11.070436+0000","last_unstale":"2023-08-27T11:25:12.151448+0000","last_undegraded":"2023-08-27T11:25:12.151448+0000","last_fullsized":"2023-08-27T11:25:12.151448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:14.078958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075087+0000","last_change":"2023-08-27T11:25:11.077334+0000","last_active":"2023-08-27T11:25:12.075087+0000","last_peered":"2023-08-27T11:25:12.075087+0000","last_clean":"2023-08-27T11:25:12.075087+0000","last_became_active":"2023-08-27T11:25:11.077046+0000","last_became_peered":"2023-08-27T11:25:11.077046+0000","last_unstale":"2023-08-27T11:25:12.075087+0000","last_undegraded":"2023-08-27T11:25:12.075087+0000","last_fullsized":"2023-08-27T11:25:12.075087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:05:34.613670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652067+0000","last_change":"2023-08-27T11:25:11.077626+0000","last_active":"2023-08-27T11:25:12.652067+0000","last_peered":"2023-08-27T11:25:12.652067+0000","last_clean":"2023-08-27T11:25:12.652067+0000","last_became_active":"2023-08-27T11:25:11.077324+0000","last_became_peered":"2023-08-27T11:25:11.077324+0000","last_unstale":"2023-08-27T11:25:12.652067+0000","last_undegraded":"2023-08-27T11:25:12.652067+0000","last_fullsized":"2023-08-27T11:25:12.652067+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:52:38.805254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651972+0000","last_change":"2023-08-27T11:25:11.084040+0000","last_active":"2023-08-27T11:25:12.651972+0000","last_peered":"2023-08-27T11:25:12.651972+0000","last_clean":"2023-08-27T11:25:12.651972+0000","last_became_active":"2023-08-27T11:25:11.083745+0000","last_became_peered":"2023-08-27T11:25:11.083745+0000","last_unstale":"2023-08-27T11:25:12.651972+0000","last_undegraded":"2023-08-27T11:25:12.651972+0000","last_fullsized":"2023-08-27T11:25:12.651972+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:48.037337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076680+0000","last_change":"2023-08-27T11:25:11.067645+0000","last_active":"2023-08-27T11:25:12.076680+0000","last_peered":"2023-08-27T11:25:12.076680+0000","last_clean":"2023-08-27T11:25:12.076680+0000","last_became_active":"2023-08-27T11:25:11.067363+0000","last_became_peered":"2023-08-27T11:25:11.067363+0000","last_unstale":"2023-08-27T11:25:12.076680+0000","last_undegraded":"2023-08-27T11:25:12.076680+0000","last_fullsized":"2023-08-27T11:25:12.076680+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:08.394353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652543+0000","last_change":"2023-08-27T11:25:11.083448+0000","last_active":"2023-08-27T11:25:12.652543+0000","last_peered":"2023-08-27T11:25:12.652543+0000","last_clean":"2023-08-27T11:25:12.652543+0000","last_became_active":"2023-08-27T11:25:11.083185+0000","last_became_peered":"2023-08-27T11:25:11.083185+0000","last_unstale":"2023-08-27T11:25:12.652543+0000","last_undegraded":"2023-08-27T11:25:12.652543+0000","last_fullsized":"2023-08-27T11:25:12.652543+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:52:38.805254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074918+0000","last_change":"2023-08-27T11:25:11.064597+0000","last_active":"2023-08-27T11:25:12.074918+0000","last_peered":"2023-08-27T11:25:12.074918+0000","last_clean":"2023-08-27T11:25:12.074918+0000","last_became_active":"2023-08-27T11:25:11.064310+0000","last_became_peered":"2023-08-27T11:25:11.064310+0000","last_unstale":"2023-08-27T11:25:12.074918+0000","last_undegraded":"2023-08-27T11:25:12.074918+0000","last_fullsized":"2023-08-27T11:25:12.074918+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:08.178590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075183+0000","last_change":"2023-08-27T11:25:11.069011+0000","last_active":"2023-08-27T11:25:12.075183+0000","last_peered":"2023-08-27T11:25:12.075183+0000","last_clean":"2023-08-27T11:25:12.075183+0000","last_became_active":"2023-08-27T11:25:11.068211+0000","last_became_peered":"2023-08-27T11:25:11.068211+0000","last_unstale":"2023-08-27T11:25:12.075183+0000","last_undegraded":"2023-08-27T11:25:12.075183+0000","last_fullsized":"2023-08-27T11:25:12.075183+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:11.370163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073026+0000","last_change":"2023-08-27T11:25:11.079492+0000","last_active":"2023-08-27T11:25:12.073026+0000","last_peered":"2023-08-27T11:25:12.073026+0000","last_clean":"2023-08-27T11:25:12.073026+0000","last_became_active":"2023-08-27T11:25:11.079226+0000","last_became_peered":"2023-08-27T11:25:11.079226+0000","last_unstale":"2023-08-27T11:25:12.073026+0000","last_undegraded":"2023-08-27T11:25:12.073026+0000","last_fullsized":"2023-08-27T11:25:12.073026+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:16.558851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151418+0000","last_change":"2023-08-27T11:25:11.082153+0000","last_active":"2023-08-27T11:25:12.151418+0000","last_peered":"2023-08-27T11:25:12.151418+0000","last_clean":"2023-08-27T11:25:12.151418+0000","last_became_active":"2023-08-27T11:25:11.081866+0000","last_became_peered":"2023-08-27T11:25:11.081866+0000","last_unstale":"2023-08-27T11:25:12.151418+0000","last_undegraded":"2023-08-27T11:25:12.151418+0000","last_fullsized":"2023-08-27T11:25:12.151418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:41.183698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652916+0000","last_change":"2023-08-27T11:25:11.078205+0000","last_active":"2023-08-27T11:25:12.652916+0000","last_peered":"2023-08-27T11:25:12.652916+0000","last_clean":"2023-08-27T11:25:12.652916+0000","last_became_active":"2023-08-27T11:25:11.077648+0000","last_became_peered":"2023-08-27T11:25:11.077648+0000","last_unstale":"2023-08-27T11:25:12.652916+0000","last_undegraded":"2023-08-27T11:25:12.652916+0000","last_fullsized":"2023-08-27T11:25:12.652916+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:17:05.972517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652025+0000","last_change":"2023-08-27T11:25:11.089266+0000","last_active":"2023-08-27T11:25:12.652025+0000","last_peered":"2023-08-27T11:25:12.652025+0000","last_clean":"2023-08-27T11:25:12.652025+0000","last_became_active":"2023-08-27T11:25:11.088926+0000","last_became_peered":"2023-08-27T11:25:11.088926+0000","last_unstale":"2023-08-27T11:25:12.652025+0000","last_undegraded":"2023-08-27T11:25:12.652025+0000","last_fullsized":"2023-08-27T11:25:12.652025+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:32:01.255063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151298+0000","last_change":"2023-08-27T11:25:11.086522+0000","last_active":"2023-08-27T11:25:12.151298+0000","last_peered":"2023-08-27T11:25:12.151298+0000","last_clean":"2023-08-27T11:25:12.151298+0000","last_became_active":"2023-08-27T11:25:11.086248+0000","last_became_peered":"2023-08-27T11:25:11.086248+0000","last_unstale":"2023-08-27T11:25:12.151298+0000","last_undegraded":"2023-08-27T11:25:12.151298+0000","last_fullsized":"2023-08-27T11:25:12.151298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:59.088239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652486+0000","last_change":"2023-08-27T11:25:11.077423+0000","last_active":"2023-08-27T11:25:12.652486+0000","last_peered":"2023-08-27T11:25:12.652486+0000","last_clean":"2023-08-27T11:25:12.652486+0000","last_became_active":"2023-08-27T11:25:11.077147+0000","last_became_peered":"2023-08-27T11:25:11.077147+0000","last_unstale":"2023-08-27T11:25:12.652486+0000","last_undegraded":"2023-08-27T11:25:12.652486+0000","last_fullsized":"2023-08-27T11:25:12.652486+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:10:35.747962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151088+0000","last_change":"2023-08-27T11:25:11.080859+0000","last_active":"2023-08-27T11:25:12.151088+0000","last_peered":"2023-08-27T11:25:12.151088+0000","last_clean":"2023-08-27T11:25:12.151088+0000","last_became_active":"2023-08-27T11:25:11.080531+0000","last_became_peered":"2023-08-27T11:25:11.080531+0000","last_unstale":"2023-08-27T11:25:12.151088+0000","last_undegraded":"2023-08-27T11:25:12.151088+0000","last_fullsized":"2023-08-27T11:25:12.151088+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:09.352024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651956+0000","last_change":"2023-08-27T11:25:11.086913+0000","last_active":"2023-08-27T11:25:12.651956+0000","last_peered":"2023-08-27T11:25:12.651956+0000","last_clean":"2023-08-27T11:25:12.651956+0000","last_became_active":"2023-08-27T11:25:11.086626+0000","last_became_peered":"2023-08-27T11:25:11.086626+0000","last_unstale":"2023-08-27T11:25:12.651956+0000","last_undegraded":"2023-08-27T11:25:12.651956+0000","last_fullsized":"2023-08-27T11:25:12.651956+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:08.542814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652635+0000","last_change":"2023-08-27T11:25:11.083359+0000","last_active":"2023-08-27T11:25:12.652635+0000","last_peered":"2023-08-27T11:25:12.652635+0000","last_clean":"2023-08-27T11:25:12.652635+0000","last_became_active":"2023-08-27T11:25:11.083108+0000","last_became_peered":"2023-08-27T11:25:11.083108+0000","last_unstale":"2023-08-27T11:25:12.652635+0000","last_undegraded":"2023-08-27T11:25:12.652635+0000","last_fullsized":"2023-08-27T11:25:12.652635+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:18:08.378262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652252+0000","last_change":"2023-08-27T11:25:11.085107+0000","last_active":"2023-08-27T11:25:12.652252+0000","last_peered":"2023-08-27T11:25:12.652252+0000","last_clean":"2023-08-27T11:25:12.652252+0000","last_became_active":"2023-08-27T11:25:11.076912+0000","last_became_peered":"2023-08-27T11:25:11.076912+0000","last_unstale":"2023-08-27T11:25:12.652252+0000","last_undegraded":"2023-08-27T11:25:12.652252+0000","last_fullsized":"2023-08-27T11:25:12.652252+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:54:55.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076685+0000","last_change":"2023-08-27T11:25:11.065003+0000","last_active":"2023-08-27T11:25:12.076685+0000","last_peered":"2023-08-27T11:25:12.076685+0000","last_clean":"2023-08-27T11:25:12.076685+0000","last_became_active":"2023-08-27T11:25:11.064698+0000","last_became_peered":"2023-08-27T11:25:11.064698+0000","last_unstale":"2023-08-27T11:25:12.076685+0000","last_undegraded":"2023-08-27T11:25:12.076685+0000","last_fullsized":"2023-08-27T11:25:12.076685+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:23:02.308613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073579+0000","last_change":"2023-08-27T11:25:11.083553+0000","last_active":"2023-08-27T11:25:12.073579+0000","last_peered":"2023-08-27T11:25:12.073579+0000","last_clean":"2023-08-27T11:25:12.073579+0000","last_became_active":"2023-08-27T11:25:11.083273+0000","last_became_peered":"2023-08-27T11:25:11.083273+0000","last_unstale":"2023-08-27T11:25:12.073579+0000","last_undegraded":"2023-08-27T11:25:12.073579+0000","last_fullsized":"2023-08-27T11:25:12.073579+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:37:29.281301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651748+0000","last_change":"2023-08-27T11:25:11.071688+0000","last_active":"2023-08-27T11:25:12.651748+0000","last_peered":"2023-08-27T11:25:12.651748+0000","last_clean":"2023-08-27T11:25:12.651748+0000","last_became_active":"2023-08-27T11:25:11.068352+0000","last_became_peered":"2023-08-27T11:25:11.068352+0000","last_unstale":"2023-08-27T11:25:12.651748+0000","last_undegraded":"2023-08-27T11:25:12.651748+0000","last_fullsized":"2023-08-27T11:25:12.651748+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:18:36.173709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151483+0000","last_change":"2023-08-27T11:25:11.078654+0000","last_active":"2023-08-27T11:25:12.151483+0000","last_peered":"2023-08-27T11:25:12.151483+0000","last_clean":"2023-08-27T11:25:12.151483+0000","last_became_active":"2023-08-27T11:25:11.078354+0000","last_became_peered":"2023-08-27T11:25:11.078354+0000","last_unstale":"2023-08-27T11:25:12.151483+0000","last_undegraded":"2023-08-27T11:25:12.151483+0000","last_fullsized":"2023-08-27T11:25:12.151483+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:15.892673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651913+0000","last_change":"2023-08-27T11:25:11.071285+0000","last_active":"2023-08-27T11:25:12.651913+0000","last_peered":"2023-08-27T11:25:12.651913+0000","last_clean":"2023-08-27T11:25:12.651913+0000","last_became_active":"2023-08-27T11:25:11.070956+0000","last_became_peered":"2023-08-27T11:25:11.070956+0000","last_unstale":"2023-08-27T11:25:12.651913+0000","last_undegraded":"2023-08-27T11:25:12.651913+0000","last_fullsized":"2023-08-27T11:25:12.651913+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:53.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651785+0000","last_change":"2023-08-27T11:25:11.077269+0000","last_active":"2023-08-27T11:25:12.651785+0000","last_peered":"2023-08-27T11:25:12.651785+0000","last_clean":"2023-08-27T11:25:12.651785+0000","last_became_active":"2023-08-27T11:25:11.076555+0000","last_became_peered":"2023-08-27T11:25:11.076555+0000","last_unstale":"2023-08-27T11:25:12.651785+0000","last_undegraded":"2023-08-27T11:25:12.651785+0000","last_fullsized":"2023-08-27T11:25:12.651785+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:39:06.180828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073134+0000","last_change":"2023-08-27T11:25:11.090246+0000","last_active":"2023-08-27T11:25:12.073134+0000","last_peered":"2023-08-27T11:25:12.073134+0000","last_clean":"2023-08-27T11:25:12.073134+0000","last_became_active":"2023-08-27T11:25:11.089944+0000","last_became_peered":"2023-08-27T11:25:11.089944+0000","last_unstale":"2023-08-27T11:25:12.073134+0000","last_undegraded":"2023-08-27T11:25:12.073134+0000","last_fullsized":"2023-08-27T11:25:12.073134+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:26:24.657680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073760+0000","last_change":"2023-08-27T11:25:11.075952+0000","last_active":"2023-08-27T11:25:12.073760+0000","last_peered":"2023-08-27T11:25:12.073760+0000","last_clean":"2023-08-27T11:25:12.073760+0000","last_became_active":"2023-08-27T11:25:11.075567+0000","last_became_peered":"2023-08-27T11:25:11.075567+0000","last_unstale":"2023-08-27T11:25:12.073760+0000","last_undegraded":"2023-08-27T11:25:12.073760+0000","last_fullsized":"2023-08-27T11:25:12.073760+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:46:44.428971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"69'4","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074235+0000","last_change":"2023-08-27T11:25:10.059196+0000","last_active":"2023-08-27T11:25:12.074235+0000","last_peered":"2023-08-27T11:25:12.074235+0000","last_clean":"2023-08-27T11:25:12.074235+0000","last_became_active":"2023-08-27T11:25:10.058927+0000","last_became_peered":"2023-08-27T11:25:10.058927+0000","last_unstale":"2023-08-27T11:25:12.074235+0000","last_undegraded":"2023-08-27T11:25:12.074235+0000","last_fullsized":"2023-08-27T11:25:12.074235+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:37.587247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652348+0000","last_change":"2023-08-27T11:25:11.075708+0000","last_active":"2023-08-27T11:25:12.652348+0000","last_peered":"2023-08-27T11:25:12.652348+0000","last_clean":"2023-08-27T11:25:12.652348+0000","last_became_active":"2023-08-27T11:25:11.075407+0000","last_became_peered":"2023-08-27T11:25:11.075407+0000","last_unstale":"2023-08-27T11:25:12.652348+0000","last_undegraded":"2023-08-27T11:25:12.652348+0000","last_fullsized":"2023-08-27T11:25:12.652348+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:19:11.472663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151479+0000","last_change":"2023-08-27T11:25:10.063585+0000","last_active":"2023-08-27T11:25:12.151479+0000","last_peered":"2023-08-27T11:25:12.151479+0000","last_clean":"2023-08-27T11:25:12.151479+0000","last_became_active":"2023-08-27T11:25:10.063302+0000","last_became_peered":"2023-08-27T11:25:10.063302+0000","last_unstale":"2023-08-27T11:25:12.151479+0000","last_undegraded":"2023-08-27T11:25:12.151479+0000","last_fullsized":"2023-08-27T11:25:12.151479+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:07.534036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073466+0000","last_change":"2023-08-27T11:25:11.082620+0000","last_active":"2023-08-27T11:25:12.073466+0000","last_peered":"2023-08-27T11:25:12.073466+0000","last_clean":"2023-08-27T11:25:12.073466+0000","last_became_active":"2023-08-27T11:25:11.082305+0000","last_became_peered":"2023-08-27T11:25:11.082305+0000","last_unstale":"2023-08-27T11:25:12.073466+0000","last_undegraded":"2023-08-27T11:25:12.073466+0000","last_fullsized":"2023-08-27T11:25:12.073466+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:51.549355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073971+0000","last_change":"2023-08-27T11:25:10.062557+0000","last_active":"2023-08-27T11:25:12.073971+0000","last_peered":"2023-08-27T11:25:12.073971+0000","last_clean":"2023-08-27T11:25:12.073971+0000","last_became_active":"2023-08-27T11:25:10.062283+0000","last_became_peered":"2023-08-27T11:25:10.062283+0000","last_unstale":"2023-08-27T11:25:12.073971+0000","last_undegraded":"2023-08-27T11:25:12.073971+0000","last_fullsized":"2023-08-27T11:25:12.073971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:52.060725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073721+0000","last_change":"2023-08-27T11:25:11.081581+0000","last_active":"2023-08-27T11:25:12.073721+0000","last_peered":"2023-08-27T11:25:12.073721+0000","last_clean":"2023-08-27T11:25:12.073721+0000","last_became_active":"2023-08-27T11:25:11.081099+0000","last_became_peered":"2023-08-27T11:25:11.081099+0000","last_unstale":"2023-08-27T11:25:12.073721+0000","last_undegraded":"2023-08-27T11:25:12.073721+0000","last_fullsized":"2023-08-27T11:25:12.073721+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:47.620310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076792+0000","last_change":"2023-08-27T11:25:10.063437+0000","last_active":"2023-08-27T11:25:12.076792+0000","last_peered":"2023-08-27T11:25:12.076792+0000","last_clean":"2023-08-27T11:25:12.076792+0000","last_became_active":"2023-08-27T11:25:10.063203+0000","last_became_peered":"2023-08-27T11:25:10.063203+0000","last_unstale":"2023-08-27T11:25:12.076792+0000","last_undegraded":"2023-08-27T11:25:12.076792+0000","last_fullsized":"2023-08-27T11:25:12.076792+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:24:45.731455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151377+0000","last_change":"2023-08-27T11:25:11.082439+0000","last_active":"2023-08-27T11:25:12.151377+0000","last_peered":"2023-08-27T11:25:12.151377+0000","last_clean":"2023-08-27T11:25:12.151377+0000","last_became_active":"2023-08-27T11:25:11.080131+0000","last_became_peered":"2023-08-27T11:25:11.080131+0000","last_unstale":"2023-08-27T11:25:12.151377+0000","last_undegraded":"2023-08-27T11:25:12.151377+0000","last_fullsized":"2023-08-27T11:25:12.151377+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:23:26.417776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074803+0000","last_change":"2023-08-27T11:25:10.062084+0000","last_active":"2023-08-27T11:25:12.074803+0000","last_peered":"2023-08-27T11:25:12.074803+0000","last_clean":"2023-08-27T11:25:12.074803+0000","last_became_active":"2023-08-27T11:25:10.061782+0000","last_became_peered":"2023-08-27T11:25:10.061782+0000","last_unstale":"2023-08-27T11:25:12.074803+0000","last_undegraded":"2023-08-27T11:25:12.074803+0000","last_fullsized":"2023-08-27T11:25:12.074803+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:37:34.190548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151222+0000","last_change":"2023-08-27T11:25:11.086124+0000","last_active":"2023-08-27T11:25:12.151222+0000","last_peered":"2023-08-27T11:25:12.151222+0000","last_clean":"2023-08-27T11:25:12.151222+0000","last_became_active":"2023-08-27T11:25:11.085932+0000","last_became_peered":"2023-08-27T11:25:11.085932+0000","last_unstale":"2023-08-27T11:25:12.151222+0000","last_undegraded":"2023-08-27T11:25:12.151222+0000","last_fullsized":"2023-08-27T11:25:12.151222+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:52:34.512979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073393+0000","last_change":"2023-08-27T11:25:10.057702+0000","last_active":"2023-08-27T11:25:12.073393+0000","last_peered":"2023-08-27T11:25:12.073393+0000","last_clean":"2023-08-27T11:25:12.073393+0000","last_became_active":"2023-08-27T11:25:10.057347+0000","last_became_peered":"2023-08-27T11:25:10.057347+0000","last_unstale":"2023-08-27T11:25:12.073393+0000","last_undegraded":"2023-08-27T11:25:12.073393+0000","last_fullsized":"2023-08-27T11:25:12.073393+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:17:04.963739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652461+0000","last_change":"2023-08-27T11:25:11.077695+0000","last_active":"2023-08-27T11:25:12.652461+0000","last_peered":"2023-08-27T11:25:12.652461+0000","last_clean":"2023-08-27T11:25:12.652461+0000","last_became_active":"2023-08-27T11:25:11.077412+0000","last_became_peered":"2023-08-27T11:25:11.077412+0000","last_unstale":"2023-08-27T11:25:12.652461+0000","last_undegraded":"2023-08-27T11:25:12.652461+0000","last_fullsized":"2023-08-27T11:25:12.652461+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:48.037337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076378+0000","last_change":"2023-08-27T11:25:10.062767+0000","last_active":"2023-08-27T11:25:12.076378+0000","last_peered":"2023-08-27T11:25:12.076378+0000","last_clean":"2023-08-27T11:25:12.076378+0000","last_became_active":"2023-08-27T11:25:10.062448+0000","last_became_peered":"2023-08-27T11:25:10.062448+0000","last_unstale":"2023-08-27T11:25:12.076378+0000","last_undegraded":"2023-08-27T11:25:12.076378+0000","last_fullsized":"2023-08-27T11:25:12.076378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:48:06.823020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151105+0000","last_change":"2023-08-27T11:25:11.081782+0000","last_active":"2023-08-27T11:25:12.151105+0000","last_peered":"2023-08-27T11:25:12.151105+0000","last_clean":"2023-08-27T11:25:12.151105+0000","last_became_active":"2023-08-27T11:25:11.081479+0000","last_became_peered":"2023-08-27T11:25:11.081479+0000","last_unstale":"2023-08-27T11:25:12.151105+0000","last_undegraded":"2023-08-27T11:25:12.151105+0000","last_fullsized":"2023-08-27T11:25:12.151105+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:15:06.456158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074607+0000","last_change":"2023-08-27T11:25:10.063530+0000","last_active":"2023-08-27T11:25:12.074607+0000","last_peered":"2023-08-27T11:25:12.074607+0000","last_clean":"2023-08-27T11:25:12.074607+0000","last_became_active":"2023-08-27T11:25:10.063284+0000","last_became_peered":"2023-08-27T11:25:10.063284+0000","last_unstale":"2023-08-27T11:25:12.074607+0000","last_undegraded":"2023-08-27T11:25:12.074607+0000","last_fullsized":"2023-08-27T11:25:12.074607+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:27:09.376425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075093+0000","last_change":"2023-08-27T11:25:11.070590+0000","last_active":"2023-08-27T11:25:12.075093+0000","last_peered":"2023-08-27T11:25:12.075093+0000","last_clean":"2023-08-27T11:25:12.075093+0000","last_became_active":"2023-08-27T11:25:11.070325+0000","last_became_peered":"2023-08-27T11:25:11.070325+0000","last_unstale":"2023-08-27T11:25:12.075093+0000","last_undegraded":"2023-08-27T11:25:12.075093+0000","last_fullsized":"2023-08-27T11:25:12.075093+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:57:32.534110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652975+0000","last_change":"2023-08-27T11:25:10.064796+0000","last_active":"2023-08-27T11:25:12.652975+0000","last_peered":"2023-08-27T11:25:12.652975+0000","last_clean":"2023-08-27T11:25:12.652975+0000","last_became_active":"2023-08-27T11:25:10.064460+0000","last_became_peered":"2023-08-27T11:25:10.064460+0000","last_unstale":"2023-08-27T11:25:12.652975+0000","last_undegraded":"2023-08-27T11:25:12.652975+0000","last_fullsized":"2023-08-27T11:25:12.652975+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:37:28.272523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.653014+0000","last_change":"2023-08-27T11:25:11.076982+0000","last_active":"2023-08-27T11:25:12.653014+0000","last_peered":"2023-08-27T11:25:12.653014+0000","last_clean":"2023-08-27T11:25:12.653014+0000","last_became_active":"2023-08-27T11:25:11.076618+0000","last_became_peered":"2023-08-27T11:25:11.076618+0000","last_unstale":"2023-08-27T11:25:12.653014+0000","last_undegraded":"2023-08-27T11:25:12.653014+0000","last_fullsized":"2023-08-27T11:25:12.653014+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:40.388403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151481+0000","last_change":"2023-08-27T11:25:10.060400+0000","last_active":"2023-08-27T11:25:12.151481+0000","last_peered":"2023-08-27T11:25:12.151481+0000","last_clean":"2023-08-27T11:25:12.151481+0000","last_became_active":"2023-08-27T11:25:10.060119+0000","last_became_peered":"2023-08-27T11:25:10.060119+0000","last_unstale":"2023-08-27T11:25:12.151481+0000","last_undegraded":"2023-08-27T11:25:12.151481+0000","last_fullsized":"2023-08-27T11:25:12.151481+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:52.218673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651858+0000","last_change":"2023-08-27T11:25:11.074836+0000","last_active":"2023-08-27T11:25:12.651858+0000","last_peered":"2023-08-27T11:25:12.651858+0000","last_clean":"2023-08-27T11:25:12.651858+0000","last_became_active":"2023-08-27T11:25:11.074522+0000","last_became_peered":"2023-08-27T11:25:11.074522+0000","last_unstale":"2023-08-27T11:25:12.651858+0000","last_undegraded":"2023-08-27T11:25:12.651858+0000","last_fullsized":"2023-08-27T11:25:12.651858+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:25.131425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"69'3","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652848+0000","last_change":"2023-08-27T11:25:10.065878+0000","last_active":"2023-08-27T11:25:12.652848+0000","last_peered":"2023-08-27T11:25:12.652848+0000","last_clean":"2023-08-27T11:25:12.652848+0000","last_became_active":"2023-08-27T11:25:10.065602+0000","last_became_peered":"2023-08-27T11:25:10.065602+0000","last_unstale":"2023-08-27T11:25:12.652848+0000","last_undegraded":"2023-08-27T11:25:12.652848+0000","last_fullsized":"2023-08-27T11:25:12.652848+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:14:27.262728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652941+0000","last_change":"2023-08-27T11:25:11.075893+0000","last_active":"2023-08-27T11:25:12.652941+0000","last_peered":"2023-08-27T11:25:12.652941+0000","last_clean":"2023-08-27T11:25:12.652941+0000","last_became_active":"2023-08-27T11:25:11.075620+0000","last_became_peered":"2023-08-27T11:25:11.075620+0000","last_unstale":"2023-08-27T11:25:12.652941+0000","last_undegraded":"2023-08-27T11:25:12.652941+0000","last_fullsized":"2023-08-27T11:25:12.652941+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:42:13.331122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"69'6","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075245+0000","last_change":"2023-08-27T11:25:10.060915+0000","last_active":"2023-08-27T11:25:12.075245+0000","last_peered":"2023-08-27T11:25:12.075245+0000","last_clean":"2023-08-27T11:25:12.075245+0000","last_became_active":"2023-08-27T11:25:10.060597+0000","last_became_peered":"2023-08-27T11:25:10.060597+0000","last_unstale":"2023-08-27T11:25:12.075245+0000","last_undegraded":"2023-08-27T11:25:12.075245+0000","last_fullsized":"2023-08-27T11:25:12.075245+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:44:44.970472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076277+0000","last_change":"2023-08-27T11:25:11.071783+0000","last_active":"2023-08-27T11:25:12.076277+0000","last_peered":"2023-08-27T11:25:12.076277+0000","last_clean":"2023-08-27T11:25:12.076277+0000","last_became_active":"2023-08-27T11:25:11.070418+0000","last_became_peered":"2023-08-27T11:25:11.070418+0000","last_unstale":"2023-08-27T11:25:12.076277+0000","last_undegraded":"2023-08-27T11:25:12.076277+0000","last_fullsized":"2023-08-27T11:25:12.076277+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:32:25.243371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151652+0000","last_change":"2023-08-27T11:25:10.060471+0000","last_active":"2023-08-27T11:25:12.151652+0000","last_peered":"2023-08-27T11:25:12.151652+0000","last_clean":"2023-08-27T11:25:12.151652+0000","last_became_active":"2023-08-27T11:25:10.060174+0000","last_became_peered":"2023-08-27T11:25:10.060174+0000","last_unstale":"2023-08-27T11:25:12.151652+0000","last_undegraded":"2023-08-27T11:25:12.151652+0000","last_fullsized":"2023-08-27T11:25:12.151652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:46.606725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652556+0000","last_change":"2023-08-27T11:25:11.078419+0000","last_active":"2023-08-27T11:25:12.652556+0000","last_peered":"2023-08-27T11:25:12.652556+0000","last_clean":"2023-08-27T11:25:12.652556+0000","last_became_active":"2023-08-27T11:25:11.078124+0000","last_became_peered":"2023-08-27T11:25:11.078124+0000","last_unstale":"2023-08-27T11:25:12.652556+0000","last_undegraded":"2023-08-27T11:25:12.652556+0000","last_fullsized":"2023-08-27T11:25:12.652556+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:20:42.020468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652350+0000","last_change":"2023-08-27T11:25:10.056373+0000","last_active":"2023-08-27T11:25:12.652350+0000","last_peered":"2023-08-27T11:25:12.652350+0000","last_clean":"2023-08-27T11:25:12.652350+0000","last_became_active":"2023-08-27T11:25:10.056089+0000","last_became_peered":"2023-08-27T11:25:10.056089+0000","last_unstale":"2023-08-27T11:25:12.652350+0000","last_undegraded":"2023-08-27T11:25:12.652350+0000","last_fullsized":"2023-08-27T11:25:12.652350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:24.122647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075035+0000","last_change":"2023-08-27T11:25:11.071794+0000","last_active":"2023-08-27T11:25:12.075035+0000","last_peered":"2023-08-27T11:25:12.075035+0000","last_clean":"2023-08-27T11:25:12.075035+0000","last_became_active":"2023-08-27T11:25:11.071515+0000","last_became_peered":"2023-08-27T11:25:11.071515+0000","last_unstale":"2023-08-27T11:25:12.075035+0000","last_undegraded":"2023-08-27T11:25:12.075035+0000","last_fullsized":"2023-08-27T11:25:12.075035+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:47.623307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076358+0000","last_change":"2023-08-27T11:25:10.063540+0000","last_active":"2023-08-27T11:25:12.076358+0000","last_peered":"2023-08-27T11:25:12.076358+0000","last_clean":"2023-08-27T11:25:12.076358+0000","last_became_active":"2023-08-27T11:25:10.063232+0000","last_became_peered":"2023-08-27T11:25:10.063232+0000","last_unstale":"2023-08-27T11:25:12.076358+0000","last_undegraded":"2023-08-27T11:25:12.076358+0000","last_fullsized":"2023-08-27T11:25:12.076358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:58.185719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652012+0000","last_change":"2023-08-27T11:25:11.087922+0000","last_active":"2023-08-27T11:25:12.652012+0000","last_peered":"2023-08-27T11:25:12.652012+0000","last_clean":"2023-08-27T11:25:12.652012+0000","last_became_active":"2023-08-27T11:25:11.087655+0000","last_became_peered":"2023-08-27T11:25:11.087655+0000","last_unstale":"2023-08-27T11:25:12.652012+0000","last_undegraded":"2023-08-27T11:25:12.652012+0000","last_fullsized":"2023-08-27T11:25:12.652012+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:08.067157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"69'8","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651820+0000","last_change":"2023-08-27T11:25:10.062911+0000","last_active":"2023-08-27T11:25:12.651820+0000","last_peered":"2023-08-27T11:25:12.651820+0000","last_clean":"2023-08-27T11:25:12.651820+0000","last_became_active":"2023-08-27T11:25:10.062611+0000","last_became_peered":"2023-08-27T11:25:10.062611+0000","last_unstale":"2023-08-27T11:25:12.651820+0000","last_undegraded":"2023-08-27T11:25:12.651820+0000","last_fullsized":"2023-08-27T11:25:12.651820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:14.883895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076851+0000","last_change":"2023-08-27T11:25:11.076967+0000","last_active":"2023-08-27T11:25:12.076851+0000","last_peered":"2023-08-27T11:25:12.076851+0000","last_clean":"2023-08-27T11:25:12.076851+0000","last_became_active":"2023-08-27T11:25:11.076687+0000","last_became_peered":"2023-08-27T11:25:11.076687+0000","last_unstale":"2023-08-27T11:25:12.076851+0000","last_undegraded":"2023-08-27T11:25:12.076851+0000","last_fullsized":"2023-08-27T11:25:12.076851+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:22.631584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652299+0000","last_change":"2023-08-27T11:25:10.059439+0000","last_active":"2023-08-27T11:25:12.652299+0000","last_peered":"2023-08-27T11:25:12.652299+0000","last_clean":"2023-08-27T11:25:12.652299+0000","last_became_active":"2023-08-27T11:25:10.059056+0000","last_became_peered":"2023-08-27T11:25:10.059056+0000","last_unstale":"2023-08-27T11:25:12.652299+0000","last_undegraded":"2023-08-27T11:25:12.652299+0000","last_fullsized":"2023-08-27T11:25:12.652299+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:34.944067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076823+0000","last_change":"2023-08-27T11:25:11.072286+0000","last_active":"2023-08-27T11:25:12.076823+0000","last_peered":"2023-08-27T11:25:12.076823+0000","last_clean":"2023-08-27T11:25:12.076823+0000","last_became_active":"2023-08-27T11:25:11.072002+0000","last_became_peered":"2023-08-27T11:25:11.072002+0000","last_unstale":"2023-08-27T11:25:12.076823+0000","last_undegraded":"2023-08-27T11:25:12.076823+0000","last_fullsized":"2023-08-27T11:25:12.076823+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:02:09.202868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074030+0000","last_change":"2023-08-27T11:25:10.060251+0000","last_active":"2023-08-27T11:25:12.074030+0000","last_peered":"2023-08-27T11:25:12.074030+0000","last_clean":"2023-08-27T11:25:12.074030+0000","last_became_active":"2023-08-27T11:25:10.059911+0000","last_became_peered":"2023-08-27T11:25:10.059911+0000","last_unstale":"2023-08-27T11:25:12.074030+0000","last_undegraded":"2023-08-27T11:25:12.074030+0000","last_fullsized":"2023-08-27T11:25:12.074030+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:39:54.589392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652007+0000","last_change":"2023-08-27T11:25:11.074463+0000","last_active":"2023-08-27T11:25:12.652007+0000","last_peered":"2023-08-27T11:25:12.652007+0000","last_clean":"2023-08-27T11:25:12.652007+0000","last_became_active":"2023-08-27T11:25:11.074121+0000","last_became_peered":"2023-08-27T11:25:11.074121+0000","last_unstale":"2023-08-27T11:25:12.652007+0000","last_undegraded":"2023-08-27T11:25:12.652007+0000","last_fullsized":"2023-08-27T11:25:12.652007+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:21.311911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652183+0000","last_change":"2023-08-27T11:25:10.052679+0000","last_active":"2023-08-27T11:25:12.652183+0000","last_peered":"2023-08-27T11:25:12.652183+0000","last_clean":"2023-08-27T11:25:12.652183+0000","last_became_active":"2023-08-27T11:25:10.052343+0000","last_became_peered":"2023-08-27T11:25:10.052343+0000","last_unstale":"2023-08-27T11:25:12.652183+0000","last_undegraded":"2023-08-27T11:25:12.652183+0000","last_fullsized":"2023-08-27T11:25:12.652183+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:20.770727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073872+0000","last_change":"2023-08-27T11:25:11.081384+0000","last_active":"2023-08-27T11:25:12.073872+0000","last_peered":"2023-08-27T11:25:12.073872+0000","last_clean":"2023-08-27T11:25:12.073872+0000","last_became_active":"2023-08-27T11:25:11.080789+0000","last_became_peered":"2023-08-27T11:25:11.080789+0000","last_unstale":"2023-08-27T11:25:12.073872+0000","last_undegraded":"2023-08-27T11:25:12.073872+0000","last_fullsized":"2023-08-27T11:25:12.073872+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:28.594935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076764+0000","last_change":"2023-08-27T11:25:10.056377+0000","last_active":"2023-08-27T11:25:12.076764+0000","last_peered":"2023-08-27T11:25:12.076764+0000","last_clean":"2023-08-27T11:25:12.076764+0000","last_became_active":"2023-08-27T11:25:10.056122+0000","last_became_peered":"2023-08-27T11:25:10.056122+0000","last_unstale":"2023-08-27T11:25:12.076764+0000","last_undegraded":"2023-08-27T11:25:12.076764+0000","last_fullsized":"2023-08-27T11:25:12.076764+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:04:23.170480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151249+0000","last_change":"2023-08-27T11:25:11.078397+0000","last_active":"2023-08-27T11:25:12.151249+0000","last_peered":"2023-08-27T11:25:12.151249+0000","last_clean":"2023-08-27T11:25:12.151249+0000","last_became_active":"2023-08-27T11:25:11.074097+0000","last_became_peered":"2023-08-27T11:25:11.074097+0000","last_unstale":"2023-08-27T11:25:12.151249+0000","last_undegraded":"2023-08-27T11:25:12.151249+0000","last_fullsized":"2023-08-27T11:25:12.151249+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:51:03.922037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651543+0000","last_change":"2023-08-27T11:25:10.058064+0000","last_active":"2023-08-27T11:25:12.651543+0000","last_peered":"2023-08-27T11:25:12.651543+0000","last_clean":"2023-08-27T11:25:12.651543+0000","last_became_active":"2023-08-27T11:25:10.057743+0000","last_became_peered":"2023-08-27T11:25:10.057743+0000","last_unstale":"2023-08-27T11:25:12.651543+0000","last_undegraded":"2023-08-27T11:25:12.651543+0000","last_fullsized":"2023-08-27T11:25:12.651543+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:35:46.801723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652636+0000","last_change":"2023-08-27T11:25:11.090519+0000","last_active":"2023-08-27T11:25:12.652636+0000","last_peered":"2023-08-27T11:25:12.652636+0000","last_clean":"2023-08-27T11:25:12.652636+0000","last_became_active":"2023-08-27T11:25:11.090200+0000","last_became_peered":"2023-08-27T11:25:11.090200+0000","last_unstale":"2023-08-27T11:25:12.652636+0000","last_undegraded":"2023-08-27T11:25:12.652636+0000","last_fullsized":"2023-08-27T11:25:12.652636+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:16.558851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074733+0000","last_change":"2023-08-27T11:25:10.060787+0000","last_active":"2023-08-27T11:25:12.074733+0000","last_peered":"2023-08-27T11:25:12.074733+0000","last_clean":"2023-08-27T11:25:12.074733+0000","last_became_active":"2023-08-27T11:25:10.060416+0000","last_became_peered":"2023-08-27T11:25:10.060416+0000","last_unstale":"2023-08-27T11:25:12.074733+0000","last_undegraded":"2023-08-27T11:25:12.074733+0000","last_fullsized":"2023-08-27T11:25:12.074733+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:16.158243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652251+0000","last_change":"2023-08-27T11:25:11.079350+0000","last_active":"2023-08-27T11:25:12.652251+0000","last_peered":"2023-08-27T11:25:12.652251+0000","last_clean":"2023-08-27T11:25:12.652251+0000","last_became_active":"2023-08-27T11:25:11.079059+0000","last_became_peered":"2023-08-27T11:25:11.079059+0000","last_unstale":"2023-08-27T11:25:12.652251+0000","last_undegraded":"2023-08-27T11:25:12.652251+0000","last_fullsized":"2023-08-27T11:25:12.652251+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:00:23.873229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651588+0000","last_change":"2023-08-27T11:25:10.061300+0000","last_active":"2023-08-27T11:25:12.651588+0000","last_peered":"2023-08-27T11:25:12.651588+0000","last_clean":"2023-08-27T11:25:12.651588+0000","last_became_active":"2023-08-27T11:25:10.061015+0000","last_became_peered":"2023-08-27T11:25:10.061015+0000","last_unstale":"2023-08-27T11:25:12.651588+0000","last_undegraded":"2023-08-27T11:25:12.651588+0000","last_fullsized":"2023-08-27T11:25:12.651588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:20.770727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151136+0000","last_change":"2023-08-27T11:25:11.081511+0000","last_active":"2023-08-27T11:25:12.151136+0000","last_peered":"2023-08-27T11:25:12.151136+0000","last_clean":"2023-08-27T11:25:12.151136+0000","last_became_active":"2023-08-27T11:25:11.081124+0000","last_became_peered":"2023-08-27T11:25:11.081124+0000","last_unstale":"2023-08-27T11:25:12.151136+0000","last_undegraded":"2023-08-27T11:25:12.151136+0000","last_fullsized":"2023-08-27T11:25:12.151136+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:21.207590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151336+0000","last_change":"2023-08-27T11:25:10.058695+0000","last_active":"2023-08-27T11:25:12.151336+0000","last_peered":"2023-08-27T11:25:12.151336+0000","last_clean":"2023-08-27T11:25:12.151336+0000","last_became_active":"2023-08-27T11:25:10.058370+0000","last_became_peered":"2023-08-27T11:25:10.058370+0000","last_unstale":"2023-08-27T11:25:12.151336+0000","last_undegraded":"2023-08-27T11:25:12.151336+0000","last_fullsized":"2023-08-27T11:25:12.151336+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:02:59.411254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076966+0000","last_change":"2023-08-27T11:25:11.074442+0000","last_active":"2023-08-27T11:25:12.076966+0000","last_peered":"2023-08-27T11:25:12.076966+0000","last_clean":"2023-08-27T11:25:12.076966+0000","last_became_active":"2023-08-27T11:25:11.074131+0000","last_became_peered":"2023-08-27T11:25:11.074131+0000","last_unstale":"2023-08-27T11:25:12.076966+0000","last_undegraded":"2023-08-27T11:25:12.076966+0000","last_fullsized":"2023-08-27T11:25:12.076966+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:18:24.128262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073847+0000","last_change":"2023-08-27T11:25:10.061400+0000","last_active":"2023-08-27T11:25:12.073847+0000","last_peered":"2023-08-27T11:25:12.073847+0000","last_clean":"2023-08-27T11:25:12.073847+0000","last_became_active":"2023-08-27T11:25:10.061105+0000","last_became_peered":"2023-08-27T11:25:10.061105+0000","last_unstale":"2023-08-27T11:25:12.073847+0000","last_undegraded":"2023-08-27T11:25:12.073847+0000","last_fullsized":"2023-08-27T11:25:12.073847+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:02:43.029300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":115,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652090+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:25:12.652090+0000","last_peered":"2023-08-27T11:25:12.652090+0000","last_clean":"2023-08-27T11:25:12.652090+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:25:12.652090+0000","last_undegraded":"2023-08-27T11:25:12.652090+0000","last_fullsized":"2023-08-27T11:25:12.652090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:05:00.194138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073245+0000","last_change":"2023-08-27T11:25:11.075517+0000","last_active":"2023-08-27T11:25:12.073245+0000","last_peered":"2023-08-27T11:25:12.073245+0000","last_clean":"2023-08-27T11:25:12.073245+0000","last_became_active":"2023-08-27T11:25:11.075044+0000","last_became_peered":"2023-08-27T11:25:11.075044+0000","last_unstale":"2023-08-27T11:25:12.073245+0000","last_undegraded":"2023-08-27T11:25:12.073245+0000","last_fullsized":"2023-08-27T11:25:12.073245+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:19:50.055671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076223+0000","last_change":"2023-08-27T11:25:10.060606+0000","last_active":"2023-08-27T11:25:12.076223+0000","last_peered":"2023-08-27T11:25:12.076223+0000","last_clean":"2023-08-27T11:25:12.076223+0000","last_became_active":"2023-08-27T11:25:10.060341+0000","last_became_peered":"2023-08-27T11:25:10.060341+0000","last_unstale":"2023-08-27T11:25:12.076223+0000","last_undegraded":"2023-08-27T11:25:12.076223+0000","last_fullsized":"2023-08-27T11:25:12.076223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:21:51.523703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":113,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076419+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:25:12.076419+0000","last_peered":"2023-08-27T11:25:12.076419+0000","last_clean":"2023-08-27T11:25:12.076419+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:25:12.076419+0000","last_undegraded":"2023-08-27T11:25:12.076419+0000","last_fullsized":"2023-08-27T11:25:12.076419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:40:01.268610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076649+0000","last_change":"2023-08-27T11:25:11.070397+0000","last_active":"2023-08-27T11:25:12.076649+0000","last_peered":"2023-08-27T11:25:12.076649+0000","last_clean":"2023-08-27T11:25:12.076649+0000","last_became_active":"2023-08-27T11:25:11.070109+0000","last_became_peered":"2023-08-27T11:25:11.070109+0000","last_unstale":"2023-08-27T11:25:12.076649+0000","last_undegraded":"2023-08-27T11:25:12.076649+0000","last_fullsized":"2023-08-27T11:25:12.076649+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:14:46.794884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151579+0000","last_change":"2023-08-27T11:25:10.061271+0000","last_active":"2023-08-27T11:25:12.151579+0000","last_peered":"2023-08-27T11:25:12.151579+0000","last_clean":"2023-08-27T11:25:12.151579+0000","last_became_active":"2023-08-27T11:25:10.060546+0000","last_became_peered":"2023-08-27T11:25:10.060546+0000","last_unstale":"2023-08-27T11:25:12.151579+0000","last_undegraded":"2023-08-27T11:25:12.151579+0000","last_fullsized":"2023-08-27T11:25:12.151579+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:36:41.413862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651396+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:25:12.651396+0000","last_peered":"2023-08-27T11:25:12.651396+0000","last_clean":"2023-08-27T11:25:12.651396+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:25:12.651396+0000","last_undegraded":"2023-08-27T11:25:12.651396+0000","last_fullsized":"2023-08-27T11:25:12.651396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:50.524608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151230+0000","last_change":"2023-08-27T11:25:11.081351+0000","last_active":"2023-08-27T11:25:12.151230+0000","last_peered":"2023-08-27T11:25:12.151230+0000","last_clean":"2023-08-27T11:25:12.151230+0000","last_became_active":"2023-08-27T11:25:11.080968+0000","last_became_peered":"2023-08-27T11:25:11.080968+0000","last_unstale":"2023-08-27T11:25:12.151230+0000","last_undegraded":"2023-08-27T11:25:12.151230+0000","last_fullsized":"2023-08-27T11:25:12.151230+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:40:04.596358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652392+0000","last_change":"2023-08-27T11:25:10.063913+0000","last_active":"2023-08-27T11:25:12.652392+0000","last_peered":"2023-08-27T11:25:12.652392+0000","last_clean":"2023-08-27T11:25:12.652392+0000","last_became_active":"2023-08-27T11:25:10.063642+0000","last_became_peered":"2023-08-27T11:25:10.063642+0000","last_unstale":"2023-08-27T11:25:12.652392+0000","last_undegraded":"2023-08-27T11:25:12.652392+0000","last_fullsized":"2023-08-27T11:25:12.652392+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:31:01.736095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":113,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076601+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:25:12.076601+0000","last_peered":"2023-08-27T11:25:12.076601+0000","last_clean":"2023-08-27T11:25:12.076601+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:25:12.076601+0000","last_undegraded":"2023-08-27T11:25:12.076601+0000","last_fullsized":"2023-08-27T11:25:12.076601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:41.635538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":347,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651844+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:25:12.651844+0000","last_peered":"2023-08-27T11:25:12.651844+0000","last_clean":"2023-08-27T11:25:12.651844+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:25:12.651844+0000","last_undegraded":"2023-08-27T11:25:12.651844+0000","last_fullsized":"2023-08-27T11:25:12.651844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:41:32.788138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073256+0000","last_change":"2023-08-27T11:25:11.078879+0000","last_active":"2023-08-27T11:25:12.073256+0000","last_peered":"2023-08-27T11:25:12.073256+0000","last_clean":"2023-08-27T11:25:12.073256+0000","last_became_active":"2023-08-27T11:25:11.078582+0000","last_became_peered":"2023-08-27T11:25:11.078582+0000","last_unstale":"2023-08-27T11:25:12.073256+0000","last_undegraded":"2023-08-27T11:25:12.073256+0000","last_fullsized":"2023-08-27T11:25:12.073256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:42:13.331122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651858+0000","last_change":"2023-08-27T11:25:10.060723+0000","last_active":"2023-08-27T11:25:12.651858+0000","last_peered":"2023-08-27T11:25:12.651858+0000","last_clean":"2023-08-27T11:25:12.651858+0000","last_became_active":"2023-08-27T11:25:10.060423+0000","last_became_peered":"2023-08-27T11:25:10.060423+0000","last_unstale":"2023-08-27T11:25:12.651858+0000","last_undegraded":"2023-08-27T11:25:12.651858+0000","last_fullsized":"2023-08-27T11:25:12.651858+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:31:01.736095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":113,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652001+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:25:12.652001+0000","last_peered":"2023-08-27T11:25:12.652001+0000","last_clean":"2023-08-27T11:25:12.652001+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:25:12.652001+0000","last_undegraded":"2023-08-27T11:25:12.652001+0000","last_fullsized":"2023-08-27T11:25:12.652001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:23:34.238516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652974+0000","last_change":"2023-08-27T11:25:11.073477+0000","last_active":"2023-08-27T11:25:12.652974+0000","last_peered":"2023-08-27T11:25:12.652974+0000","last_clean":"2023-08-27T11:25:12.652974+0000","last_became_active":"2023-08-27T11:25:11.073225+0000","last_became_peered":"2023-08-27T11:25:11.073225+0000","last_unstale":"2023-08-27T11:25:12.652974+0000","last_undegraded":"2023-08-27T11:25:12.652974+0000","last_fullsized":"2023-08-27T11:25:12.652974+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:51.549355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151489+0000","last_change":"2023-08-27T11:25:10.060521+0000","last_active":"2023-08-27T11:25:12.151489+0000","last_peered":"2023-08-27T11:25:12.151489+0000","last_clean":"2023-08-27T11:25:12.151489+0000","last_became_active":"2023-08-27T11:25:10.060236+0000","last_became_peered":"2023-08-27T11:25:10.060236+0000","last_unstale":"2023-08-27T11:25:12.151489+0000","last_undegraded":"2023-08-27T11:25:12.151489+0000","last_fullsized":"2023-08-27T11:25:12.151489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:07.058379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":113,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.653026+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:25:12.653026+0000","last_peered":"2023-08-27T11:25:12.653026+0000","last_clean":"2023-08-27T11:25:12.653026+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:25:12.653026+0000","last_undegraded":"2023-08-27T11:25:12.653026+0000","last_fullsized":"2023-08-27T11:25:12.653026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:39.436186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652730+0000","last_change":"2023-08-27T11:25:11.074279+0000","last_active":"2023-08-27T11:25:12.652730+0000","last_peered":"2023-08-27T11:25:12.652730+0000","last_clean":"2023-08-27T11:25:12.652730+0000","last_became_active":"2023-08-27T11:25:11.073985+0000","last_became_peered":"2023-08-27T11:25:11.073985+0000","last_unstale":"2023-08-27T11:25:12.652730+0000","last_undegraded":"2023-08-27T11:25:12.652730+0000","last_fullsized":"2023-08-27T11:25:12.652730+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:26:24.657680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073498+0000","last_change":"2023-08-27T11:25:10.066265+0000","last_active":"2023-08-27T11:25:12.073498+0000","last_peered":"2023-08-27T11:25:12.073498+0000","last_clean":"2023-08-27T11:25:12.073498+0000","last_became_active":"2023-08-27T11:25:10.065959+0000","last_became_peered":"2023-08-27T11:25:10.065959+0000","last_unstale":"2023-08-27T11:25:12.073498+0000","last_undegraded":"2023-08-27T11:25:12.073498+0000","last_fullsized":"2023-08-27T11:25:12.073498+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:39.379625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":115,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652123+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:25:12.652123+0000","last_peered":"2023-08-27T11:25:12.652123+0000","last_clean":"2023-08-27T11:25:12.652123+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:25:12.652123+0000","last_undegraded":"2023-08-27T11:25:12.652123+0000","last_fullsized":"2023-08-27T11:25:12.652123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:58.020795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151399+0000","last_change":"2023-08-27T11:25:11.071240+0000","last_active":"2023-08-27T11:25:12.151399+0000","last_peered":"2023-08-27T11:25:12.151399+0000","last_clean":"2023-08-27T11:25:12.151399+0000","last_became_active":"2023-08-27T11:25:11.070969+0000","last_became_peered":"2023-08-27T11:25:11.070969+0000","last_unstale":"2023-08-27T11:25:12.151399+0000","last_undegraded":"2023-08-27T11:25:12.151399+0000","last_fullsized":"2023-08-27T11:25:12.151399+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:18.122114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076584+0000","last_change":"2023-08-27T11:25:10.064086+0000","last_active":"2023-08-27T11:25:12.076584+0000","last_peered":"2023-08-27T11:25:12.076584+0000","last_clean":"2023-08-27T11:25:12.076584+0000","last_became_active":"2023-08-27T11:25:10.063790+0000","last_became_peered":"2023-08-27T11:25:10.063790+0000","last_unstale":"2023-08-27T11:25:12.076584+0000","last_undegraded":"2023-08-27T11:25:12.076584+0000","last_fullsized":"2023-08-27T11:25:12.076584+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:48.474309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":107,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652382+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:25:12.652382+0000","last_peered":"2023-08-27T11:25:12.652382+0000","last_clean":"2023-08-27T11:25:12.652382+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:25:12.652382+0000","last_undegraded":"2023-08-27T11:25:12.652382+0000","last_fullsized":"2023-08-27T11:25:12.652382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:36:13.127787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":1022112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.875,"15min":0.875},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.55900000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.314},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.48299999999999998,"5min":0.35199999999999998,"15min":0.252},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.439,"5min":0.439,"15min":0.29099999999999998},"max":{"1min":0.79100000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65600000000000003,"15min":0.61599999999999999},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65700000000000003,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.39900000000000002},"max":{"1min":0.82199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56899999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.26500000000000001,"15min":0.23499999999999999},"max":{"1min":0.65400000000000003,"5min":0.67500000000000004,"15min":0.72499999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.52900000000000003},"min":{"1min":0.39100000000000001,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.66700000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.74199999999999999,"15min":0.84699999999999998},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.30099999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.90100000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":431584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.59699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.377,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":1.3540000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":3.3780000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.63900000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.71199999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":9.3770000000000007},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.746,"5min":0.82799999999999996,"15min":1.325},"last":0.55900000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60399999999999998,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34100000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.84899999999999998,"15min":1.395},"last":0.433}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.25900000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34300000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68500000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53300000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.5},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.82099999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.32900000000000001},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72299999999999998,"5min":0.76100000000000001,"15min":0.88900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.34899999999999998},"max":{"1min":0.76400000000000001,"5min":0.77400000000000002,"15min":0.86299999999999999},"last":0.621}]}]},{"osd":1,"up_from":13,"seq":55834575010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40324,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331516,"statfs":{"total":96636764160,"available":96595472384,"internally_reserved":0,"allocated":843776,"data_stored":431674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.20399999999999999},"max":{"1min":0.52700000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.60699999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.313}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.52700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":0.68100000000000005,"5min":0.68300000000000005,"15min":0.78600000000000003},"last":0.33600000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.32500000000000001,"5min":0.311,"15min":0.27600000000000002},"max":{"1min":0.61899999999999999,"5min":0.70399999999999996,"15min":0.77100000000000002},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45700000000000002,"15min":0.434},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.23400000000000001},"max":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.80000000000000004},"last":0.35699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.51700000000000002,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.97999999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.94199999999999995,"15min":2.9380000000000002},"last":0.51700000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.42099999999999999},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97999999999999998,"15min":2.9740000000000002},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.54700000000000004,"5min":0.47399999999999998,"15min":0.42499999999999999},"max":{"1min":0.85499999999999998,"5min":1.002,"15min":1.139},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.876,"5min":0.91600000000000004,"15min":1.0169999999999999},"last":0.57899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":1024146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41999999999999998,"15min":0.40600000000000003},"min":{"1min":0.28199999999999997,"5min":0.27200000000000002,"15min":0.23699999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.90200000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.47499999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":1.2889999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42899999999999999,"15min":0.41399999999999998},"min":{"1min":0.317,"5min":0.313,"15min":0.22900000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":1.3340000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.505},"min":{"1min":0.30199999999999999,"5min":0.29099999999999998,"15min":0.255},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":8.1349999999999998},"last":0.314}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.56399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.23899999999999999},"max":{"1min":0.56899999999999995,"5min":0.69899999999999995,"15min":11.026999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.29699999999999999},"max":{"1min":0.63800000000000001,"5min":0.73299999999999998,"15min":11.926},"last":0.52000000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66500000000000004,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":1.0069999999999999,"15min":10.964},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":1.3640000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.73699999999999999},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":8.0779999999999994},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70199999999999996,"15min":0.81000000000000005},"min":{"1min":0.53300000000000003,"5min":0.42799999999999999,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1,"15min":10.922000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.35699999999999998},"max":{"1min":0.82499999999999996,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.51400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":8.0380000000000003},"last":0.71099999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.78500000000000003},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":8.1069999999999993},"last":0.627},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":1.036,"15min":10.996},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":847872,"data_stored":431640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38500000000000001,"15min":0.372},"min":{"1min":0.25800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.65400000000000003},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.437,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.20799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.35999999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.753},"last":0.307},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46000000000000002,"15min":0.443},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.24099999999999999},"max":{"1min":0.71299999999999997,"5min":0.71699999999999997,"15min":0.77800000000000002},"last":0.40600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48099999999999998,"15min":0.46200000000000002},"min":{"1min":0.34599999999999997,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.29299999999999998},"max":{"1min":0.61899999999999999,"5min":0.65300000000000002,"15min":0.69099999999999995},"last":0.33900000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76500000000000001,"5min":1.163,"15min":1.163},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0660000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95199999999999996,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.55000000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68100000000000005,"15min":0.68000000000000005},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.1319999999999999,"15min":1.2050000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0189999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.499,"15min":0.40100000000000002},"max":{"1min":1.0409999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45700000000000002,"15min":0.42299999999999999},"max":{"1min":0.96999999999999997,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":431640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40999999999999998,"15min":0.52300000000000002},"min":{"1min":0.26800000000000002,"5min":0.23899999999999999,"15min":0.22},"max":{"1min":0.53800000000000003,"5min":0.63300000000000001,"15min":11.885999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44500000000000001,"15min":0.40000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.224},"max":{"1min":0.59899999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.27100000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.32500000000000001,"5min":0.29299999999999998,"15min":0.255},"max":{"1min":0.57399999999999995,"5min":0.67900000000000005,"15min":6.0830000000000002},"last":0.498},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.503,"15min":0.60599999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.245},"max":{"1min":0.68899999999999995,"5min":0.80700000000000005,"15min":12.050000000000001},"last":0.29999999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.504,"15min":0.45800000000000002},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.27100000000000002},"max":{"1min":0.66700000000000004,"5min":0.93100000000000005,"15min":1.25},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54200000000000004,"15min":0.63800000000000001},"min":{"1min":0.376,"5min":0.374,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.83999999999999997,"15min":11.939},"last":0.47499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.76900000000000002},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.376},"max":{"1min":0.86099999999999999,"5min":1.0329999999999999,"15min":11.007},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70199999999999996,"15min":0.65400000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":0.91000000000000003,"5min":1.113,"15min":1.2929999999999999},"last":0.60199999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70499999999999996,"15min":0.81200000000000006},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.436},"max":{"1min":0.92500000000000004,"5min":1.0800000000000001,"15min":11.981},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.79300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.218,"15min":11.09},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72799999999999998,"15min":0.91400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.38200000000000001},"max":{"1min":0.88900000000000001,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.748},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75800000000000001,"15min":0.85499999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.47799999999999998},"max":{"1min":0.95899999999999996,"5min":1.0389999999999999,"15min":12.016},"last":0.78100000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.80900000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.41599999999999998},"max":{"1min":0.91100000000000003,"5min":1.006,"15min":11.054},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.69299999999999995},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":1.0640000000000001,"15min":1.361},"last":0.64900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575010,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":433196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.23499999999999999},"max":{"1min":0.71099999999999997,"5min":0.81799999999999995,"15min":0.89800000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.60399999999999998},"min":{"1min":0.501,"5min":0.40699999999999997,"15min":0.255},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.873},"last":0.56899999999999995}]},{"osd":1,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.622,"15min":0.59699999999999998},"min":{"1min":0.51800000000000002,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82999999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.747,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.312},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.629},"min":{"1min":0.52100000000000002,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.503,"5min":0.40200000000000002,"15min":0.30099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.49399999999999999,"5min":0.371,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.88800000000000001},"last":0.41099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.497,"15min":0.51600000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.26200000000000001},"max":{"1min":0.72299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70899999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.27800000000000002},"max":{"1min":0.72199999999999998,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.441},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.93899999999999995},"last":0.72399999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.874,"15min":0.878},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.496}]}]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36176,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335664,"statfs":{"total":96636764160,"available":96599719936,"internally_reserved":0,"allocated":847872,"data_stored":432236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.93300000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.26200000000000001},"max":{"1min":0.749,"5min":0.84999999999999998,"15min":0.88100000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.248},"max":{"1min":0.80400000000000005,"5min":0.81899999999999995,"15min":0.96999999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":0.77400000000000002,"15min":0.82299999999999995},"last":0.50600000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.33400000000000002,"15min":0.32800000000000001},"max":{"1min":0.75,"5min":0.87,"15min":0.87},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":1.0109999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.80600000000000005,"15min":0.94199999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.34799999999999998,"15min":0.34000000000000002},"max":{"1min":0.79400000000000004,"5min":0.79500000000000004,"15min":0.83799999999999997},"last":0.56200000000000006}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71299999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.33300000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47999999999999998,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.89000000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.379,"15min":0.379},"max":{"1min":0.75,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45100000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.88100000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.84999999999999998},"last":0.42899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:19.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:25:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.088+0000 7f1ebd8bd700 1 Processor -- start 2023-08-27T11:25:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.088+0000 7f1ebd8bd700 1 -- start start 2023-08-27T11:25:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1ebd8bd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 0x7f1eb80cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1ebd8bd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb815d6a0 con 0x7f1eb815d2c0 2023-08-27T11:25:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1ebd8bd700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb80d2c90 con 0x7f1eb80ce2e0 2023-08-27T11:25:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1ebd8bd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80ce770 con 0x7f1eb80b8c20 2023-08-27T11:25:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb6ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1eb815d2c0 0x7f1eb80d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43126/0 (socket says 172.21.15.45:43126) 2023-08-27T11:25:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb6ffd700 1 -- 172.21.15.45:0/1883112204 learned_addr learned my addr 172.21.15.45:0/1883112204 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb67fc700 1 --2- 172.21.15.45:0/1883112204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 0x7f1eb80cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb5ffb700 1 -- 172.21.15.45:0/1883112204 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779149612 0 0) 0x7f1ea8001110 con 0x7f1eb815d2c0 2023-08-27T11:25:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb5ffb700 1 -- 172.21.15.45:0/1883112204 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea00035b0 con 0x7f1eb815d2c0 2023-08-27T11:25:20.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb5ffb700 1 -- 172.21.15.45:0/1883112204 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168156741 0 0) 0x7f1eb80d2c90 con 0x7f1eb80ce2e0 2023-08-27T11:25:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb5ffb700 1 -- 172.21.15.45:0/1883112204 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea8001110 con 0x7f1eb80ce2e0 2023-08-27T11:25:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb67fc700 1 -- 172.21.15.45:0/1883112204 >> v1:172.21.15.135:6789/0 conn(0x7f1eb80ce2e0 legacy=0x7f1eb80b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb5ffb700 1 -- 172.21.15.45:0/1883112204 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1220854011 0 0) 0x7f1ea00035b0 con 0x7f1eb815d2c0 2023-08-27T11:25:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb67fc700 1 -- 172.21.15.45:0/1883112204 >> v1:172.21.15.45:6789/0 conn(0x7f1eb815d2c0 legacy=0x7f1eb80d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.089+0000 7f1eb67fc700 1 -- 172.21.15.45:0/1883112204 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb80d1bf0 con 0x7f1eb80b8c20 2023-08-27T11:25:20.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1eb67fc700 1 --2- 172.21.15.45:0/1883112204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 0x7f1eb80cdda0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009870 tx=0x7f1eac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2fd988da261d9e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1eb5ffb700 1 -- 172.21.15.45:0/1883112204 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8001800 con 0x7f1eb80b8c20 2023-08-27T11:25:20.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1eb5ffb700 1 -- 172.21.15.45:0/1883112204 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac004030 con 0x7f1eb80b8c20 2023-08-27T11:25:20.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1eb5ffb700 1 -- 172.21.15.45:0/1883112204 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0043c0 con 0x7f1eb80b8c20 2023-08-27T11:25:20.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/1883112204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 msgr2=0x7f1eb80cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1ebd8bd700 1 --2- 172.21.15.45:0/1883112204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 0x7f1eb80cdda0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009870 tx=0x7f1eac009b80 comp rx=0 tx=0).stop 2023-08-27T11:25:20.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/1883112204 shutdown_connections 2023-08-27T11:25:20.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1ebd8bd700 1 --2- 172.21.15.45:0/1883112204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 0x7f1eb80cdda0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/1883112204 >> 172.21.15.45:0/1883112204 conn(0x7f1eb80d2fd0 msgr2=0x7f1eb80d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/1883112204 shutdown_connections 2023-08-27T11:25:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.090+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/1883112204 wait complete. 2023-08-27T11:25:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.092+0000 7f1ebd8bd700 1 Processor -- start 2023-08-27T11:25:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.092+0000 7f1ebd8bd700 1 -- start start 2023-08-27T11:25:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1ebd8bd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 0x7f1eb80c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1ebd8bd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1eb80ce2e0 0x7f1eb80c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1ebd8bd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1eb815d2c0 0x7f1eb815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1ebd8bd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c9030 con 0x7f1eb815d2c0 2023-08-27T11:25:20.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1ebd8bd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c9310 con 0x7f1eb80ce2e0 2023-08-27T11:25:20.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1ebd8bd700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c6ac0 con 0x7f1eb80b8c20 2023-08-27T11:25:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1eb815d2c0 0x7f1eb815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1eb815d2c0 0x7f1eb815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56360/0 (socket says 172.21.15.45:56360) 2023-08-27T11:25:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb77fe700 1 -- 172.21.15.45:0/2865921868 learned_addr learned my addr 172.21.15.45:0/2865921868 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb6ffd700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 0x7f1eb80c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb67fc700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1eb80ce2e0 0x7f1eb80c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb77fe700 1 -- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 msgr2=0x7f1eb80c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb77fe700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 0x7f1eb80c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb77fe700 1 -- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1eb80ce2e0 msgr2=0x7f1eb80c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb77fe700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1eb80ce2e0 0x7f1eb80c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb77fe700 1 -- 172.21.15.45:0/2865921868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac0094a0 con 0x7f1eb815d2c0 2023-08-27T11:25:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb67fc700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1eb80ce2e0 0x7f1eb80c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1eb77fe700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1eb815d2c0 0x7f1eb815c090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cc20 tx=0x7f1eb000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1e9ffff700 1 -- 172.21.15.45:0/2865921868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000ac20 con 0x7f1eb815d2c0 2023-08-27T11:25:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1e9ffff700 1 -- 172.21.15.45:0/2865921868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000ad80 con 0x7f1eb815d2c0 2023-08-27T11:25:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb80c6da0 con 0x7f1eb815d2c0 2023-08-27T11:25:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1e9ffff700 1 -- 172.21.15.45:0/2865921868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0003c20 con 0x7f1eb815d2c0 2023-08-27T11:25:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.093+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb8150fa0 con 0x7f1eb815d2c0 2023-08-27T11:25:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.094+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb8151130 con 0x7f1eb815d2c0 2023-08-27T11:25:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.095+0000 7f1e9ffff700 1 -- 172.21.15.45:0/2865921868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1eb00050d0 con 0x7f1eb815d2c0 2023-08-27T11:25:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.095+0000 7f1e9ffff700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f1ea406a200 0x7f1ea406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.095+0000 7f1e9ffff700 1 -- 172.21.15.45:0/2865921868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1eb00889f0 con 0x7f1eb815d2c0 2023-08-27T11:25:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.095+0000 7f1eb6ffd700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f1ea406a200 0x7f1ea406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.096+0000 7f1eb6ffd700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f1ea406a200 0x7f1ea406c6c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8000f60 tx=0x7f1ea8009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.100+0000 7f1e9ffff700 1 -- 172.21.15.45:0/2865921868 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1eb0053e70 con 0x7f1eb815d2c0 2023-08-27T11:25:20.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.298+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 --> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1eb80d21b0 con 0x7f1ea406a200 2023-08-27T11:25:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.307+0000 7f1e9ffff700 1 -- 172.21.15.45:0/2865921868 <== mgr.5402 v2:172.21.15.135:6832/3181978685 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311619 (secure 0 0 0) 0x7f1ea80070d0 con 0x7f1ea406a200 2023-08-27T11:25:20.312 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f1ea406a200 msgr2=0x7f1ea406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f1ea406a200 0x7f1ea406c6c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8000f60 tx=0x7f1ea8009040 comp rx=0 tx=0).stop 2023-08-27T11:25:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1eb815d2c0 msgr2=0x7f1eb815c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1eb815d2c0 0x7f1eb815c090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cc20 tx=0x7f1eb000a060 comp rx=0 tx=0).stop 2023-08-27T11:25:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 shutdown_connections 2023-08-27T11:25:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f1ea406a200 0x7f1ea406c6c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1eb815d2c0 0x7f1eb815c090 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1eb80b8c20 0x7f1eb80c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 --2- 172.21.15.45:0/2865921868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1eb80ce2e0 0x7f1eb80c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 >> 172.21.15.45:0/2865921868 conn(0x7f1eb80d2fd0 msgr2=0x7f1eb8159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 shutdown_connections 2023-08-27T11:25:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.311+0000 7f1ebd8bd700 1 -- 172.21.15.45:0/2865921868 wait complete. 2023-08-27T11:25:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:25:20.324 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-08-27T11:25:18.323260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":244,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":192,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265568,"kb_used_data":7784,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754709152,"statfs":{"total":773094113280,"available":772822171648,"internally_reserved":0,"allocated":7970816,"data_stored":4638228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002335"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651999+0000","last_change":"2023-08-27T11:25:11.089737+0000","last_active":"2023-08-27T11:25:12.651999+0000","last_peered":"2023-08-27T11:25:12.651999+0000","last_clean":"2023-08-27T11:25:12.651999+0000","last_became_active":"2023-08-27T11:25:11.088793+0000","last_became_peered":"2023-08-27T11:25:11.088793+0000","last_unstale":"2023-08-27T11:25:12.651999+0000","last_undegraded":"2023-08-27T11:25:12.651999+0000","last_fullsized":"2023-08-27T11:25:12.651999+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:24:46.661058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151313+0000","last_change":"2023-08-27T11:25:11.083482+0000","last_active":"2023-08-27T11:25:12.151313+0000","last_peered":"2023-08-27T11:25:12.151313+0000","last_clean":"2023-08-27T11:25:12.151313+0000","last_became_active":"2023-08-27T11:25:11.083108+0000","last_became_peered":"2023-08-27T11:25:11.083108+0000","last_unstale":"2023-08-27T11:25:12.151313+0000","last_undegraded":"2023-08-27T11:25:12.151313+0000","last_fullsized":"2023-08-27T11:25:12.151313+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:30:42.181255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652740+0000","last_change":"2023-08-27T11:25:11.074228+0000","last_active":"2023-08-27T11:25:12.652740+0000","last_peered":"2023-08-27T11:25:12.652740+0000","last_clean":"2023-08-27T11:25:12.652740+0000","last_became_active":"2023-08-27T11:25:11.073701+0000","last_became_peered":"2023-08-27T11:25:11.073701+0000","last_unstale":"2023-08-27T11:25:12.652740+0000","last_undegraded":"2023-08-27T11:25:12.652740+0000","last_fullsized":"2023-08-27T11:25:12.652740+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:19:50.055671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076532+0000","last_change":"2023-08-27T11:25:11.069621+0000","last_active":"2023-08-27T11:25:12.076532+0000","last_peered":"2023-08-27T11:25:12.076532+0000","last_clean":"2023-08-27T11:25:12.076532+0000","last_became_active":"2023-08-27T11:25:11.069346+0000","last_became_peered":"2023-08-27T11:25:11.069346+0000","last_unstale":"2023-08-27T11:25:12.076532+0000","last_undegraded":"2023-08-27T11:25:12.076532+0000","last_fullsized":"2023-08-27T11:25:12.076532+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:45.624206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651752+0000","last_change":"2023-08-27T11:25:11.065611+0000","last_active":"2023-08-27T11:25:12.651752+0000","last_peered":"2023-08-27T11:25:12.651752+0000","last_clean":"2023-08-27T11:25:12.651752+0000","last_became_active":"2023-08-27T11:25:11.065315+0000","last_became_peered":"2023-08-27T11:25:11.065315+0000","last_unstale":"2023-08-27T11:25:12.651752+0000","last_undegraded":"2023-08-27T11:25:12.651752+0000","last_fullsized":"2023-08-27T11:25:12.651752+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:53.418624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073327+0000","last_change":"2023-08-27T11:25:11.075312+0000","last_active":"2023-08-27T11:25:12.073327+0000","last_peered":"2023-08-27T11:25:12.073327+0000","last_clean":"2023-08-27T11:25:12.073327+0000","last_became_active":"2023-08-27T11:25:11.075019+0000","last_became_peered":"2023-08-27T11:25:11.075019+0000","last_unstale":"2023-08-27T11:25:12.073327+0000","last_undegraded":"2023-08-27T11:25:12.073327+0000","last_fullsized":"2023-08-27T11:25:12.073327+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:14:28.271506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075021+0000","last_change":"2023-08-27T11:25:11.066375+0000","last_active":"2023-08-27T11:25:12.075021+0000","last_peered":"2023-08-27T11:25:12.075021+0000","last_clean":"2023-08-27T11:25:12.075021+0000","last_became_active":"2023-08-27T11:25:11.066087+0000","last_became_peered":"2023-08-27T11:25:11.066087+0000","last_unstale":"2023-08-27T11:25:12.075021+0000","last_undegraded":"2023-08-27T11:25:12.075021+0000","last_fullsized":"2023-08-27T11:25:12.075021+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:57:52.752858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151448+0000","last_change":"2023-08-27T11:25:11.070672+0000","last_active":"2023-08-27T11:25:12.151448+0000","last_peered":"2023-08-27T11:25:12.151448+0000","last_clean":"2023-08-27T11:25:12.151448+0000","last_became_active":"2023-08-27T11:25:11.070436+0000","last_became_peered":"2023-08-27T11:25:11.070436+0000","last_unstale":"2023-08-27T11:25:12.151448+0000","last_undegraded":"2023-08-27T11:25:12.151448+0000","last_fullsized":"2023-08-27T11:25:12.151448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:14.078958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075087+0000","last_change":"2023-08-27T11:25:11.077334+0000","last_active":"2023-08-27T11:25:12.075087+0000","last_peered":"2023-08-27T11:25:12.075087+0000","last_clean":"2023-08-27T11:25:12.075087+0000","last_became_active":"2023-08-27T11:25:11.077046+0000","last_became_peered":"2023-08-27T11:25:11.077046+0000","last_unstale":"2023-08-27T11:25:12.075087+0000","last_undegraded":"2023-08-27T11:25:12.075087+0000","last_fullsized":"2023-08-27T11:25:12.075087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:05:34.613670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652067+0000","last_change":"2023-08-27T11:25:11.077626+0000","last_active":"2023-08-27T11:25:12.652067+0000","last_peered":"2023-08-27T11:25:12.652067+0000","last_clean":"2023-08-27T11:25:12.652067+0000","last_became_active":"2023-08-27T11:25:11.077324+0000","last_became_peered":"2023-08-27T11:25:11.077324+0000","last_unstale":"2023-08-27T11:25:12.652067+0000","last_undegraded":"2023-08-27T11:25:12.652067+0000","last_fullsized":"2023-08-27T11:25:12.652067+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:52:38.805254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651972+0000","last_change":"2023-08-27T11:25:11.084040+0000","last_active":"2023-08-27T11:25:12.651972+0000","last_peered":"2023-08-27T11:25:12.651972+0000","last_clean":"2023-08-27T11:25:12.651972+0000","last_became_active":"2023-08-27T11:25:11.083745+0000","last_became_peered":"2023-08-27T11:25:11.083745+0000","last_unstale":"2023-08-27T11:25:12.651972+0000","last_undegraded":"2023-08-27T11:25:12.651972+0000","last_fullsized":"2023-08-27T11:25:12.651972+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:48.037337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076680+0000","last_change":"2023-08-27T11:25:11.067645+0000","last_active":"2023-08-27T11:25:12.076680+0000","last_peered":"2023-08-27T11:25:12.076680+0000","last_clean":"2023-08-27T11:25:12.076680+0000","last_became_active":"2023-08-27T11:25:11.067363+0000","last_became_peered":"2023-08-27T11:25:11.067363+0000","last_unstale":"2023-08-27T11:25:12.076680+0000","last_undegraded":"2023-08-27T11:25:12.076680+0000","last_fullsized":"2023-08-27T11:25:12.076680+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:54:08.394353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652543+0000","last_change":"2023-08-27T11:25:11.083448+0000","last_active":"2023-08-27T11:25:12.652543+0000","last_peered":"2023-08-27T11:25:12.652543+0000","last_clean":"2023-08-27T11:25:12.652543+0000","last_became_active":"2023-08-27T11:25:11.083185+0000","last_became_peered":"2023-08-27T11:25:11.083185+0000","last_unstale":"2023-08-27T11:25:12.652543+0000","last_undegraded":"2023-08-27T11:25:12.652543+0000","last_fullsized":"2023-08-27T11:25:12.652543+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:52:38.805254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074918+0000","last_change":"2023-08-27T11:25:11.064597+0000","last_active":"2023-08-27T11:25:12.074918+0000","last_peered":"2023-08-27T11:25:12.074918+0000","last_clean":"2023-08-27T11:25:12.074918+0000","last_became_active":"2023-08-27T11:25:11.064310+0000","last_became_peered":"2023-08-27T11:25:11.064310+0000","last_unstale":"2023-08-27T11:25:12.074918+0000","last_undegraded":"2023-08-27T11:25:12.074918+0000","last_fullsized":"2023-08-27T11:25:12.074918+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:50:08.178590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075183+0000","last_change":"2023-08-27T11:25:11.069011+0000","last_active":"2023-08-27T11:25:12.075183+0000","last_peered":"2023-08-27T11:25:12.075183+0000","last_clean":"2023-08-27T11:25:12.075183+0000","last_became_active":"2023-08-27T11:25:11.068211+0000","last_became_peered":"2023-08-27T11:25:11.068211+0000","last_unstale":"2023-08-27T11:25:12.075183+0000","last_undegraded":"2023-08-27T11:25:12.075183+0000","last_fullsized":"2023-08-27T11:25:12.075183+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:11.370163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073026+0000","last_change":"2023-08-27T11:25:11.079492+0000","last_active":"2023-08-27T11:25:12.073026+0000","last_peered":"2023-08-27T11:25:12.073026+0000","last_clean":"2023-08-27T11:25:12.073026+0000","last_became_active":"2023-08-27T11:25:11.079226+0000","last_became_peered":"2023-08-27T11:25:11.079226+0000","last_unstale":"2023-08-27T11:25:12.073026+0000","last_undegraded":"2023-08-27T11:25:12.073026+0000","last_fullsized":"2023-08-27T11:25:12.073026+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:16.558851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151418+0000","last_change":"2023-08-27T11:25:11.082153+0000","last_active":"2023-08-27T11:25:12.151418+0000","last_peered":"2023-08-27T11:25:12.151418+0000","last_clean":"2023-08-27T11:25:12.151418+0000","last_became_active":"2023-08-27T11:25:11.081866+0000","last_became_peered":"2023-08-27T11:25:11.081866+0000","last_unstale":"2023-08-27T11:25:12.151418+0000","last_undegraded":"2023-08-27T11:25:12.151418+0000","last_fullsized":"2023-08-27T11:25:12.151418+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:41.183698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652916+0000","last_change":"2023-08-27T11:25:11.078205+0000","last_active":"2023-08-27T11:25:12.652916+0000","last_peered":"2023-08-27T11:25:12.652916+0000","last_clean":"2023-08-27T11:25:12.652916+0000","last_became_active":"2023-08-27T11:25:11.077648+0000","last_became_peered":"2023-08-27T11:25:11.077648+0000","last_unstale":"2023-08-27T11:25:12.652916+0000","last_undegraded":"2023-08-27T11:25:12.652916+0000","last_fullsized":"2023-08-27T11:25:12.652916+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:17:05.972517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652025+0000","last_change":"2023-08-27T11:25:11.089266+0000","last_active":"2023-08-27T11:25:12.652025+0000","last_peered":"2023-08-27T11:25:12.652025+0000","last_clean":"2023-08-27T11:25:12.652025+0000","last_became_active":"2023-08-27T11:25:11.088926+0000","last_became_peered":"2023-08-27T11:25:11.088926+0000","last_unstale":"2023-08-27T11:25:12.652025+0000","last_undegraded":"2023-08-27T11:25:12.652025+0000","last_fullsized":"2023-08-27T11:25:12.652025+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:32:01.255063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151298+0000","last_change":"2023-08-27T11:25:11.086522+0000","last_active":"2023-08-27T11:25:12.151298+0000","last_peered":"2023-08-27T11:25:12.151298+0000","last_clean":"2023-08-27T11:25:12.151298+0000","last_became_active":"2023-08-27T11:25:11.086248+0000","last_became_peered":"2023-08-27T11:25:11.086248+0000","last_unstale":"2023-08-27T11:25:12.151298+0000","last_undegraded":"2023-08-27T11:25:12.151298+0000","last_fullsized":"2023-08-27T11:25:12.151298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:59.088239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652486+0000","last_change":"2023-08-27T11:25:11.077423+0000","last_active":"2023-08-27T11:25:12.652486+0000","last_peered":"2023-08-27T11:25:12.652486+0000","last_clean":"2023-08-27T11:25:12.652486+0000","last_became_active":"2023-08-27T11:25:11.077147+0000","last_became_peered":"2023-08-27T11:25:11.077147+0000","last_unstale":"2023-08-27T11:25:12.652486+0000","last_undegraded":"2023-08-27T11:25:12.652486+0000","last_fullsized":"2023-08-27T11:25:12.652486+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:10:35.747962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151088+0000","last_change":"2023-08-27T11:25:11.080859+0000","last_active":"2023-08-27T11:25:12.151088+0000","last_peered":"2023-08-27T11:25:12.151088+0000","last_clean":"2023-08-27T11:25:12.151088+0000","last_became_active":"2023-08-27T11:25:11.080531+0000","last_became_peered":"2023-08-27T11:25:11.080531+0000","last_unstale":"2023-08-27T11:25:12.151088+0000","last_undegraded":"2023-08-27T11:25:12.151088+0000","last_fullsized":"2023-08-27T11:25:12.151088+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:09.352024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651956+0000","last_change":"2023-08-27T11:25:11.086913+0000","last_active":"2023-08-27T11:25:12.651956+0000","last_peered":"2023-08-27T11:25:12.651956+0000","last_clean":"2023-08-27T11:25:12.651956+0000","last_became_active":"2023-08-27T11:25:11.086626+0000","last_became_peered":"2023-08-27T11:25:11.086626+0000","last_unstale":"2023-08-27T11:25:12.651956+0000","last_undegraded":"2023-08-27T11:25:12.651956+0000","last_fullsized":"2023-08-27T11:25:12.651956+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:08.542814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652635+0000","last_change":"2023-08-27T11:25:11.083359+0000","last_active":"2023-08-27T11:25:12.652635+0000","last_peered":"2023-08-27T11:25:12.652635+0000","last_clean":"2023-08-27T11:25:12.652635+0000","last_became_active":"2023-08-27T11:25:11.083108+0000","last_became_peered":"2023-08-27T11:25:11.083108+0000","last_unstale":"2023-08-27T11:25:12.652635+0000","last_undegraded":"2023-08-27T11:25:12.652635+0000","last_fullsized":"2023-08-27T11:25:12.652635+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:18:08.378262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652252+0000","last_change":"2023-08-27T11:25:11.085107+0000","last_active":"2023-08-27T11:25:12.652252+0000","last_peered":"2023-08-27T11:25:12.652252+0000","last_clean":"2023-08-27T11:25:12.652252+0000","last_became_active":"2023-08-27T11:25:11.076912+0000","last_became_peered":"2023-08-27T11:25:11.076912+0000","last_unstale":"2023-08-27T11:25:12.652252+0000","last_undegraded":"2023-08-27T11:25:12.652252+0000","last_fullsized":"2023-08-27T11:25:12.652252+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:54:55.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076685+0000","last_change":"2023-08-27T11:25:11.065003+0000","last_active":"2023-08-27T11:25:12.076685+0000","last_peered":"2023-08-27T11:25:12.076685+0000","last_clean":"2023-08-27T11:25:12.076685+0000","last_became_active":"2023-08-27T11:25:11.064698+0000","last_became_peered":"2023-08-27T11:25:11.064698+0000","last_unstale":"2023-08-27T11:25:12.076685+0000","last_undegraded":"2023-08-27T11:25:12.076685+0000","last_fullsized":"2023-08-27T11:25:12.076685+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:23:02.308613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073579+0000","last_change":"2023-08-27T11:25:11.083553+0000","last_active":"2023-08-27T11:25:12.073579+0000","last_peered":"2023-08-27T11:25:12.073579+0000","last_clean":"2023-08-27T11:25:12.073579+0000","last_became_active":"2023-08-27T11:25:11.083273+0000","last_became_peered":"2023-08-27T11:25:11.083273+0000","last_unstale":"2023-08-27T11:25:12.073579+0000","last_undegraded":"2023-08-27T11:25:12.073579+0000","last_fullsized":"2023-08-27T11:25:12.073579+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:37:29.281301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651748+0000","last_change":"2023-08-27T11:25:11.071688+0000","last_active":"2023-08-27T11:25:12.651748+0000","last_peered":"2023-08-27T11:25:12.651748+0000","last_clean":"2023-08-27T11:25:12.651748+0000","last_became_active":"2023-08-27T11:25:11.068352+0000","last_became_peered":"2023-08-27T11:25:11.068352+0000","last_unstale":"2023-08-27T11:25:12.651748+0000","last_undegraded":"2023-08-27T11:25:12.651748+0000","last_fullsized":"2023-08-27T11:25:12.651748+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:18:36.173709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151483+0000","last_change":"2023-08-27T11:25:11.078654+0000","last_active":"2023-08-27T11:25:12.151483+0000","last_peered":"2023-08-27T11:25:12.151483+0000","last_clean":"2023-08-27T11:25:12.151483+0000","last_became_active":"2023-08-27T11:25:11.078354+0000","last_became_peered":"2023-08-27T11:25:11.078354+0000","last_unstale":"2023-08-27T11:25:12.151483+0000","last_undegraded":"2023-08-27T11:25:12.151483+0000","last_fullsized":"2023-08-27T11:25:12.151483+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:15.892673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651913+0000","last_change":"2023-08-27T11:25:11.071285+0000","last_active":"2023-08-27T11:25:12.651913+0000","last_peered":"2023-08-27T11:25:12.651913+0000","last_clean":"2023-08-27T11:25:12.651913+0000","last_became_active":"2023-08-27T11:25:11.070956+0000","last_became_peered":"2023-08-27T11:25:11.070956+0000","last_unstale":"2023-08-27T11:25:12.651913+0000","last_undegraded":"2023-08-27T11:25:12.651913+0000","last_fullsized":"2023-08-27T11:25:12.651913+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:53.227452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651785+0000","last_change":"2023-08-27T11:25:11.077269+0000","last_active":"2023-08-27T11:25:12.651785+0000","last_peered":"2023-08-27T11:25:12.651785+0000","last_clean":"2023-08-27T11:25:12.651785+0000","last_became_active":"2023-08-27T11:25:11.076555+0000","last_became_peered":"2023-08-27T11:25:11.076555+0000","last_unstale":"2023-08-27T11:25:12.651785+0000","last_undegraded":"2023-08-27T11:25:12.651785+0000","last_fullsized":"2023-08-27T11:25:12.651785+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:39:06.180828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073134+0000","last_change":"2023-08-27T11:25:11.090246+0000","last_active":"2023-08-27T11:25:12.073134+0000","last_peered":"2023-08-27T11:25:12.073134+0000","last_clean":"2023-08-27T11:25:12.073134+0000","last_became_active":"2023-08-27T11:25:11.089944+0000","last_became_peered":"2023-08-27T11:25:11.089944+0000","last_unstale":"2023-08-27T11:25:12.073134+0000","last_undegraded":"2023-08-27T11:25:12.073134+0000","last_fullsized":"2023-08-27T11:25:12.073134+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:26:24.657680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073760+0000","last_change":"2023-08-27T11:25:11.075952+0000","last_active":"2023-08-27T11:25:12.073760+0000","last_peered":"2023-08-27T11:25:12.073760+0000","last_clean":"2023-08-27T11:25:12.073760+0000","last_became_active":"2023-08-27T11:25:11.075567+0000","last_became_peered":"2023-08-27T11:25:11.075567+0000","last_unstale":"2023-08-27T11:25:12.073760+0000","last_undegraded":"2023-08-27T11:25:12.073760+0000","last_fullsized":"2023-08-27T11:25:12.073760+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:46:44.428971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"69'4","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074235+0000","last_change":"2023-08-27T11:25:10.059196+0000","last_active":"2023-08-27T11:25:12.074235+0000","last_peered":"2023-08-27T11:25:12.074235+0000","last_clean":"2023-08-27T11:25:12.074235+0000","last_became_active":"2023-08-27T11:25:10.058927+0000","last_became_peered":"2023-08-27T11:25:10.058927+0000","last_unstale":"2023-08-27T11:25:12.074235+0000","last_undegraded":"2023-08-27T11:25:12.074235+0000","last_fullsized":"2023-08-27T11:25:12.074235+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:17:37.587247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652348+0000","last_change":"2023-08-27T11:25:11.075708+0000","last_active":"2023-08-27T11:25:12.652348+0000","last_peered":"2023-08-27T11:25:12.652348+0000","last_clean":"2023-08-27T11:25:12.652348+0000","last_became_active":"2023-08-27T11:25:11.075407+0000","last_became_peered":"2023-08-27T11:25:11.075407+0000","last_unstale":"2023-08-27T11:25:12.652348+0000","last_undegraded":"2023-08-27T11:25:12.652348+0000","last_fullsized":"2023-08-27T11:25:12.652348+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:19:11.472663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151479+0000","last_change":"2023-08-27T11:25:10.063585+0000","last_active":"2023-08-27T11:25:12.151479+0000","last_peered":"2023-08-27T11:25:12.151479+0000","last_clean":"2023-08-27T11:25:12.151479+0000","last_became_active":"2023-08-27T11:25:10.063302+0000","last_became_peered":"2023-08-27T11:25:10.063302+0000","last_unstale":"2023-08-27T11:25:12.151479+0000","last_undegraded":"2023-08-27T11:25:12.151479+0000","last_fullsized":"2023-08-27T11:25:12.151479+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:07.534036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073466+0000","last_change":"2023-08-27T11:25:11.082620+0000","last_active":"2023-08-27T11:25:12.073466+0000","last_peered":"2023-08-27T11:25:12.073466+0000","last_clean":"2023-08-27T11:25:12.073466+0000","last_became_active":"2023-08-27T11:25:11.082305+0000","last_became_peered":"2023-08-27T11:25:11.082305+0000","last_unstale":"2023-08-27T11:25:12.073466+0000","last_undegraded":"2023-08-27T11:25:12.073466+0000","last_fullsized":"2023-08-27T11:25:12.073466+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:51.549355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073971+0000","last_change":"2023-08-27T11:25:10.062557+0000","last_active":"2023-08-27T11:25:12.073971+0000","last_peered":"2023-08-27T11:25:12.073971+0000","last_clean":"2023-08-27T11:25:12.073971+0000","last_became_active":"2023-08-27T11:25:10.062283+0000","last_became_peered":"2023-08-27T11:25:10.062283+0000","last_unstale":"2023-08-27T11:25:12.073971+0000","last_undegraded":"2023-08-27T11:25:12.073971+0000","last_fullsized":"2023-08-27T11:25:12.073971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:52.060725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073721+0000","last_change":"2023-08-27T11:25:11.081581+0000","last_active":"2023-08-27T11:25:12.073721+0000","last_peered":"2023-08-27T11:25:12.073721+0000","last_clean":"2023-08-27T11:25:12.073721+0000","last_became_active":"2023-08-27T11:25:11.081099+0000","last_became_peered":"2023-08-27T11:25:11.081099+0000","last_unstale":"2023-08-27T11:25:12.073721+0000","last_undegraded":"2023-08-27T11:25:12.073721+0000","last_fullsized":"2023-08-27T11:25:12.073721+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:47.620310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076792+0000","last_change":"2023-08-27T11:25:10.063437+0000","last_active":"2023-08-27T11:25:12.076792+0000","last_peered":"2023-08-27T11:25:12.076792+0000","last_clean":"2023-08-27T11:25:12.076792+0000","last_became_active":"2023-08-27T11:25:10.063203+0000","last_became_peered":"2023-08-27T11:25:10.063203+0000","last_unstale":"2023-08-27T11:25:12.076792+0000","last_undegraded":"2023-08-27T11:25:12.076792+0000","last_fullsized":"2023-08-27T11:25:12.076792+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:24:45.731455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151377+0000","last_change":"2023-08-27T11:25:11.082439+0000","last_active":"2023-08-27T11:25:12.151377+0000","last_peered":"2023-08-27T11:25:12.151377+0000","last_clean":"2023-08-27T11:25:12.151377+0000","last_became_active":"2023-08-27T11:25:11.080131+0000","last_became_peered":"2023-08-27T11:25:11.080131+0000","last_unstale":"2023-08-27T11:25:12.151377+0000","last_undegraded":"2023-08-27T11:25:12.151377+0000","last_fullsized":"2023-08-27T11:25:12.151377+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:23:26.417776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074803+0000","last_change":"2023-08-27T11:25:10.062084+0000","last_active":"2023-08-27T11:25:12.074803+0000","last_peered":"2023-08-27T11:25:12.074803+0000","last_clean":"2023-08-27T11:25:12.074803+0000","last_became_active":"2023-08-27T11:25:10.061782+0000","last_became_peered":"2023-08-27T11:25:10.061782+0000","last_unstale":"2023-08-27T11:25:12.074803+0000","last_undegraded":"2023-08-27T11:25:12.074803+0000","last_fullsized":"2023-08-27T11:25:12.074803+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:37:34.190548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151222+0000","last_change":"2023-08-27T11:25:11.086124+0000","last_active":"2023-08-27T11:25:12.151222+0000","last_peered":"2023-08-27T11:25:12.151222+0000","last_clean":"2023-08-27T11:25:12.151222+0000","last_became_active":"2023-08-27T11:25:11.085932+0000","last_became_peered":"2023-08-27T11:25:11.085932+0000","last_unstale":"2023-08-27T11:25:12.151222+0000","last_undegraded":"2023-08-27T11:25:12.151222+0000","last_fullsized":"2023-08-27T11:25:12.151222+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:52:34.512979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073393+0000","last_change":"2023-08-27T11:25:10.057702+0000","last_active":"2023-08-27T11:25:12.073393+0000","last_peered":"2023-08-27T11:25:12.073393+0000","last_clean":"2023-08-27T11:25:12.073393+0000","last_became_active":"2023-08-27T11:25:10.057347+0000","last_became_peered":"2023-08-27T11:25:10.057347+0000","last_unstale":"2023-08-27T11:25:12.073393+0000","last_undegraded":"2023-08-27T11:25:12.073393+0000","last_fullsized":"2023-08-27T11:25:12.073393+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:17:04.963739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652461+0000","last_change":"2023-08-27T11:25:11.077695+0000","last_active":"2023-08-27T11:25:12.652461+0000","last_peered":"2023-08-27T11:25:12.652461+0000","last_clean":"2023-08-27T11:25:12.652461+0000","last_became_active":"2023-08-27T11:25:11.077412+0000","last_became_peered":"2023-08-27T11:25:11.077412+0000","last_unstale":"2023-08-27T11:25:12.652461+0000","last_undegraded":"2023-08-27T11:25:12.652461+0000","last_fullsized":"2023-08-27T11:25:12.652461+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:21:48.037337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076378+0000","last_change":"2023-08-27T11:25:10.062767+0000","last_active":"2023-08-27T11:25:12.076378+0000","last_peered":"2023-08-27T11:25:12.076378+0000","last_clean":"2023-08-27T11:25:12.076378+0000","last_became_active":"2023-08-27T11:25:10.062448+0000","last_became_peered":"2023-08-27T11:25:10.062448+0000","last_unstale":"2023-08-27T11:25:12.076378+0000","last_undegraded":"2023-08-27T11:25:12.076378+0000","last_fullsized":"2023-08-27T11:25:12.076378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:48:06.823020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151105+0000","last_change":"2023-08-27T11:25:11.081782+0000","last_active":"2023-08-27T11:25:12.151105+0000","last_peered":"2023-08-27T11:25:12.151105+0000","last_clean":"2023-08-27T11:25:12.151105+0000","last_became_active":"2023-08-27T11:25:11.081479+0000","last_became_peered":"2023-08-27T11:25:11.081479+0000","last_unstale":"2023-08-27T11:25:12.151105+0000","last_undegraded":"2023-08-27T11:25:12.151105+0000","last_fullsized":"2023-08-27T11:25:12.151105+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:15:06.456158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074607+0000","last_change":"2023-08-27T11:25:10.063530+0000","last_active":"2023-08-27T11:25:12.074607+0000","last_peered":"2023-08-27T11:25:12.074607+0000","last_clean":"2023-08-27T11:25:12.074607+0000","last_became_active":"2023-08-27T11:25:10.063284+0000","last_became_peered":"2023-08-27T11:25:10.063284+0000","last_unstale":"2023-08-27T11:25:12.074607+0000","last_undegraded":"2023-08-27T11:25:12.074607+0000","last_fullsized":"2023-08-27T11:25:12.074607+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:27:09.376425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075093+0000","last_change":"2023-08-27T11:25:11.070590+0000","last_active":"2023-08-27T11:25:12.075093+0000","last_peered":"2023-08-27T11:25:12.075093+0000","last_clean":"2023-08-27T11:25:12.075093+0000","last_became_active":"2023-08-27T11:25:11.070325+0000","last_became_peered":"2023-08-27T11:25:11.070325+0000","last_unstale":"2023-08-27T11:25:12.075093+0000","last_undegraded":"2023-08-27T11:25:12.075093+0000","last_fullsized":"2023-08-27T11:25:12.075093+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:57:32.534110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652975+0000","last_change":"2023-08-27T11:25:10.064796+0000","last_active":"2023-08-27T11:25:12.652975+0000","last_peered":"2023-08-27T11:25:12.652975+0000","last_clean":"2023-08-27T11:25:12.652975+0000","last_became_active":"2023-08-27T11:25:10.064460+0000","last_became_peered":"2023-08-27T11:25:10.064460+0000","last_unstale":"2023-08-27T11:25:12.652975+0000","last_undegraded":"2023-08-27T11:25:12.652975+0000","last_fullsized":"2023-08-27T11:25:12.652975+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:37:28.272523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.653014+0000","last_change":"2023-08-27T11:25:11.076982+0000","last_active":"2023-08-27T11:25:12.653014+0000","last_peered":"2023-08-27T11:25:12.653014+0000","last_clean":"2023-08-27T11:25:12.653014+0000","last_became_active":"2023-08-27T11:25:11.076618+0000","last_became_peered":"2023-08-27T11:25:11.076618+0000","last_unstale":"2023-08-27T11:25:12.653014+0000","last_undegraded":"2023-08-27T11:25:12.653014+0000","last_fullsized":"2023-08-27T11:25:12.653014+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:40.388403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151481+0000","last_change":"2023-08-27T11:25:10.060400+0000","last_active":"2023-08-27T11:25:12.151481+0000","last_peered":"2023-08-27T11:25:12.151481+0000","last_clean":"2023-08-27T11:25:12.151481+0000","last_became_active":"2023-08-27T11:25:10.060119+0000","last_became_peered":"2023-08-27T11:25:10.060119+0000","last_unstale":"2023-08-27T11:25:12.151481+0000","last_undegraded":"2023-08-27T11:25:12.151481+0000","last_fullsized":"2023-08-27T11:25:12.151481+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:46:52.218673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651858+0000","last_change":"2023-08-27T11:25:11.074836+0000","last_active":"2023-08-27T11:25:12.651858+0000","last_peered":"2023-08-27T11:25:12.651858+0000","last_clean":"2023-08-27T11:25:12.651858+0000","last_became_active":"2023-08-27T11:25:11.074522+0000","last_became_peered":"2023-08-27T11:25:11.074522+0000","last_unstale":"2023-08-27T11:25:12.651858+0000","last_undegraded":"2023-08-27T11:25:12.651858+0000","last_fullsized":"2023-08-27T11:25:12.651858+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:25.131425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"69'3","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652848+0000","last_change":"2023-08-27T11:25:10.065878+0000","last_active":"2023-08-27T11:25:12.652848+0000","last_peered":"2023-08-27T11:25:12.652848+0000","last_clean":"2023-08-27T11:25:12.652848+0000","last_became_active":"2023-08-27T11:25:10.065602+0000","last_became_peered":"2023-08-27T11:25:10.065602+0000","last_unstale":"2023-08-27T11:25:12.652848+0000","last_undegraded":"2023-08-27T11:25:12.652848+0000","last_fullsized":"2023-08-27T11:25:12.652848+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:14:27.262728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652941+0000","last_change":"2023-08-27T11:25:11.075893+0000","last_active":"2023-08-27T11:25:12.652941+0000","last_peered":"2023-08-27T11:25:12.652941+0000","last_clean":"2023-08-27T11:25:12.652941+0000","last_became_active":"2023-08-27T11:25:11.075620+0000","last_became_peered":"2023-08-27T11:25:11.075620+0000","last_unstale":"2023-08-27T11:25:12.652941+0000","last_undegraded":"2023-08-27T11:25:12.652941+0000","last_fullsized":"2023-08-27T11:25:12.652941+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:42:13.331122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"69'6","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075245+0000","last_change":"2023-08-27T11:25:10.060915+0000","last_active":"2023-08-27T11:25:12.075245+0000","last_peered":"2023-08-27T11:25:12.075245+0000","last_clean":"2023-08-27T11:25:12.075245+0000","last_became_active":"2023-08-27T11:25:10.060597+0000","last_became_peered":"2023-08-27T11:25:10.060597+0000","last_unstale":"2023-08-27T11:25:12.075245+0000","last_undegraded":"2023-08-27T11:25:12.075245+0000","last_fullsized":"2023-08-27T11:25:12.075245+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:44:44.970472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076277+0000","last_change":"2023-08-27T11:25:11.071783+0000","last_active":"2023-08-27T11:25:12.076277+0000","last_peered":"2023-08-27T11:25:12.076277+0000","last_clean":"2023-08-27T11:25:12.076277+0000","last_became_active":"2023-08-27T11:25:11.070418+0000","last_became_peered":"2023-08-27T11:25:11.070418+0000","last_unstale":"2023-08-27T11:25:12.076277+0000","last_undegraded":"2023-08-27T11:25:12.076277+0000","last_fullsized":"2023-08-27T11:25:12.076277+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:32:25.243371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151652+0000","last_change":"2023-08-27T11:25:10.060471+0000","last_active":"2023-08-27T11:25:12.151652+0000","last_peered":"2023-08-27T11:25:12.151652+0000","last_clean":"2023-08-27T11:25:12.151652+0000","last_became_active":"2023-08-27T11:25:10.060174+0000","last_became_peered":"2023-08-27T11:25:10.060174+0000","last_unstale":"2023-08-27T11:25:12.151652+0000","last_undegraded":"2023-08-27T11:25:12.151652+0000","last_fullsized":"2023-08-27T11:25:12.151652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:46.606725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652556+0000","last_change":"2023-08-27T11:25:11.078419+0000","last_active":"2023-08-27T11:25:12.652556+0000","last_peered":"2023-08-27T11:25:12.652556+0000","last_clean":"2023-08-27T11:25:12.652556+0000","last_became_active":"2023-08-27T11:25:11.078124+0000","last_became_peered":"2023-08-27T11:25:11.078124+0000","last_unstale":"2023-08-27T11:25:12.652556+0000","last_undegraded":"2023-08-27T11:25:12.652556+0000","last_fullsized":"2023-08-27T11:25:12.652556+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:20:42.020468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652350+0000","last_change":"2023-08-27T11:25:10.056373+0000","last_active":"2023-08-27T11:25:12.652350+0000","last_peered":"2023-08-27T11:25:12.652350+0000","last_clean":"2023-08-27T11:25:12.652350+0000","last_became_active":"2023-08-27T11:25:10.056089+0000","last_became_peered":"2023-08-27T11:25:10.056089+0000","last_unstale":"2023-08-27T11:25:12.652350+0000","last_undegraded":"2023-08-27T11:25:12.652350+0000","last_fullsized":"2023-08-27T11:25:12.652350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:24.122647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.075035+0000","last_change":"2023-08-27T11:25:11.071794+0000","last_active":"2023-08-27T11:25:12.075035+0000","last_peered":"2023-08-27T11:25:12.075035+0000","last_clean":"2023-08-27T11:25:12.075035+0000","last_became_active":"2023-08-27T11:25:11.071515+0000","last_became_peered":"2023-08-27T11:25:11.071515+0000","last_unstale":"2023-08-27T11:25:12.075035+0000","last_undegraded":"2023-08-27T11:25:12.075035+0000","last_fullsized":"2023-08-27T11:25:12.075035+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:47.623307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076358+0000","last_change":"2023-08-27T11:25:10.063540+0000","last_active":"2023-08-27T11:25:12.076358+0000","last_peered":"2023-08-27T11:25:12.076358+0000","last_clean":"2023-08-27T11:25:12.076358+0000","last_became_active":"2023-08-27T11:25:10.063232+0000","last_became_peered":"2023-08-27T11:25:10.063232+0000","last_unstale":"2023-08-27T11:25:12.076358+0000","last_undegraded":"2023-08-27T11:25:12.076358+0000","last_fullsized":"2023-08-27T11:25:12.076358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:42:58.185719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652012+0000","last_change":"2023-08-27T11:25:11.087922+0000","last_active":"2023-08-27T11:25:12.652012+0000","last_peered":"2023-08-27T11:25:12.652012+0000","last_clean":"2023-08-27T11:25:12.652012+0000","last_became_active":"2023-08-27T11:25:11.087655+0000","last_became_peered":"2023-08-27T11:25:11.087655+0000","last_unstale":"2023-08-27T11:25:12.652012+0000","last_undegraded":"2023-08-27T11:25:12.652012+0000","last_fullsized":"2023-08-27T11:25:12.652012+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:08.067157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"69'8","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651820+0000","last_change":"2023-08-27T11:25:10.062911+0000","last_active":"2023-08-27T11:25:12.651820+0000","last_peered":"2023-08-27T11:25:12.651820+0000","last_clean":"2023-08-27T11:25:12.651820+0000","last_became_active":"2023-08-27T11:25:10.062611+0000","last_became_peered":"2023-08-27T11:25:10.062611+0000","last_unstale":"2023-08-27T11:25:12.651820+0000","last_undegraded":"2023-08-27T11:25:12.651820+0000","last_fullsized":"2023-08-27T11:25:12.651820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:14.883895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076851+0000","last_change":"2023-08-27T11:25:11.076967+0000","last_active":"2023-08-27T11:25:12.076851+0000","last_peered":"2023-08-27T11:25:12.076851+0000","last_clean":"2023-08-27T11:25:12.076851+0000","last_became_active":"2023-08-27T11:25:11.076687+0000","last_became_peered":"2023-08-27T11:25:11.076687+0000","last_unstale":"2023-08-27T11:25:12.076851+0000","last_undegraded":"2023-08-27T11:25:12.076851+0000","last_fullsized":"2023-08-27T11:25:12.076851+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:18:22.631584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652299+0000","last_change":"2023-08-27T11:25:10.059439+0000","last_active":"2023-08-27T11:25:12.652299+0000","last_peered":"2023-08-27T11:25:12.652299+0000","last_clean":"2023-08-27T11:25:12.652299+0000","last_became_active":"2023-08-27T11:25:10.059056+0000","last_became_peered":"2023-08-27T11:25:10.059056+0000","last_unstale":"2023-08-27T11:25:12.652299+0000","last_undegraded":"2023-08-27T11:25:12.652299+0000","last_fullsized":"2023-08-27T11:25:12.652299+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:34.944067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076823+0000","last_change":"2023-08-27T11:25:11.072286+0000","last_active":"2023-08-27T11:25:12.076823+0000","last_peered":"2023-08-27T11:25:12.076823+0000","last_clean":"2023-08-27T11:25:12.076823+0000","last_became_active":"2023-08-27T11:25:11.072002+0000","last_became_peered":"2023-08-27T11:25:11.072002+0000","last_unstale":"2023-08-27T11:25:12.076823+0000","last_undegraded":"2023-08-27T11:25:12.076823+0000","last_fullsized":"2023-08-27T11:25:12.076823+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:02:09.202868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"70'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074030+0000","last_change":"2023-08-27T11:25:10.060251+0000","last_active":"2023-08-27T11:25:12.074030+0000","last_peered":"2023-08-27T11:25:12.074030+0000","last_clean":"2023-08-27T11:25:12.074030+0000","last_became_active":"2023-08-27T11:25:10.059911+0000","last_became_peered":"2023-08-27T11:25:10.059911+0000","last_unstale":"2023-08-27T11:25:12.074030+0000","last_undegraded":"2023-08-27T11:25:12.074030+0000","last_fullsized":"2023-08-27T11:25:12.074030+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:39:54.589392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652007+0000","last_change":"2023-08-27T11:25:11.074463+0000","last_active":"2023-08-27T11:25:12.652007+0000","last_peered":"2023-08-27T11:25:12.652007+0000","last_clean":"2023-08-27T11:25:12.652007+0000","last_became_active":"2023-08-27T11:25:11.074121+0000","last_became_peered":"2023-08-27T11:25:11.074121+0000","last_unstale":"2023-08-27T11:25:12.652007+0000","last_undegraded":"2023-08-27T11:25:12.652007+0000","last_fullsized":"2023-08-27T11:25:12.652007+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:49:21.311911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652183+0000","last_change":"2023-08-27T11:25:10.052679+0000","last_active":"2023-08-27T11:25:12.652183+0000","last_peered":"2023-08-27T11:25:12.652183+0000","last_clean":"2023-08-27T11:25:12.652183+0000","last_became_active":"2023-08-27T11:25:10.052343+0000","last_became_peered":"2023-08-27T11:25:10.052343+0000","last_unstale":"2023-08-27T11:25:12.652183+0000","last_undegraded":"2023-08-27T11:25:12.652183+0000","last_fullsized":"2023-08-27T11:25:12.652183+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:20.770727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073872+0000","last_change":"2023-08-27T11:25:11.081384+0000","last_active":"2023-08-27T11:25:12.073872+0000","last_peered":"2023-08-27T11:25:12.073872+0000","last_clean":"2023-08-27T11:25:12.073872+0000","last_became_active":"2023-08-27T11:25:11.080789+0000","last_became_peered":"2023-08-27T11:25:11.080789+0000","last_unstale":"2023-08-27T11:25:12.073872+0000","last_undegraded":"2023-08-27T11:25:12.073872+0000","last_fullsized":"2023-08-27T11:25:12.073872+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:28.594935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076764+0000","last_change":"2023-08-27T11:25:10.056377+0000","last_active":"2023-08-27T11:25:12.076764+0000","last_peered":"2023-08-27T11:25:12.076764+0000","last_clean":"2023-08-27T11:25:12.076764+0000","last_became_active":"2023-08-27T11:25:10.056122+0000","last_became_peered":"2023-08-27T11:25:10.056122+0000","last_unstale":"2023-08-27T11:25:12.076764+0000","last_undegraded":"2023-08-27T11:25:12.076764+0000","last_fullsized":"2023-08-27T11:25:12.076764+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:04:23.170480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151249+0000","last_change":"2023-08-27T11:25:11.078397+0000","last_active":"2023-08-27T11:25:12.151249+0000","last_peered":"2023-08-27T11:25:12.151249+0000","last_clean":"2023-08-27T11:25:12.151249+0000","last_became_active":"2023-08-27T11:25:11.074097+0000","last_became_peered":"2023-08-27T11:25:11.074097+0000","last_unstale":"2023-08-27T11:25:12.151249+0000","last_undegraded":"2023-08-27T11:25:12.151249+0000","last_fullsized":"2023-08-27T11:25:12.151249+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:51:03.922037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651543+0000","last_change":"2023-08-27T11:25:10.058064+0000","last_active":"2023-08-27T11:25:12.651543+0000","last_peered":"2023-08-27T11:25:12.651543+0000","last_clean":"2023-08-27T11:25:12.651543+0000","last_became_active":"2023-08-27T11:25:10.057743+0000","last_became_peered":"2023-08-27T11:25:10.057743+0000","last_unstale":"2023-08-27T11:25:12.651543+0000","last_undegraded":"2023-08-27T11:25:12.651543+0000","last_fullsized":"2023-08-27T11:25:12.651543+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:35:46.801723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652636+0000","last_change":"2023-08-27T11:25:11.090519+0000","last_active":"2023-08-27T11:25:12.652636+0000","last_peered":"2023-08-27T11:25:12.652636+0000","last_clean":"2023-08-27T11:25:12.652636+0000","last_became_active":"2023-08-27T11:25:11.090200+0000","last_became_peered":"2023-08-27T11:25:11.090200+0000","last_unstale":"2023-08-27T11:25:12.652636+0000","last_undegraded":"2023-08-27T11:25:12.652636+0000","last_fullsized":"2023-08-27T11:25:12.652636+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:16.558851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.074733+0000","last_change":"2023-08-27T11:25:10.060787+0000","last_active":"2023-08-27T11:25:12.074733+0000","last_peered":"2023-08-27T11:25:12.074733+0000","last_clean":"2023-08-27T11:25:12.074733+0000","last_became_active":"2023-08-27T11:25:10.060416+0000","last_became_peered":"2023-08-27T11:25:10.060416+0000","last_unstale":"2023-08-27T11:25:12.074733+0000","last_undegraded":"2023-08-27T11:25:12.074733+0000","last_fullsized":"2023-08-27T11:25:12.074733+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:32:16.158243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652251+0000","last_change":"2023-08-27T11:25:11.079350+0000","last_active":"2023-08-27T11:25:12.652251+0000","last_peered":"2023-08-27T11:25:12.652251+0000","last_clean":"2023-08-27T11:25:12.652251+0000","last_became_active":"2023-08-27T11:25:11.079059+0000","last_became_peered":"2023-08-27T11:25:11.079059+0000","last_unstale":"2023-08-27T11:25:12.652251+0000","last_undegraded":"2023-08-27T11:25:12.652251+0000","last_fullsized":"2023-08-27T11:25:12.652251+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:00:23.873229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651588+0000","last_change":"2023-08-27T11:25:10.061300+0000","last_active":"2023-08-27T11:25:12.651588+0000","last_peered":"2023-08-27T11:25:12.651588+0000","last_clean":"2023-08-27T11:25:12.651588+0000","last_became_active":"2023-08-27T11:25:10.061015+0000","last_became_peered":"2023-08-27T11:25:10.061015+0000","last_unstale":"2023-08-27T11:25:12.651588+0000","last_undegraded":"2023-08-27T11:25:12.651588+0000","last_fullsized":"2023-08-27T11:25:12.651588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:12:20.770727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151136+0000","last_change":"2023-08-27T11:25:11.081511+0000","last_active":"2023-08-27T11:25:12.151136+0000","last_peered":"2023-08-27T11:25:12.151136+0000","last_clean":"2023-08-27T11:25:12.151136+0000","last_became_active":"2023-08-27T11:25:11.081124+0000","last_became_peered":"2023-08-27T11:25:11.081124+0000","last_unstale":"2023-08-27T11:25:12.151136+0000","last_undegraded":"2023-08-27T11:25:12.151136+0000","last_fullsized":"2023-08-27T11:25:12.151136+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:18:21.207590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151336+0000","last_change":"2023-08-27T11:25:10.058695+0000","last_active":"2023-08-27T11:25:12.151336+0000","last_peered":"2023-08-27T11:25:12.151336+0000","last_clean":"2023-08-27T11:25:12.151336+0000","last_became_active":"2023-08-27T11:25:10.058370+0000","last_became_peered":"2023-08-27T11:25:10.058370+0000","last_unstale":"2023-08-27T11:25:12.151336+0000","last_undegraded":"2023-08-27T11:25:12.151336+0000","last_fullsized":"2023-08-27T11:25:12.151336+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:02:59.411254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076966+0000","last_change":"2023-08-27T11:25:11.074442+0000","last_active":"2023-08-27T11:25:12.076966+0000","last_peered":"2023-08-27T11:25:12.076966+0000","last_clean":"2023-08-27T11:25:12.076966+0000","last_became_active":"2023-08-27T11:25:11.074131+0000","last_became_peered":"2023-08-27T11:25:11.074131+0000","last_unstale":"2023-08-27T11:25:12.076966+0000","last_undegraded":"2023-08-27T11:25:12.076966+0000","last_fullsized":"2023-08-27T11:25:12.076966+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:18:24.128262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073847+0000","last_change":"2023-08-27T11:25:10.061400+0000","last_active":"2023-08-27T11:25:12.073847+0000","last_peered":"2023-08-27T11:25:12.073847+0000","last_clean":"2023-08-27T11:25:12.073847+0000","last_became_active":"2023-08-27T11:25:10.061105+0000","last_became_peered":"2023-08-27T11:25:10.061105+0000","last_unstale":"2023-08-27T11:25:12.073847+0000","last_undegraded":"2023-08-27T11:25:12.073847+0000","last_fullsized":"2023-08-27T11:25:12.073847+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:02:43.029300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":115,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652090+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:25:12.652090+0000","last_peered":"2023-08-27T11:25:12.652090+0000","last_clean":"2023-08-27T11:25:12.652090+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:25:12.652090+0000","last_undegraded":"2023-08-27T11:25:12.652090+0000","last_fullsized":"2023-08-27T11:25:12.652090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:05:00.194138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073245+0000","last_change":"2023-08-27T11:25:11.075517+0000","last_active":"2023-08-27T11:25:12.073245+0000","last_peered":"2023-08-27T11:25:12.073245+0000","last_clean":"2023-08-27T11:25:12.073245+0000","last_became_active":"2023-08-27T11:25:11.075044+0000","last_became_peered":"2023-08-27T11:25:11.075044+0000","last_unstale":"2023-08-27T11:25:12.073245+0000","last_undegraded":"2023-08-27T11:25:12.073245+0000","last_fullsized":"2023-08-27T11:25:12.073245+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:19:50.055671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076223+0000","last_change":"2023-08-27T11:25:10.060606+0000","last_active":"2023-08-27T11:25:12.076223+0000","last_peered":"2023-08-27T11:25:12.076223+0000","last_clean":"2023-08-27T11:25:12.076223+0000","last_became_active":"2023-08-27T11:25:10.060341+0000","last_became_peered":"2023-08-27T11:25:10.060341+0000","last_unstale":"2023-08-27T11:25:12.076223+0000","last_undegraded":"2023-08-27T11:25:12.076223+0000","last_fullsized":"2023-08-27T11:25:12.076223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:21:51.523703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":113,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076419+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:25:12.076419+0000","last_peered":"2023-08-27T11:25:12.076419+0000","last_clean":"2023-08-27T11:25:12.076419+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:25:12.076419+0000","last_undegraded":"2023-08-27T11:25:12.076419+0000","last_fullsized":"2023-08-27T11:25:12.076419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:40:01.268610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076649+0000","last_change":"2023-08-27T11:25:11.070397+0000","last_active":"2023-08-27T11:25:12.076649+0000","last_peered":"2023-08-27T11:25:12.076649+0000","last_clean":"2023-08-27T11:25:12.076649+0000","last_became_active":"2023-08-27T11:25:11.070109+0000","last_became_peered":"2023-08-27T11:25:11.070109+0000","last_unstale":"2023-08-27T11:25:12.076649+0000","last_undegraded":"2023-08-27T11:25:12.076649+0000","last_fullsized":"2023-08-27T11:25:12.076649+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:14:46.794884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"69'2","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151579+0000","last_change":"2023-08-27T11:25:10.061271+0000","last_active":"2023-08-27T11:25:12.151579+0000","last_peered":"2023-08-27T11:25:12.151579+0000","last_clean":"2023-08-27T11:25:12.151579+0000","last_became_active":"2023-08-27T11:25:10.060546+0000","last_became_peered":"2023-08-27T11:25:10.060546+0000","last_unstale":"2023-08-27T11:25:12.151579+0000","last_undegraded":"2023-08-27T11:25:12.151579+0000","last_fullsized":"2023-08-27T11:25:12.151579+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:36:41.413862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651396+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:25:12.651396+0000","last_peered":"2023-08-27T11:25:12.651396+0000","last_clean":"2023-08-27T11:25:12.651396+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:25:12.651396+0000","last_undegraded":"2023-08-27T11:25:12.651396+0000","last_fullsized":"2023-08-27T11:25:12.651396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:50.524608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151230+0000","last_change":"2023-08-27T11:25:11.081351+0000","last_active":"2023-08-27T11:25:12.151230+0000","last_peered":"2023-08-27T11:25:12.151230+0000","last_clean":"2023-08-27T11:25:12.151230+0000","last_became_active":"2023-08-27T11:25:11.080968+0000","last_became_peered":"2023-08-27T11:25:11.080968+0000","last_unstale":"2023-08-27T11:25:12.151230+0000","last_undegraded":"2023-08-27T11:25:12.151230+0000","last_fullsized":"2023-08-27T11:25:12.151230+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:40:04.596358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652392+0000","last_change":"2023-08-27T11:25:10.063913+0000","last_active":"2023-08-27T11:25:12.652392+0000","last_peered":"2023-08-27T11:25:12.652392+0000","last_clean":"2023-08-27T11:25:12.652392+0000","last_became_active":"2023-08-27T11:25:10.063642+0000","last_became_peered":"2023-08-27T11:25:10.063642+0000","last_unstale":"2023-08-27T11:25:12.652392+0000","last_undegraded":"2023-08-27T11:25:12.652392+0000","last_fullsized":"2023-08-27T11:25:12.652392+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:31:01.736095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":113,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076601+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:25:12.076601+0000","last_peered":"2023-08-27T11:25:12.076601+0000","last_clean":"2023-08-27T11:25:12.076601+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:25:12.076601+0000","last_undegraded":"2023-08-27T11:25:12.076601+0000","last_fullsized":"2023-08-27T11:25:12.076601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:11:41.635538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"58'139","reported_seq":347,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651844+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:25:12.651844+0000","last_peered":"2023-08-27T11:25:12.651844+0000","last_clean":"2023-08-27T11:25:12.651844+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:25:12.651844+0000","last_undegraded":"2023-08-27T11:25:12.651844+0000","last_fullsized":"2023-08-27T11:25:12.651844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:41:32.788138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073256+0000","last_change":"2023-08-27T11:25:11.078879+0000","last_active":"2023-08-27T11:25:12.073256+0000","last_peered":"2023-08-27T11:25:12.073256+0000","last_clean":"2023-08-27T11:25:12.073256+0000","last_became_active":"2023-08-27T11:25:11.078582+0000","last_became_peered":"2023-08-27T11:25:11.078582+0000","last_unstale":"2023-08-27T11:25:12.073256+0000","last_undegraded":"2023-08-27T11:25:12.073256+0000","last_fullsized":"2023-08-27T11:25:12.073256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:42:13.331122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.651858+0000","last_change":"2023-08-27T11:25:10.060723+0000","last_active":"2023-08-27T11:25:12.651858+0000","last_peered":"2023-08-27T11:25:12.651858+0000","last_clean":"2023-08-27T11:25:12.651858+0000","last_became_active":"2023-08-27T11:25:10.060423+0000","last_became_peered":"2023-08-27T11:25:10.060423+0000","last_unstale":"2023-08-27T11:25:12.651858+0000","last_undegraded":"2023-08-27T11:25:12.651858+0000","last_fullsized":"2023-08-27T11:25:12.651858+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:31:01.736095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":113,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652001+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:25:12.652001+0000","last_peered":"2023-08-27T11:25:12.652001+0000","last_clean":"2023-08-27T11:25:12.652001+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:25:12.652001+0000","last_undegraded":"2023-08-27T11:25:12.652001+0000","last_fullsized":"2023-08-27T11:25:12.652001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:23:34.238516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652974+0000","last_change":"2023-08-27T11:25:11.073477+0000","last_active":"2023-08-27T11:25:12.652974+0000","last_peered":"2023-08-27T11:25:12.652974+0000","last_clean":"2023-08-27T11:25:12.652974+0000","last_became_active":"2023-08-27T11:25:11.073225+0000","last_became_peered":"2023-08-27T11:25:11.073225+0000","last_unstale":"2023-08-27T11:25:12.652974+0000","last_undegraded":"2023-08-27T11:25:12.652974+0000","last_fullsized":"2023-08-27T11:25:12.652974+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:24:51.549355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151489+0000","last_change":"2023-08-27T11:25:10.060521+0000","last_active":"2023-08-27T11:25:12.151489+0000","last_peered":"2023-08-27T11:25:12.151489+0000","last_clean":"2023-08-27T11:25:12.151489+0000","last_became_active":"2023-08-27T11:25:10.060236+0000","last_became_peered":"2023-08-27T11:25:10.060236+0000","last_unstale":"2023-08-27T11:25:12.151489+0000","last_undegraded":"2023-08-27T11:25:12.151489+0000","last_fullsized":"2023-08-27T11:25:12.151489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:21:07.058379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":113,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.653026+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:25:12.653026+0000","last_peered":"2023-08-27T11:25:12.653026+0000","last_clean":"2023-08-27T11:25:12.653026+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:25:12.653026+0000","last_undegraded":"2023-08-27T11:25:12.653026+0000","last_fullsized":"2023-08-27T11:25:12.653026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:59:39.436186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652730+0000","last_change":"2023-08-27T11:25:11.074279+0000","last_active":"2023-08-27T11:25:12.652730+0000","last_peered":"2023-08-27T11:25:12.652730+0000","last_clean":"2023-08-27T11:25:12.652730+0000","last_became_active":"2023-08-27T11:25:11.073985+0000","last_became_peered":"2023-08-27T11:25:11.073985+0000","last_unstale":"2023-08-27T11:25:12.652730+0000","last_undegraded":"2023-08-27T11:25:12.652730+0000","last_fullsized":"2023-08-27T11:25:12.652730+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:26:24.657680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.073498+0000","last_change":"2023-08-27T11:25:10.066265+0000","last_active":"2023-08-27T11:25:12.073498+0000","last_peered":"2023-08-27T11:25:12.073498+0000","last_clean":"2023-08-27T11:25:12.073498+0000","last_became_active":"2023-08-27T11:25:10.065959+0000","last_became_peered":"2023-08-27T11:25:10.065959+0000","last_unstale":"2023-08-27T11:25:12.073498+0000","last_undegraded":"2023-08-27T11:25:12.073498+0000","last_fullsized":"2023-08-27T11:25:12.073498+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:39.379625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":115,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652123+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:25:12.652123+0000","last_peered":"2023-08-27T11:25:12.652123+0000","last_clean":"2023-08-27T11:25:12.652123+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:25:12.652123+0000","last_undegraded":"2023-08-27T11:25:12.652123+0000","last_fullsized":"2023-08-27T11:25:12.652123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:58.020795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.151399+0000","last_change":"2023-08-27T11:25:11.071240+0000","last_active":"2023-08-27T11:25:12.151399+0000","last_peered":"2023-08-27T11:25:12.151399+0000","last_clean":"2023-08-27T11:25:12.151399+0000","last_became_active":"2023-08-27T11:25:11.070969+0000","last_became_peered":"2023-08-27T11:25:11.070969+0000","last_unstale":"2023-08-27T11:25:12.151399+0000","last_undegraded":"2023-08-27T11:25:12.151399+0000","last_fullsized":"2023-08-27T11:25:12.151399+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:10.038390+0000","last_clean_scrub_stamp":"2023-08-27T11:25:10.038390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:50:18.122114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"69'1","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.076584+0000","last_change":"2023-08-27T11:25:10.064086+0000","last_active":"2023-08-27T11:25:12.076584+0000","last_peered":"2023-08-27T11:25:12.076584+0000","last_clean":"2023-08-27T11:25:12.076584+0000","last_became_active":"2023-08-27T11:25:10.063790+0000","last_became_peered":"2023-08-27T11:25:10.063790+0000","last_unstale":"2023-08-27T11:25:12.076584+0000","last_undegraded":"2023-08-27T11:25:12.076584+0000","last_fullsized":"2023-08-27T11:25:12.076584+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:25:09.029612+0000","last_clean_scrub_stamp":"2023-08-27T11:25:09.029612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:50:48.474309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":107,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-08-27T11:25:12.652382+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:25:12.652382+0000","last_peered":"2023-08-27T11:25:12.652382+0000","last_clean":"2023-08-27T11:25:12.652382+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:25:12.652382+0000","last_undegraded":"2023-08-27T11:25:12.652382+0000","last_fullsized":"2023-08-27T11:25:12.652382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:36:13.127787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":1022112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.35199999999999998,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.875,"15min":0.875},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.55900000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60899999999999999,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.314},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.48299999999999998,"5min":0.35199999999999998,"15min":0.252},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.439,"5min":0.439,"15min":0.29099999999999998},"max":{"1min":0.79100000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65600000000000003,"15min":0.61599999999999999},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.78100000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65700000000000003,"15min":0.61899999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.39900000000000002},"max":{"1min":0.82199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56899999999999995}]},{"osd":4,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.442,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.26500000000000001,"15min":0.23499999999999999},"max":{"1min":0.65400000000000003,"5min":0.67500000000000004,"15min":0.72499999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.52900000000000003},"min":{"1min":0.39100000000000001,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.66700000000000004}]},{"osd":5,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46200000000000002,"15min":0.48099999999999998},"min":{"1min":0.32200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71699999999999997,"5min":0.74199999999999999,"15min":0.84699999999999998},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.30099999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.90100000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":431584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.59699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.377,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":1.3540000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.621,"15min":0.63100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":0.79700000000000004,"15min":3.3780000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.63900000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60399999999999998,"15min":0.71199999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":9.3770000000000007},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.746,"5min":0.82799999999999996,"15min":1.325},"last":0.55900000000000005}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60399999999999998,"15min":0.68600000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34100000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73599999999999999,"5min":0.84899999999999998,"15min":1.395},"last":0.433}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.25900000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34300000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68500000000000005,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53300000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.496,"15min":0.5},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.82099999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.32900000000000001},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72299999999999998,"5min":0.76100000000000001,"15min":0.88900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.34899999999999998},"max":{"1min":0.76400000000000001,"5min":0.77400000000000002,"15min":0.86299999999999999},"last":0.621}]}]},{"osd":1,"up_from":13,"seq":55834575010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40324,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331516,"statfs":{"total":96636764160,"available":96595472384,"internally_reserved":0,"allocated":843776,"data_stored":431674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.20399999999999999},"max":{"1min":0.52700000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.34999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.60699999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.313}]},{"osd":2,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.251},"max":{"1min":0.52700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":0.68100000000000005,"5min":0.68300000000000005,"15min":0.78600000000000003},"last":0.33600000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.32500000000000001,"5min":0.311,"15min":0.27600000000000002},"max":{"1min":0.61899999999999999,"5min":0.70399999999999996,"15min":0.77100000000000002},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45700000000000002,"15min":0.434},"min":{"1min":0.29699999999999999,"5min":0.28599999999999998,"15min":0.23400000000000001},"max":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.80000000000000004},"last":0.35699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.51700000000000002,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.97999999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.47899999999999998,"5min":0.378,"15min":0.35199999999999998},"max":{"1min":0.79700000000000004,"5min":0.94199999999999995,"15min":2.9380000000000002},"last":0.51700000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.32900000000000001},"max":{"1min":0.86099999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.53200000000000003,"5min":0.45600000000000002,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.42099999999999999},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81599999999999995,"5min":0.97999999999999998,"15min":2.9740000000000002},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.54700000000000004,"5min":0.47399999999999998,"15min":0.42499999999999999},"max":{"1min":0.85499999999999998,"5min":1.002,"15min":1.139},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.441},"max":{"1min":0.876,"5min":0.91600000000000004,"15min":1.0169999999999999},"last":0.57899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":1024146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41999999999999998,"15min":0.40600000000000003},"min":{"1min":0.28199999999999997,"5min":0.27200000000000002,"15min":0.23699999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.90200000000000002},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.47499999999999998,"15min":0.46000000000000002},"min":{"1min":0.33200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":1.2889999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42899999999999999,"15min":0.41399999999999998},"min":{"1min":0.317,"5min":0.313,"15min":0.22900000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":1.3340000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.505},"min":{"1min":0.30199999999999999,"5min":0.29099999999999998,"15min":0.255},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":8.1349999999999998},"last":0.314}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.56399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.23899999999999999},"max":{"1min":0.56899999999999995,"5min":0.69899999999999995,"15min":11.026999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52800000000000002,"15min":0.65300000000000002},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.29699999999999999},"max":{"1min":0.63800000000000001,"5min":0.73299999999999998,"15min":11.926},"last":0.52000000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66500000000000004,"15min":0.77600000000000002},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":1.0069999999999999,"15min":10.964},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.27900000000000003},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":1.3640000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.73699999999999999},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":8.0779999999999994},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70199999999999996,"15min":0.81000000000000005},"min":{"1min":0.53300000000000003,"5min":0.42799999999999999,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":1,"15min":10.922000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.66600000000000004},"min":{"1min":0.51400000000000001,"5min":0.46200000000000002,"15min":0.35699999999999998},"max":{"1min":0.82499999999999996,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.77400000000000002},"min":{"1min":0.51400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":8.0380000000000003},"last":0.71099999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.78500000000000003},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":8.1069999999999993},"last":0.627},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":1.036,"15min":10.996},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":847872,"data_stored":431640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38500000000000001,"15min":0.372},"min":{"1min":0.25800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56200000000000006,"5min":0.57799999999999996,"15min":0.65400000000000003},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.437,"15min":0.41499999999999998},"min":{"1min":0.30599999999999999,"5min":0.25,"15min":0.20799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.35999999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.753},"last":0.307},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46000000000000002,"15min":0.443},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.24099999999999999},"max":{"1min":0.71299999999999997,"5min":0.71699999999999997,"15min":0.77800000000000002},"last":0.40600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48099999999999998,"15min":0.46200000000000002},"min":{"1min":0.34599999999999997,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.46700000000000003,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.29299999999999998},"max":{"1min":0.61899999999999999,"5min":0.65300000000000002,"15min":0.69099999999999995},"last":0.33900000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76500000000000001,"5min":1.163,"15min":1.163},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.39400000000000002,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0660000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95199999999999996,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.55000000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68100000000000005,"15min":0.68000000000000005},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.41699999999999998},"max":{"1min":0.98899999999999999,"5min":1.1319999999999999,"15min":1.2050000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.52300000000000002,"5min":0.39700000000000002,"15min":0.38400000000000001},"max":{"1min":1.0189999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.499,"15min":0.40100000000000002},"max":{"1min":1.0409999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45700000000000002,"15min":0.42299999999999999},"max":{"1min":0.96999999999999997,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":431640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40999999999999998,"15min":0.52300000000000002},"min":{"1min":0.26800000000000002,"5min":0.23899999999999999,"15min":0.22},"max":{"1min":0.53800000000000003,"5min":0.63300000000000001,"15min":11.885999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44500000000000001,"15min":0.40000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.224},"max":{"1min":0.59899999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.27100000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.47999999999999998},"min":{"1min":0.32500000000000001,"5min":0.29299999999999998,"15min":0.255},"max":{"1min":0.57399999999999995,"5min":0.67900000000000005,"15min":6.0830000000000002},"last":0.498},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.503,"15min":0.60599999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.245},"max":{"1min":0.68899999999999995,"5min":0.80700000000000005,"15min":12.050000000000001},"last":0.29999999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.504,"15min":0.45800000000000002},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.27100000000000002},"max":{"1min":0.66700000000000004,"5min":0.93100000000000005,"15min":1.25},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54200000000000004,"15min":0.63800000000000001},"min":{"1min":0.376,"5min":0.374,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.83999999999999997,"15min":11.939},"last":0.47499999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.76900000000000002},"min":{"1min":0.436,"5min":0.39700000000000002,"15min":0.376},"max":{"1min":0.86099999999999999,"5min":1.0329999999999999,"15min":11.007},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70199999999999996,"15min":0.65400000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":0.91000000000000003,"5min":1.113,"15min":1.2929999999999999},"last":0.60199999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70499999999999996,"15min":0.81200000000000006},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.436},"max":{"1min":0.92500000000000004,"5min":1.0800000000000001,"15min":11.981},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70899999999999996,"15min":0.79300000000000004},"min":{"1min":0.46999999999999997,"5min":0.46700000000000003,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.218,"15min":11.09},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72799999999999998,"15min":0.91400000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.38200000000000001},"max":{"1min":0.88900000000000001,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.748},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75800000000000001,"15min":0.85499999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.47799999999999998},"max":{"1min":0.95899999999999996,"5min":1.0389999999999999,"15min":12.016},"last":0.78100000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.80900000000000005},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.41599999999999998},"max":{"1min":0.91100000000000003,"5min":1.006,"15min":11.054},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73999999999999999,"15min":0.69299999999999995},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":1.0640000000000001,"15min":1.361},"last":0.64900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575010,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":433196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.23499999999999999},"max":{"1min":0.71099999999999997,"5min":0.81799999999999995,"15min":0.89800000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63,"15min":0.60399999999999998},"min":{"1min":0.501,"5min":0.40699999999999997,"15min":0.255},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.873},"last":0.56899999999999995}]},{"osd":1,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.622,"15min":0.59699999999999998},"min":{"1min":0.51800000000000002,"5min":0.35999999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82999999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.45800000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.747,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.312},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.629},"min":{"1min":0.52100000000000002,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.503,"5min":0.40200000000000002,"15min":0.30099999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.49399999999999999,"5min":0.371,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.88800000000000001},"last":0.41099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47499999999999998,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59099999999999997,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.497,"15min":0.51600000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.26200000000000001},"max":{"1min":0.72299999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70899999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.495,"15min":0.52900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.27800000000000002},"max":{"1min":0.72199999999999998,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.441},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.93899999999999995},"last":0.72399999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.874,"15min":0.878},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.496}]}]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36176,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335664,"statfs":{"total":96636764160,"available":96599719936,"internally_reserved":0,"allocated":847872,"data_stored":432236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.93300000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.26200000000000001},"max":{"1min":0.749,"5min":0.84999999999999998,"15min":0.88100000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.59999999999999998},"min":{"1min":0.51600000000000001,"5min":0.36199999999999999,"15min":0.248},"max":{"1min":0.80400000000000005,"5min":0.81899999999999995,"15min":0.96999999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.32600000000000001},"max":{"1min":0.752,"5min":0.77400000000000002,"15min":0.82299999999999995},"last":0.50600000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.33400000000000002,"15min":0.32800000000000001},"max":{"1min":0.75,"5min":0.87,"15min":0.87},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":1.0109999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.48099999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.80600000000000005,"15min":0.94199999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.46100000000000002,"5min":0.34799999999999998,"15min":0.34000000000000002},"max":{"1min":0.79400000000000004,"5min":0.79500000000000004,"15min":0.83799999999999997},"last":0.56200000000000006}]},{"osd":4,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.36399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.71299999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.33300000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47999999999999998,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.89000000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.379,"15min":0.379},"max":{"1min":0.75,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45100000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.88100000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.84999999999999998},"last":0.42899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:25:20.334 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:25:20.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:25:20.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.496+0000 7f63c0ce5700 1 Processor -- start 2023-08-27T11:25:20.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.496+0000 7f63c0ce5700 1 -- start start 2023-08-27T11:25:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63c0ce5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b65d0 0x7f63bc0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63c0ce5700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f63bc0cb5c0 0x7f63bc15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63c0ce5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63bc15ec50 con 0x7f63bc0cfe10 2023-08-27T11:25:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63c0ce5700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63bc15c620 con 0x7f63bc0cb5c0 2023-08-27T11:25:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63c0ce5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc15eea0 con 0x7f63bc0b65d0 2023-08-27T11:25:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63bad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f63bc0cfe10 0x7f63bc0cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43142/0 (socket says 172.21.15.45:43142) 2023-08-27T11:25:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63bad9d700 1 -- 172.21.15.45:0/1674754782 learned_addr learned my addr 172.21.15.45:0/1674754782 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63ba59c700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b65d0 0x7f63bc0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b9d9b700 1 --2- 172.21.15.45:0/1674754782 >> v2:172.21.15.135:3300/0 conn(0x7f63bc0cb5c0 0x7f63bc15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709025873 0 0) 0x7f63bc15ec50 con 0x7f63bc0cfe10 2023-08-27T11:25:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a40035b0 con 0x7f63bc0cfe10 2023-08-27T11:25:20.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3614724540 0 0) 0x7f63a40035b0 con 0x7f63bc0cfe10 2023-08-27T11:25:20.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b65d0 msgr2=0x7f63bc0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:20.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b65d0 0x7f63bc0ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 >> v2:172.21.15.135:3300/0 conn(0x7f63bc0cb5c0 msgr2=0x7f63bc15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:20.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 --2- 172.21.15.45:0/1674754782 >> v2:172.21.15.135:3300/0 conn(0x7f63bc0cb5c0 0x7f63bc15c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc15f180 con 0x7f63bc0cfe10 2023-08-27T11:25:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f63b40019f0 con 0x7f63bc0cfe10 2023-08-27T11:25:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 >> v1:172.21.15.45:6789/0 conn(0x7f63bc0cfe10 legacy=0x7f63bc0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63a4003a00 0x7f63a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b65d0 0x7f63a4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63a4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc15c620 con 0x7f63a4004d90 2023-08-27T11:25:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc15eea0 con 0x7f63a4003a00 2023-08-27T11:25:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.497+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a400ba10 con 0x7f63bc0b65d0 2023-08-27T11:25:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63ba59c700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63a4003a00 0x7f63a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63bad9d700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63a4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63b9d9b700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b65d0 0x7f63a4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63bad9d700 1 -- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b65d0 msgr2=0x7f63a4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63bad9d700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b65d0 0x7f63a4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63bad9d700 1 -- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63a4003a00 msgr2=0x7f63a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:20.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63bad9d700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63a4003a00 0x7f63a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63bad9d700 1 -- 172.21.15.45:0/1674754782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a40035b0 con 0x7f63a4004d90 2023-08-27T11:25:20.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63b9d9b700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b65d0 0x7f63a4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:20.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63bad9d700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63a4005210 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f63b400a8d0 tx=0x7f63a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2062eeb9fcfdcf7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b400a340 con 0x7f63a4004d90 2023-08-27T11:25:20.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b4012600 con 0x7f63a4004d90 2023-08-27T11:25:20.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.498+0000 7f63b959a700 1 -- 172.21.15.45:0/1674754782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b40129b0 con 0x7f63a4004d90 2023-08-27T11:25:20.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.499+0000 7f63c0ce5700 1 -- 172.21.15.45:0/1674754782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 msgr2=0x7f63a4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.499+0000 7f63c0ce5700 1 --2- 172.21.15.45:0/1674754782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63a4005210 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f63b400a8d0 tx=0x7f63a4005880 comp rx=0 tx=0).stop 2023-08-27T11:25:20.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.499+0000 7f63bad9d700 1 -- 172.21.15.45:0/1674754782 reap_dead start 2023-08-27T11:25:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.499+0000 7f63c0ce5700 1 -- 172.21.15.45:0/1674754782 shutdown_connections 2023-08-27T11:25:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.499+0000 7f63c0ce5700 1 -- 172.21.15.45:0/1674754782 >> 172.21.15.45:0/1674754782 conn(0x7f63bc06f960 msgr2=0x7f63bc06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.499+0000 7f63c0ce5700 1 -- 172.21.15.45:0/1674754782 shutdown_connections 2023-08-27T11:25:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.499+0000 7f63c0ce5700 1 -- 172.21.15.45:0/1674754782 wait complete. 2023-08-27T11:25:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.500+0000 7f63c0ce5700 1 Processor -- start 2023-08-27T11:25:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.500+0000 7f63c0ce5700 1 -- start start 2023-08-27T11:25:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63c0ce5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63bc06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63c0ce5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63bc06b390 0x7f63bc149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63c0ce5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b0f40 0x7f63bc0b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63c0ce5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0b0b20 con 0x7f63a4004d90 2023-08-27T11:25:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63c0ce5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc151050 con 0x7f63bc06b390 2023-08-27T11:25:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63c0ce5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0b07d0 con 0x7f63bc0b0f40 2023-08-27T11:25:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63ba59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63bc06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63ba59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63bc06d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56378/0 (socket says 172.21.15.45:56378) 2023-08-27T11:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63ba59c700 1 -- 172.21.15.45:0/2399062872 learned_addr learned my addr 172.21.15.45:0/2399062872 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63bad9d700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b0f40 0x7f63bc0b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63b9d9b700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63bc06b390 0x7f63bc149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63ba59c700 1 -- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b0f40 msgr2=0x7f63bc0b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63ba59c700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b0f40 0x7f63bc0b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63ba59c700 1 -- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63bc06b390 msgr2=0x7f63bc149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63ba59c700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63bc06b390 0x7f63bc149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63ba59c700 1 -- 172.21.15.45:0/2399062872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc06b810 con 0x7f63a4004d90 2023-08-27T11:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63b9d9b700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63bc06b390 0x7f63bc149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63bad9d700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b0f40 0x7f63bc0b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:20.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.501+0000 7f63ba59c700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63bc06d030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0119a0 tx=0x7f63ac011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.502+0000 7f63aaffd700 1 -- 172.21.15.45:0/2399062872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00f940 con 0x7f63a4004d90 2023-08-27T11:25:20.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.502+0000 7f63aaffd700 1 -- 172.21.15.45:0/2399062872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac00faa0 con 0x7f63a4004d90 2023-08-27T11:25:20.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.502+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc1497b0 con 0x7f63a4004d90 2023-08-27T11:25:20.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.502+0000 7f63aaffd700 1 -- 172.21.15.45:0/2399062872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00fe30 con 0x7f63a4004d90 2023-08-27T11:25:20.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.502+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc06bd70 con 0x7f63a4004d90 2023-08-27T11:25:20.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.503+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc0b0c60 con 0x7f63a4004d90 2023-08-27T11:25:20.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.503+0000 7f63aaffd700 1 -- 172.21.15.45:0/2399062872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f63ac014440 con 0x7f63a4004d90 2023-08-27T11:25:20.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.504+0000 7f63aaffd700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f639c06a230 0x7f639c06c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.504+0000 7f63aaffd700 1 -- 172.21.15.45:0/2399062872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f63ac088fb0 con 0x7f63a4004d90 2023-08-27T11:25:20.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.504+0000 7f63b9d9b700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f639c06a230 0x7f639c06c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.504+0000 7f63b9d9b700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f639c06a230 0x7f639c06c6f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0b6160 tx=0x7f63b0008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.508+0000 7f63aaffd700 1 -- 172.21.15.45:0/2399062872 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f63bc0b0c60 con 0x7f63a4004d90 2023-08-27T11:25:20.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.791+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f63bc0b0c60 con 0x7f63a4004d90 2023-08-27T11:25:20.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.794+0000 7f63aaffd700 1 -- 172.21.15.45:0/2399062872 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f63bc0b0c60 con 0x7f63a4004d90 2023-08-27T11:25:20.795 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-08-27T11:25:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ== 2023-08-27T11:25:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2023-08-27T11:25:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2023-08-27T11:25:20.796 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:25:20.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f639c06a230 msgr2=0x7f639c06c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f639c06a230 0x7f639c06c6f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0b6160 tx=0x7f63b0008040 comp rx=0 tx=0).stop 2023-08-27T11:25:20.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 msgr2=0x7f63bc06d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63bc06d030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0119a0 tx=0x7f63ac011d60 comp rx=0 tx=0).stop 2023-08-27T11:25:20.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 shutdown_connections 2023-08-27T11:25:20.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f639c06a230 0x7f639c06c6f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a4004d90 0x7f63bc06d030 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f63bc0b0f40 0x7f63bc0b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 --2- 172.21.15.45:0/2399062872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63bc06b390 0x7f63bc149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 >> 172.21.15.45:0/2399062872 conn(0x7f63bc06f960 msgr2=0x7f63bc06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:20.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 shutdown_connections 2023-08-27T11:25:20.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.797+0000 7f63c0ce5700 1 -- 172.21.15.45:0/2399062872 wait complete. 2023-08-27T11:25:20.801 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-08-27T11:25:20.810 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:25:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.934+0000 7fdcd86da700 1 Processor -- start 2023-08-27T11:25:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.934+0000 7fdcd86da700 1 -- start start 2023-08-27T11:25:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd86da700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdcd015a0f0 0x7fdcd00cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd86da700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd86da700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd00cfae0 con 0x7fdcd015e030 2023-08-27T11:25:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd86da700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd015e900 con 0x7fdcd015a0f0 2023-08-27T11:25:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd86da700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd00cae60 con 0x7fdcd00cd300 2023-08-27T11:25:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd5c75700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd6476700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdcd015a0f0 0x7fdcd00cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd5c75700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd015daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:54294/0 (socket says 172.21.15.45:54294) 2023-08-27T11:25:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd5c75700 1 -- 172.21.15.45:0/1291076553 learned_addr learned my addr 172.21.15.45:0/1291076553 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.935+0000 7fdcd6c77700 1 --1- 172.21.15.45:0/1291076553 >> v1:172.21.15.135:6789/0 conn(0x7fdcd015e030 0x7fdcd00cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:38828/0 (socket says 172.21.15.45:38828) 2023-08-27T11:25:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.936+0000 7fdcd5c75700 1 -- 172.21.15.45:0/1291076553 >> v2:172.21.15.135:3300/0 conn(0x7fdcd015a0f0 msgr2=0x7fdcd00cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.936+0000 7fdcd5c75700 1 --2- 172.21.15.45:0/1291076553 >> v2:172.21.15.135:3300/0 conn(0x7fdcd015a0f0 0x7fdcd00cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.936+0000 7fdcd5c75700 1 -- 172.21.15.45:0/1291076553 >> v1:172.21.15.135:6789/0 conn(0x7fdcd015e030 legacy=0x7fdcd00cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.936+0000 7fdcd5c75700 1 -- 172.21.15.45:0/1291076553 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd00cb140 con 0x7fdcd00cd300 2023-08-27T11:25:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.936+0000 7fdcd5c75700 1 --2- 172.21.15.45:0/1291076553 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd015daf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0098f0 tx=0x7fdccc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=125b71819984686a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.936+0000 7fdcd5474700 1 -- 172.21.15.45:0/1291076553 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc011830 con 0x7fdcd00cd300 2023-08-27T11:25:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.936+0000 7fdcd5474700 1 -- 172.21.15.45:0/1291076553 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc011990 con 0x7fdcd00cd300 2023-08-27T11:25:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.936+0000 7fdcd5474700 1 -- 172.21.15.45:0/1291076553 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc011d20 con 0x7fdcd00cd300 2023-08-27T11:25:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.937+0000 7fdcd86da700 1 -- 172.21.15.45:0/1291076553 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 msgr2=0x7fdcd015daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.937+0000 7fdcd86da700 1 --2- 172.21.15.45:0/1291076553 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd015daf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0098f0 tx=0x7fdccc009c00 comp rx=0 tx=0).stop 2023-08-27T11:25:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.937+0000 7fdcd86da700 1 -- 172.21.15.45:0/1291076553 shutdown_connections 2023-08-27T11:25:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.937+0000 7fdcd86da700 1 --2- 172.21.15.45:0/1291076553 >> v2:172.21.15.135:3300/0 conn(0x7fdcd015a0f0 0x7fdcd00cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.937+0000 7fdcd86da700 1 --2- 172.21.15.45:0/1291076553 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd015daf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.937+0000 7fdcd86da700 1 -- 172.21.15.45:0/1291076553 >> 172.21.15.45:0/1291076553 conn(0x7fdcd00cc7e0 msgr2=0x7fdcd00ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.937+0000 7fdcd86da700 1 -- 172.21.15.45:0/1291076553 shutdown_connections 2023-08-27T11:25:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.937+0000 7fdcd86da700 1 -- 172.21.15.45:0/1291076553 wait complete. 2023-08-27T11:25:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.938+0000 7fdcd86da700 1 Processor -- start 2023-08-27T11:25:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.938+0000 7fdcd86da700 1 -- start start 2023-08-27T11:25:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd86da700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd01493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd86da700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd015a0f0 0x7fdcd0153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd86da700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcd015e030 0x7fdcd01542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd86da700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd00b29c0 con 0x7fdcd015a0f0 2023-08-27T11:25:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd86da700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd00b3540 con 0x7fdcd015e030 2023-08-27T11:25:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd86da700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd01498e0 con 0x7fdcd00cd300 2023-08-27T11:25:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd5c75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd015a0f0 0x7fdcd0153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd5c75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd015a0f0 0x7fdcd0153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56394/0 (socket says 172.21.15.45:56394) 2023-08-27T11:25:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd5c75700 1 -- 172.21.15.45:0/2211889437 learned_addr learned my addr 172.21.15.45:0/2211889437 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd6476700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd01493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd6c77700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcd015e030 0x7fdcd01542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd5c75700 1 -- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 msgr2=0x7fdcd01493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd5c75700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd01493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd5c75700 1 -- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcd015e030 msgr2=0x7fdcd01542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd5c75700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcd015e030 0x7fdcd01542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.939+0000 7fdcd5c75700 1 -- 172.21.15.45:0/2211889437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc009550 con 0x7fdcd015a0f0 2023-08-27T11:25:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.940+0000 7fdcd5c75700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd015a0f0 0x7fdcd0153da0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdccc014040 tx=0x7fdccc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.940+0000 7fdcc6ffd700 1 -- 172.21.15.45:0/2211889437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004510 con 0x7fdcd015a0f0 2023-08-27T11:25:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.940+0000 7fdcc6ffd700 1 -- 172.21.15.45:0/2211889437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc01e070 con 0x7fdcd015a0f0 2023-08-27T11:25:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.940+0000 7fdcc6ffd700 1 -- 172.21.15.45:0/2211889437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0038e0 con 0x7fdcd015a0f0 2023-08-27T11:25:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.940+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd0154b90 con 0x7fdcd015a0f0 2023-08-27T11:25:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.940+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd00b2ef0 con 0x7fdcd015a0f0 2023-08-27T11:25:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.941+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd00b36d0 con 0x7fdcd015a0f0 2023-08-27T11:25:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.941+0000 7fdcc6ffd700 1 -- 172.21.15.45:0/2211889437 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fdccc003a40 con 0x7fdcd015a0f0 2023-08-27T11:25:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.942+0000 7fdcc6ffd700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdcb406a200 0x7fdcb406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:20.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.942+0000 7fdcc6ffd700 1 -- 172.21.15.45:0/2211889437 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fdccc089700 con 0x7fdcd015a0f0 2023-08-27T11:25:20.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.942+0000 7fdcd6476700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdcb406a200 0x7fdcb406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:20.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.942+0000 7fdcd6476700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdcb406a200 0x7fdcb406c6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0015230 tx=0x7fdcc000f040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:20.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:20.946+0000 7fdcc6ffd700 1 -- 172.21.15.45:0/2211889437 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdccc054b80 con 0x7fdcd015a0f0 2023-08-27T11:25:21.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.193+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcd00726d0 con 0x7fdcd015a0f0 2023-08-27T11:25:21.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.193+0000 7fdcc6ffd700 1 -- 172.21.15.45:0/2211889437 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4872 (secure 0 0 0) 0x7fdccc0581a0 con 0x7fdcd015a0f0 2023-08-27T11:25:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:21.195 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/72415568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":72415568},{"type":"v1","addr":"172.21.15.135:6837","nonce":72415568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2554143619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2554143619},{"type":"v1","addr":"172.21.15.135:6835","nonce":2554143619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6306,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/990560009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":990560009},{"type":"v1","addr":"172.21.15.45:6834","nonce":990560009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:10.472982+0000","modified":"2023-08-27T11:25:14.080477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6309":{"gid":6309,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6835/2659160949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2659160949},{"type":"v1","addr":"172.21.15.45:6835","nonce":2659160949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-27T11:25:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdcb406a200 msgr2=0x7fdcb406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdcb406a200 0x7fdcb406c6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0015230 tx=0x7fdcc000f040 comp rx=0 tx=0).stop 2023-08-27T11:25:21.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd015a0f0 msgr2=0x7fdcd0153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd015a0f0 0x7fdcd0153da0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdccc014040 tx=0x7fdccc004750 comp rx=0 tx=0).stop 2023-08-27T11:25:21.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 shutdown_connections 2023-08-27T11:25:21.197 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fdcb406a200 0x7fdcb406c6c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcd015e030 0x7fdcd01542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdcd00cd300 0x7fdcd01493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 --2- 172.21.15.45:0/2211889437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcd015a0f0 0x7fdcd0153da0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 >> 172.21.15.45:0/2211889437 conn(0x7fdcd00cc7e0 msgr2=0x7fdcd0070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 shutdown_connections 2023-08-27T11:25:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.195+0000 7fdcd86da700 1 -- 172.21.15.45:0/2211889437 wait complete. 2023-08-27T11:25:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-08-27T11:25:21.612 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:25:10.472982+0000', 'modified': '2023-08-27T11:25:14.080477+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6309}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6309': {'gid': 6309, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.45:6835/2659160949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 2659160949}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2659160949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:25:21.612 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:25:21.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-27T11:25:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.745+0000 7f676e13c700 1 Processor -- start 2023-08-27T11:25:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.746+0000 7f676e13c700 1 -- start start 2023-08-27T11:25:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.746+0000 7f676e13c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f676815d340 0x7f6768160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.746+0000 7f676e13c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f67680cd880 0x7f676806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.746+0000 7f676e13c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676806b160 con 0x7f67680cdd00 2023-08-27T11:25:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.746+0000 7f676e13c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6768072010 con 0x7f676815d340 2023-08-27T11:25:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.746+0000 7f676e13c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67680af590 con 0x7f67680cd880 2023-08-27T11:25:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.746+0000 7f67677fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f676815d340 0x7f6768160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.746+0000 7f67677fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f676815d340 0x7f6768160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56398/0 (socket says 172.21.15.45:56398) 2023-08-27T11:25:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f67677fe700 1 -- 172.21.15.45:0/2200315753 learned_addr learned my addr 172.21.15.45:0/2200315753 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f6766ffd700 1 --2- 172.21.15.45:0/2200315753 >> v2:172.21.15.135:3300/0 conn(0x7f67680cd880 0x7f676806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f67677fe700 1 -- 172.21.15.45:0/2200315753 >> v2:172.21.15.135:3300/0 conn(0x7f67680cd880 msgr2=0x7f676806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f67677fe700 1 --2- 172.21.15.45:0/2200315753 >> v2:172.21.15.135:3300/0 conn(0x7f67680cd880 0x7f676806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f67677fe700 1 -- 172.21.15.45:0/2200315753 >> v1:172.21.15.135:6789/0 conn(0x7f67680cdd00 legacy=0x7f67680b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f67677fe700 1 -- 172.21.15.45:0/2200315753 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67680af2b0 con 0x7f676815d340 2023-08-27T11:25:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f67677fe700 1 --2- 172.21.15.45:0/2200315753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676815d340 0x7f6768160d00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f67680cd7f0 tx=0x7f6750009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59811027ea9485f8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f67667fc700 1 -- 172.21.15.45:0/2200315753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750004230 con 0x7f676815d340 2023-08-27T11:25:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f67667fc700 1 -- 172.21.15.45:0/2200315753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6750004390 con 0x7f676815d340 2023-08-27T11:25:21.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.747+0000 7f67667fc700 1 -- 172.21.15.45:0/2200315753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500036a0 con 0x7f676815d340 2023-08-27T11:25:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.748+0000 7f676e13c700 1 -- 172.21.15.45:0/2200315753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676815d340 msgr2=0x7f6768160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.748+0000 7f676e13c700 1 --2- 172.21.15.45:0/2200315753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676815d340 0x7f6768160d00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f67680cd7f0 tx=0x7f6750009bd0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.748+0000 7f676e13c700 1 -- 172.21.15.45:0/2200315753 shutdown_connections 2023-08-27T11:25:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.748+0000 7f676e13c700 1 --2- 172.21.15.45:0/2200315753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f676815d340 0x7f6768160d00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.748+0000 7f676e13c700 1 --2- 172.21.15.45:0/2200315753 >> v2:172.21.15.135:3300/0 conn(0x7f67680cd880 0x7f676806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.748+0000 7f676e13c700 1 -- 172.21.15.45:0/2200315753 >> 172.21.15.45:0/2200315753 conn(0x7f67680cc7c0 msgr2=0x7f67680ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.748+0000 7f676e13c700 1 -- 172.21.15.45:0/2200315753 shutdown_connections 2023-08-27T11:25:21.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.748+0000 7f676e13c700 1 -- 172.21.15.45:0/2200315753 wait complete. 2023-08-27T11:25:21.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.749+0000 7f676e13c700 1 Processor -- start 2023-08-27T11:25:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.749+0000 7f676e13c700 1 -- start start 2023-08-27T11:25:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f676e13c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67680cd880 0x7f6768154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f676e13c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cdd00 0x7f67680c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f676e13c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f676815d340 0x7f6768147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f676e13c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67680c8630 con 0x7f67680cd880 2023-08-27T11:25:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f676e13c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67680c72c0 con 0x7f676815d340 2023-08-27T11:25:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f676e13c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67680c7590 con 0x7f67680cdd00 2023-08-27T11:25:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f67677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67680cd880 0x7f6768154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f67677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67680cd880 0x7f6768154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56404/0 (socket says 172.21.15.45:56404) 2023-08-27T11:25:21.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f67677fe700 1 -- 172.21.15.45:0/2597502130 learned_addr learned my addr 172.21.15.45:0/2597502130 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:21.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f6767fff700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f676815d340 0x7f6768147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f6766ffd700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cdd00 0x7f67680c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f67677fe700 1 -- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cdd00 msgr2=0x7f67680c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f67677fe700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cdd00 0x7f67680c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f67677fe700 1 -- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f676815d340 msgr2=0x7f6768147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f67677fe700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f676815d340 0x7f6768147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f67677fe700 1 -- 172.21.15.45:0/2597502130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6750009550 con 0x7f67680cd880 2023-08-27T11:25:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.750+0000 7f67677fe700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67680cd880 0x7f6768154fa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6750009130 tx=0x7f6750004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.751+0000 7f675ffff700 1 -- 172.21.15.45:0/2597502130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750011630 con 0x7f67680cd880 2023-08-27T11:25:21.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.751+0000 7f675ffff700 1 -- 172.21.15.45:0/2597502130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6750011790 con 0x7f67680cd880 2023-08-27T11:25:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.751+0000 7f676e13c700 1 -- 172.21.15.45:0/2597502130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67680c7870 con 0x7f67680cd880 2023-08-27T11:25:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.751+0000 7f675ffff700 1 -- 172.21.15.45:0/2597502130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750011b40 con 0x7f67680cd880 2023-08-27T11:25:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.751+0000 7f676e13c700 1 -- 172.21.15.45:0/2597502130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67680c7e30 con 0x7f67680cd880 2023-08-27T11:25:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.752+0000 7f675ffff700 1 -- 172.21.15.45:0/2597502130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6750011ca0 con 0x7f67680cd880 2023-08-27T11:25:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.753+0000 7f675ffff700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f674806a200 0x7f674806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.753+0000 7f675ffff700 1 -- 172.21.15.45:0/2597502130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f6750089df0 con 0x7f67680cd880 2023-08-27T11:25:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.753+0000 7f6766ffd700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f674806a200 0x7f674806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.753+0000 7f6766ffd700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f674806a200 0x7f674806c6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6768155990 tx=0x7f6758008040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.755+0000 7f677a2feb80 1 Processor -- start 2023-08-27T11:25:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.755+0000 7f677a2feb80 1 -- start start 2023-08-27T11:25:21.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.755+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f352b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.755+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f353c960 0x5566f3527b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.755+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f34fe3d0 0x5566f35280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.755+0000 7f677a2feb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f3459730 con 0x5566f352af40 2023-08-27T11:25:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.755+0000 7f677a2feb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f3457050 con 0x5566f353c960 2023-08-27T11:25:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.755+0000 7f677a2feb80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566f3452940 con 0x5566f34fe3d0 2023-08-27T11:25:21.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f67677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f352b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f67677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f352b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56412/0 (socket says 172.21.15.45:56412) 2023-08-27T11:25:21.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f67677fe700 1 -- 172.21.15.45:0/2875221602 learned_addr learned my addr 172.21.15.45:0/2875221602 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:21.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f67677fe700 1 -- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f34fe3d0 msgr2=0x5566f35280b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f6767fff700 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f34fe3d0 0x5566f35280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f6766ffd700 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f353c960 0x5566f3527b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f67677fe700 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f34fe3d0 0x5566f35280b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f67677fe700 1 -- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f353c960 msgr2=0x5566f3527b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f67677fe700 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f353c960 0x5566f3527b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f67677fe700 1 -- 172.21.15.45:0/2875221602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566f32d25e0 con 0x5566f352af40 2023-08-27T11:25:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f6767fff700 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f34fe3d0 0x5566f35280b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f6766ffd700 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f353c960 0x5566f3527b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f67677fe700 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f352b320 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6750060790 tx=0x7f6750058ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f69b3e70b9f946c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f675d7fa700 1 -- 172.21.15.45:0/2875221602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750070070 con 0x5566f352af40 2023-08-27T11:25:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f675d7fa700 1 -- 172.21.15.45:0/2875221602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6750057260 con 0x5566f352af40 2023-08-27T11:25:21.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.756+0000 7f675d7fa700 1 -- 172.21.15.45:0/2875221602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750058790 con 0x5566f352af40 2023-08-27T11:25:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 -- 172.21.15.45:0/2875221602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 msgr2=0x5566f352b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f352b320 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6750060790 tx=0x7f6750058ca0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 -- 172.21.15.45:0/2875221602 shutdown_connections 2023-08-27T11:25:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f352b320 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f34fe3d0 0x5566f35280b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2875221602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f353c960 0x5566f3527b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 -- 172.21.15.45:0/2875221602 >> 172.21.15.45:0/2875221602 conn(0x5566f345ec30 msgr2=0x5566f3545d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.770 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 -- 172.21.15.45:0/2875221602 shutdown_connections 2023-08-27T11:25:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 -- 172.21.15.45:0/2875221602 wait complete. 2023-08-27T11:25:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.757+0000 7f677a2feb80 1 -- start start 2023-08-27T11:25:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f677a2feb80 1 Processor -- start 2023-08-27T11:25:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 0x5566f352ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3521c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f3522150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f677a2feb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f33f0b60 con 0x5566f352af40 2023-08-27T11:25:21.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f677a2feb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f33f0860 con 0x5566f34fe3d0 2023-08-27T11:25:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f677a2feb80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566f32d48d0 con 0x5566f353c960 2023-08-27T11:25:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f677a2feb80 10 client.0 _ll_register_callbacks cb 0x5566f33ee630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:25:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f6766ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3521c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f6766ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3521c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56420/0 (socket says 172.21.15.45:56420) 2023-08-27T11:25:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.758+0000 7f6766ffd700 1 -- 172.21.15.45:0/3638940314 learned_addr learned my addr 172.21.15.45:0/3638940314 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f67677fe700 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 0x5566f352ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f6767fff700 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f3522150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f6766ffd700 1 -- 172.21.15.45:0/3638940314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 msgr2=0x5566f3522150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f6766ffd700 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f3522150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f6766ffd700 1 -- 172.21.15.45:0/3638940314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 msgr2=0x5566f352ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f6766ffd700 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 0x5566f352ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f6766ffd700 1 -- 172.21.15.45:0/3638940314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67680c7e30 con 0x5566f352af40 2023-08-27T11:25:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f6767fff700 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f3522150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f6766ffd700 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3521c10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f675800fea0 tx=0x7f67580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31543bab5ce2b7f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f67467fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:25:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f67467fc700 1 -- 172.21.15.45:0/3638940314 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758021610 con 0x5566f352af40 2023-08-27T11:25:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f67467fc700 1 -- 172.21.15.45:0/3638940314 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f6758021770 con 0x5566f352af40 2023-08-27T11:25:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f677a2feb80 1 -- 172.21.15.45:0/3638940314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566f344b920 con 0x5566f352af40 2023-08-27T11:25:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f67467fc700 1 -- 172.21.15.45:0/3638940314 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f675800fed0 con 0x5566f352af40 2023-08-27T11:25:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f677a2feb80 10 client.6483 fetch_fsmap learned FSMap version 26 2023-08-27T11:25:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f677a2feb80 1 -- 172.21.15.45:0/3638940314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5566f346a750 con 0x5566f352af40 2023-08-27T11:25:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f67467fc700 1 -- 172.21.15.45:0/3638940314 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5566f344b920 con 0x5566f352af40 2023-08-27T11:25:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f67467fc700 1 -- 172.21.15.45:0/3638940314 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6758021930 con 0x5566f352af40 2023-08-27T11:25:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f677a2feb80 10 client.6483 fetch_fsmap finished waiting for FSMap version 26 2023-08-27T11:25:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.759+0000 7f677a2feb80 10 client.6483 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:25:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f677a2feb80 20 client.6483 populate_metadata read hostname 'smithi045' 2023-08-27T11:25:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] conn(0x5566f352d6f0 0x5566f35628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f677a2feb80 4 client.6483 mds_command: new command op to 6309 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:25:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f677a2feb80 1 -- 172.21.15.45:0/3638940314 --> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566f340c000 con 0x5566f352d6f0 2023-08-27T11:25:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f67677fe700 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] conn(0x5566f352d6f0 0x5566f35628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f67677fe700 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] conn(0x5566f352d6f0 0x5566f35628b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab00d9daddee86c server_cookie=5312f1e9b5460774 in_seq=0 out_seq=0 2023-08-27T11:25:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f67467fc700 10 client.6483 ms_handle_connect on v2:172.21.15.45:6833/2659160949 2023-08-27T11:25:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f67467fc700 1 -- 172.21.15.45:0/3638940314 <== mds.0 v2:172.21.15.45:6833/2659160949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5566f340c000 con 0x5566f352d6f0 2023-08-27T11:25:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f67467fc700 10 client.6483 handle_command_reply: tid=0 2023-08-27T11:25:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.760+0000 7f677a2feb80 1 client.6483 shutdown 2023-08-27T11:25:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 -- 172.21.15.45:0/3638940314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 msgr2=0x5566f3521c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3521c10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f675800fea0 tx=0x7f67580047c0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 -- 172.21.15.45:0/3638940314 shutdown_connections 2023-08-27T11:25:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] conn(0x5566f352d6f0 0x5566f35628b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f3522150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 0x5566f352ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f67467fc700 0 client.6483 ms_handle_reset on v2:172.21.15.45:6833/2659160949 2023-08-27T11:25:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3638940314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3521c10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 -- 172.21.15.45:0/3638940314 >> 172.21.15.45:0/3638940314 conn(0x5566f345ec30 msgr2=0x5566f3524b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 -- 172.21.15.45:0/3638940314 shutdown_connections 2023-08-27T11:25:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 1 -- 172.21.15.45:0/3638940314 wait complete. 2023-08-27T11:25:21.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.761+0000 7f677a2feb80 20 client.6483 trim_cache size 0 max 16384 2023-08-27T11:25:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.778+0000 7f677a2feb80 1 Processor -- start 2023-08-27T11:25:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.779+0000 7f677a2feb80 1 -- start start 2023-08-27T11:25:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.779+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f34fe3d0 0x5566f35220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.779+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f352af40 0x5566f3521940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.779+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f352dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.779+0000 7f677a2feb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f352eff0 con 0x5566f34fe3d0 2023-08-27T11:25:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.779+0000 7f677a2feb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f3521410 con 0x5566f352af40 2023-08-27T11:25:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.779+0000 7f677a2feb80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566f3502860 con 0x5566f353c960 2023-08-27T11:25:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.779+0000 7f67677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f34fe3d0 0x5566f35220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f67677fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f34fe3d0 0x5566f35220d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56426/0 (socket says 172.21.15.45:56426) 2023-08-27T11:25:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f67677fe700 1 -- 172.21.15.45:0/3929875479 learned_addr learned my addr 172.21.15.45:0/3929875479 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f6767fff700 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f352dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f6766ffd700 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f352af40 0x5566f3521940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f67677fe700 1 -- 172.21.15.45:0/3929875479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 msgr2=0x5566f352dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f67677fe700 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f352dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f67677fe700 1 -- 172.21.15.45:0/3929875479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f352af40 msgr2=0x5566f3521940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f67677fe700 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f352af40 0x5566f3521940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f67677fe700 1 -- 172.21.15.45:0/3929875479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566f327ed90 con 0x5566f34fe3d0 2023-08-27T11:25:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f6766ffd700 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f352af40 0x5566f3521940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f67677fe700 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f34fe3d0 0x5566f35220d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6750055510 tx=0x7f675005c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1db5af688fb3c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f675dffb700 1 -- 172.21.15.45:0/3929875479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750070040 con 0x5566f34fe3d0 2023-08-27T11:25:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f675dffb700 1 -- 172.21.15.45:0/3929875479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6750058ba0 con 0x5566f34fe3d0 2023-08-27T11:25:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.780+0000 7f675dffb700 1 -- 172.21.15.45:0/3929875479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750077380 con 0x5566f34fe3d0 2023-08-27T11:25:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 -- 172.21.15.45:0/3929875479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f34fe3d0 msgr2=0x5566f35220d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f34fe3d0 0x5566f35220d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6750055510 tx=0x7f675005c000 comp rx=0 tx=0).stop 2023-08-27T11:25:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 -- 172.21.15.45:0/3929875479 shutdown_connections 2023-08-27T11:25:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f34fe3d0 0x5566f35220d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f352dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 --2- 172.21.15.45:0/3929875479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f352af40 0x5566f3521940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 -- 172.21.15.45:0/3929875479 >> 172.21.15.45:0/3929875479 conn(0x5566f345ec30 msgr2=0x5566f3501d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 -- 172.21.15.45:0/3929875479 shutdown_connections 2023-08-27T11:25:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 -- 172.21.15.45:0/3929875479 wait complete. 2023-08-27T11:25:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 -- start start 2023-08-27T11:25:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.781+0000 7f677a2feb80 1 Processor -- start 2023-08-27T11:25:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 0x5566f35ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3522b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f677a2feb80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f35230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f677a2feb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f3557990 con 0x5566f352af40 2023-08-27T11:25:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f677a2feb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566f35ac6d0 con 0x5566f34fe3d0 2023-08-27T11:25:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f677a2feb80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566f3521fe0 con 0x5566f353c960 2023-08-27T11:25:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f677a2feb80 10 client.0 _ll_register_callbacks cb 0x5566f33ee630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:25:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f6766ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3522b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f6766ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3522b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56434/0 (socket says 172.21.15.45:56434) 2023-08-27T11:25:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f6766ffd700 1 -- 172.21.15.45:0/2543687932 learned_addr learned my addr 172.21.15.45:0/2543687932 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.782+0000 7f67677fe700 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 0x5566f35ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f6766ffd700 1 -- 172.21.15.45:0/2543687932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 msgr2=0x5566f35230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f6767fff700 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f35230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f6766ffd700 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f35230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f6766ffd700 1 -- 172.21.15.45:0/2543687932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 msgr2=0x5566f35ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f6766ffd700 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 0x5566f35ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f6766ffd700 1 -- 172.21.15.45:0/2543687932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566f32d25e0 con 0x5566f352af40 2023-08-27T11:25:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f6767fff700 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f35230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f6766ffd700 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3522b80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6758000c00 tx=0x7f6758014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d77a248bca1aa93 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f67467fc700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:25:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f67467fc700 1 -- 172.21.15.45:0/2543687932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758029550 con 0x5566f352af40 2023-08-27T11:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f67467fc700 1 -- 172.21.15.45:0/2543687932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f6758021e40 con 0x5566f352af40 2023-08-27T11:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f677a2feb80 1 -- 172.21.15.45:0/2543687932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566f34f34f0 con 0x5566f352af40 2023-08-27T11:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f67467fc700 1 -- 172.21.15.45:0/2543687932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6758014e90 con 0x5566f352af40 2023-08-27T11:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f677a2feb80 10 client.6489 fetch_fsmap learned FSMap version 26 2023-08-27T11:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f677a2feb80 1 -- 172.21.15.45:0/2543687932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5566f3267c10 con 0x5566f352af40 2023-08-27T11:25:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f67467fc700 1 -- 172.21.15.45:0/2543687932 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5566f34f34f0 con 0x5566f352af40 2023-08-27T11:25:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f67467fc700 1 -- 172.21.15.45:0/2543687932 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67580296b0 con 0x5566f352af40 2023-08-27T11:25:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f677a2feb80 10 client.6489 fetch_fsmap finished waiting for FSMap version 26 2023-08-27T11:25:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f677a2feb80 10 client.6489 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-27T11:25:21.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f677a2feb80 20 client.6489 populate_metadata read hostname 'smithi045' 2023-08-27T11:25:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] conn(0x5566f35497b0 0x5566f3549b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f677a2feb80 4 client.6489 mds_command: new command op to 6309 tid=0[{"prefix": "status"}] 2023-08-27T11:25:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f677a2feb80 1 -- 172.21.15.45:0/2543687932 --> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5566f329ff20 con 0x5566f35497b0 2023-08-27T11:25:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.783+0000 7f67677fe700 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] conn(0x5566f35497b0 0x5566f3549b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f67467fc700 10 client.6489 ms_handle_connect on v2:172.21.15.45:6833/2659160949 2023-08-27T11:25:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f67677fe700 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] conn(0x5566f35497b0 0x5566f3549b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b39fc488b01d4c server_cookie=14130bc10c857f08 in_seq=0 out_seq=0 2023-08-27T11:25:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f67467fc700 1 -- 172.21.15.45:0/2543687932 <== mds.0 v2:172.21.15.45:6833/2659160949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5566f329ff20 con 0x5566f35497b0 2023-08-27T11:25:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f67467fc700 10 client.6489 handle_command_reply: tid=0 2023-08-27T11:25:21.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 client.6489 shutdown 2023-08-27T11:25:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 -- 172.21.15.45:0/2543687932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 msgr2=0x5566f3522b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3522b80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6758000c00 tx=0x7f6758014000 comp rx=0 tx=0).stop 2023-08-27T11:25:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 -- 172.21.15.45:0/2543687932 shutdown_connections 2023-08-27T11:25:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.45:6833/2659160949,v1:172.21.15.45:6835/2659160949] conn(0x5566f35497b0 0x5566f3549b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x5566f353c960 0x5566f35230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f67467fc700 0 client.6489 ms_handle_reset on v2:172.21.15.45:6833/2659160949 2023-08-27T11:25:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5566f34fe3d0 0x5566f35ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 --2- 172.21.15.45:0/2543687932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566f352af40 0x5566f3522b80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 -- 172.21.15.45:0/2543687932 >> 172.21.15.45:0/2543687932 conn(0x5566f345ec30 msgr2=0x5566f352dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 -- 172.21.15.45:0/2543687932 shutdown_connections 2023-08-27T11:25:21.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 1 -- 172.21.15.45:0/2543687932 wait complete. 2023-08-27T11:25:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.784+0000 7f677a2feb80 20 client.6489 trim_cache size 0 max 16384 2023-08-27T11:25:21.812 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:25:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "8ea1ae34-1697-4b79-b2ce-3f741cd3c828", 2023-08-27T11:25:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-08-27T11:25:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6309, 2023-08-27T11:25:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-08-27T11:25:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-08-27T11:25:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-08-27T11:25:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.715980373000001, 2023-08-27T11:25:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2023-08-27T11:25:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 70, 2023-08-27T11:25:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 70, 2023-08-27T11:25:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 23.079238693000001 2023-08-27T11:25:21.815 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:25:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 -- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f674806a200 msgr2=0x7f674806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f674806a200 0x7f674806c6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6768155990 tx=0x7f6758008040 comp rx=0 tx=0).stop 2023-08-27T11:25:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 -- 172.21.15.45:0/2597502130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67680cd880 msgr2=0x7f6768154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67680cd880 0x7f6768154fa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6750009130 tx=0x7f6750004750 comp rx=0 tx=0).stop 2023-08-27T11:25:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 -- 172.21.15.45:0/2597502130 shutdown_connections 2023-08-27T11:25:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f674806a200 0x7f674806c6c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67680cd880 0x7f6768154fa0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f676815d340 0x7f6768147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 --2- 172.21.15.45:0/2597502130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f67680cdd00 0x7f67680c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 -- 172.21.15.45:0/2597502130 >> 172.21.15.45:0/2597502130 conn(0x7f67680cc7c0 msgr2=0x7f676815e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 -- 172.21.15.45:0/2597502130 shutdown_connections 2023-08-27T11:25:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.785+0000 7f676e13c700 1 -- 172.21.15.45:0/2597502130 wait complete. 2023-08-27T11:25:21.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:25:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.944+0000 7f4222916700 1 Processor -- start 2023-08-27T11:25:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.944+0000 7f4222916700 1 -- start start 2023-08-27T11:25:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f4222916700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f421c15d450 0x7f421c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f4222916700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f421c0d0440 0x7f421c161250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f4222916700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421c0ce1c0 con 0x7f421c0cbaf0 2023-08-27T11:25:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f4222916700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421c0cf290 con 0x7f421c15d450 2023-08-27T11:25:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f4222916700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0ce440 con 0x7f421c0d0440 2023-08-27T11:25:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f421c0cbaf0 0x7f421c0cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43216/0 (socket says 172.21.15.45:43216) 2023-08-27T11:25:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421b7fe700 1 -- 172.21.15.45:0/665044914 learned_addr learned my addr 172.21.15.45:0/665044914 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f4220eb3700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f421c0d0440 0x7f421c161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421bfff700 1 --2- 172.21.15.45:0/665044914 >> v2:172.21.15.135:3300/0 conn(0x7f421c15d450 0x7f421c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171016867 0 0) 0x7f421c0ce1c0 con 0x7f421c0cbaf0 2023-08-27T11:25:21.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42080035b0 con 0x7f421c0cbaf0 2023-08-27T11:25:21.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 276352509 0 0) 0x7f42080035b0 con 0x7f421c0cbaf0 2023-08-27T11:25:21.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f421c0d0440 msgr2=0x7f421c161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f421c0d0440 0x7f421c161250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 >> v2:172.21.15.135:3300/0 conn(0x7f421c15d450 msgr2=0x7f421c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 --2- 172.21.15.45:0/665044914 >> v2:172.21.15.135:3300/0 conn(0x7f421c15d450 0x7f421c0cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c0ce720 con 0x7f421c0cbaf0 2023-08-27T11:25:21.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f4204001670 con 0x7f421c0cbaf0 2023-08-27T11:25:21.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.945+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 >> v1:172.21.15.45:6789/0 conn(0x7f421c0cbaf0 legacy=0x7f421c0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421affd700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4208003a00 0x7f4208003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421affd700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208004850 0x7f4208004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421affd700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f421c0d0440 0x7f4208005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0cf290 con 0x7f4208004850 2023-08-27T11:25:21.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0ce440 con 0x7f4208003a00 2023-08-27T11:25:21.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420800ba40 con 0x7f421c0d0440 2023-08-27T11:25:21.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421b7fe700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208004850 0x7f4208004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421bfff700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4208003a00 0x7f4208003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f4220eb3700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f421c0d0440 0x7f4208005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421b7fe700 1 -- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f421c0d0440 msgr2=0x7f4208005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421b7fe700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f421c0d0440 0x7f4208005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421b7fe700 1 -- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4208003a00 msgr2=0x7f4208003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421b7fe700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4208003a00 0x7f4208003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421b7fe700 1 -- 172.21.15.45:0/665044914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42080035b0 con 0x7f4208004850 2023-08-27T11:25:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421b7fe700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208004850 0x7f4208004cd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f421c0cbed0 tx=0x7f4208005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7837a512b990340d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204007c60 con 0x7f4208004850 2023-08-27T11:25:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420400ce20 con 0x7f4208004850 2023-08-27T11:25:21.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.946+0000 7f421affd700 1 -- 172.21.15.45:0/665044914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204005ba0 con 0x7f4208004850 2023-08-27T11:25:21.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.947+0000 7f4222916700 1 -- 172.21.15.45:0/665044914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208004850 msgr2=0x7f4208004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.947+0000 7f4222916700 1 --2- 172.21.15.45:0/665044914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208004850 0x7f4208004cd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f421c0cbed0 tx=0x7f4208005880 comp rx=0 tx=0).stop 2023-08-27T11:25:21.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.947+0000 7f4220eb3700 1 -- 172.21.15.45:0/665044914 reap_dead start 2023-08-27T11:25:21.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.947+0000 7f4222916700 1 -- 172.21.15.45:0/665044914 shutdown_connections 2023-08-27T11:25:21.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.947+0000 7f4222916700 1 -- 172.21.15.45:0/665044914 >> 172.21.15.45:0/665044914 conn(0x7f421c0d0f60 msgr2=0x7f421c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:21.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.947+0000 7f4222916700 1 -- 172.21.15.45:0/665044914 shutdown_connections 2023-08-27T11:25:21.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.947+0000 7f4222916700 1 -- 172.21.15.45:0/665044914 wait complete. 2023-08-27T11:25:21.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.948+0000 7f4222916700 1 Processor -- start 2023-08-27T11:25:21.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.949+0000 7f4222916700 1 -- start start 2023-08-27T11:25:21.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4222916700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4208004850 0x7f421c1f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4222916700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f421c0ca380 0x7f421c1f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4222916700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f421c1f6a40 0x7f421c150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4222916700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c1f6ec0 con 0x7f421c0ca380 2023-08-27T11:25:21.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4222916700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c1f4020 con 0x7f421c1f6a40 2023-08-27T11:25:21.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4222916700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c1f73c0 con 0x7f4208004850 2023-08-27T11:25:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f421b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f421c0ca380 0x7f421c1f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f421b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f421c0ca380 0x7f421c1f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56454/0 (socket says 172.21.15.45:56454) 2023-08-27T11:25:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f421b7fe700 1 -- 172.21.15.45:0/393114778 learned_addr learned my addr 172.21.15.45:0/393114778 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:21.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f421b7fe700 1 -- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4208004850 msgr2=0x7f421c1f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f421b7fe700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4208004850 0x7f421c1f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f421b7fe700 1 -- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f421c1f6a40 msgr2=0x7f421c150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:21.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f421b7fe700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f421c1f6a40 0x7f421c150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f421b7fe700 1 -- 172.21.15.45:0/393114778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c1f39f0 con 0x7f421c0ca380 2023-08-27T11:25:21.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f421b7fe700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f421c0ca380 0x7f421c1f6500 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4204001820 tx=0x7f4204000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4218ff9700 1 -- 172.21.15.45:0/393114778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204004e60 con 0x7f421c0ca380 2023-08-27T11:25:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4218ff9700 1 -- 172.21.15.45:0/393114778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42040070d0 con 0x7f421c0ca380 2023-08-27T11:25:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4218ff9700 1 -- 172.21.15.45:0/393114778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204007480 con 0x7f421c0ca380 2023-08-27T11:25:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421c0b60e0 con 0x7f421c0ca380 2023-08-27T11:25:21.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.950+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421c1f3cb0 con 0x7f421c0ca380 2023-08-27T11:25:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.951+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421c0b66c0 con 0x7f421c0ca380 2023-08-27T11:25:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.952+0000 7f4218ff9700 1 -- 172.21.15.45:0/393114778 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f42040075e0 con 0x7f421c0ca380 2023-08-27T11:25:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.952+0000 7f4218ff9700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f41fc06a230 0x7f41fc06c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.952+0000 7f4218ff9700 1 -- 172.21.15.45:0/393114778 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4204089d80 con 0x7f421c0ca380 2023-08-27T11:25:21.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.952+0000 7f421bfff700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f41fc06a230 0x7f41fc06c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.953+0000 7f421bfff700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f41fc06a230 0x7f41fc06c6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f420c006ba0 tx=0x7f420c006b30 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:21.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:21.956+0000 7f4218ff9700 1 -- 172.21.15.45:0/393114778 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4204055280 con 0x7f421c0ca380 2023-08-27T11:25:22.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.200+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f421c1f4160 con 0x7f421c0ca380 2023-08-27T11:25:22.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.201+0000 7f4218ff9700 1 -- 172.21.15.45:0/393114778 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f421c1f4160 con 0x7f421c0ca380 2023-08-27T11:25:22.203 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:25:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f41fc06a230 msgr2=0x7f41fc06c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f41fc06a230 0x7f41fc06c6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f420c006ba0 tx=0x7f420c006b30 comp rx=0 tx=0).stop 2023-08-27T11:25:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f421c0ca380 msgr2=0x7f421c1f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f421c0ca380 0x7f421c1f6500 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4204001820 tx=0x7f4204000fe0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 shutdown_connections 2023-08-27T11:25:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f41fc06a230 0x7f41fc06c6f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f421c1f6a40 0x7f421c150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4208004850 0x7f421c1f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 --2- 172.21.15.45:0/393114778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f421c0ca380 0x7f421c1f6500 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 >> 172.21.15.45:0/393114778 conn(0x7f421c0d0f60 msgr2=0x7f421c15af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 shutdown_connections 2023-08-27T11:25:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:22.203+0000 7f4222916700 1 -- 172.21.15.45:0/393114778 wait complete. 2023-08-27T11:25:22.217 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:25:22.217 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:25:22.217 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-08-27T11:25:22.218 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:25:22.218 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:22.218 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:25:22.219 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:25:22.219 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:25:22.219 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:25:22.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.220 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-08-27T11:25:22.238 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:25:22.238 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:25:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:25:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:25:22.239 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:22.239 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:25:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:22.240 INFO:teuthology.orchestra.run.smithi045.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:25:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-27T11:25:22.240 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41619sec preferred_lft 41619sec 2023-08-27T11:25:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-08-27T11:25:22.241 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:22.241 INFO:teuthology.orchestra.run.smithi045.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:25:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:22.242 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:25:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:22.242 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:25:22.242 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:25:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:25:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a41d:a1ff:fed9:3bf2/64 scope link 2023-08-27T11:25:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:22.243 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:25:22.243 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether a6:e4:29:97:ce:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:25:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a4e4:29ff:fe97:ced1/64 scope link 2023-08-27T11:25:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:22.244 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:25:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:25:22.244 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c84:f3ff:fe1a:a84/64 scope link 2023-08-27T11:25:22.245 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:25:22.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.246 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:25:22.304 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-08-27T11:25:22.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.306 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-08-27T11:25:22.365 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-08-27T11:25:22.366 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:25:22.366 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-27T11:25:22.366 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-27T11:25:22.366 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:25:22.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.367 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:25:22.368 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:25:22.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-27T11:25:22.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-27T11:25:22.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-27T11:25:22.369 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:25:22.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.473 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:25:22.490 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:25:22.491 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:25:22.491 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:25:22.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-08-27T11:25:22.558 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:25:22.558 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:22.615 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:25:22.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:25:22.616 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:22.672 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:25:22.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:25:22.759 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:25:22.759 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:25:22.760 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:25:22.768 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:25:22.769 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:25:22.769 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:25:22.769 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:25:22.769 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:25:22.813 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:25:22.814 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:25:22.814 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:25:22.814 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:25:22.814 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:25:22.814 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790" 2023-08-27T11:25:22.815 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2023-08-27T11:25:22.815 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8ea1ae34-1697-4b79-b2ce-3f741cd3c828" will pass to kernel 2023-08-27T11:25:22.829 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:25:22.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:25:22.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:22.857 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:25:22.940 INFO:teuthology.orchestra.run.smithi045.stdout:client6124 2023-08-27T11:25:22.941 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:25:22.941 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:25:22.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:25:23.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.165+0000 7fbc155a0700 1 Processor -- start 2023-08-27T11:25:23.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.165+0000 7fbc155a0700 1 -- start start 2023-08-27T11:25:23.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.166+0000 7fbc155a0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbc100cfb20 0x7fbc100cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.166+0000 7fbc155a0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc100cd780 con 0x7fbc100cb2d0 2023-08-27T11:25:23.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.166+0000 7fbc155a0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc100ccb70 con 0x7fbc1015e3f0 2023-08-27T11:25:23.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.166+0000 7fbc155a0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1015e7d0 con 0x7fbc100cfb20 2023-08-27T11:25:23.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.166+0000 7fbc0f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbc100cb2d0 0x7fbc100cdc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43240/0 (socket says 172.21.15.45:43240) 2023-08-27T11:25:23.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.166+0000 7fbc0f7fe700 1 -- 172.21.15.45:0/930844862 learned_addr learned my addr 172.21.15.45:0/930844862 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:23.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.166+0000 7fbc14d9f700 1 --2- 172.21.15.45:0/930844862 >> v2:172.21.15.135:3300/0 conn(0x7fbc100cfb20 0x7fbc100cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc0effd700 1 -- 172.21.15.45:0/930844862 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846462634 0 0) 0x7fbc100cd780 con 0x7fbc100cb2d0 2023-08-27T11:25:23.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc0effd700 1 -- 172.21.15.45:0/930844862 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbf80035b0 con 0x7fbc100cb2d0 2023-08-27T11:25:23.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc0effd700 1 -- 172.21.15.45:0/930844862 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734497038 0 0) 0x7fbc100ccb70 con 0x7fbc1015e3f0 2023-08-27T11:25:23.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc0effd700 1 -- 172.21.15.45:0/930844862 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc100cd780 con 0x7fbc1015e3f0 2023-08-27T11:25:23.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc14d9f700 1 -- 172.21.15.45:0/930844862 >> v1:172.21.15.135:6789/0 conn(0x7fbc1015e3f0 legacy=0x7fbc100b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc14d9f700 1 -- 172.21.15.45:0/930844862 >> v1:172.21.15.45:6789/0 conn(0x7fbc100cb2d0 legacy=0x7fbc100cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc14d9f700 1 -- 172.21.15.45:0/930844862 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc100cce50 con 0x7fbc100cfb20 2023-08-27T11:25:23.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc14d9f700 1 --2- 172.21.15.45:0/930844862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc100cfb20 0x7fbc100cffe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009d90 tx=0x7fbc0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14d2deb56ceb97fa server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc0effd700 1 -- 172.21.15.45:0/930844862 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800e070 con 0x7fbc100cfb20 2023-08-27T11:25:23.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc0effd700 1 -- 172.21.15.45:0/930844862 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0800bb20 con 0x7fbc100cfb20 2023-08-27T11:25:23.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.167+0000 7fbc0effd700 1 -- 172.21.15.45:0/930844862 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08012430 con 0x7fbc100cfb20 2023-08-27T11:25:23.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.168+0000 7fbc155a0700 1 -- 172.21.15.45:0/930844862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc100cfb20 msgr2=0x7fbc100cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.168+0000 7fbc155a0700 1 --2- 172.21.15.45:0/930844862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc100cfb20 0x7fbc100cffe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009d90 tx=0x7fbc0800b330 comp rx=0 tx=0).stop 2023-08-27T11:25:23.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.168+0000 7fbc155a0700 1 -- 172.21.15.45:0/930844862 shutdown_connections 2023-08-27T11:25:23.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.168+0000 7fbc155a0700 1 --2- 172.21.15.45:0/930844862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc100cfb20 0x7fbc100cffe0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.168+0000 7fbc155a0700 1 -- 172.21.15.45:0/930844862 >> 172.21.15.45:0/930844862 conn(0x7fbc1006dd40 msgr2=0x7fbc1006e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:23.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.168+0000 7fbc155a0700 1 -- 172.21.15.45:0/930844862 shutdown_connections 2023-08-27T11:25:23.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.168+0000 7fbc155a0700 1 -- 172.21.15.45:0/930844862 wait complete. 2023-08-27T11:25:23.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.169+0000 7fbc155a0700 1 Processor -- start 2023-08-27T11:25:23.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.169+0000 7fbc155a0700 1 -- start start 2023-08-27T11:25:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc155a0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc100cb2d0 0x7fbc10153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc155a0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc100cfb20 0x7fbc100b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc155a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1015e3f0 0x7fbc100b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc155a0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10154280 con 0x7fbc1015e3f0 2023-08-27T11:25:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc155a0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10151580 con 0x7fbc100cb2d0 2023-08-27T11:25:23.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc155a0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1015c9b0 con 0x7fbc100cfb20 2023-08-27T11:25:23.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc14d9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1015e3f0 0x7fbc100b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc14d9f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1015e3f0 0x7fbc100b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56474/0 (socket says 172.21.15.45:56474) 2023-08-27T11:25:23.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc14d9f700 1 -- 172.21.15.45:0/2704496935 learned_addr learned my addr 172.21.15.45:0/2704496935 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:23.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc0f7fe700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc100cfb20 0x7fbc100b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc0ffff700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc100cb2d0 0x7fbc10153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc14d9f700 1 -- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc100cfb20 msgr2=0x7fbc100b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc14d9f700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc100cfb20 0x7fbc100b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc14d9f700 1 -- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc100cb2d0 msgr2=0x7fbc10153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc14d9f700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc100cb2d0 0x7fbc10153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.170+0000 7fbc14d9f700 1 -- 172.21.15.45:0/2704496935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08009a70 con 0x7fbc1015e3f0 2023-08-27T11:25:23.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.171+0000 7fbc14d9f700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1015e3f0 0x7fbc100b4980 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbc08013000 tx=0x7fbc08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.171+0000 7fbc0cff9700 1 -- 172.21.15.45:0/2704496935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08012430 con 0x7fbc1015e3f0 2023-08-27T11:25:23.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.171+0000 7fbc0cff9700 1 -- 172.21.15.45:0/2704496935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc08020070 con 0x7fbc1015e3f0 2023-08-27T11:25:23.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.171+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1015cc90 con 0x7fbc1015e3f0 2023-08-27T11:25:23.180 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.171+0000 7fbc0cff9700 1 -- 172.21.15.45:0/2704496935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0801ba70 con 0x7fbc1015e3f0 2023-08-27T11:25:23.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.171+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1015d250 con 0x7fbc1015e3f0 2023-08-27T11:25:23.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.172+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1004ee70 con 0x7fbc1015e3f0 2023-08-27T11:25:23.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.172+0000 7fbc0cff9700 1 -- 172.21.15.45:0/2704496935 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbc0801bbd0 con 0x7fbc1015e3f0 2023-08-27T11:25:23.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.173+0000 7fbc0cff9700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbbfc06a200 0x7fbbfc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.173+0000 7fbc0cff9700 1 -- 172.21.15.45:0/2704496935 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fbc08022070 con 0x7fbc1015e3f0 2023-08-27T11:25:23.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.173+0000 7fbc0ffff700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbbfc06a200 0x7fbbfc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.173+0000 7fbc0ffff700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbbfc06a200 0x7fbbfc06c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc000011e0 tx=0x7fbc00009040 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.177+0000 7fbc0cff9700 1 -- 172.21.15.45:0/2704496935 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbc08056040 con 0x7fbc1015e3f0 2023-08-27T11:25:23.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.404+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 --> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc100cd410 con 0x7fbbfc06a200 2023-08-27T11:25:23.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.406+0000 7fbc0cff9700 1 -- 172.21.15.45:0/2704496935 <== mgr.5402 v2:172.21.15.135:6832/3181978685 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbc00001cd0 con 0x7fbbfc06a200 2023-08-27T11:25:23.407 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:25:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:25:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephfs" 2023-08-27T11:25:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:25:23.408 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:25:23.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbbfc06a200 msgr2=0x7fbbfc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbbfc06a200 0x7fbbfc06c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc000011e0 tx=0x7fbc00009040 comp rx=0 tx=0).stop 2023-08-27T11:25:23.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1015e3f0 msgr2=0x7fbc100b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1015e3f0 0x7fbc100b4980 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbc08013000 tx=0x7fbc08004ad0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 shutdown_connections 2023-08-27T11:25:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fbbfc06a200 0x7fbbfc06c6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1015e3f0 0x7fbc100b4980 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc100cb2d0 0x7fbc10153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 --2- 172.21.15.45:0/2704496935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbc100cfb20 0x7fbc100b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 >> 172.21.15.45:0/2704496935 conn(0x7fbc1006dd40 msgr2=0x7fbc10150d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 shutdown_connections 2023-08-27T11:25:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.408+0000 7fbc155a0700 1 -- 172.21.15.45:0/2704496935 wait complete. 2023-08-27T11:25:23.421 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-27T11:25:23.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.547+0000 7f28664ea700 1 Processor -- start 2023-08-27T11:25:23.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.547+0000 7f28664ea700 1 -- start start 2023-08-27T11:25:23.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.548+0000 7f28664ea700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f28600ca620 0x7f28600cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.548+0000 7f28664ea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600caa00 0x7f28600acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.548+0000 7f28664ea700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28600ce340 con 0x7f28600af200 2023-08-27T11:25:23.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.548+0000 7f28664ea700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28600ceec0 con 0x7f28600ca620 2023-08-27T11:25:23.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.548+0000 7f28664ea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286015d7c0 con 0x7f28600caa00 2023-08-27T11:25:23.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.548+0000 7f285f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f28600af200 0x7f28600ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43254/0 (socket says 172.21.15.45:43254) 2023-08-27T11:25:23.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.548+0000 7f285f7fe700 1 -- 172.21.15.45:0/3517746243 learned_addr learned my addr 172.21.15.45:0/3517746243 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:23.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.548+0000 7f2864a87700 1 --2- 172.21.15.45:0/3517746243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600caa00 0x7f28600acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f285ffff700 1 --2- 172.21.15.45:0/3517746243 >> v2:172.21.15.135:3300/0 conn(0x7f28600ca620 0x7f28600cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f285effd700 1 -- 172.21.15.45:0/3517746243 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188446770 0 0) 0x7f28600ce340 con 0x7f28600af200 2023-08-27T11:25:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f285effd700 1 -- 172.21.15.45:0/3517746243 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f284c0035b0 con 0x7f28600af200 2023-08-27T11:25:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f2864a87700 1 -- 172.21.15.45:0/3517746243 >> v2:172.21.15.135:3300/0 conn(0x7f28600ca620 msgr2=0x7f28600cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f2864a87700 1 --2- 172.21.15.45:0/3517746243 >> v2:172.21.15.135:3300/0 conn(0x7f28600ca620 0x7f28600cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f2864a87700 1 -- 172.21.15.45:0/3517746243 >> v1:172.21.15.45:6789/0 conn(0x7f28600af200 legacy=0x7f28600ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f285effd700 1 -- 172.21.15.45:0/3517746243 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4181868670 0 0) 0x7f284c0035b0 con 0x7f28600af200 2023-08-27T11:25:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f2864a87700 1 -- 172.21.15.45:0/3517746243 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28600cc570 con 0x7f28600caa00 2023-08-27T11:25:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f285ffff700 1 --2- 172.21.15.45:0/3517746243 >> v2:172.21.15.135:3300/0 conn(0x7f28600ca620 0x7f28600cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:25:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f2864a87700 1 --2- 172.21.15.45:0/3517746243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600caa00 0x7f28600acfc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2858009d90 tx=0x7f285800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3675ad71f62931 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f285effd700 1 -- 172.21.15.45:0/3517746243 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800e070 con 0x7f28600caa00 2023-08-27T11:25:23.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f285effd700 1 -- 172.21.15.45:0/3517746243 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2858003a20 con 0x7f28600caa00 2023-08-27T11:25:23.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.549+0000 7f285effd700 1 -- 172.21.15.45:0/3517746243 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850001670 con 0x7f28600caa00 2023-08-27T11:25:23.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.550+0000 7f28664ea700 1 -- 172.21.15.45:0/3517746243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600caa00 msgr2=0x7f28600acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.550+0000 7f28664ea700 1 --2- 172.21.15.45:0/3517746243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600caa00 0x7f28600acfc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2858009d90 tx=0x7f285800b330 comp rx=0 tx=0).stop 2023-08-27T11:25:23.555 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.550+0000 7f28664ea700 1 -- 172.21.15.45:0/3517746243 shutdown_connections 2023-08-27T11:25:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.550+0000 7f28664ea700 1 --2- 172.21.15.45:0/3517746243 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600caa00 0x7f28600acfc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.550+0000 7f28664ea700 1 --2- 172.21.15.45:0/3517746243 >> v2:172.21.15.135:3300/0 conn(0x7f28600ca620 0x7f28600cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.550+0000 7f28664ea700 1 -- 172.21.15.45:0/3517746243 >> 172.21.15.45:0/3517746243 conn(0x7f28600cc7e0 msgr2=0x7f28600ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.550+0000 7f28664ea700 1 -- 172.21.15.45:0/3517746243 shutdown_connections 2023-08-27T11:25:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.550+0000 7f28664ea700 1 -- 172.21.15.45:0/3517746243 wait complete. 2023-08-27T11:25:23.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.551+0000 7f28664ea700 1 Processor -- start 2023-08-27T11:25:23.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.551+0000 7f28664ea700 1 -- start start 2023-08-27T11:25:23.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f28664ea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600af200 0x7f2860156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f28664ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28600ca620 0x7f28601f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f28664ea700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600caa00 0x7f2860158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f28664ea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28601f5750 con 0x7f28600ca620 2023-08-27T11:25:23.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f28664ea700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28601593f0 con 0x7f28600caa00 2023-08-27T11:25:23.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f28664ea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2860158b50 con 0x7f28600af200 2023-08-27T11:25:23.560 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f285f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28600ca620 0x7f28601f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f285f7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28600ca620 0x7f28601f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56486/0 (socket says 172.21.15.45:56486) 2023-08-27T11:25:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f285f7fe700 1 -- 172.21.15.45:0/3932411937 learned_addr learned my addr 172.21.15.45:0/3932411937 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f285ffff700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600af200 0x7f2860156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f2864a87700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600caa00 0x7f2860158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f285f7fe700 1 -- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600af200 msgr2=0x7f2860156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f285f7fe700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600af200 0x7f2860156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f285f7fe700 1 -- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600caa00 msgr2=0x7f2860158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f285f7fe700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600caa00 0x7f2860158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.552+0000 7f285f7fe700 1 -- 172.21.15.45:0/3932411937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480094a0 con 0x7f28600ca620 2023-08-27T11:25:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.553+0000 7f285f7fe700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28600ca620 0x7f28601f5210 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f285000c730 tx=0x7f285000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.553+0000 7f285cff9700 1 -- 172.21.15.45:0/3932411937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850004c80 con 0x7f28600ca620 2023-08-27T11:25:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.553+0000 7f285cff9700 1 -- 172.21.15.45:0/3932411937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2850004de0 con 0x7f28600ca620 2023-08-27T11:25:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.553+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2858009a70 con 0x7f28600ca620 2023-08-27T11:25:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.553+0000 7f285cff9700 1 -- 172.21.15.45:0/3932411937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850007310 con 0x7f28600ca620 2023-08-27T11:25:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.553+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2860159110 con 0x7f28600ca620 2023-08-27T11:25:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.554+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28600cbce0 con 0x7f28600ca620 2023-08-27T11:25:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.554+0000 7f285cff9700 1 -- 172.21.15.45:0/3932411937 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2850007470 con 0x7f28600ca620 2023-08-27T11:25:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.555+0000 7f285cff9700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f284006a200 0x7f284006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.555+0000 7f285cff9700 1 -- 172.21.15.45:0/3932411937 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2850088560 con 0x7f28600ca620 2023-08-27T11:25:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.555+0000 7f285ffff700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f284006a200 0x7f284006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:23.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.555+0000 7f285ffff700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f284006a200 0x7f284006c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2860155a90 tx=0x7f2848009210 comp rx=0 tx=0).ready entity=mgr.5402 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:23.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.559+0000 7f285cff9700 1 -- 172.21.15.45:0/3932411937 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2850053a60 con 0x7f28600ca620 2023-08-27T11:25:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:23.819+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f28600734a0 con 0x7f28600ca620 2023-08-27T11:25:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.117+0000 7f285cff9700 1 -- 172.21.15.45:0/3932411937 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 19) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f2850053e30 con 0x7f28600ca620 2023-08-27T11:25:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.118+0000 7f285cff9700 1 -- 172.21.15.45:0/3932411937 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2850053a60 con 0x7f28600ca620 2023-08-27T11:25:24.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f284006a200 msgr2=0x7f284006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f284006a200 0x7f284006c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2860155a90 tx=0x7f2848009210 comp rx=0 tx=0).stop 2023-08-27T11:25:24.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28600ca620 msgr2=0x7f28601f5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28600ca620 0x7f28601f5210 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f285000c730 tx=0x7f285000ca40 comp rx=0 tx=0).stop 2023-08-27T11:25:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 shutdown_connections 2023-08-27T11:25:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7f284006a200 0x7f284006c6c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28600caa00 0x7f2860158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28600af200 0x7f2860156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 --2- 172.21.15.45:0/3932411937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28600ca620 0x7f28601f5210 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 >> 172.21.15.45:0/3932411937 conn(0x7f28600cc7e0 msgr2=0x7f28600adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 shutdown_connections 2023-08-27T11:25:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.121+0000 7f28664ea700 1 -- 172.21.15.45:0/3932411937 wait complete. 2023-08-27T11:25:24.136 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-27T11:25:24.263 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:24.261+0000 7ff1d5722200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:25:24.266 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:24.265+0000 7f50f44e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:25:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.269+0000 7fe7d32bc700 1 Processor -- start 2023-08-27T11:25:24.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.269+0000 7fe7d32bc700 1 -- start start 2023-08-27T11:25:24.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d32bc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe7cc15a0f0 0x7fe7cc0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d32bc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc06ed90 0x7fe7cc15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d32bc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0cfae0 con 0x7fe7cc0cd300 2023-08-27T11:25:24.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d32bc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc15e900 con 0x7fe7cc15a0f0 2023-08-27T11:25:24.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d32bc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc0cae60 con 0x7fe7cc06ed90 2023-08-27T11:25:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d1058700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe7cc15a0f0 0x7fe7cc0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d0857700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe7cc0cd300 0x7fe7cc15daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43294/0 (socket says 172.21.15.45:43294) 2023-08-27T11:25:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d0857700 1 -- 172.21.15.45:0/340990058 learned_addr learned my addr 172.21.15.45:0/340990058 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:24.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d1859700 1 --2- 172.21.15.45:0/340990058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc06ed90 0x7fe7cc15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d1058700 1 -- 172.21.15.45:0/340990058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc06ed90 msgr2=0x7fe7cc15e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d1058700 1 --2- 172.21.15.45:0/340990058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc06ed90 0x7fe7cc15e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d1058700 1 -- 172.21.15.45:0/340990058 >> v1:172.21.15.45:6789/0 conn(0x7fe7cc0cd300 legacy=0x7fe7cc15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.270+0000 7fe7d1058700 1 -- 172.21.15.45:0/340990058 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc0cb140 con 0x7fe7cc15a0f0 2023-08-27T11:25:24.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7d1058700 1 --2- 172.21.15.45:0/340990058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc15a0f0 0x7fe7cc0cbc20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009840 tx=0x7fe7c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55752b70291a3ca1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7c3fff700 1 -- 172.21.15.45:0/340990058 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8011830 con 0x7fe7cc15a0f0 2023-08-27T11:25:24.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7c3fff700 1 -- 172.21.15.45:0/340990058 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c8011990 con 0x7fe7cc15a0f0 2023-08-27T11:25:24.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7c3fff700 1 -- 172.21.15.45:0/340990058 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8011d20 con 0x7fe7cc15a0f0 2023-08-27T11:25:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7d32bc700 1 -- 172.21.15.45:0/340990058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc15a0f0 msgr2=0x7fe7cc0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7d32bc700 1 --2- 172.21.15.45:0/340990058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc15a0f0 0x7fe7cc0cbc20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009840 tx=0x7fe7c8009b50 comp rx=0 tx=0).stop 2023-08-27T11:25:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7d32bc700 1 -- 172.21.15.45:0/340990058 shutdown_connections 2023-08-27T11:25:24.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7d32bc700 1 --2- 172.21.15.45:0/340990058 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc06ed90 0x7fe7cc15e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7d32bc700 1 --2- 172.21.15.45:0/340990058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc15a0f0 0x7fe7cc0cbc20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.271+0000 7fe7d32bc700 1 -- 172.21.15.45:0/340990058 >> 172.21.15.45:0/340990058 conn(0x7fe7cc0cc7e0 msgr2=0x7fe7cc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.272+0000 7fe7d32bc700 1 -- 172.21.15.45:0/340990058 shutdown_connections 2023-08-27T11:25:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.272+0000 7fe7d32bc700 1 -- 172.21.15.45:0/340990058 wait complete. 2023-08-27T11:25:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.273+0000 7fe7d32bc700 1 Processor -- start 2023-08-27T11:25:24.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.273+0000 7fe7d32bc700 1 -- start start 2023-08-27T11:25:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.273+0000 7fe7d32bc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7cc06ed90 0x7fe7cc0c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d32bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc0cd300 0x7fe7cc0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d32bc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc15a0f0 0x7fe7cc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d32bc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc0aef60 con 0x7fe7cc0cd300 2023-08-27T11:25:24.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d32bc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc0b5b50 con 0x7fe7cc06ed90 2023-08-27T11:25:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d32bc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc1f60e0 con 0x7fe7cc15a0f0 2023-08-27T11:25:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d0857700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc0cd300 0x7fe7cc0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d0857700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc0cd300 0x7fe7cc0ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56526/0 (socket says 172.21.15.45:56526) 2023-08-27T11:25:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d0857700 1 -- 172.21.15.45:0/2328602097 learned_addr learned my addr 172.21.15.45:0/2328602097 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:24.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d1859700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc15a0f0 0x7fe7cc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d1058700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7cc06ed90 0x7fe7cc0c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d0857700 1 -- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc15a0f0 msgr2=0x7fe7cc0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d0857700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc15a0f0 0x7fe7cc0ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d0857700 1 -- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7cc06ed90 msgr2=0x7fe7cc0c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d0857700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7cc06ed90 0x7fe7cc0c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d0857700 1 -- 172.21.15.45:0/2328602097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c80094a0 con 0x7fe7cc0cd300 2023-08-27T11:25:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d0857700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc0cd300 0x7fe7cc0ae340 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00c520 tx=0x7fe7bc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7c1ffb700 1 -- 172.21.15.45:0/2328602097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc005d70 con 0x7fe7cc0cd300 2023-08-27T11:25:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7c1ffb700 1 -- 172.21.15.45:0/2328602097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc0070d0 con 0x7fe7cc0cd300 2023-08-27T11:25:24.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7c1ffb700 1 -- 172.21.15.45:0/2328602097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc007440 con 0x7fe7cc0cd300 2023-08-27T11:25:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.274+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc1f63c0 con 0x7fe7cc0cd300 2023-08-27T11:25:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.275+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc1f6980 con 0x7fe7cc0cd300 2023-08-27T11:25:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.276+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc1f6ba0 con 0x7fe7cc0cd300 2023-08-27T11:25:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.276+0000 7fe7c1ffb700 1 -- 172.21.15.45:0/2328602097 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fe7bc0075a0 con 0x7fe7cc0cd300 2023-08-27T11:25:24.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.276+0000 7fe7c1ffb700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe7b006a250 0x7fe7b006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.276+0000 7fe7c1ffb700 1 -- 172.21.15.45:0/2328602097 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe7bc087ae0 con 0x7fe7cc0cd300 2023-08-27T11:25:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.276+0000 7fe7d1058700 1 -- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe7b006a250 msgr2=0x7fe7b006c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3181978685 2023-08-27T11:25:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.276+0000 7fe7d1058700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe7b006a250 0x7fe7b006c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:25:24.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.281+0000 7fe7c1ffb700 1 -- 172.21.15.45:0/2328602097 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe7bc053090 con 0x7fe7cc0cd300 2023-08-27T11:25:24.330 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:24.328+0000 7ff1d5722200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:25:24.352 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:24.351+0000 7f50f44e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:25:24.402 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:24.400+0000 7ff1d5722200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:25:24.426 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:24.426+0000 7f50f44e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:25:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.477+0000 7fe7d1058700 1 -- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe7b006a250 msgr2=0x7fe7b006c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3181978685 2023-08-27T11:25:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.477+0000 7fe7d1058700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe7b006a250 0x7fe7b006c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:25:24.480 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:24.478+0000 7ff1d5722200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:25:24.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.499+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe7cc0726d0 con 0x7fe7cc0cd300 2023-08-27T11:25:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.502+0000 7fe7c1ffb700 1 -- 172.21.15.45:0/2328602097 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe7bc0566b0 con 0x7fe7cc0cd300 2023-08-27T11:25:24.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.504+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe7b006a250 msgr2=0x7fe7b006c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:25:24.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.504+0000 7fe7d32bc700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe7b006a250 0x7fe7b006c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.504+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc0cd300 msgr2=0x7fe7cc0ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.504+0000 7fe7d32bc700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc0cd300 0x7fe7cc0ae340 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00c520 tx=0x7fe7bc00c830 comp rx=0 tx=0).stop 2023-08-27T11:25:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.505+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 shutdown_connections 2023-08-27T11:25:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.505+0000 7fe7d32bc700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe7b006a250 0x7fe7b006c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.505+0000 7fe7d32bc700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7cc15a0f0 0x7fe7cc0ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.505+0000 7fe7d32bc700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7cc06ed90 0x7fe7cc0c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.505+0000 7fe7d32bc700 1 --2- 172.21.15.45:0/2328602097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7cc0cd300 0x7fe7cc0ae340 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.505+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 >> 172.21.15.45:0/2328602097 conn(0x7fe7cc0cc7e0 msgr2=0x7fe7cc0af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.505+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 shutdown_connections 2023-08-27T11:25:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.505+0000 7fe7d32bc700 1 -- 172.21.15.45:0/2328602097 wait complete. 2023-08-27T11:25:24.509 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:24.509+0000 7f50f44e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:25:24.518 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-27T11:25:24.621 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:24.619+0000 7ff1d5722200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:25:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.647+0000 7fb77f50b700 1 Processor -- start 2023-08-27T11:25:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.647+0000 7fb77f50b700 1 -- start start 2023-08-27T11:25:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77f50b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb778157be0 0x7fb7780716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77f50b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7780b17c0 0x7fb7780b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77f50b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77815b460 con 0x7fb77806ced0 2023-08-27T11:25:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77f50b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb778157fc0 con 0x7fb7780b17c0 2023-08-27T11:25:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77f50b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780b2280 con 0x7fb778157be0 2023-08-27T11:25:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77caa6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7780b17c0 0x7fb7780b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77daa8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb77806ced0 0x7fb7780cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43358/0 (socket says 172.21.15.45:43358) 2023-08-27T11:25:24.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77daa8700 1 -- 172.21.15.45:0/4218490274 learned_addr learned my addr 172.21.15.45:0/4218490274 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:24.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77d2a7700 1 --2- 172.21.15.45:0/4218490274 >> v2:172.21.15.135:3300/0 conn(0x7fb778157be0 0x7fb7780716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77caa6700 1 -- 172.21.15.45:0/4218490274 >> v2:172.21.15.135:3300/0 conn(0x7fb778157be0 msgr2=0x7fb7780716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:24.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77caa6700 1 --2- 172.21.15.45:0/4218490274 >> v2:172.21.15.135:3300/0 conn(0x7fb778157be0 0x7fb7780716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77caa6700 1 -- 172.21.15.45:0/4218490274 >> v1:172.21.15.45:6789/0 conn(0x7fb77806ced0 legacy=0x7fb7780cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb76ffff700 1 -- 172.21.15.45:0/4218490274 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637408951 0 0) 0x7fb77815b460 con 0x7fb77806ced0 2023-08-27T11:25:24.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77caa6700 1 -- 172.21.15.45:0/4218490274 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780b2560 con 0x7fb7780b17c0 2023-08-27T11:25:24.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.648+0000 7fb77caa6700 1 --2- 172.21.15.45:0/4218490274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7780b17c0 0x7fb7780b1c70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb778066e70 tx=0x7fb768009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17db52aca0b4c722 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb76ffff700 1 -- 172.21.15.45:0/4218490274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768011830 con 0x7fb7780b17c0 2023-08-27T11:25:24.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb76ffff700 1 -- 172.21.15.45:0/4218490274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb768011990 con 0x7fb7780b17c0 2023-08-27T11:25:24.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb76ffff700 1 -- 172.21.15.45:0/4218490274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768011d20 con 0x7fb7780b17c0 2023-08-27T11:25:24.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb77f50b700 1 -- 172.21.15.45:0/4218490274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7780b17c0 msgr2=0x7fb7780b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb77f50b700 1 --2- 172.21.15.45:0/4218490274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7780b17c0 0x7fb7780b1c70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb778066e70 tx=0x7fb768009b20 comp rx=0 tx=0).stop 2023-08-27T11:25:24.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb77f50b700 1 -- 172.21.15.45:0/4218490274 shutdown_connections 2023-08-27T11:25:24.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb77f50b700 1 --2- 172.21.15.45:0/4218490274 >> v2:172.21.15.135:3300/0 conn(0x7fb778157be0 0x7fb7780716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb77f50b700 1 --2- 172.21.15.45:0/4218490274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7780b17c0 0x7fb7780b1c70 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb77f50b700 1 -- 172.21.15.45:0/4218490274 >> 172.21.15.45:0/4218490274 conn(0x7fb7780cc7c0 msgr2=0x7fb7780ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:24.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb77f50b700 1 -- 172.21.15.45:0/4218490274 shutdown_connections 2023-08-27T11:25:24.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.649+0000 7fb77f50b700 1 -- 172.21.15.45:0/4218490274 wait complete. 2023-08-27T11:25:24.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.650+0000 7fb77f50b700 1 Processor -- start 2023-08-27T11:25:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77f50b700 1 -- start start 2023-08-27T11:25:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77f50b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb77806ced0 0x7fb7780c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77f50b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7780b17c0 0x7fb7780c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77f50b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb778157be0 0x7fb7780c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77f50b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77815d2e0 con 0x7fb778157be0 2023-08-27T11:25:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77f50b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780c7480 con 0x7fb77806ced0 2023-08-27T11:25:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77f50b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780c7760 con 0x7fb7780b17c0 2023-08-27T11:25:24.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77daa8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb778157be0 0x7fb7780c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77daa8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb778157be0 0x7fb7780c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56588/0 (socket says 172.21.15.45:56588) 2023-08-27T11:25:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.651+0000 7fb77daa8700 1 -- 172.21.15.45:0/3873499102 learned_addr learned my addr 172.21.15.45:0/3873499102 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77caa6700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7780b17c0 0x7fb7780c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77d2a7700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb77806ced0 0x7fb7780c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77daa8700 1 -- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7780b17c0 msgr2=0x7fb7780c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77daa8700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7780b17c0 0x7fb7780c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77daa8700 1 -- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb77806ced0 msgr2=0x7fb7780c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77daa8700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb77806ced0 0x7fb7780c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77daa8700 1 -- 172.21.15.45:0/3873499102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7680094a0 con 0x7fb778157be0 2023-08-27T11:25:24.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77d2a7700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb77806ced0 0x7fb7780c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77caa6700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7780b17c0 0x7fb7780c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:25:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77daa8700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb778157be0 0x7fb7780c8520 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb77000cba0 tx=0x7fb77000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb76dffb700 1 -- 172.21.15.45:0/3873499102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77000ab80 con 0x7fb778157be0 2023-08-27T11:25:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb76dffb700 1 -- 172.21.15.45:0/3873499102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77000ad00 con 0x7fb778157be0 2023-08-27T11:25:24.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb76dffb700 1 -- 172.21.15.45:0/3873499102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770007310 con 0x7fb778157be0 2023-08-27T11:25:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7780c7a40 con 0x7fb778157be0 2023-08-27T11:25:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.652+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7780c8000 con 0x7fb778157be0 2023-08-27T11:25:24.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.653+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb778072690 con 0x7fb778157be0 2023-08-27T11:25:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.654+0000 7fb76dffb700 1 -- 172.21.15.45:0/3873499102 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fb770007470 con 0x7fb778157be0 2023-08-27T11:25:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.654+0000 7fb76dffb700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fb75c06a250 0x7fb75c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:24.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.654+0000 7fb77d2a7700 1 -- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fb75c06a250 msgr2=0x7fb75c06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3181978685 2023-08-27T11:25:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.654+0000 7fb76dffb700 1 -- 172.21.15.45:0/3873499102 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fb770088340 con 0x7fb778157be0 2023-08-27T11:25:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.654+0000 7fb77d2a7700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fb75c06a250 0x7fb75c06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:25:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.658+0000 7fb76dffb700 1 -- 172.21.15.45:0/3873499102 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb7700537b0 con 0x7fb778157be0 2023-08-27T11:25:24.670 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:24.667+0000 7f50f44e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:25:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.855+0000 7fb77d2a7700 1 -- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fb75c06a250 msgr2=0x7fb75c06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3181978685 2023-08-27T11:25:24.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.855+0000 7fb77d2a7700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fb75c06a250 0x7fb75c06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:25:24.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.860+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fb7780734a0 con 0x7fb778157be0 2023-08-27T11:25:24.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.864+0000 7fb76dffb700 1 -- 172.21.15.45:0/3873499102 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb770056dd0 con 0x7fb778157be0 2023-08-27T11:25:24.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fb75c06a250 msgr2=0x7fb75c06c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:25:24.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fb75c06a250 0x7fb75c06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb778157be0 msgr2=0x7fb7780c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:24.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb778157be0 0x7fb7780c8520 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb77000cba0 tx=0x7fb77000cf60 comp rx=0 tx=0).stop 2023-08-27T11:25:24.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 shutdown_connections 2023-08-27T11:25:24.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fb75c06a250 0x7fb75c06c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb778157be0 0x7fb7780c8520 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb77806ced0 0x7fb7780c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 --2- 172.21.15.45:0/3873499102 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7780b17c0 0x7fb7780c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:24.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 >> 172.21.15.45:0/3873499102 conn(0x7fb7780cc7c0 msgr2=0x7fb7780cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:24.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 shutdown_connections 2023-08-27T11:25:24.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:24.866+0000 7fb77f50b700 1 -- 172.21.15.45:0/3873499102 wait complete. 2023-08-27T11:25:24.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:24.880 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-27T11:25:24.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-08-27T11:25:24.950 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:24.948+0000 7ff1d5722200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:25:25.021 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:25.021+0000 7f50f44e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:25:25.066 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:25.064+0000 7ff1d5722200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:25:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.072+0000 7fe54b79e700 1 Processor -- start 2023-08-27T11:25:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.072+0000 7fe54b79e700 1 -- start start 2023-08-27T11:25:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe54b79e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe544157be0 0x7fe5440716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe54b79e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5440b17c0 0x7fe5440b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe54b79e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe54406ced0 0x7fe54406d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe54b79e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe544157fc0 con 0x7fe5440b17c0 2023-08-27T11:25:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe54b79e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54406d880 con 0x7fe544157be0 2023-08-27T11:25:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe54b79e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54415b460 con 0x7fe54406ced0 2023-08-27T11:25:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe548d39700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5440b17c0 0x7fe5440b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe548d39700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5440b17c0 0x7fe5440b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56638/0 (socket says 172.21.15.45:56638) 2023-08-27T11:25:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe548d39700 1 -- 172.21.15.45:0/1809823661 learned_addr learned my addr 172.21.15.45:0/1809823661 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe549d3b700 1 --2- 172.21.15.45:0/1809823661 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe54406ced0 0x7fe54406d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.073+0000 7fe54953a700 1 --2- 172.21.15.45:0/1809823661 >> v2:172.21.15.135:3300/0 conn(0x7fe544157be0 0x7fe5440716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.074+0000 7fe548d39700 1 -- 172.21.15.45:0/1809823661 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe54406ced0 msgr2=0x7fe54406d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.074+0000 7fe548d39700 1 --2- 172.21.15.45:0/1809823661 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe54406ced0 0x7fe54406d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.074+0000 7fe548d39700 1 -- 172.21.15.45:0/1809823661 >> v2:172.21.15.135:3300/0 conn(0x7fe544157be0 msgr2=0x7fe5440716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.074+0000 7fe548d39700 1 --2- 172.21.15.45:0/1809823661 >> v2:172.21.15.135:3300/0 conn(0x7fe544157be0 0x7fe5440716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.074+0000 7fe548d39700 1 -- 172.21.15.45:0/1809823661 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5440b2280 con 0x7fe5440b17c0 2023-08-27T11:25:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.074+0000 7fe548d39700 1 --2- 172.21.15.45:0/1809823661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5440b17c0 0x7fe5440b1c70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe544066e70 tx=0x7fe540009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e8541cda35899e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.074+0000 7fe53bfff700 1 -- 172.21.15.45:0/1809823661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540011830 con 0x7fe5440b17c0 2023-08-27T11:25:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.074+0000 7fe53bfff700 1 -- 172.21.15.45:0/1809823661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe540011990 con 0x7fe5440b17c0 2023-08-27T11:25:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.074+0000 7fe53bfff700 1 -- 172.21.15.45:0/1809823661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540011d20 con 0x7fe5440b17c0 2023-08-27T11:25:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.075+0000 7fe54b79e700 1 -- 172.21.15.45:0/1809823661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5440b17c0 msgr2=0x7fe5440b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.075+0000 7fe54b79e700 1 --2- 172.21.15.45:0/1809823661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5440b17c0 0x7fe5440b1c70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe544066e70 tx=0x7fe540009c00 comp rx=0 tx=0).stop 2023-08-27T11:25:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.075+0000 7fe54b79e700 1 -- 172.21.15.45:0/1809823661 shutdown_connections 2023-08-27T11:25:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.075+0000 7fe54b79e700 1 --2- 172.21.15.45:0/1809823661 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe54406ced0 0x7fe54406d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.075+0000 7fe54b79e700 1 --2- 172.21.15.45:0/1809823661 >> v2:172.21.15.135:3300/0 conn(0x7fe544157be0 0x7fe5440716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.075+0000 7fe54b79e700 1 --2- 172.21.15.45:0/1809823661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5440b17c0 0x7fe5440b1c70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.075+0000 7fe54b79e700 1 -- 172.21.15.45:0/1809823661 >> 172.21.15.45:0/1809823661 conn(0x7fe5440cc7c0 msgr2=0x7fe5440ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.075+0000 7fe54b79e700 1 -- 172.21.15.45:0/1809823661 shutdown_connections 2023-08-27T11:25:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.075+0000 7fe54b79e700 1 -- 172.21.15.45:0/1809823661 wait complete. 2023-08-27T11:25:25.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.076+0000 7fe54b79e700 1 Processor -- start 2023-08-27T11:25:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.076+0000 7fe54b79e700 1 -- start start 2023-08-27T11:25:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54b79e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe54406ced0 0x7fe544153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54b79e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5440b17c0 0x7fe54415c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54b79e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe544157be0 0x7fe54414f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54b79e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440b4ec0 con 0x7fe54406ced0 2023-08-27T11:25:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54b79e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440b42b0 con 0x7fe5440b17c0 2023-08-27T11:25:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54b79e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440b4590 con 0x7fe544157be0 2023-08-27T11:25:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54953a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe54406ced0 0x7fe544153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54953a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe54406ced0 0x7fe544153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56644/0 (socket says 172.21.15.45:56644) 2023-08-27T11:25:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54953a700 1 -- 172.21.15.45:0/2473943602 learned_addr learned my addr 172.21.15.45:0/2473943602 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe548d39700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5440b17c0 0x7fe54415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54953a700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe544157be0 msgr2=0x7fe54414f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54953a700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe544157be0 0x7fe54414f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54953a700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5440b17c0 msgr2=0x7fe54415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:25.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54953a700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5440b17c0 0x7fe54415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54953a700 1 -- 172.21.15.45:0/2473943602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe540009550 con 0x7fe54406ced0 2023-08-27T11:25:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe54953a700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe54406ced0 0x7fe544153d40 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe53400e9b0 tx=0x7fe53400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.077+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534009710 con 0x7fe54406ced0 2023-08-27T11:25:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.078+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53400c7d0 con 0x7fe54406ced0 2023-08-27T11:25:25.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.078+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5440b4870 con 0x7fe54406ced0 2023-08-27T11:25:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.078+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400cb40 con 0x7fe54406ced0 2023-08-27T11:25:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.078+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe544154280 con 0x7fe54406ced0 2023-08-27T11:25:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.079+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54406dc80 con 0x7fe54406ced0 2023-08-27T11:25:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.079+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fe53400cca0 con 0x7fe54406ced0 2023-08-27T11:25:25.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.080+0000 7fe539ffb700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 0x7fe52806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.080+0000 7fe548d39700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 msgr2=0x7fe52806c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3181978685 2023-08-27T11:25:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.080+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe534088950 con 0x7fe54406ced0 2023-08-27T11:25:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.080+0000 7fe548d39700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 0x7fe52806c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:25:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.084+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe534053d40 con 0x7fe54406ced0 2023-08-27T11:25:25.140 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:25.138+0000 7ff1d5722200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:25:25.148 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:25.147+0000 7f50f44e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:25:25.221 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:25.218+0000 7ff1d5722200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:25:25.225 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:25.225+0000 7f50f44e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:25:25.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.281+0000 7fe548d39700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 msgr2=0x7fe52806c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3181978685 2023-08-27T11:25:25.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.281+0000 7fe548d39700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 0x7fe52806c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:25:25.310 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:25.310+0000 7f50f44e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:25:25.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.311+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 --> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5440c3c30 con 0x7fe52806a250 2023-08-27T11:25:25.591 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:25.590+0000 7ff1d5722200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:25:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.682+0000 7fe548d39700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 msgr2=0x7fe52806c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3181978685 2023-08-27T11:25:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:25.682+0000 7fe548d39700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 0x7fe52806c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:25:25.699 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:25.698+0000 7f50f44e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:25:26.162 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:26.161+0000 7ff1d5722200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:25:26.233 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:26.232+0000 7ff1d5722200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:25:26.308 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:26.307+0000 7f50f44e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:25:26.381 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:26.380+0000 7f50f44e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:25:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:26.483+0000 7fe548d39700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 msgr2=0x7fe52806c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3181978685 2023-08-27T11:25:26.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:26.483+0000 7fe548d39700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 0x7fe52806c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:25:26.861 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:26.860+0000 7ff1d5722200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:25:26.939 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:26.938+0000 7ff1d5722200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:25:27.044 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:27.043+0000 7f50f44e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:25:27.086 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:27.084+0000 7ff1d5722200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:25:27.128 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:27.128+0000 7f50f44e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:25:27.281 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:27.280+0000 7f50f44e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:25:27.315 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:27.314+0000 7ff1d5722200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:25:27.389 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:27.388+0000 7ff1d5722200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:25:27.518 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:27.517+0000 7f50f44e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:25:27.595 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:27.595+0000 7f50f44e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:25:27.654 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:27.653+0000 7ff1d5722200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:25:27.871 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:27.870+0000 7f50f44e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:25:27.875 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:27.874+0000 7ff1d5722200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:25:28.024 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:28.022+0000 7ff1d5722200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:25:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:28.084+0000 7fe548d39700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 msgr2=0x7fe52806c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3181978685 2023-08-27T11:25:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:28.084+0000 7fe548d39700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 0x7fe52806c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-27T11:25:28.101 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:28.101+0000 7f50f44e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:25:28.137 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:28.136+0000 7ff1d5722200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:25:28.257 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:28.256+0000 7f50f44e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:25:28.291 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:28.290+0000 7ff1d5722200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:25:28.376 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:28.376+0000 7f50f44e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:25:28.538 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:28.537+0000 7f50f44e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:25:28.554 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:28.553+0000 7ff1d5722200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:25:28.714 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:28.713+0000 7ff1d5722200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:25:28.809 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:28.808+0000 7f50f44e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:25:28.975 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:28.975+0000 7f50f44e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:25:29.241 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.240+0000 7ff1d5722200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:25:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:29.261+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 20) v1 ==== 88069+0+0 (secure 0 0 0) 0x7fe534086ba0 con 0x7fe54406ced0 2023-08-27T11:25:29.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:29.261+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 msgr2=0x7fe52806c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:25:29.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:29.261+0000 7fe539ffb700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/3181978685,v1:172.21.15.135:6833/3181978685] conn(0x7fe52806a250 0x7fe52806c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:29.387 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff148780700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.387 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff148780700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.387 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff148780700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.387 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff148780700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.388 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff148780700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.388 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff14b786700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.388 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff14b786700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.388 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff14b786700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.388 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff14b786700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.388 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:25:29.385+0000 7ff14b786700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:25:29.545 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:25:29.544+0000 7f50f44e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:25:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.265+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 21) v1 ==== 88196+0+0 (secure 0 0 0) 0x7fe53405d510 con 0x7fe54406ced0 2023-08-27T11:25:30.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.266+0000 7fe539ffb700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fe528072cc0 0x7fe5280750b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.266+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 --> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5440c3c30 con 0x7fe528072cc0 2023-08-27T11:25:30.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.267+0000 7fe548d39700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fe528072cc0 0x7fe5280750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.269+0000 7fe548d39700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fe528072cc0 0x7fe5280750b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe544066e70 tx=0x7fe540008030 comp rx=0 tx=0).ready entity=mgr.6157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.332+0000 7fe539ffb700 1 -- 172.21.15.45:0/2473943602 <== mgr.6157 v2:172.21.15.135:6832/1664443653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe5440c3c30 con 0x7fe528072cc0 2023-08-27T11:25:30.334 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path snap_test_dir1 2023-08-27T11:25:30.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.335+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fe528072cc0 msgr2=0x7fe5280750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.335+0000 7fe54b79e700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fe528072cc0 0x7fe5280750b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe544066e70 tx=0x7fe540008030 comp rx=0 tx=0).stop 2023-08-27T11:25:30.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.335+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe54406ced0 msgr2=0x7fe544153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.335+0000 7fe54b79e700 1 --2- 172.21.15.45:0/2473943602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe54406ced0 0x7fe544153d40 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe53400e9b0 tx=0x7fe53400ecc0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.335+0000 7fe549d3b700 1 -- 172.21.15.45:0/2473943602 reap_dead start 2023-08-27T11:25:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.335+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 shutdown_connections 2023-08-27T11:25:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.335+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 >> 172.21.15.45:0/2473943602 conn(0x7fe5440cc7c0 msgr2=0x7fe5440ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.335+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 shutdown_connections 2023-08-27T11:25:30.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.335+0000 7fe54b79e700 1 -- 172.21.15.45:0/2473943602 wait complete. 2023-08-27T11:25:30.349 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-08-27T11:25:30.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-08-27T11:25:30.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.482+0000 7f87c8dac700 1 Processor -- start 2023-08-27T11:25:30.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.482+0000 7f87c8dac700 1 -- start start 2023-08-27T11:25:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.483+0000 7f87c8dac700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 0x7f87c40ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.483+0000 7f87c8dac700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c406b010 con 0x7f87c415d4f0 2023-08-27T11:25:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.483+0000 7f87c8dac700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c40ad9f0 con 0x7f87c40aefb0 2023-08-27T11:25:30.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.483+0000 7f87c8dac700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c415d8d0 con 0x7f87c40cacd0 2023-08-27T11:25:30.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.483+0000 7f87c259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f87c415d4f0 0x7f87c40ca5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43526/0 (socket says 172.21.15.45:43526) 2023-08-27T11:25:30.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.483+0000 7f87c259c700 1 -- 172.21.15.45:0/253909835 learned_addr learned my addr 172.21.15.45:0/253909835 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:30.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.483+0000 7f87c1d9b700 1 --2- 172.21.15.45:0/253909835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 0x7f87c40ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.483+0000 7f87c159a700 1 -- 172.21.15.45:0/253909835 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448982517 0 0) 0x7f87c406b010 con 0x7f87c415d4f0 2023-08-27T11:25:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.483+0000 7f87c159a700 1 -- 172.21.15.45:0/253909835 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87ac0035b0 con 0x7f87c415d4f0 2023-08-27T11:25:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c159a700 1 -- 172.21.15.45:0/253909835 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978598679 0 0) 0x7f87c40ad9f0 con 0x7f87c40aefb0 2023-08-27T11:25:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c159a700 1 -- 172.21.15.45:0/253909835 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c406b010 con 0x7f87c40aefb0 2023-08-27T11:25:30.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c159a700 1 -- 172.21.15.45:0/253909835 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 293780504 0 0) 0x7f87ac0035b0 con 0x7f87c415d4f0 2023-08-27T11:25:30.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c1d9b700 1 -- 172.21.15.45:0/253909835 >> v1:172.21.15.135:6789/0 conn(0x7f87c40aefb0 legacy=0x7f87c40cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c1d9b700 1 -- 172.21.15.45:0/253909835 >> v1:172.21.15.45:6789/0 conn(0x7f87c415d4f0 legacy=0x7f87c40ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c1d9b700 1 -- 172.21.15.45:0/253909835 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c406ad30 con 0x7f87c40cacd0 2023-08-27T11:25:30.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c1d9b700 1 --2- 172.21.15.45:0/253909835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 0x7f87c40ae490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009870 tx=0x7f87b8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc993522ea7d13c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c159a700 1 -- 172.21.15.45:0/253909835 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4001670 con 0x7f87c40cacd0 2023-08-27T11:25:30.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c159a700 1 -- 172.21.15.45:0/253909835 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b8004030 con 0x7f87c40cacd0 2023-08-27T11:25:30.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.484+0000 7f87c159a700 1 -- 172.21.15.45:0/253909835 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80043c0 con 0x7f87c40cacd0 2023-08-27T11:25:30.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.485+0000 7f87c8dac700 1 -- 172.21.15.45:0/253909835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 msgr2=0x7f87c40ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.485+0000 7f87c8dac700 1 --2- 172.21.15.45:0/253909835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 0x7f87c40ae490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009870 tx=0x7f87b8009b80 comp rx=0 tx=0).stop 2023-08-27T11:25:30.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.485+0000 7f87c8dac700 1 -- 172.21.15.45:0/253909835 shutdown_connections 2023-08-27T11:25:30.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.485+0000 7f87c8dac700 1 --2- 172.21.15.45:0/253909835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 0x7f87c40ae490 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.485+0000 7f87c8dac700 1 -- 172.21.15.45:0/253909835 >> 172.21.15.45:0/253909835 conn(0x7f87c40cc7e0 msgr2=0x7f87c40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:30.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.485+0000 7f87c8dac700 1 -- 172.21.15.45:0/253909835 shutdown_connections 2023-08-27T11:25:30.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.485+0000 7f87c8dac700 1 -- 172.21.15.45:0/253909835 wait complete. 2023-08-27T11:25:30.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.486+0000 7f87c8dac700 1 Processor -- start 2023-08-27T11:25:30.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.486+0000 7f87c8dac700 1 -- start start 2023-08-27T11:25:30.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c8dac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c40aefb0 0x7f87c40c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c8dac700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 0x7f87c40c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c8dac700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87c415d4f0 0x7f87c40c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c8dac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40c7880 con 0x7f87c40aefb0 2023-08-27T11:25:30.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c8dac700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40c7b60 con 0x7f87c415d4f0 2023-08-27T11:25:30.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c8dac700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40c7e40 con 0x7f87c40cacd0 2023-08-27T11:25:30.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c40aefb0 0x7f87c40c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c40aefb0 0x7f87c40c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56756/0 (socket says 172.21.15.45:56756) 2023-08-27T11:25:30.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c259c700 1 -- 172.21.15.45:0/750723549 learned_addr learned my addr 172.21.15.45:0/750723549 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:30.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c1d9b700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 0x7f87c40c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c2d9d700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87c415d4f0 0x7f87c40c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c259c700 1 -- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 msgr2=0x7f87c40c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c259c700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 0x7f87c40c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c259c700 1 -- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87c415d4f0 msgr2=0x7f87c40c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c259c700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87c415d4f0 0x7f87c40c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c259c700 1 -- 172.21.15.45:0/750723549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80094a0 con 0x7f87c40aefb0 2023-08-27T11:25:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.487+0000 7f87c259c700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c40aefb0 0x7f87c40c6e30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c750 tx=0x7f87b400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.488+0000 7f87b2ffd700 1 -- 172.21.15.45:0/750723549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4007560 con 0x7f87c40aefb0 2023-08-27T11:25:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.488+0000 7f87b2ffd700 1 -- 172.21.15.45:0/750723549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b40076c0 con 0x7f87c40aefb0 2023-08-27T11:25:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.488+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c40c8120 con 0x7f87c40aefb0 2023-08-27T11:25:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.488+0000 7f87b2ffd700 1 -- 172.21.15.45:0/750723549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4004a70 con 0x7f87c40aefb0 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.488+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c4159660 con 0x7f87c40aefb0 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.489+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c40676b0 con 0x7f87c40aefb0 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.489+0000 7f87b2ffd700 1 -- 172.21.15.45:0/750723549 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f87b4004bd0 con 0x7f87c40aefb0 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.490+0000 7f87b2ffd700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f87a4069f80 0x7f87a406c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.490+0000 7f87b2ffd700 1 -- 172.21.15.45:0/750723549 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f87b40883a0 con 0x7f87c40aefb0 2023-08-27T11:25:30.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.490+0000 7f87c1d9b700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f87a4069f80 0x7f87a406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.490+0000 7f87c1d9b700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f87a4069f80 0x7f87a406c440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f87b80097c0 tx=0x7f87b8013040 comp rx=0 tx=0).ready entity=mgr.6157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.494+0000 7f87b2ffd700 1 -- 172.21.15.45:0/750723549 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f87c40676b0 con 0x7f87c40aefb0 2023-08-27T11:25:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.721+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 --> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f87c40adc90 con 0x7f87a4069f80 2023-08-27T11:25:30.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.778+0000 7f87b2ffd700 1 -- 172.21.15.45:0/750723549 <== mgr.6157 v2:172.21.15.135:6832/1664443653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f87b8004320 con 0x7f87a4069f80 2023-08-27T11:25:30.780 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path snap_test_dir1 2023-08-27T11:25:30.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.780+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f87a4069f80 msgr2=0x7f87a406c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.780+0000 7f87c8dac700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f87a4069f80 0x7f87a406c440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f87b80097c0 tx=0x7f87b8013040 comp rx=0 tx=0).stop 2023-08-27T11:25:30.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c40aefb0 msgr2=0x7f87c40c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c40aefb0 0x7f87c40c6e30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c750 tx=0x7f87b400cb10 comp rx=0 tx=0).stop 2023-08-27T11:25:30.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 shutdown_connections 2023-08-27T11:25:30.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f87a4069f80 0x7f87a406c440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c40aefb0 0x7f87c40c6e30 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87c415d4f0 0x7f87c40c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 --2- 172.21.15.45:0/750723549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87c40cacd0 0x7f87c40c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 >> 172.21.15.45:0/750723549 conn(0x7f87c40cc7e0 msgr2=0x7f87c415c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:30.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 shutdown_connections 2023-08-27T11:25:30.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.781+0000 7f87c8dac700 1 -- 172.21.15.45:0/750723549 wait complete. 2023-08-27T11:25:30.794 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-08-27T11:25:30.794 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-08-27-11_26_00 in ~30s... 2023-08-27T11:25:30.794 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-08-27-11_27_00 in ~90s... 2023-08-27T11:25:30.795 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-08-27T11:25:30.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-08-27T11:25:30.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.923+0000 7faf25c2d700 1 Processor -- start 2023-08-27T11:25:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.923+0000 7faf25c2d700 1 -- start start 2023-08-27T11:25:30.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.924+0000 7faf25c2d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7faf20160d00 0x7faf2006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.924+0000 7faf25c2d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2006b160 con 0x7faf2015d340 2023-08-27T11:25:30.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.924+0000 7faf25c2d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf200cdf90 con 0x7faf2006a7c0 2023-08-27T11:25:30.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.924+0000 7faf25c2d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf20072010 con 0x7faf20160d00 2023-08-27T11:25:30.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.924+0000 7faf1f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faf2015d340 0x7faf200cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43542/0 (socket says 172.21.15.45:43542) 2023-08-27T11:25:30.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.924+0000 7faf1f7fe700 1 -- 172.21.15.45:0/735440126 learned_addr learned my addr 172.21.15.45:0/735440126 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:25:30.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1effd700 1 --2- 172.21.15.45:0/735440126 >> v2:172.21.15.135:3300/0 conn(0x7faf20160d00 0x7faf2006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603479572 0 0) 0x7faf2006b160 con 0x7faf2015d340 2023-08-27T11:25:30.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf0c0035b0 con 0x7faf2015d340 2023-08-27T11:25:30.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2725562150 0 0) 0x7faf0c0035b0 con 0x7faf2015d340 2023-08-27T11:25:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 >> v2:172.21.15.135:3300/0 conn(0x7faf20160d00 msgr2=0x7faf2006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 --2- 172.21.15.45:0/735440126 >> v2:172.21.15.135:3300/0 conn(0x7faf20160d00 0x7faf2006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 >> v1:172.21.15.135:6789/0 conn(0x7faf2006a7c0 legacy=0x7faf200b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf200af590 con 0x7faf2015d340 2023-08-27T11:25:30.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7faf10001670 con 0x7faf2015d340 2023-08-27T11:25:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 >> v1:172.21.15.45:6789/0 conn(0x7faf2015d340 legacy=0x7faf200cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c003a00 0x7faf0c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf0c004850 0x7faf0c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf0c005210 0x7faf0c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf20072010 con 0x7faf0c003a00 2023-08-27T11:25:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c00c1b0 con 0x7faf0c004850 2023-08-27T11:25:30.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c00c370 con 0x7faf0c005210 2023-08-27T11:25:30.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1f7fe700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c003a00 0x7faf0c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1effd700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf0c004850 0x7faf0c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1ffff700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf0c005210 0x7faf0c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1f7fe700 1 -- 172.21.15.45:0/735440126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf0c005210 msgr2=0x7faf0c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1f7fe700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf0c005210 0x7faf0c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1f7fe700 1 -- 172.21.15.45:0/735440126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf0c004850 msgr2=0x7faf0c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:25:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1f7fe700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf0c004850 0x7faf0c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1f7fe700 1 -- 172.21.15.45:0/735440126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0035b0 con 0x7faf0c003a00 2023-08-27T11:25:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.925+0000 7faf1ffff700 1 -- 172.21.15.45:0/735440126 reap_dead start 2023-08-27T11:25:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf1f7fe700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c003a00 0x7faf0c003e20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7faf1000c2a0 tx=0x7faf0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ae9deb1f493ed2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10007c10 con 0x7faf0c003a00 2023-08-27T11:25:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1000cde0 con 0x7faf0c003a00 2023-08-27T11:25:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf1e7fc700 1 -- 172.21.15.45:0/735440126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10005ba0 con 0x7faf0c003a00 2023-08-27T11:25:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf25c2d700 1 -- 172.21.15.45:0/735440126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c003a00 msgr2=0x7faf0c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf25c2d700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c003a00 0x7faf0c003e20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7faf1000c2a0 tx=0x7faf0c005880 comp rx=0 tx=0).stop 2023-08-27T11:25:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf25c2d700 1 -- 172.21.15.45:0/735440126 shutdown_connections 2023-08-27T11:25:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf25c2d700 1 --2- 172.21.15.45:0/735440126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf0c003a00 0x7faf0c003e20 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf25c2d700 1 -- 172.21.15.45:0/735440126 >> 172.21.15.45:0/735440126 conn(0x7faf200cc7c0 msgr2=0x7faf200ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf25c2d700 1 -- 172.21.15.45:0/735440126 shutdown_connections 2023-08-27T11:25:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.926+0000 7faf25c2d700 1 -- 172.21.15.45:0/735440126 wait complete. 2023-08-27T11:25:30.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.928+0000 7faf25c2d700 1 Processor -- start 2023-08-27T11:25:30.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.928+0000 7faf25c2d700 1 -- start start 2023-08-27T11:25:30.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.928+0000 7faf25c2d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf0c003a00 0x7faf20151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.928+0000 7faf25c2d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf200c73d0 0x7faf2015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.928+0000 7faf25c2d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf200c7850 0x7faf201602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.928+0000 7faf25c2d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf200adb80 con 0x7faf200c73d0 2023-08-27T11:25:30.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf25c2d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf200ad2c0 con 0x7faf200c7850 2023-08-27T11:25:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf25c2d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf200b1a20 con 0x7faf0c003a00 2023-08-27T11:25:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf200c73d0 0x7faf2015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf200c73d0 0x7faf2015fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56778/0 (socket says 172.21.15.45:56778) 2023-08-27T11:25:30.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1effd700 1 -- 172.21.15.45:0/2564386736 learned_addr learned my addr 172.21.15.45:0/2564386736 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:25:30.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1ffff700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf200c7850 0x7faf201602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1effd700 1 -- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf0c003a00 msgr2=0x7faf20151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1effd700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf0c003a00 0x7faf20151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1effd700 1 -- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf200c7850 msgr2=0x7faf201602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:30.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1effd700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf200c7850 0x7faf201602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:30.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1effd700 1 -- 172.21.15.45:0/2564386736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf200b1d00 con 0x7faf200c73d0 2023-08-27T11:25:30.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1ffff700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf200c7850 0x7faf201602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:25:30.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf1effd700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf200c73d0 0x7faf2015fd60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7faf08011aa0 tx=0x7faf08011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf17fff700 1 -- 172.21.15.45:0/2564386736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800f9e0 con 0x7faf200c73d0 2023-08-27T11:25:30.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf17fff700 1 -- 172.21.15.45:0/2564386736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0800fb40 con 0x7faf200c73d0 2023-08-27T11:25:30.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf200ad4c0 con 0x7faf200c73d0 2023-08-27T11:25:30.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf17fff700 1 -- 172.21.15.45:0/2564386736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08014440 con 0x7faf200c73d0 2023-08-27T11:25:30.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.929+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf201518a0 con 0x7faf200c73d0 2023-08-27T11:25:30.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.930+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf20072690 con 0x7faf200c73d0 2023-08-27T11:25:30.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.931+0000 7faf17fff700 1 -- 172.21.15.45:0/2564386736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) v1 ==== 88196+0+0 (secure 0 0 0) 0x7faf080145a0 con 0x7faf200c73d0 2023-08-27T11:25:30.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.931+0000 7faf17fff700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7faf00069f80 0x7faf0006c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:25:30.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.931+0000 7faf17fff700 1 -- 172.21.15.45:0/2564386736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6770+0+0 (secure 0 0 0) 0x7faf08088bf0 con 0x7faf200c73d0 2023-08-27T11:25:30.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.931+0000 7faf1f7fe700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7faf00069f80 0x7faf0006c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:25:30.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.932+0000 7faf1f7fe700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7faf00069f80 0x7faf0006c440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf2006a720 tx=0x7faf1000a000 comp rx=0 tx=0).ready entity=mgr.6157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:25:30.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:30.936+0000 7faf17fff700 1 -- 172.21.15.45:0/2564386736 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faf0808f080 con 0x7faf200c73d0 2023-08-27T11:25:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.158+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 --> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf20071950 con 0x7faf00069f80 2023-08-27T11:25:31.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.170+0000 7faf17fff700 1 -- 172.21.15.45:0/2564386736 <== mgr.6157 v2:172.21.15.135:6832/1664443653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7faf10007b30 con 0x7faf00069f80 2023-08-27T11:25:31.171 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:25:31.171 INFO:teuthology.orchestra.run.smithi045.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-08-27T11:25:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7faf00069f80 msgr2=0x7faf0006c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7faf00069f80 0x7faf0006c440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf2006a720 tx=0x7faf1000a000 comp rx=0 tx=0).stop 2023-08-27T11:25:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf200c73d0 msgr2=0x7faf2015fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:25:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf200c73d0 0x7faf2015fd60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7faf08011aa0 tx=0x7faf08011e60 comp rx=0 tx=0).stop 2023-08-27T11:25:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 shutdown_connections 2023-08-27T11:25:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf200c7850 0x7faf201602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7faf0c003a00 0x7faf20151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7faf00069f80 0x7faf0006c440 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 --2- 172.21.15.45:0/2564386736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf200c73d0 0x7faf2015fd60 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:25:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 >> 172.21.15.45:0/2564386736 conn(0x7faf200cc7c0 msgr2=0x7faf20070d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:25:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 shutdown_connections 2023-08-27T11:25:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:25:31.172+0000 7faf25c2d700 1 -- 172.21.15.45:0/2564386736 wait complete. 2023-08-27T11:25:31.185 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-08-27T11:25:31.185 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-08-27T11:25:31.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:31.186 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:31.204 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:31.204 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:31.205 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:32.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:32.207 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:32.229 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:32.230 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:32.230 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:33.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:33.232 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:33.254 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:33.255 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:33.255 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:34.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:34.257 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:34.280 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:34.280 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:34.281 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:35.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:35.283 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:35.305 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:35.306 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:35.306 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:36.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:36.308 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:36.331 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:36.332 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:36.332 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:36.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:25:36.612 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:25:37.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:37.334 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:37.357 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:37.357 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:37.358 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:38.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:38.360 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:38.383 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:38.383 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:38.383 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:39.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:39.385 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:39.407 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:39.408 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:39.408 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:40.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:40.410 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:40.432 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:40.433 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:40.433 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:41.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:41.435 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:41.458 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:41.458 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:41.459 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:42.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:42.461 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:42.483 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:42.484 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:42.484 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:43.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:43.485 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:43.508 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:43.509 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:43.509 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:44.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:44.511 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:44.533 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:44.534 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:44.534 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:45.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:45.535 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:45.558 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:45.559 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:45.559 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:46.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:46.561 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:46.584 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:46.584 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:46.585 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:47.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:47.587 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:47.609 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:47.610 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:47.610 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:48.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:48.614 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:48.636 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:48.636 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:48.637 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:49.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:49.639 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:49.661 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:49.662 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:49.662 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:50.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:50.664 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:50.689 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:50.690 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:50.690 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:51.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:51.692 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:51.714 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:51.715 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:51.715 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:52.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:52.717 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:52.739 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:52.740 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:52.740 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:53.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:53.742 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:53.765 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:53.765 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:53.766 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:54.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:54.767 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:54.789 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:54.790 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:54.790 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:55.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:55.792 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:55.815 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:55.815 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:55.816 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:56.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:56.818 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:56.840 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:56.841 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:56.841 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:57.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:57.843 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:57.866 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:57.866 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:57.866 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:58.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:58.868 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:58.891 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:58.891 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:58.892 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:25:59.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:25:59.894 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:25:59.916 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-27T11:25:59.916 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:25:59.917 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:00.790 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:26:00.789+0000 7ff13a764700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:00.791 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:26:00.789+0000 7ff13a764700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:00.791 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:26:00.789+0000 7ff13a764700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:00.791 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:26:00.789+0000 7ff13a764700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:00.791 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:26:00.789+0000 7ff13a764700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:26:00.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:00.918 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:00.939 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:00.940 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:00.940 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:00.941 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:00.942 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-08-27-11_26_00_UTC'] 2023-08-27T11:26:00.943 DEBUG:tasks.cephfs.test_snap_schedules:exec=1693135530.7945108, now = 1693135560.9429786, timo = 30 2023-08-27T11:26:01.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:01.944 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:01.966 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:01.967 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:01.967 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:01.967 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:02.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:02.969 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:02.991 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:02.991 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:02.992 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:02.992 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:03.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:03.994 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:04.016 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:04.017 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:04.017 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:04.018 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:05.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:05.019 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:05.042 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:05.042 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:05.043 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:05.043 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:06.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:06.045 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:06.069 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:06.070 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:06.070 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:06.070 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:06.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:26:06.701 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:26:07.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:07.073 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:07.096 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:07.096 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:07.096 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:07.097 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:08.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:08.098 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:08.119 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:08.121 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:08.121 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:08.121 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:09.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:09.123 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:09.145 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:09.146 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:09.146 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:09.146 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:10.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:10.149 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:10.170 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:10.171 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:10.172 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:10.172 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:11.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:11.174 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:11.198 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:11.198 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:11.199 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:11.199 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:12.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:12.202 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:12.225 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:12.226 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:12.226 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:12.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:13.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:13.229 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:13.251 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:13.252 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:13.252 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:13.253 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:14.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:14.255 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:14.277 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:14.277 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:14.278 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:14.278 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:15.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:15.280 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:15.302 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:15.302 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:15.302 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:15.303 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:16.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:16.305 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:16.328 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:16.328 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:16.329 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:16.329 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:17.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:17.330 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:17.353 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:17.354 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:17.354 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:17.354 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:18.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:18.356 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:18.382 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:18.383 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:18.383 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:18.383 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:19.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:19.386 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:19.408 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:19.409 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:19.409 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:19.410 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:20.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:20.411 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:20.434 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:20.434 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:20.435 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:20.435 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:21.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:21.437 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:21.459 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:21.460 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:21.460 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:21.460 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:22.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:22.462 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:22.485 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:22.486 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:22.486 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:22.487 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:23.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:23.488 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:23.510 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:23.511 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:23.511 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:23.511 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:24.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:24.514 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:24.536 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:24.537 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:24.537 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:24.537 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:25.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:25.539 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:25.561 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:25.562 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:25.562 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:25.562 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:26.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:26.565 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:26.587 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:26.588 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:26.588 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:26.588 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:27.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:27.590 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:27.613 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:27.613 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:27.613 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:27.614 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:28.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:28.615 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:28.637 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:28.638 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:28.639 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:28.639 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:29.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:29.640 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:29.662 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:29.663 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:29.664 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:29.664 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:30.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:30.666 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:30.689 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:30.689 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:30.689 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:30.690 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:31.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:31.691 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:31.714 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:31.715 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:31.715 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:31.715 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:32.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:32.717 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:32.739 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:32.740 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:32.740 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:32.741 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:33.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:33.743 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:33.764 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:33.765 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:33.766 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:33.766 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:34.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:34.768 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:34.789 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:34.790 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:34.791 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:34.791 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:35.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:35.792 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:35.815 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:35.815 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:35.816 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:35.816 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:36.787 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:26:36.790 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:26:36.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:36.818 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:36.853 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:36.854 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:36.854 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:36.855 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:37.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:37.856 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:37.880 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:37.880 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:37.880 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:38.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:38.882 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:38.904 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:38.905 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:38.905 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:38.905 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:39.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:39.907 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:39.928 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:39.929 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:39.930 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:39.930 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:40.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:40.932 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:40.954 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:40.955 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:40.956 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:40.956 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:41.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:41.957 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:41.980 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:41.980 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:41.981 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:41.981 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:42.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:42.982 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:43.005 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:43.006 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:43.006 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:43.006 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:44.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:44.008 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:44.031 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:44.031 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:44.032 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:44.032 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:45.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:45.034 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:45.057 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:45.057 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:45.058 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:45.058 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:46.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:46.059 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:46.081 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:46.082 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:46.083 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:46.083 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:47.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:47.084 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:47.106 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:47.107 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:47.107 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:47.107 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:48.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:48.109 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:48.131 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:48.132 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:48.132 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:48.132 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:49.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:49.134 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:49.156 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:49.156 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:49.157 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:49.157 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:50.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:50.159 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:50.180 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:50.181 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:50.182 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:50.182 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:51.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:51.183 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:51.205 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:51.206 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:51.207 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:51.207 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:52.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:52.208 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:52.231 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:52.231 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:52.232 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:52.232 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:53.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:53.234 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:53.256 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:53.257 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:53.257 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:53.258 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:54.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:54.260 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:54.282 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:54.283 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:54.283 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:54.283 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:55.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:55.285 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:55.306 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:55.307 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:55.308 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:55.308 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:56.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:56.310 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:56.331 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:56.332 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:56.332 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:56.333 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:57.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:57.335 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:57.357 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:57.358 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:57.358 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:57.359 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:58.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:58.361 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:58.383 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:58.384 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:58.384 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:58.384 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:26:59.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:26:59.386 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:26:59.407 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:26:59.408 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:26:59.409 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:26:59.409 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:27:00.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:00.411 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:27:00.433 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:27:00.433 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC'} 2023-08-27T11:27:00.434 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-27T11:27:00.434 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:27:01.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:01.436 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:27:01.457 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:27:01.457 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_27_00_UTC 2023-08-27T11:27:01.458 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-27-11_26_00_UTC', 'scheduled-2023-08-27-11_27_00_UTC'} 2023-08-27T11:27:01.459 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-08-27-11_27_00_UTC'} 2023-08-27T11:27:01.459 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-27T11:27:01.459 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-08-27-11_27_00_UTC'] 2023-08-27T11:27:01.460 DEBUG:tasks.cephfs.test_snap_schedules:exec=1693135530.7945108, now = 1693135621.4600852, timo = 90 2023-08-27T11:27:02.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-08-27T11:27:02.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.602+0000 7f2ccf84a700 1 Processor -- start 2023-08-27T11:27:02.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.603+0000 7f2ccf84a700 1 -- start start 2023-08-27T11:27:02.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.603+0000 7f2ccf84a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2cc815a0f0 0x7f2cc80cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccf84a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc80cfae0 con 0x7f2cc806ed90 2023-08-27T11:27:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccf84a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc815e900 con 0x7f2cc80cd300 2023-08-27T11:27:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccd5e6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2cc815a0f0 0x7f2cc80cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccf84a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc80cae60 con 0x7f2cc815a0f0 2023-08-27T11:27:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccd5e6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2cc815a0f0 0x7f2cc80cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56786/0 (socket says 172.21.15.45:56786) 2023-08-27T11:27:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccd5e6700 1 -- 172.21.15.45:0/34174247 learned_addr learned my addr 172.21.15.45:0/34174247 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccd5e6700 1 -- 172.21.15.45:0/34174247 >> v1:172.21.15.135:6789/0 conn(0x7f2cc80cd300 legacy=0x7f2cc815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccd5e6700 1 -- 172.21.15.45:0/34174247 >> v1:172.21.15.45:6789/0 conn(0x7f2cc806ed90 legacy=0x7f2cc80cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccd5e6700 1 -- 172.21.15.45:0/34174247 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc80cb140 con 0x7f2cc815a0f0 2023-08-27T11:27:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2cbbfff700 1 -- 172.21.15.45:0/34174247 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848121163 0 0) 0x7f2cc80cfae0 con 0x7f2cc806ed90 2023-08-27T11:27:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2ccd5e6700 1 --2- 172.21.15.45:0/34174247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc815a0f0 0x7f2cc80cbc20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40098f0 tx=0x7f2cc4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bc54233b81ea8b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2cbbfff700 1 -- 172.21.15.45:0/34174247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400b8a0 con 0x7f2cc815a0f0 2023-08-27T11:27:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2cbbfff700 1 -- 172.21.15.45:0/34174247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc400ba00 con 0x7f2cc815a0f0 2023-08-27T11:27:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.604+0000 7f2cbbfff700 1 -- 172.21.15.45:0/34174247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400bd90 con 0x7f2cc815a0f0 2023-08-27T11:27:02.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.605+0000 7f2ccf84a700 1 -- 172.21.15.45:0/34174247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc815a0f0 msgr2=0x7f2cc80cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.605+0000 7f2ccf84a700 1 --2- 172.21.15.45:0/34174247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc815a0f0 0x7f2cc80cbc20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40098f0 tx=0x7f2cc4009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.605+0000 7f2ccf84a700 1 -- 172.21.15.45:0/34174247 shutdown_connections 2023-08-27T11:27:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.605+0000 7f2ccf84a700 1 --2- 172.21.15.45:0/34174247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc815a0f0 0x7f2cc80cbc20 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.605+0000 7f2ccf84a700 1 -- 172.21.15.45:0/34174247 >> 172.21.15.45:0/34174247 conn(0x7f2cc80cc7e0 msgr2=0x7f2cc80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.605+0000 7f2ccf84a700 1 -- 172.21.15.45:0/34174247 shutdown_connections 2023-08-27T11:27:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.605+0000 7f2ccf84a700 1 -- 172.21.15.45:0/34174247 wait complete. 2023-08-27T11:27:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.606+0000 7f2ccf84a700 1 Processor -- start 2023-08-27T11:27:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.606+0000 7f2ccf84a700 1 -- start start 2023-08-27T11:27:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccf84a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc806ed90 0x7f2cc80b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccf84a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2cc80cd300 0x7f2cc814e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccf84a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cc815a0f0 0x7f2cc814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccf84a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80b39b0 con 0x7f2cc806ed90 2023-08-27T11:27:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccf84a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80b5940 con 0x7f2cc815a0f0 2023-08-27T11:27:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccf84a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80b4fe0 con 0x7f2cc80cd300 2023-08-27T11:27:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccd5e6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc806ed90 0x7f2cc80b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccd5e6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc806ed90 0x7f2cc80b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56792/0 (socket says 172.21.15.45:56792) 2023-08-27T11:27:02.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccd5e6700 1 -- 172.21.15.45:0/294594039 learned_addr learned my addr 172.21.15.45:0/294594039 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2ccdde7700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cc815a0f0 0x7f2cc814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.607+0000 7f2cccde5700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2cc80cd300 0x7f2cc814e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2ccd5e6700 1 -- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2cc80cd300 msgr2=0x7f2cc814e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2ccd5e6700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2cc80cd300 0x7f2cc814e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2ccd5e6700 1 -- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cc815a0f0 msgr2=0x7f2cc814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2ccd5e6700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cc815a0f0 0x7f2cc814f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2ccd5e6700 1 -- 172.21.15.45:0/294594039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc4009550 con 0x7f2cc806ed90 2023-08-27T11:27:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2cccde5700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2cc80cd300 0x7f2cc814e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2ccdde7700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cc815a0f0 0x7f2cc814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:02.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2ccd5e6700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc806ed90 0x7f2cc80b5400 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4014040 tx=0x7f2cc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2cb9ffb700 1 -- 172.21.15.45:0/294594039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4011a70 con 0x7f2cc806ed90 2023-08-27T11:27:02.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2cb9ffb700 1 -- 172.21.15.45:0/294594039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc401e070 con 0x7f2cc806ed90 2023-08-27T11:27:02.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc80b4640 con 0x7f2cc806ed90 2023-08-27T11:27:02.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2cb9ffb700 1 -- 172.21.15.45:0/294594039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4011dd0 con 0x7f2cc806ed90 2023-08-27T11:27:02.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.608+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc80b4c00 con 0x7f2cc806ed90 2023-08-27T11:27:02.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.609+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc80b5250 con 0x7f2cc806ed90 2023-08-27T11:27:02.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.610+0000 7f2cb9ffb700 1 -- 172.21.15.45:0/294594039 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 88472+0+0 (secure 0 0 0) 0x7f2cc402d450 con 0x7f2cc806ed90 2023-08-27T11:27:02.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.610+0000 7f2cb9ffb700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f2cac06a250 0x7f2cac06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:02.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.610+0000 7f2cb9ffb700 1 -- 172.21.15.45:0/294594039 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f2cc4089ac0 con 0x7f2cc806ed90 2023-08-27T11:27:02.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.610+0000 7f2cccde5700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f2cac06a250 0x7f2cac06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:02.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.611+0000 7f2cccde5700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f2cac06a250 0x7f2cac06c710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80c38b0 tx=0x7f2cbc009040 comp rx=0 tx=0).ready entity=mgr.6157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:02.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.615+0000 7f2cb9ffb700 1 -- 172.21.15.45:0/294594039 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2cc4054e60 con 0x7f2cc806ed90 2023-08-27T11:27:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.845+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 --> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cc80cb700 con 0x7f2cac06a250 2023-08-27T11:27:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.909+0000 7f2cb9ffb700 1 -- 172.21.15.45:0/294594039 <== mgr.6157 v2:172.21.15.135:6832/1664443653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2cbc001440 con 0x7f2cac06a250 2023-08-27T11:27:02.912 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path snap_test_dir1 2023-08-27T11:27:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f2cac06a250 msgr2=0x7f2cac06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f2cac06a250 0x7f2cac06c710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80c38b0 tx=0x7f2cbc009040 comp rx=0 tx=0).stop 2023-08-27T11:27:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc806ed90 msgr2=0x7f2cc80b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc806ed90 0x7f2cc80b5400 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4014040 tx=0x7f2cc4004750 comp rx=0 tx=0).stop 2023-08-27T11:27:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 shutdown_connections 2023-08-27T11:27:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f2cac06a250 0x7f2cac06c710 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cc806ed90 0x7f2cc80b5400 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cc815a0f0 0x7f2cc814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 --2- 172.21.15.45:0/294594039 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2cc80cd300 0x7f2cc814e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:02.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 >> 172.21.15.45:0/294594039 conn(0x7f2cc80cc7e0 msgr2=0x7f2cc80ca390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:02.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 shutdown_connections 2023-08-27T11:27:02.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:02.912+0000 7f2ccf84a700 1 -- 172.21.15.45:0/294594039 wait complete. 2023-08-27T11:27:02.925 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-08-27T11:27:02.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:02.926 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-27T11:27:02.944 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:27:02.944 INFO:teuthology.orchestra.run.smithi045.stdout:scheduled-2023-08-27-11_27_00_UTC 2023-08-27T11:27:02.944 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-27-11_26_00_UTC 2023-08-27T11:27:02.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:02.945 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-27-11_26_00_UTC) 2023-08-27T11:27:03.006 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-27-11_27_00_UTC 2023-08-27T11:27:03.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:03.007 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-27-11_27_00_UTC) 2023-08-27T11:27:03.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:03.068 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-08-27T11:27:03.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-27T11:27:03.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.293+0000 7fb2a23b5700 1 Processor -- start 2023-08-27T11:27:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.293+0000 7fb2a23b5700 1 -- start start 2023-08-27T11:27:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.293+0000 7fb2a23b5700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb29c15a0f0 0x7fb29c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.293+0000 7fb2a23b5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 0x7fb29c15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.293+0000 7fb2a23b5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb29c15e030 0x7fb29c0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.293+0000 7fb2a23b5700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29c0cfae0 con 0x7fb29c15e030 2023-08-27T11:27:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.293+0000 7fb2a23b5700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29c15e900 con 0x7fb29c15a0f0 2023-08-27T11:27:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.293+0000 7fb2a23b5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0cae60 con 0x7fb29c0cd300 2023-08-27T11:27:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb2a0952700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb29c15e030 0x7fb29c0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb2a0952700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb29c15e030 0x7fb29c0cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56802/0 (socket says 172.21.15.45:56802) 2023-08-27T11:27:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb2a0952700 1 -- 172.21.15.45:0/2197776586 learned_addr learned my addr 172.21.15.45:0/2197776586 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb29b7fe700 1 --2- 172.21.15.45:0/2197776586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 0x7fb29c15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb29bfff700 1 --2- 172.21.15.45:0/2197776586 >> v2:172.21.15.135:3300/0 conn(0x7fb29c15a0f0 0x7fb29c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb2a0952700 1 -- 172.21.15.45:0/2197776586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 msgr2=0x7fb29c15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb2a0952700 1 --2- 172.21.15.45:0/2197776586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 0x7fb29c15daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb2a0952700 1 -- 172.21.15.45:0/2197776586 >> v2:172.21.15.135:3300/0 conn(0x7fb29c15a0f0 msgr2=0x7fb29c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb2a0952700 1 --2- 172.21.15.45:0/2197776586 >> v2:172.21.15.135:3300/0 conn(0x7fb29c15a0f0 0x7fb29c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb2a0952700 1 -- 172.21.15.45:0/2197776586 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0cb140 con 0x7fb29c15e030 2023-08-27T11:27:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb29b7fe700 1 --2- 172.21.15.45:0/2197776586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 0x7fb29c15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb29bfff700 1 --2- 172.21.15.45:0/2197776586 >> v2:172.21.15.135:3300/0 conn(0x7fb29c15a0f0 0x7fb29c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb2a0952700 1 --2- 172.21.15.45:0/2197776586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15e030 0x7fb29c0cd930 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0730c0 tx=0x7fb29400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74aeab5b1c2ed93c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb29affd700 1 -- 172.21.15.45:0/2197776586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400e070 con 0x7fb29c15e030 2023-08-27T11:27:03.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb29affd700 1 -- 172.21.15.45:0/2197776586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29400b9c0 con 0x7fb29c15e030 2023-08-27T11:27:03.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.294+0000 7fb29affd700 1 -- 172.21.15.45:0/2197776586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400bd70 con 0x7fb29c15e030 2023-08-27T11:27:03.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.295+0000 7fb2a23b5700 1 -- 172.21.15.45:0/2197776586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15e030 msgr2=0x7fb29c0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:03.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.295+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/2197776586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15e030 0x7fb29c0cd930 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0730c0 tx=0x7fb29400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:03.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.295+0000 7fb2a23b5700 1 -- 172.21.15.45:0/2197776586 shutdown_connections 2023-08-27T11:27:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.295+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/2197776586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15e030 0x7fb29c0cd930 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.295+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/2197776586 >> v2:172.21.15.135:3300/0 conn(0x7fb29c15a0f0 0x7fb29c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.295+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/2197776586 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 0x7fb29c15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.295+0000 7fb2a23b5700 1 -- 172.21.15.45:0/2197776586 >> 172.21.15.45:0/2197776586 conn(0x7fb29c0cc7e0 msgr2=0x7fb29c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.295+0000 7fb2a23b5700 1 -- 172.21.15.45:0/2197776586 shutdown_connections 2023-08-27T11:27:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.295+0000 7fb2a23b5700 1 -- 172.21.15.45:0/2197776586 wait complete. 2023-08-27T11:27:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.297+0000 7fb2a23b5700 1 Processor -- start 2023-08-27T11:27:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.297+0000 7fb2a23b5700 1 -- start start 2023-08-27T11:27:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb2a23b5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 0x7fb29c0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb2a23b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15a0f0 0x7fb29c1f0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb2a23b5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb29c15e030 0x7fb29c15ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb2a23b5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0b5d30 con 0x7fb29c15a0f0 2023-08-27T11:27:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb2a23b5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c15d3f0 con 0x7fb29c15e030 2023-08-27T11:27:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb2a23b5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c1f3f60 con 0x7fb29c0cd300 2023-08-27T11:27:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15a0f0 0x7fb29c1f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15a0f0 0x7fb29c1f0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56806/0 (socket says 172.21.15.45:56806) 2023-08-27T11:27:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29b7fe700 1 -- 172.21.15.45:0/1532503230 learned_addr learned my addr 172.21.15.45:0/1532503230 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29bfff700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 0x7fb29c0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb2a0952700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb29c15e030 0x7fb29c15ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29b7fe700 1 -- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 msgr2=0x7fb29c0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29b7fe700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 0x7fb29c0b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29b7fe700 1 -- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb29c15e030 msgr2=0x7fb29c15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29b7fe700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb29c15e030 0x7fb29c15ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29b7fe700 1 -- 172.21.15.45:0/1532503230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294009b30 con 0x7fb29c15a0f0 2023-08-27T11:27:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb29b7fe700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15a0f0 0x7fb29c1f0470 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb29000d800 tx=0x7fb29000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb298ff9700 1 -- 172.21.15.45:0/1532503230 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290004030 con 0x7fb29c15a0f0 2023-08-27T11:27:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb298ff9700 1 -- 172.21.15.45:0/1532503230 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb290004190 con 0x7fb29c15a0f0 2023-08-27T11:27:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c1f4240 con 0x7fb29c15a0f0 2023-08-27T11:27:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.298+0000 7fb298ff9700 1 -- 172.21.15.45:0/1532503230 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290004520 con 0x7fb29c15a0f0 2023-08-27T11:27:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.299+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c1f4800 con 0x7fb29c15a0f0 2023-08-27T11:27:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.299+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29c06d2c0 con 0x7fb29c15a0f0 2023-08-27T11:27:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.300+0000 7fb298ff9700 1 -- 172.21.15.45:0/1532503230 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) v1 ==== 88472+0+0 (secure 0 0 0) 0x7fb2900036a0 con 0x7fb29c15a0f0 2023-08-27T11:27:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.300+0000 7fb298ff9700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb28806a250 0x7fb28806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.300+0000 7fb298ff9700 1 -- 172.21.15.45:0/1532503230 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fb290087eb0 con 0x7fb29c15a0f0 2023-08-27T11:27:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.301+0000 7fb29bfff700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb28806a250 0x7fb28806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.301+0000 7fb29bfff700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb28806a250 0x7fb28806c710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb29c06b300 tx=0x7fb28c008040 comp rx=0 tx=0).ready entity=mgr.6157 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.305+0000 7fb298ff9700 1 -- 172.21.15.45:0/1532503230 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb290053300 con 0x7fb29c15a0f0 2023-08-27T11:27:03.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:03.565+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb29c0726d0 con 0x7fb29c15a0f0 2023-08-27T11:27:04.175 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.174+0000 7fb298ff9700 1 -- 172.21.15.45:0/1532503230 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fb2900536d0 con 0x7fb29c15a0f0 2023-08-27T11:27:04.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.176+0000 7fb298ff9700 1 -- 172.21.15.45:0/1532503230 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v26) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb290053300 con 0x7fb29c15a0f0 2023-08-27T11:27:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb28806a250 msgr2=0x7fb28806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb28806a250 0x7fb28806c710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb29c06b300 tx=0x7fb28c008040 comp rx=0 tx=0).stop 2023-08-27T11:27:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15a0f0 msgr2=0x7fb29c1f0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15a0f0 0x7fb29c1f0470 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb29000d800 tx=0x7fb29000db10 comp rx=0 tx=0).stop 2023-08-27T11:27:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 shutdown_connections 2023-08-27T11:27:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb29c15e030 0x7fb29c15ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb29c0cd300 0x7fb29c0b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb28806a250 0x7fb28806c710 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 --2- 172.21.15.45:0/1532503230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb29c15a0f0 0x7fb29c1f0470 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.178+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 >> 172.21.15.45:0/1532503230 conn(0x7fb29c0cc7e0 msgr2=0x7fb29c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:04.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.179+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 shutdown_connections 2023-08-27T11:27:04.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.179+0000 7fb2a23b5700 1 -- 172.21.15.45:0/1532503230 wait complete. 2023-08-27T11:27:04.195 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:27:04.196 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:04.247 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:04.288 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:27:04.289 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:27:04.290 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:04.307 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:27:04.307 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:04.326 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:04.325+0000 7f29b2b55200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:27:04.331 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:04.330+0000 7f9639e02200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:27:04.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:04.396 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:04.395+0000 7f9639e02200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:04.397 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:04.396+0000 7f29b2b55200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:04.467 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:04.466+0000 7f9639e02200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:27:04.473 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:04.471+0000 7f29b2b55200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:27:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.510+0000 7f82b3bca700 1 Processor -- start 2023-08-27T11:27:04.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.511+0000 7f82b3bca700 1 -- start start 2023-08-27T11:27:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.511+0000 7f82b3bca700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac155660 0x7f82ac06e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.511+0000 7f82b3bca700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82ac06a990 0x7f82ac15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.511+0000 7f82b3bca700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ac0cbf00 con 0x7f82ac070e80 2023-08-27T11:27:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.511+0000 7f82b3bca700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac15c760 con 0x7f82ac06a990 2023-08-27T11:27:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.511+0000 7f82b3bca700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0cc180 con 0x7f82ac155660 2023-08-27T11:27:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b2167700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82ac06a990 0x7f82ac15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b1165700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f82ac070e80 0x7f82ac06a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43658/0 (socket says 172.21.15.45:43658) 2023-08-27T11:27:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b2167700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82ac06a990 0x7f82ac15c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56884/0 (socket says 172.21.15.45:56884) 2023-08-27T11:27:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b1165700 1 -- 172.21.15.45:0/987239818 learned_addr learned my addr 172.21.15.45:0/987239818 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b1966700 1 --2- 172.21.15.45:0/987239818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac155660 0x7f82ac06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b2167700 1 -- 172.21.15.45:0/987239818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac155660 msgr2=0x7f82ac06e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b2167700 1 --2- 172.21.15.45:0/987239818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac155660 0x7f82ac06e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b2167700 1 -- 172.21.15.45:0/987239818 >> v1:172.21.15.45:6789/0 conn(0x7f82ac070e80 legacy=0x7f82ac06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b2167700 1 -- 172.21.15.45:0/987239818 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac0caa60 con 0x7f82ac06a990 2023-08-27T11:27:04.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b0964700 1 -- 172.21.15.45:0/987239818 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778946350 0 0) 0x7f82ac0cbf00 con 0x7f82ac070e80 2023-08-27T11:27:04.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b1966700 1 --2- 172.21.15.45:0/987239818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac155660 0x7f82ac06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:04.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b2167700 1 --2- 172.21.15.45:0/987239818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac06a990 0x7f82ac15c220 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009d90 tx=0x7f82a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f173c338085fdffe server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:04.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b0964700 1 -- 172.21.15.45:0/987239818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a400e070 con 0x7f82ac06a990 2023-08-27T11:27:04.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b0964700 1 -- 172.21.15.45:0/987239818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a400bbd0 con 0x7f82ac06a990 2023-08-27T11:27:04.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.512+0000 7f82b0964700 1 -- 172.21.15.45:0/987239818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4012430 con 0x7f82ac06a990 2023-08-27T11:27:04.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.513+0000 7f82b3bca700 1 -- 172.21.15.45:0/987239818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac06a990 msgr2=0x7f82ac15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.513+0000 7f82b3bca700 1 --2- 172.21.15.45:0/987239818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac06a990 0x7f82ac15c220 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009d90 tx=0x7f82a400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:04.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.513+0000 7f82b3bca700 1 -- 172.21.15.45:0/987239818 shutdown_connections 2023-08-27T11:27:04.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.513+0000 7f82b3bca700 1 --2- 172.21.15.45:0/987239818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac06a990 0x7f82ac15c220 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.513+0000 7f82b3bca700 1 --2- 172.21.15.45:0/987239818 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac155660 0x7f82ac06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.513+0000 7f82b3bca700 1 -- 172.21.15.45:0/987239818 >> 172.21.15.45:0/987239818 conn(0x7f82ac0cfe50 msgr2=0x7f82ac0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:04.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.513+0000 7f82b3bca700 1 -- 172.21.15.45:0/987239818 shutdown_connections 2023-08-27T11:27:04.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.513+0000 7f82b3bca700 1 -- 172.21.15.45:0/987239818 wait complete. 2023-08-27T11:27:04.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.514+0000 7f82b3bca700 1 Processor -- start 2023-08-27T11:27:04.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.514+0000 7f82b3bca700 1 -- start start 2023-08-27T11:27:04.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b3bca700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac06a990 0x7f82ac0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b3bca700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82ac070e80 0x7f82ac0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b3bca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac155660 0x7f82ac0a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b3bca700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0ce670 con 0x7f82ac155660 2023-08-27T11:27:04.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b3bca700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0ce950 con 0x7f82ac070e80 2023-08-27T11:27:04.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b3bca700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0cec30 con 0x7f82ac06a990 2023-08-27T11:27:04.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b2167700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac155660 0x7f82ac0a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b2167700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac155660 0x7f82ac0a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56890/0 (socket says 172.21.15.45:56890) 2023-08-27T11:27:04.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b2167700 1 -- 172.21.15.45:0/970657070 learned_addr learned my addr 172.21.15.45:0/970657070 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:04.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b1966700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac06a990 0x7f82ac0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.515+0000 7f82b1165700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82ac070e80 0x7f82ac0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82b2167700 1 -- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac06a990 msgr2=0x7f82ac0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82b2167700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac06a990 0x7f82ac0c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82b2167700 1 -- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82ac070e80 msgr2=0x7f82ac0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82b2167700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82ac070e80 0x7f82ac0cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82b2167700 1 -- 172.21.15.45:0/970657070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4009a70 con 0x7f82ac155660 2023-08-27T11:27:04.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82b2167700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac155660 0x7f82ac0a8eb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0cfcd0 tx=0x7f82a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82a27fc700 1 -- 172.21.15.45:0/970657070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4012430 con 0x7f82ac155660 2023-08-27T11:27:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82a27fc700 1 -- 172.21.15.45:0/970657070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a4020070 con 0x7f82ac155660 2023-08-27T11:27:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac0cef10 con 0x7f82ac155660 2023-08-27T11:27:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82a27fc700 1 -- 172.21.15.45:0/970657070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4004280 con 0x7f82ac155660 2023-08-27T11:27:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.516+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac0c59e0 con 0x7f82ac155660 2023-08-27T11:27:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.517+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac0cf430 con 0x7f82ac155660 2023-08-27T11:27:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.517+0000 7f82a27fc700 1 -- 172.21.15.45:0/970657070 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f82a40043e0 con 0x7f82ac155660 2023-08-27T11:27:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.518+0000 7f82a27fc700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f829006a200 0x7f829006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.518+0000 7f82a27fc700 1 -- 172.21.15.45:0/970657070 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6858+0+0 (secure 0 0 0) 0x7f82a4022070 con 0x7f82ac155660 2023-08-27T11:27:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.518+0000 7f82b1966700 1 -- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f829006a200 msgr2=0x7f829006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.518+0000 7f82b1966700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f829006a200 0x7f829006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.522+0000 7f82a27fc700 1 -- 172.21.15.45:0/970657070 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f82a4055e10 con 0x7f82ac155660 2023-08-27T11:27:04.545 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:04.544+0000 7f9639e02200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:27:04.558 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:04.556+0000 7f29b2b55200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:27:04.685 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:04.684+0000 7f9639e02200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:27:04.706 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:04.705+0000 7f29b2b55200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:27:04.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.719+0000 7f82b1966700 1 -- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f829006a200 msgr2=0x7f829006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:04.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.719+0000 7f82b1966700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f829006a200 0x7f829006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:04.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.727+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82ac0d0230 con 0x7f82ac155660 2023-08-27T11:27:04.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.728+0000 7f82a27fc700 1 -- 172.21.15.45:0/970657070 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+19297 (secure 0 0 0) 0x7f82a4059430 con 0x7f82ac155660 2023-08-27T11:27:04.731 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:04.731 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":72,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:04.163017+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:08.337329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:09.446448+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":72,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/3859433979":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/485008861":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/3010439837":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/2511615876":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1537214801":"2023-08-28T11:25:29.254479+0000","172.21.15.135:6832/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:6833/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1231508777":"2023-08-28T11:25:29.254479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]},{"pool":9,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f829006a200 msgr2=0x7f829006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f829006a200 0x7f829006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac155660 msgr2=0x7f82ac0a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac155660 0x7f82ac0a8eb0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0cfcd0 tx=0x7f82a4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 shutdown_connections 2023-08-27T11:27:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f829006a200 0x7f829006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82ac155660 0x7f82ac0a8eb0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f82ac06a990 0x7f82ac0c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 --2- 172.21.15.45:0/970657070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82ac070e80 0x7f82ac0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 >> 172.21.15.45:0/970657070 conn(0x7f82ac0cfe50 msgr2=0x7f82ac0b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:04.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 shutdown_connections 2023-08-27T11:27:04.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.731+0000 7f82b3bca700 1 -- 172.21.15.45:0/970657070 wait complete. 2023-08-27T11:27:04.745 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:25:08.337329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:25:09.446448+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 72, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:04.746 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:04.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.912+0000 7f23f13ec700 1 Processor -- start 2023-08-27T11:27:04.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.912+0000 7f23f13ec700 1 -- start start 2023-08-27T11:27:04.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23f13ec700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23ec0ca620 0x7f23ec0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23f13ec700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ec0ce340 con 0x7f23ec0caa00 2023-08-27T11:27:04.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23f13ec700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ec0ceec0 con 0x7f23ec0af200 2023-08-27T11:27:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23f13ec700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ec15d7c0 con 0x7f23ec0ca620 2023-08-27T11:27:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23eaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23ec0ca620 0x7f23ec0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23eaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23ec0ca620 0x7f23ec0cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56930/0 (socket says 172.21.15.45:56930) 2023-08-27T11:27:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23eb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f23ec0caa00 0x7f23ec0acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43710/0 (socket says 172.21.15.45:43710) 2023-08-27T11:27:04.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23eaffd700 1 -- 172.21.15.45:0/1705832093 learned_addr learned my addr 172.21.15.45:0/1705832093 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23eaffd700 1 -- 172.21.15.45:0/1705832093 >> v1:172.21.15.135:6789/0 conn(0x7f23ec0af200 legacy=0x7f23ec0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23eaffd700 1 -- 172.21.15.45:0/1705832093 >> v1:172.21.15.45:6789/0 conn(0x7f23ec0caa00 legacy=0x7f23ec0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23e9ffb700 1 -- 172.21.15.45:0/1705832093 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217345019 0 0) 0x7f23ec0ce340 con 0x7f23ec0caa00 2023-08-27T11:27:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.913+0000 7f23eaffd700 1 -- 172.21.15.45:0/1705832093 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec0cc570 con 0x7f23ec0ca620 2023-08-27T11:27:04.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23eaffd700 1 --2- 172.21.15.45:0/1705832093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0ca620 0x7f23ec0cf350 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0098f0 tx=0x7f23dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15049146e9a9aabc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23e9ffb700 1 -- 172.21.15.45:0/1705832093 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc00b8a0 con 0x7f23ec0ca620 2023-08-27T11:27:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23e9ffb700 1 -- 172.21.15.45:0/1705832093 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23dc00ba00 con 0x7f23ec0ca620 2023-08-27T11:27:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23e9ffb700 1 -- 172.21.15.45:0/1705832093 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc00bd90 con 0x7f23ec0ca620 2023-08-27T11:27:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23f13ec700 1 -- 172.21.15.45:0/1705832093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0ca620 msgr2=0x7f23ec0cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23f13ec700 1 --2- 172.21.15.45:0/1705832093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0ca620 0x7f23ec0cf350 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0098f0 tx=0x7f23dc009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23f13ec700 1 -- 172.21.15.45:0/1705832093 shutdown_connections 2023-08-27T11:27:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23f13ec700 1 --2- 172.21.15.45:0/1705832093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0ca620 0x7f23ec0cf350 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23f13ec700 1 -- 172.21.15.45:0/1705832093 >> 172.21.15.45:0/1705832093 conn(0x7f23ec0cc7e0 msgr2=0x7f23ec0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23f13ec700 1 -- 172.21.15.45:0/1705832093 shutdown_connections 2023-08-27T11:27:04.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.914+0000 7f23f13ec700 1 -- 172.21.15.45:0/1705832093 wait complete. 2023-08-27T11:27:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.916+0000 7f23f13ec700 1 Processor -- start 2023-08-27T11:27:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.916+0000 7f23f13ec700 1 -- start start 2023-08-27T11:27:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.916+0000 7f23f13ec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0af200 0x7f23ec147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.916+0000 7f23f13ec700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23ec0ca620 0x7f23ec1f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23f13ec700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ec0caa00 0x7f23ec147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23f13ec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0c59a0 con 0x7f23ec0af200 2023-08-27T11:27:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23f13ec700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec148090 con 0x7f23ec0caa00 2023-08-27T11:27:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23f13ec700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0c5b30 con 0x7f23ec0ca620 2023-08-27T11:27:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0af200 0x7f23ec147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0af200 0x7f23ec147510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56936/0 (socket says 172.21.15.45:56936) 2023-08-27T11:27:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eaffd700 1 -- 172.21.15.45:0/3620313114 learned_addr learned my addr 172.21.15.45:0/3620313114 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eb7fe700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ec0caa00 0x7f23ec147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23ea7fc700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23ec0ca620 0x7f23ec1f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eaffd700 1 -- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23ec0ca620 msgr2=0x7f23ec1f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eaffd700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23ec0ca620 0x7f23ec1f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eaffd700 1 -- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ec0caa00 msgr2=0x7f23ec147b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:04.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eaffd700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ec0caa00 0x7f23ec147b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:04.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eaffd700 1 -- 172.21.15.45:0/3620313114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc009550 con 0x7f23ec0af200 2023-08-27T11:27:04.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23eaffd700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0af200 0x7f23ec147510 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f23ec074300 tx=0x7f23dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.917+0000 7f23d37fe700 1 -- 172.21.15.45:0/3620313114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc01a900 con 0x7f23ec0af200 2023-08-27T11:27:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.918+0000 7f23d37fe700 1 -- 172.21.15.45:0/3620313114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23dc01f070 con 0x7f23ec0af200 2023-08-27T11:27:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.918+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ec1f2be0 con 0x7f23ec0af200 2023-08-27T11:27:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.918+0000 7f23d37fe700 1 -- 172.21.15.45:0/3620313114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc01ac60 con 0x7f23ec0af200 2023-08-27T11:27:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.918+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ec1f31a0 con 0x7f23ec0af200 2023-08-27T11:27:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.919+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ec15bc60 con 0x7f23ec0af200 2023-08-27T11:27:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.919+0000 7f23d37fe700 1 -- 172.21.15.45:0/3620313114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f23dc026480 con 0x7f23ec0af200 2023-08-27T11:27:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.920+0000 7f23d37fe700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f23d806a200 0x7f23d806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.920+0000 7f23d37fe700 1 -- 172.21.15.45:0/3620313114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6858+0+0 (secure 0 0 0) 0x7f23dc021070 con 0x7f23ec0af200 2023-08-27T11:27:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.920+0000 7f23ea7fc700 1 -- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f23d806a200 msgr2=0x7f23d806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.920+0000 7f23ea7fc700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f23d806a200 0x7f23d806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:04.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:04.924+0000 7f23d37fe700 1 -- 172.21.15.45:0/3620313114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f23dc0555d0 con 0x7f23ec0af200 2023-08-27T11:27:05.009 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:05.008+0000 7f9639e02200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:27:05.056 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:05.055+0000 7f29b2b55200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:27:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.121+0000 7f23ea7fc700 1 -- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f23d806a200 msgr2=0x7f23d806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.121+0000 7f23ea7fc700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f23d806a200 0x7f23d806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:05.126 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:05.125+0000 7f9639e02200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:27:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.129+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23ec0734a0 con 0x7f23ec0af200 2023-08-27T11:27:05.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.130+0000 7f23d37fe700 1 -- 172.21.15.45:0/3620313114 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23dc015020 con 0x7f23ec0af200 2023-08-27T11:27:05.132 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:27:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f23d806a200 msgr2=0x7f23d806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f23d806a200 0x7f23d806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0af200 msgr2=0x7f23ec147510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0af200 0x7f23ec147510 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f23ec074300 tx=0x7f23dc004750 comp rx=0 tx=0).stop 2023-08-27T11:27:05.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 shutdown_connections 2023-08-27T11:27:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f23d806a200 0x7f23d806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23ec0af200 0x7f23ec147510 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ec0caa00 0x7f23ec147b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 --2- 172.21.15.45:0/3620313114 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f23ec0ca620 0x7f23ec1f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 >> 172.21.15.45:0/3620313114 conn(0x7f23ec0cc7e0 msgr2=0x7f23ec0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 shutdown_connections 2023-08-27T11:27:05.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.132+0000 7f23f13ec700 1 -- 172.21.15.45:0/3620313114 wait complete. 2023-08-27T11:27:05.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:05.185 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:05.183+0000 7f29b2b55200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:27:05.203 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:05.202+0000 7f9639e02200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:27:05.263 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:05.261+0000 7f29b2b55200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:27:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.277+0000 7f9e42a49700 1 Processor -- start 2023-08-27T11:27:05.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.277+0000 7f9e42a49700 1 -- start start 2023-08-27T11:27:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.278+0000 7f9e42a49700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9e3c15a0f0 0x7f9e3c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.278+0000 7f9e42a49700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e3c0cd300 0x7f9e3c15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.278+0000 7f9e42a49700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0cfae0 con 0x7f9e3c15e030 2023-08-27T11:27:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.278+0000 7f9e42a49700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c15e900 con 0x7f9e3c0cd300 2023-08-27T11:27:05.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.278+0000 7f9e42a49700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c0cae60 con 0x7f9e3c15a0f0 2023-08-27T11:27:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.278+0000 7f9e3b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e3c0cd300 0x7f9e3c15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.278+0000 7f9e3b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e3c0cd300 0x7f9e3c15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56946/0 (socket says 172.21.15.45:56946) 2023-08-27T11:27:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.278+0000 7f9e3b7fe700 1 -- 172.21.15.45:0/2999909558 learned_addr learned my addr 172.21.15.45:0/2999909558 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.279+0000 7f9e3bfff700 1 --2- 172.21.15.45:0/2999909558 >> v2:172.21.15.135:3300/0 conn(0x7f9e3c15a0f0 0x7f9e3c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.279+0000 7f9e3b7fe700 1 -- 172.21.15.45:0/2999909558 >> v2:172.21.15.135:3300/0 conn(0x7f9e3c15a0f0 msgr2=0x7f9e3c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.279+0000 7f9e3b7fe700 1 --2- 172.21.15.45:0/2999909558 >> v2:172.21.15.135:3300/0 conn(0x7f9e3c15a0f0 0x7f9e3c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.279+0000 7f9e3b7fe700 1 -- 172.21.15.45:0/2999909558 >> v1:172.21.15.135:6789/0 conn(0x7f9e3c15e030 legacy=0x7f9e3c0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.279+0000 7f9e3b7fe700 1 -- 172.21.15.45:0/2999909558 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0cb140 con 0x7f9e3c0cd300 2023-08-27T11:27:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.279+0000 7f9e3b7fe700 1 --2- 172.21.15.45:0/2999909558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0cd300 0x7f9e3c15daf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0730c0 tx=0x7f9e30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4d177a4b4dcecd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:05.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.279+0000 7f9e3affd700 1 -- 172.21.15.45:0/2999909558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30011830 con 0x7f9e3c0cd300 2023-08-27T11:27:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.279+0000 7f9e3affd700 1 -- 172.21.15.45:0/2999909558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e30011990 con 0x7f9e3c0cd300 2023-08-27T11:27:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.279+0000 7f9e3affd700 1 -- 172.21.15.45:0/2999909558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30011d20 con 0x7f9e3c0cd300 2023-08-27T11:27:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.280+0000 7f9e42a49700 1 -- 172.21.15.45:0/2999909558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0cd300 msgr2=0x7f9e3c15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.280+0000 7f9e42a49700 1 --2- 172.21.15.45:0/2999909558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0cd300 0x7f9e3c15daf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0730c0 tx=0x7f9e30009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.280+0000 7f9e42a49700 1 -- 172.21.15.45:0/2999909558 shutdown_connections 2023-08-27T11:27:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.280+0000 7f9e42a49700 1 --2- 172.21.15.45:0/2999909558 >> v2:172.21.15.135:3300/0 conn(0x7f9e3c15a0f0 0x7f9e3c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.280+0000 7f9e42a49700 1 --2- 172.21.15.45:0/2999909558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c0cd300 0x7f9e3c15daf0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.280+0000 7f9e42a49700 1 -- 172.21.15.45:0/2999909558 >> 172.21.15.45:0/2999909558 conn(0x7f9e3c0cc7e0 msgr2=0x7f9e3c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.280+0000 7f9e42a49700 1 -- 172.21.15.45:0/2999909558 shutdown_connections 2023-08-27T11:27:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.280+0000 7f9e42a49700 1 -- 172.21.15.45:0/2999909558 wait complete. 2023-08-27T11:27:05.287 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:05.282+0000 7f9639e02200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:27:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.282+0000 7f9e42a49700 1 Processor -- start 2023-08-27T11:27:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.282+0000 7f9e42a49700 1 -- start start 2023-08-27T11:27:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e42a49700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e3c0cd300 0x7f9e3c152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e42a49700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e3c15a0f0 0x7f9e3c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e42a49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c15e030 0x7f9e3c15a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e42a49700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c153f80 con 0x7f9e3c15e030 2023-08-27T11:27:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e42a49700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c153400 con 0x7f9e3c0cd300 2023-08-27T11:27:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e42a49700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c1536e0 con 0x7f9e3c15a0f0 2023-08-27T11:27:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e40fe6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c15e030 0x7f9e3c15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e40fe6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c15e030 0x7f9e3c15a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56954/0 (socket says 172.21.15.45:56954) 2023-08-27T11:27:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e40fe6700 1 -- 172.21.15.45:0/3949893444 learned_addr learned my addr 172.21.15.45:0/3949893444 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e3bfff700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e3c0cd300 0x7f9e3c152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.283+0000 7f9e3b7fe700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e3c15a0f0 0x7f9e3c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e3bfff700 1 -- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e3c15a0f0 msgr2=0x7f9e3c06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e3bfff700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e3c15a0f0 0x7f9e3c06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e3bfff700 1 -- 172.21.15.45:0/3949893444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c15e030 msgr2=0x7f9e3c15a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e3bfff700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c15e030 0x7f9e3c15a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e3bfff700 1 -- 172.21.15.45:0/3949893444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30009550 con 0x7f9e3c0cd300 2023-08-27T11:27:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e40fe6700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c15e030 0x7f9e3c15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e3bfff700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e3c0cd300 0x7f9e3c152bb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00e900 tx=0x7f9e2c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e38ff9700 1 -- 172.21.15.45:0/3949893444 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c013680 con 0x7f9e3c0cd300 2023-08-27T11:27:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e38ff9700 1 -- 172.21.15.45:0/3949893444 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c0137e0 con 0x7f9e3c0cd300 2023-08-27T11:27:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c1539c0 con 0x7f9e3c0cd300 2023-08-27T11:27:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e38ff9700 1 -- 172.21.15.45:0/3949893444 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c013b50 con 0x7f9e3c0cd300 2023-08-27T11:27:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.284+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c1530f0 con 0x7f9e3c0cd300 2023-08-27T11:27:05.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.285+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c0726d0 con 0x7f9e3c0cd300 2023-08-27T11:27:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.286+0000 7f9e38ff9700 1 -- 172.21.15.45:0/3949893444 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f9e2c013cb0 con 0x7f9e3c0cd300 2023-08-27T11:27:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.286+0000 7f9e38ff9700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f9e2806a200 0x7f9e2806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.286+0000 7f9e38ff9700 1 -- 172.21.15.45:0/3949893444 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7f9e2c0889e0 con 0x7f9e3c0cd300 2023-08-27T11:27:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.286+0000 7f9e3b7fe700 1 -- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f9e2806a200 msgr2=0x7f9e2806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.286+0000 7f9e3b7fe700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f9e2806a200 0x7f9e2806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.290+0000 7f9e38ff9700 1 -- 172.21.15.45:0/3949893444 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9e2c053c10 con 0x7f9e3c0cd300 2023-08-27T11:27:05.353 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:05.351+0000 7f29b2b55200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:27:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.487+0000 7f9e3b7fe700 1 -- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f9e2806a200 msgr2=0x7f9e2806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.487+0000 7f9e3b7fe700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f9e2806a200 0x7f9e2806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.495+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9e3c0734a0 con 0x7f9e3c0cd300 2023-08-27T11:27:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.495+0000 7f9e38ff9700 1 -- 172.21.15.45:0/3949893444 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v73) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9e2c017020 con 0x7f9e3c0cd300 2023-08-27T11:27:05.498 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:27:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f9e2806a200 msgr2=0x7f9e2806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f9e2806a200 0x7f9e2806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e3c0cd300 msgr2=0x7f9e3c152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e3c0cd300 0x7f9e3c152bb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00e900 tx=0x7f9e2c00ec10 comp rx=0 tx=0).stop 2023-08-27T11:27:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 shutdown_connections 2023-08-27T11:27:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f9e2806a200 0x7f9e2806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e3c15e030 0x7f9e3c15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e3c0cd300 0x7f9e3c152bb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 --2- 172.21.15.45:0/3949893444 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9e3c15a0f0 0x7f9e3c06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 >> 172.21.15.45:0/3949893444 conn(0x7f9e3c0cc7e0 msgr2=0x7f9e3c0ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 shutdown_connections 2023-08-27T11:27:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.498+0000 7f9e42a49700 1 -- 172.21.15.45:0/3949893444 wait complete. 2023-08-27T11:27:05.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:05.658 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:05.657+0000 7f9639e02200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:27:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.686+0000 7f6bd0f33700 1 Processor -- start 2023-08-27T11:27:05.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.686+0000 7f6bd0f33700 1 -- start start 2023-08-27T11:27:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bd0f33700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6bcc15af30 0x7f6bcc0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bd0f33700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6bcc0cf960 0x7f6bcc0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bd0f33700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc0cb8b0 con 0x7f6bcc15b310 2023-08-27T11:27:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bd0f33700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc06f9c0 con 0x7f6bcc0cf960 2023-08-27T11:27:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bd0f33700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc0cecd0 con 0x7f6bcc15af30 2023-08-27T11:27:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bc9d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6bcc0cf960 0x7f6bcc0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bc9d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6bcc0cf960 0x7f6bcc0cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56994/0 (socket says 172.21.15.45:56994) 2023-08-27T11:27:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bc9d9b700 1 -- 172.21.15.45:0/370557227 learned_addr learned my addr 172.21.15.45:0/370557227 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bcad9d700 1 --1- 172.21.15.45:0/370557227 >> v1:172.21.15.45:6789/0 conn(0x7f6bcc15b310 0x7f6bcc15ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43772/0 (socket says 172.21.15.45:43772) 2023-08-27T11:27:05.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bca59c700 1 --2- 172.21.15.45:0/370557227 >> v2:172.21.15.135:3300/0 conn(0x7f6bcc15af30 0x7f6bcc0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bc9d9b700 1 -- 172.21.15.45:0/370557227 >> v2:172.21.15.135:3300/0 conn(0x7f6bcc15af30 msgr2=0x7f6bcc0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bc9d9b700 1 --2- 172.21.15.45:0/370557227 >> v2:172.21.15.135:3300/0 conn(0x7f6bcc15af30 0x7f6bcc0cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bc9d9b700 1 -- 172.21.15.45:0/370557227 >> v1:172.21.15.45:6789/0 conn(0x7f6bcc15b310 legacy=0x7f6bcc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.687+0000 7f6bc9d9b700 1 -- 172.21.15.45:0/370557227 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc0cbae0 con 0x7f6bcc0cf960 2023-08-27T11:27:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.688+0000 7f6bc9d9b700 1 --2- 172.21.15.45:0/370557227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc0cf960 0x7f6bcc0cd770 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc09a8d0 tx=0x7f6bc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6462724766c614 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.688+0000 7f6bc959a700 1 -- 172.21.15.45:0/370557227 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0011830 con 0x7f6bcc0cf960 2023-08-27T11:27:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.688+0000 7f6bc959a700 1 -- 172.21.15.45:0/370557227 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc0011990 con 0x7f6bcc0cf960 2023-08-27T11:27:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.688+0000 7f6bc959a700 1 -- 172.21.15.45:0/370557227 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0011d20 con 0x7f6bcc0cf960 2023-08-27T11:27:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.688+0000 7f6bd0f33700 1 -- 172.21.15.45:0/370557227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc0cf960 msgr2=0x7f6bcc0cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.688+0000 7f6bd0f33700 1 --2- 172.21.15.45:0/370557227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc0cf960 0x7f6bcc0cd770 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc09a8d0 tx=0x7f6bc0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.689+0000 7f6bd0f33700 1 -- 172.21.15.45:0/370557227 shutdown_connections 2023-08-27T11:27:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.689+0000 7f6bd0f33700 1 --2- 172.21.15.45:0/370557227 >> v2:172.21.15.135:3300/0 conn(0x7f6bcc15af30 0x7f6bcc0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.689+0000 7f6bd0f33700 1 --2- 172.21.15.45:0/370557227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc0cf960 0x7f6bcc0cd770 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.689+0000 7f6bd0f33700 1 -- 172.21.15.45:0/370557227 >> 172.21.15.45:0/370557227 conn(0x7f6bcc09a1a0 msgr2=0x7f6bcc0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:05.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.689+0000 7f6bd0f33700 1 -- 172.21.15.45:0/370557227 shutdown_connections 2023-08-27T11:27:05.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.689+0000 7f6bd0f33700 1 -- 172.21.15.45:0/370557227 wait complete. 2023-08-27T11:27:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.690+0000 7f6bd0f33700 1 Processor -- start 2023-08-27T11:27:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.690+0000 7f6bd0f33700 1 -- start start 2023-08-27T11:27:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bd0f33700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bcc0cf960 0x7f6bcc0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bd0f33700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc15af30 0x7f6bcc1f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bd0f33700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc15b310 0x7f6bcc0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bd0f33700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc15cee0 con 0x7f6bcc15b310 2023-08-27T11:27:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bd0f33700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc15c260 con 0x7f6bcc15af30 2023-08-27T11:27:05.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bd0f33700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc15c540 con 0x7f6bcc0cf960 2023-08-27T11:27:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bcad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc15b310 0x7f6bcc0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bcad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc15b310 0x7f6bcc0b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57002/0 (socket says 172.21.15.45:57002) 2023-08-27T11:27:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bcad9d700 1 -- 172.21.15.45:0/2980325962 learned_addr learned my addr 172.21.15.45:0/2980325962 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bc9d9b700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc15af30 0x7f6bcc1f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bca59c700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bcc0cf960 0x7f6bcc0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bcad9d700 1 -- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bcc0cf960 msgr2=0x7f6bcc0b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bcad9d700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bcc0cf960 0x7f6bcc0b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bcad9d700 1 -- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc15af30 msgr2=0x7f6bcc1f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bcad9d700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc15af30 0x7f6bcc1f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bcad9d700 1 -- 172.21.15.45:0/2980325962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc0009550 con 0x7f6bcc15b310 2023-08-27T11:27:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.691+0000 7f6bcad9d700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc15b310 0x7f6bcc0b5f60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400cb10 tx=0x7f6bc400ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.692+0000 7f6bbaffd700 1 -- 172.21.15.45:0/2980325962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400a9a0 con 0x7f6bcc15b310 2023-08-27T11:27:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.692+0000 7f6bbaffd700 1 -- 172.21.15.45:0/2980325962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc400ab00 con 0x7f6bcc15b310 2023-08-27T11:27:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.692+0000 7f6bbaffd700 1 -- 172.21.15.45:0/2980325962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400ae70 con 0x7f6bcc15b310 2023-08-27T11:27:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.692+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc15c820 con 0x7f6bcc15b310 2023-08-27T11:27:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.692+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc1f3dd0 con 0x7f6bcc15b310 2023-08-27T11:27:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.693+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc15d020 con 0x7f6bcc15b310 2023-08-27T11:27:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.693+0000 7f6bbaffd700 1 -- 172.21.15.45:0/2980325962 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f6bc4007070 con 0x7f6bcc15b310 2023-08-27T11:27:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.694+0000 7f6bbaffd700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f6bac06a200 0x7f6bac06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.694+0000 7f6bca59c700 1 -- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f6bac06a200 msgr2=0x7f6bac06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.694+0000 7f6bbaffd700 1 -- 172.21.15.45:0/2980325962 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7f6bc4088510 con 0x7f6bcc15b310 2023-08-27T11:27:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.694+0000 7f6bca59c700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f6bac06a200 0x7f6bac06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:05.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.698+0000 7f6bbaffd700 1 -- 172.21.15.45:0/2980325962 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6bc4053890 con 0x7f6bcc15b310 2023-08-27T11:27:05.753 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:05.751+0000 7f29b2b55200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:27:05.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.895+0000 7f6bca59c700 1 -- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f6bac06a200 msgr2=0x7f6bac06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:05.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.895+0000 7f6bca59c700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f6bac06a200 0x7f6bac06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.900+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6bcc0734a0 con 0x7f6bcc15b310 2023-08-27T11:27:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.900+0000 7f6bbaffd700 1 -- 172.21.15.45:0/2980325962 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6bc4056eb0 con 0x7f6bcc15b310 2023-08-27T11:27:05.902 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:27:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f6bac06a200 msgr2=0x7f6bac06c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f6bac06a200 0x7f6bac06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc15b310 msgr2=0x7f6bcc0b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc15b310 0x7f6bcc0b5f60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400cb10 tx=0x7f6bc400ce20 comp rx=0 tx=0).stop 2023-08-27T11:27:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 shutdown_connections 2023-08-27T11:27:05.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f6bac06a200 0x7f6bac06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bcc15b310 0x7f6bcc0b5f60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bcc0cf960 0x7f6bcc0b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 --2- 172.21.15.45:0/2980325962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bcc15af30 0x7f6bcc1f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.902+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 >> 172.21.15.45:0/2980325962 conn(0x7f6bcc09a1a0 msgr2=0x7f6bcc0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:05.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.903+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 shutdown_connections 2023-08-27T11:27:05.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:05.903+0000 7f6bd0f33700 1 -- 172.21.15.45:0/2980325962 wait complete. 2023-08-27T11:27:05.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.048+0000 7f1feea0f700 1 Processor -- start 2023-08-27T11:27:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.048+0000 7f1feea0f700 1 -- start start 2023-08-27T11:27:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.049+0000 7f1feea0f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1fe815d2c0 0x7f1fe80d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1feea0f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1fe80b8c20 0x7f1fe80cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1feea0f700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe815d6a0 con 0x7f1fe80ce2e0 2023-08-27T11:27:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1feea0f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe80ce730 con 0x7f1fe815d2c0 2023-08-27T11:27:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1feea0f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe80d2c90 con 0x7f1fe80b8c20 2023-08-27T11:27:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1fe815d2c0 0x7f1fe80d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1fe815d2c0 0x7f1fe80d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57008/0 (socket says 172.21.15.45:57008) 2023-08-27T11:27:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe7fff700 1 -- 172.21.15.45:0/3516342303 learned_addr learned my addr 172.21.15.45:0/3516342303 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe77fe700 1 --2- 172.21.15.45:0/3516342303 >> v2:172.21.15.135:3300/0 conn(0x7f1fe80b8c20 0x7f1fe80cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe7fff700 1 -- 172.21.15.45:0/3516342303 >> v2:172.21.15.135:3300/0 conn(0x7f1fe80b8c20 msgr2=0x7f1fe80cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe7fff700 1 --2- 172.21.15.45:0/3516342303 >> v2:172.21.15.135:3300/0 conn(0x7f1fe80b8c20 0x7f1fe80cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe7fff700 1 -- 172.21.15.45:0/3516342303 >> v1:172.21.15.135:6789/0 conn(0x7f1fe80ce2e0 legacy=0x7f1fe806c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe7fff700 1 -- 172.21.15.45:0/3516342303 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80d1bf0 con 0x7f1fe815d2c0 2023-08-27T11:27:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe7fff700 1 --2- 172.21.15.45:0/3516342303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe815d2c0 0x7f1fe80d13a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098f0 tx=0x7f1fd8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615a7ba0f16c89ff server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe6ffd700 1 -- 172.21.15.45:0/3516342303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011830 con 0x7f1fe815d2c0 2023-08-27T11:27:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe6ffd700 1 -- 172.21.15.45:0/3516342303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8011990 con 0x7f1fe815d2c0 2023-08-27T11:27:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.050+0000 7f1fe6ffd700 1 -- 172.21.15.45:0/3516342303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011d20 con 0x7f1fe815d2c0 2023-08-27T11:27:06.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.051+0000 7f1feea0f700 1 -- 172.21.15.45:0/3516342303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe815d2c0 msgr2=0x7f1fe80d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.051+0000 7f1feea0f700 1 --2- 172.21.15.45:0/3516342303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe815d2c0 0x7f1fe80d13a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098f0 tx=0x7f1fd8009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.051+0000 7f1feea0f700 1 -- 172.21.15.45:0/3516342303 shutdown_connections 2023-08-27T11:27:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.051+0000 7f1feea0f700 1 --2- 172.21.15.45:0/3516342303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe815d2c0 0x7f1fe80d13a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.051+0000 7f1feea0f700 1 --2- 172.21.15.45:0/3516342303 >> v2:172.21.15.135:3300/0 conn(0x7f1fe80b8c20 0x7f1fe80cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.051+0000 7f1feea0f700 1 -- 172.21.15.45:0/3516342303 >> 172.21.15.45:0/3516342303 conn(0x7f1fe80d2fd0 msgr2=0x7f1fe80d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.051+0000 7f1feea0f700 1 -- 172.21.15.45:0/3516342303 shutdown_connections 2023-08-27T11:27:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.051+0000 7f1feea0f700 1 -- 172.21.15.45:0/3516342303 wait complete. 2023-08-27T11:27:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.052+0000 7f1feea0f700 1 Processor -- start 2023-08-27T11:27:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.053+0000 7f1feea0f700 1 -- start start 2023-08-27T11:27:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.053+0000 7f1feea0f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80b8c20 0x7f1fe80c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.053+0000 7f1feea0f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe80ce2e0 0x7f1fe80c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.053+0000 7f1feea0f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe815d2c0 0x7f1fe815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.053+0000 7f1feea0f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80caeb0 con 0x7f1fe80b8c20 2023-08-27T11:27:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.053+0000 7f1feea0f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c99a0 con 0x7f1fe80ce2e0 2023-08-27T11:27:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.053+0000 7f1feea0f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c7170 con 0x7f1fe815d2c0 2023-08-27T11:27:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80b8c20 0x7f1fe80c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80b8c20 0x7f1fe80c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57014/0 (socket says 172.21.15.45:57014) 2023-08-27T11:27:06.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe80ce2e0 0x7f1fe80c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe7fff700 1 -- 172.21.15.45:0/1635303602 learned_addr learned my addr 172.21.15.45:0/1635303602 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:06.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fecfac700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe815d2c0 0x7f1fe815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe7fff700 1 -- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe815d2c0 msgr2=0x7f1fe815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe7fff700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe815d2c0 0x7f1fe815eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe7fff700 1 -- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe80ce2e0 msgr2=0x7f1fe80c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe7fff700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe80ce2e0 0x7f1fe80c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe7fff700 1 -- 172.21.15.45:0/1635303602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8009550 con 0x7f1fe80b8c20 2023-08-27T11:27:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe7fff700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80b8c20 0x7f1fe80c6ae0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009c40 tx=0x7f1fd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe4ff9700 1 -- 172.21.15.45:0/1635303602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8004300 con 0x7f1fe80b8c20 2023-08-27T11:27:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe4ff9700 1 -- 172.21.15.45:0/1635303602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd801f070 con 0x7f1fe80b8c20 2023-08-27T11:27:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1fe4ff9700 1 -- 172.21.15.45:0/1635303602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80036a0 con 0x7f1fe80b8c20 2023-08-27T11:27:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe80ca580 con 0x7f1fe80b8c20 2023-08-27T11:27:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.054+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe80cab40 con 0x7f1fe80b8c20 2023-08-27T11:27:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.056+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe804e620 con 0x7f1fe80b8c20 2023-08-27T11:27:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.056+0000 7f1fe4ff9700 1 -- 172.21.15.45:0/1635303602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f1fd8021030 con 0x7f1fe80b8c20 2023-08-27T11:27:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.056+0000 7f1fe4ff9700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1fd406a200 0x7f1fd406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.056+0000 7f1fe4ff9700 1 -- 172.21.15.45:0/1635303602 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7f1fd8089d50 con 0x7f1fe80b8c20 2023-08-27T11:27:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.056+0000 7f1fe77fe700 1 -- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1fd406a200 msgr2=0x7f1fd406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.056+0000 7f1fe77fe700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1fd406a200 0x7f1fd406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.061+0000 7f1fe4ff9700 1 -- 172.21.15.45:0/1635303602 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1fd808d030 con 0x7f1fe80b8c20 2023-08-27T11:27:06.228 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:06.227+0000 7f9639e02200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:27:06.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.257+0000 7f1fe77fe700 1 -- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1fd406a200 msgr2=0x7f1fd406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.257+0000 7f1fe77fe700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1fd406a200 0x7f1fd406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:06.298 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:06.297+0000 7f9639e02200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:27:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.306+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fe80d2760 con 0x7f1fe80b8c20 2023-08-27T11:27:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.306+0000 7f1fe4ff9700 1 -- 172.21.15.45:0/1635303602 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fd8055030 con 0x7f1fe80b8c20 2023-08-27T11:27:06.308 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:27:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.308+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1fd406a200 msgr2=0x7f1fd406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.308+0000 7f1feea0f700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1fd406a200 0x7f1fd406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80b8c20 msgr2=0x7f1fe80c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80b8c20 0x7f1fe80c6ae0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009c40 tx=0x7f1fd8004750 comp rx=0 tx=0).stop 2023-08-27T11:27:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 shutdown_connections 2023-08-27T11:27:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1fd406a200 0x7f1fd406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80b8c20 0x7f1fe80c6ae0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1fe815d2c0 0x7f1fe815eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 --2- 172.21.15.45:0/1635303602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1fe80ce2e0 0x7f1fe80c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 >> 172.21.15.45:0/1635303602 conn(0x7f1fe80d2fd0 msgr2=0x7f1fe80b6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 shutdown_connections 2023-08-27T11:27:06.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.309+0000 7f1feea0f700 1 -- 172.21.15.45:0/1635303602 wait complete. 2023-08-27T11:27:06.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:27:06.376 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:06.374+0000 7f29b2b55200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:27:06.450 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:06.449+0000 7f29b2b55200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:27:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.453+0000 7fab504b8700 1 Processor -- start 2023-08-27T11:27:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.453+0000 7fab504b8700 1 -- start start 2023-08-27T11:27:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab504b8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 0x7fab480b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab504b8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4806b160 con 0x7fab48160d00 2023-08-27T11:27:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab504b8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480cdf90 con 0x7fab4815d340 2023-08-27T11:27:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab504b8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab48072010 con 0x7fab4806a990 2023-08-27T11:27:06.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4da53700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fab48160d00 0x7fab4806a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43802/0 (socket says 172.21.15.45:43802) 2023-08-27T11:27:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4da53700 1 -- 172.21.15.45:0/3761192265 learned_addr learned my addr 172.21.15.45:0/3761192265 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4ea55700 1 --2- 172.21.15.45:0/3761192265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 0x7fab480b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4d252700 1 -- 172.21.15.45:0/3761192265 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231500503 0 0) 0x7fab4806b160 con 0x7fab48160d00 2023-08-27T11:27:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4d252700 1 -- 172.21.15.45:0/3761192265 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab340035b0 con 0x7fab48160d00 2023-08-27T11:27:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4d252700 1 -- 172.21.15.45:0/3761192265 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248966509 0 0) 0x7fab480cdf90 con 0x7fab4815d340 2023-08-27T11:27:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4d252700 1 -- 172.21.15.45:0/3761192265 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab4806b160 con 0x7fab4815d340 2023-08-27T11:27:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4ea55700 1 -- 172.21.15.45:0/3761192265 >> v1:172.21.15.135:6789/0 conn(0x7fab4815d340 legacy=0x7fab480cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4ea55700 1 -- 172.21.15.45:0/3761192265 >> v1:172.21.15.45:6789/0 conn(0x7fab48160d00 legacy=0x7fab4806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4ea55700 1 -- 172.21.15.45:0/3761192265 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480af590 con 0x7fab4806a990 2023-08-27T11:27:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.454+0000 7fab4ea55700 1 --2- 172.21.15.45:0/3761192265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 0x7fab480b18a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab40009dc0 tx=0x7fab4000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ab7172b49ca0d42 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab4d252700 1 -- 172.21.15.45:0/3761192265 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000e070 con 0x7fab4806a990 2023-08-27T11:27:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab4d252700 1 -- 172.21.15.45:0/3761192265 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4000bb20 con 0x7fab4806a990 2023-08-27T11:27:06.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab4d252700 1 -- 172.21.15.45:0/3761192265 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40012430 con 0x7fab4806a990 2023-08-27T11:27:06.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab504b8700 1 -- 172.21.15.45:0/3761192265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 msgr2=0x7fab480b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab504b8700 1 --2- 172.21.15.45:0/3761192265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 0x7fab480b18a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab40009dc0 tx=0x7fab4000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab504b8700 1 -- 172.21.15.45:0/3761192265 shutdown_connections 2023-08-27T11:27:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab504b8700 1 --2- 172.21.15.45:0/3761192265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 0x7fab480b18a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab504b8700 1 -- 172.21.15.45:0/3761192265 >> 172.21.15.45:0/3761192265 conn(0x7fab480cc7c0 msgr2=0x7fab480ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab504b8700 1 -- 172.21.15.45:0/3761192265 shutdown_connections 2023-08-27T11:27:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.455+0000 7fab504b8700 1 -- 172.21.15.45:0/3761192265 wait complete. 2023-08-27T11:27:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.457+0000 7fab504b8700 1 Processor -- start 2023-08-27T11:27:06.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.457+0000 7fab504b8700 1 -- start start 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.457+0000 7fab504b8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 0x7fab481f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.457+0000 7fab504b8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4815d340 0x7fab481f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.457+0000 7fab504b8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab48160d00 0x7fab480ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.457+0000 7fab504b8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480adf90 con 0x7fab4815d340 2023-08-27T11:27:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.457+0000 7fab504b8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480ad380 con 0x7fab48160d00 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.457+0000 7fab504b8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480ad660 con 0x7fab4806a990 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4da53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4815d340 0x7fab481f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4da53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4815d340 0x7fab481f0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57032/0 (socket says 172.21.15.45:57032) 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4da53700 1 -- 172.21.15.45:0/2074417303 learned_addr learned my addr 172.21.15.45:0/2074417303 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4ea55700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab48160d00 0x7fab480ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4e254700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 0x7fab481f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4da53700 1 -- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 msgr2=0x7fab481f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4da53700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 0x7fab481f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4da53700 1 -- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab48160d00 msgr2=0x7fab480ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4da53700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab48160d00 0x7fab480ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4da53700 1 -- 172.21.15.45:0/2074417303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab40009a70 con 0x7fab4815d340 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4ea55700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab48160d00 0x7fab480ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab4da53700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4815d340 0x7fab481f0100 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fab4400c700 tx=0x7fab4400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab3effd700 1 -- 172.21.15.45:0/2074417303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44005d70 con 0x7fab4815d340 2023-08-27T11:27:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab3effd700 1 -- 172.21.15.45:0/2074417303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab440070d0 con 0x7fab4815d340 2023-08-27T11:27:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab480ad940 con 0x7fab4815d340 2023-08-27T11:27:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab3effd700 1 -- 172.21.15.45:0/2074417303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44007460 con 0x7fab4815d340 2023-08-27T11:27:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.458+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab480ca190 con 0x7fab4815d340 2023-08-27T11:27:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.459+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab48072690 con 0x7fab4815d340 2023-08-27T11:27:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.460+0000 7fab3effd700 1 -- 172.21.15.45:0/2074417303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fab440075c0 con 0x7fab4815d340 2023-08-27T11:27:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.460+0000 7fab3effd700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fab2c06a200 0x7fab2c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.460+0000 7fab4e254700 1 -- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fab2c06a200 msgr2=0x7fab2c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.460+0000 7fab3effd700 1 -- 172.21.15.45:0/2074417303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7fab44087e50 con 0x7fab4815d340 2023-08-27T11:27:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.460+0000 7fab4e254700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fab2c06a200 0x7fab2c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.465+0000 7fab3effd700 1 -- 172.21.15.45:0/2074417303 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fab44053090 con 0x7fab4815d340 2023-08-27T11:27:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.661+0000 7fab4e254700 1 -- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fab2c06a200 msgr2=0x7fab2c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.661+0000 7fab4e254700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fab2c06a200 0x7fab2c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.685+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fab480ae0d0 con 0x7fab4815d340 2023-08-27T11:27:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.685+0000 7fab3effd700 1 -- 172.21.15.45:0/2074417303 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v73) v1 ==== 80+0+477 (secure 0 0 0) 0x7fab440566b0 con 0x7fab4815d340 2023-08-27T11:27:06.688 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/3859433979 2023-08-28T11:25:29.254479+0000 2023-08-27T11:27:06.688 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/485008861 2023-08-28T11:25:29.254479+0000 2023-08-27T11:27:06.688 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/3010439837 2023-08-28T11:25:29.254479+0000 2023-08-27T11:27:06.688 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/2511615876 2023-08-28T11:25:29.254479+0000 2023-08-27T11:27:06.689 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/1537214801 2023-08-28T11:25:29.254479+0000 2023-08-27T11:27:06.689 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6832/3181978685 2023-08-28T11:25:29.254479+0000 2023-08-27T11:27:06.689 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6833/3181978685 2023-08-28T11:25:29.254479+0000 2023-08-27T11:27:06.689 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/1231508777 2023-08-28T11:25:29.254479+0000 2023-08-27T11:27:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fab2c06a200 msgr2=0x7fab2c06c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fab2c06a200 0x7fab2c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4815d340 msgr2=0x7fab481f0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4815d340 0x7fab481f0100 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fab4400c700 tx=0x7fab4400ca10 comp rx=0 tx=0).stop 2023-08-27T11:27:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 shutdown_connections 2023-08-27T11:27:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab48160d00 0x7fab480ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fab4806a990 0x7fab481f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fab2c06a200 0x7fab2c06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 --2- 172.21.15.45:0/2074417303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4815d340 0x7fab481f0100 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 >> 172.21.15.45:0/2074417303 conn(0x7fab480cc7c0 msgr2=0x7fab480af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 shutdown_connections 2023-08-27T11:27:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:06.689+0000 7fab504b8700 1 -- 172.21.15.45:0/2074417303 wait complete. 2023-08-27T11:27:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:listed 8 entries 2023-08-27T11:27:06.708 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:27:06.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:06.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:06.751 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:27:06.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:06.752 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:27:06.809 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:27:06.810 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:06.828 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:06.829 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:27:06.829 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:06.853 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:27:06.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:27:06.868 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:27:06.868 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:06.869 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:27:06.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:06.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:06.936 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:06.935+0000 7f9639e02200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:07.015 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:07.014+0000 7f9639e02200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:27:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.035+0000 7fb2fd174700 1 Processor -- start 2023-08-27T11:27:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.035+0000 7fb2fd174700 1 -- start start 2023-08-27T11:27:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2fd174700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80b45d0 0x7fb2f80d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2fd174700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2f80cfd60 0x7fb2f80b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2fd174700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f80ceae0 con 0x7fb2f80cf980 2023-08-27T11:27:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2fd174700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f80cf0a0 con 0x7fb2f80cfd60 2023-08-27T11:27:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2fd174700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80cedc0 con 0x7fb2f80b45d0 2023-08-27T11:27:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2ee59c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80b45d0 0x7fb2f80d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2f759e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2f80cfd60 0x7fb2f80b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2f759e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb2f80cfd60 0x7fb2f80b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:45900/0 (socket says 172.21.15.45:45900) 2023-08-27T11:27:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2ee59c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80b45d0 0x7fb2f80d1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:54952/0 (socket says 172.21.15.45:54952) 2023-08-27T11:27:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2f759e700 1 -- 172.21.15.45:0/1957791701 learned_addr learned my addr 172.21.15.45:0/1957791701 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2f6d9d700 1 --1- 172.21.15.45:0/1957791701 >> v1:172.21.15.135:6789/0 conn(0x7fb2f80cf980 0x7fb2f80d0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:39480/0 (socket says 172.21.15.45:39480) 2023-08-27T11:27:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2f759e700 1 -- 172.21.15.45:0/1957791701 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80b45d0 msgr2=0x7fb2f80d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2f759e700 1 --2- 172.21.15.45:0/1957791701 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80b45d0 0x7fb2f80d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2f759e700 1 -- 172.21.15.45:0/1957791701 >> v1:172.21.15.135:6789/0 conn(0x7fb2f80cf980 legacy=0x7fb2f80d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2f759e700 1 -- 172.21.15.45:0/1957791701 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f80b22c0 con 0x7fb2f80cfd60 2023-08-27T11:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.036+0000 7fb2ee59c700 1 --2- 172.21.15.45:0/1957791701 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80b45d0 0x7fb2f80d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.037+0000 7fb2f759e700 1 --2- 172.21.15.45:0/1957791701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 0x7fb2f80b2ac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009e80 tx=0x7fb2f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5642592c62d0eb71 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.037+0000 7fb2f659c700 1 -- 172.21.15.45:0/1957791701 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000e070 con 0x7fb2f80cfd60 2023-08-27T11:27:07.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.037+0000 7fb2f659c700 1 -- 172.21.15.45:0/1957791701 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f000b8d0 con 0x7fb2f80cfd60 2023-08-27T11:27:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.037+0000 7fb2f659c700 1 -- 172.21.15.45:0/1957791701 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000bc80 con 0x7fb2f80cfd60 2023-08-27T11:27:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.037+0000 7fb2fd174700 1 -- 172.21.15.45:0/1957791701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 msgr2=0x7fb2f80b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.037+0000 7fb2fd174700 1 --2- 172.21.15.45:0/1957791701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 0x7fb2f80b2ac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009e80 tx=0x7fb2f000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.038+0000 7fb2fd174700 1 -- 172.21.15.45:0/1957791701 shutdown_connections 2023-08-27T11:27:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.038+0000 7fb2fd174700 1 --2- 172.21.15.45:0/1957791701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 0x7fb2f80b2ac0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.038+0000 7fb2fd174700 1 --2- 172.21.15.45:0/1957791701 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80b45d0 0x7fb2f80d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.038+0000 7fb2fd174700 1 -- 172.21.15.45:0/1957791701 >> 172.21.15.45:0/1957791701 conn(0x7fb2f80d3020 msgr2=0x7fb2f80d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.038+0000 7fb2fd174700 1 -- 172.21.15.45:0/1957791701 shutdown_connections 2023-08-27T11:27:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.038+0000 7fb2fd174700 1 -- 172.21.15.45:0/1957791701 wait complete. 2023-08-27T11:27:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.039+0000 7fb2fd174700 1 Processor -- start 2023-08-27T11:27:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.039+0000 7fb2fd174700 1 -- start start 2023-08-27T11:27:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2fd174700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f80b45d0 0x7fb2f81f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2fd174700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80cf980 0x7fb2f80b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2fd174700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 0x7fb2f80b8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2fd174700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f81f7480 con 0x7fb2f80b45d0 2023-08-27T11:27:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2fd174700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80b5c40 con 0x7fb2f80cfd60 2023-08-27T11:27:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2fd174700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80b5ee0 con 0x7fb2f80cf980 2023-08-27T11:27:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f80b45d0 0x7fb2f81f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f759e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 0x7fb2f80b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2ee59c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80cf980 0x7fb2f80b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f80b45d0 0x7fb2f81f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57046/0 (socket says 172.21.15.45:57046) 2023-08-27T11:27:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f759e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 0x7fb2f80b8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:45906/0 (socket says 172.21.15.45:45906) 2023-08-27T11:27:07.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f6d9d700 1 -- 172.21.15.45:0/3250552616 learned_addr learned my addr 172.21.15.45:0/3250552616 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f759e700 1 -- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80cf980 msgr2=0x7fb2f80b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f759e700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80cf980 0x7fb2f80b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f759e700 1 -- 172.21.15.45:0/3250552616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f80b45d0 msgr2=0x7fb2f81f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f759e700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f80b45d0 0x7fb2f81f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.040+0000 7fb2f759e700 1 -- 172.21.15.45:0/3250552616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e00094a0 con 0x7fb2f80cfd60 2023-08-27T11:27:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.041+0000 7fb2f759e700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 0x7fb2f80b8350 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0015000 tx=0x7fb2f0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.041+0000 7fb2effff700 1 -- 172.21.15.45:0/3250552616 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0004880 con 0x7fb2f80cfd60 2023-08-27T11:27:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.041+0000 7fb2effff700 1 -- 172.21.15.45:0/3250552616 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f00092e0 con 0x7fb2f80cfd60 2023-08-27T11:27:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.041+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f0009b30 con 0x7fb2f80cfd60 2023-08-27T11:27:07.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.041+0000 7fb2effff700 1 -- 172.21.15.45:0/3250552616 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f001c680 con 0x7fb2f80cfd60 2023-08-27T11:27:07.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.041+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f80b64a0 con 0x7fb2f80cfd60 2023-08-27T11:27:07.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.042+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f80a1030 con 0x7fb2f80cfd60 2023-08-27T11:27:07.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.043+0000 7fb2effff700 1 -- 172.21.15.45:0/3250552616 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fb2f001c7e0 con 0x7fb2f80cfd60 2023-08-27T11:27:07.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.043+0000 7fb2effff700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb2d806a200 0x7fb2d806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.044+0000 7fb2effff700 1 -- 172.21.15.45:0/3250552616 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7fb2f008a390 con 0x7fb2f80cfd60 2023-08-27T11:27:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.044+0000 7fb2f6d9d700 1 -- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb2d806a200 msgr2=0x7fb2d806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.044+0000 7fb2f6d9d700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb2d806a200 0x7fb2d806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:07.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.052+0000 7fb2effff700 1 -- 172.21.15.45:0/3250552616 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb2f00555e0 con 0x7fb2f80cfd60 2023-08-27T11:27:07.136 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:07.135+0000 7f29b2b55200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:07.164 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:07.163+0000 7f9639e02200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:07.219 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:07.217+0000 7f29b2b55200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:27:07.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.245+0000 7fb2f6d9d700 1 -- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb2d806a200 msgr2=0x7fb2d806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:07.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.245+0000 7fb2f6d9d700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb2d806a200 0x7fb2d806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:07.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.301+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2f80734a0 con 0x7fb2f80cfd60 2023-08-27T11:27:07.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.301+0000 7fb2effff700 1 -- 172.21.15.45:0/3250552616 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4872 (secure 0 0 0) 0x7fb2f0058c00 con 0x7fb2f80cfd60 2023-08-27T11:27:07.304 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:07.304 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/72415568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":72415568},{"type":"v1","addr":"172.21.15.135:6837","nonce":72415568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2554143619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2554143619},{"type":"v1","addr":"172.21.15.135:6835","nonce":2554143619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6306,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/990560009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":990560009},{"type":"v1","addr":"172.21.15.45:6834","nonce":990560009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:10.472982+0000","modified":"2023-08-27T11:25:14.080477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6309":{"gid":6309,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6835/2659160949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2659160949},{"type":"v1","addr":"172.21.15.45:6835","nonce":2659160949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-27T11:27:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb2d806a200 msgr2=0x7fb2d806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb2d806a200 0x7fb2d806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 msgr2=0x7fb2f80b8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 0x7fb2f80b8350 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0015000 tx=0x7fb2f0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 shutdown_connections 2023-08-27T11:27:07.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fb2d806a200 0x7fb2d806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2f80b45d0 0x7fb2f81f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2f80cfd60 0x7fb2f80b8350 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 --2- 172.21.15.45:0/3250552616 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb2f80cf980 0x7fb2f80b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 >> 172.21.15.45:0/3250552616 conn(0x7fb2f80d3020 msgr2=0x7fb2f806bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 shutdown_connections 2023-08-27T11:27:07.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.304+0000 7fb2fd174700 1 -- 172.21.15.45:0/3250552616 wait complete. 2023-08-27T11:27:07.309 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-08-27T11:27:07.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:07.374 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:07.372+0000 7f29b2b55200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:07.393 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:07.392+0000 7f9639e02200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:27:07.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.443+0000 7fc439f49700 1 Processor -- start 2023-08-27T11:27:07.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.443+0000 7fc439f49700 1 -- start start 2023-08-27T11:27:07.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc439f49700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc4340cf980 0x7fc4340d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc439f49700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340b45d0 0x7fc4340d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc439f49700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4340cfd60 0x7fc4340d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc439f49700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4340cf0a0 con 0x7fc4340cfd60 2023-08-27T11:27:07.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc439f49700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4340ceae0 con 0x7fc4340cf980 2023-08-27T11:27:07.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc439f49700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4340cedc0 con 0x7fc4340b45d0 2023-08-27T11:27:07.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc433fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4340cfd60 0x7fc4340d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc433fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4340cfd60 0x7fc4340d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57104/0 (socket says 172.21.15.45:57104) 2023-08-27T11:27:07.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc433fff700 1 -- 172.21.15.45:0/1538879549 learned_addr learned my addr 172.21.15.45:0/1538879549 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:07.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc432ffd700 1 --2- 172.21.15.45:0/1538879549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340b45d0 0x7fc4340d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc4337fe700 1 --2- 172.21.15.45:0/1538879549 >> v2:172.21.15.135:3300/0 conn(0x7fc4340cf980 0x7fc4340d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc433fff700 1 -- 172.21.15.45:0/1538879549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340b45d0 msgr2=0x7fc4340d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc433fff700 1 --2- 172.21.15.45:0/1538879549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340b45d0 0x7fc4340d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc433fff700 1 -- 172.21.15.45:0/1538879549 >> v2:172.21.15.135:3300/0 conn(0x7fc4340cf980 msgr2=0x7fc4340d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc433fff700 1 --2- 172.21.15.45:0/1538879549 >> v2:172.21.15.135:3300/0 conn(0x7fc4340cf980 0x7fc4340d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc433fff700 1 -- 172.21.15.45:0/1538879549 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4340b22c0 con 0x7fc4340cfd60 2023-08-27T11:27:07.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.445+0000 7fc433fff700 1 --2- 172.21.15.45:0/1538879549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340cfd60 0x7fc4340d19a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc4340b5c10 tx=0x7fc424009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60e0e1731488f2b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc4327fc700 1 -- 172.21.15.45:0/1538879549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc424011830 con 0x7fc4340cfd60 2023-08-27T11:27:07.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc4327fc700 1 -- 172.21.15.45:0/1538879549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc424011990 con 0x7fc4340cfd60 2023-08-27T11:27:07.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc4327fc700 1 -- 172.21.15.45:0/1538879549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc424011d20 con 0x7fc4340cfd60 2023-08-27T11:27:07.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc439f49700 1 -- 172.21.15.45:0/1538879549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340cfd60 msgr2=0x7fc4340d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc439f49700 1 --2- 172.21.15.45:0/1538879549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340cfd60 0x7fc4340d19a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc4340b5c10 tx=0x7fc424009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc439f49700 1 -- 172.21.15.45:0/1538879549 shutdown_connections 2023-08-27T11:27:07.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc439f49700 1 --2- 172.21.15.45:0/1538879549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340cfd60 0x7fc4340d19a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc439f49700 1 --2- 172.21.15.45:0/1538879549 >> v2:172.21.15.135:3300/0 conn(0x7fc4340cf980 0x7fc4340d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc439f49700 1 --2- 172.21.15.45:0/1538879549 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340b45d0 0x7fc4340d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc439f49700 1 -- 172.21.15.45:0/1538879549 >> 172.21.15.45:0/1538879549 conn(0x7fc4340d3020 msgr2=0x7fc4340d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc439f49700 1 -- 172.21.15.45:0/1538879549 shutdown_connections 2023-08-27T11:27:07.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.446+0000 7fc439f49700 1 -- 172.21.15.45:0/1538879549 wait complete. 2023-08-27T11:27:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.447+0000 7fc439f49700 1 Processor -- start 2023-08-27T11:27:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.448+0000 7fc439f49700 1 -- start start 2023-08-27T11:27:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.448+0000 7fc439f49700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340b45d0 0x7fc43414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.448+0000 7fc439f49700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4340cf980 0x7fc43415d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.448+0000 7fc439f49700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340cfd60 0x7fc4340b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.448+0000 7fc439f49700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4340cae30 con 0x7fc4340b45d0 2023-08-27T11:27:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.448+0000 7fc439f49700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4340ca220 con 0x7fc4340cf980 2023-08-27T11:27:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.448+0000 7fc439f49700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4340ca500 con 0x7fc4340cfd60 2023-08-27T11:27:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc4337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340b45d0 0x7fc43414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc4337fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340b45d0 0x7fc43414e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57106/0 (socket says 172.21.15.45:57106) 2023-08-27T11:27:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc4337fe700 1 -- 172.21.15.45:0/1997764920 learned_addr learned my addr 172.21.15.45:0/1997764920 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc433fff700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340cfd60 0x7fc4340b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc432ffd700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4340cf980 0x7fc43415d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc4337fe700 1 -- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340cfd60 msgr2=0x7fc4340b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc4337fe700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340cfd60 0x7fc4340b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc4337fe700 1 -- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4340cf980 msgr2=0x7fc43415d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc4337fe700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4340cf980 0x7fc43415d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc4337fe700 1 -- 172.21.15.45:0/1997764920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc424009550 con 0x7fc4340b45d0 2023-08-27T11:27:07.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc432ffd700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4340cf980 0x7fc43415d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:07.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc4337fe700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340b45d0 0x7fc43414e600 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00efc0 tx=0x7fc42c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc42bfff700 1 -- 172.21.15.45:0/1997764920 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00cd10 con 0x7fc4340b45d0 2023-08-27T11:27:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc42bfff700 1 -- 172.21.15.45:0/1997764920 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c00f040 con 0x7fc4340b45d0 2023-08-27T11:27:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4340ca7e0 con 0x7fc4340b45d0 2023-08-27T11:27:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc42bfff700 1 -- 172.21.15.45:0/1997764920 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0146a0 con 0x7fc4340b45d0 2023-08-27T11:27:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.449+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4340cb710 con 0x7fc4340b45d0 2023-08-27T11:27:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.450+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4340a1030 con 0x7fc4340b45d0 2023-08-27T11:27:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.451+0000 7fc42bfff700 1 -- 172.21.15.45:0/1997764920 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fc42c014800 con 0x7fc4340b45d0 2023-08-27T11:27:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.452+0000 7fc42bfff700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fc41406a200 0x7fc41406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.452+0000 7fc42bfff700 1 -- 172.21.15.45:0/1997764920 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7fc42c089580 con 0x7fc4340b45d0 2023-08-27T11:27:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.452+0000 7fc432ffd700 1 -- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fc41406a200 msgr2=0x7fc41406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.452+0000 7fc432ffd700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fc41406a200 0x7fc41406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.456+0000 7fc42bfff700 1 -- 172.21.15.45:0/1997764920 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc42c0547d0 con 0x7fc4340b45d0 2023-08-27T11:27:07.466 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:07.465+0000 7f9639e02200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:27:07.619 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:07.618+0000 7f29b2b55200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:27:07.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.653+0000 7fc432ffd700 1 -- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fc41406a200 msgr2=0x7fc41406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:07.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.653+0000 7fc432ffd700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fc41406a200 0x7fc41406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:07.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.659+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4340734a0 con 0x7fc4340b45d0 2023-08-27T11:27:07.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.660+0000 7fc42bfff700 1 -- 172.21.15.45:0/1997764920 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+19274 (secure 0 0 0) 0x7fc42c057df0 con 0x7fc4340b45d0 2023-08-27T11:27:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:07.663 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:05.169863+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:08.337329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:09.446448+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":73,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/3859433979":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/485008861":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/3010439837":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/2511615876":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1537214801":"2023-08-28T11:25:29.254479+0000","172.21.15.135:6832/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:6833/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1231508777":"2023-08-28T11:25:29.254479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]},{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:07.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.662+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fc41406a200 msgr2=0x7fc41406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:07.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.662+0000 7fc439f49700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fc41406a200 0x7fc41406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340b45d0 msgr2=0x7fc43414e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340b45d0 0x7fc43414e600 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00efc0 tx=0x7fc42c00c3a0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 shutdown_connections 2023-08-27T11:27:07.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fc41406a200 0x7fc41406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4340b45d0 0x7fc43414e600 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc4340cfd60 0x7fc4340b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 --2- 172.21.15.45:0/1997764920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4340cf980 0x7fc43415d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 >> 172.21.15.45:0/1997764920 conn(0x7fc4340d3020 msgr2=0x7fc4340b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 shutdown_connections 2023-08-27T11:27:07.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.663+0000 7fc439f49700 1 -- 172.21.15.45:0/1997764920 wait complete. 2023-08-27T11:27:07.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:25:08.337329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:25:09.446448+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 73, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:07.678 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:07.700 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:07.698+0000 7f29b2b55200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:27:07.732 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:07.731+0000 7f9639e02200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:27:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.810+0000 7f5b29d01700 1 Processor -- start 2023-08-27T11:27:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.810+0000 7f5b29d01700 1 -- start start 2023-08-27T11:27:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b29d01700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b2415a0f0 0x7f5b240cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b29d01700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5b240cd300 0x7f5b2415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b29d01700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b240cfae0 con 0x7f5b2415e030 2023-08-27T11:27:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b29d01700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2415e900 con 0x7f5b240cd300 2023-08-27T11:27:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b29d01700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240cae60 con 0x7f5b2415a0f0 2023-08-27T11:27:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b22ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5b240cd300 0x7f5b2415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b22ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5b240cd300 0x7f5b2415daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57118/0 (socket says 172.21.15.45:57118) 2023-08-27T11:27:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b23fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5b2415e030 0x7f5b240cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43896/0 (socket says 172.21.15.45:43896) 2023-08-27T11:27:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b22ffd700 1 -- 172.21.15.45:0/2019421489 learned_addr learned my addr 172.21.15.45:0/2019421489 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b237fe700 1 --2- 172.21.15.45:0/2019421489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b2415a0f0 0x7f5b240cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b227fc700 1 -- 172.21.15.45:0/2019421489 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141850544 0 0) 0x7f5b240cfae0 con 0x7f5b2415e030 2023-08-27T11:27:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b227fc700 1 -- 172.21.15.45:0/2019421489 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b0c0057c0 con 0x7f5b2415e030 2023-08-27T11:27:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b22ffd700 1 -- 172.21.15.45:0/2019421489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b2415a0f0 msgr2=0x7f5b240cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b22ffd700 1 --2- 172.21.15.45:0/2019421489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b2415a0f0 0x7f5b240cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b22ffd700 1 -- 172.21.15.45:0/2019421489 >> v1:172.21.15.45:6789/0 conn(0x7f5b2415e030 legacy=0x7f5b240cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.811+0000 7f5b22ffd700 1 -- 172.21.15.45:0/2019421489 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b240cb140 con 0x7f5b240cd300 2023-08-27T11:27:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.812+0000 7f5b22ffd700 1 --2- 172.21.15.45:0/2019421489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b240cd300 0x7f5b2415daf0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5b180098f0 tx=0x7f5b18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653442d6ff510734 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.812+0000 7f5b227fc700 1 -- 172.21.15.45:0/2019421489 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18011830 con 0x7f5b240cd300 2023-08-27T11:27:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.812+0000 7f5b227fc700 1 -- 172.21.15.45:0/2019421489 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b18011990 con 0x7f5b240cd300 2023-08-27T11:27:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.812+0000 7f5b227fc700 1 -- 172.21.15.45:0/2019421489 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18011d20 con 0x7f5b240cd300 2023-08-27T11:27:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.812+0000 7f5b29d01700 1 -- 172.21.15.45:0/2019421489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b240cd300 msgr2=0x7f5b2415daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.812+0000 7f5b29d01700 1 --2- 172.21.15.45:0/2019421489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b240cd300 0x7f5b2415daf0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5b180098f0 tx=0x7f5b18009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.813+0000 7f5b29d01700 1 -- 172.21.15.45:0/2019421489 shutdown_connections 2023-08-27T11:27:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.813+0000 7f5b29d01700 1 --2- 172.21.15.45:0/2019421489 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b2415a0f0 0x7f5b240cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.813+0000 7f5b29d01700 1 --2- 172.21.15.45:0/2019421489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b240cd300 0x7f5b2415daf0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.813+0000 7f5b29d01700 1 -- 172.21.15.45:0/2019421489 >> 172.21.15.45:0/2019421489 conn(0x7f5b240cc7e0 msgr2=0x7f5b240ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.813+0000 7f5b29d01700 1 -- 172.21.15.45:0/2019421489 shutdown_connections 2023-08-27T11:27:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.813+0000 7f5b29d01700 1 -- 172.21.15.45:0/2019421489 wait complete. 2023-08-27T11:27:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.814+0000 7f5b29d01700 1 Processor -- start 2023-08-27T11:27:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.814+0000 7f5b29d01700 1 -- start start 2023-08-27T11:27:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b29d01700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b240cd300 0x7f5b241f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b29d01700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b2415a0f0 0x7f5b240aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b29d01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b2415e030 0x7f5b240acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b29d01700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241f6e50 con 0x7f5b2415e030 2023-08-27T11:27:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b29d01700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b24146a50 con 0x7f5b2415a0f0 2023-08-27T11:27:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b29d01700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241f63a0 con 0x7f5b240cd300 2023-08-27T11:27:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b23fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b2415e030 0x7f5b240acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b23fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b2415e030 0x7f5b240acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57126/0 (socket says 172.21.15.45:57126) 2023-08-27T11:27:07.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b23fff700 1 -- 172.21.15.45:0/2764596417 learned_addr learned my addr 172.21.15.45:0/2764596417 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:07.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b237fe700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b240cd300 0x7f5b241f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b22ffd700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b2415a0f0 0x7f5b240aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:07.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b23fff700 1 -- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b240cd300 msgr2=0x7f5b241f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b23fff700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b240cd300 0x7f5b241f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b23fff700 1 -- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b2415a0f0 msgr2=0x7f5b240aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b23fff700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b2415a0f0 0x7f5b240aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.815+0000 7f5b23fff700 1 -- 172.21.15.45:0/2764596417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18009550 con 0x7f5b2415e030 2023-08-27T11:27:07.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.816+0000 7f5b23fff700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b2415e030 0x7f5b240acf70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00c4d0 tx=0x7f5b1c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.816+0000 7f5b0bfff700 1 -- 172.21.15.45:0/2764596417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c005cb0 con 0x7f5b2415e030 2023-08-27T11:27:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.816+0000 7f5b0bfff700 1 -- 172.21.15.45:0/2764596417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c005e10 con 0x7f5b2415e030 2023-08-27T11:27:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.816+0000 7f5b0bfff700 1 -- 172.21.15.45:0/2764596417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c0072f0 con 0x7f5b2415e030 2023-08-27T11:27:07.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.816+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b241f65b0 con 0x7f5b2415e030 2023-08-27T11:27:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.816+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b241f6b70 con 0x7f5b2415e030 2023-08-27T11:27:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.817+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b240726d0 con 0x7f5b2415e030 2023-08-27T11:27:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.817+0000 7f5b0bfff700 1 -- 172.21.15.45:0/2764596417 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f5b1c007450 con 0x7f5b2415e030 2023-08-27T11:27:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.818+0000 7f5b0bfff700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f5b1006a200 0x7f5b1006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:07.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.818+0000 7f5b0bfff700 1 -- 172.21.15.45:0/2764596417 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7f5b1c087df0 con 0x7f5b2415e030 2023-08-27T11:27:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.818+0000 7f5b237fe700 1 -- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f5b1006a200 msgr2=0x7f5b1006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.818+0000 7f5b237fe700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f5b1006a200 0x7f5b1006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:07.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:07.822+0000 7f5b0bfff700 1 -- 172.21.15.45:0/2764596417 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5b1c0530c0 con 0x7f5b2415e030 2023-08-27T11:27:07.951 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:07.950+0000 7f9639e02200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:27:07.981 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:07.979+0000 7f29b2b55200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:27:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.019+0000 7f5b237fe700 1 -- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f5b1006a200 msgr2=0x7f5b1006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.019+0000 7f5b237fe700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f5b1006a200 0x7f5b1006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:08.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.021+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b240734a0 con 0x7f5b2415e030 2023-08-27T11:27:08.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.022+0000 7f5b0bfff700 1 -- 172.21.15.45:0/2764596417 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b1c0170a0 con 0x7f5b2415e030 2023-08-27T11:27:08.024 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:27:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f5b1006a200 msgr2=0x7f5b1006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f5b1006a200 0x7f5b1006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b2415e030 msgr2=0x7f5b240acf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b2415e030 0x7f5b240acf70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00c4d0 tx=0x7f5b1c00c7e0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 shutdown_connections 2023-08-27T11:27:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f5b1006a200 0x7f5b1006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b2415e030 0x7f5b240acf70 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5b240cd300 0x7f5b241f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 --2- 172.21.15.45:0/2764596417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b2415a0f0 0x7f5b240aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 >> 172.21.15.45:0/2764596417 conn(0x7f5b240cc7e0 msgr2=0x7f5b2406e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 shutdown_connections 2023-08-27T11:27:08.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.024+0000 7f5b29d01700 1 -- 172.21.15.45:0/2764596417 wait complete. 2023-08-27T11:27:08.039 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:08.101 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:08.100+0000 7f9639e02200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.208+0000 7ff7484d3700 1 Processor -- start 2023-08-27T11:27:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.208+0000 7ff7484d3700 1 -- start start 2023-08-27T11:27:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff7484d3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff74015d480 0x7ff7400d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff7484d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff740073a10 0x7ff740160f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff7484d3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7401614b0 0x7ff74006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff7484d3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff740161e60 con 0x7ff74015d480 2023-08-27T11:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff7484d3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7401618e0 con 0x7ff7401614b0 2023-08-27T11:27:08.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff7484d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74015d860 con 0x7ff740073a10 2023-08-27T11:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff74626f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff74015d480 0x7ff7400d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff74626f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff74015d480 0x7ff7400d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57132/0 (socket says 172.21.15.45:57132) 2023-08-27T11:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff74626f700 1 -- 172.21.15.45:0/242956350 learned_addr learned my addr 172.21.15.45:0/242956350 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff746a70700 1 --2- 172.21.15.45:0/242956350 >> v2:172.21.15.135:3300/0 conn(0x7ff7401614b0 0x7ff74006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff745a6e700 1 --2- 172.21.15.45:0/242956350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff740073a10 0x7ff740160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff74626f700 1 -- 172.21.15.45:0/242956350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff740073a10 msgr2=0x7ff740160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff74626f700 1 --2- 172.21.15.45:0/242956350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff740073a10 0x7ff740160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff74626f700 1 -- 172.21.15.45:0/242956350 >> v2:172.21.15.135:3300/0 conn(0x7ff7401614b0 msgr2=0x7ff74006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff74626f700 1 --2- 172.21.15.45:0/242956350 >> v2:172.21.15.135:3300/0 conn(0x7ff7401614b0 0x7ff74006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.209+0000 7ff74626f700 1 -- 172.21.15.45:0/242956350 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff740073e70 con 0x7ff74015d480 2023-08-27T11:27:08.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff74626f700 1 --2- 172.21.15.45:0/242956350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74015d480 0x7ff7400d0040 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff7400b5450 tx=0x7ff73c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27d0605923b525d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff74526d700 1 -- 172.21.15.45:0/242956350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c011830 con 0x7ff74015d480 2023-08-27T11:27:08.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff74526d700 1 -- 172.21.15.45:0/242956350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c011990 con 0x7ff74015d480 2023-08-27T11:27:08.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff74526d700 1 -- 172.21.15.45:0/242956350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c011d20 con 0x7ff74015d480 2023-08-27T11:27:08.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff7484d3700 1 -- 172.21.15.45:0/242956350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74015d480 msgr2=0x7ff7400d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff7484d3700 1 --2- 172.21.15.45:0/242956350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74015d480 0x7ff7400d0040 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff7400b5450 tx=0x7ff73c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff7484d3700 1 -- 172.21.15.45:0/242956350 shutdown_connections 2023-08-27T11:27:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff7484d3700 1 --2- 172.21.15.45:0/242956350 >> v2:172.21.15.135:3300/0 conn(0x7ff7401614b0 0x7ff74006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff7484d3700 1 --2- 172.21.15.45:0/242956350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff74015d480 0x7ff7400d0040 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff7484d3700 1 --2- 172.21.15.45:0/242956350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff740073a10 0x7ff740160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff7484d3700 1 -- 172.21.15.45:0/242956350 >> 172.21.15.45:0/242956350 conn(0x7ff7400cf520 msgr2=0x7ff7400cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff7484d3700 1 -- 172.21.15.45:0/242956350 shutdown_connections 2023-08-27T11:27:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.210+0000 7ff7484d3700 1 -- 172.21.15.45:0/242956350 wait complete. 2023-08-27T11:27:08.219 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:08.214+0000 7f9639e02200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:27:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.212+0000 7ff7484d3700 1 Processor -- start 2023-08-27T11:27:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.212+0000 7ff7484d3700 1 -- start start 2023-08-27T11:27:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.212+0000 7ff7484d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff740073a10 0x7ff7400b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.212+0000 7ff7484d3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff74015d480 0x7ff7401f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.212+0000 7ff7484d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7401614b0 0x7ff7401f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.212+0000 7ff7484d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7400b1e70 con 0x7ff740073a10 2023-08-27T11:27:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.212+0000 7ff7484d3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74014e210 con 0x7ff74015d480 2023-08-27T11:27:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.212+0000 7ff7484d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7401f6f50 con 0x7ff7401614b0 2023-08-27T11:27:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff74626f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff740073a10 0x7ff7400b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff74626f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff740073a10 0x7ff7400b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57138/0 (socket says 172.21.15.45:57138) 2023-08-27T11:27:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff74626f700 1 -- 172.21.15.45:0/1834516465 learned_addr learned my addr 172.21.15.45:0/1834516465 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff745a6e700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff74015d480 0x7ff7401f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff746a70700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7401614b0 0x7ff7401f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff74626f700 1 -- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7401614b0 msgr2=0x7ff7401f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff74626f700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7401614b0 0x7ff7401f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff74626f700 1 -- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff74015d480 msgr2=0x7ff7401f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff74626f700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff74015d480 0x7ff7401f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff74626f700 1 -- 172.21.15.45:0/1834516465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c009550 con 0x7ff740073a10 2023-08-27T11:27:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff74626f700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff740073a10 0x7ff7400b1930 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009c10 tx=0x7ff73c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff736ffd700 1 -- 172.21.15.45:0/1834516465 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c004450 con 0x7ff740073a10 2023-08-27T11:27:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff736ffd700 1 -- 172.21.15.45:0/1834516465 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c01e070 con 0x7ff740073a10 2023-08-27T11:27:08.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7401f28a0 con 0x7ff740073a10 2023-08-27T11:27:08.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff736ffd700 1 -- 172.21.15.45:0/1834516465 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0038e0 con 0x7ff740073a10 2023-08-27T11:27:08.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.213+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7400b0e50 con 0x7ff740073a10 2023-08-27T11:27:08.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.214+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7400b54a0 con 0x7ff740073a10 2023-08-27T11:27:08.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.215+0000 7ff736ffd700 1 -- 172.21.15.45:0/1834516465 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7ff73c003a40 con 0x7ff740073a10 2023-08-27T11:27:08.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.215+0000 7ff736ffd700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7ff72406a200 0x7ff72406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.215+0000 7ff745a6e700 1 -- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7ff72406a200 msgr2=0x7ff72406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:08.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.215+0000 7ff736ffd700 1 -- 172.21.15.45:0/1834516465 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7ff73c0895f0 con 0x7ff740073a10 2023-08-27T11:27:08.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.215+0000 7ff745a6e700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7ff72406a200 0x7ff72406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:08.228 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:08.223+0000 7f29b2b55200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:27:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.224+0000 7ff736ffd700 1 -- 172.21.15.45:0/1834516465 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff73c054950 con 0x7ff740073a10 2023-08-27T11:27:08.370 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:08.368+0000 7f9639e02200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:27:08.383 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:08.381+0000 7f29b2b55200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:27:08.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.416+0000 7ff745a6e700 1 -- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7ff72406a200 msgr2=0x7ff72406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:08.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.416+0000 7ff745a6e700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7ff72406a200 0x7ff72406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.426+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff7400cbb30 con 0x7ff740073a10 2023-08-27T11:27:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.427+0000 7ff736ffd700 1 -- 172.21.15.45:0/1834516465 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v73) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff73c057f70 con 0x7ff740073a10 2023-08-27T11:27:08.429 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:27:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7ff72406a200 msgr2=0x7ff72406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7ff72406a200 0x7ff72406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff740073a10 msgr2=0x7ff7400b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff740073a10 0x7ff7400b1930 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009c10 tx=0x7ff73c004750 comp rx=0 tx=0).stop 2023-08-27T11:27:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 shutdown_connections 2023-08-27T11:27:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7ff72406a200 0x7ff72406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff740073a10 0x7ff7400b1930 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7401614b0 0x7ff7401f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 --2- 172.21.15.45:0/1834516465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff74015d480 0x7ff7401f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 >> 172.21.15.45:0/1834516465 conn(0x7ff7400cf520 msgr2=0x7ff7400c5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 shutdown_connections 2023-08-27T11:27:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.429+0000 7ff7484d3700 1 -- 172.21.15.45:0/1834516465 wait complete. 2023-08-27T11:27:08.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:08.508 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:08.506+0000 7f29b2b55200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:27:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.573+0000 7fa703292700 1 Processor -- start 2023-08-27T11:27:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.573+0000 7fa703292700 1 -- start start 2023-08-27T11:27:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.574+0000 7fa703292700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa6fc071e80 0x7fa6fc06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.574+0000 7fa703292700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc06f8b0 con 0x7fa6fc15b270 2023-08-27T11:27:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.574+0000 7fa703292700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc06f010 con 0x7fa6fc06cf40 2023-08-27T11:27:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.574+0000 7fa703292700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc0ca650 con 0x7fa6fc071e80 2023-08-27T11:27:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.574+0000 7fa70082d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa6fc15b270 0x7fa6fc0caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43924/0 (socket says 172.21.15.45:43924) 2023-08-27T11:27:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.574+0000 7fa70082d700 1 -- 172.21.15.45:0/2562282683 learned_addr learned my addr 172.21.15.45:0/2562282683 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.574+0000 7fa70182f700 1 --2- 172.21.15.45:0/2562282683 >> v2:172.21.15.135:3300/0 conn(0x7fa6fc071e80 0x7fa6fc06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.575+0000 7fa6f3fff700 1 -- 172.21.15.45:0/2562282683 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311779787 0 0) 0x7fa6ec001110 con 0x7fa6fc15b270 2023-08-27T11:27:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.575+0000 7fa6f3fff700 1 -- 172.21.15.45:0/2562282683 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e80035b0 con 0x7fa6fc15b270 2023-08-27T11:27:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.575+0000 7fa70182f700 1 -- 172.21.15.45:0/2562282683 >> v1:172.21.15.135:6789/0 conn(0x7fa6fc06cf40 legacy=0x7fa6fc06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.575+0000 7fa70182f700 1 -- 172.21.15.45:0/2562282683 >> v1:172.21.15.45:6789/0 conn(0x7fa6fc15b270 legacy=0x7fa6fc0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.575+0000 7fa70182f700 1 -- 172.21.15.45:0/2562282683 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc06f2f0 con 0x7fa6fc071e80 2023-08-27T11:27:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.575+0000 7fa70182f700 1 --2- 172.21.15.45:0/2562282683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc071e80 0x7fa6fc06ac90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ced0115b4109f9ee server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.575+0000 7fa6f3fff700 1 -- 172.21.15.45:0/2562282683 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400e070 con 0x7fa6fc071e80 2023-08-27T11:27:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.575+0000 7fa6f3fff700 1 -- 172.21.15.45:0/2562282683 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f400bb20 con 0x7fa6fc071e80 2023-08-27T11:27:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.575+0000 7fa6f3fff700 1 -- 172.21.15.45:0/2562282683 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4012430 con 0x7fa6fc071e80 2023-08-27T11:27:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.576+0000 7fa703292700 1 -- 172.21.15.45:0/2562282683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc071e80 msgr2=0x7fa6fc06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.576+0000 7fa703292700 1 --2- 172.21.15.45:0/2562282683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc071e80 0x7fa6fc06ac90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009d90 tx=0x7fa6f400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.576+0000 7fa703292700 1 -- 172.21.15.45:0/2562282683 shutdown_connections 2023-08-27T11:27:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.576+0000 7fa703292700 1 --2- 172.21.15.45:0/2562282683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc071e80 0x7fa6fc06ac90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.576+0000 7fa703292700 1 -- 172.21.15.45:0/2562282683 >> 172.21.15.45:0/2562282683 conn(0x7fa6fc0cc7e0 msgr2=0x7fa6fc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.576+0000 7fa703292700 1 -- 172.21.15.45:0/2562282683 shutdown_connections 2023-08-27T11:27:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.576+0000 7fa703292700 1 -- 172.21.15.45:0/2562282683 wait complete. 2023-08-27T11:27:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.577+0000 7fa703292700 1 Processor -- start 2023-08-27T11:27:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.577+0000 7fa703292700 1 -- start start 2023-08-27T11:27:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa703292700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6fc06cf40 0x7fa6fc0b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa703292700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6fc071e80 0x7fa6fc155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa703292700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc15b270 0x7fa6fc0ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa703292700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0ce7f0 con 0x7fa6fc06cf40 2023-08-27T11:27:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa703292700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0b2fb0 con 0x7fa6fc15b270 2023-08-27T11:27:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa703292700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc1e72e0 con 0x7fa6fc071e80 2023-08-27T11:27:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70102e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6fc06cf40 0x7fa6fc0b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70102e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6fc06cf40 0x7fa6fc0b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57154/0 (socket says 172.21.15.45:57154) 2023-08-27T11:27:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70102e700 1 -- 172.21.15.45:0/329174965 learned_addr learned my addr 172.21.15.45:0/329174965 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70182f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc15b270 0x7fa6fc0ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70082d700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6fc071e80 0x7fa6fc155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70102e700 1 -- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6fc071e80 msgr2=0x7fa6fc155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70102e700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6fc071e80 0x7fa6fc155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70102e700 1 -- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc15b270 msgr2=0x7fa6fc0ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70102e700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc15b270 0x7fa6fc0ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.578+0000 7fa70102e700 1 -- 172.21.15.45:0/329174965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f4009a70 con 0x7fa6fc06cf40 2023-08-27T11:27:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.579+0000 7fa70102e700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6fc06cf40 0x7fa6fc0b2a70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800c4e0 tx=0x7fa6f800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.579+0000 7fa6f1ffb700 1 -- 172.21.15.45:0/329174965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f80052d0 con 0x7fa6fc06cf40 2023-08-27T11:27:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.579+0000 7fa6f1ffb700 1 -- 172.21.15.45:0/329174965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f8005430 con 0x7fa6fc06cf40 2023-08-27T11:27:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.579+0000 7fa6f1ffb700 1 -- 172.21.15.45:0/329174965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f80038c0 con 0x7fa6fc06cf40 2023-08-27T11:27:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.579+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc0cde50 con 0x7fa6fc06cf40 2023-08-27T11:27:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.579+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc0ce410 con 0x7fa6fc06cf40 2023-08-27T11:27:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.580+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc0726d0 con 0x7fa6fc06cf40 2023-08-27T11:27:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.580+0000 7fa6f1ffb700 1 -- 172.21.15.45:0/329174965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fa6f8003a20 con 0x7fa6fc06cf40 2023-08-27T11:27:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.581+0000 7fa6f1ffb700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fa6e006a200 0x7fa6e006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.581+0000 7fa70082d700 1 -- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fa6e006a200 msgr2=0x7fa6e006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.581+0000 7fa6f1ffb700 1 -- 172.21.15.45:0/329174965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7fa6f8087e40 con 0x7fa6fc06cf40 2023-08-27T11:27:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.581+0000 7fa70082d700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fa6e006a200 0x7fa6e006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.585+0000 7fa6f1ffb700 1 -- 172.21.15.45:0/329174965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa6f8053070 con 0x7fa6fc06cf40 2023-08-27T11:27:08.632 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:08.631+0000 7f9639e02200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:08.674 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:08.672+0000 7f29b2b55200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:27:08.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.782+0000 7fa70082d700 1 -- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fa6e006a200 msgr2=0x7fa6e006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:08.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.782+0000 7fa70082d700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fa6e006a200 0x7fa6e006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.787+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6fc072440 con 0x7fa6fc06cf40 2023-08-27T11:27:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.787+0000 7fa6f1ffb700 1 -- 172.21.15.45:0/329174965 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6f8056690 con 0x7fa6fc06cf40 2023-08-27T11:27:08.790 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:27:08.791 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:08.790+0000 7f9639e02200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:27:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fa6e006a200 msgr2=0x7fa6e006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fa6e006a200 0x7fa6e006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6fc06cf40 msgr2=0x7fa6fc0b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6fc06cf40 0x7fa6fc0b2a70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800c4e0 tx=0x7fa6f800c7f0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 shutdown_connections 2023-08-27T11:27:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fa6e006a200 0x7fa6e006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6fc06cf40 0x7fa6fc0b2a70 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6fc15b270 0x7fa6fc0ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 --2- 172.21.15.45:0/329174965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6fc071e80 0x7fa6fc155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 >> 172.21.15.45:0/329174965 conn(0x7fa6fc0cc7e0 msgr2=0x7fa6fc0b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 shutdown_connections 2023-08-27T11:27:08.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.790+0000 7fa703292700 1 -- 172.21.15.45:0/329174965 wait complete. 2023-08-27T11:27:08.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.936+0000 7fbe1281a700 1 Processor -- start 2023-08-27T11:27:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.936+0000 7fbe1281a700 1 -- start start 2023-08-27T11:27:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe1281a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c160890 0x7fbe0c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe1281a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe0c0b62c0 0x7fbe0c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe1281a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbe0c160c70 0x7fbe0c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe1281a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0c0cdf40 con 0x7fbe0c0b62c0 2023-08-27T11:27:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe1281a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0c0d0fd0 con 0x7fbe0c160c70 2023-08-27T11:27:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe1281a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c0ce190 con 0x7fbe0c160890 2023-08-27T11:27:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe0b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe0c0b62c0 0x7fbe0c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe0b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe0c0b62c0 0x7fbe0c0b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57166/0 (socket says 172.21.15.45:57166) 2023-08-27T11:27:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe0b7fe700 1 -- 172.21.15.45:0/332327558 learned_addr learned my addr 172.21.15.45:0/332327558 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe10db7700 1 --2- 172.21.15.45:0/332327558 >> v2:172.21.15.135:3300/0 conn(0x7fbe0c160c70 0x7fbe0c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe0bfff700 1 --2- 172.21.15.45:0/332327558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c160890 0x7fbe0c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe0b7fe700 1 -- 172.21.15.45:0/332327558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c160890 msgr2=0x7fbe0c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe0b7fe700 1 --2- 172.21.15.45:0/332327558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c160890 0x7fbe0c0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe0b7fe700 1 -- 172.21.15.45:0/332327558 >> v2:172.21.15.135:3300/0 conn(0x7fbe0c160c70 msgr2=0x7fbe0c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe0b7fe700 1 --2- 172.21.15.45:0/332327558 >> v2:172.21.15.135:3300/0 conn(0x7fbe0c160c70 0x7fbe0c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.937+0000 7fbe0b7fe700 1 -- 172.21.15.45:0/332327558 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c0ce470 con 0x7fbe0c0b62c0 2023-08-27T11:27:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe0b7fe700 1 --2- 172.21.15.45:0/332327558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c0b62c0 0x7fbe0c0b4b10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c162220 tx=0x7fbe00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe0332fb1032de2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe0affd700 1 -- 172.21.15.45:0/332327558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00011830 con 0x7fbe0c0b62c0 2023-08-27T11:27:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe0affd700 1 -- 172.21.15.45:0/332327558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe00011990 con 0x7fbe0c0b62c0 2023-08-27T11:27:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe0affd700 1 -- 172.21.15.45:0/332327558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00011d20 con 0x7fbe0c0b62c0 2023-08-27T11:27:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe1281a700 1 -- 172.21.15.45:0/332327558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c0b62c0 msgr2=0x7fbe0c0b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe1281a700 1 --2- 172.21.15.45:0/332327558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c0b62c0 0x7fbe0c0b4b10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c162220 tx=0x7fbe00009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe1281a700 1 -- 172.21.15.45:0/332327558 shutdown_connections 2023-08-27T11:27:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe1281a700 1 --2- 172.21.15.45:0/332327558 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c160890 0x7fbe0c0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe1281a700 1 --2- 172.21.15.45:0/332327558 >> v2:172.21.15.135:3300/0 conn(0x7fbe0c160c70 0x7fbe0c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe1281a700 1 --2- 172.21.15.45:0/332327558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c0b62c0 0x7fbe0c0b4b10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.938+0000 7fbe1281a700 1 -- 172.21.15.45:0/332327558 >> 172.21.15.45:0/332327558 conn(0x7fbe0c0cfb10 msgr2=0x7fbe0c0cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.939+0000 7fbe1281a700 1 -- 172.21.15.45:0/332327558 shutdown_connections 2023-08-27T11:27:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.939+0000 7fbe1281a700 1 -- 172.21.15.45:0/332327558 wait complete. 2023-08-27T11:27:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.940+0000 7fbe1281a700 1 Processor -- start 2023-08-27T11:27:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.940+0000 7fbe1281a700 1 -- start start 2023-08-27T11:27:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe1281a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c0b62c0 0x7fbe0c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe1281a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c160890 0x7fbe0c0abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe1281a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe0c160c70 0x7fbe0c0ac120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe1281a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c1f3150 con 0x7fbe0c160890 2023-08-27T11:27:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe1281a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c0ade10 con 0x7fbe0c160c70 2023-08-27T11:27:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe1281a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c0ac750 con 0x7fbe0c0b62c0 2023-08-27T11:27:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c160890 0x7fbe0c0abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0bfff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c0b62c0 0x7fbe0c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c160890 0x7fbe0c0abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57172/0 (socket says 172.21.15.45:57172) 2023-08-27T11:27:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0b7fe700 1 -- 172.21.15.45:0/3825255225 learned_addr learned my addr 172.21.15.45:0/3825255225 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe10db7700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe0c160c70 0x7fbe0c0ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0b7fe700 1 -- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c0b62c0 msgr2=0x7fbe0c0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0b7fe700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c0b62c0 0x7fbe0c0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0b7fe700 1 -- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe0c160c70 msgr2=0x7fbe0c0ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0b7fe700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe0c160c70 0x7fbe0c0ac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0b7fe700 1 -- 172.21.15.45:0/3825255225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00009550 con 0x7fbe0c160890 2023-08-27T11:27:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe0b7fe700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c160890 0x7fbe0c0abbe0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbe00000c00 tx=0x7fbe00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.941+0000 7fbe08ff9700 1 -- 172.21.15.45:0/3825255225 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00004240 con 0x7fbe0c160890 2023-08-27T11:27:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.942+0000 7fbe08ff9700 1 -- 172.21.15.45:0/3825255225 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0001f070 con 0x7fbe0c160890 2023-08-27T11:27:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.942+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0c0aca30 con 0x7fbe0c160890 2023-08-27T11:27:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.942+0000 7fbe08ff9700 1 -- 172.21.15.45:0/3825255225 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000045e0 con 0x7fbe0c160890 2023-08-27T11:27:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.942+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0c0c8c20 con 0x7fbe0c160890 2023-08-27T11:27:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.943+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0c0acf50 con 0x7fbe0c160890 2023-08-27T11:27:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.943+0000 7fbe08ff9700 1 -- 172.21.15.45:0/3825255225 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fbe00021030 con 0x7fbe0c160890 2023-08-27T11:27:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.944+0000 7fbe08ff9700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fbdf806a200 0x7fbdf806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.944+0000 7fbe0bfff700 1 -- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fbdf806a200 msgr2=0x7fbdf806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.944+0000 7fbe08ff9700 1 -- 172.21.15.45:0/3825255225 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7fbe00089ef0 con 0x7fbe0c160890 2023-08-27T11:27:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.944+0000 7fbe0bfff700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fbdf806a200 0x7fbdf806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:08.948+0000 7fbe08ff9700 1 -- 172.21.15.45:0/3825255225 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbe000552b0 con 0x7fbe0c160890 2023-08-27T11:27:08.956 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:08.951+0000 7f29b2b55200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:09.122 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:09.120+0000 7f29b2b55200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:27:09.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.145+0000 7fbe0bfff700 1 -- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fbdf806a200 msgr2=0x7fbdf806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:09.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.145+0000 7fbe0bfff700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fbdf806a200 0x7fbdf806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:09.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.154+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe0c0c8db0 con 0x7fbe0c160890 2023-08-27T11:27:09.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.155+0000 7fbe08ff9700 1 -- 172.21.15.45:0/3825255225 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe000588d0 con 0x7fbe0c160890 2023-08-27T11:27:09.157 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:27:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fbdf806a200 msgr2=0x7fbdf806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:09.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fbdf806a200 0x7fbdf806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c160890 msgr2=0x7fbe0c0abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c160890 0x7fbe0c0abbe0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbe00000c00 tx=0x7fbe00004750 comp rx=0 tx=0).stop 2023-08-27T11:27:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 shutdown_connections 2023-08-27T11:27:09.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe0c160c70 0x7fbe0c0ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe0c0b62c0 0x7fbe0c0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7fbdf806a200 0x7fbdf806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 --2- 172.21.15.45:0/3825255225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe0c160890 0x7fbe0c0abbe0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 >> 172.21.15.45:0/3825255225 conn(0x7fbe0c0cfb10 msgr2=0x7fbe0c0b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 shutdown_connections 2023-08-27T11:27:09.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.157+0000 7fbe1281a700 1 -- 172.21.15.45:0/3825255225 wait complete. 2023-08-27T11:27:09.172 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:09.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.297+0000 7f1f64e75700 1 Processor -- start 2023-08-27T11:27:09.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.297+0000 7f1f64e75700 1 -- start start 2023-08-27T11:27:09.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.298+0000 7f1f64e75700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1f60161970 0x7f1f600cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.298+0000 7f1f64e75700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f600b2780 con 0x7f1f600cba10 2023-08-27T11:27:09.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.298+0000 7f1f64e75700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f600ce2b0 con 0x7f1f600b02e0 2023-08-27T11:27:09.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.298+0000 7f1f64e75700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600d1950 con 0x7f1f60161970 2023-08-27T11:27:09.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5dd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1f600cba10 0x7f1f6006b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43958/0 (socket says 172.21.15.45:43958) 2023-08-27T11:27:09.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5dd9b700 1 -- 172.21.15.45:0/1182518866 learned_addr learned my addr 172.21.15.45:0/1182518866 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:09.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5e59c700 1 --2- 172.21.15.45:0/1182518866 >> v2:172.21.15.135:3300/0 conn(0x7f1f60161970 0x7f1f600cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5d59a700 1 -- 172.21.15.45:0/1182518866 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698875886 0 0) 0x7f1f600b2780 con 0x7f1f600cba10 2023-08-27T11:27:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5d59a700 1 -- 172.21.15.45:0/1182518866 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f480035b0 con 0x7f1f600cba10 2023-08-27T11:27:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5e59c700 1 -- 172.21.15.45:0/1182518866 >> v1:172.21.15.135:6789/0 conn(0x7f1f600b02e0 legacy=0x7f1f600c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5e59c700 1 -- 172.21.15.45:0/1182518866 >> v1:172.21.15.45:6789/0 conn(0x7f1f600cba10 legacy=0x7f1f6006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5e59c700 1 -- 172.21.15.45:0/1182518866 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600ce550 con 0x7f1f60161970 2023-08-27T11:27:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5e59c700 1 --2- 172.21.15.45:0/1182518866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f60161970 0x7f1f600cb4d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009840 tx=0x7f1f50009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c2470b0e834dd56 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.299+0000 7f1f5d59a700 1 -- 172.21.15.45:0/1182518866 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50011830 con 0x7f1f60161970 2023-08-27T11:27:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.300+0000 7f1f5d59a700 1 -- 172.21.15.45:0/1182518866 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f50011990 con 0x7f1f60161970 2023-08-27T11:27:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.300+0000 7f1f5d59a700 1 -- 172.21.15.45:0/1182518866 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50011d20 con 0x7f1f60161970 2023-08-27T11:27:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.300+0000 7f1f64e75700 1 -- 172.21.15.45:0/1182518866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f60161970 msgr2=0x7f1f600cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.300+0000 7f1f64e75700 1 --2- 172.21.15.45:0/1182518866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f60161970 0x7f1f600cb4d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009840 tx=0x7f1f50009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.300+0000 7f1f64e75700 1 -- 172.21.15.45:0/1182518866 shutdown_connections 2023-08-27T11:27:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.300+0000 7f1f64e75700 1 --2- 172.21.15.45:0/1182518866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f60161970 0x7f1f600cb4d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.300+0000 7f1f64e75700 1 -- 172.21.15.45:0/1182518866 >> 172.21.15.45:0/1182518866 conn(0x7f1f6009a160 msgr2=0x7f1f600d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.300+0000 7f1f64e75700 1 -- 172.21.15.45:0/1182518866 shutdown_connections 2023-08-27T11:27:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.300+0000 7f1f64e75700 1 -- 172.21.15.45:0/1182518866 wait complete. 2023-08-27T11:27:09.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.302+0000 7f1f64e75700 1 Processor -- start 2023-08-27T11:27:09.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.302+0000 7f1f64e75700 1 -- start start 2023-08-27T11:27:09.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f64e75700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f600b02e0 0x7f1f60160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.313 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f64e75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cba10 0x7f1f6015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f64e75700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f60161970 0x7f1f6015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f64e75700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6015eb70 con 0x7f1f600cba10 2023-08-27T11:27:09.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f64e75700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6015f620 con 0x7f1f600b02e0 2023-08-27T11:27:09.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f64e75700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6015ed80 con 0x7f1f60161970 2023-08-27T11:27:09.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cba10 0x7f1f6015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cba10 0x7f1f6015e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57188/0 (socket says 172.21.15.45:57188) 2023-08-27T11:27:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5dd9b700 1 -- 172.21.15.45:0/980107388 learned_addr learned my addr 172.21.15.45:0/980107388 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5e59c700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f600b02e0 0x7f1f60160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5ed9d700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f60161970 0x7f1f6015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5dd9b700 1 -- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f60161970 msgr2=0x7f1f6015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5dd9b700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f60161970 0x7f1f6015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5dd9b700 1 -- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f600b02e0 msgr2=0x7f1f60160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5dd9b700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f600b02e0 0x7f1f60160450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.303+0000 7f1f5dd9b700 1 -- 172.21.15.45:0/980107388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500094a0 con 0x7f1f600cba10 2023-08-27T11:27:09.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.304+0000 7f1f5dd9b700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cba10 0x7f1f6015e630 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005910 tx=0x7f1f5400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.304+0000 7f1f4effd700 1 -- 172.21.15.45:0/980107388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54005d70 con 0x7f1f600cba10 2023-08-27T11:27:09.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.304+0000 7f1f4effd700 1 -- 172.21.15.45:0/980107388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f540070d0 con 0x7f1f600cba10 2023-08-27T11:27:09.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.304+0000 7f1f64e75700 1 -- 172.21.15.45:0/980107388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6015f060 con 0x7f1f600cba10 2023-08-27T11:27:09.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.304+0000 7f1f4effd700 1 -- 172.21.15.45:0/980107388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54007460 con 0x7f1f600cba10 2023-08-27T11:27:09.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.304+0000 7f1f64e75700 1 -- 172.21.15.45:0/980107388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f600c7ff0 con 0x7f1f600cba10 2023-08-27T11:27:09.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.305+0000 7f1f64e75700 1 -- 172.21.15.45:0/980107388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6009edd0 con 0x7f1f600cba10 2023-08-27T11:27:09.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.305+0000 7f1f4effd700 1 -- 172.21.15.45:0/980107388 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f1f540075c0 con 0x7f1f600cba10 2023-08-27T11:27:09.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.306+0000 7f1f4effd700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1f4006a200 0x7f1f4006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.306+0000 7f1f4effd700 1 -- 172.21.15.45:0/980107388 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6842+0+0 (secure 0 0 0) 0x7f1f54087e70 con 0x7f1f600cba10 2023-08-27T11:27:09.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.306+0000 7f1f5e59c700 1 -- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1f4006a200 msgr2=0x7f1f4006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1664443653 2023-08-27T11:27:09.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.306+0000 7f1f5e59c700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1f4006a200 0x7f1f4006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:09.321 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.315+0000 7f9639e02200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:27:09.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.310+0000 7f1f4effd700 1 -- 172.21.15.45:0/980107388 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1f54053140 con 0x7f1f600cba10 2023-08-27T11:27:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.338+0000 7f1f4effd700 1 -- 172.21.15.45:0/980107388 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 27) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f1f54085f20 con 0x7f1f600cba10 2023-08-27T11:27:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.339+0000 7f1f4effd700 1 -- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1f4006a200 msgr2=0x7f1f4006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.339+0000 7f1f4effd700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1f4006a200 0x7f1f4006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.398 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b366d700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.398 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b366d700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.398 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b366d700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.399 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b366d700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.399 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b366d700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.400 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b5e72700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.400 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b5e72700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.400 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b5e72700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.400 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b5e72700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.400 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:09.396+0000 7f95b5e72700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.581+0000 7f1f64e75700 1 -- 172.21.15.45:0/980107388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f600cd110 con 0x7f1f600cba10 2023-08-27T11:27:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.582+0000 7f1f4effd700 1 -- 172.21.15.45:0/980107388 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4872 (secure 0 0 0) 0x7f1f54071360 con 0x7f1f600cba10 2023-08-27T11:27:09.584 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:09.585 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/72415568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":72415568},{"type":"v1","addr":"172.21.15.135:6837","nonce":72415568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2554143619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2554143619},{"type":"v1","addr":"172.21.15.135:6835","nonce":2554143619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6306,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/990560009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":990560009},{"type":"v1","addr":"172.21.15.45:6834","nonce":990560009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:10.472982+0000","modified":"2023-08-27T11:25:14.080477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6309":{"gid":6309,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6835/2659160949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2659160949},{"type":"v1","addr":"172.21.15.45:6835","nonce":2659160949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-27T11:27:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 -- 172.21.15.45:0/980107388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cba10 msgr2=0x7f1f6015e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cba10 0x7f1f6015e630 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005910 tx=0x7f1f5400c9e0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 -- 172.21.15.45:0/980107388 shutdown_connections 2023-08-27T11:27:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f60161970 0x7f1f6015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f600b02e0 0x7f1f60160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.135:6832/1664443653,v1:172.21.15.135:6833/1664443653] conn(0x7f1f4006a200 0x7f1f4006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 --2- 172.21.15.45:0/980107388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cba10 0x7f1f6015e630 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 -- 172.21.15.45:0/980107388 >> 172.21.15.45:0/980107388 conn(0x7f1f6009a160 msgr2=0x7f1f6006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 -- 172.21.15.45:0/980107388 shutdown_connections 2023-08-27T11:27:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.584+0000 7f1f64e75700 1 -- 172.21.15.45:0/980107388 wait complete. 2023-08-27T11:27:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-08-27T11:27:09.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:09.708 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:09.706+0000 7f29b2b55200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:27:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.725+0000 7f0b5e115700 1 Processor -- start 2023-08-27T11:27:09.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.725+0000 7f0b5e115700 1 -- start start 2023-08-27T11:27:09.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b5e115700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0b580ce4b0 0x7f0b58167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b5e115700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5815d6a0 con 0x7f0b580b8c20 2023-08-27T11:27:09.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b5e115700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b580d2c90 con 0x7f0b5815d2c0 2023-08-27T11:27:09.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b5e115700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b580d1bf0 con 0x7f0b580ce4b0 2023-08-27T11:27:09.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b56ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0b580b8c20 0x7f0b580cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43974/0 (socket says 172.21.15.45:43974) 2023-08-27T11:27:09.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b56ffd700 1 -- 172.21.15.45:0/1279074463 learned_addr learned my addr 172.21.15.45:0/1279074463 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:09.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b57fff700 1 --2- 172.21.15.45:0/1279074463 >> v2:172.21.15.135:3300/0 conn(0x7f0b580ce4b0 0x7f0b58167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846903780 0 0) 0x7f0b48001110 con 0x7f0b580b8c20 2023-08-27T11:27:09.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b400035b0 con 0x7f0b580b8c20 2023-08-27T11:27:09.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2124759904 0 0) 0x7f0b400035b0 con 0x7f0b580b8c20 2023-08-27T11:27:09.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 >> v2:172.21.15.135:3300/0 conn(0x7f0b580ce4b0 msgr2=0x7f0b58167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:09.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 --2- 172.21.15.45:0/1279074463 >> v2:172.21.15.135:3300/0 conn(0x7f0b580ce4b0 0x7f0b58167d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 >> v1:172.21.15.135:6789/0 conn(0x7f0b5815d2c0 legacy=0x7f0b580d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b580d1ed0 con 0x7f0b580b8c20 2023-08-27T11:27:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f0b48001800 con 0x7f0b580b8c20 2023-08-27T11:27:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b57fff700 1 --2- 172.21.15.45:0/1279074463 >> v2:172.21.15.135:3300/0 conn(0x7f0b580ce4b0 0x7f0b58167d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:09.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 >> v1:172.21.15.45:6789/0 conn(0x7f0b580b8c20 legacy=0x7f0b580cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b40003a00 0x7f0b40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.726+0000 7f0b567fc700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b40004850 0x7f0b40004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b567fc700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b4000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580d1bf0 con 0x7f0b40005210 2023-08-27T11:27:09.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4000c0b0 con 0x7f0b40003a00 2023-08-27T11:27:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4000c270 con 0x7f0b40004850 2023-08-27T11:27:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b57fff700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b4000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b577fe700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b40003a00 0x7f0b40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b56ffd700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b40004850 0x7f0b40004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b57fff700 1 -- 172.21.15.45:0/1279074463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b40004850 msgr2=0x7f0b40004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:09.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b57fff700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b40004850 0x7f0b40004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b57fff700 1 -- 172.21.15.45:0/1279074463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b40003a00 msgr2=0x7f0b40003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:09.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b57fff700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b40003a00 0x7f0b40003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b57fff700 1 -- 172.21.15.45:0/1279074463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400035b0 con 0x7f0b40005210 2023-08-27T11:27:09.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b57fff700 1 -- 172.21.15.45:0/1279074463 reap_dead start 2023-08-27T11:27:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b57fff700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b4000ba10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000e320 tx=0x7f0b40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeda586e4661e295 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5000edc0 con 0x7f0b40005210 2023-08-27T11:27:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5000f040 con 0x7f0b40005210 2023-08-27T11:27:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.727+0000 7f0b567fc700 1 -- 172.21.15.45:0/1279074463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b500146c0 con 0x7f0b40005210 2023-08-27T11:27:09.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.728+0000 7f0b5e115700 1 -- 172.21.15.45:0/1279074463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 msgr2=0x7f0b4000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.728+0000 7f0b5e115700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b4000ba10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000e320 tx=0x7f0b40005880 comp rx=0 tx=0).stop 2023-08-27T11:27:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.728+0000 7f0b5e115700 1 -- 172.21.15.45:0/1279074463 shutdown_connections 2023-08-27T11:27:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.728+0000 7f0b5e115700 1 --2- 172.21.15.45:0/1279074463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b4000ba10 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.728+0000 7f0b5e115700 1 -- 172.21.15.45:0/1279074463 >> 172.21.15.45:0/1279074463 conn(0x7f0b580d2fd0 msgr2=0x7f0b580d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.728+0000 7f0b5e115700 1 -- 172.21.15.45:0/1279074463 shutdown_connections 2023-08-27T11:27:09.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.728+0000 7f0b5e115700 1 -- 172.21.15.45:0/1279074463 wait complete. 2023-08-27T11:27:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.729+0000 7f0b5e115700 1 Processor -- start 2023-08-27T11:27:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.730+0000 7f0b5e115700 1 -- start start 2023-08-27T11:27:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b5e115700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b581512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b5e115700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b580c53e0 0x7f0b580cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b5e115700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b580ca0a0 0x7f0b580ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b5e115700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580cd030 con 0x7f0b40005210 2023-08-27T11:27:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b5e115700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580caa70 con 0x7f0b580ca0a0 2023-08-27T11:27:09.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b5e115700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580cca70 con 0x7f0b580c53e0 2023-08-27T11:27:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b577fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b581512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b577fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b581512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57208/0 (socket says 172.21.15.45:57208) 2023-08-27T11:27:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b577fe700 1 -- 172.21.15.45:0/3134763787 learned_addr learned my addr 172.21.15.45:0/3134763787 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b57fff700 1 --2- 172.21.15.45:0/3134763787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b580ca0a0 0x7f0b580ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b56ffd700 1 --2- 172.21.15.45:0/3134763787 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b580c53e0 0x7f0b580cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b577fe700 1 -- 172.21.15.45:0/3134763787 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b580c53e0 msgr2=0x7f0b580cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b577fe700 1 --2- 172.21.15.45:0/3134763787 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b580c53e0 0x7f0b580cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b577fe700 1 -- 172.21.15.45:0/3134763787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b580ca0a0 msgr2=0x7f0b580ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b577fe700 1 --2- 172.21.15.45:0/3134763787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b580ca0a0 0x7f0b580ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b577fe700 1 -- 172.21.15.45:0/3134763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b580ccd50 con 0x7f0b40005210 2023-08-27T11:27:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b577fe700 1 --2- 172.21.15.45:0/3134763787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b581512a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00f7f0 tx=0x7f0b4c00fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b3ffff700 1 -- 172.21.15.45:0/3134763787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0052e0 con 0x7f0b40005210 2023-08-27T11:27:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b3ffff700 1 -- 172.21.15.45:0/3134763787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c005440 con 0x7f0b40005210 2023-08-27T11:27:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.731+0000 7f0b5e115700 1 -- 172.21.15.45:0/3134763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b580c7390 con 0x7f0b40005210 2023-08-27T11:27:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.732+0000 7f0b3ffff700 1 -- 172.21.15.45:0/3134763787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0038a0 con 0x7f0b40005210 2023-08-27T11:27:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.732+0000 7f0b5e115700 1 -- 172.21.15.45:0/3134763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b580c5860 con 0x7f0b40005210 2023-08-27T11:27:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.733+0000 7f0b5e115700 1 -- 172.21.15.45:0/3134763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b580c71f0 con 0x7f0b40005210 2023-08-27T11:27:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.733+0000 7f0b3ffff700 1 -- 172.21.15.45:0/3134763787 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f0b4c003a00 con 0x7f0b40005210 2023-08-27T11:27:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.733+0000 7f0b3ffff700 1 -- 172.21.15.45:0/3134763787 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f0b4c0880c0 con 0x7f0b40005210 2023-08-27T11:27:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.738+0000 7f0b3ffff700 1 -- 172.21.15.45:0/3134763787 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0b580c71f0 con 0x7f0b40005210 2023-08-27T11:27:09.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.942+0000 7f0b5e115700 1 -- 172.21.15.45:0/3134763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b580c71f0 con 0x7f0b40005210 2023-08-27T11:27:09.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.943+0000 7f0b3ffff700 1 -- 172.21.15.45:0/3134763787 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19716 (secure 0 0 0) 0x7f0b580c71f0 con 0x7f0b40005210 2023-08-27T11:27:09.946 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:09.946 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":74,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:09.329686+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:08.337329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:09.446448+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":73,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/1159058976":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/1568057874":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/3797775120":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/143531536":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/2756325788":"2023-08-28T11:27:09.329619+0000","172.21.15.135:6833/1664443653":"2023-08-28T11:27:09.329619+0000","172.21.15.135:6832/1664443653":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/3859433979":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/485008861":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/3010439837":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/2511615876":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1537214801":"2023-08-28T11:25:29.254479+0000","172.21.15.135:6832/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/726344780":"2023-08-28T11:27:09.329619+0000","172.21.15.135:6833/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1231508777":"2023-08-28T11:25:29.254479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]},{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:09.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.946+0000 7f0b5e115700 1 -- 172.21.15.45:0/3134763787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 msgr2=0x7f0b581512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:09.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.946+0000 7f0b5e115700 1 --2- 172.21.15.45:0/3134763787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b581512a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00f7f0 tx=0x7f0b4c00fb00 comp rx=0 tx=0).stop 2023-08-27T11:27:09.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.946+0000 7f0b5e115700 1 -- 172.21.15.45:0/3134763787 shutdown_connections 2023-08-27T11:27:09.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.946+0000 7f0b5e115700 1 --2- 172.21.15.45:0/3134763787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b40005210 0x7f0b581512a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.946+0000 7f0b5e115700 1 --2- 172.21.15.45:0/3134763787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b580ca0a0 0x7f0b580ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.946+0000 7f0b5e115700 1 --2- 172.21.15.45:0/3134763787 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0b580c53e0 0x7f0b580cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.946+0000 7f0b5e115700 1 -- 172.21.15.45:0/3134763787 >> 172.21.15.45:0/3134763787 conn(0x7f0b580d2fd0 msgr2=0x7f0b5815b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.946+0000 7f0b5e115700 1 -- 172.21.15.45:0/3134763787 shutdown_connections 2023-08-27T11:27:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:09.946+0000 7f0b5e115700 1 -- 172.21.15.45:0/3134763787 wait complete. 2023-08-27T11:27:09.960 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:27:09.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.085+0000 7fe69c76c700 1 Processor -- start 2023-08-27T11:27:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.085+0000 7fe69c76c700 1 -- start start 2023-08-27T11:27:10.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.086+0000 7fe69c76c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940ceba0 0x7fe6940cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.086+0000 7fe69c76c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6940cbe70 0x7fe69406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.086+0000 7fe69c76c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69406d690 con 0x7fe69415a240 2023-08-27T11:27:10.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.086+0000 7fe69c76c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6940cc440 con 0x7fe6940cbe70 2023-08-27T11:27:10.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.086+0000 7fe69c76c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69406d040 con 0x7fe6940ceba0 2023-08-27T11:27:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.086+0000 7fe69ad09700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6940cbe70 0x7fe69406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe69ad09700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6940cbe70 0x7fe69406db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57218/0 (socket says 172.21.15.45:57218) 2023-08-27T11:27:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe69ad09700 1 -- 172.21.15.45:0/629332536 learned_addr learned my addr 172.21.15.45:0/629332536 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe699d07700 1 --2- 172.21.15.45:0/629332536 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940ceba0 0x7fe6940cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe69ad09700 1 -- 172.21.15.45:0/629332536 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940ceba0 msgr2=0x7fe6940cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe69ad09700 1 --2- 172.21.15.45:0/629332536 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940ceba0 0x7fe6940cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe69ad09700 1 -- 172.21.15.45:0/629332536 >> v1:172.21.15.135:6789/0 conn(0x7fe69415a240 legacy=0x7fe6940ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe69ad09700 1 -- 172.21.15.45:0/629332536 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69406d320 con 0x7fe6940cbe70 2023-08-27T11:27:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe69ad09700 1 --2- 172.21.15.45:0/629332536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6940cbe70 0x7fe69406db20 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009e50 tx=0x7fe68c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd3213ea90d46a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe699506700 1 -- 172.21.15.45:0/629332536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c00e070 con 0x7fe6940cbe70 2023-08-27T11:27:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe699506700 1 -- 172.21.15.45:0/629332536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68c003a20 con 0x7fe6940cbe70 2023-08-27T11:27:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.087+0000 7fe699506700 1 -- 172.21.15.45:0/629332536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c003dd0 con 0x7fe6940cbe70 2023-08-27T11:27:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.088+0000 7fe69c76c700 1 -- 172.21.15.45:0/629332536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6940cbe70 msgr2=0x7fe69406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.088+0000 7fe69c76c700 1 --2- 172.21.15.45:0/629332536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6940cbe70 0x7fe69406db20 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009e50 tx=0x7fe68c00b330 comp rx=0 tx=0).stop 2023-08-27T11:27:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.088+0000 7fe69c76c700 1 -- 172.21.15.45:0/629332536 shutdown_connections 2023-08-27T11:27:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.088+0000 7fe69c76c700 1 --2- 172.21.15.45:0/629332536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6940cbe70 0x7fe69406db20 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.088+0000 7fe69c76c700 1 --2- 172.21.15.45:0/629332536 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940ceba0 0x7fe6940cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.088+0000 7fe69c76c700 1 -- 172.21.15.45:0/629332536 >> 172.21.15.45:0/629332536 conn(0x7fe6940cfe50 msgr2=0x7fe6940d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.088+0000 7fe69c76c700 1 -- 172.21.15.45:0/629332536 shutdown_connections 2023-08-27T11:27:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.088+0000 7fe69c76c700 1 -- 172.21.15.45:0/629332536 wait complete. 2023-08-27T11:27:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.089+0000 7fe69c76c700 1 Processor -- start 2023-08-27T11:27:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.089+0000 7fe69c76c700 1 -- start start 2023-08-27T11:27:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69c76c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940cbe70 0x7fe6941f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69c76c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6940ceba0 0x7fe694152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69c76c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69415a240 0x7fe69406a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69c76c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe694157c00 con 0x7fe69415a240 2023-08-27T11:27:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69c76c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe694157ea0 con 0x7fe6940ceba0 2023-08-27T11:27:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69c76c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6941f65c0 con 0x7fe6940cbe70 2023-08-27T11:27:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69ad09700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69415a240 0x7fe69406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69ad09700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69415a240 0x7fe69406a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57224/0 (socket says 172.21.15.45:57224) 2023-08-27T11:27:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69ad09700 1 -- 172.21.15.45:0/2358936525 learned_addr learned my addr 172.21.15.45:0/2358936525 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69a508700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940cbe70 0x7fe6941f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe699d07700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6940ceba0 0x7fe694152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69ad09700 1 -- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940cbe70 msgr2=0x7fe6941f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69ad09700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940cbe70 0x7fe6941f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69ad09700 1 -- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6940ceba0 msgr2=0x7fe694152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69ad09700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6940ceba0 0x7fe694152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69ad09700 1 -- 172.21.15.45:0/2358936525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c009b30 con 0x7fe69415a240 2023-08-27T11:27:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.090+0000 7fe69ad09700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69415a240 0x7fe69406a280 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe69406c730 tx=0x7fe68c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.091+0000 7fe68affd700 1 -- 172.21.15.45:0/2358936525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c004440 con 0x7fe69415a240 2023-08-27T11:27:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.091+0000 7fe68affd700 1 -- 172.21.15.45:0/2358936525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68c0045c0 con 0x7fe69415a240 2023-08-27T11:27:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.091+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe694152a70 con 0x7fe69415a240 2023-08-27T11:27:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.091+0000 7fe68affd700 1 -- 172.21.15.45:0/2358936525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c004970 con 0x7fe69415a240 2023-08-27T11:27:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.091+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6941f6b40 con 0x7fe69415a240 2023-08-27T11:27:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.091+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe694072810 con 0x7fe69415a240 2023-08-27T11:27:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.092+0000 7fe68affd700 1 -- 172.21.15.45:0/2358936525 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fe68c025450 con 0x7fe69415a240 2023-08-27T11:27:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.092+0000 7fe68affd700 1 -- 172.21.15.45:0/2358936525 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7142+0+0 (secure 0 0 0) 0x7fe68c089a60 con 0x7fe69415a240 2023-08-27T11:27:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.097+0000 7fe68affd700 1 -- 172.21.15.45:0/2358936525 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe68c054f90 con 0x7fe69415a240 2023-08-27T11:27:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.339+0000 7fe68affd700 1 -- 172.21.15.45:0/2358936525 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 28) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fe68c054c10 con 0x7fe69415a240 2023-08-27T11:27:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.339+0000 7fe68affd700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fe6780704c0 0x7fe6780728b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.340+0000 7fe69a508700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fe6780704c0 0x7fe6780728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.342+0000 7fe69a508700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fe6780704c0 0x7fe6780728b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe684009960 tx=0x7fe6840090e0 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.343+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe694157fe0 con 0x7fe69415a240 2023-08-27T11:27:10.345 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.343+0000 7fe68affd700 1 -- 172.21.15.45:0/2358936525 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4872 (secure 0 0 0) 0x7fe68c06e740 con 0x7fe69415a240 2023-08-27T11:27:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:10.346 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/72415568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":72415568},{"type":"v1","addr":"172.21.15.135:6837","nonce":72415568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2554143619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2554143619},{"type":"v1","addr":"172.21.15.135:6835","nonce":2554143619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6306,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/990560009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":990560009},{"type":"v1","addr":"172.21.15.45:6834","nonce":990560009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:10.472982+0000","modified":"2023-08-27T11:25:14.080477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6309":{"gid":6309,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6835/2659160949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2659160949},{"type":"v1","addr":"172.21.15.45:6835","nonce":2659160949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-27T11:27:10.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fe6780704c0 msgr2=0x7fe6780728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fe6780704c0 0x7fe6780728b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe684009960 tx=0x7fe6840090e0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69415a240 msgr2=0x7fe69406a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69415a240 0x7fe69406a280 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe69406c730 tx=0x7fe68c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 shutdown_connections 2023-08-27T11:27:10.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fe6780704c0 0x7fe6780728b0 secure :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe684009960 tx=0x7fe6840090e0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69415a240 0x7fe69406a280 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe6940cbe70 0x7fe6941f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 --2- 172.21.15.45:0/2358936525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6940ceba0 0x7fe694152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 >> 172.21.15.45:0/2358936525 conn(0x7fe6940cfe50 msgr2=0x7fe69415c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 shutdown_connections 2023-08-27T11:27:10.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.346+0000 7fe69c76c700 1 -- 172.21.15.45:0/2358936525 wait complete. 2023-08-27T11:27:10.351 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-08-27T11:27:10.360 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.484+0000 7f70effbc700 1 Processor -- start 2023-08-27T11:27:10.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.484+0000 7f70effbc700 1 -- start start 2023-08-27T11:27:10.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70effbc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f70e815d620 0x7f70e80d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70effbc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70e80b43c0 0x7f70e80d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70effbc700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e80d3fe0 con 0x7f70e80d0f30 2023-08-27T11:27:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70effbc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e80b25f0 con 0x7f70e80b43c0 2023-08-27T11:27:10.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70effbc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e80b28d0 con 0x7f70e815d620 2023-08-27T11:27:10.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70ee559700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70e80b43c0 0x7f70e80d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70ee559700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70e80b43c0 0x7f70e80d1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57246/0 (socket says 172.21.15.45:57246) 2023-08-27T11:27:10.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70ee559700 1 -- 172.21.15.45:0/4112277009 learned_addr learned my addr 172.21.15.45:0/4112277009 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:10.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70edd58700 1 --2- 172.21.15.45:0/4112277009 >> v2:172.21.15.135:3300/0 conn(0x7f70e815d620 0x7f70e80d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70ee559700 1 -- 172.21.15.45:0/4112277009 >> v2:172.21.15.135:3300/0 conn(0x7f70e815d620 msgr2=0x7f70e80d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70ee559700 1 --2- 172.21.15.45:0/4112277009 >> v2:172.21.15.135:3300/0 conn(0x7f70e815d620 0x7f70e80d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70ee559700 1 -- 172.21.15.45:0/4112277009 >> v1:172.21.15.135:6789/0 conn(0x7f70e80d0f30 legacy=0x7f70e80cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70ee559700 1 -- 172.21.15.45:0/4112277009 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e80b2bb0 con 0x7f70e80b43c0 2023-08-27T11:27:10.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.485+0000 7f70ee559700 1 --2- 172.21.15.45:0/4112277009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e80b43c0 0x7f70e80d1c70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f70e806d2c0 tx=0x7f70e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddb76736f10cbf2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70ecd56700 1 -- 172.21.15.45:0/4112277009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e000e070 con 0x7f70e80b43c0 2023-08-27T11:27:10.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70ecd56700 1 -- 172.21.15.45:0/4112277009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e0003a20 con 0x7f70e80b43c0 2023-08-27T11:27:10.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70ecd56700 1 -- 172.21.15.45:0/4112277009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0003dd0 con 0x7f70e80b43c0 2023-08-27T11:27:10.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70effbc700 1 -- 172.21.15.45:0/4112277009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e80b43c0 msgr2=0x7f70e80d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70effbc700 1 --2- 172.21.15.45:0/4112277009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e80b43c0 0x7f70e80d1c70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f70e806d2c0 tx=0x7f70e000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:10.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70effbc700 1 -- 172.21.15.45:0/4112277009 shutdown_connections 2023-08-27T11:27:10.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70effbc700 1 --2- 172.21.15.45:0/4112277009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e80b43c0 0x7f70e80d1c70 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70effbc700 1 --2- 172.21.15.45:0/4112277009 >> v2:172.21.15.135:3300/0 conn(0x7f70e815d620 0x7f70e80d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70effbc700 1 -- 172.21.15.45:0/4112277009 >> 172.21.15.45:0/4112277009 conn(0x7f70e80d3150 msgr2=0x7f70e80d3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70effbc700 1 -- 172.21.15.45:0/4112277009 shutdown_connections 2023-08-27T11:27:10.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.486+0000 7f70effbc700 1 -- 172.21.15.45:0/4112277009 wait complete. 2023-08-27T11:27:10.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.487+0000 7f70effbc700 1 Processor -- start 2023-08-27T11:27:10.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.487+0000 7f70effbc700 1 -- start start 2023-08-27T11:27:10.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70effbc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70e80b43c0 0x7f70e815fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70effbc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70e80d0f30 0x7f70e815e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70effbc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e815d620 0x7f70e8152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70effbc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80ca220 con 0x7f70e815d620 2023-08-27T11:27:10.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70effbc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e815edf0 con 0x7f70e80d0f30 2023-08-27T11:27:10.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70effbc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e815f0d0 con 0x7f70e80b43c0 2023-08-27T11:27:10.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70ee559700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e815d620 0x7f70e8152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70ee559700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e815d620 0x7f70e8152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57252/0 (socket says 172.21.15.45:57252) 2023-08-27T11:27:10.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70ee559700 1 -- 172.21.15.45:0/2783933909 learned_addr learned my addr 172.21.15.45:0/2783933909 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:10.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70edd58700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70e80b43c0 0x7f70e815fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70ed557700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70e80d0f30 0x7f70e815e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70ee559700 1 -- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70e80b43c0 msgr2=0x7f70e815fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70ee559700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70e80b43c0 0x7f70e815fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70ee559700 1 -- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70e80d0f30 msgr2=0x7f70e815e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70ee559700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70e80d0f30 0x7f70e815e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.488+0000 7f70ee559700 1 -- 172.21.15.45:0/2783933909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e0009b30 con 0x7f70e815d620 2023-08-27T11:27:10.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.489+0000 7f70ee559700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e815d620 0x7f70e8152370 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b370 tx=0x7f70e0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.489+0000 7f70de7fc700 1 -- 172.21.15.45:0/2783933909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0004440 con 0x7f70e815d620 2023-08-27T11:27:10.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.489+0000 7f70de7fc700 1 -- 172.21.15.45:0/2783933909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e00045a0 con 0x7f70e815d620 2023-08-27T11:27:10.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.489+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e80ca4e0 con 0x7f70e815d620 2023-08-27T11:27:10.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.489+0000 7f70de7fc700 1 -- 172.21.15.45:0/2783933909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0004950 con 0x7f70e815d620 2023-08-27T11:27:10.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.489+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e8160140 con 0x7f70e815d620 2023-08-27T11:27:10.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.490+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e81602d0 con 0x7f70e815d620 2023-08-27T11:27:10.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.490+0000 7f70de7fc700 1 -- 172.21.15.45:0/2783933909 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 28) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f70e0012430 con 0x7f70e815d620 2023-08-27T11:27:10.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.490+0000 7f70de7fc700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f70cc069f30 0x7f70cc06c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.491+0000 7f70de7fc700 1 -- 172.21.15.45:0/2783933909 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f70e0089a60 con 0x7f70e815d620 2023-08-27T11:27:10.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.491+0000 7f70edd58700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f70cc069f30 0x7f70cc06c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.491+0000 7f70edd58700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f70cc069f30 0x7f70cc06c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009770 tx=0x7f70e4008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.495+0000 7f70de7fc700 1 -- 172.21.15.45:0/2783933909 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f70e0055050 con 0x7f70e815d620 2023-08-27T11:27:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.738+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70e80ca9a0 con 0x7f70e815d620 2023-08-27T11:27:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.738+0000 7f70de7fc700 1 -- 172.21.15.45:0/2783933909 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4872 (secure 0 0 0) 0x7f70e0058670 con 0x7f70e815d620 2023-08-27T11:27:10.741 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:10.741 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/72415568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":72415568},{"type":"v1","addr":"172.21.15.135:6837","nonce":72415568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2554143619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2554143619},{"type":"v1","addr":"172.21.15.135:6835","nonce":2554143619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6306,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/990560009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":990560009},{"type":"v1","addr":"172.21.15.45:6834","nonce":990560009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:25:10.472982+0000","modified":"2023-08-27T11:25:14.080477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6309":{"gid":6309,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":5,"addr":"172.21.15.45:6835/2659160949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2659160949},{"type":"v1","addr":"172.21.15.45:6835","nonce":2659160949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-27T11:27:10.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f70cc069f30 msgr2=0x7f70cc06c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f70cc069f30 0x7f70cc06c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009770 tx=0x7f70e4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:10.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e815d620 msgr2=0x7f70e8152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e815d620 0x7f70e8152370 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b370 tx=0x7f70e0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 shutdown_connections 2023-08-27T11:27:10.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f70cc069f30 0x7f70cc06c3f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e815d620 0x7f70e8152370 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f70e80b43c0 0x7f70e815fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 --2- 172.21.15.45:0/2783933909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70e80d0f30 0x7f70e815e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.740+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 >> 172.21.15.45:0/2783933909 conn(0x7f70e80d3150 msgr2=0x7f70e80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.741+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 shutdown_connections 2023-08-27T11:27:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.741+0000 7f70effbc700 1 -- 172.21.15.45:0/2783933909 wait complete. 2023-08-27T11:27:10.745 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2023-08-27T11:27:10.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.880+0000 7f7988f8b700 1 Processor -- start 2023-08-27T11:27:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.880+0000 7f7988f8b700 1 -- start start 2023-08-27T11:27:10.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f7988f8b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f798415d3d0 0x7f79841619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f7988f8b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f798406e3a0 0x7f798406e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f7988f8b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79840cfb30 con 0x7f79840af040 2023-08-27T11:27:10.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f7988f8b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79840cdf90 con 0x7f798415d3d0 2023-08-27T11:27:10.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f7988f8b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798406ed70 con 0x7f798406e3a0 2023-08-27T11:27:10.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f798259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f798415d3d0 0x7f79841619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f798259c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f798415d3d0 0x7f79841619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57258/0 (socket says 172.21.15.45:57258) 2023-08-27T11:27:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f798259c700 1 -- 172.21.15.45:0/2384254524 learned_addr learned my addr 172.21.15.45:0/2384254524 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f7982d9d700 1 --2- 172.21.15.45:0/2384254524 >> v2:172.21.15.135:3300/0 conn(0x7f798406e3a0 0x7f798406e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f798259c700 1 -- 172.21.15.45:0/2384254524 >> v2:172.21.15.135:3300/0 conn(0x7f798406e3a0 msgr2=0x7f798406e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f798259c700 1 --2- 172.21.15.45:0/2384254524 >> v2:172.21.15.135:3300/0 conn(0x7f798406e3a0 0x7f798406e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f798259c700 1 -- 172.21.15.45:0/2384254524 >> v1:172.21.15.45:6789/0 conn(0x7f79840af040 legacy=0x7f79840cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f798259c700 1 -- 172.21.15.45:0/2384254524 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7984161550 con 0x7f798415d3d0 2023-08-27T11:27:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.881+0000 7f798259c700 1 --2- 172.21.15.45:0/2384254524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798415d3d0 0x7f79841619e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7984072a20 tx=0x7f7974009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb061354abddf7e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f798159a700 1 -- 172.21.15.45:0/2384254524 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400b870 con 0x7f798415d3d0 2023-08-27T11:27:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f798159a700 1 -- 172.21.15.45:0/2384254524 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797400b9d0 con 0x7f798415d3d0 2023-08-27T11:27:10.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f798159a700 1 -- 172.21.15.45:0/2384254524 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400bd60 con 0x7f798415d3d0 2023-08-27T11:27:10.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f7988f8b700 1 -- 172.21.15.45:0/2384254524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798415d3d0 msgr2=0x7f79841619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f7988f8b700 1 --2- 172.21.15.45:0/2384254524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798415d3d0 0x7f79841619e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7984072a20 tx=0x7f7974009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f7988f8b700 1 -- 172.21.15.45:0/2384254524 shutdown_connections 2023-08-27T11:27:10.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f7988f8b700 1 --2- 172.21.15.45:0/2384254524 >> v2:172.21.15.135:3300/0 conn(0x7f798406e3a0 0x7f798406e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f7988f8b700 1 --2- 172.21.15.45:0/2384254524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798415d3d0 0x7f79841619e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f7988f8b700 1 -- 172.21.15.45:0/2384254524 >> 172.21.15.45:0/2384254524 conn(0x7f79840cfd60 msgr2=0x7f79840d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f7988f8b700 1 -- 172.21.15.45:0/2384254524 shutdown_connections 2023-08-27T11:27:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.882+0000 7f7988f8b700 1 -- 172.21.15.45:0/2384254524 wait complete. 2023-08-27T11:27:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f7988f8b700 1 Processor -- start 2023-08-27T11:27:10.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f7988f8b700 1 -- start start 2023-08-27T11:27:10.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f7988f8b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798406e3a0 0x7f79841f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f7988f8b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79840af040 0x7f79841f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f7988f8b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f798415d3d0 0x7f798414be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f7988f8b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79841512a0 con 0x7f798406e3a0 2023-08-27T11:27:10.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f7988f8b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840b2b00 con 0x7f79840af040 2023-08-27T11:27:10.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f7988f8b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798414ca70 con 0x7f798415d3d0 2023-08-27T11:27:10.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f798259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798406e3a0 0x7f79841f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f798259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798406e3a0 0x7f79841f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57264/0 (socket says 172.21.15.45:57264) 2023-08-27T11:27:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.884+0000 7f798259c700 1 -- 172.21.15.45:0/3677137174 learned_addr learned my addr 172.21.15.45:0/3677137174 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f798259c700 1 -- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f798415d3d0 msgr2=0x7f798414be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f7982d9d700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f798415d3d0 0x7f798414be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f798259c700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f798415d3d0 0x7f798414be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f798259c700 1 -- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79840af040 msgr2=0x7f79841f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:10.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f7981d9b700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79840af040 0x7f79841f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f798259c700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79840af040 0x7f79841f67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f798259c700 1 -- 172.21.15.45:0/3677137174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7974009550 con 0x7f798406e3a0 2023-08-27T11:27:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f798259c700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798406e3a0 0x7f79841f6260 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7974000c00 tx=0x7f7974004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f7972ffd700 1 -- 172.21.15.45:0/3677137174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974004540 con 0x7f798406e3a0 2023-08-27T11:27:10.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f7972ffd700 1 -- 172.21.15.45:0/3677137174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79740036a0 con 0x7f798406e3a0 2023-08-27T11:27:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798414cd50 con 0x7f798406e3a0 2023-08-27T11:27:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f7972ffd700 1 -- 172.21.15.45:0/3677137174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974003a30 con 0x7f798406e3a0 2023-08-27T11:27:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.885+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798414ae80 con 0x7f798406e3a0 2023-08-27T11:27:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.886+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79840b2c70 con 0x7f798406e3a0 2023-08-27T11:27:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.886+0000 7f7972ffd700 1 -- 172.21.15.45:0/3677137174 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 28) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f7974021070 con 0x7f798406e3a0 2023-08-27T11:27:10.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.887+0000 7f7972ffd700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f7964069f30 0x7f796406c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:10.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.887+0000 7f7972ffd700 1 -- 172.21.15.45:0/3677137174 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f7974089d50 con 0x7f798406e3a0 2023-08-27T11:27:10.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.887+0000 7f7981d9b700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f7964069f30 0x7f796406c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:10.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.888+0000 7f7981d9b700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f7964069f30 0x7f796406c3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f79841519b0 tx=0x7f7978009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:10.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:10.891+0000 7f7972ffd700 1 -- 172.21.15.45:0/3677137174 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7974055340 con 0x7f798406e3a0 2023-08-27T11:27:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.094+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f798406e830 con 0x7f798406e3a0 2023-08-27T11:27:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.095+0000 7f7972ffd700 1 -- 172.21.15.45:0/3677137174 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19716 (secure 0 0 0) 0x7f7974015020 con 0x7f798406e3a0 2023-08-27T11:27:11.097 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:11.098 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":74,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:09.329686+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:25:08.337329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-27T11:25:09.446448+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":73,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/1159058976":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/1568057874":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/3797775120":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/143531536":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/2756325788":"2023-08-28T11:27:09.329619+0000","172.21.15.135:6833/1664443653":"2023-08-28T11:27:09.329619+0000","172.21.15.135:6832/1664443653":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/3859433979":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/485008861":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/3010439837":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/2511615876":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1537214801":"2023-08-28T11:25:29.254479+0000","172.21.15.135:6832/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/726344780":"2023-08-28T11:27:09.329619+0000","172.21.15.135:6833/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1231508777":"2023-08-28T11:25:29.254479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]},{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:11.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f7964069f30 msgr2=0x7f796406c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f7964069f30 0x7f796406c3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f79841519b0 tx=0x7f7978009040 comp rx=0 tx=0).stop 2023-08-27T11:27:11.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798406e3a0 msgr2=0x7f79841f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798406e3a0 0x7f79841f6260 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7974000c00 tx=0x7f7974004750 comp rx=0 tx=0).stop 2023-08-27T11:27:11.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 shutdown_connections 2023-08-27T11:27:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f7964069f30 0x7f796406c3f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f798406e3a0 0x7f79841f6260 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f798415d3d0 0x7f798414be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 --2- 172.21.15.45:0/3677137174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79840af040 0x7f79841f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 >> 172.21.15.45:0/3677137174 conn(0x7f79840cfd60 msgr2=0x7f79840adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 shutdown_connections 2023-08-27T11:27:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.097+0000 7f7988f8b700 1 -- 172.21.15.45:0/3677137174 wait complete. 2023-08-27T11:27:11.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:27:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.235+0000 7f426f4ca700 1 Processor -- start 2023-08-27T11:27:11.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.236+0000 7f426f4ca700 1 -- start start 2023-08-27T11:27:11.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.236+0000 7f426f4ca700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680ca620 0x7f42680cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.236+0000 7f426f4ca700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42680caa00 0x7f42680acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.236+0000 7f426f4ca700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42680ce340 con 0x7f42680af200 2023-08-27T11:27:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.236+0000 7f426f4ca700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42680ceec0 con 0x7f42680caa00 2023-08-27T11:27:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.236+0000 7f426f4ca700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426815d7c0 con 0x7f42680ca620 2023-08-27T11:27:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.236+0000 7f426da67700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42680caa00 0x7f42680acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f426da67700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42680caa00 0x7f42680acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57276/0 (socket says 172.21.15.45:57276) 2023-08-27T11:27:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f426da67700 1 -- 172.21.15.45:0/2023759425 learned_addr learned my addr 172.21.15.45:0/2023759425 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f426d266700 1 --2- 172.21.15.45:0/2023759425 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680ca620 0x7f42680cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f426da67700 1 -- 172.21.15.45:0/2023759425 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680ca620 msgr2=0x7f42680cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f426da67700 1 --2- 172.21.15.45:0/2023759425 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680ca620 0x7f42680cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f426da67700 1 -- 172.21.15.45:0/2023759425 >> v1:172.21.15.135:6789/0 conn(0x7f42680af200 legacy=0x7f42680ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f426da67700 1 -- 172.21.15.45:0/2023759425 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680cc570 con 0x7f42680caa00 2023-08-27T11:27:11.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f426d266700 1 --2- 172.21.15.45:0/2023759425 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680ca620 0x7f42680cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f426da67700 1 --2- 172.21.15.45:0/2023759425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680caa00 0x7f42680acfc0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4260009e50 tx=0x7f426000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8397d05539800a18 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f425ffff700 1 -- 172.21.15.45:0/2023759425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000e070 con 0x7f42680caa00 2023-08-27T11:27:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f425ffff700 1 -- 172.21.15.45:0/2023759425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426000b9c0 con 0x7f42680caa00 2023-08-27T11:27:11.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.237+0000 7f425ffff700 1 -- 172.21.15.45:0/2023759425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000bd70 con 0x7f42680caa00 2023-08-27T11:27:11.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.238+0000 7f426f4ca700 1 -- 172.21.15.45:0/2023759425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680caa00 msgr2=0x7f42680acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.238+0000 7f426f4ca700 1 --2- 172.21.15.45:0/2023759425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680caa00 0x7f42680acfc0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4260009e50 tx=0x7f426000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:11.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.238+0000 7f426f4ca700 1 -- 172.21.15.45:0/2023759425 shutdown_connections 2023-08-27T11:27:11.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.238+0000 7f426f4ca700 1 --2- 172.21.15.45:0/2023759425 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680ca620 0x7f42680cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.238+0000 7f426f4ca700 1 --2- 172.21.15.45:0/2023759425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680caa00 0x7f42680acfc0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.238+0000 7f426f4ca700 1 -- 172.21.15.45:0/2023759425 >> 172.21.15.45:0/2023759425 conn(0x7f42680cc7e0 msgr2=0x7f42680ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.238+0000 7f426f4ca700 1 -- 172.21.15.45:0/2023759425 shutdown_connections 2023-08-27T11:27:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.238+0000 7f426f4ca700 1 -- 172.21.15.45:0/2023759425 wait complete. 2023-08-27T11:27:11.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.239+0000 7f426f4ca700 1 Processor -- start 2023-08-27T11:27:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.239+0000 7f426f4ca700 1 -- start start 2023-08-27T11:27:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426f4ca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680af200 0x7f426806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426f4ca700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42680ca620 0x7f42681440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426f4ca700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680caa00 0x7f426806ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426f4ca700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4268144610 con 0x7f42680af200 2023-08-27T11:27:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426f4ca700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426806da80 con 0x7f42680ca620 2023-08-27T11:27:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426f4ca700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680b5990 con 0x7f42680caa00 2023-08-27T11:27:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426d266700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680af200 0x7f426806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426d266700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680af200 0x7f426806d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57278/0 (socket says 172.21.15.45:57278) 2023-08-27T11:27:11.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426d266700 1 -- 172.21.15.45:0/3967447370 learned_addr learned my addr 172.21.15.45:0/3967447370 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:11.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426da67700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680caa00 0x7f426806ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426ca65700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42680ca620 0x7f42681440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426d266700 1 -- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680caa00 msgr2=0x7f426806ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426d266700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680caa00 0x7f426806ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426d266700 1 -- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42680ca620 msgr2=0x7f42681440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426d266700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42680ca620 0x7f42681440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426d266700 1 -- 172.21.15.45:0/3967447370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260009b30 con 0x7f42680af200 2023-08-27T11:27:11.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.240+0000 7f426d266700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680af200 0x7f426806d3e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f425800e810 tx=0x7f425800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.241+0000 7f425dffb700 1 -- 172.21.15.45:0/3967447370 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425800c070 con 0x7f42680af200 2023-08-27T11:27:11.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.241+0000 7f425dffb700 1 -- 172.21.15.45:0/3967447370 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4258004230 con 0x7f42680af200 2023-08-27T11:27:11.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.241+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42680c5f40 con 0x7f42680af200 2023-08-27T11:27:11.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.241+0000 7f425dffb700 1 -- 172.21.15.45:0/3967447370 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42580036a0 con 0x7f42680af200 2023-08-27T11:27:11.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.241+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42680c6500 con 0x7f42680af200 2023-08-27T11:27:11.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.242+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426806dd20 con 0x7f42680af200 2023-08-27T11:27:11.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.242+0000 7f425dffb700 1 -- 172.21.15.45:0/3967447370 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 28) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f4258003800 con 0x7f42680af200 2023-08-27T11:27:11.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.242+0000 7f425dffb700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f424c069f30 0x7f424c06c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.243+0000 7f425dffb700 1 -- 172.21.15.45:0/3967447370 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7142+0+0 (secure 0 0 0) 0x7f4258088260 con 0x7f42680af200 2023-08-27T11:27:11.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.243+0000 7f426ca65700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f424c069f30 0x7f424c06c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.243+0000 7f426ca65700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f424c069f30 0x7f424c06c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42680ae6c0 tx=0x7f4264009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.247+0000 7f425dffb700 1 -- 172.21.15.45:0/3967447370 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4258053850 con 0x7f42680af200 2023-08-27T11:27:11.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.496+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f42680cbce0 con 0x7f42680af200 2023-08-27T11:27:11.498 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-08-27T11:27:11.496+0000 7f87a1c48700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:27:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.501+0000 7f425dffb700 1 -- 172.21.15.45:0/3967447370 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4258056e70 con 0x7f42680af200 2023-08-27T11:27:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f424c069f30 msgr2=0x7f424c06c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f424c069f30 0x7f424c06c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f42680ae6c0 tx=0x7f4264009040 comp rx=0 tx=0).stop 2023-08-27T11:27:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680af200 msgr2=0x7f426806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680af200 0x7f426806d3e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f425800e810 tx=0x7f425800eb20 comp rx=0 tx=0).stop 2023-08-27T11:27:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 shutdown_connections 2023-08-27T11:27:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f424c069f30 0x7f424c06c3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42680af200 0x7f426806d3e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f42680caa00 0x7f426806ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 --2- 172.21.15.45:0/3967447370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42680ca620 0x7f42681440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 >> 172.21.15.45:0/3967447370 conn(0x7f42680cc7e0 msgr2=0x7f42680cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 shutdown_connections 2023-08-27T11:27:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.503+0000 7f426f4ca700 1 -- 172.21.15.45:0/3967447370 wait complete. 2023-08-27T11:27:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:27:11.518 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:27:11.539 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-08-27T11:27:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.646+0000 7f0eb2eba700 1 Processor -- start 2023-08-27T11:27:11.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.646+0000 7f0eb2eba700 1 -- start start 2023-08-27T11:27:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eb2eba700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0eac15d0a0 0x7f0eac06ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eb2eba700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0eac0d0a50 0x7f0eac0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eb2eba700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 0x7f0eac0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eb2eba700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eac06d3f0 con 0x7f0eac0d0a50 2023-08-27T11:27:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eb2eba700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eac161870 con 0x7f0eac15d0a0 2023-08-27T11:27:11.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eb2eba700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac161da0 con 0x7f0eac15d480 2023-08-27T11:27:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eabfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0eac0d0a50 0x7f0eac0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eabfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0eac0d0a50 0x7f0eac0cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57300/0 (socket says 172.21.15.45:57300) 2023-08-27T11:27:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eabfff700 1 -- 172.21.15.45:0/1681983785 learned_addr learned my addr 172.21.15.45:0/1681983785 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eb0c56700 1 --2- 172.21.15.45:0/1681983785 >> v2:172.21.15.135:3300/0 conn(0x7f0eac15d0a0 0x7f0eac06ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eb1457700 1 --2- 172.21.15.45:0/1681983785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 0x7f0eac0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eabfff700 1 -- 172.21.15.45:0/1681983785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 msgr2=0x7f0eac0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eabfff700 1 --2- 172.21.15.45:0/1681983785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 0x7f0eac0cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eabfff700 1 -- 172.21.15.45:0/1681983785 >> v2:172.21.15.135:3300/0 conn(0x7f0eac15d0a0 msgr2=0x7f0eac06ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eabfff700 1 --2- 172.21.15.45:0/1681983785 >> v2:172.21.15.135:3300/0 conn(0x7f0eac15d0a0 0x7f0eac06ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.647+0000 7f0eabfff700 1 -- 172.21.15.45:0/1681983785 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac161ac0 con 0x7f0eac0d0a50 2023-08-27T11:27:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eabfff700 1 --2- 172.21.15.45:0/1681983785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 0x7f0eac0cd880 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0eac06ad60 tx=0x7f0e98009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbefaf841a1ca9d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eab7fe700 1 -- 172.21.15.45:0/1681983785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800b870 con 0x7f0eac0d0a50 2023-08-27T11:27:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eab7fe700 1 -- 172.21.15.45:0/1681983785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9800b9d0 con 0x7f0eac0d0a50 2023-08-27T11:27:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eab7fe700 1 -- 172.21.15.45:0/1681983785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800bd60 con 0x7f0eac0d0a50 2023-08-27T11:27:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eb2eba700 1 -- 172.21.15.45:0/1681983785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 msgr2=0x7f0eac0cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/1681983785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 0x7f0eac0cd880 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0eac06ad60 tx=0x7f0e98009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eb2eba700 1 -- 172.21.15.45:0/1681983785 shutdown_connections 2023-08-27T11:27:11.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/1681983785 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 0x7f0eac0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/1681983785 >> v2:172.21.15.135:3300/0 conn(0x7f0eac15d0a0 0x7f0eac06ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/1681983785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 0x7f0eac0cd880 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eb2eba700 1 -- 172.21.15.45:0/1681983785 >> 172.21.15.45:0/1681983785 conn(0x7f0eac06f960 msgr2=0x7f0eac06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eb2eba700 1 -- 172.21.15.45:0/1681983785 shutdown_connections 2023-08-27T11:27:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.648+0000 7f0eb2eba700 1 -- 172.21.15.45:0/1681983785 wait complete. 2023-08-27T11:27:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.650+0000 7f0eb2eba700 1 Processor -- start 2023-08-27T11:27:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.650+0000 7f0eb2eba700 1 -- start start 2023-08-27T11:27:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.650+0000 7f0eb2eba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 0x7f0eac14a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.650+0000 7f0eb2eba700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eac15d0a0 0x7f0eac15ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.650+0000 7f0eb2eba700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 0x7f0eac15ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.650+0000 7f0eb2eba700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac14aec0 con 0x7f0eac0d0a50 2023-08-27T11:27:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.650+0000 7f0eb2eba700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0b2850 con 0x7f0eac15d0a0 2023-08-27T11:27:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb2eba700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0b2b30 con 0x7f0eac15d480 2023-08-27T11:27:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb0c56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 0x7f0eac14a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb0c56700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 0x7f0eac14a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57304/0 (socket says 172.21.15.45:57304) 2023-08-27T11:27:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb0c56700 1 -- 172.21.15.45:0/384059456 learned_addr learned my addr 172.21.15.45:0/384059456 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:11.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eabfff700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eac15d0a0 0x7f0eac15ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb0c56700 1 -- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 msgr2=0x7f0eac15ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb1457700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 0x7f0eac15ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb0c56700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 0x7f0eac15ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb0c56700 1 -- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eac15d0a0 msgr2=0x7f0eac15ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:11.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb0c56700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eac15d0a0 0x7f0eac15ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb0c56700 1 -- 172.21.15.45:0/384059456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e98009550 con 0x7f0eac0d0a50 2023-08-27T11:27:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eabfff700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eac15d0a0 0x7f0eac15ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb1457700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 0x7f0eac15ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb0c56700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 0x7f0eac14a5c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000e810 tx=0x7f0ea000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0ea97fa700 1 -- 172.21.15.45:0/384059456 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0013610 con 0x7f0eac0d0a50 2023-08-27T11:27:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac0b2e10 con 0x7f0eac0d0a50 2023-08-27T11:27:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0ea97fa700 1 -- 172.21.15.45:0/384059456 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea0013770 con 0x7f0eac0d0a50 2023-08-27T11:27:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0ea97fa700 1 -- 172.21.15.45:0/384059456 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0013b20 con 0x7f0eac0d0a50 2023-08-27T11:27:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.651+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac14ab00 con 0x7f0eac0d0a50 2023-08-27T11:27:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.652+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eac06f4a0 con 0x7f0eac0d0a50 2023-08-27T11:27:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.653+0000 7f0ea97fa700 1 -- 172.21.15.45:0/384059456 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 28) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f0ea0013c80 con 0x7f0eac0d0a50 2023-08-27T11:27:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.653+0000 7f0ea97fa700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f0e90069f30 0x7f0e9006c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.653+0000 7f0ea97fa700 1 -- 172.21.15.45:0/384059456 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7228+0+0 (secure 0 0 0) 0x7f0ea0088460 con 0x7f0eac0d0a50 2023-08-27T11:27:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.653+0000 7f0eabfff700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f0e90069f30 0x7f0e9006c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.654+0000 7f0eabfff700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f0e90069f30 0x7f0e9006c3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0eac15dc30 tx=0x7f0e98008030 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:11.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.657+0000 7f0ea97fa700 1 -- 172.21.15.45:0/384059456 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0ea0053980 con 0x7f0eac0d0a50 2023-08-27T11:27:11.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:11.903+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0eac0734a0 con 0x7f0eac0d0a50 2023-08-27T11:27:12.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.347+0000 7f0ea97fa700 1 -- 172.21.15.45:0/384059456 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 29) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f0ea0053d50 con 0x7f0eac0d0a50 2023-08-27T11:27:12.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.501+0000 7f0ea97fa700 1 -- 172.21.15.45:0/384059456 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0ea0057410 con 0x7f0eac0d0a50 2023-08-27T11:27:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f0e90069f30 msgr2=0x7f0e9006c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f0e90069f30 0x7f0e9006c3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0eac15dc30 tx=0x7f0e98008030 comp rx=0 tx=0).stop 2023-08-27T11:27:12.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 msgr2=0x7f0eac14a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 0x7f0eac14a5c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000e810 tx=0x7f0ea000ebd0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 shutdown_connections 2023-08-27T11:27:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f0e90069f30 0x7f0e9006c3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0eac0d0a50 0x7f0eac14a5c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0eac15d480 0x7f0eac15ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 --2- 172.21.15.45:0/384059456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eac15d0a0 0x7f0eac15ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 >> 172.21.15.45:0/384059456 conn(0x7f0eac06f960 msgr2=0x7f0eac15ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:12.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 shutdown_connections 2023-08-27T11:27:12.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.504+0000 7f0eb2eba700 1 -- 172.21.15.45:0/384059456 wait complete. 2023-08-27T11:27:12.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:27:12.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc2202b8700 1 Processor -- start 2023-08-27T11:27:12.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc2202b8700 1 -- start start 2023-08-27T11:27:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc2202b8700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc21815b360 0x7fc2180cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc2202b8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 0x7fc2180b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc2202b8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21815b740 con 0x7fc218073800 2023-08-27T11:27:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc2202b8700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21815ef80 con 0x7fc21815b360 2023-08-27T11:27:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc2202b8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21815f260 con 0x7fc218073c80 2023-08-27T11:27:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc21e054700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc21815b360 0x7fc2180cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc21e855700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 0x7fc2180b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc21e054700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc21815b360 0x7fc2180cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:46170/0 (socket says 172.21.15.45:46170) 2023-08-27T11:27:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.696+0000 7fc21e054700 1 -- 172.21.15.45:0/3687842813 learned_addr learned my addr 172.21.15.45:0/3687842813 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21e855700 1 -- 172.21.15.45:0/3687842813 >> v2:172.21.15.135:3300/0 conn(0x7fc21815b360 msgr2=0x7fc2180cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21e855700 1 --2- 172.21.15.45:0/3687842813 >> v2:172.21.15.135:3300/0 conn(0x7fc21815b360 0x7fc2180cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21e855700 1 -- 172.21.15.45:0/3687842813 >> v1:172.21.15.135:6789/0 conn(0x7fc218073800 legacy=0x7fc21806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21d052700 1 -- 172.21.15.45:0/3687842813 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695375782 0 0) 0x7fc208001110 con 0x7fc218073800 2023-08-27T11:27:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21e855700 1 -- 172.21.15.45:0/3687842813 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21815f540 con 0x7fc218073c80 2023-08-27T11:27:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21e054700 1 --2- 172.21.15.45:0/3687842813 >> v2:172.21.15.135:3300/0 conn(0x7fc21815b360 0x7fc2180cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21e855700 1 --2- 172.21.15.45:0/3687842813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 0x7fc2180b0f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc210009d90 tx=0x7fc21000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26fe83d9834029f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21d052700 1 -- 172.21.15.45:0/3687842813 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21000e070 con 0x7fc218073c80 2023-08-27T11:27:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21d052700 1 -- 172.21.15.45:0/3687842813 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21000bb20 con 0x7fc218073c80 2023-08-27T11:27:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.697+0000 7fc21d052700 1 -- 172.21.15.45:0/3687842813 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210012430 con 0x7fc218073c80 2023-08-27T11:27:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.698+0000 7fc2202b8700 1 -- 172.21.15.45:0/3687842813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 msgr2=0x7fc2180b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.698+0000 7fc2202b8700 1 --2- 172.21.15.45:0/3687842813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 0x7fc2180b0f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc210009d90 tx=0x7fc21000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.698+0000 7fc2202b8700 1 -- 172.21.15.45:0/3687842813 shutdown_connections 2023-08-27T11:27:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.698+0000 7fc2202b8700 1 --2- 172.21.15.45:0/3687842813 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 0x7fc2180b0f70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.698+0000 7fc2202b8700 1 --2- 172.21.15.45:0/3687842813 >> v2:172.21.15.135:3300/0 conn(0x7fc21815b360 0x7fc2180cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.698+0000 7fc2202b8700 1 -- 172.21.15.45:0/3687842813 >> 172.21.15.45:0/3687842813 conn(0x7fc2180b2440 msgr2=0x7fc2180b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.698+0000 7fc2202b8700 1 -- 172.21.15.45:0/3687842813 shutdown_connections 2023-08-27T11:27:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.698+0000 7fc2202b8700 1 -- 172.21.15.45:0/3687842813 wait complete. 2023-08-27T11:27:12.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.699+0000 7fc2202b8700 1 Processor -- start 2023-08-27T11:27:12.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.699+0000 7fc2202b8700 1 -- start start 2023-08-27T11:27:12.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.700+0000 7fc2202b8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc218073800 0x7fc218158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.700+0000 7fc2202b8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 0x7fc21815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.700+0000 7fc2202b8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21815b360 0x7fc21815de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.700+0000 7fc2202b8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180afd90 con 0x7fc21815b360 2023-08-27T11:27:12.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.700+0000 7fc2202b8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b0070 con 0x7fc218073800 2023-08-27T11:27:12.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.700+0000 7fc2202b8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b0350 con 0x7fc218073c80 2023-08-27T11:27:12.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.700+0000 7fc21e855700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21815b360 0x7fc21815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21e855700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21815b360 0x7fc21815de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57324/0 (socket says 172.21.15.45:57324) 2023-08-27T11:27:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21e855700 1 -- 172.21.15.45:0/3505263300 learned_addr learned my addr 172.21.15.45:0/3505263300 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21d853700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 0x7fc21815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21e054700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc218073800 0x7fc218158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21e855700 1 -- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 msgr2=0x7fc21815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21e855700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 0x7fc21815d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21e855700 1 -- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc218073800 msgr2=0x7fc218158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21e855700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc218073800 0x7fc218158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21e855700 1 -- 172.21.15.45:0/3505263300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214009500 con 0x7fc21815b360 2023-08-27T11:27:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc21e855700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21815b360 0x7fc21815de30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc210014040 tx=0x7fc210004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc20effd700 1 -- 172.21.15.45:0/3505263300 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210012430 con 0x7fc21815b360 2023-08-27T11:27:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc20effd700 1 -- 172.21.15.45:0/3505263300 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc210020070 con 0x7fc21815b360 2023-08-27T11:27:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc210009a70 con 0x7fc21815b360 2023-08-27T11:27:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc20effd700 1 -- 172.21.15.45:0/3505263300 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210004280 con 0x7fc21815b360 2023-08-27T11:27:12.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.701+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2180b0910 con 0x7fc21815b360 2023-08-27T11:27:12.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.702+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21809d4b0 con 0x7fc21815b360 2023-08-27T11:27:12.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.703+0000 7fc20effd700 1 -- 172.21.15.45:0/3505263300 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 29) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fc2100043e0 con 0x7fc21815b360 2023-08-27T11:27:12.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.703+0000 7fc20effd700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fc1fc06a170 0x7fc1fc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:12.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.703+0000 7fc20effd700 1 -- 172.21.15.45:0/3505263300 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7228+0+0 (secure 0 0 0) 0x7fc210022070 con 0x7fc21815b360 2023-08-27T11:27:12.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.703+0000 7fc21e054700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fc1fc06a170 0x7fc1fc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:12.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.704+0000 7fc21e054700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fc1fc06a170 0x7fc1fc06c630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc2180af2e0 tx=0x7fc214009290 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:12.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.709+0000 7fc20effd700 1 -- 172.21.15.45:0/3505263300 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc210055d10 con 0x7fc21815b360 2023-08-27T11:27:12.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:12.907+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc2180734a0 con 0x7fc21815b360 2023-08-27T11:27:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.034+0000 7fc20effd700 1 -- 172.21.15.45:0/3505263300 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc210059330 con 0x7fc21815b360 2023-08-27T11:27:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fc1fc06a170 msgr2=0x7fc1fc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fc1fc06a170 0x7fc1fc06c630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc2180af2e0 tx=0x7fc214009290 comp rx=0 tx=0).stop 2023-08-27T11:27:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21815b360 msgr2=0x7fc21815de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21815b360 0x7fc21815de30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc210014040 tx=0x7fc210004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 shutdown_connections 2023-08-27T11:27:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fc1fc06a170 0x7fc1fc06c630 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21815b360 0x7fc21815de30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc218073800 0x7fc218158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 --2- 172.21.15.45:0/3505263300 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc218073c80 0x7fc21815d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 >> 172.21.15.45:0/3505263300 conn(0x7fc2180b2440 msgr2=0x7fc2180ca330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 shutdown_connections 2023-08-27T11:27:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.037+0000 7fc2202b8700 1 -- 172.21.15.45:0/3505263300 wait complete. 2023-08-27T11:27:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:27:13.053 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:27:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.180+0000 7f8eddaf3700 1 Processor -- start 2023-08-27T11:27:13.183 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.180+0000 7f8eddaf3700 1 -- start start 2023-08-27T11:27:13.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.181+0000 7f8eddaf3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8ed815b310 0x7f8ed80cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.181+0000 7f8eddaf3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed80cdcb0 0x7f8ed8164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.181+0000 7f8eddaf3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed80cb8b0 con 0x7f8ed815af30 2023-08-27T11:27:13.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.181+0000 7f8eddaf3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed806f9c0 con 0x7f8ed815b310 2023-08-27T11:27:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.181+0000 7f8eddaf3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80ce140 con 0x7f8ed80cdcb0 2023-08-27T11:27:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.181+0000 7f8ed77fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8ed815af30 0x7f8ed80cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44104/0 (socket says 172.21.15.45:44104) 2023-08-27T11:27:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.181+0000 7f8ed77fe700 1 -- 172.21.15.45:0/2626362038 learned_addr learned my addr 172.21.15.45:0/2626362038 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed6ffd700 1 --2- 172.21.15.45:0/2626362038 >> v2:172.21.15.135:3300/0 conn(0x7f8ed815b310 0x7f8ed80cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed7fff700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed80cdcb0 0x7f8ed8164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091506287 0 0) 0x7f8ed80cb8b0 con 0x7f8ed815af30 2023-08-27T11:27:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ec40035b0 con 0x7f8ed815af30 2023-08-27T11:27:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907733996 0 0) 0x7f8ec40035b0 con 0x7f8ed815af30 2023-08-27T11:27:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed80cdcb0 msgr2=0x7f8ed8164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.188 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed80cdcb0 0x7f8ed8164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 >> v2:172.21.15.135:3300/0 conn(0x7f8ed815b310 msgr2=0x7f8ed80cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 --2- 172.21.15.45:0/2626362038 >> v2:172.21.15.135:3300/0 conn(0x7f8ed815b310 0x7f8ed80cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed80cecd0 con 0x7f8ed815af30 2023-08-27T11:27:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f8ec8001670 con 0x7f8ed815af30 2023-08-27T11:27:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 >> v1:172.21.15.45:6789/0 conn(0x7f8ed815af30 legacy=0x7f8ed80cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ec4003a00 0x7f8ec4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ec4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed80cdcb0 0x7f8ec4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80ce140 con 0x7f8ec4004850 2023-08-27T11:27:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec400ba10 con 0x7f8ec4003a00 2023-08-27T11:27:13.191 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec400bb80 con 0x7f8ed80cdcb0 2023-08-27T11:27:13.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed6ffd700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ec4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed77fe700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ec4003a00 0x7f8ec4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed7fff700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed80cdcb0 0x7f8ec4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed6ffd700 1 -- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed80cdcb0 msgr2=0x7f8ec4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.192 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed6ffd700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed80cdcb0 0x7f8ec4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed6ffd700 1 -- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ec4003a00 msgr2=0x7f8ec4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:13.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed6ffd700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ec4003a00 0x7f8ec4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.182+0000 7f8ed6ffd700 1 -- 172.21.15.45:0/2626362038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40035b0 con 0x7f8ec4004850 2023-08-27T11:27:13.193 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8ed6ffd700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ec4004cd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009b60 tx=0x7f8ec4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c826ec1ac6152cf4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec000e6c0 con 0x7f8ec4004850 2023-08-27T11:27:13.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec000e820 con 0x7f8ec4004850 2023-08-27T11:27:13.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8ed67fc700 1 -- 172.21.15.45:0/2626362038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec000eb90 con 0x7f8ec4004850 2023-08-27T11:27:13.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8eddaf3700 1 -- 172.21.15.45:0/2626362038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 msgr2=0x7f8ec4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8eddaf3700 1 --2- 172.21.15.45:0/2626362038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ec4004cd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009b60 tx=0x7f8ec4005880 comp rx=0 tx=0).stop 2023-08-27T11:27:13.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8ed7fff700 1 -- 172.21.15.45:0/2626362038 reap_dead start 2023-08-27T11:27:13.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8eddaf3700 1 -- 172.21.15.45:0/2626362038 shutdown_connections 2023-08-27T11:27:13.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8eddaf3700 1 -- 172.21.15.45:0/2626362038 >> 172.21.15.45:0/2626362038 conn(0x7f8ed809a1a0 msgr2=0x7f8ed80d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:13.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8eddaf3700 1 -- 172.21.15.45:0/2626362038 shutdown_connections 2023-08-27T11:27:13.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.183+0000 7f8eddaf3700 1 -- 172.21.15.45:0/2626362038 wait complete. 2023-08-27T11:27:13.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.185+0000 7f8eddaf3700 1 Processor -- start 2023-08-27T11:27:13.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.185+0000 7f8eddaf3700 1 -- start start 2023-08-27T11:27:13.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8eddaf3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ed80b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8eddaf3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed815e1b0 0x7f8ed806ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8eddaf3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed806f140 0x7f8ed81f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8eddaf3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed815dd90 con 0x7f8ec4004850 2023-08-27T11:27:13.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8eddaf3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed806f5d0 con 0x7f8ed815e1b0 2023-08-27T11:27:13.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8eddaf3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed815d1e0 con 0x7f8ed806f140 2023-08-27T11:27:13.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8ed77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ed80b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8ed77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ed80b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57340/0 (socket says 172.21.15.45:57340) 2023-08-27T11:27:13.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8ed77fe700 1 -- 172.21.15.45:0/3033499881 learned_addr learned my addr 172.21.15.45:0/3033499881 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:13.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8ed7fff700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed806f140 0x7f8ed81f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.186+0000 7f8ed6ffd700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed815e1b0 0x7f8ed806ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ed77fe700 1 -- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed806f140 msgr2=0x7f8ed81f0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ed77fe700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed806f140 0x7f8ed81f0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ed77fe700 1 -- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed815e1b0 msgr2=0x7f8ed806ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:13.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ed77fe700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed815e1b0 0x7f8ed806ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:13.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ed77fe700 1 -- 172.21.15.45:0/3033499881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed81f17d0 con 0x7f8ec4004850 2023-08-27T11:27:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ed6ffd700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed815e1b0 0x7f8ed806ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ed77fe700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ed80b4f70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8007310 tx=0x7f8ec8007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ecffff700 1 -- 172.21.15.45:0/3033499881 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800f6c0 con 0x7f8ec4004850 2023-08-27T11:27:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ecffff700 1 -- 172.21.15.45:0/3033499881 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec800f840 con 0x7f8ec4004850 2023-08-27T11:27:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed815d3f0 con 0x7f8ec4004850 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8ecffff700 1 -- 172.21.15.45:0/3033499881 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800fbf0 con 0x7f8ec4004850 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.187+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed815d9b0 con 0x7f8ec4004850 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.188+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed815db70 con 0x7f8ec4004850 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.188+0000 7f8ecffff700 1 -- 172.21.15.45:0/3033499881 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 29) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f8ec800fd50 con 0x7f8ec4004850 2023-08-27T11:27:13.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.189+0000 7f8ecffff700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8eb806a170 0x7f8eb806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:13.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.189+0000 7f8ecffff700 1 -- 172.21.15.45:0/3033499881 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f8ec80888d0 con 0x7f8ec4004850 2023-08-27T11:27:13.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.189+0000 7f8ed6ffd700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8eb806a170 0x7f8eb806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:13.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.189+0000 7f8ed6ffd700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8eb806a170 0x7f8eb806c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00090b0 tx=0x7f8ec0008030 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:13.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.194+0000 7f8ecffff700 1 -- 172.21.15.45:0/3033499881 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8ec8053cf0 con 0x7f8ec4004850 2023-08-27T11:27:13.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:13.398+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ed815e000 con 0x7f8ec4004850 2023-08-27T11:27:14.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.358+0000 7f8ecffff700 1 -- 172.21.15.45:0/3033499881 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8ec80540c0 con 0x7f8ec4004850 2023-08-27T11:27:14.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.360+0000 7f8ecffff700 1 -- 172.21.15.45:0/3033499881 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8ec8053cf0 con 0x7f8ec4004850 2023-08-27T11:27:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8eb806a170 msgr2=0x7f8eb806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8eb806a170 0x7f8eb806c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00090b0 tx=0x7f8ec0008030 comp rx=0 tx=0).stop 2023-08-27T11:27:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 msgr2=0x7f8ed80b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ed80b4f70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8007310 tx=0x7f8ec8007620 comp rx=0 tx=0).stop 2023-08-27T11:27:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 shutdown_connections 2023-08-27T11:27:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8eb806a170 0x7f8eb806c630 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ec4004850 0x7f8ed80b4f70 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ed806f140 0x7f8ed81f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 --2- 172.21.15.45:0/3033499881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ed815e1b0 0x7f8ed806ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 >> 172.21.15.45:0/3033499881 conn(0x7f8ed809a1a0 msgr2=0x7f8ed80b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 shutdown_connections 2023-08-27T11:27:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.364+0000 7f8eddaf3700 1 -- 172.21.15.45:0/3033499881 wait complete. 2023-08-27T11:27:14.368 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-08-27T11:27:14.380 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:27:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.513+0000 7ff1ad0b5700 1 Processor -- start 2023-08-27T11:27:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.513+0000 7ff1ad0b5700 1 -- start start 2023-08-27T11:27:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.514+0000 7ff1ad0b5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a815d340 0x7ff1a8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.514+0000 7ff1ad0b5700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff1a80cd880 0x7ff1a806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.514+0000 7ff1ad0b5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a806b160 con 0x7ff1a80cdd00 2023-08-27T11:27:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.514+0000 7ff1ad0b5700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a8072010 con 0x7ff1a80cd880 2023-08-27T11:27:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.514+0000 7ff1ad0b5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80af590 con 0x7ff1a815d340 2023-08-27T11:27:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.514+0000 7ff1a759e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff1a80cdd00 0x7ff1a80b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44128/0 (socket says 172.21.15.45:44128) 2023-08-27T11:27:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.514+0000 7ff1a759e700 1 -- 172.21.15.45:0/1782421788 learned_addr learned my addr 172.21.15.45:0/1782421788 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a659c700 1 --2- 172.21.15.45:0/1782421788 >> v2:172.21.15.135:3300/0 conn(0x7ff1a80cd880 0x7ff1a806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a6d9d700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a815d340 0x7ff1a8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083873281 0 0) 0x7ff1a806b160 con 0x7ff1a80cdd00 2023-08-27T11:27:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1940035b0 con 0x7ff1a80cdd00 2023-08-27T11:27:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2254675177 0 0) 0x7ff1940035b0 con 0x7ff1a80cdd00 2023-08-27T11:27:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a815d340 msgr2=0x7ff1a8160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a815d340 0x7ff1a8160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 >> v2:172.21.15.135:3300/0 conn(0x7ff1a80cd880 msgr2=0x7ff1a806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 --2- 172.21.15.45:0/1782421788 >> v2:172.21.15.135:3300/0 conn(0x7ff1a80cd880 0x7ff1a806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a80af2b0 con 0x7ff1a80cdd00 2023-08-27T11:27:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7ff1a00019f0 con 0x7ff1a80cdd00 2023-08-27T11:27:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 >> v1:172.21.15.45:6789/0 conn(0x7ff1a80cdd00 legacy=0x7ff1a80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a815d340 0x7ff194003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff194003f80 0x7ff194004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff194004d90 0x7ff194005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8072010 con 0x7ff194004d90 2023-08-27T11:27:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80af590 con 0x7ff194003f80 2023-08-27T11:27:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19400ba60 con 0x7ff1a815d340 2023-08-27T11:27:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a759e700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff194004d90 0x7ff194005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a6d9d700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a815d340 0x7ff194003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.515+0000 7ff1a659c700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff194003f80 0x7ff194004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a759e700 1 -- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a815d340 msgr2=0x7ff194003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a759e700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a815d340 0x7ff194003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a759e700 1 -- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff194003f80 msgr2=0x7ff194004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a759e700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff194003f80 0x7ff194004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a759e700 1 -- 172.21.15.45:0/1782421788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940035b0 con 0x7ff194004d90 2023-08-27T11:27:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a759e700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff194004d90 0x7ff194005210 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000a8d0 tx=0x7ff194005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986e652c0db12a9e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000a340 con 0x7ff194004d90 2023-08-27T11:27:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a0012600 con 0x7ff194004d90 2023-08-27T11:27:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1782421788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00129b0 con 0x7ff194004d90 2023-08-27T11:27:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1782421788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff194004d90 msgr2=0x7ff194005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1ad0b5700 1 --2- 172.21.15.45:0/1782421788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff194004d90 0x7ff194005210 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000a8d0 tx=0x7ff194005880 comp rx=0 tx=0).stop 2023-08-27T11:27:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1a759e700 1 -- 172.21.15.45:0/1782421788 reap_dead start 2023-08-27T11:27:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1782421788 shutdown_connections 2023-08-27T11:27:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.516+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1782421788 >> 172.21.15.45:0/1782421788 conn(0x7ff1a80cc7c0 msgr2=0x7ff1a80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.517+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1782421788 shutdown_connections 2023-08-27T11:27:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.517+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1782421788 wait complete. 2023-08-27T11:27:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.518+0000 7ff1ad0b5700 1 Processor -- start 2023-08-27T11:27:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.518+0000 7ff1ad0b5700 1 -- start start 2023-08-27T11:27:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1ad0b5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a81f6830 0x7ff1a815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1ad0b5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a81f6c10 0x7ff1a81f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1ad0b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a81f3510 0x7ff1a81f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1ad0b5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81596e0 con 0x7ff1a81f3510 2023-08-27T11:27:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1ad0b5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81f3ed0 con 0x7ff1a81f6830 2023-08-27T11:27:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1ad0b5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81607e0 con 0x7ff1a81f6c10 2023-08-27T11:27:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a81f3510 0x7ff1a81f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a81f3510 0x7ff1a81f3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57364/0 (socket says 172.21.15.45:57364) 2023-08-27T11:27:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a759e700 1 -- 172.21.15.45:0/1880240639 learned_addr learned my addr 172.21.15.45:0/1880240639 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a6d9d700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a81f6830 0x7ff1a815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a659c700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a81f6c10 0x7ff1a81f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a759e700 1 -- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a81f6c10 msgr2=0x7ff1a81f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a759e700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a81f6c10 0x7ff1a81f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a759e700 1 -- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a81f6830 msgr2=0x7ff1a815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a759e700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a81f6830 0x7ff1a815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a759e700 1 -- 172.21.15.45:0/1880240639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a81598f0 con 0x7ff1a81f3510 2023-08-27T11:27:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.519+0000 7ff1a759e700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a81f3510 0x7ff1a81f3990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0012000 tx=0x7ff1a0001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.520+0000 7ff19f7fe700 1 -- 172.21.15.45:0/1880240639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000a340 con 0x7ff1a81f3510 2023-08-27T11:27:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.520+0000 7ff19f7fe700 1 -- 172.21.15.45:0/1880240639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a00019c0 con 0x7ff1a81f3510 2023-08-27T11:27:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.520+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a8159bd0 con 0x7ff1a81f3510 2023-08-27T11:27:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.520+0000 7ff19f7fe700 1 -- 172.21.15.45:0/1880240639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0027410 con 0x7ff1a81f3510 2023-08-27T11:27:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.520+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a815a190 con 0x7ff1a81f3510 2023-08-27T11:27:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.521+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a815aed0 con 0x7ff1a81f3510 2023-08-27T11:27:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.521+0000 7ff19f7fe700 1 -- 172.21.15.45:0/1880240639 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff1a0019070 con 0x7ff1a81f3510 2023-08-27T11:27:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.522+0000 7ff19f7fe700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7ff18806a200 0x7ff18806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.522+0000 7ff19f7fe700 1 -- 172.21.15.45:0/1880240639 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7ff1a0089c10 con 0x7ff1a81f3510 2023-08-27T11:27:14.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.522+0000 7ff1a6d9d700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7ff18806a200 0x7ff18806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.522+0000 7ff1a6d9d700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7ff18806a200 0x7ff18806c6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80c6da0 tx=0x7ff190008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.526+0000 7ff19f7fe700 1 -- 172.21.15.45:0/1880240639 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff1a00550e0 con 0x7ff1a81f3510 2023-08-27T11:27:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.729+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff1a8072690 con 0x7ff1a81f3510 2023-08-27T11:27:14.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.733+0000 7ff19f7fe700 1 -- 172.21.15.45:0/1880240639 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff1a0058700 con 0x7ff1a81f3510 2023-08-27T11:27:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7ff18806a200 msgr2=0x7ff18806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7ff18806a200 0x7ff18806c6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80c6da0 tx=0x7ff190008040 comp rx=0 tx=0).stop 2023-08-27T11:27:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a81f3510 msgr2=0x7ff1a81f3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a81f3510 0x7ff1a81f3990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0012000 tx=0x7ff1a0001360 comp rx=0 tx=0).stop 2023-08-27T11:27:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 shutdown_connections 2023-08-27T11:27:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7ff18806a200 0x7ff18806c6c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a81f3510 0x7ff1a81f3990 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a81f6830 0x7ff1a815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 --2- 172.21.15.45:0/1880240639 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a81f6c10 0x7ff1a81f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 >> 172.21.15.45:0/1880240639 conn(0x7ff1a80cc7c0 msgr2=0x7ff1a815c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:14.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 shutdown_connections 2023-08-27T11:27:14.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.735+0000 7ff1ad0b5700 1 -- 172.21.15.45:0/1880240639 wait complete. 2023-08-27T11:27:14.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-27T11:27:14.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.874+0000 7f9ff1c1e700 1 Processor -- start 2023-08-27T11:27:14.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.874+0000 7f9ff1c1e700 1 -- start start 2023-08-27T11:27:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9ff1c1e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9fec0b17c0 0x7f9fec0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9ff1c1e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec15b460 con 0x7f9fec157be0 2023-08-27T11:27:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9ff1c1e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec0b2280 con 0x7f9fec06ced0 2023-08-27T11:27:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9ff1c1e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fec157fc0 con 0x7f9fec0b17c0 2023-08-27T11:27:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9feaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9fec0b17c0 0x7f9fec0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9feb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9fec157be0 0x7f9fec0716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44144/0 (socket says 172.21.15.45:44144) 2023-08-27T11:27:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9feb7fe700 1 -- 172.21.15.45:0/2962403312 learned_addr learned my addr 172.21.15.45:0/2962403312 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9feaffd700 1 -- 172.21.15.45:0/2962403312 >> v1:172.21.15.135:6789/0 conn(0x7f9fec06ced0 legacy=0x7f9fec0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9feaffd700 1 -- 172.21.15.45:0/2962403312 >> v1:172.21.15.45:6789/0 conn(0x7f9fec157be0 legacy=0x7f9fec0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9feaffd700 1 -- 172.21.15.45:0/2962403312 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec0b2560 con 0x7f9fec0b17c0 2023-08-27T11:27:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.875+0000 7f9fea7fc700 1 -- 172.21.15.45:0/2962403312 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431992847 0 0) 0x7f9fec15b460 con 0x7f9fec157be0 2023-08-27T11:27:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9feaffd700 1 --2- 172.21.15.45:0/2962403312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 0x7f9fec0b1c70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009840 tx=0x7f9fdc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec440c749e0efba9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9fea7fc700 1 -- 172.21.15.45:0/2962403312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011830 con 0x7f9fec0b17c0 2023-08-27T11:27:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9fea7fc700 1 -- 172.21.15.45:0/2962403312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc011990 con 0x7f9fec0b17c0 2023-08-27T11:27:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9fea7fc700 1 -- 172.21.15.45:0/2962403312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011d20 con 0x7f9fec0b17c0 2023-08-27T11:27:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2962403312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 msgr2=0x7f9fec0b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9ff1c1e700 1 --2- 172.21.15.45:0/2962403312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 0x7f9fec0b1c70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009840 tx=0x7f9fdc009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2962403312 shutdown_connections 2023-08-27T11:27:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9ff1c1e700 1 --2- 172.21.15.45:0/2962403312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 0x7f9fec0b1c70 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2962403312 >> 172.21.15.45:0/2962403312 conn(0x7f9fec0cc7c0 msgr2=0x7f9fec0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2962403312 shutdown_connections 2023-08-27T11:27:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.876+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2962403312 wait complete. 2023-08-27T11:27:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.878+0000 7f9ff1c1e700 1 Processor -- start 2023-08-27T11:27:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.878+0000 7f9ff1c1e700 1 -- start start 2023-08-27T11:27:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.878+0000 7f9ff1c1e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fec06ced0 0x7f9fec0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.878+0000 7f9ff1c1e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 0x7f9fec1e6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9ff1c1e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec157be0 0x7f9fec0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9ff1c1e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1f2f00 con 0x7f9fec0b17c0 2023-08-27T11:27:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9ff1c1e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1f31e0 con 0x7f9fec157be0 2023-08-27T11:27:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9ff1c1e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec1f34c0 con 0x7f9fec06ced0 2023-08-27T11:27:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9feaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 0x7f9fec1e6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9feaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 0x7f9fec1e6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57376/0 (socket says 172.21.15.45:57376) 2023-08-27T11:27:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9feaffd700 1 -- 172.21.15.45:0/2533801700 learned_addr learned my addr 172.21.15.45:0/2533801700 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9febfff700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec157be0 0x7f9fec0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9feaffd700 1 -- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fec06ced0 msgr2=0x7f9fec0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9feaffd700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fec06ced0 0x7f9fec0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9feaffd700 1 -- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec157be0 msgr2=0x7f9fec0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9feaffd700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec157be0 0x7f9fec0c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9feaffd700 1 -- 172.21.15.45:0/2533801700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0094a0 con 0x7f9fec0b17c0 2023-08-27T11:27:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9feaffd700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 0x7f9fec1e6d80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc014040 tx=0x7f9fdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9fe3fff700 1 -- 172.21.15.45:0/2533801700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc01a900 con 0x7f9fec0b17c0 2023-08-27T11:27:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9fe3fff700 1 -- 172.21.15.45:0/2533801700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc01aa60 con 0x7f9fec0b17c0 2023-08-27T11:27:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec1f37a0 con 0x7f9fec0b17c0 2023-08-27T11:27:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9fe3fff700 1 -- 172.21.15.45:0/2533801700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc01add0 con 0x7f9fec0b17c0 2023-08-27T11:27:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.879+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec1e72c0 con 0x7f9fec0b17c0 2023-08-27T11:27:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.880+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fec1e74b0 con 0x7f9fec0b17c0 2023-08-27T11:27:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.881+0000 7f9fe3fff700 1 -- 172.21.15.45:0/2533801700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9fdc022070 con 0x7f9fec0b17c0 2023-08-27T11:27:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.881+0000 7f9fe3fff700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fcc06a200 0x7f9fcc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.881+0000 7f9fe3fff700 1 -- 172.21.15.45:0/2533801700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f9fdc089a50 con 0x7f9fec0b17c0 2023-08-27T11:27:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.881+0000 7f9feb7fe700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fcc06a200 0x7f9fcc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.882+0000 7f9feb7fe700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fcc06a200 0x7f9fcc06c6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000c00 tx=0x7f9fd4007990 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:14.886+0000 7f9fe3fff700 1 -- 172.21.15.45:0/2533801700 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9fdc054fa0 con 0x7f9fec0b17c0 2023-08-27T11:27:15.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.085+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f9fec072690 con 0x7f9fec0b17c0 2023-08-27T11:27:15.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.089+0000 7f9fe3fff700 1 -- 172.21.15.45:0/2533801700 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7f9fdc0585c0 con 0x7f9fec0b17c0 2023-08-27T11:27:15.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fcc06a200 msgr2=0x7f9fcc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fcc06a200 0x7f9fcc06c6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000c00 tx=0x7f9fd4007990 comp rx=0 tx=0).stop 2023-08-27T11:27:15.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 msgr2=0x7f9fec1e6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 0x7f9fec1e6d80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc014040 tx=0x7f9fdc004750 comp rx=0 tx=0).stop 2023-08-27T11:27:15.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 shutdown_connections 2023-08-27T11:27:15.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fcc06a200 0x7f9fcc06c6c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec157be0 0x7f9fec0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fec06ced0 0x7f9fec0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 --2- 172.21.15.45:0/2533801700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0b17c0 0x7f9fec1e6d80 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 >> 172.21.15.45:0/2533801700 conn(0x7f9fec0cc7c0 msgr2=0x7f9fec0b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:15.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 shutdown_connections 2023-08-27T11:27:15.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.091+0000 7f9ff1c1e700 1 -- 172.21.15.45:0/2533801700 wait complete. 2023-08-27T11:27:15.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-27T11:27:15.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.236+0000 7f6852c93700 1 Processor -- start 2023-08-27T11:27:15.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.237+0000 7f6852c93700 1 -- start start 2023-08-27T11:27:15.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.237+0000 7f6852c93700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f684c0cd300 0x7f684c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.237+0000 7f6852c93700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f684c06ed90 0x7f684c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.237+0000 7f6852c93700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0cfae0 con 0x7f684c15a0f0 2023-08-27T11:27:15.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.237+0000 7f6852c93700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c15e900 con 0x7f684c06ed90 2023-08-27T11:27:15.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.237+0000 7f6852c93700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c0cae60 con 0x7f684c0cd300 2023-08-27T11:27:15.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.237+0000 7f6851230700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f684c06ed90 0x7f684c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f6851230700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f684c06ed90 0x7f684c15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57386/0 (socket says 172.21.15.45:57386) 2023-08-27T11:27:15.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f6851230700 1 -- 172.21.15.45:0/1616439240 learned_addr learned my addr 172.21.15.45:0/1616439240 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:15.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f6851230700 1 -- 172.21.15.45:0/1616439240 >> v2:172.21.15.135:3300/0 conn(0x7f684c0cd300 msgr2=0x7f684c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f684bfff700 1 --2- 172.21.15.45:0/1616439240 >> v2:172.21.15.135:3300/0 conn(0x7f684c0cd300 0x7f684c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f6851230700 1 --2- 172.21.15.45:0/1616439240 >> v2:172.21.15.135:3300/0 conn(0x7f684c0cd300 0x7f684c0cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f6851230700 1 -- 172.21.15.45:0/1616439240 >> v1:172.21.15.135:6789/0 conn(0x7f684c15a0f0 legacy=0x7f684c15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f6851230700 1 -- 172.21.15.45:0/1616439240 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0cb140 con 0x7f684c06ed90 2023-08-27T11:27:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f6851230700 1 --2- 172.21.15.45:0/1616439240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c06ed90 0x7f684c15e200 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f684c0cf9e0 tx=0x7f684400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd7015cb1dc7e49 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f684b7fe700 1 -- 172.21.15.45:0/1616439240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400e070 con 0x7f684c06ed90 2023-08-27T11:27:15.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f684b7fe700 1 -- 172.21.15.45:0/1616439240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844003a20 con 0x7f684c06ed90 2023-08-27T11:27:15.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.238+0000 7f684b7fe700 1 -- 172.21.15.45:0/1616439240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844003dd0 con 0x7f684c06ed90 2023-08-27T11:27:15.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.239+0000 7f6852c93700 1 -- 172.21.15.45:0/1616439240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c06ed90 msgr2=0x7f684c15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.239+0000 7f6852c93700 1 --2- 172.21.15.45:0/1616439240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c06ed90 0x7f684c15e200 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f684c0cf9e0 tx=0x7f684400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:15.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.239+0000 7f6852c93700 1 -- 172.21.15.45:0/1616439240 shutdown_connections 2023-08-27T11:27:15.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.239+0000 7f6852c93700 1 --2- 172.21.15.45:0/1616439240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c06ed90 0x7f684c15e200 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.239+0000 7f6852c93700 1 --2- 172.21.15.45:0/1616439240 >> v2:172.21.15.135:3300/0 conn(0x7f684c0cd300 0x7f684c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.239+0000 7f6852c93700 1 -- 172.21.15.45:0/1616439240 >> 172.21.15.45:0/1616439240 conn(0x7f684c0cc7e0 msgr2=0x7f684c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.239+0000 7f6852c93700 1 -- 172.21.15.45:0/1616439240 shutdown_connections 2023-08-27T11:27:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.239+0000 7f6852c93700 1 -- 172.21.15.45:0/1616439240 wait complete. 2023-08-27T11:27:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.240+0000 7f6852c93700 1 Processor -- start 2023-08-27T11:27:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.240+0000 7f6852c93700 1 -- start start 2023-08-27T11:27:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6852c93700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f684c06ed90 0x7f684c1f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6852c93700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f684c0cd300 0x7f684c1f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6852c93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c15a0f0 0x7f684c1eff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6852c93700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1f0460 con 0x7f684c15a0f0 2023-08-27T11:27:15.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6852c93700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0ac950 con 0x7f684c06ed90 2023-08-27T11:27:15.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6852c93700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1f2fe0 con 0x7f684c0cd300 2023-08-27T11:27:15.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6851230700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c15a0f0 0x7f684c1eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6851230700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c15a0f0 0x7f684c1eff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57390/0 (socket says 172.21.15.45:57390) 2023-08-27T11:27:15.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6851230700 1 -- 172.21.15.45:0/2084923233 learned_addr learned my addr 172.21.15.45:0/2084923233 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:15.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6850a2f700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f684c06ed90 0x7f684c1f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f684bfff700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f684c0cd300 0x7f684c1f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6851230700 1 -- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f684c0cd300 msgr2=0x7f684c1f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6851230700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f684c0cd300 0x7f684c1f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6851230700 1 -- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f684c06ed90 msgr2=0x7f684c1f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6851230700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f684c06ed90 0x7f684c1f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.241+0000 7f6851230700 1 -- 172.21.15.45:0/2084923233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6844009b30 con 0x7f684c15a0f0 2023-08-27T11:27:15.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.242+0000 7f6851230700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c15a0f0 0x7f684c1eff20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f684c074300 tx=0x7f6844004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.242+0000 7f68497fa700 1 -- 172.21.15.45:0/2084923233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844004440 con 0x7f684c15a0f0 2023-08-27T11:27:15.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.242+0000 7f68497fa700 1 -- 172.21.15.45:0/2084923233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68440092e0 con 0x7f684c15a0f0 2023-08-27T11:27:15.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.242+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c1f32c0 con 0x7f684c15a0f0 2023-08-27T11:27:15.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.242+0000 7f68497fa700 1 -- 172.21.15.45:0/2084923233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68440047e0 con 0x7f684c15a0f0 2023-08-27T11:27:15.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.242+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c1f3880 con 0x7f684c15a0f0 2023-08-27T11:27:15.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.243+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c0acb20 con 0x7f684c15a0f0 2023-08-27T11:27:15.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.243+0000 7f68497fa700 1 -- 172.21.15.45:0/2084923233 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f684401c440 con 0x7f684c15a0f0 2023-08-27T11:27:15.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.244+0000 7f68497fa700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f683006a200 0x7f683006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.244+0000 7f68497fa700 1 -- 172.21.15.45:0/2084923233 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f6844089b50 con 0x7f684c15a0f0 2023-08-27T11:27:15.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.244+0000 7f6850a2f700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f683006a200 0x7f683006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.244+0000 7f6850a2f700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f683006a200 0x7f683006c6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6840009960 tx=0x7f68400090e0 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.248+0000 7f68497fa700 1 -- 172.21.15.45:0/2084923233 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f684c0acb20 con 0x7f684c15a0f0 2023-08-27T11:27:15.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.452+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f684c0acb20 con 0x7f684c15a0f0 2023-08-27T11:27:15.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.456+0000 7f68497fa700 1 -- 172.21.15.45:0/2084923233 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f684c0acb20 con 0x7f684c15a0f0 2023-08-27T11:27:15.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f683006a200 msgr2=0x7f683006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f683006a200 0x7f683006c6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6840009960 tx=0x7f68400090e0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c15a0f0 msgr2=0x7f684c1eff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c15a0f0 0x7f684c1eff20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f684c074300 tx=0x7f6844004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 shutdown_connections 2023-08-27T11:27:15.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f683006a200 0x7f683006c6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f684c15a0f0 0x7f684c1eff20 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f684c06ed90 0x7f684c1f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 --2- 172.21.15.45:0/2084923233 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f684c0cd300 0x7f684c1f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 >> 172.21.15.45:0/2084923233 conn(0x7f684c0cc7e0 msgr2=0x7f684c159500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:15.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 shutdown_connections 2023-08-27T11:27:15.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.458+0000 7f6852c93700 1 -- 172.21.15.45:0/2084923233 wait complete. 2023-08-27T11:27:15.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-08-27T11:27:15.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.595+0000 7f958d6d3700 1 Processor -- start 2023-08-27T11:27:15.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.596+0000 7f958d6d3700 1 -- start start 2023-08-27T11:27:15.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.596+0000 7f958d6d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 0x7f9588070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.596+0000 7f958d6d3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9588161130 0x7f95880cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.596+0000 7f958d6d3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95880d0bf0 con 0x7f95880cd770 2023-08-27T11:27:15.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.596+0000 7f958d6d3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95881615b0 con 0x7f9588161130 2023-08-27T11:27:15.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.596+0000 7f958d6d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9588162040 con 0x7f958815d7d0 2023-08-27T11:27:15.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95877fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9588161130 0x7f95880cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95877fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9588161130 0x7f95880cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:46256/0 (socket says 172.21.15.45:46256) 2023-08-27T11:27:15.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95877fe700 1 -- 172.21.15.45:0/1466833105 learned_addr learned my addr 172.21.15.45:0/1466833105 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:15.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95867fc700 1 --2- 172.21.15.45:0/1466833105 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 0x7f9588070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95867fc700 1 -- 172.21.15.45:0/1466833105 >> v2:172.21.15.135:3300/0 conn(0x7f9588161130 msgr2=0x7f95880cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:15.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95867fc700 1 --2- 172.21.15.45:0/1466833105 >> v2:172.21.15.135:3300/0 conn(0x7f9588161130 0x7f95880cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95867fc700 1 -- 172.21.15.45:0/1466833105 >> v1:172.21.15.135:6789/0 conn(0x7f95880cd770 legacy=0x7f95880cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95867fc700 1 -- 172.21.15.45:0/1466833105 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95881617a0 con 0x7f958815d7d0 2023-08-27T11:27:15.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95877fe700 1 --2- 172.21.15.45:0/1466833105 >> v2:172.21.15.135:3300/0 conn(0x7f9588161130 0x7f95880cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:15.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.597+0000 7f95867fc700 1 --2- 172.21.15.45:0/1466833105 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 0x7f9588070480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f957c009840 tx=0x7f957c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3de9c4668540592 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f9585ffb700 1 -- 172.21.15.45:0/1466833105 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c011830 con 0x7f958815d7d0 2023-08-27T11:27:15.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f9585ffb700 1 -- 172.21.15.45:0/1466833105 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c011990 con 0x7f958815d7d0 2023-08-27T11:27:15.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f9585ffb700 1 -- 172.21.15.45:0/1466833105 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c011d20 con 0x7f958815d7d0 2023-08-27T11:27:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f958d6d3700 1 -- 172.21.15.45:0/1466833105 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 msgr2=0x7f9588070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f958d6d3700 1 --2- 172.21.15.45:0/1466833105 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 0x7f9588070480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f957c009840 tx=0x7f957c009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f958d6d3700 1 -- 172.21.15.45:0/1466833105 shutdown_connections 2023-08-27T11:27:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f958d6d3700 1 --2- 172.21.15.45:0/1466833105 >> v2:172.21.15.135:3300/0 conn(0x7f9588161130 0x7f95880cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f958d6d3700 1 --2- 172.21.15.45:0/1466833105 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 0x7f9588070480 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f958d6d3700 1 -- 172.21.15.45:0/1466833105 >> 172.21.15.45:0/1466833105 conn(0x7f958806f960 msgr2=0x7f958806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:15.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f958d6d3700 1 -- 172.21.15.45:0/1466833105 shutdown_connections 2023-08-27T11:27:15.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.598+0000 7f958d6d3700 1 -- 172.21.15.45:0/1466833105 wait complete. 2023-08-27T11:27:15.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.599+0000 7f958d6d3700 1 Processor -- start 2023-08-27T11:27:15.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f958d6d3700 1 -- start start 2023-08-27T11:27:15.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f958d6d3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95880cd770 0x7f95880af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f958d6d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 0x7f95880c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f958d6d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588161130 0x7f958806c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f958d6d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95881f6a70 con 0x7f9588161130 2023-08-27T11:27:15.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f958d6d3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95881f6160 con 0x7f95880cd770 2023-08-27T11:27:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f958d6d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95881f2a20 con 0x7f958815d7d0 2023-08-27T11:27:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f95877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588161130 0x7f958806c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f95877fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588161130 0x7f958806c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57406/0 (socket says 172.21.15.45:57406) 2023-08-27T11:27:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.600+0000 7f95877fe700 1 -- 172.21.15.45:0/759807173 learned_addr learned my addr 172.21.15.45:0/759807173 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:15.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f9586ffd700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95880cd770 0x7f95880af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f95867fc700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 0x7f95880c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f95877fe700 1 -- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 msgr2=0x7f95880c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f95877fe700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 0x7f95880c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f95877fe700 1 -- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95880cd770 msgr2=0x7f95880af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:15.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f95877fe700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95880cd770 0x7f95880af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:15.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f95877fe700 1 -- 172.21.15.45:0/759807173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580009b30 con 0x7f9588161130 2023-08-27T11:27:15.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f95877fe700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588161130 0x7f958806c4d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f958000efc0 tx=0x7f958000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f956f7fe700 1 -- 172.21.15.45:0/759807173 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958000cb30 con 0x7f9588161130 2023-08-27T11:27:15.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f956f7fe700 1 -- 172.21.15.45:0/759807173 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958000f040 con 0x7f9588161130 2023-08-27T11:27:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f956f7fe700 1 -- 172.21.15.45:0/759807173 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580014480 con 0x7f9588161130 2023-08-27T11:27:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c0094a0 con 0x7f9588161130 2023-08-27T11:27:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.601+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95880c6fc0 con 0x7f9588161130 2023-08-27T11:27:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.602+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958804e620 con 0x7f9588161130 2023-08-27T11:27:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.602+0000 7f956f7fe700 1 -- 172.21.15.45:0/759807173 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f95800145e0 con 0x7f9588161130 2023-08-27T11:27:15.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.603+0000 7f956f7fe700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f957406a200 0x7f957406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:15.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.603+0000 7f956f7fe700 1 -- 172.21.15.45:0/759807173 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f9580088fa0 con 0x7f9588161130 2023-08-27T11:27:15.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.603+0000 7f9586ffd700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f957406a200 0x7f957406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:15.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.604+0000 7f9586ffd700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f957406a200 0x7f957406c6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9578000f60 tx=0x7f9578009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:15.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.607+0000 7f956f7fe700 1 -- 172.21.15.45:0/759807173 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f95800bb990 con 0x7f9588161130 2023-08-27T11:27:15.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:15.892+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f95880ce780 con 0x7f9588161130 2023-08-27T11:27:16.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.356+0000 7f956f7fe700 1 -- 172.21.15.45:0/759807173 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v561) v1 ==== 154+0+0 (secure 0 0 0) 0x7f95800543e0 con 0x7f9588161130 2023-08-27T11:27:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.358+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f957406a200 msgr2=0x7f957406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.358+0000 7f958d6d3700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f957406a200 0x7f957406c6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9578000f60 tx=0x7f9578009040 comp rx=0 tx=0).stop 2023-08-27T11:27:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.358+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588161130 msgr2=0x7f958806c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.358+0000 7f958d6d3700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588161130 0x7f958806c4d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f958000efc0 tx=0x7f958000c3a0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.359+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 shutdown_connections 2023-08-27T11:27:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.359+0000 7f958d6d3700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f957406a200 0x7f957406c6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.359+0000 7f958d6d3700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588161130 0x7f958806c4d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.359+0000 7f958d6d3700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95880cd770 0x7f95880af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.359+0000 7f958d6d3700 1 --2- 172.21.15.45:0/759807173 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f958815d7d0 0x7f95880c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.359+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 >> 172.21.15.45:0/759807173 conn(0x7f958806f960 msgr2=0x7f958815f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.359+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 shutdown_connections 2023-08-27T11:27:16.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.359+0000 7f958d6d3700 1 -- 172.21.15.45:0/759807173 wait complete. 2023-08-27T11:27:16.374 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-08-27T11:27:16.374 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-08-27T11:27:16.374 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-27T11:27:16.375 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-08-27T11:27:16.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.503+0000 7fd523852700 1 Processor -- start 2023-08-27T11:27:16.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.504+0000 7fd523852700 1 -- start start 2023-08-27T11:27:16.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.504+0000 7fd523852700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.504+0000 7fd523852700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd51c15d830 0x7fd51c0d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.504+0000 7fd523852700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c0ce1c0 con 0x7fd51c15d450 2023-08-27T11:27:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.504+0000 7fd523852700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c0cf290 con 0x7fd51c15d830 2023-08-27T11:27:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.504+0000 7fd523852700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0ce440 con 0x7fd51c0b2140 2023-08-27T11:27:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd520ded700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd521def700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd51c15d830 0x7fd51c0d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd520ded700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c0cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:55322/0 (socket says 172.21.15.45:55322) 2023-08-27T11:27:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd521def700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd51c15d830 0x7fd51c0d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:46270/0 (socket says 172.21.15.45:46270) 2023-08-27T11:27:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd5215ee700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd51c15d450 0x7fd51c0cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:39850/0 (socket says 172.21.15.45:39850) 2023-08-27T11:27:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd520ded700 1 -- 172.21.15.45:0/2634466976 learned_addr learned my addr 172.21.15.45:0/2634466976 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd520ded700 1 -- 172.21.15.45:0/2634466976 >> v2:172.21.15.135:3300/0 conn(0x7fd51c15d830 msgr2=0x7fd51c0d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd520ded700 1 --2- 172.21.15.45:0/2634466976 >> v2:172.21.15.135:3300/0 conn(0x7fd51c15d830 0x7fd51c0d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd50ffff700 1 -- 172.21.15.45:0/2634466976 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207876591 0 0) 0x7fd51c0ce1c0 con 0x7fd51c15d450 2023-08-27T11:27:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd520ded700 1 -- 172.21.15.45:0/2634466976 >> v1:172.21.15.135:6789/0 conn(0x7fd51c15d450 legacy=0x7fd51c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd520ded700 1 -- 172.21.15.45:0/2634466976 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0ce720 con 0x7fd51c0b2140 2023-08-27T11:27:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.505+0000 7fd520ded700 1 --2- 172.21.15.45:0/2634466976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c0cfd30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd5100098f0 tx=0x7fd510009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=305f2db8afe955bf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd50ffff700 1 -- 172.21.15.45:0/2634466976 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510011830 con 0x7fd51c0b2140 2023-08-27T11:27:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd50ffff700 1 -- 172.21.15.45:0/2634466976 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510011990 con 0x7fd51c0b2140 2023-08-27T11:27:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd50ffff700 1 -- 172.21.15.45:0/2634466976 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510011d20 con 0x7fd51c0b2140 2023-08-27T11:27:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd523852700 1 -- 172.21.15.45:0/2634466976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 msgr2=0x7fd51c0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd523852700 1 --2- 172.21.15.45:0/2634466976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c0cfd30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd5100098f0 tx=0x7fd510009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd523852700 1 -- 172.21.15.45:0/2634466976 shutdown_connections 2023-08-27T11:27:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd523852700 1 --2- 172.21.15.45:0/2634466976 >> v2:172.21.15.135:3300/0 conn(0x7fd51c15d830 0x7fd51c0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd523852700 1 --2- 172.21.15.45:0/2634466976 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c0cfd30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd523852700 1 -- 172.21.15.45:0/2634466976 >> 172.21.15.45:0/2634466976 conn(0x7fd51c0d0f60 msgr2=0x7fd51c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:16.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd523852700 1 -- 172.21.15.45:0/2634466976 shutdown_connections 2023-08-27T11:27:16.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.506+0000 7fd523852700 1 -- 172.21.15.45:0/2634466976 wait complete. 2023-08-27T11:27:16.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.507+0000 7fd523852700 1 Processor -- start 2023-08-27T11:27:16.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd523852700 1 -- start start 2023-08-27T11:27:16.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd523852700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd523852700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd51c15d450 0x7fd51c1580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd523852700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd51c15d830 0x7fd51c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd523852700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c157030 con 0x7fd51c15d830 2023-08-27T11:27:16.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd523852700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c157310 con 0x7fd51c15d450 2023-08-27T11:27:16.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd523852700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c1575f0 con 0x7fd51c0b2140 2023-08-27T11:27:16.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd521def700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd51c15d830 0x7fd51c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd521def700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd51c15d830 0x7fd51c1589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57420/0 (socket says 172.21.15.45:57420) 2023-08-27T11:27:16.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd521def700 1 -- 172.21.15.45:0/3605674311 learned_addr learned my addr 172.21.15.45:0/3605674311 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:16.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.508+0000 7fd5215ee700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd520ded700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd51c15d450 0x7fd51c1580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd521def700 1 -- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 msgr2=0x7fd51c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd521def700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd521def700 1 -- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd51c15d450 msgr2=0x7fd51c1580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:16.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd521def700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd51c15d450 0x7fd51c1580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:16.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd521def700 1 -- 172.21.15.45:0/3605674311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510009550 con 0x7fd51c15d830 2023-08-27T11:27:16.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd521def700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd51c15d830 0x7fd51c1589a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd51400efc0 tx=0x7fd51400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:16.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd50dffb700 1 -- 172.21.15.45:0/3605674311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400cad0 con 0x7fd51c15d830 2023-08-27T11:27:16.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd50dffb700 1 -- 172.21.15.45:0/3605674311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51400f040 con 0x7fd51c15d830 2023-08-27T11:27:16.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c1578d0 con 0x7fd51c15d830 2023-08-27T11:27:16.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd50dffb700 1 -- 172.21.15.45:0/3605674311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400ce70 con 0x7fd51c15d830 2023-08-27T11:27:16.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.509+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c158610 con 0x7fd51c15d830 2023-08-27T11:27:16.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.510+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c157df0 con 0x7fd51c15d830 2023-08-27T11:27:16.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.511+0000 7fd50dffb700 1 -- 172.21.15.45:0/3605674311 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd514014480 con 0x7fd51c15d830 2023-08-27T11:27:16.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.511+0000 7fd50dffb700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd50006a1d0 0x7fd50006c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:16.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.511+0000 7fd50dffb700 1 -- 172.21.15.45:0/3605674311 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fd514089070 con 0x7fd51c15d830 2023-08-27T11:27:16.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.511+0000 7fd5215ee700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd50006a1d0 0x7fd50006c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:16.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.512+0000 7fd5215ee700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd50006a1d0 0x7fd50006c690 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd518007fd0 tx=0x7fd518007de0 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:16.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.516+0000 7fd50dffb700 1 -- 172.21.15.45:0/3605674311 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd51c157df0 con 0x7fd51c15d830 2023-08-27T11:27:16.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:16.803+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fd51c157df0 con 0x7fd51c15d830 2023-08-27T11:27:17.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.360+0000 7fd50dffb700 1 -- 172.21.15.45:0/3605674311 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v562) v1 ==== 159+0+0 (secure 0 0 0) 0x7fd51c157df0 con 0x7fd51c15d830 2023-08-27T11:27:17.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd50006a1d0 msgr2=0x7fd50006c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd50006a1d0 0x7fd50006c690 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd518007fd0 tx=0x7fd518007de0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd51c15d830 msgr2=0x7fd51c1589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd51c15d830 0x7fd51c1589a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd51400efc0 tx=0x7fd51400c0c0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 shutdown_connections 2023-08-27T11:27:17.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd50006a1d0 0x7fd50006c690 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd51c15d830 0x7fd51c1589a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd51c0b2140 0x7fd51c156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 --2- 172.21.15.45:0/3605674311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd51c15d450 0x7fd51c1580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 >> 172.21.15.45:0/3605674311 conn(0x7fd51c0d0f60 msgr2=0x7fd51c0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:17.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 shutdown_connections 2023-08-27T11:27:17.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.363+0000 7fd523852700 1 -- 172.21.15.45:0/3605674311 wait complete. 2023-08-27T11:27:17.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.505+0000 7fadf23f4700 1 Processor -- start 2023-08-27T11:27:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.506+0000 7fadf23f4700 1 -- start start 2023-08-27T11:27:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadf23f4700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fadec0cc390 0x7fadec164f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadf23f4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec0caef0 con 0x7fadec070e40 2023-08-27T11:27:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadf23f4700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec0ce000 con 0x7fadec15dbc0 2023-08-27T11:27:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadf23f4700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec0cac50 con 0x7fadec0cc390 2023-08-27T11:27:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadebfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fadec070e40 0x7fadec071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44200/0 (socket says 172.21.15.45:44200) 2023-08-27T11:27:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadebfff700 1 -- 172.21.15.45:0/1401784242 learned_addr learned my addr 172.21.15.45:0/1401784242 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadf0991700 1 --2- 172.21.15.45:0/1401784242 >> v2:172.21.15.135:3300/0 conn(0x7fadec0cc390 0x7fadec164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741232832 0 0) 0x7fadec0caef0 con 0x7fadec070e40 2023-08-27T11:27:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadd40035b0 con 0x7fadec070e40 2023-08-27T11:27:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813736831 0 0) 0x7fadd40035b0 con 0x7fadec070e40 2023-08-27T11:27:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 >> v2:172.21.15.135:3300/0 conn(0x7fadec0cc390 msgr2=0x7fadec164f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 --2- 172.21.15.45:0/1401784242 >> v2:172.21.15.135:3300/0 conn(0x7fadec0cc390 0x7fadec164f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 >> v1:172.21.15.135:6789/0 conn(0x7fadec15dbc0 legacy=0x7fadec15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec15b5c0 con 0x7fadec070e40 2023-08-27T11:27:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7faddc001670 con 0x7fadec070e40 2023-08-27T11:27:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 >> v1:172.21.15.45:6789/0 conn(0x7fadec070e40 legacy=0x7fadec071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadd4003a00 0x7fadd4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fadd4004850 0x7fadd4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadd4005210 0x7fadd400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.507+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0cac50 con 0x7fadd4003a00 2023-08-27T11:27:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd400c0b0 con 0x7fadd4005210 2023-08-27T11:27:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd400c270 con 0x7fadd4004850 2023-08-27T11:27:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadebfff700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadd4003a00 0x7fadd4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadf0991700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadd4005210 0x7fadd400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadeb7fe700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fadd4004850 0x7fadd4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadebfff700 1 -- 172.21.15.45:0/1401784242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fadd4004850 msgr2=0x7fadd4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadebfff700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fadd4004850 0x7fadd4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadebfff700 1 -- 172.21.15.45:0/1401784242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadd4005210 msgr2=0x7fadd400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadebfff700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadd4005210 0x7fadd400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadebfff700 1 -- 172.21.15.45:0/1401784242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40035b0 con 0x7fadd4003a00 2023-08-27T11:27:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadf0991700 1 -- 172.21.15.45:0/1401784242 reap_dead start 2023-08-27T11:27:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadebfff700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadd4003a00 0x7fadd4003e20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c240 tx=0x7fadd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cd3a9954518cf0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc007c10 con 0x7fadd4003a00 2023-08-27T11:27:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc00cd80 con 0x7fadd4003a00 2023-08-27T11:27:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.508+0000 7fadeaffd700 1 -- 172.21.15.45:0/1401784242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc005ba0 con 0x7fadd4003a00 2023-08-27T11:27:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.509+0000 7fadf23f4700 1 -- 172.21.15.45:0/1401784242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadd4003a00 msgr2=0x7fadd4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.509+0000 7fadf23f4700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadd4003a00 0x7fadd4003e20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c240 tx=0x7fadd4005880 comp rx=0 tx=0).stop 2023-08-27T11:27:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.509+0000 7fadf23f4700 1 -- 172.21.15.45:0/1401784242 shutdown_connections 2023-08-27T11:27:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.509+0000 7fadf23f4700 1 --2- 172.21.15.45:0/1401784242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadd4003a00 0x7fadd4003e20 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.509+0000 7fadf23f4700 1 -- 172.21.15.45:0/1401784242 >> 172.21.15.45:0/1401784242 conn(0x7fadec0cfd60 msgr2=0x7fadec0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.509+0000 7fadf23f4700 1 -- 172.21.15.45:0/1401784242 shutdown_connections 2023-08-27T11:27:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.509+0000 7fadf23f4700 1 -- 172.21.15.45:0/1401784242 wait complete. 2023-08-27T11:27:17.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.510+0000 7fadf23f4700 1 Processor -- start 2023-08-27T11:27:17.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.510+0000 7fadf23f4700 1 -- start start 2023-08-27T11:27:17.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadf23f4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fadd4003a00 0x7fadec14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadf23f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadec1581e0 0x7fadec158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadf23f4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadec0c8770 0x7fadec0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadf23f4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec158ba0 con 0x7fadec1581e0 2023-08-27T11:27:17.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadf23f4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec14d710 con 0x7fadec0c8770 2023-08-27T11:27:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadf23f4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1f6590 con 0x7fadd4003a00 2023-08-27T11:27:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadeb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadec1581e0 0x7fadec158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadeb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadec1581e0 0x7fadec158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57438/0 (socket says 172.21.15.45:57438) 2023-08-27T11:27:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadeb7fe700 1 -- 172.21.15.45:0/2853750254 learned_addr learned my addr 172.21.15.45:0/2853750254 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadebfff700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fadd4003a00 0x7fadec14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadf0991700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadec0c8770 0x7fadec0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadeb7fe700 1 -- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fadd4003a00 msgr2=0x7fadec14d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadeb7fe700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fadd4003a00 0x7fadec14d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadeb7fe700 1 -- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadec0c8770 msgr2=0x7fadec0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadeb7fe700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadec0c8770 0x7fadec0c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.511+0000 7fadeb7fe700 1 -- 172.21.15.45:0/2853750254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec1f6870 con 0x7fadec1581e0 2023-08-27T11:27:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.512+0000 7fadeb7fe700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadec1581e0 0x7fadec158660 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fade000f820 tx=0x7fade000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.512+0000 7fade8ff9700 1 -- 172.21.15.45:0/2853750254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade00052e0 con 0x7fadec1581e0 2023-08-27T11:27:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.512+0000 7fade8ff9700 1 -- 172.21.15.45:0/2853750254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade0005440 con 0x7fadec1581e0 2023-08-27T11:27:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.512+0000 7fade8ff9700 1 -- 172.21.15.45:0/2853750254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade00038a0 con 0x7fadec1581e0 2023-08-27T11:27:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.512+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec1f6b50 con 0x7fadec1581e0 2023-08-27T11:27:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.512+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec1f7110 con 0x7fadec1581e0 2023-08-27T11:27:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.513+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec072180 con 0x7fadec1581e0 2023-08-27T11:27:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.513+0000 7fade8ff9700 1 -- 172.21.15.45:0/2853750254 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fade0003a00 con 0x7fadec1581e0 2023-08-27T11:27:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.514+0000 7fade8ff9700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fadd806a230 0x7fadd806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.514+0000 7fade8ff9700 1 -- 172.21.15.45:0/2853750254 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fade00883a0 con 0x7fadec1581e0 2023-08-27T11:27:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.514+0000 7fadebfff700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fadd806a230 0x7fadd806c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.514+0000 7fadebfff700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fadd806a230 0x7fadd806c6f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faddc000c00 tx=0x7faddc007000 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.518+0000 7fade8ff9700 1 -- 172.21.15.45:0/2853750254 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fade00538f0 con 0x7fadec1581e0 2023-08-27T11:27:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.717+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadec072780 con 0x7fadec1581e0 2023-08-27T11:27:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.718+0000 7fade8ff9700 1 -- 172.21.15.45:0/2853750254 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16072 (secure 0 0 0) 0x7fade0003cb0 con 0x7fadec1581e0 2023-08-27T11:27:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:17.720 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":77,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:14.347726+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2659160949":"2023-08-28T11:27:11.498263+0000","172.21.15.45:6833/2659160949":"2023-08-28T11:27:11.498263+0000","172.21.15.135:0/1159058976":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/1568057874":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/3797775120":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/143531536":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/2756325788":"2023-08-28T11:27:09.329619+0000","172.21.15.135:6833/1664443653":"2023-08-28T11:27:09.329619+0000","172.21.15.135:6832/1664443653":"2023-08-28T11:27:09.329619+0000","172.21.15.135:0/3859433979":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/485008861":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/3010439837":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/2511615876":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1537214801":"2023-08-28T11:25:29.254479+0000","172.21.15.135:6832/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/726344780":"2023-08-28T11:27:09.329619+0000","172.21.15.135:6833/3181978685":"2023-08-28T11:25:29.254479+0000","172.21.15.135:0/1231508777":"2023-08-28T11:25:29.254479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fadd806a230 msgr2=0x7fadd806c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fadd806a230 0x7fadd806c6f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faddc000c00 tx=0x7faddc007000 comp rx=0 tx=0).stop 2023-08-27T11:27:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadec1581e0 msgr2=0x7fadec158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadec1581e0 0x7fadec158660 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fade000f820 tx=0x7fade000fb30 comp rx=0 tx=0).stop 2023-08-27T11:27:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 shutdown_connections 2023-08-27T11:27:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fadd806a230 0x7fadd806c6f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadec0c8770 0x7fadec0c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fadd4003a00 0x7fadec14d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 --2- 172.21.15.45:0/2853750254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fadec1581e0 0x7fadec158660 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 >> 172.21.15.45:0/2853750254 conn(0x7fadec0cfd60 msgr2=0x7fadec06e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 shutdown_connections 2023-08-27T11:27:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.720+0000 7fadf23f4700 1 -- 172.21.15.45:0/2853750254 wait complete. 2023-08-27T11:27:17.734 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:17.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:17.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.862+0000 7fd1f03a2700 1 Processor -- start 2023-08-27T11:27:17.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.862+0000 7fd1f03a2700 1 -- start start 2023-08-27T11:27:17.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1f03a2700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd1e815d340 0x7fd1e8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1f03a2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd1e80cd880 0x7fd1e806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1f03a2700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e806b160 con 0x7fd1e80cdd00 2023-08-27T11:27:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1f03a2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e8072010 con 0x7fd1e80cd880 2023-08-27T11:27:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1f03a2700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e80af590 con 0x7fd1e815d340 2023-08-27T11:27:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed93d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd1e80cd880 0x7fd1e806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed93d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd1e80cd880 0x7fd1e806a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57446/0 (socket says 172.21.15.45:57446) 2023-08-27T11:27:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed93d700 1 -- 172.21.15.45:0/2746321502 learned_addr learned my addr 172.21.15.45:0/2746321502 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ee13e700 1 --2- 172.21.15.45:0/2746321502 >> v2:172.21.15.135:3300/0 conn(0x7fd1e815d340 0x7fd1e8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed93d700 1 -- 172.21.15.45:0/2746321502 >> v2:172.21.15.135:3300/0 conn(0x7fd1e815d340 msgr2=0x7fd1e8160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed93d700 1 --2- 172.21.15.45:0/2746321502 >> v2:172.21.15.135:3300/0 conn(0x7fd1e815d340 0x7fd1e8160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed93d700 1 -- 172.21.15.45:0/2746321502 >> v1:172.21.15.135:6789/0 conn(0x7fd1e80cdd00 legacy=0x7fd1e80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed93d700 1 -- 172.21.15.45:0/2746321502 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80af2b0 con 0x7fd1e80cd880 2023-08-27T11:27:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed93d700 1 --2- 172.21.15.45:0/2746321502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cd880 0x7fd1e806a280 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80732d0 tx=0x7fd1d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5691eb4bb1d4d30 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed13c700 1 -- 172.21.15.45:0/2746321502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d800b870 con 0x7fd1e80cd880 2023-08-27T11:27:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.863+0000 7fd1ed13c700 1 -- 172.21.15.45:0/2746321502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d800b9d0 con 0x7fd1e80cd880 2023-08-27T11:27:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.864+0000 7fd1ed13c700 1 -- 172.21.15.45:0/2746321502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d800bd60 con 0x7fd1e80cd880 2023-08-27T11:27:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.864+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2746321502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cd880 msgr2=0x7fd1e806a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.864+0000 7fd1f03a2700 1 --2- 172.21.15.45:0/2746321502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cd880 0x7fd1e806a280 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80732d0 tx=0x7fd1d8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.864+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2746321502 shutdown_connections 2023-08-27T11:27:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.864+0000 7fd1f03a2700 1 --2- 172.21.15.45:0/2746321502 >> v2:172.21.15.135:3300/0 conn(0x7fd1e815d340 0x7fd1e8160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.864+0000 7fd1f03a2700 1 --2- 172.21.15.45:0/2746321502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cd880 0x7fd1e806a280 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.864+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2746321502 >> 172.21.15.45:0/2746321502 conn(0x7fd1e80cc7c0 msgr2=0x7fd1e80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.864+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2746321502 shutdown_connections 2023-08-27T11:27:17.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.864+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2746321502 wait complete. 2023-08-27T11:27:17.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.865+0000 7fd1f03a2700 1 Processor -- start 2023-08-27T11:27:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.865+0000 7fd1f03a2700 1 -- start start 2023-08-27T11:27:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1f03a2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd1e80cd880 0x7fd1e80b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1f03a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cdd00 0x7fd1e814c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1f03a2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1e815d340 0x7fd1e806ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1f03a2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80b5b80 con 0x7fd1e80cdd00 2023-08-27T11:27:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1f03a2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e814ce20 con 0x7fd1e815d340 2023-08-27T11:27:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1f03a2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e814be90 con 0x7fd1e80cd880 2023-08-27T11:27:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1ed93d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cdd00 0x7fd1e814c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1ed93d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cdd00 0x7fd1e814c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57454/0 (socket says 172.21.15.45:57454) 2023-08-27T11:27:17.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1ed93d700 1 -- 172.21.15.45:0/2224315808 learned_addr learned my addr 172.21.15.45:0/2224315808 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1ee13e700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd1e80cd880 0x7fd1e80b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1ee93f700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1e815d340 0x7fd1e806ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1ed93d700 1 -- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd1e80cd880 msgr2=0x7fd1e80b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1ed93d700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd1e80cd880 0x7fd1e80b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1ed93d700 1 -- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1e815d340 msgr2=0x7fd1e806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.866+0000 7fd1ed93d700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1e815d340 0x7fd1e806ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.867+0000 7fd1ed93d700 1 -- 172.21.15.45:0/2224315808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d8009550 con 0x7fd1e80cdd00 2023-08-27T11:27:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.867+0000 7fd1ee93f700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1e815d340 0x7fd1e806ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.867+0000 7fd1ed93d700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cdd00 0x7fd1e814c8e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8000c00 tx=0x7fd1d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:17.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.867+0000 7fd1deffd700 1 -- 172.21.15.45:0/2224315808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8004520 con 0x7fd1e80cdd00 2023-08-27T11:27:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.867+0000 7fd1deffd700 1 -- 172.21.15.45:0/2224315808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d801e070 con 0x7fd1e80cdd00 2023-08-27T11:27:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.867+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e806d840 con 0x7fd1e80cdd00 2023-08-27T11:27:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.867+0000 7fd1deffd700 1 -- 172.21.15.45:0/2224315808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d80038a0 con 0x7fd1e80cdd00 2023-08-27T11:27:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.867+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e814c0e0 con 0x7fd1e80cdd00 2023-08-27T11:27:17.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.868+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e8072690 con 0x7fd1e80cdd00 2023-08-27T11:27:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.868+0000 7fd1deffd700 1 -- 172.21.15.45:0/2224315808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd1d8003a00 con 0x7fd1e80cdd00 2023-08-27T11:27:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.869+0000 7fd1deffd700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd1cc06a200 0x7fd1cc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.869+0000 7fd1deffd700 1 -- 172.21.15.45:0/2224315808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fd1d8020070 con 0x7fd1e80cdd00 2023-08-27T11:27:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.869+0000 7fd1ee13e700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd1cc06a200 0x7fd1cc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.869+0000 7fd1ee13e700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd1cc06a200 0x7fd1cc06c6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006b20 tx=0x7fd1e4008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:17.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:17.873+0000 7fd1deffd700 1 -- 172.21.15.45:0/2224315808 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd1d8055260 con 0x7fd1e80cdd00 2023-08-27T11:27:18.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.070+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1e80734a0 con 0x7fd1e80cdd00 2023-08-27T11:27:18.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.070+0000 7fd1deffd700 1 -- 172.21.15.45:0/2224315808 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1d80150a0 con 0x7fd1e80cdd00 2023-08-27T11:27:18.072 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:27:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd1cc06a200 msgr2=0x7fd1cc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd1cc06a200 0x7fd1cc06c6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006b20 tx=0x7fd1e4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cdd00 msgr2=0x7fd1e814c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cdd00 0x7fd1e814c8e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8000c00 tx=0x7fd1d8004750 comp rx=0 tx=0).stop 2023-08-27T11:27:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 shutdown_connections 2023-08-27T11:27:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fd1cc06a200 0x7fd1cc06c6c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1e815d340 0x7fd1e806ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd1e80cd880 0x7fd1e80b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 --2- 172.21.15.45:0/2224315808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1e80cdd00 0x7fd1e814c8e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 >> 172.21.15.45:0/2224315808 conn(0x7fd1e80cc7c0 msgr2=0x7fd1e806f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 shutdown_connections 2023-08-27T11:27:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.073+0000 7fd1f03a2700 1 -- 172.21.15.45:0/2224315808 wait complete. 2023-08-27T11:27:18.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:18.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.213+0000 7f659952b700 1 Processor -- start 2023-08-27T11:27:18.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.213+0000 7f659952b700 1 -- start start 2023-08-27T11:27:18.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f659952b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f659415d2c0 0x7f65940d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f659952b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f65940ce4b0 0x7f65940b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f659952b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659415d6a0 con 0x7f65940b8c20 2023-08-27T11:27:18.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f659952b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65940d2c90 con 0x7f65940ce4b0 2023-08-27T11:27:18.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f659952b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65940d1bf0 con 0x7f659415d2c0 2023-08-27T11:27:18.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f65937fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f65940ce4b0 0x7f65940b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f65937fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f65940ce4b0 0x7f65940b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57462/0 (socket says 172.21.15.45:57462) 2023-08-27T11:27:18.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f65937fe700 1 -- 172.21.15.45:0/2553375348 learned_addr learned my addr 172.21.15.45:0/2553375348 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:18.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f6592ffd700 1 --2- 172.21.15.45:0/2553375348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f659415d2c0 0x7f65940d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f65937fe700 1 -- 172.21.15.45:0/2553375348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f659415d2c0 msgr2=0x7f65940d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f65937fe700 1 --2- 172.21.15.45:0/2553375348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f659415d2c0 0x7f65940d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f65937fe700 1 -- 172.21.15.45:0/2553375348 >> v1:172.21.15.135:6789/0 conn(0x7f65940b8c20 legacy=0x7f65940cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f65937fe700 1 -- 172.21.15.45:0/2553375348 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940d1ed0 con 0x7f65940ce4b0 2023-08-27T11:27:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f6592ffd700 1 --2- 172.21.15.45:0/2553375348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f659415d2c0 0x7f65940d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f65937fe700 1 --2- 172.21.15.45:0/2553375348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65940ce4b0 0x7f65940b6100 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f658c009e50 tx=0x7f658c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d520410d204e8c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.214+0000 7f6591ffb700 1 -- 172.21.15.45:0/2553375348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c00e070 con 0x7f65940ce4b0 2023-08-27T11:27:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f6591ffb700 1 -- 172.21.15.45:0/2553375348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c003a20 con 0x7f65940ce4b0 2023-08-27T11:27:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f6591ffb700 1 -- 172.21.15.45:0/2553375348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c003dd0 con 0x7f65940ce4b0 2023-08-27T11:27:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f659952b700 1 -- 172.21.15.45:0/2553375348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65940ce4b0 msgr2=0x7f65940b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f659952b700 1 --2- 172.21.15.45:0/2553375348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65940ce4b0 0x7f65940b6100 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f658c009e50 tx=0x7f658c00b330 comp rx=0 tx=0).stop 2023-08-27T11:27:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f659952b700 1 -- 172.21.15.45:0/2553375348 shutdown_connections 2023-08-27T11:27:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f659952b700 1 --2- 172.21.15.45:0/2553375348 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f659415d2c0 0x7f65940d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f659952b700 1 --2- 172.21.15.45:0/2553375348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65940ce4b0 0x7f65940b6100 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f659952b700 1 -- 172.21.15.45:0/2553375348 >> 172.21.15.45:0/2553375348 conn(0x7f65940d2fd0 msgr2=0x7f65940d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f659952b700 1 -- 172.21.15.45:0/2553375348 shutdown_connections 2023-08-27T11:27:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.215+0000 7f659952b700 1 -- 172.21.15.45:0/2553375348 wait complete. 2023-08-27T11:27:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.217+0000 7f659952b700 1 Processor -- start 2023-08-27T11:27:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.217+0000 7f659952b700 1 -- start start 2023-08-27T11:27:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f659952b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65940b8c20 0x7f65941574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f659952b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65940ce4b0 0x7f6594157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f659952b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f659415d2c0 0x7f65940c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f659952b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940c9960 con 0x7f659415d2c0 2023-08-27T11:27:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f659952b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940c6c10 con 0x7f65940b8c20 2023-08-27T11:27:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f659952b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659415e380 con 0x7f65940ce4b0 2023-08-27T11:27:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f659415d2c0 0x7f65940c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f659415d2c0 0x7f65940c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57468/0 (socket says 172.21.15.45:57468) 2023-08-27T11:27:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65937fe700 1 -- 172.21.15.45:0/4245066561 learned_addr learned my addr 172.21.15.45:0/4245066561 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f6592ffd700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65940b8c20 0x7f65941574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65927fc700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65940ce4b0 0x7f6594157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65937fe700 1 -- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65940ce4b0 msgr2=0x7f6594157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65937fe700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65940ce4b0 0x7f6594157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65937fe700 1 -- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65940b8c20 msgr2=0x7f65941574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65937fe700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65940b8c20 0x7f65941574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65937fe700 1 -- 172.21.15.45:0/4245066561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c009b30 con 0x7f659415d2c0 2023-08-27T11:27:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f6592ffd700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65940b8c20 0x7f65941574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65927fc700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65940ce4b0 0x7f6594157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.218+0000 7f65937fe700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f659415d2c0 0x7f65940c8a10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f658c004000 tx=0x7f658c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.219+0000 7f657b7fe700 1 -- 172.21.15.45:0/4245066561 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c004440 con 0x7f659415d2c0 2023-08-27T11:27:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.219+0000 7f657b7fe700 1 -- 172.21.15.45:0/4245066561 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c0045a0 con 0x7f659415d2c0 2023-08-27T11:27:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.219+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65940c6eb0 con 0x7f659415d2c0 2023-08-27T11:27:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.219+0000 7f657b7fe700 1 -- 172.21.15.45:0/4245066561 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c004950 con 0x7f659415d2c0 2023-08-27T11:27:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.219+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65941543f0 con 0x7f659415d2c0 2023-08-27T11:27:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.220+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659415e540 con 0x7f659415d2c0 2023-08-27T11:27:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.220+0000 7f657b7fe700 1 -- 172.21.15.45:0/4245066561 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f658c012430 con 0x7f659415d2c0 2023-08-27T11:27:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.220+0000 7f657b7fe700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f658006a200 0x7f658006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.221+0000 7f657b7fe700 1 -- 172.21.15.45:0/4245066561 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f658c089c40 con 0x7f659415d2c0 2023-08-27T11:27:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.221+0000 7f6592ffd700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f658006a200 0x7f658006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.221+0000 7f6592ffd700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f658006a200 0x7f658006c6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6588008910 tx=0x7f6588008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.225+0000 7f657b7fe700 1 -- 172.21.15.45:0/4245066561 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f658c055000 con 0x7f659415d2c0 2023-08-27T11:27:18.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.426+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f659415d6a0 con 0x7f659415d2c0 2023-08-27T11:27:18.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.427+0000 7f657b7fe700 1 -- 172.21.15.45:0/4245066561 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v77) v1 ==== 92+0+10 (secure 0 0 0) 0x7f659415d6a0 con 0x7f659415d2c0 2023-08-27T11:27:18.428 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:27:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f658006a200 msgr2=0x7f658006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f658006a200 0x7f658006c6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6588008910 tx=0x7f6588008040 comp rx=0 tx=0).stop 2023-08-27T11:27:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f659415d2c0 msgr2=0x7f65940c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f659415d2c0 0x7f65940c8a10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f658c004000 tx=0x7f658c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 shutdown_connections 2023-08-27T11:27:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f658006a200 0x7f658006c6c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f659415d2c0 0x7f65940c8a10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65940b8c20 0x7f65941574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 --2- 172.21.15.45:0/4245066561 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f65940ce4b0 0x7f6594157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 >> 172.21.15.45:0/4245066561 conn(0x7f65940d2fd0 msgr2=0x7f6594158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:18.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 shutdown_connections 2023-08-27T11:27:18.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.429+0000 7f659952b700 1 -- 172.21.15.45:0/4245066561 wait complete. 2023-08-27T11:27:18.442 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:27:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.570+0000 7f35cf20f700 1 Processor -- start 2023-08-27T11:27:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.570+0000 7f35cf20f700 1 -- start start 2023-08-27T11:27:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cf20f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 0x7f35c80716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cf20f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f35c80b17c0 0x7f35c80b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cf20f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35c806ced0 0x7f35c806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cf20f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c8157fc0 con 0x7f35c806ced0 2023-08-27T11:27:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cf20f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c806d860 con 0x7f35c80b17c0 2023-08-27T11:27:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cf20f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c815b460 con 0x7f35c8157be0 2023-08-27T11:27:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cd7ac700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35c806ced0 0x7f35c806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cd7ac700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35c806ced0 0x7f35c806d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57476/0 (socket says 172.21.15.45:57476) 2023-08-27T11:27:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cd7ac700 1 -- 172.21.15.45:0/3925915263 learned_addr learned my addr 172.21.15.45:0/3925915263 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35ccfab700 1 --2- 172.21.15.45:0/3925915263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 0x7f35c80716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35c7fff700 1 --2- 172.21.15.45:0/3925915263 >> v2:172.21.15.135:3300/0 conn(0x7f35c80b17c0 0x7f35c80b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cd7ac700 1 -- 172.21.15.45:0/3925915263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 msgr2=0x7f35c80716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cd7ac700 1 --2- 172.21.15.45:0/3925915263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 0x7f35c80716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cd7ac700 1 -- 172.21.15.45:0/3925915263 >> v2:172.21.15.135:3300/0 conn(0x7f35c80b17c0 msgr2=0x7f35c80b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cd7ac700 1 --2- 172.21.15.45:0/3925915263 >> v2:172.21.15.135:3300/0 conn(0x7f35c80b17c0 0x7f35c80b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cd7ac700 1 -- 172.21.15.45:0/3925915263 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c80b2280 con 0x7f35c806ced0 2023-08-27T11:27:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35cd7ac700 1 --2- 172.21.15.45:0/3925915263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 0x7f35c806d320 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f35c806c3b0 tx=0x7f35c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dea7c77837b7e28 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.571+0000 7f35c77fe700 1 -- 172.21.15.45:0/3925915263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c000e070 con 0x7f35c806ced0 2023-08-27T11:27:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35c77fe700 1 -- 172.21.15.45:0/3925915263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c000b9c0 con 0x7f35c806ced0 2023-08-27T11:27:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35c77fe700 1 -- 172.21.15.45:0/3925915263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c000bd70 con 0x7f35c806ced0 2023-08-27T11:27:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35cf20f700 1 -- 172.21.15.45:0/3925915263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 msgr2=0x7f35c806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35cf20f700 1 --2- 172.21.15.45:0/3925915263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 0x7f35c806d320 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f35c806c3b0 tx=0x7f35c000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35cf20f700 1 -- 172.21.15.45:0/3925915263 shutdown_connections 2023-08-27T11:27:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35cf20f700 1 --2- 172.21.15.45:0/3925915263 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 0x7f35c80716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35cf20f700 1 --2- 172.21.15.45:0/3925915263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 0x7f35c806d320 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35cf20f700 1 --2- 172.21.15.45:0/3925915263 >> v2:172.21.15.135:3300/0 conn(0x7f35c80b17c0 0x7f35c80b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35cf20f700 1 -- 172.21.15.45:0/3925915263 >> 172.21.15.45:0/3925915263 conn(0x7f35c80cc7c0 msgr2=0x7f35c80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:18.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35cf20f700 1 -- 172.21.15.45:0/3925915263 shutdown_connections 2023-08-27T11:27:18.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.572+0000 7f35cf20f700 1 -- 172.21.15.45:0/3925915263 wait complete. 2023-08-27T11:27:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.573+0000 7f35cf20f700 1 Processor -- start 2023-08-27T11:27:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.573+0000 7f35cf20f700 1 -- start start 2023-08-27T11:27:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35cf20f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 0x7f35c815d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35cf20f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35c80b17c0 0x7f35c814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35cf20f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 0x7f35c80afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35cf20f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80b2cd0 con 0x7f35c806ced0 2023-08-27T11:27:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35cf20f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80b54c0 con 0x7f35c80b17c0 2023-08-27T11:27:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35cf20f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80c7e70 con 0x7f35c8157be0 2023-08-27T11:27:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35ccfab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 0x7f35c815d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35ccfab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 0x7f35c815d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57478/0 (socket says 172.21.15.45:57478) 2023-08-27T11:27:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35ccfab700 1 -- 172.21.15.45:0/2024427778 learned_addr learned my addr 172.21.15.45:0/2024427778 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35cd7ac700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 0x7f35c80afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.574+0000 7f35c7fff700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35c80b17c0 0x7f35c814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35ccfab700 1 -- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 msgr2=0x7f35c80afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35ccfab700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 0x7f35c80afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35ccfab700 1 -- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35c80b17c0 msgr2=0x7f35c814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35ccfab700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35c80b17c0 0x7f35c814da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35ccfab700 1 -- 172.21.15.45:0/2024427778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c0009b30 con 0x7f35c806ced0 2023-08-27T11:27:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35ccfab700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 0x7f35c815d070 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00c7f0 tx=0x7f35bc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35c57fa700 1 -- 172.21.15.45:0/2024427778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc011800 con 0x7f35c806ced0 2023-08-27T11:27:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35c57fa700 1 -- 172.21.15.45:0/2024427778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc011960 con 0x7f35c806ced0 2023-08-27T11:27:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c80b5710 con 0x7f35c806ced0 2023-08-27T11:27:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35c57fa700 1 -- 172.21.15.45:0/2024427778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc011cf0 con 0x7f35c806ced0 2023-08-27T11:27:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.575+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c80c83f0 con 0x7f35c806ced0 2023-08-27T11:27:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.576+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c80c8610 con 0x7f35c806ced0 2023-08-27T11:27:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.576+0000 7f35c57fa700 1 -- 172.21.15.45:0/2024427778 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f35bc018360 con 0x7f35c806ced0 2023-08-27T11:27:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.577+0000 7f35c57fa700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f35ac06a200 0x7f35ac06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.577+0000 7f35c57fa700 1 -- 172.21.15.45:0/2024427778 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f35bc088880 con 0x7f35c806ced0 2023-08-27T11:27:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.577+0000 7f35c7fff700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f35ac06a200 0x7f35ac06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:18.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.577+0000 7f35c7fff700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f35ac06a200 0x7f35ac06c6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f35c815cdf0 tx=0x7f35b4008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:18.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.581+0000 7f35c57fa700 1 -- 172.21.15.45:0/2024427778 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f35bc053c40 con 0x7f35c806ced0 2023-08-27T11:27:18.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.786+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f35c815af00 con 0x7f35c806ced0 2023-08-27T11:27:18.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.789+0000 7f35c57fa700 1 -- 172.21.15.45:0/2024427778 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f35bc057260 con 0x7f35c806ced0 2023-08-27T11:27:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f35ac06a200 msgr2=0x7f35ac06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f35ac06a200 0x7f35ac06c6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f35c815cdf0 tx=0x7f35b4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 msgr2=0x7f35c815d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 0x7f35c815d070 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00c7f0 tx=0x7f35bc00cbb0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 shutdown_connections 2023-08-27T11:27:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f35ac06a200 0x7f35ac06c6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35c806ced0 0x7f35c815d070 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f35c8157be0 0x7f35c80afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 --2- 172.21.15.45:0/2024427778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35c80b17c0 0x7f35c814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 >> 172.21.15.45:0/2024427778 conn(0x7f35c80cc7c0 msgr2=0x7f35c80cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 shutdown_connections 2023-08-27T11:27:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:18.792+0000 7f35cf20f700 1 -- 172.21.15.45:0/2024427778 wait complete. 2023-08-27T11:27:18.805 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:27:18.805 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:18.851 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:18.903 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:27:18.903 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:27:18.904 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:18.922 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:27:18.922 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:27:18.923 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:18.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:18.943 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:27:18.943 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:18.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:27:18.979 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:27:18.980 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:19.000 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:19.001 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:27:19.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:19.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.142+0000 7f8043057700 1 Processor -- start 2023-08-27T11:27:19.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.143+0000 7f8043057700 1 -- start start 2023-08-27T11:27:19.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.143+0000 7f8043057700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0b45d0 0x7f803c0d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.143+0000 7f8043057700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c0ceae0 con 0x7f803c0cfd60 2023-08-27T11:27:19.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.143+0000 7f8043057700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c0cedc0 con 0x7f803c0cf980 2023-08-27T11:27:19.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.143+0000 7f8043057700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0cf0a0 con 0x7f803c0b45d0 2023-08-27T11:27:19.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.143+0000 7f80415f4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f803c0cfd60 0x7f803c0b2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44266/0 (socket says 172.21.15.45:44266) 2023-08-27T11:27:19.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f80415f4700 1 -- 172.21.15.45:0/3338346599 learned_addr learned my addr 172.21.15.45:0/3338346599 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:19.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915704541 0 0) 0x7f803c0ceae0 con 0x7f803c0cfd60 2023-08-27T11:27:19.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803bfff700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0b45d0 0x7f803c0d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c0035b0 con 0x7f803c0cfd60 2023-08-27T11:27:19.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477056794 0 0) 0x7f802c0035b0 con 0x7f803c0cfd60 2023-08-27T11:27:19.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0b45d0 msgr2=0x7f803c0d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0b45d0 0x7f803c0d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 >> v1:172.21.15.135:6789/0 conn(0x7f803c0cf980 legacy=0x7f803c0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0b22c0 con 0x7f803c0cfd60 2023-08-27T11:27:19.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f80340019f0 con 0x7f803c0cfd60 2023-08-27T11:27:19.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 >> v1:172.21.15.45:6789/0 conn(0x7f803c0cfd60 legacy=0x7f803c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0b45d0 0x7f802c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f802c003fb0 0x7f802c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c004d90 0x7f802c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0cf0a0 con 0x7f802c004d90 2023-08-27T11:27:19.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00baf0 con 0x7f802c003fb0 2023-08-27T11:27:19.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00bcb0 con 0x7f803c0b45d0 2023-08-27T11:27:19.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f80415f4700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c004d90 0x7f802c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f8040df3700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0b45d0 0x7f802c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.144+0000 7f803bfff700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f802c003fb0 0x7f802c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f80415f4700 1 -- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0b45d0 msgr2=0x7f802c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f80415f4700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0b45d0 0x7f802c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f80415f4700 1 -- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f802c003fb0 msgr2=0x7f802c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f80415f4700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f802c003fb0 0x7f802c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f80415f4700 1 -- 172.21.15.45:0/3338346599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0035b0 con 0x7f802c004d90 2023-08-27T11:27:19.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f80415f4700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c004d90 0x7f802c005210 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f803400a800 tx=0x7f802c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21aae5c1aef91444 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400a2f0 con 0x7f802c004d90 2023-08-27T11:27:19.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034012620 con 0x7f802c004d90 2023-08-27T11:27:19.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f803b7fe700 1 -- 172.21.15.45:0/3338346599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80340129d0 con 0x7f802c004d90 2023-08-27T11:27:19.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f8043057700 1 -- 172.21.15.45:0/3338346599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c004d90 msgr2=0x7f802c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f8043057700 1 --2- 172.21.15.45:0/3338346599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f802c004d90 0x7f802c005210 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f803400a800 tx=0x7f802c005880 comp rx=0 tx=0).stop 2023-08-27T11:27:19.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.145+0000 7f80415f4700 1 -- 172.21.15.45:0/3338346599 reap_dead start 2023-08-27T11:27:19.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.146+0000 7f8043057700 1 -- 172.21.15.45:0/3338346599 shutdown_connections 2023-08-27T11:27:19.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.146+0000 7f8043057700 1 -- 172.21.15.45:0/3338346599 >> 172.21.15.45:0/3338346599 conn(0x7f803c0d3020 msgr2=0x7f803c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:19.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.146+0000 7f8043057700 1 -- 172.21.15.45:0/3338346599 shutdown_connections 2023-08-27T11:27:19.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.146+0000 7f8043057700 1 -- 172.21.15.45:0/3338346599 wait complete. 2023-08-27T11:27:19.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.147+0000 7f8043057700 1 Processor -- start 2023-08-27T11:27:19.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.147+0000 7f8043057700 1 -- start start 2023-08-27T11:27:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.148+0000 7f8043057700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f802c004d90 0x7f803c14d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.148+0000 7f8043057700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0c6d10 0x7f803c0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.148+0000 7f8043057700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f803c1e7160 0x7f803c1e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.148+0000 7f8043057700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1e7b20 con 0x7f803c1e7160 2023-08-27T11:27:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.148+0000 7f8043057700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0cafe0 con 0x7f802c004d90 2023-08-27T11:27:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.148+0000 7f8043057700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0ca0f0 con 0x7f803c0c6d10 2023-08-27T11:27:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.148+0000 7f80415f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f803c1e7160 0x7f803c1e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.148+0000 7f80415f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f803c1e7160 0x7f803c1e75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57502/0 (socket says 172.21.15.45:57502) 2023-08-27T11:27:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.148+0000 7f80415f4700 1 -- 172.21.15.45:0/1006793009 learned_addr learned my addr 172.21.15.45:0/1006793009 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:19.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f803bfff700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0c6d10 0x7f803c0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f8040df3700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f802c004d90 0x7f803c14d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f80415f4700 1 -- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0c6d10 msgr2=0x7f803c0c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f80415f4700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0c6d10 0x7f803c0c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f80415f4700 1 -- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f802c004d90 msgr2=0x7f803c14d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f80415f4700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f802c004d90 0x7f803c14d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f80415f4700 1 -- 172.21.15.45:0/1006793009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0ca3d0 con 0x7f803c1e7160 2023-08-27T11:27:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f80415f4700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f803c1e7160 0x7f803c1e75e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f802c00bac0 tx=0x7f8034001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f80397fa700 1 -- 172.21.15.45:0/1006793009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80340019f0 con 0x7f803c1e7160 2023-08-27T11:27:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f80397fa700 1 -- 172.21.15.45:0/1006793009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803401e430 con 0x7f803c1e7160 2023-08-27T11:27:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f80397fa700 1 -- 172.21.15.45:0/1006793009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803401e7e0 con 0x7f803c1e7160 2023-08-27T11:27:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c0ca6b0 con 0x7f803c1e7160 2023-08-27T11:27:19.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.149+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c0cac70 con 0x7f803c1e7160 2023-08-27T11:27:19.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.150+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c0cae30 con 0x7f803c1e7160 2023-08-27T11:27:19.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.150+0000 7f80397fa700 1 -- 172.21.15.45:0/1006793009 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f803401e940 con 0x7f803c1e7160 2023-08-27T11:27:19.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.151+0000 7f80397fa700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f802006a200 0x7f802006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.151+0000 7f80397fa700 1 -- 172.21.15.45:0/1006793009 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f803408a970 con 0x7f803c1e7160 2023-08-27T11:27:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.151+0000 7f8040df3700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f802006a200 0x7f802006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.152+0000 7f8040df3700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f802006a200 0x7f802006c6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8030000c00 tx=0x7f8030009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.155+0000 7f80397fa700 1 -- 172.21.15.45:0/1006793009 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8034055ec0 con 0x7f803c1e7160 2023-08-27T11:27:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.402+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f803c0734a0 con 0x7f803c1e7160 2023-08-27T11:27:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.403+0000 7f80397fa700 1 -- 172.21.15.45:0/1006793009 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f80340594e0 con 0x7f803c1e7160 2023-08-27T11:27:19.404 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:19.405 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/72415568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":72415568},{"type":"v1","addr":"172.21.15.135:6837","nonce":72415568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5974,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2554143619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2554143619},{"type":"v1","addr":"172.21.15.135:6835","nonce":2554143619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6306,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/990560009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":990560009},{"type":"v1","addr":"172.21.15.45:6834","nonce":990560009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3538246625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3538246625},{"type":"v1","addr":"172.21.15.45:6835","nonce":3538246625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2023-08-27T11:27:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f802006a200 msgr2=0x7f802006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f802006a200 0x7f802006c6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8030000c00 tx=0x7f8030009040 comp rx=0 tx=0).stop 2023-08-27T11:27:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f803c1e7160 msgr2=0x7f803c1e75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f803c1e7160 0x7f803c1e75e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f802c00bac0 tx=0x7f8034001d80 comp rx=0 tx=0).stop 2023-08-27T11:27:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 shutdown_connections 2023-08-27T11:27:19.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f802006a200 0x7f802006c6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f803c1e7160 0x7f803c1e75e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f802c004d90 0x7f803c14d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 --2- 172.21.15.45:0/1006793009 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f803c0c6d10 0x7f803c0c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 >> 172.21.15.45:0/1006793009 conn(0x7f803c0d3020 msgr2=0x7f803c160a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 shutdown_connections 2023-08-27T11:27:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.405+0000 7f8043057700 1 -- 172.21.15.45:0/1006793009 wait complete. 2023-08-27T11:27:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2023-08-27T11:27:19.418 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:27:19.419 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:27:19.419 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:27:19.419 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:27:19.420 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:27:19.420 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:27:19.420 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:27:19.420 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:27:19.421 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:27:19.421 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:27:19.421 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:27:19.422 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:27:19.422 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:27:19.422 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:27:19.422 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:27:19.423 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:27:19.428 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:27:19.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:27:19.429 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:27:19.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:27:19.433 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:27:19.433 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:27:19.434 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:27:19.434 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:27:19.471 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:27:19.472 DEBUG:teuthology.parallel:result is None 2023-08-27T11:27:19.473 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:27:19.473 DEBUG:teuthology.parallel:result is None 2023-08-27T11:27:19.477 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:27:19.478 DEBUG:teuthology.parallel:result is None 2023-08-27T11:27:19.479 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:27:19.479 DEBUG:teuthology.parallel:result is None 2023-08-27T11:27:19.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:27:19.554 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-08-27T11:27:19.555 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-08-27T11:27:19.565 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2023-08-27T11:27:19.565 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2023-08-27T11:27:19.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.611+0000 7f4713aca700 1 Processor -- start 2023-08-27T11:27:19.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.612+0000 7f4713aca700 1 -- start start 2023-08-27T11:27:19.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.612+0000 7f4713aca700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f470c0d02c0 0x7f470c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.612+0000 7f4713aca700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f470c0b0320 0x7f470c06c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.612+0000 7f4713aca700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470c15d780 con 0x7f470c15d3a0 2023-08-27T11:27:19.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.612+0000 7f4713aca700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c0cfe30 con 0x7f470c0d02c0 2023-08-27T11:27:19.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.612+0000 7f4713aca700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c0cf220 con 0x7f470c0b0320 2023-08-27T11:27:19.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4711065700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f470c0d02c0 0x7f470c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4711866700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f470c15d3a0 0x7f470c0d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44306/0 (socket says 172.21.15.45:44306) 2023-08-27T11:27:19.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4711866700 1 -- 172.21.15.45:0/4268012448 learned_addr learned my addr 172.21.15.45:0/4268012448 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:19.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4712067700 1 --2- 172.21.15.45:0/4268012448 >> v2:172.21.15.135:3300/0 conn(0x7f470c0b0320 0x7f470c06c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4711065700 1 -- 172.21.15.45:0/4268012448 >> v2:172.21.15.135:3300/0 conn(0x7f470c0b0320 msgr2=0x7f470c06c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:19.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4711065700 1 --2- 172.21.15.45:0/4268012448 >> v2:172.21.15.135:3300/0 conn(0x7f470c0b0320 0x7f470c06c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4711065700 1 -- 172.21.15.45:0/4268012448 >> v1:172.21.15.45:6789/0 conn(0x7f470c15d3a0 legacy=0x7f470c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4711065700 1 -- 172.21.15.45:0/4268012448 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c0cf500 con 0x7f470c0d02c0 2023-08-27T11:27:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4710864700 1 -- 172.21.15.45:0/4268012448 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227943561 0 0) 0x7f46fc001110 con 0x7f470c15d3a0 2023-08-27T11:27:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4711065700 1 --2- 172.21.15.45:0/4268012448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 0x7f470c06bda0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f470c08b1f0 tx=0x7f4708009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab6b98839c6f27 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4710864700 1 -- 172.21.15.45:0/4268012448 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708011830 con 0x7f470c0d02c0 2023-08-27T11:27:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4710864700 1 -- 172.21.15.45:0/4268012448 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4708011990 con 0x7f470c0d02c0 2023-08-27T11:27:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.613+0000 7f4710864700 1 -- 172.21.15.45:0/4268012448 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708011d20 con 0x7f470c0d02c0 2023-08-27T11:27:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.614+0000 7f4713aca700 1 -- 172.21.15.45:0/4268012448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 msgr2=0x7f470c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.614+0000 7f4713aca700 1 --2- 172.21.15.45:0/4268012448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 0x7f470c06bda0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f470c08b1f0 tx=0x7f4708009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.614+0000 7f4713aca700 1 -- 172.21.15.45:0/4268012448 shutdown_connections 2023-08-27T11:27:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.614+0000 7f4713aca700 1 --2- 172.21.15.45:0/4268012448 >> v2:172.21.15.135:3300/0 conn(0x7f470c0b0320 0x7f470c06c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.614+0000 7f4713aca700 1 --2- 172.21.15.45:0/4268012448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 0x7f470c06bda0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.614+0000 7f4713aca700 1 -- 172.21.15.45:0/4268012448 >> 172.21.15.45:0/4268012448 conn(0x7f470c09a1a0 msgr2=0x7f470c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:19.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.614+0000 7f4713aca700 1 -- 172.21.15.45:0/4268012448 shutdown_connections 2023-08-27T11:27:19.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.614+0000 7f4713aca700 1 -- 172.21.15.45:0/4268012448 wait complete. 2023-08-27T11:27:19.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.615+0000 7f4713aca700 1 Processor -- start 2023-08-27T11:27:19.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.615+0000 7f4713aca700 1 -- start start 2023-08-27T11:27:19.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4713aca700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c0b0320 0x7f470c0af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4713aca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 0x7f470c15f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4713aca700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470c15d3a0 0x7f470c15f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4713aca700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0affb0 con 0x7f470c0d02c0 2023-08-27T11:27:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4713aca700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0c4820 con 0x7f470c0b0320 2023-08-27T11:27:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4713aca700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0c4b00 con 0x7f470c15d3a0 2023-08-27T11:27:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4711065700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 0x7f470c15f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4711065700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 0x7f470c15f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57538/0 (socket says 172.21.15.45:57538) 2023-08-27T11:27:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4711065700 1 -- 172.21.15.45:0/3370202144 learned_addr learned my addr 172.21.15.45:0/3370202144 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4711866700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c0b0320 0x7f470c0af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4712067700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470c15d3a0 0x7f470c15f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4711065700 1 -- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470c15d3a0 msgr2=0x7f470c15f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4711065700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470c15d3a0 0x7f470c15f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4711065700 1 -- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c0b0320 msgr2=0x7f470c0af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4711065700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c0b0320 0x7f470c0af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.616+0000 7f4711065700 1 -- 172.21.15.45:0/3370202144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47080094a0 con 0x7f470c0d02c0 2023-08-27T11:27:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.617+0000 7f4711065700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 0x7f470c15f1e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f47080090b0 tx=0x7f4708004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.617+0000 7f47027fc700 1 -- 172.21.15.45:0/3370202144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47080036a0 con 0x7f470c0d02c0 2023-08-27T11:27:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.617+0000 7f47027fc700 1 -- 172.21.15.45:0/3370202144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470801f070 con 0x7f470c0d02c0 2023-08-27T11:27:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.617+0000 7f47027fc700 1 -- 172.21.15.45:0/3370202144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708003a60 con 0x7f470c0d02c0 2023-08-27T11:27:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.617+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c0c4de0 con 0x7f470c0d02c0 2023-08-27T11:27:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.617+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c0afca0 con 0x7f470c0d02c0 2023-08-27T11:27:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.618+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c0b0120 con 0x7f470c0d02c0 2023-08-27T11:27:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.618+0000 7f47027fc700 1 -- 172.21.15.45:0/3370202144 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4708003bc0 con 0x7f470c0d02c0 2023-08-27T11:27:19.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.618+0000 7f47027fc700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f46f006a200 0x7f46f006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:19.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.619+0000 7f47027fc700 1 -- 172.21.15.45:0/3370202144 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f4708021070 con 0x7f470c0d02c0 2023-08-27T11:27:19.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.619+0000 7f4711866700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f46f006a200 0x7f46f006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:19.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.619+0000 7f4711866700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f46f006a200 0x7f46f006c6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000f60 tx=0x7f46fc007e70 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:19.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.623+0000 7f47027fc700 1 -- 172.21.15.45:0/3370202144 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4708055500 con 0x7f470c0d02c0 2023-08-27T11:27:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:19.845+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f470c0cf7b0 con 0x7f470c0d02c0 2023-08-27T11:27:20.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.802+0000 7f47027fc700 1 -- 172.21.15.45:0/3370202144 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v82) v1 ==== 97+0+0 (secure 0 0 0) 0x7f470802b080 con 0x7f470c0d02c0 2023-08-27T11:27:20.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.805+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f46f006a200 msgr2=0x7f46f006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.805+0000 7f4713aca700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f46f006a200 0x7f46f006c6c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000f60 tx=0x7f46fc007e70 comp rx=0 tx=0).stop 2023-08-27T11:27:20.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.805+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 msgr2=0x7f470c15f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.805+0000 7f4713aca700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 0x7f470c15f1e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f47080090b0 tx=0x7f4708004750 comp rx=0 tx=0).stop 2023-08-27T11:27:20.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.805+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 shutdown_connections 2023-08-27T11:27:20.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.805+0000 7f4713aca700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f46f006a200 0x7f46f006c6c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.805+0000 7f4713aca700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f470c15d3a0 0x7f470c15f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.806+0000 7f4713aca700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f470c0b0320 0x7f470c0af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.806+0000 7f4713aca700 1 --2- 172.21.15.45:0/3370202144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f470c0d02c0 0x7f470c15f1e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.806+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 >> 172.21.15.45:0/3370202144 conn(0x7f470c09a1a0 msgr2=0x7f470c0cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:20.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.806+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 shutdown_connections 2023-08-27T11:27:20.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.806+0000 7f4713aca700 1 -- 172.21.15.45:0/3370202144 wait complete. 2023-08-27T11:27:20.809 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-08-27T11:27:20.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:27:20.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.953+0000 7f14192a0700 1 Processor -- start 2023-08-27T11:27:20.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.953+0000 7f14192a0700 1 -- start start 2023-08-27T11:27:20.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.953+0000 7f14192a0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f141406bda0 0x7f141406c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.953+0000 7f14192a0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14140cc790 0x7f14140ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.953+0000 7f14192a0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14140cff50 con 0x7f141415d140 2023-08-27T11:27:20.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.953+0000 7f14192a0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14140cc4e0 con 0x7f141406bda0 2023-08-27T11:27:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.953+0000 7f14192a0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140ce820 con 0x7f14140cc790 2023-08-27T11:27:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f14127fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f141406bda0 0x7f141406c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f1412ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f141415d140 0x7f14140ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44324/0 (socket says 172.21.15.45:44324) 2023-08-27T11:27:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f1412ffd700 1 -- 172.21.15.45:0/3274315602 learned_addr learned my addr 172.21.15.45:0/3274315602 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f14137fe700 1 --2- 172.21.15.45:0/3274315602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14140cc790 0x7f14140ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f14127fc700 1 -- 172.21.15.45:0/3274315602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14140cc790 msgr2=0x7f14140ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:20.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f14127fc700 1 --2- 172.21.15.45:0/3274315602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14140cc790 0x7f14140ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f14127fc700 1 -- 172.21.15.45:0/3274315602 >> v1:172.21.15.45:6789/0 conn(0x7f141415d140 legacy=0x7f14140ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f1411ffb700 1 -- 172.21.15.45:0/3274315602 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709725122 0 0) 0x7f14140cff50 con 0x7f141415d140 2023-08-27T11:27:20.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f14127fc700 1 -- 172.21.15.45:0/3274315602 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14140cf0a0 con 0x7f141406bda0 2023-08-27T11:27:20.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f14137fe700 1 --2- 172.21.15.45:0/3274315602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14140cc790 0x7f14140ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:20.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f14127fc700 1 --2- 172.21.15.45:0/3274315602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f141406bda0 0x7f141406c220 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009840 tx=0x7f13fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3114ca4cd38896e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:20.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f1411ffb700 1 -- 172.21.15.45:0/3274315602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc011830 con 0x7f141406bda0 2023-08-27T11:27:20.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f1411ffb700 1 -- 172.21.15.45:0/3274315602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc011990 con 0x7f141406bda0 2023-08-27T11:27:20.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.954+0000 7f1411ffb700 1 -- 172.21.15.45:0/3274315602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc011d20 con 0x7f141406bda0 2023-08-27T11:27:20.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.955+0000 7f14192a0700 1 -- 172.21.15.45:0/3274315602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f141406bda0 msgr2=0x7f141406c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.955+0000 7f14192a0700 1 --2- 172.21.15.45:0/3274315602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f141406bda0 0x7f141406c220 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009840 tx=0x7f13fc009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:20.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.955+0000 7f14192a0700 1 -- 172.21.15.45:0/3274315602 shutdown_connections 2023-08-27T11:27:20.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.955+0000 7f14192a0700 1 --2- 172.21.15.45:0/3274315602 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14140cc790 0x7f14140ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.955+0000 7f14192a0700 1 --2- 172.21.15.45:0/3274315602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f141406bda0 0x7f141406c220 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.955+0000 7f14192a0700 1 -- 172.21.15.45:0/3274315602 >> 172.21.15.45:0/3274315602 conn(0x7f141409a160 msgr2=0x7f14140d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:20.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.955+0000 7f14192a0700 1 -- 172.21.15.45:0/3274315602 shutdown_connections 2023-08-27T11:27:20.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.955+0000 7f14192a0700 1 -- 172.21.15.45:0/3274315602 wait complete. 2023-08-27T11:27:20.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.956+0000 7f14192a0700 1 Processor -- start 2023-08-27T11:27:20.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.956+0000 7f14192a0700 1 -- start start 2023-08-27T11:27:20.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14192a0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f141406bda0 0x7f141409b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14192a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14140cc790 0x7f141409d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14192a0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f141415d140 0x7f14141594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14192a0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141409bc20 con 0x7f14140cc790 2023-08-27T11:27:20.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14192a0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141409d640 con 0x7f141415d140 2023-08-27T11:27:20.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14192a0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141409e490 con 0x7f141406bda0 2023-08-27T11:27:20.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14127fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14140cc790 0x7f141409d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14127fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14140cc790 0x7f141409d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57556/0 (socket says 172.21.15.45:57556) 2023-08-27T11:27:20.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14127fc700 1 -- 172.21.15.45:0/275300807 learned_addr learned my addr 172.21.15.45:0/275300807 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f1412ffd700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f141406bda0 0x7f141409b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14137fe700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f141415d140 0x7f14141594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14127fc700 1 -- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f141406bda0 msgr2=0x7f141409b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14127fc700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f141406bda0 0x7f141409b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14127fc700 1 -- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f141415d140 msgr2=0x7f14141594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14127fc700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f141415d140 0x7f14141594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.957+0000 7f14127fc700 1 -- 172.21.15.45:0/275300807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc0094a0 con 0x7f14140cc790 2023-08-27T11:27:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.958+0000 7f14127fc700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14140cc790 0x7f141409d100 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f13fc014040 tx=0x7f13fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.958+0000 7f140b7fe700 1 -- 172.21.15.45:0/275300807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0045f0 con 0x7f14140cc790 2023-08-27T11:27:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.958+0000 7f140b7fe700 1 -- 172.21.15.45:0/275300807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc01e070 con 0x7f14140cc790 2023-08-27T11:27:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.958+0000 7f140b7fe700 1 -- 172.21.15.45:0/275300807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc003ae0 con 0x7f14140cc790 2023-08-27T11:27:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.958+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1414159a00 con 0x7f14140cc790 2023-08-27T11:27:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.958+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141409db60 con 0x7f14140cc790 2023-08-27T11:27:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.959+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14140cd0d0 con 0x7f14140cc790 2023-08-27T11:27:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.959+0000 7f140b7fe700 1 -- 172.21.15.45:0/275300807 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f13fc003c40 con 0x7f14140cc790 2023-08-27T11:27:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.960+0000 7f140b7fe700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f13f406a200 0x7f13f406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.960+0000 7f140b7fe700 1 -- 172.21.15.45:0/275300807 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f13fc0896c0 con 0x7f14140cc790 2023-08-27T11:27:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.960+0000 7f1412ffd700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f13f406a200 0x7f13f406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:20.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.960+0000 7f1412ffd700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f13f406a200 0x7f13f406c6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1404000f60 tx=0x7f1404007be0 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:20.964+0000 7f140b7fe700 1 -- 172.21.15.45:0/275300807 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f13fc054f10 con 0x7f14140cc790 2023-08-27T11:27:21.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.246+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f141406c360 con 0x7f14140cc790 2023-08-27T11:27:21.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.246+0000 7f140b7fe700 1 -- 172.21.15.45:0/275300807 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f13fc020090 con 0x7f14140cc790 2023-08-27T11:27:21.247 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:21.248 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:27:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-08-27T11:27:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-08-27T11:27:21.248 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tknzGlBhAABVGsUvHGyOdTgH9miKO0Cg==", 2023-08-27T11:27:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:27:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:27:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:27:21.249 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-08-27T11:27:21.250 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkEWbPExAAgt/9BpPD1vK9QoG8l3HeFw==", 2023-08-27T11:27:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:27:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:27:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:27:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-08-27T11:27:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk6qRCDhAAGaj+NY32IDjdRcuSNbASpw==", 2023-08-27T11:27:21.252 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:27:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:27:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:27:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-08-27T11:27:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkE9NxGxAAnWum05rSbYQSLZ9IjjqDpQ==", 2023-08-27T11:27:21.254 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:27:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:27:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:27:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-08-27T11:27:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBML+tkOzvBBBAA196QdZiLzcsm5hLBGDl0Qw==", 2023-08-27T11:27:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.256 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:27:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.257 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-08-27T11:27:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBPL+tkIXaxBBAAfHwZ2SsmOENhnNxZsG230Q==", 2023-08-27T11:27:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:27:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-08-27T11:27:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBSL+tkpzphBBAA37UzwlmuLdSWDIrkBu/8Rw==", 2023-08-27T11:27:21.259 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:27:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-08-27T11:27:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVL+tkt63hAxAAJReGReJHqyggSbdFEhm3+A==", 2023-08-27T11:27:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:27:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-08-27T11:27:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBbL+tkTOFXBxAAZ1I44e50IrNzgj7mjKyuGA==", 2023-08-27T11:27:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:27:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-08-27T11:27:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBeL+tkdezFBhAANof2C80WiBZ6hWm3oB3kfg==", 2023-08-27T11:27:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:27:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-08-27T11:27:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBhL+tk7auXBhAAJPwKwTVIUzao5D3t4iXYcw==", 2023-08-27T11:27:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:27:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-08-27T11:27:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBkL+tkY+ztBRAAvM2oxELLN8WV0i0spqfY6Q==", 2023-08-27T11:27:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:27:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:27:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-08-27T11:27:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ==", 2023-08-27T11:27:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:27:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2023-08-27T11:27:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:27:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-08-27T11:27:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkMCDoJhAAevee0PlTshlWJcPzX8fpYg==", 2023-08-27T11:27:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:27:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:27:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:27:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:27:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-08-27T11:27:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk1xh+IxAAswoUpd+D70hUwfeKI2gc0w==", 2023-08-27T11:27:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:27:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:27:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:27:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:27:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-08-27T11:27:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkGWuaLBAAYM2t+TMEs59u635jL9f9kg==", 2023-08-27T11:27:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:27:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:27:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:27:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:27:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-08-27T11:27:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkkw7pGRAAVim28nKfumQg/LcBfJpivA==", 2023-08-27T11:27:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:27:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:27:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:27:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-08-27T11:27:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkNxfXAhAA/K5r8Mx/C7nRitNuzXXvKg==", 2023-08-27T11:27:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:27:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:27:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-08-27T11:27:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkW6B8NRAAF2hsNVRi3Ji4M8anuYwpDg==", 2023-08-27T11:27:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:27:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:27:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:27:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:27:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-08-27T11:27:21.282 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:27:21.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f13f406a200 msgr2=0x7f13f406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f13f406a200 0x7f13f406c6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1404000f60 tx=0x7f1404007be0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14140cc790 msgr2=0x7f141409d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14140cc790 0x7f141409d100 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f13fc014040 tx=0x7f13fc004750 comp rx=0 tx=0).stop 2023-08-27T11:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 shutdown_connections 2023-08-27T11:27:21.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f13f406a200 0x7f13f406c6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f141415d140 0x7f14141594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f141406bda0 0x7f141409b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 --2- 172.21.15.45:0/275300807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14140cc790 0x7f141409d100 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 >> 172.21.15.45:0/275300807 conn(0x7f141409a160 msgr2=0x7f14140ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 shutdown_connections 2023-08-27T11:27:21.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.248+0000 7f14192a0700 1 -- 172.21.15.45:0/275300807 wait complete. 2023-08-27T11:27:21.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.412+0000 7f20896cb700 1 Processor -- start 2023-08-27T11:27:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.412+0000 7f20896cb700 1 -- start start 2023-08-27T11:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20896cb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20840af040 0x7f20840cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20896cb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2084161550 con 0x7f208415d3d0 2023-08-27T11:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20896cb700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20840cd9d0 con 0x7f208406e3a0 2023-08-27T11:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20896cb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20840cdf90 con 0x7f20840af040 2023-08-27T11:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20827fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20840af040 0x7f20840cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f2082ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f208415d3d0 0x7f20841619e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44338/0 (socket says 172.21.15.45:44338) 2023-08-27T11:27:21.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20827fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20840af040 0x7f20840cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57564/0 (socket says 172.21.15.45:57564) 2023-08-27T11:27:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f2082ffd700 1 -- 172.21.15.45:0/1018651348 learned_addr learned my addr 172.21.15.45:0/1018651348 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20827fc700 1 -- 172.21.15.45:0/1018651348 >> v1:172.21.15.135:6789/0 conn(0x7f208406e3a0 legacy=0x7f2084070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20827fc700 1 -- 172.21.15.45:0/1018651348 >> v1:172.21.15.45:6789/0 conn(0x7f208415d3d0 legacy=0x7f20841619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20827fc700 1 -- 172.21.15.45:0/1018651348 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20840cdcb0 con 0x7f20840af040 2023-08-27T11:27:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f2081ffb700 1 -- 172.21.15.45:0/1018651348 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154898125 0 0) 0x7f2084161550 con 0x7f208415d3d0 2023-08-27T11:27:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.413+0000 7f20827fc700 1 --2- 172.21.15.45:0/1018651348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20840af040 0x7f20840cf420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2074009840 tx=0x7f2074009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5874907937ca9a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f2081ffb700 1 -- 172.21.15.45:0/1018651348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074004230 con 0x7f20840af040 2023-08-27T11:27:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f2081ffb700 1 -- 172.21.15.45:0/1018651348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2074004390 con 0x7f20840af040 2023-08-27T11:27:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f2081ffb700 1 -- 172.21.15.45:0/1018651348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20740036a0 con 0x7f20840af040 2023-08-27T11:27:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f20896cb700 1 -- 172.21.15.45:0/1018651348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20840af040 msgr2=0x7f20840cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f20896cb700 1 --2- 172.21.15.45:0/1018651348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20840af040 0x7f20840cf420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2074009840 tx=0x7f2074009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f20896cb700 1 -- 172.21.15.45:0/1018651348 shutdown_connections 2023-08-27T11:27:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f20896cb700 1 --2- 172.21.15.45:0/1018651348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20840af040 0x7f20840cf420 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f20896cb700 1 -- 172.21.15.45:0/1018651348 >> 172.21.15.45:0/1018651348 conn(0x7f20840cfd60 msgr2=0x7f20840d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f20896cb700 1 -- 172.21.15.45:0/1018651348 shutdown_connections 2023-08-27T11:27:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.414+0000 7f20896cb700 1 -- 172.21.15.45:0/1018651348 wait complete. 2023-08-27T11:27:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.415+0000 7f20896cb700 1 Processor -- start 2023-08-27T11:27:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20896cb700 1 -- start start 2023-08-27T11:27:21.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20896cb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f208406e3a0 0x7f20841f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20896cb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20840af040 0x7f20841f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20896cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f208415d3d0 0x7f20840b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20896cb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20840c48e0 con 0x7f208415d3d0 2023-08-27T11:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20896cb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20841f2c80 con 0x7f20840af040 2023-08-27T11:27:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20896cb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20841f2e70 con 0x7f208406e3a0 2023-08-27T11:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f208415d3d0 0x7f20840b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f208415d3d0 0x7f20840b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57572/0 (socket says 172.21.15.45:57572) 2023-08-27T11:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.416+0000 7f20837fe700 1 -- 172.21.15.45:0/546190124 learned_addr learned my addr 172.21.15.45:0/546190124 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f2082ffd700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f208406e3a0 0x7f20841f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f20827fc700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20840af040 0x7f20841f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f20837fe700 1 -- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f208406e3a0 msgr2=0x7f20841f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f20837fe700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f208406e3a0 0x7f20841f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f20837fe700 1 -- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20840af040 msgr2=0x7f20841f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f20837fe700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20840af040 0x7f20841f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f20837fe700 1 -- 172.21.15.45:0/546190124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20740094a0 con 0x7f208415d3d0 2023-08-27T11:27:21.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f20837fe700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f208415d3d0 0x7f20840b3390 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f207c00b970 tx=0x7f207c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f206b7fe700 1 -- 172.21.15.45:0/546190124 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c00c950 con 0x7f208415d3d0 2023-08-27T11:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f206b7fe700 1 -- 172.21.15.45:0/546190124 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c00cab0 con 0x7f208415d3d0 2023-08-27T11:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20841f3150 con 0x7f208415d3d0 2023-08-27T11:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f206b7fe700 1 -- 172.21.15.45:0/546190124 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c00ce60 con 0x7f208415d3d0 2023-08-27T11:27:21.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.417+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20841f3710 con 0x7f208415d3d0 2023-08-27T11:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.418+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20840c4a20 con 0x7f208415d3d0 2023-08-27T11:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.418+0000 7f206b7fe700 1 -- 172.21.15.45:0/546190124 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f207c01d3b0 con 0x7f208415d3d0 2023-08-27T11:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.419+0000 7f206b7fe700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f207006a200 0x7f207006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.419+0000 7f206b7fe700 1 -- 172.21.15.45:0/546190124 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f207c0880e0 con 0x7f208415d3d0 2023-08-27T11:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.419+0000 7f2082ffd700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f207006a200 0x7f207006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.420+0000 7f2082ffd700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f207006a200 0x7f207006c6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2078000f60 tx=0x7f2078007be0 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.423+0000 7f206b7fe700 1 -- 172.21.15.45:0/546190124 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f20840c4a20 con 0x7f208415d3d0 2023-08-27T11:27:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.624+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20840c4a20 con 0x7f208415d3d0 2023-08-27T11:27:21.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.625+0000 7f206b7fe700 1 -- 172.21.15.45:0/546190124 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+14926 (secure 0 0 0) 0x7f20840c4a20 con 0x7f208415d3d0 2023-08-27T11:27:21.626 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:21.626 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:20.799515+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f207006a200 msgr2=0x7f207006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f207006a200 0x7f207006c6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2078000f60 tx=0x7f2078007be0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f208415d3d0 msgr2=0x7f20840b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f208415d3d0 0x7f20840b3390 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f207c00b970 tx=0x7f207c00bd30 comp rx=0 tx=0).stop 2023-08-27T11:27:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 shutdown_connections 2023-08-27T11:27:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f207006a200 0x7f207006c6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f208415d3d0 0x7f20840b3390 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f208406e3a0 0x7f20841f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 --2- 172.21.15.45:0/546190124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20840af040 0x7f20841f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 >> 172.21.15.45:0/546190124 conn(0x7f20840cfd60 msgr2=0x7f20841619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 shutdown_connections 2023-08-27T11:27:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.627+0000 7f20896cb700 1 -- 172.21.15.45:0/546190124 wait complete. 2023-08-27T11:27:21.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:21.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.768+0000 7f6dd787e700 1 Processor -- start 2023-08-27T11:27:21.769 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.768+0000 7f6dd787e700 1 -- start start 2023-08-27T11:27:21.771 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.769+0000 7f6dd787e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6dd00ca3b0 0x7f6dd00ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.769+0000 7f6dd787e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6dd015d600 0x7f6dd0160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.769+0000 7f6dd787e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd00cacd0 con 0x7f6dd00aed20 2023-08-27T11:27:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.769+0000 7f6dd787e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00af270 con 0x7f6dd015d600 2023-08-27T11:27:21.772 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.769+0000 7f6dd787e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00af550 con 0x7f6dd00ca3b0 2023-08-27T11:27:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.769+0000 7f6dd4e19700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6dd015d600 0x7f6dd0160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.769+0000 7f6dd4e19700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6dd015d600 0x7f6dd0160ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57578/0 (socket says 172.21.15.45:57578) 2023-08-27T11:27:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.769+0000 7f6dd4e19700 1 -- 172.21.15.45:0/2896065913 learned_addr learned my addr 172.21.15.45:0/2896065913 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.769+0000 7f6dd561a700 1 --2- 172.21.15.45:0/2896065913 >> v2:172.21.15.135:3300/0 conn(0x7f6dd00ca3b0 0x7f6dd00ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dd4e19700 1 -- 172.21.15.45:0/2896065913 >> v2:172.21.15.135:3300/0 conn(0x7f6dd00ca3b0 msgr2=0x7f6dd00ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dd4e19700 1 --2- 172.21.15.45:0/2896065913 >> v2:172.21.15.135:3300/0 conn(0x7f6dd00ca3b0 0x7f6dd00ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dd4e19700 1 -- 172.21.15.45:0/2896065913 >> v1:172.21.15.135:6789/0 conn(0x7f6dd00aed20 legacy=0x7f6dd0161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dd4e19700 1 -- 172.21.15.45:0/2896065913 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd006ac90 con 0x7f6dd015d600 2023-08-27T11:27:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dd4e19700 1 --2- 172.21.15.45:0/2896065913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd015d600 0x7f6dd0160ef0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00cd8e0 tx=0x7f6dcc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be73568792a16bdc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dc7fff700 1 -- 172.21.15.45:0/2896065913 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc004230 con 0x7f6dd015d600 2023-08-27T11:27:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dc7fff700 1 -- 172.21.15.45:0/2896065913 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc004390 con 0x7f6dd015d600 2023-08-27T11:27:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dc7fff700 1 -- 172.21.15.45:0/2896065913 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0036a0 con 0x7f6dd015d600 2023-08-27T11:27:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dd787e700 1 -- 172.21.15.45:0/2896065913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd015d600 msgr2=0x7f6dd0160ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dd787e700 1 --2- 172.21.15.45:0/2896065913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd015d600 0x7f6dd0160ef0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00cd8e0 tx=0x7f6dcc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.770+0000 7f6dd787e700 1 -- 172.21.15.45:0/2896065913 shutdown_connections 2023-08-27T11:27:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.771+0000 7f6dd787e700 1 --2- 172.21.15.45:0/2896065913 >> v2:172.21.15.135:3300/0 conn(0x7f6dd00ca3b0 0x7f6dd00ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.771+0000 7f6dd787e700 1 --2- 172.21.15.45:0/2896065913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd015d600 0x7f6dd0160ef0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.771+0000 7f6dd787e700 1 -- 172.21.15.45:0/2896065913 >> 172.21.15.45:0/2896065913 conn(0x7f6dd00cc7e0 msgr2=0x7f6dd00ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.771+0000 7f6dd787e700 1 -- 172.21.15.45:0/2896065913 shutdown_connections 2023-08-27T11:27:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.771+0000 7f6dd787e700 1 -- 172.21.15.45:0/2896065913 wait complete. 2023-08-27T11:27:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.772+0000 7f6dd787e700 1 Processor -- start 2023-08-27T11:27:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.772+0000 7f6dd787e700 1 -- start start 2023-08-27T11:27:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd787e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd00aed20 0x7f6dd00b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd787e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dd00ca3b0 0x7f6dd01f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd787e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd015c080 0x7f6dd015fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd787e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00b3a20 con 0x7f6dd00aed20 2023-08-27T11:27:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd787e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01f3fe0 con 0x7f6dd015c080 2023-08-27T11:27:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd787e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00c6cb0 con 0x7f6dd00ca3b0 2023-08-27T11:27:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd561a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd00aed20 0x7f6dd00b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd561a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd00aed20 0x7f6dd00b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57582/0 (socket says 172.21.15.45:57582) 2023-08-27T11:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd561a700 1 -- 172.21.15.45:0/2217456059 learned_addr learned my addr 172.21.15.45:0/2217456059 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd561a700 1 -- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dd00ca3b0 msgr2=0x7f6dd01f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd561a700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dd00ca3b0 0x7f6dd01f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd561a700 1 -- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd015c080 msgr2=0x7f6dd015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd561a700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd015c080 0x7f6dd015fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd561a700 1 -- 172.21.15.45:0/2217456059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc009550 con 0x7f6dd00aed20 2023-08-27T11:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd561a700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd00aed20 0x7f6dd00b34e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000c7f0 tx=0x7f6dc000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dc5ffb700 1 -- 172.21.15.45:0/2217456059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0011800 con 0x7f6dd00aed20 2023-08-27T11:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dc5ffb700 1 -- 172.21.15.45:0/2217456059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc0011960 con 0x7f6dd00aed20 2023-08-27T11:27:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd015e360 con 0x7f6dd00aed20 2023-08-27T11:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.773+0000 7f6dc5ffb700 1 -- 172.21.15.45:0/2217456059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0011cf0 con 0x7f6dd00aed20 2023-08-27T11:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.774+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd015e8e0 con 0x7f6dd00aed20 2023-08-27T11:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.774+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd00726d0 con 0x7f6dd00aed20 2023-08-27T11:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.775+0000 7f6dc5ffb700 1 -- 172.21.15.45:0/2217456059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6dc0018360 con 0x7f6dd00aed20 2023-08-27T11:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.775+0000 7f6dc5ffb700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f6db406a230 0x7f6db406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.775+0000 7f6dc5ffb700 1 -- 172.21.15.45:0/2217456059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f6dc0088560 con 0x7f6dd00aed20 2023-08-27T11:27:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.775+0000 7f6dd4e19700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f6db406a230 0x7f6db406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.776+0000 7f6dd4e19700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f6db406a230 0x7f6db406c6f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000c00 tx=0x7f6dcc010000 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.780+0000 7f6dc5ffb700 1 -- 172.21.15.45:0/2217456059 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6dc00576b0 con 0x7f6dd00aed20 2023-08-27T11:27:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.978+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6dd00734a0 con 0x7f6dd00aed20 2023-08-27T11:27:21.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.978+0000 7f6dc5ffb700 1 -- 172.21.15.45:0/2217456059 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6dc0057240 con 0x7f6dd00aed20 2023-08-27T11:27:21.979 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:27:21.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f6db406a230 msgr2=0x7f6db406c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f6db406a230 0x7f6db406c6f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000c00 tx=0x7f6dcc010000 comp rx=0 tx=0).stop 2023-08-27T11:27:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd00aed20 msgr2=0x7f6dd00b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd00aed20 0x7f6dd00b34e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000c7f0 tx=0x7f6dc000cbb0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 shutdown_connections 2023-08-27T11:27:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f6db406a230 0x7f6db406c6f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dd00aed20 0x7f6dd00b34e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dd015c080 0x7f6dd015fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 --2- 172.21.15.45:0/2217456059 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dd00ca3b0 0x7f6dd01f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 >> 172.21.15.45:0/2217456059 conn(0x7f6dd00cc7e0 msgr2=0x7f6dd00afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 shutdown_connections 2023-08-27T11:27:21.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:21.980+0000 7f6dd787e700 1 -- 172.21.15.45:0/2217456059 wait complete. 2023-08-27T11:27:21.993 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:22.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.120+0000 7feae7726700 1 Processor -- start 2023-08-27T11:27:22.122 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.121+0000 7feae7726700 1 -- start start 2023-08-27T11:27:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.121+0000 7feae7726700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feae006d5e0 0x7feae00b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.121+0000 7feae7726700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae015b460 con 0x7feae00b17c0 2023-08-27T11:27:22.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.121+0000 7feae7726700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae00b2280 con 0x7feae0157be0 2023-08-27T11:27:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.121+0000 7feae7726700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae0157fc0 con 0x7feae006d5e0 2023-08-27T11:27:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7feae5cc3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feae006d5e0 0x7feae00b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7feae5cc3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feae006d5e0 0x7feae00b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57594/0 (socket says 172.21.15.45:57594) 2023-08-27T11:27:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7feae5cc3700 1 -- 172.21.15.45:0/23751032 learned_addr learned my addr 172.21.15.45:0/23751032 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:22.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7feae4cc1700 1 --1- 172.21.15.45:0/23751032 >> v1:172.21.15.45:6789/0 conn(0x7feae00b17c0 0x7feae006ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44366/0 (socket says 172.21.15.45:44366) 2023-08-27T11:27:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7feae5cc3700 1 -- 172.21.15.45:0/23751032 >> v1:172.21.15.135:6789/0 conn(0x7feae0157be0 legacy=0x7feae00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7feae5cc3700 1 -- 172.21.15.45:0/23751032 >> v1:172.21.15.45:6789/0 conn(0x7feae00b17c0 legacy=0x7feae006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7feae5cc3700 1 -- 172.21.15.45:0/23751032 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae00b2560 con 0x7feae006d5e0 2023-08-27T11:27:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7feae5cc3700 1 --2- 172.21.15.45:0/23751032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae006d5e0 0x7feae00b1c70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fead8009e50 tx=0x7fead800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f586d212f69cd1e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7fead7fff700 1 -- 172.21.15.45:0/23751032 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead800e070 con 0x7feae006d5e0 2023-08-27T11:27:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7fead7fff700 1 -- 172.21.15.45:0/23751032 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead800b9c0 con 0x7feae006d5e0 2023-08-27T11:27:22.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.122+0000 7fead7fff700 1 -- 172.21.15.45:0/23751032 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead800bd70 con 0x7feae006d5e0 2023-08-27T11:27:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.123+0000 7feae7726700 1 -- 172.21.15.45:0/23751032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae006d5e0 msgr2=0x7feae00b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.123+0000 7feae7726700 1 --2- 172.21.15.45:0/23751032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae006d5e0 0x7feae00b1c70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fead8009e50 tx=0x7fead800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.123+0000 7feae7726700 1 -- 172.21.15.45:0/23751032 shutdown_connections 2023-08-27T11:27:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.123+0000 7feae7726700 1 --2- 172.21.15.45:0/23751032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae006d5e0 0x7feae00b1c70 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.123+0000 7feae7726700 1 -- 172.21.15.45:0/23751032 >> 172.21.15.45:0/23751032 conn(0x7feae00cc7c0 msgr2=0x7feae00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.123+0000 7feae7726700 1 -- 172.21.15.45:0/23751032 shutdown_connections 2023-08-27T11:27:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.123+0000 7feae7726700 1 -- 172.21.15.45:0/23751032 wait complete. 2023-08-27T11:27:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.124+0000 7feae7726700 1 Processor -- start 2023-08-27T11:27:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.124+0000 7feae7726700 1 -- start start 2023-08-27T11:27:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae7726700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feae006d5e0 0x7feae00c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae7726700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feae00b17c0 0x7feae0153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae7726700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae0157be0 0x7feae006a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae7726700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae00c8340 con 0x7feae0157be0 2023-08-27T11:27:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae7726700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae00c8620 con 0x7feae00b17c0 2023-08-27T11:27:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae7726700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae00c8900 con 0x7feae006d5e0 2023-08-27T11:27:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae5cc3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae0157be0 0x7feae006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae5cc3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae0157be0 0x7feae006a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57600/0 (socket says 172.21.15.45:57600) 2023-08-27T11:27:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae5cc3700 1 -- 172.21.15.45:0/877886882 learned_addr learned my addr 172.21.15.45:0/877886882 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae4cc1700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feae00b17c0 0x7feae0153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae54c2700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feae006d5e0 0x7feae00c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae5cc3700 1 -- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feae006d5e0 msgr2=0x7feae00c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae5cc3700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feae006d5e0 0x7feae00c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae5cc3700 1 -- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feae00b17c0 msgr2=0x7feae0153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae5cc3700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feae00b17c0 0x7feae0153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae5cc3700 1 -- 172.21.15.45:0/877886882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8009b30 con 0x7feae0157be0 2023-08-27T11:27:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.125+0000 7feae54c2700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feae006d5e0 0x7feae00c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.126+0000 7feae5cc3700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae0157be0 0x7feae006a280 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7feae006c3e0 tx=0x7fead8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.126+0000 7fead5ffb700 1 -- 172.21.15.45:0/877886882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8004970 con 0x7feae0157be0 2023-08-27T11:27:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.126+0000 7fead5ffb700 1 -- 172.21.15.45:0/877886882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead80092e0 con 0x7feae0157be0 2023-08-27T11:27:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.126+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae00c8be0 con 0x7feae0157be0 2023-08-27T11:27:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.126+0000 7fead5ffb700 1 -- 172.21.15.45:0/877886882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead801b630 con 0x7feae0157be0 2023-08-27T11:27:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.126+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae0153a70 con 0x7feae0157be0 2023-08-27T11:27:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.127+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae0072690 con 0x7feae0157be0 2023-08-27T11:27:22.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.127+0000 7fead5ffb700 1 -- 172.21.15.45:0/877886882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fead801b790 con 0x7feae0157be0 2023-08-27T11:27:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.128+0000 7fead5ffb700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7feac406a200 0x7feac406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.128+0000 7fead5ffb700 1 -- 172.21.15.45:0/877886882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fead808a000 con 0x7feae0157be0 2023-08-27T11:27:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.128+0000 7feae54c2700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7feac406a200 0x7feac406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.129+0000 7feae54c2700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7feac406a200 0x7feac406c6c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feadc001780 tx=0x7feadc009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.132+0000 7fead5ffb700 1 -- 172.21.15.45:0/877886882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fead80556a0 con 0x7feae0157be0 2023-08-27T11:27:22.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.332+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feae00b1c70 con 0x7feae0157be0 2023-08-27T11:27:22.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.332+0000 7fead5ffb700 1 -- 172.21.15.45:0/877886882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v82) v1 ==== 92+0+10 (secure 0 0 0) 0x7fead8058cc0 con 0x7feae0157be0 2023-08-27T11:27:22.334 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:27:22.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7feac406a200 msgr2=0x7feac406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7feac406a200 0x7feac406c6c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feadc001780 tx=0x7feadc009040 comp rx=0 tx=0).stop 2023-08-27T11:27:22.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae0157be0 msgr2=0x7feae006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae0157be0 0x7feae006a280 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7feae006c3e0 tx=0x7fead8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 shutdown_connections 2023-08-27T11:27:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7feac406a200 0x7feac406c6c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feae0157be0 0x7feae006a280 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feae006d5e0 0x7feae00c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 --2- 172.21.15.45:0/877886882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feae00b17c0 0x7feae0153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 >> 172.21.15.45:0/877886882 conn(0x7feae00cc7c0 msgr2=0x7feae00ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:22.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 shutdown_connections 2023-08-27T11:27:22.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.335+0000 7feae7726700 1 -- 172.21.15.45:0/877886882 wait complete. 2023-08-27T11:27:22.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:27:22.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.472+0000 7fa168e60700 1 Processor -- start 2023-08-27T11:27:22.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.472+0000 7fa168e60700 1 -- start start 2023-08-27T11:27:22.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa168e60700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa1640b8c20 0x7fa1640cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa168e60700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16415d6a0 con 0x7fa16415d2c0 2023-08-27T11:27:22.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa168e60700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1640ce730 con 0x7fa1640ce2e0 2023-08-27T11:27:22.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa168e60700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1640d2c90 con 0x7fa1640b8c20 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa16415d2c0 0x7fa1640d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44380/0 (socket says 172.21.15.45:44380) 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16259c700 1 -- 172.21.15.45:0/1753709952 learned_addr learned my addr 172.21.15.45:0/1753709952 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212877619 0 0) 0x7fa154001110 con 0x7fa16415d2c0 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa14c0035b0 con 0x7fa16415d2c0 2023-08-27T11:27:22.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa161d9b700 1 --2- 172.21.15.45:0/1753709952 >> v2:172.21.15.135:3300/0 conn(0x7fa1640b8c20 0x7fa1640cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1325494077 0 0) 0x7fa14c0035b0 con 0x7fa16415d2c0 2023-08-27T11:27:22.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 >> v2:172.21.15.135:3300/0 conn(0x7fa1640b8c20 msgr2=0x7fa1640cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:22.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 --2- 172.21.15.45:0/1753709952 >> v2:172.21.15.135:3300/0 conn(0x7fa1640b8c20 0x7fa1640cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 >> v1:172.21.15.135:6789/0 conn(0x7fa1640ce2e0 legacy=0x7fa1640b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1640d1bf0 con 0x7fa16415d2c0 2023-08-27T11:27:22.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fa154001800 con 0x7fa16415d2c0 2023-08-27T11:27:22.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 >> v1:172.21.15.45:6789/0 conn(0x7fa16415d2c0 legacy=0x7fa1640d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14c003a30 0x7fa14c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c004850 0x7fa14c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c005260 0x7fa14c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640d2c90 con 0x7fa14c004850 2023-08-27T11:27:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c00c190 con 0x7fa14c003a30 2023-08-27T11:27:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.473+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c00c350 con 0x7fa14c005260 2023-08-27T11:27:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa161d9b700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c004850 0x7fa14c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa16259c700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14c003a30 0x7fa14c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa162d9d700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c005260 0x7fa14c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa161d9b700 1 -- 172.21.15.45:0/1753709952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c005260 msgr2=0x7fa14c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa161d9b700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c005260 0x7fa14c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa161d9b700 1 -- 172.21.15.45:0/1753709952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14c003a30 msgr2=0x7fa14c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:22.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa161d9b700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa14c003a30 0x7fa14c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa161d9b700 1 -- 172.21.15.45:0/1753709952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0035b0 con 0x7fa14c004850 2023-08-27T11:27:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa162d9d700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c005260 0x7fa14c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa162d9d700 1 -- 172.21.15.45:0/1753709952 reap_dead start 2023-08-27T11:27:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa161d9b700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c004850 0x7fa14c004cd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa158009860 tx=0x7fa14c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51dafd98b93e857 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15800c070 con 0x7fa14c004850 2023-08-27T11:27:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158013610 con 0x7fa14c004850 2023-08-27T11:27:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.474+0000 7fa16159a700 1 -- 172.21.15.45:0/1753709952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580139a0 con 0x7fa14c004850 2023-08-27T11:27:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.475+0000 7fa168e60700 1 -- 172.21.15.45:0/1753709952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c004850 msgr2=0x7fa14c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.475+0000 7fa168e60700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c004850 0x7fa14c004cd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa158009860 tx=0x7fa14c005880 comp rx=0 tx=0).stop 2023-08-27T11:27:22.483 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.475+0000 7fa168e60700 1 -- 172.21.15.45:0/1753709952 shutdown_connections 2023-08-27T11:27:22.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.475+0000 7fa168e60700 1 --2- 172.21.15.45:0/1753709952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa14c004850 0x7fa14c004cd0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.475+0000 7fa168e60700 1 -- 172.21.15.45:0/1753709952 >> 172.21.15.45:0/1753709952 conn(0x7fa1640d2fd0 msgr2=0x7fa1640d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:22.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.475+0000 7fa168e60700 1 -- 172.21.15.45:0/1753709952 shutdown_connections 2023-08-27T11:27:22.484 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.475+0000 7fa168e60700 1 -- 172.21.15.45:0/1753709952 wait complete. 2023-08-27T11:27:22.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.476+0000 7fa168e60700 1 Processor -- start 2023-08-27T11:27:22.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.476+0000 7fa168e60700 1 -- start start 2023-08-27T11:27:22.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa168e60700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c004850 0x7fa16414a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa168e60700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa16415fac0 0x7fa1640c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa168e60700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640c74f0 0x7fa1640cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa168e60700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640c88d0 con 0x7fa1640c74f0 2023-08-27T11:27:22.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa168e60700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16414ae70 con 0x7fa16415fac0 2023-08-27T11:27:22.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa168e60700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640c92a0 con 0x7fa14c004850 2023-08-27T11:27:22.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa162d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640c74f0 0x7fa1640cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa162d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640c74f0 0x7fa1640cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57620/0 (socket says 172.21.15.45:57620) 2023-08-27T11:27:22.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa162d9d700 1 -- 172.21.15.45:0/1731699916 learned_addr learned my addr 172.21.15.45:0/1731699916 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:22.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa16259c700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c004850 0x7fa16414a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa161d9b700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa16415fac0 0x7fa1640c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa162d9d700 1 -- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c004850 msgr2=0x7fa16414a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.477+0000 7fa162d9d700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c004850 0x7fa16414a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.478+0000 7fa162d9d700 1 -- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa16415fac0 msgr2=0x7fa1640c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.478+0000 7fa162d9d700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa16415fac0 0x7fa1640c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.478+0000 7fa162d9d700 1 -- 172.21.15.45:0/1731699916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164160080 con 0x7fa1640c74f0 2023-08-27T11:27:22.490 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.478+0000 7fa162d9d700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640c74f0 0x7fa1640cfa50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fa16406a3f0 tx=0x7fa15c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.478+0000 7fa152ffd700 1 -- 172.21.15.45:0/1731699916 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c003c10 con 0x7fa1640c74f0 2023-08-27T11:27:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.478+0000 7fa152ffd700 1 -- 172.21.15.45:0/1731699916 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c003d90 con 0x7fa1640c74f0 2023-08-27T11:27:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.478+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1640c7fa0 con 0x7fa1640c74f0 2023-08-27T11:27:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.478+0000 7fa152ffd700 1 -- 172.21.15.45:0/1731699916 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c005310 con 0x7fa1640c74f0 2023-08-27T11:27:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.478+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1640c8560 con 0x7fa1640c74f0 2023-08-27T11:27:22.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.479+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16404e620 con 0x7fa1640c74f0 2023-08-27T11:27:22.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.479+0000 7fa152ffd700 1 -- 172.21.15.45:0/1731699916 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa15c005470 con 0x7fa1640c74f0 2023-08-27T11:27:22.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.480+0000 7fa152ffd700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fa14406a200 0x7fa14406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.480+0000 7fa152ffd700 1 -- 172.21.15.45:0/1731699916 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fa15c0883a0 con 0x7fa1640c74f0 2023-08-27T11:27:22.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.480+0000 7fa16259c700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fa14406a200 0x7fa14406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.481+0000 7fa16259c700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fa14406a200 0x7fa14406c6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa154001050 tx=0x7fa154009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.493 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.484+0000 7fa152ffd700 1 -- 172.21.15.45:0/1731699916 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa15c053b60 con 0x7fa1640c74f0 2023-08-27T11:27:22.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.689+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa1640ce730 con 0x7fa1640c74f0 2023-08-27T11:27:22.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.690+0000 7fa152ffd700 1 -- 172.21.15.45:0/1731699916 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa1640ce730 con 0x7fa1640c74f0 2023-08-27T11:27:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:22.691 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:27:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-08-27T11:27:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-08-27T11:27:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:27:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-08-27T11:27:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-08-27T11:27:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:22.694 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:27:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fa14406a200 msgr2=0x7fa14406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fa14406a200 0x7fa14406c6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa154001050 tx=0x7fa154009040 comp rx=0 tx=0).stop 2023-08-27T11:27:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640c74f0 msgr2=0x7fa1640cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640c74f0 0x7fa1640cfa50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fa16406a3f0 tx=0x7fa15c00ee50 comp rx=0 tx=0).stop 2023-08-27T11:27:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 shutdown_connections 2023-08-27T11:27:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fa14406a200 0x7fa14406c6c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640c74f0 0x7fa1640cfa50 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa14c004850 0x7fa16414a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 --2- 172.21.15.45:0/1731699916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa16415fac0 0x7fa1640c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 >> 172.21.15.45:0/1731699916 conn(0x7fa1640d2fd0 msgr2=0x7fa1640b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 shutdown_connections 2023-08-27T11:27:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.692+0000 7fa168e60700 1 -- 172.21.15.45:0/1731699916 wait complete. 2023-08-27T11:27:22.706 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:27:22.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:27:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.835+0000 7f36e3796700 1 Processor -- start 2023-08-27T11:27:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.835+0000 7f36e3796700 1 -- start start 2023-08-27T11:27:22.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.835+0000 7f36e3796700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36dc15a0f0 0x7f36dc0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e3796700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc06ed90 0x7f36dc15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e3796700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36dc0cfae0 con 0x7f36dc0cd300 2023-08-27T11:27:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e3796700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36dc15e900 con 0x7f36dc15a0f0 2023-08-27T11:27:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e3796700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc0cae60 con 0x7f36dc06ed90 2023-08-27T11:27:22.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e1532700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36dc15a0f0 0x7f36dc0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e1532700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36dc15a0f0 0x7f36dc0cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57624/0 (socket says 172.21.15.45:57624) 2023-08-27T11:27:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e1532700 1 -- 172.21.15.45:0/1418284670 learned_addr learned my addr 172.21.15.45:0/1418284670 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e1d33700 1 --2- 172.21.15.45:0/1418284670 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc06ed90 0x7f36dc15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e1532700 1 -- 172.21.15.45:0/1418284670 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc06ed90 msgr2=0x7f36dc15e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e1532700 1 --2- 172.21.15.45:0/1418284670 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc06ed90 0x7f36dc15e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e1532700 1 -- 172.21.15.45:0/1418284670 >> v1:172.21.15.135:6789/0 conn(0x7f36dc0cd300 legacy=0x7f36dc15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e1532700 1 -- 172.21.15.45:0/1418284670 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc0cb140 con 0x7f36dc15a0f0 2023-08-27T11:27:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36e1532700 1 --2- 172.21.15.45:0/1418284670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 0x7f36dc0cbc20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f36d00098f0 tx=0x7f36d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce30aa91815d268c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36cffff700 1 -- 172.21.15.45:0/1418284670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0011830 con 0x7f36dc15a0f0 2023-08-27T11:27:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36cffff700 1 -- 172.21.15.45:0/1418284670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d0011990 con 0x7f36dc15a0f0 2023-08-27T11:27:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.836+0000 7f36cffff700 1 -- 172.21.15.45:0/1418284670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0011d20 con 0x7f36dc15a0f0 2023-08-27T11:27:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.837+0000 7f36e3796700 1 -- 172.21.15.45:0/1418284670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 msgr2=0x7f36dc0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.837+0000 7f36e3796700 1 --2- 172.21.15.45:0/1418284670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 0x7f36dc0cbc20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f36d00098f0 tx=0x7f36d0009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.837+0000 7f36e3796700 1 -- 172.21.15.45:0/1418284670 shutdown_connections 2023-08-27T11:27:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.837+0000 7f36e3796700 1 --2- 172.21.15.45:0/1418284670 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc06ed90 0x7f36dc15e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.837+0000 7f36e3796700 1 --2- 172.21.15.45:0/1418284670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 0x7f36dc0cbc20 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.837+0000 7f36e3796700 1 -- 172.21.15.45:0/1418284670 >> 172.21.15.45:0/1418284670 conn(0x7f36dc0cc7e0 msgr2=0x7f36dc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.837+0000 7f36e3796700 1 -- 172.21.15.45:0/1418284670 shutdown_connections 2023-08-27T11:27:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.837+0000 7f36e3796700 1 -- 172.21.15.45:0/1418284670 wait complete. 2023-08-27T11:27:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.838+0000 7f36e3796700 1 Processor -- start 2023-08-27T11:27:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.838+0000 7f36e3796700 1 -- start start 2023-08-27T11:27:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e3796700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36dc06ed90 0x7f36dc14c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e3796700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc0cd300 0x7f36dc0c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e3796700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 0x7f36dc0ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e3796700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc0ada60 con 0x7f36dc15a0f0 2023-08-27T11:27:22.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e3796700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc0c8790 con 0x7f36dc06ed90 2023-08-27T11:27:22.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e3796700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc0ace50 con 0x7f36dc0cd300 2023-08-27T11:27:22.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e1d33700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 0x7f36dc0ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e1d33700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 0x7f36dc0ae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57634/0 (socket says 172.21.15.45:57634) 2023-08-27T11:27:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e1d33700 1 -- 172.21.15.45:0/3942981610 learned_addr learned my addr 172.21.15.45:0/3942981610 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e1532700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36dc06ed90 0x7f36dc14c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e0d31700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc0cd300 0x7f36dc0c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e1d33700 1 -- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc0cd300 msgr2=0x7f36dc0c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e1d33700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc0cd300 0x7f36dc0c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e1d33700 1 -- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36dc06ed90 msgr2=0x7f36dc14c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e1d33700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36dc06ed90 0x7f36dc14c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.839+0000 7f36e1d33700 1 -- 172.21.15.45:0/3942981610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d0009550 con 0x7f36dc15a0f0 2023-08-27T11:27:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.840+0000 7f36e1d33700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 0x7f36dc0ae380 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f36d400efc0 tx=0x7f36d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.840+0000 7f36cdffb700 1 -- 172.21.15.45:0/3942981610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4009ea0 con 0x7f36dc15a0f0 2023-08-27T11:27:22.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.840+0000 7f36cdffb700 1 -- 172.21.15.45:0/3942981610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d400f040 con 0x7f36dc15a0f0 2023-08-27T11:27:22.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.840+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36dc0ad130 con 0x7f36dc15a0f0 2023-08-27T11:27:22.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.840+0000 7f36cdffb700 1 -- 172.21.15.45:0/3942981610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d400ced0 con 0x7f36dc15a0f0 2023-08-27T11:27:22.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.840+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc0ad6f0 con 0x7f36dc15a0f0 2023-08-27T11:27:22.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.841+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36dc0ad8b0 con 0x7f36dc15a0f0 2023-08-27T11:27:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.841+0000 7f36cdffb700 1 -- 172.21.15.45:0/3942981610 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f36d4014480 con 0x7f36dc15a0f0 2023-08-27T11:27:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.842+0000 7f36cdffb700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f36c006a200 0x7f36c006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.842+0000 7f36cdffb700 1 -- 172.21.15.45:0/3942981610 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f36d4089090 con 0x7f36dc15a0f0 2023-08-27T11:27:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.842+0000 7f36e1532700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f36c006a200 0x7f36c006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.842+0000 7f36e1532700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f36c006a200 0x7f36c006c6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f36d0014040 tx=0x7f36d0008030 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:22.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:22.846+0000 7f36cdffb700 1 -- 172.21.15.45:0/3942981610 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f36d40547d0 con 0x7f36dc15a0f0 2023-08-27T11:27:23.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:23.051+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f36dc0726d0 con 0x7f36dc15a0f0 2023-08-27T11:27:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.045+0000 7f36cdffb700 1 -- 172.21.15.45:0/3942981610 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f36d4057df0 con 0x7f36dc15a0f0 2023-08-27T11:27:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f36c006a200 msgr2=0x7f36c006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f36c006a200 0x7f36c006c6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f36d0014040 tx=0x7f36d0008030 comp rx=0 tx=0).stop 2023-08-27T11:27:24.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 msgr2=0x7f36dc0ae380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 0x7f36dc0ae380 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f36d400efc0 tx=0x7f36d400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 shutdown_connections 2023-08-27T11:27:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f36c006a200 0x7f36c006c6c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36dc15a0f0 0x7f36dc0ae380 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36dc06ed90 0x7f36dc14c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 --2- 172.21.15.45:0/3942981610 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36dc0cd300 0x7f36dc0c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.048+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 >> 172.21.15.45:0/3942981610 conn(0x7f36dc0cc7e0 msgr2=0x7f36dc0cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:24.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.049+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 shutdown_connections 2023-08-27T11:27:24.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.049+0000 7f36e3796700 1 -- 172.21.15.45:0/3942981610 wait complete. 2023-08-27T11:27:24.056 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-08-27T11:27:24.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:27:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.209+0000 7fb4acd03700 1 Processor -- start 2023-08-27T11:27:24.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.209+0000 7fb4acd03700 1 -- start start 2023-08-27T11:27:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.210+0000 7fb4acd03700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb4a80caa00 0x7fb4a80cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.210+0000 7fb4acd03700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a80ce340 con 0x7fb4a80af200 2023-08-27T11:27:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.210+0000 7fb4acd03700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a80ceec0 con 0x7fb4a80ca620 2023-08-27T11:27:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.210+0000 7fb4acd03700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a815d7c0 con 0x7fb4a80caa00 2023-08-27T11:27:24.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.210+0000 7fb4a5d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb4a80af200 0x7fb4a80acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44416/0 (socket says 172.21.15.45:44416) 2023-08-27T11:27:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.210+0000 7fb4a5d9b700 1 -- 172.21.15.45:0/3771756315 learned_addr learned my addr 172.21.15.45:0/3771756315 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.210+0000 7fb4a6d9d700 1 --2- 172.21.15.45:0/3771756315 >> v2:172.21.15.135:3300/0 conn(0x7fb4a80caa00 0x7fb4a80cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.210+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515224158 0 0) 0x7fb4a80ce340 con 0x7fb4a80af200 2023-08-27T11:27:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.210+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4900035b0 con 0x7fb4a80af200 2023-08-27T11:27:24.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2444762408 0 0) 0x7fb4900035b0 con 0x7fb4a80af200 2023-08-27T11:27:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 >> v2:172.21.15.135:3300/0 conn(0x7fb4a80caa00 msgr2=0x7fb4a80cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:24.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 --2- 172.21.15.45:0/3771756315 >> v2:172.21.15.135:3300/0 conn(0x7fb4a80caa00 0x7fb4a80cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 >> v1:172.21.15.135:6789/0 conn(0x7fb4a80ca620 legacy=0x7fb4a80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a80cc570 con 0x7fb4a80af200 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fb498001670 con 0x7fb4a80af200 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 >> v1:172.21.15.45:6789/0 conn(0x7fb4a80af200 legacy=0x7fb4a80acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb490003a00 0x7fb490003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb490004850 0x7fb490004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490005210 0x7fb49000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a815d7c0 con 0x7fb490003a00 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49000c0b0 con 0x7fb490005210 2023-08-27T11:27:24.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49000c270 con 0x7fb490004850 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a659c700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb490003a00 0x7fb490003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a5d9b700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb490004850 0x7fb490004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a6d9d700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490005210 0x7fb49000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a659c700 1 -- 172.21.15.45:0/3771756315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb490004850 msgr2=0x7fb490004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a659c700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb490004850 0x7fb490004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a659c700 1 -- 172.21.15.45:0/3771756315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490005210 msgr2=0x7fb49000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a659c700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490005210 0x7fb49000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a659c700 1 -- 172.21.15.45:0/3771756315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900035b0 con 0x7fb490003a00 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a6d9d700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490005210 0x7fb49000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a6d9d700 1 -- 172.21.15.45:0/3771756315 reap_dead start 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a659c700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb490003a00 0x7fb490003e20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0017c0 tx=0x7fb490005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc960228868c7dc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.211+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0052d0 con 0x7fb490003a00 2023-08-27T11:27:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.212+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c005430 con 0x7fb490003a00 2023-08-27T11:27:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.212+0000 7fb4a559a700 1 -- 172.21.15.45:0/3771756315 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0038c0 con 0x7fb490003a00 2023-08-27T11:27:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.212+0000 7fb4acd03700 1 -- 172.21.15.45:0/3771756315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb490003a00 msgr2=0x7fb490003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.212+0000 7fb4acd03700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb490003a00 0x7fb490003e20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0017c0 tx=0x7fb490005880 comp rx=0 tx=0).stop 2023-08-27T11:27:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.212+0000 7fb4acd03700 1 -- 172.21.15.45:0/3771756315 shutdown_connections 2023-08-27T11:27:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.212+0000 7fb4acd03700 1 --2- 172.21.15.45:0/3771756315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb490003a00 0x7fb490003e20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.212+0000 7fb4acd03700 1 -- 172.21.15.45:0/3771756315 >> 172.21.15.45:0/3771756315 conn(0x7fb4a80cc7e0 msgr2=0x7fb4a80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.212+0000 7fb4acd03700 1 -- 172.21.15.45:0/3771756315 shutdown_connections 2023-08-27T11:27:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.212+0000 7fb4acd03700 1 -- 172.21.15.45:0/3771756315 wait complete. 2023-08-27T11:27:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4acd03700 1 Processor -- start 2023-08-27T11:27:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4acd03700 1 -- start start 2023-08-27T11:27:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4acd03700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490003a00 0x7fb4a81557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4acd03700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4a80c6e30 0x7fb4a806e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4acd03700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a80c72b0 0x7fb4a806ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4acd03700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80c4950 con 0x7fb4a80c6e30 2023-08-27T11:27:24.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4acd03700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80c57b0 con 0x7fb490003a00 2023-08-27T11:27:24.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4acd03700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80c5c40 con 0x7fb4a80c72b0 2023-08-27T11:27:24.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4a5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4a80c6e30 0x7fb4a806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4a5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4a80c6e30 0x7fb4a806e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57652/0 (socket says 172.21.15.45:57652) 2023-08-27T11:27:24.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.214+0000 7fb4a5d9b700 1 -- 172.21.15.45:0/3466693492 learned_addr learned my addr 172.21.15.45:0/3466693492 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:24.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4a659c700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490003a00 0x7fb4a81557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4a6d9d700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a80c72b0 0x7fb4a806ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4a5d9b700 1 -- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a80c72b0 msgr2=0x7fb4a806ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4a5d9b700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a80c72b0 0x7fb4a806ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4a5d9b700 1 -- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490003a00 msgr2=0x7fb4a81557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4a5d9b700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490003a00 0x7fb4a81557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4a5d9b700 1 -- 172.21.15.45:0/3466693492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a8155d30 con 0x7fb4a80c6e30 2023-08-27T11:27:24.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4a6d9d700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a80c72b0 0x7fb4a806ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4a5d9b700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4a80c6e30 0x7fb4a806e530 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb498007290 tx=0x7fb498007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb496ffd700 1 -- 172.21.15.45:0/3466693492 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498001450 con 0x7fb4a80c6e30 2023-08-27T11:27:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb496ffd700 1 -- 172.21.15.45:0/3466693492 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49800f850 con 0x7fb4a80c6e30 2023-08-27T11:27:24.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a80c9c90 con 0x7fb4a80c6e30 2023-08-27T11:27:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb496ffd700 1 -- 172.21.15.45:0/3466693492 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49800fc00 con 0x7fb4a80c6e30 2023-08-27T11:27:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.215+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a80c4e80 con 0x7fb4a80c6e30 2023-08-27T11:27:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.216+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a80cbce0 con 0x7fb4a80c6e30 2023-08-27T11:27:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.217+0000 7fb496ffd700 1 -- 172.21.15.45:0/3466693492 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb49800fd60 con 0x7fb4a80c6e30 2023-08-27T11:27:24.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.217+0000 7fb496ffd700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb48806a200 0x7fb48806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.217+0000 7fb496ffd700 1 -- 172.21.15.45:0/3466693492 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5944+0+0 (secure 0 0 0) 0x7fb498088760 con 0x7fb4a80c6e30 2023-08-27T11:27:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.217+0000 7fb4a659c700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb48806a200 0x7fb48806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.218+0000 7fb4a659c700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb48806a200 0x7fb48806c6c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb49c000c00 tx=0x7fb49c009030 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:24.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.222+0000 7fb496ffd700 1 -- 172.21.15.45:0/3466693492 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb498053d30 con 0x7fb4a80c6e30 2023-08-27T11:27:24.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:24.422+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb4a80734a0 con 0x7fb4a80c6e30 2023-08-27T11:27:25.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.055+0000 7fb496ffd700 1 -- 172.21.15.45:0/3466693492 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb498057350 con 0x7fb4a80c6e30 2023-08-27T11:27:25.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.060+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb48806a200 msgr2=0x7fb48806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.060+0000 7fb4acd03700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb48806a200 0x7fb48806c6c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb49c000c00 tx=0x7fb49c009030 comp rx=0 tx=0).stop 2023-08-27T11:27:25.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.060+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4a80c6e30 msgr2=0x7fb4a806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.060+0000 7fb4acd03700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4a80c6e30 0x7fb4a806e530 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb498007290 tx=0x7fb498007650 comp rx=0 tx=0).stop 2023-08-27T11:27:25.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.061+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 shutdown_connections 2023-08-27T11:27:25.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.061+0000 7fb4acd03700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb48806a200 0x7fb48806c6c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.061+0000 7fb4acd03700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb4a80c72b0 0x7fb4a806ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.061+0000 7fb4acd03700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb490003a00 0x7fb4a81557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.061+0000 7fb4acd03700 1 --2- 172.21.15.45:0/3466693492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4a80c6e30 0x7fb4a806e530 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.061+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 >> 172.21.15.45:0/3466693492 conn(0x7fb4a80cc7e0 msgr2=0x7fb4a80ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.063+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 shutdown_connections 2023-08-27T11:27:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.065+0000 7fb4acd03700 1 -- 172.21.15.45:0/3466693492 wait complete. 2023-08-27T11:27:25.071 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-08-27T11:27:25.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:27:25.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.216+0000 7f4a4c1a8700 1 Processor -- start 2023-08-27T11:27:25.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.217+0000 7f4a4c1a8700 1 -- start start 2023-08-27T11:27:25.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.217+0000 7f4a4c1a8700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4a4406c260 0x7f4a440cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.217+0000 7f4a4c1a8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a440d04f0 con 0x7f4a440b02c0 2023-08-27T11:27:25.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.217+0000 7f4a4c1a8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a440b06a0 con 0x7f4a440cdd40 2023-08-27T11:27:25.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.217+0000 7f4a4c1a8700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a440ce1d0 con 0x7f4a4406c260 2023-08-27T11:27:25.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.217+0000 7f4a49f44700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4a440b02c0 0x7f4a440cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44434/0 (socket says 172.21.15.45:44434) 2023-08-27T11:27:25.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.217+0000 7f4a49f44700 1 -- 172.21.15.45:0/2797559666 learned_addr learned my addr 172.21.15.45:0/2797559666 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:25.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a49743700 1 --2- 172.21.15.45:0/2797559666 >> v2:172.21.15.135:3300/0 conn(0x7f4a4406c260 0x7f4a440cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288901214 0 0) 0x7f4a440d04f0 con 0x7f4a440b02c0 2023-08-27T11:27:25.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a300035b0 con 0x7f4a440b02c0 2023-08-27T11:27:25.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1177067726 0 0) 0x7f4a300035b0 con 0x7f4a440b02c0 2023-08-27T11:27:25.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 >> v2:172.21.15.135:3300/0 conn(0x7f4a4406c260 msgr2=0x7f4a440cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:25.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 --2- 172.21.15.45:0/2797559666 >> v2:172.21.15.135:3300/0 conn(0x7f4a4406c260 0x7f4a440cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 >> v1:172.21.15.135:6789/0 conn(0x7f4a440cdd40 legacy=0x7f4a4415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440cf8e0 con 0x7f4a440b02c0 2023-08-27T11:27:25.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f4a34001670 con 0x7f4a440b02c0 2023-08-27T11:27:25.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 >> v1:172.21.15.45:6789/0 conn(0x7f4a440b02c0 legacy=0x7f4a440cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a30003a00 0x7f4a30003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a30004850 0x7f4a30004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a30005210 0x7f4a3000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440ce1d0 con 0x7f4a30004850 2023-08-27T11:27:25.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000c0b0 con 0x7f4a30005210 2023-08-27T11:27:25.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000c270 con 0x7f4a30003a00 2023-08-27T11:27:25.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a49743700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a30004850 0x7f4a30004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a49f44700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a30003a00 0x7f4a30003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.218+0000 7f4a4a745700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a30005210 0x7f4a3000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a49743700 1 -- 172.21.15.45:0/2797559666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a30003a00 msgr2=0x7f4a30003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:25.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a49743700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a30003a00 0x7f4a30003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a49743700 1 -- 172.21.15.45:0/2797559666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a30005210 msgr2=0x7f4a3000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:25.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a49743700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a30005210 0x7f4a3000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a49743700 1 -- 172.21.15.45:0/2797559666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a300035b0 con 0x7f4a30004850 2023-08-27T11:27:25.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a4a745700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a30005210 0x7f4a3000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:25.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a4a745700 1 -- 172.21.15.45:0/2797559666 reap_dead start 2023-08-27T11:27:25.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a49743700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a30004850 0x7f4a30004cd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009b60 tx=0x7f4a30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c06691f61a5e001 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000e6c0 con 0x7f4a30004850 2023-08-27T11:27:25.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000e820 con 0x7f4a30004850 2023-08-27T11:27:25.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a48f42700 1 -- 172.21.15.45:0/2797559666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000eb90 con 0x7f4a30004850 2023-08-27T11:27:25.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/2797559666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a30004850 msgr2=0x7f4a30004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a4c1a8700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a30004850 0x7f4a30004cd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009b60 tx=0x7f4a30005880 comp rx=0 tx=0).stop 2023-08-27T11:27:25.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/2797559666 shutdown_connections 2023-08-27T11:27:25.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a4c1a8700 1 --2- 172.21.15.45:0/2797559666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a30004850 0x7f4a30004cd0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.219+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/2797559666 >> 172.21.15.45:0/2797559666 conn(0x7f4a4409a1a0 msgr2=0x7f4a440d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:25.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.220+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/2797559666 shutdown_connections 2023-08-27T11:27:25.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.220+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/2797559666 wait complete. 2023-08-27T11:27:25.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.221+0000 7f4a4c1a8700 1 Processor -- start 2023-08-27T11:27:25.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.221+0000 7f4a4c1a8700 1 -- start start 2023-08-27T11:27:25.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a4c1a8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a30004850 0x7f4a440c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a4c1a8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4409e680 0x7f4a441f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a4c1a8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a44158220 0x7f4a440cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a4c1a8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a441f39c0 con 0x7f4a4409e680 2023-08-27T11:27:25.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a4c1a8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a44159660 con 0x7f4a44158220 2023-08-27T11:27:25.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a4c1a8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a44158d70 con 0x7f4a30004850 2023-08-27T11:27:25.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a49743700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4409e680 0x7f4a441f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a49743700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4409e680 0x7f4a441f3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57674/0 (socket says 172.21.15.45:57674) 2023-08-27T11:27:25.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a49743700 1 -- 172.21.15.45:0/1365622495 learned_addr learned my addr 172.21.15.45:0/1365622495 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:25.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a49f44700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a30004850 0x7f4a440c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.222+0000 7f4a4a745700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a44158220 0x7f4a440cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a49743700 1 -- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a30004850 msgr2=0x7f4a440c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a49743700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a30004850 0x7f4a440c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a49743700 1 -- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a44158220 msgr2=0x7f4a440cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a49743700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a44158220 0x7f4a440cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a49743700 1 -- 172.21.15.45:0/1365622495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4415f8d0 con 0x7f4a4409e680 2023-08-27T11:27:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a49743700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4409e680 0x7f4a441f3480 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c080 tx=0x7f4a400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a3a7fc700 1 -- 172.21.15.45:0/1365622495 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000e6c0 con 0x7f4a4409e680 2023-08-27T11:27:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a3a7fc700 1 -- 172.21.15.45:0/1365622495 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a400038c0 con 0x7f4a4409e680 2023-08-27T11:27:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a44159010 con 0x7f4a4409e680 2023-08-27T11:27:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a3a7fc700 1 -- 172.21.15.45:0/1365622495 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40003c70 con 0x7f4a4409e680 2023-08-27T11:27:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.223+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4409eb40 con 0x7f4a4409e680 2023-08-27T11:27:25.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.224+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a441597d0 con 0x7f4a4409e680 2023-08-27T11:27:25.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.225+0000 7f4a3a7fc700 1 -- 172.21.15.45:0/1365622495 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4a40013470 con 0x7f4a4409e680 2023-08-27T11:27:25.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.225+0000 7f4a3a7fc700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2806a200 0x7f4a2806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:25.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.225+0000 7f4a3a7fc700 1 -- 172.21.15.45:0/1365622495 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f4a4008a3f0 con 0x7f4a4409e680 2023-08-27T11:27:25.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.225+0000 7f4a49f44700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2806a200 0x7f4a2806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:25.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.226+0000 7f4a49f44700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2806a200 0x7f4a2806c6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a34000c00 tx=0x7f4a34000f80 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:25.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.229+0000 7f4a3a7fc700 1 -- 172.21.15.45:0/1365622495 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4a40055a50 con 0x7f4a4409e680 2023-08-27T11:27:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:25.477+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4a440734a0 con 0x7f4a4409e680 2023-08-27T11:27:26.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.060+0000 7f4a3a7fc700 1 -- 172.21.15.45:0/1365622495 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v30) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4a40059070 con 0x7f4a4409e680 2023-08-27T11:27:26.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.070+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2806a200 msgr2=0x7f4a2806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.070+0000 7f4a4c1a8700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2806a200 0x7f4a2806c6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a34000c00 tx=0x7f4a34000f80 comp rx=0 tx=0).stop 2023-08-27T11:27:26.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.070+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4409e680 msgr2=0x7f4a441f3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.070+0000 7f4a4c1a8700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4409e680 0x7f4a441f3480 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c080 tx=0x7f4a400047c0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.071+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 shutdown_connections 2023-08-27T11:27:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.071+0000 7f4a4c1a8700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2806a200 0x7f4a2806c6c0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4a34000c00 tx=0x7f4a34000f80 comp rx=0 tx=0).stop 2023-08-27T11:27:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.071+0000 7f4a4c1a8700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a44158220 0x7f4a440cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.071+0000 7f4a4c1a8700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a30004850 0x7f4a440c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.071+0000 7f4a4c1a8700 1 --2- 172.21.15.45:0/1365622495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4409e680 0x7f4a441f3480 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.071+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 >> 172.21.15.45:0/1365622495 conn(0x7f4a4409a1a0 msgr2=0x7f4a440cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:26.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.073+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 shutdown_connections 2023-08-27T11:27:26.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.073+0000 7f4a4c1a8700 1 -- 172.21.15.45:0/1365622495 wait complete. 2023-08-27T11:27:26.086 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 10 and data pool 11 2023-08-27T11:27:26.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:26.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.226+0000 7f3de746b700 1 Processor -- start 2023-08-27T11:27:26.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.226+0000 7f3de746b700 1 -- start start 2023-08-27T11:27:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.227+0000 7f3de746b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3de015b070 0x7f3de00cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.227+0000 7f3de746b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3de00cdf60 0x7f3de006e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.227+0000 7f3de746b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de006cec0 0x7f3de006e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.227+0000 7f3de746b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de015d8e0 con 0x7f3de00cdf60 2023-08-27T11:27:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.227+0000 7f3de746b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de015dad0 con 0x7f3de015b070 2023-08-27T11:27:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.227+0000 7f3de746b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015ddb0 con 0x7f3de006cec0 2023-08-27T11:27:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.227+0000 7f3de4a06700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3de00cdf60 0x7f3de006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.227+0000 7f3de4a06700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3de00cdf60 0x7f3de006e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57678/0 (socket says 172.21.15.45:57678) 2023-08-27T11:27:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.227+0000 7f3de4a06700 1 -- 172.21.15.45:0/2982710204 learned_addr learned my addr 172.21.15.45:0/2982710204 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3de5207700 1 --2- 172.21.15.45:0/2982710204 >> v2:172.21.15.135:3300/0 conn(0x7f3de015b070 0x7f3de00cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3de5a08700 1 --2- 172.21.15.45:0/2982710204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de006cec0 0x7f3de006e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3de4a06700 1 -- 172.21.15.45:0/2982710204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de006cec0 msgr2=0x7f3de006e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3de4a06700 1 --2- 172.21.15.45:0/2982710204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de006cec0 0x7f3de006e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3de4a06700 1 -- 172.21.15.45:0/2982710204 >> v2:172.21.15.135:3300/0 conn(0x7f3de015b070 msgr2=0x7f3de00cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:26.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3de4a06700 1 --2- 172.21.15.45:0/2982710204 >> v2:172.21.15.135:3300/0 conn(0x7f3de015b070 0x7f3de00cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3de4a06700 1 -- 172.21.15.45:0/2982710204 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de015e090 con 0x7f3de00cdf60 2023-08-27T11:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3de5207700 1 --2- 172.21.15.45:0/2982710204 >> v2:172.21.15.135:3300/0 conn(0x7f3de015b070 0x7f3de00cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3de4a06700 1 --2- 172.21.15.45:0/2982710204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cdf60 0x7f3de006e270 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3de006ef10 tx=0x7f3dd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88423e3548fd71a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3dd3fff700 1 -- 172.21.15.45:0/2982710204 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400b870 con 0x7f3de00cdf60 2023-08-27T11:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3dd3fff700 1 -- 172.21.15.45:0/2982710204 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd400b9d0 con 0x7f3de00cdf60 2023-08-27T11:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.228+0000 7f3dd3fff700 1 -- 172.21.15.45:0/2982710204 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400bd60 con 0x7f3de00cdf60 2023-08-27T11:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.229+0000 7f3de746b700 1 -- 172.21.15.45:0/2982710204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cdf60 msgr2=0x7f3de006e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.229+0000 7f3de746b700 1 --2- 172.21.15.45:0/2982710204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cdf60 0x7f3de006e270 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3de006ef10 tx=0x7f3dd4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.229+0000 7f3de746b700 1 -- 172.21.15.45:0/2982710204 shutdown_connections 2023-08-27T11:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.229+0000 7f3de746b700 1 --2- 172.21.15.45:0/2982710204 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de006cec0 0x7f3de006e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.229+0000 7f3de746b700 1 --2- 172.21.15.45:0/2982710204 >> v2:172.21.15.135:3300/0 conn(0x7f3de015b070 0x7f3de00cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.229+0000 7f3de746b700 1 --2- 172.21.15.45:0/2982710204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cdf60 0x7f3de006e270 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.229+0000 7f3de746b700 1 -- 172.21.15.45:0/2982710204 >> 172.21.15.45:0/2982710204 conn(0x7f3de00cfe50 msgr2=0x7f3de00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.229+0000 7f3de746b700 1 -- 172.21.15.45:0/2982710204 shutdown_connections 2023-08-27T11:27:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.229+0000 7f3de746b700 1 -- 172.21.15.45:0/2982710204 wait complete. 2023-08-27T11:27:26.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.230+0000 7f3de746b700 1 Processor -- start 2023-08-27T11:27:26.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.230+0000 7f3de746b700 1 -- start start 2023-08-27T11:27:26.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de746b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de006cec0 0x7f3de00c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de746b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de00cdf60 0x7f3de00b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de746b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de015b070 0x7f3de01f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de746b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0144ae0 con 0x7f3de006cec0 2023-08-27T11:27:26.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de746b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0145440 con 0x7f3de00cdf60 2023-08-27T11:27:26.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de746b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0145720 con 0x7f3de015b070 2023-08-27T11:27:26.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5207700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de006cec0 0x7f3de00c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5207700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de006cec0 0x7f3de00c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57684/0 (socket says 172.21.15.45:57684) 2023-08-27T11:27:26.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5207700 1 -- 172.21.15.45:0/266351944 learned_addr learned my addr 172.21.15.45:0/266351944 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:26.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de4a06700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de00cdf60 0x7f3de00b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5a08700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de015b070 0x7f3de01f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5207700 1 -- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de015b070 msgr2=0x7f3de01f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5207700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de015b070 0x7f3de01f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5207700 1 -- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de00cdf60 msgr2=0x7f3de00b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5207700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de00cdf60 0x7f3de00b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5207700 1 -- 172.21.15.45:0/266351944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4009550 con 0x7f3de006cec0 2023-08-27T11:27:26.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5a08700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de015b070 0x7f3de01f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de5207700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de006cec0 0x7f3de00c6390 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00e760 tx=0x7f3ddc00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3dd1ffb700 1 -- 172.21.15.45:0/266351944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc013610 con 0x7f3de006cec0 2023-08-27T11:27:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3dd1ffb700 1 -- 172.21.15.45:0/266351944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc013770 con 0x7f3de006cec0 2023-08-27T11:27:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de0145a00 con 0x7f3de006cec0 2023-08-27T11:27:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.231+0000 7f3dd1ffb700 1 -- 172.21.15.45:0/266351944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc013b20 con 0x7f3de006cec0 2023-08-27T11:27:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.232+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de0145fc0 con 0x7f3de006cec0 2023-08-27T11:27:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.233+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de0146330 con 0x7f3de006cec0 2023-08-27T11:27:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.233+0000 7f3dd1ffb700 1 -- 172.21.15.45:0/266351944 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3ddc013c80 con 0x7f3de006cec0 2023-08-27T11:27:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.234+0000 7f3dd1ffb700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f3dc406a200 0x7f3dc406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.234+0000 7f3dd1ffb700 1 -- 172.21.15.45:0/266351944 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f3ddc0884b0 con 0x7f3de006cec0 2023-08-27T11:27:26.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.234+0000 7f3de4a06700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f3dc406a200 0x7f3dc406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.234+0000 7f3de4a06700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f3dc406a200 0x7f3dc406c6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3de006ef10 tx=0x7f3dd4008030 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.238+0000 7f3dd1ffb700 1 -- 172.21.15.45:0/266351944 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ddc0572a0 con 0x7f3de006cec0 2023-08-27T11:27:26.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.439+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3de015e340 con 0x7f3de006cec0 2023-08-27T11:27:26.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.440+0000 7f3dd1ffb700 1 -- 172.21.15.45:0/266351944 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+18650 (secure 0 0 0) 0x7f3ddc056e30 con 0x7f3de006cec0 2023-08-27T11:27:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:26.441 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:25.478781+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:23.082458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:24.453679+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:26.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f3dc406a200 msgr2=0x7f3dc406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f3dc406a200 0x7f3dc406c6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3de006ef10 tx=0x7f3dd4008030 comp rx=0 tx=0).stop 2023-08-27T11:27:26.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de006cec0 msgr2=0x7f3de00c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de006cec0 0x7f3de00c6390 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00e760 tx=0x7f3ddc00ea70 comp rx=0 tx=0).stop 2023-08-27T11:27:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 shutdown_connections 2023-08-27T11:27:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f3dc406a200 0x7f3dc406c6c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de006cec0 0x7f3de00c6390 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de015b070 0x7f3de01f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 --2- 172.21.15.45:0/266351944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de00cdf60 0x7f3de00b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 >> 172.21.15.45:0/266351944 conn(0x7f3de00cfe50 msgr2=0x7f3de006e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:26.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 shutdown_connections 2023-08-27T11:27:26.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.442+0000 7f3de746b700 1 -- 172.21.15.45:0/266351944 wait complete. 2023-08-27T11:27:26.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:26.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.579+0000 7f8befb94700 1 Processor -- start 2023-08-27T11:27:26.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.579+0000 7f8befb94700 1 -- start start 2023-08-27T11:27:26.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.580+0000 7f8befb94700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8be80d0440 0x7f8be8168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.580+0000 7f8befb94700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be80ce1c0 con 0x7f8be815d450 2023-08-27T11:27:26.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.580+0000 7f8befb94700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be80cf290 con 0x7f8be80b2140 2023-08-27T11:27:26.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.580+0000 7f8befb94700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be80ce440 con 0x7f8be80d0440 2023-08-27T11:27:26.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.580+0000 7f8bed930700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8be815d450 0x7f8be80cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44468/0 (socket says 172.21.15.45:44468) 2023-08-27T11:27:26.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.580+0000 7f8bed930700 1 -- 172.21.15.45:0/500970388 learned_addr learned my addr 172.21.15.45:0/500970388 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:26.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.580+0000 7f8bee131700 1 --2- 172.21.15.45:0/500970388 >> v2:172.21.15.135:3300/0 conn(0x7f8be80d0440 0x7f8be8168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.580+0000 7f8bec92e700 1 -- 172.21.15.45:0/500970388 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754776851 0 0) 0x7f8be80ce1c0 con 0x7f8be815d450 2023-08-27T11:27:26.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.580+0000 7f8bec92e700 1 -- 172.21.15.45:0/500970388 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bd40035b0 con 0x7f8be815d450 2023-08-27T11:27:26.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.581+0000 7f8bee131700 1 -- 172.21.15.45:0/500970388 >> v1:172.21.15.135:6789/0 conn(0x7f8be80b2140 legacy=0x7f8be80cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.581+0000 7f8bec92e700 1 -- 172.21.15.45:0/500970388 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119086606 0 0) 0x7f8be80cf290 con 0x7f8be80b2140 2023-08-27T11:27:26.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.581+0000 7f8bee131700 1 -- 172.21.15.45:0/500970388 >> v1:172.21.15.45:6789/0 conn(0x7f8be815d450 legacy=0x7f8be80cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.581+0000 7f8bee131700 1 -- 172.21.15.45:0/500970388 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be80ce720 con 0x7f8be80d0440 2023-08-27T11:27:26.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.581+0000 7f8bee131700 1 --2- 172.21.15.45:0/500970388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 0x7f8be8168580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009d90 tx=0x7f8be000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac517a18810a9287 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.581+0000 7f8bec92e700 1 -- 172.21.15.45:0/500970388 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be000e070 con 0x7f8be80d0440 2023-08-27T11:27:26.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.581+0000 7f8bec92e700 1 -- 172.21.15.45:0/500970388 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0003a20 con 0x7f8be80d0440 2023-08-27T11:27:26.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.581+0000 7f8bec92e700 1 -- 172.21.15.45:0/500970388 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8001670 con 0x7f8be80d0440 2023-08-27T11:27:26.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.582+0000 7f8befb94700 1 -- 172.21.15.45:0/500970388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 msgr2=0x7f8be8168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.582+0000 7f8befb94700 1 --2- 172.21.15.45:0/500970388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 0x7f8be8168580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009d90 tx=0x7f8be000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.582+0000 7f8befb94700 1 -- 172.21.15.45:0/500970388 shutdown_connections 2023-08-27T11:27:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.582+0000 7f8befb94700 1 --2- 172.21.15.45:0/500970388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 0x7f8be8168580 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.582+0000 7f8befb94700 1 -- 172.21.15.45:0/500970388 >> 172.21.15.45:0/500970388 conn(0x7f8be80d0f60 msgr2=0x7f8be80d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.582+0000 7f8befb94700 1 -- 172.21.15.45:0/500970388 shutdown_connections 2023-08-27T11:27:26.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.582+0000 7f8befb94700 1 -- 172.21.15.45:0/500970388 wait complete. 2023-08-27T11:27:26.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.583+0000 7f8befb94700 1 Processor -- start 2023-08-27T11:27:26.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.583+0000 7f8befb94700 1 -- start start 2023-08-27T11:27:26.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8befb94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8be80b2140 0x7f8be81502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8befb94700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 0x7f8be809ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8befb94700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8be815d450 0x7f8be809b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8befb94700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be8150800 con 0x7f8be80b2140 2023-08-27T11:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8befb94700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be80b5200 con 0x7f8be80d0440 2023-08-27T11:27:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8befb94700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be80b54e0 con 0x7f8be815d450 2023-08-27T11:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed930700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8be80b2140 0x7f8be81502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed930700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8be80b2140 0x7f8be81502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57698/0 (socket says 172.21.15.45:57698) 2023-08-27T11:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed930700 1 -- 172.21.15.45:0/1887332299 learned_addr learned my addr 172.21.15.45:0/1887332299 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed12f700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 0x7f8be809ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bee131700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8be815d450 0x7f8be809b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed930700 1 -- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8be815d450 msgr2=0x7f8be809b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed930700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8be815d450 0x7f8be809b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed930700 1 -- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 msgr2=0x7f8be809ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed930700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 0x7f8be809ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed930700 1 -- 172.21.15.45:0/1887332299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be0009a70 con 0x7f8be80b2140 2023-08-27T11:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed12f700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 0x7f8be809ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bee131700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8be815d450 0x7f8be809b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bed930700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8be80b2140 0x7f8be81502c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80011e0 tx=0x7f8bd800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bde7fc700 1 -- 172.21.15.45:0/1887332299 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8004c80 con 0x7f8be80b2140 2023-08-27T11:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bde7fc700 1 -- 172.21.15.45:0/1887332299 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd8004de0 con 0x7f8be80b2140 2023-08-27T11:27:26.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be80b57c0 con 0x7f8be80b2140 2023-08-27T11:27:26.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.584+0000 7f8bde7fc700 1 -- 172.21.15.45:0/1887332299 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80072f0 con 0x7f8be80b2140 2023-08-27T11:27:26.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.585+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be80b5d80 con 0x7f8be80b2140 2023-08-27T11:27:26.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.585+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be809ee50 con 0x7f8be80b2140 2023-08-27T11:27:26.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.586+0000 7f8bde7fc700 1 -- 172.21.15.45:0/1887332299 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8bd8007450 con 0x7f8be80b2140 2023-08-27T11:27:26.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.586+0000 7f8bde7fc700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8bcc06a200 0x7f8bcc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.586+0000 7f8bde7fc700 1 -- 172.21.15.45:0/1887332299 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8bd8087b00 con 0x7f8be80b2140 2023-08-27T11:27:26.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.586+0000 7f8bed12f700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8bcc06a200 0x7f8bcc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.587+0000 7f8bed12f700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8bcc06a200 0x7f8bcc06c6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8be4000f60 tx=0x7f8be4009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.591+0000 7f8bde7fc700 1 -- 172.21.15.45:0/1887332299 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8bd8052f70 con 0x7f8be80b2140 2023-08-27T11:27:26.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.791+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8be80b6ee0 con 0x7f8be80b2140 2023-08-27T11:27:26.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.792+0000 7f8bde7fc700 1 -- 172.21.15.45:0/1887332299 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+18650 (secure 0 0 0) 0x7f8bd8018020 con 0x7f8be80b2140 2023-08-27T11:27:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:26.793 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:25.478781+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:23.082458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:24.453679+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8bcc06a200 msgr2=0x7f8bcc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8bcc06a200 0x7f8bcc06c6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8be4000f60 tx=0x7f8be4009040 comp rx=0 tx=0).stop 2023-08-27T11:27:26.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8be80b2140 msgr2=0x7f8be81502c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8be80b2140 0x7f8be81502c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80011e0 tx=0x7f8bd800cae0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 shutdown_connections 2023-08-27T11:27:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f8bcc06a200 0x7f8bcc06c6c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8be80b2140 0x7f8be81502c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8be815d450 0x7f8be809b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 --2- 172.21.15.45:0/1887332299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8be80d0440 0x7f8be809ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 >> 172.21.15.45:0/1887332299 conn(0x7f8be80d0f60 msgr2=0x7f8be80d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 shutdown_connections 2023-08-27T11:27:26.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.794+0000 7f8befb94700 1 -- 172.21.15.45:0/1887332299 wait complete. 2023-08-27T11:27:26.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.935+0000 7f408fc83700 1 Processor -- start 2023-08-27T11:27:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.935+0000 7f408fc83700 1 -- start start 2023-08-27T11:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408fc83700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f408806a990 0x7f408815f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408fc83700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f408806cf40 0x7f40880ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408fc83700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408815ecb0 con 0x7f408815b200 2023-08-27T11:27:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408fc83700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408815c5f0 con 0x7f408806cf40 2023-08-27T11:27:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408fc83700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408806d4b0 con 0x7f408806a990 2023-08-27T11:27:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408e220700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f408806cf40 0x7f40880ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408e220700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f408806cf40 0x7f40880ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57710/0 (socket says 172.21.15.45:57710) 2023-08-27T11:27:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408e220700 1 -- 172.21.15.45:0/4006326905 learned_addr learned my addr 172.21.15.45:0/4006326905 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408da1f700 1 --1- 172.21.15.45:0/4006326905 >> v1:172.21.15.45:6789/0 conn(0x7f408815b200 0x7f408806a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44482/0 (socket says 172.21.15.45:44482) 2023-08-27T11:27:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408d21e700 1 --2- 172.21.15.45:0/4006326905 >> v2:172.21.15.135:3300/0 conn(0x7f408806a990 0x7f408815f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408e220700 1 -- 172.21.15.45:0/4006326905 >> v2:172.21.15.135:3300/0 conn(0x7f408806a990 msgr2=0x7f408815f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408e220700 1 --2- 172.21.15.45:0/4006326905 >> v2:172.21.15.135:3300/0 conn(0x7f408806a990 0x7f408815f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408e220700 1 -- 172.21.15.45:0/4006326905 >> v1:172.21.15.45:6789/0 conn(0x7f408815b200 legacy=0x7f408806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408e220700 1 -- 172.21.15.45:0/4006326905 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408806d790 con 0x7f408806cf40 2023-08-27T11:27:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408d21e700 1 --2- 172.21.15.45:0/4006326905 >> v2:172.21.15.135:3300/0 conn(0x7f408806a990 0x7f408815f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.936+0000 7f408e220700 1 --2- 172.21.15.45:0/4006326905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 0x7f40880ca3b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f40880af810 tx=0x7f408000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97e3af196a9595b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408ca1d700 1 -- 172.21.15.45:0/4006326905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408000e070 con 0x7f408806cf40 2023-08-27T11:27:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408ca1d700 1 -- 172.21.15.45:0/4006326905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408000b9c0 con 0x7f408806cf40 2023-08-27T11:27:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408ca1d700 1 -- 172.21.15.45:0/4006326905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408000bd70 con 0x7f408806cf40 2023-08-27T11:27:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408fc83700 1 -- 172.21.15.45:0/4006326905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 msgr2=0x7f40880ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408fc83700 1 --2- 172.21.15.45:0/4006326905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 0x7f40880ca3b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f40880af810 tx=0x7f408000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408fc83700 1 -- 172.21.15.45:0/4006326905 shutdown_connections 2023-08-27T11:27:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408fc83700 1 --2- 172.21.15.45:0/4006326905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 0x7f40880ca3b0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408fc83700 1 --2- 172.21.15.45:0/4006326905 >> v2:172.21.15.135:3300/0 conn(0x7f408806a990 0x7f408815f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408fc83700 1 -- 172.21.15.45:0/4006326905 >> 172.21.15.45:0/4006326905 conn(0x7f40880cc7e0 msgr2=0x7f40880ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408fc83700 1 -- 172.21.15.45:0/4006326905 shutdown_connections 2023-08-27T11:27:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.937+0000 7f408fc83700 1 -- 172.21.15.45:0/4006326905 wait complete. 2023-08-27T11:27:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.938+0000 7f408fc83700 1 Processor -- start 2023-08-27T11:27:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.938+0000 7f408fc83700 1 -- start start 2023-08-27T11:27:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408fc83700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f408806a990 0x7f40881f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408fc83700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 0x7f4088157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408fc83700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f408815b200 0x7f40880cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408fc83700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40880c6590 con 0x7f408806cf40 2023-08-27T11:27:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408fc83700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40881f16b0 con 0x7f408815b200 2023-08-27T11:27:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408fc83700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40880c6f40 con 0x7f408806a990 2023-08-27T11:27:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408d21e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 0x7f4088157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408d21e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 0x7f4088157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57714/0 (socket says 172.21.15.45:57714) 2023-08-27T11:27:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408d21e700 1 -- 172.21.15.45:0/3970126016 learned_addr learned my addr 172.21.15.45:0/3970126016 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:26.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408e220700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f408815b200 0x7f40880cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408da1f700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f408806a990 0x7f40881f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.939+0000 7f408d21e700 1 -- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f408806a990 msgr2=0x7f40881f1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f408d21e700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f408806a990 0x7f40881f1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f408d21e700 1 -- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f408815b200 msgr2=0x7f40880cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f408d21e700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f408815b200 0x7f40880cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f408d21e700 1 -- 172.21.15.45:0/3970126016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4080009b30 con 0x7f408806cf40 2023-08-27T11:27:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f408e220700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f408815b200 0x7f40880cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f408da1f700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f408806a990 0x7f40881f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f408d21e700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 0x7f4088157ba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f408400c7f0 tx=0x7f408400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f407e7fc700 1 -- 172.21.15.45:0/3970126016 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084011800 con 0x7f408806cf40 2023-08-27T11:27:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f407e7fc700 1 -- 172.21.15.45:0/3970126016 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084011960 con 0x7f408806cf40 2023-08-27T11:27:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40880c7220 con 0x7f408806cf40 2023-08-27T11:27:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f407e7fc700 1 -- 172.21.15.45:0/3970126016 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084011d10 con 0x7f408806cf40 2023-08-27T11:27:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.940+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40880c77e0 con 0x7f408806cf40 2023-08-27T11:27:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.941+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40880c3610 con 0x7f408806cf40 2023-08-27T11:27:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.941+0000 7f407e7fc700 1 -- 172.21.15.45:0/3970126016 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4084018360 con 0x7f408806cf40 2023-08-27T11:27:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.942+0000 7f407e7fc700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f406c06a200 0x7f406c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.942+0000 7f407e7fc700 1 -- 172.21.15.45:0/3970126016 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4084088970 con 0x7f408806cf40 2023-08-27T11:27:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.942+0000 7f408da1f700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f406c06a200 0x7f406c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.943+0000 7f408da1f700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f406c06a200 0x7f406c06c6c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4078000f60 tx=0x7f4078009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:26.946+0000 7f407e7fc700 1 -- 172.21.15.45:0/3970126016 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4084057760 con 0x7f408806cf40 2023-08-27T11:27:27.194 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:27.193+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f40880734a0 con 0x7f408806cf40 2023-08-27T11:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.095+0000 7f407e7fc700 1 -- 172.21.15.45:0/3970126016 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7f40840572f0 con 0x7f408806cf40 2023-08-27T11:27:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f406c06a200 msgr2=0x7f406c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f406c06a200 0x7f406c06c6c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4078000f60 tx=0x7f4078009040 comp rx=0 tx=0).stop 2023-08-27T11:27:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 msgr2=0x7f4088157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 0x7f4088157ba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f408400c7f0 tx=0x7f408400cbb0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 shutdown_connections 2023-08-27T11:27:28.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f406c06a200 0x7f406c06c6c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f408815b200 0x7f40880cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f408806a990 0x7f40881f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 --2- 172.21.15.45:0/3970126016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f408806cf40 0x7f4088157ba0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 >> 172.21.15.45:0/3970126016 conn(0x7f40880cc7e0 msgr2=0x7f408806a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 shutdown_connections 2023-08-27T11:27:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.098+0000 7f408fc83700 1 -- 172.21.15.45:0/3970126016 wait complete. 2023-08-27T11:27:28.114 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:27:28.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:27:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.251+0000 7f8178b15700 1 Processor -- start 2023-08-27T11:27:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.251+0000 7f8178b15700 1 -- start start 2023-08-27T11:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f8178b15700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8174157be0 0x7f81740716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f8178b15700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817415b460 con 0x7f817406d5e0 2023-08-27T11:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f8178b15700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81740b2280 con 0x7f81740b17c0 2023-08-27T11:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f8178b15700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8174157fc0 con 0x7f8174157be0 2023-08-27T11:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f8172d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f817406d5e0 0x7f81740cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44498/0 (socket says 172.21.15.45:44498) 2023-08-27T11:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f8172d9d700 1 -- 172.21.15.45:0/2307912523 learned_addr learned my addr 172.21.15.45:0/2307912523 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515387091 0 0) 0x7f817415b460 con 0x7f817406d5e0 2023-08-27T11:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817259c700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8174157be0 0x7f81740716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815c0035b0 con 0x7f817406d5e0 2023-08-27T11:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855309452 0 0) 0x7f815c0035b0 con 0x7f817406d5e0 2023-08-27T11:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8174157be0 msgr2=0x7f81740716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8174157be0 0x7f81740716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 >> v1:172.21.15.135:6789/0 conn(0x7f81740b17c0 legacy=0x7f817406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81740b2560 con 0x7f817406d5e0 2023-08-27T11:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f816c0019f0 con 0x7f817406d5e0 2023-08-27T11:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 >> v1:172.21.15.45:6789/0 conn(0x7f817406d5e0 legacy=0x7f81740cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.252+0000 7f817159a700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8174157be0 0x7f815c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f817159a700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f815c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f817159a700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f815c004d90 0x7f815c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8174157fc0 con 0x7f815c003fb0 2023-08-27T11:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c00baf0 con 0x7f815c004d90 2023-08-27T11:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c00bcb0 con 0x7f8174157be0 2023-08-27T11:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f8171d9b700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f815c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f817259c700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8174157be0 0x7f815c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f8172d9d700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f815c004d90 0x7f815c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f8171d9b700 1 -- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8174157be0 msgr2=0x7f815c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f8171d9b700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8174157be0 0x7f815c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f8171d9b700 1 -- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f815c004d90 msgr2=0x7f815c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f8171d9b700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f815c004d90 0x7f815c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f8171d9b700 1 -- 172.21.15.45:0/2307912523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c0035b0 con 0x7f815c003fb0 2023-08-27T11:27:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f8171d9b700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f815c004850 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f81640017c0 tx=0x7f815c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0899f6e746063b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640052d0 con 0x7f815c003fb0 2023-08-27T11:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8164005430 con 0x7f815c003fb0 2023-08-27T11:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.253+0000 7f817159a700 1 -- 172.21.15.45:0/2307912523 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640038c0 con 0x7f815c003fb0 2023-08-27T11:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.254+0000 7f8178b15700 1 -- 172.21.15.45:0/2307912523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 msgr2=0x7f815c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.254+0000 7f8178b15700 1 --2- 172.21.15.45:0/2307912523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f815c004850 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f81640017c0 tx=0x7f815c005880 comp rx=0 tx=0).stop 2023-08-27T11:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.254+0000 7f8172d9d700 1 -- 172.21.15.45:0/2307912523 reap_dead start 2023-08-27T11:27:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.254+0000 7f8178b15700 1 -- 172.21.15.45:0/2307912523 shutdown_connections 2023-08-27T11:27:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.254+0000 7f8178b15700 1 -- 172.21.15.45:0/2307912523 >> 172.21.15.45:0/2307912523 conn(0x7f81740cc7c0 msgr2=0x7f81740ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.254+0000 7f8178b15700 1 -- 172.21.15.45:0/2307912523 shutdown_connections 2023-08-27T11:27:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.254+0000 7f8178b15700 1 -- 172.21.15.45:0/2307912523 wait complete. 2023-08-27T11:27:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.255+0000 7f8178b15700 1 Processor -- start 2023-08-27T11:27:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.255+0000 7f8178b15700 1 -- start start 2023-08-27T11:27:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f8178b15700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f8174159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f8178b15700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f81740adbd0 0x7f8174154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f8178b15700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81740ae050 0x7f8174154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f8178b15700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8174159060 con 0x7f815c003fb0 2023-08-27T11:27:28.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f8178b15700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8174158100 con 0x7f81740ae050 2023-08-27T11:27:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f8178b15700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81741583e0 con 0x7f81740adbd0 2023-08-27T11:27:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f817259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f8174159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f817259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f8174159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57730/0 (socket says 172.21.15.45:57730) 2023-08-27T11:27:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f817259c700 1 -- 172.21.15.45:0/2636483911 learned_addr learned my addr 172.21.15.45:0/2636483911 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f8171d9b700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f81740adbd0 0x7f8174154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f817259c700 1 -- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f81740adbd0 msgr2=0x7f8174154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f8172d9d700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81740ae050 0x7f8174154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f817259c700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f81740adbd0 0x7f8174154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f817259c700 1 -- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81740ae050 msgr2=0x7f8174154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f817259c700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81740ae050 0x7f8174154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.256+0000 7f817259c700 1 -- 172.21.15.45:0/2636483911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81741586c0 con 0x7f815c003fb0 2023-08-27T11:27:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.257+0000 7f817259c700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f8174159480 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8168011aa0 tx=0x7f8168011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.257+0000 7f8162ffd700 1 -- 172.21.15.45:0/2636483911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816800f920 con 0x7f815c003fb0 2023-08-27T11:27:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.257+0000 7f8162ffd700 1 -- 172.21.15.45:0/2636483911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816800fa80 con 0x7f815c003fb0 2023-08-27T11:27:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.257+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81741589a0 con 0x7f815c003fb0 2023-08-27T11:27:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.257+0000 7f8162ffd700 1 -- 172.21.15.45:0/2636483911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816800fe30 con 0x7f815c003fb0 2023-08-27T11:27:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.257+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81741599c0 con 0x7f815c003fb0 2023-08-27T11:27:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.258+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8174072690 con 0x7f815c003fb0 2023-08-27T11:27:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.258+0000 7f8162ffd700 1 -- 172.21.15.45:0/2636483911 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8168014440 con 0x7f815c003fb0 2023-08-27T11:27:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.259+0000 7f8162ffd700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f815406a200 0x7f815406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.259+0000 7f8162ffd700 1 -- 172.21.15.45:0/2636483911 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8168088f10 con 0x7f815c003fb0 2023-08-27T11:27:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.259+0000 7f8171d9b700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f815406a200 0x7f815406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.260+0000 7f8171d9b700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f815406a200 0x7f815406c6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f81741560d0 tx=0x7f8164009030 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.263+0000 7f8162ffd700 1 -- 172.21.15.45:0/2636483911 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8168058010 con 0x7f815c003fb0 2023-08-27T11:27:28.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:28.511+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f81740734a0 con 0x7f815c003fb0 2023-08-27T11:27:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.098+0000 7f8162ffd700 1 -- 172.21.15.45:0/2636483911 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8168022020 con 0x7f815c003fb0 2023-08-27T11:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f815406a200 msgr2=0x7f815406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f815406a200 0x7f815406c6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f81741560d0 tx=0x7f8164009030 comp rx=0 tx=0).stop 2023-08-27T11:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 msgr2=0x7f8174159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f8174159480 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8168011aa0 tx=0x7f8168011e60 comp rx=0 tx=0).stop 2023-08-27T11:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 shutdown_connections 2023-08-27T11:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f815406a200 0x7f815406c6c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f815c003fb0 0x7f8174159480 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81740ae050 0x7f8174154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 --2- 172.21.15.45:0/2636483911 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f81740adbd0 0x7f8174154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 >> 172.21.15.45:0/2636483911 conn(0x7f81740cc7c0 msgr2=0x7f817406a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 shutdown_connections 2023-08-27T11:27:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.101+0000 7f8178b15700 1 -- 172.21.15.45:0/2636483911 wait complete. 2023-08-27T11:27:29.117 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:27:29.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.250+0000 7f00076ce700 1 Processor -- start 2023-08-27T11:27:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.250+0000 7f00076ce700 1 -- start start 2023-08-27T11:27:29.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f00076ce700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00000b8c20 0x7f00000cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f00076ce700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000015d6a0 con 0x7f00000ce2e0 2023-08-27T11:27:29.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f00076ce700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00000ce730 con 0x7f000015d2c0 2023-08-27T11:27:29.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f00076ce700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00000d2c90 con 0x7f00000b8c20 2023-08-27T11:27:29.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f0004c69700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00000b8c20 0x7f00000cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f0005c6b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f00000ce2e0 0x7f00000b6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44518/0 (socket says 172.21.15.45:44518) 2023-08-27T11:27:29.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f0005c6b700 1 -- 172.21.15.45:0/3916528220 learned_addr learned my addr 172.21.15.45:0/3916528220 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:29.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f0004c69700 1 -- 172.21.15.45:0/3916528220 >> v1:172.21.15.135:6789/0 conn(0x7f000015d2c0 legacy=0x7f00000d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:29.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f0004c69700 1 -- 172.21.15.45:0/3916528220 >> v1:172.21.15.45:6789/0 conn(0x7f00000ce2e0 legacy=0x7f00000b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.251+0000 7f0004c69700 1 -- 172.21.15.45:0/3916528220 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00000d1bf0 con 0x7f00000b8c20 2023-08-27T11:27:29.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7f0004c69700 1 --2- 172.21.15.45:0/3916528220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00000b8c20 0x7f00000cdda0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7efffc009870 tx=0x7efffc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a196922ac254b2b9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7efff7fff700 1 -- 172.21.15.45:0/3916528220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc004230 con 0x7f00000b8c20 2023-08-27T11:27:29.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7efff7fff700 1 -- 172.21.15.45:0/3916528220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc004390 con 0x7f00000b8c20 2023-08-27T11:27:29.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7efff7fff700 1 -- 172.21.15.45:0/3916528220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc0036a0 con 0x7f00000b8c20 2023-08-27T11:27:29.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7f00076ce700 1 -- 172.21.15.45:0/3916528220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00000b8c20 msgr2=0x7f00000cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7f00076ce700 1 --2- 172.21.15.45:0/3916528220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00000b8c20 0x7f00000cdda0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7efffc009870 tx=0x7efffc009b80 comp rx=0 tx=0).stop 2023-08-27T11:27:29.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7f00076ce700 1 -- 172.21.15.45:0/3916528220 shutdown_connections 2023-08-27T11:27:29.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7f00076ce700 1 --2- 172.21.15.45:0/3916528220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00000b8c20 0x7f00000cdda0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7f00076ce700 1 -- 172.21.15.45:0/3916528220 >> 172.21.15.45:0/3916528220 conn(0x7f00000d2fd0 msgr2=0x7f00000d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:29.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7f00076ce700 1 -- 172.21.15.45:0/3916528220 shutdown_connections 2023-08-27T11:27:29.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.252+0000 7f00076ce700 1 -- 172.21.15.45:0/3916528220 wait complete. 2023-08-27T11:27:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.254+0000 7f00076ce700 1 Processor -- start 2023-08-27T11:27:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.254+0000 7f00076ce700 1 -- start start 2023-08-27T11:27:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.254+0000 7f00076ce700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00000b8c20 0x7f0000153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.254+0000 7f00076ce700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00000ce2e0 0x7f00000b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.254+0000 7f00076ce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000015d2c0 0x7f000014db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.254+0000 7f00076ce700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001575f0 con 0x7f000015d2c0 2023-08-27T11:27:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.254+0000 7f00076ce700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001f2c60 con 0x7f00000b8c20 2023-08-27T11:27:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.254+0000 7f00076ce700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00001f2f40 con 0x7f00000ce2e0 2023-08-27T11:27:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0005c6b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000015d2c0 0x7f000014db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0005c6b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000015d2c0 0x7f000014db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57748/0 (socket says 172.21.15.45:57748) 2023-08-27T11:27:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0005c6b700 1 -- 172.21.15.45:0/1056400313 learned_addr learned my addr 172.21.15.45:0/1056400313 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f000546a700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00000b8c20 0x7f0000153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0004c69700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00000ce2e0 0x7f00000b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0005c6b700 1 -- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00000ce2e0 msgr2=0x7f00000b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0005c6b700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00000ce2e0 0x7f00000b7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0005c6b700 1 -- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00000b8c20 msgr2=0x7f0000153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0005c6b700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00000b8c20 0x7f0000153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0005c6b700 1 -- 172.21.15.45:0/1056400313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc0094a0 con 0x7f000015d2c0 2023-08-27T11:27:29.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f000546a700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00000b8c20 0x7f0000153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:29.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f0005c6b700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000015d2c0 0x7f000014db30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7efff800cbc0 tx=0x7efff800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7efff5ffb700 1 -- 172.21.15.45:0/1056400313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff800ab50 con 0x7f000015d2c0 2023-08-27T11:27:29.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7efff5ffb700 1 -- 172.21.15.45:0/1056400313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff800acb0 con 0x7f000015d2c0 2023-08-27T11:27:29.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7efff5ffb700 1 -- 172.21.15.45:0/1056400313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8012600 con 0x7f000015d2c0 2023-08-27T11:27:29.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.255+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00001f3220 con 0x7f000015d2c0 2023-08-27T11:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.256+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00001f28c0 con 0x7f000015d2c0 2023-08-27T11:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.256+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00001f3410 con 0x7f000015d2c0 2023-08-27T11:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.257+0000 7efff5ffb700 1 -- 172.21.15.45:0/1056400313 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7efff801b3b0 con 0x7f000015d2c0 2023-08-27T11:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.257+0000 7efff5ffb700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7effe406a200 0x7effe406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.257+0000 7efff5ffb700 1 -- 172.21.15.45:0/1056400313 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7efff80882f0 con 0x7f000015d2c0 2023-08-27T11:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.257+0000 7f000546a700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7effe406a200 0x7effe406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.258+0000 7f000546a700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7effe406a200 0x7effe406c6c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efff0006fd0 tx=0x7efff0008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.262+0000 7efff5ffb700 1 -- 172.21.15.45:0/1056400313 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efff8053810 con 0x7f000015d2c0 2023-08-27T11:27:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.513+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f000015d6a0 con 0x7f000015d2c0 2023-08-27T11:27:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.514+0000 7efff5ffb700 1 -- 172.21.15.45:0/1056400313 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f000015d6a0 con 0x7f000015d2c0 2023-08-27T11:27:29.515 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:29.515 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2178366980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2178366980},{"type":"v1","addr":"172.21.15.135:6836","nonce":2178366980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2939076938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2939076938},{"type":"v1","addr":"172.21.15.45:6833","nonce":2939076938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/148627485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":148627485},{"type":"v1","addr":"172.21.15.45:6835","nonce":148627485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:25.479170+0000","modified":"2023-08-27T11:27:29.096440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1457694749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1457694749},{"type":"v1","addr":"172.21.15.135:6837","nonce":1457694749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:27:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7effe406a200 msgr2=0x7effe406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7effe406a200 0x7effe406c6c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efff0006fd0 tx=0x7efff0008040 comp rx=0 tx=0).stop 2023-08-27T11:27:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000015d2c0 msgr2=0x7f000014db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000015d2c0 0x7f000014db30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7efff800cbc0 tx=0x7efff800cf80 comp rx=0 tx=0).stop 2023-08-27T11:27:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 shutdown_connections 2023-08-27T11:27:29.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7effe406a200 0x7effe406c6c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000015d2c0 0x7f000014db30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00000b8c20 0x7f0000153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 --2- 172.21.15.45:0/1056400313 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00000ce2e0 0x7f00000b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 >> 172.21.15.45:0/1056400313 conn(0x7f00000d2fd0 msgr2=0x7f0000159110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:29.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 shutdown_connections 2023-08-27T11:27:29.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.516+0000 7f00076ce700 1 -- 172.21.15.45:0/1056400313 wait complete. 2023-08-27T11:27:29.520 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2023-08-27T11:27:29.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.653+0000 7f61eb559700 1 Processor -- start 2023-08-27T11:27:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.654+0000 7f61eb559700 1 -- start start 2023-08-27T11:27:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.654+0000 7f61eb559700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61e415b120 0x7f61e40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61eb559700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e40ada50 con 0x7f61e40d1a20 2023-08-27T11:27:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61eb559700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e40ccd60 con 0x7f61e40cdb80 2023-08-27T11:27:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61eb559700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40b0860 con 0x7f61e415b120 2023-08-27T11:27:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61e8af4700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f61e40d1a20 0x7f61e40cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44530/0 (socket says 172.21.15.45:44530) 2023-08-27T11:27:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61e8af4700 1 -- 172.21.15.45:0/2255379622 learned_addr learned my addr 172.21.15.45:0/2255379622 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:29.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61e92f5700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61e415b120 0x7f61e40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389361178 0 0) 0x7f61e40ada50 con 0x7f61e40d1a20 2023-08-27T11:27:29.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d00035b0 con 0x7f61e40d1a20 2023-08-27T11:27:29.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1773145258 0 0) 0x7f61d00035b0 con 0x7f61e40d1a20 2023-08-27T11:27:29.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61e415b120 msgr2=0x7f61e40cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:29.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61e415b120 0x7f61e40cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 >> v1:172.21.15.135:6789/0 conn(0x7f61e40cdb80 legacy=0x7f61e40b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e40b0b40 con 0x7f61e40d1a20 2023-08-27T11:27:29.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f61d4001670 con 0x7f61e40d1a20 2023-08-27T11:27:29.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 >> v1:172.21.15.45:6789/0 conn(0x7f61e40d1a20 legacy=0x7f61e40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61d0003a30 0x7f61d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d00043e0 0x7f61d00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61e415b120 0x7f61d0005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40b0860 con 0x7f61d00043e0 2023-08-27T11:27:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d000ba10 con 0x7f61d0003a30 2023-08-27T11:27:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.655+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d000bbd0 con 0x7f61e415b120 2023-08-27T11:27:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61e8af4700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d00043e0 0x7f61d00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61e92f5700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61d0003a30 0x7f61d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61e9af6700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61e415b120 0x7f61d0005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61e8af4700 1 -- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61e415b120 msgr2=0x7f61d0005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61e8af4700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61e415b120 0x7f61d0005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61e8af4700 1 -- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61d0003a30 msgr2=0x7f61d0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61e8af4700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61d0003a30 0x7f61d0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61e8af4700 1 -- 172.21.15.45:0/2255379622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00035b0 con 0x7f61d00043e0 2023-08-27T11:27:29.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61e8af4700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d00043e0 0x7f61d00048b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c2f0 tx=0x7f61d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120d2f0bf6e311a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d400ce30 con 0x7f61d00043e0 2023-08-27T11:27:29.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d4005960 con 0x7f61d00043e0 2023-08-27T11:27:29.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.656+0000 7f61dbfff700 1 -- 172.21.15.45:0/2255379622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4005d10 con 0x7f61d00043e0 2023-08-27T11:27:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.657+0000 7f61eb559700 1 -- 172.21.15.45:0/2255379622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d00043e0 msgr2=0x7f61d00048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.657+0000 7f61eb559700 1 --2- 172.21.15.45:0/2255379622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d00043e0 0x7f61d00048b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c2f0 tx=0x7f61d0005880 comp rx=0 tx=0).stop 2023-08-27T11:27:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.657+0000 7f61e9af6700 1 -- 172.21.15.45:0/2255379622 reap_dead start 2023-08-27T11:27:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.657+0000 7f61eb559700 1 -- 172.21.15.45:0/2255379622 shutdown_connections 2023-08-27T11:27:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.657+0000 7f61eb559700 1 -- 172.21.15.45:0/2255379622 >> 172.21.15.45:0/2255379622 conn(0x7f61e409a160 msgr2=0x7f61e40d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.657+0000 7f61eb559700 1 -- 172.21.15.45:0/2255379622 shutdown_connections 2023-08-27T11:27:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.657+0000 7f61eb559700 1 -- 172.21.15.45:0/2255379622 wait complete. 2023-08-27T11:27:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.658+0000 7f61eb559700 1 Processor -- start 2023-08-27T11:27:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.658+0000 7f61eb559700 1 -- start start 2023-08-27T11:27:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.659+0000 7f61eb559700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61d00043e0 0x7f61e415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.659+0000 7f61eb559700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e40b22b0 0x7f61e40b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.659+0000 7f61eb559700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e40c8fa0 0x7f61e40c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.659+0000 7f61eb559700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e415c140 con 0x7f61e40b22b0 2023-08-27T11:27:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.659+0000 7f61eb559700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40b2c70 con 0x7f61e40c8fa0 2023-08-27T11:27:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.659+0000 7f61eb559700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40c9970 con 0x7f61d00043e0 2023-08-27T11:27:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e8af4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e40b22b0 0x7f61e40b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e8af4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e40b22b0 0x7f61e40b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57766/0 (socket says 172.21.15.45:57766) 2023-08-27T11:27:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e8af4700 1 -- 172.21.15.45:0/3048976500 learned_addr learned my addr 172.21.15.45:0/3048976500 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e92f5700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61d00043e0 0x7f61e415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e9af6700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e40c8fa0 0x7f61e40c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e8af4700 1 -- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61d00043e0 msgr2=0x7f61e415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e8af4700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61d00043e0 0x7f61e415e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e8af4700 1 -- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e40c8fa0 msgr2=0x7f61e40c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e8af4700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e40c8fa0 0x7f61e40c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e8af4700 1 -- 172.21.15.45:0/3048976500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e415e7d0 con 0x7f61e40b22b0 2023-08-27T11:27:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e92f5700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61d00043e0 0x7f61e415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e9af6700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e40c8fa0 0x7f61e40c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61e8af4700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e40b22b0 0x7f61e40b2730 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f61d4001930 tx=0x7f61d4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61d9ffb700 1 -- 172.21.15.45:0/3048976500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4001660 con 0x7f61e40b22b0 2023-08-27T11:27:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61d9ffb700 1 -- 172.21.15.45:0/3048976500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d4004a80 con 0x7f61e40b22b0 2023-08-27T11:27:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61d9ffb700 1 -- 172.21.15.45:0/3048976500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4004e10 con 0x7f61e40b22b0 2023-08-27T11:27:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e40cab20 con 0x7f61e40b22b0 2023-08-27T11:27:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.660+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e40c9e60 con 0x7f61e40b22b0 2023-08-27T11:27:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.661+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e415bea0 con 0x7f61e40b22b0 2023-08-27T11:27:29.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.662+0000 7f61d9ffb700 1 -- 172.21.15.45:0/3048976500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f61d40070d0 con 0x7f61e40b22b0 2023-08-27T11:27:29.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.662+0000 7f61d9ffb700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f61c806a200 0x7f61c806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:29.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.662+0000 7f61d9ffb700 1 -- 172.21.15.45:0/3048976500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f61d4089fb0 con 0x7f61e40b22b0 2023-08-27T11:27:29.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.662+0000 7f61e92f5700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f61c806a200 0x7f61c806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:29.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.663+0000 7f61e92f5700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f61c806a200 0x7f61c806c6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009470 tx=0x7f61e0008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:29.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.667+0000 7f61d9ffb700 1 -- 172.21.15.45:0/3048976500 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f61d40bc990 con 0x7f61e40b22b0 2023-08-27T11:27:29.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.868+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61e409edd0 con 0x7f61e40b22b0 2023-08-27T11:27:29.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.869+0000 7f61d9ffb700 1 -- 172.21.15.45:0/3048976500 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18637 (secure 0 0 0) 0x7f61d4017020 con 0x7f61e40b22b0 2023-08-27T11:27:29.871 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:29.871 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":86,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:27.055669+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:23.082458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:24.453679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:29.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f61c806a200 msgr2=0x7f61c806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f61c806a200 0x7f61c806c6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009470 tx=0x7f61e0008040 comp rx=0 tx=0).stop 2023-08-27T11:27:29.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e40b22b0 msgr2=0x7f61e40b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:29.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e40b22b0 0x7f61e40b2730 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f61d4001930 tx=0x7f61d4001a00 comp rx=0 tx=0).stop 2023-08-27T11:27:29.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 shutdown_connections 2023-08-27T11:27:29.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f61c806a200 0x7f61c806c6c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e40c8fa0 0x7f61e40c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61d00043e0 0x7f61e415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 --2- 172.21.15.45:0/3048976500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61e40b22b0 0x7f61e40b2730 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:29.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 >> 172.21.15.45:0/3048976500 conn(0x7f61e409a160 msgr2=0x7f61e40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:29.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 shutdown_connections 2023-08-27T11:27:29.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:29.871+0000 7f61eb559700 1 -- 172.21.15.45:0/3048976500 wait complete. 2023-08-27T11:27:29.885 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:27:29.886 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:27:30.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.009+0000 7f8282985700 1 Processor -- start 2023-08-27T11:27:30.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.009+0000 7f8282985700 1 -- start start 2023-08-27T11:27:30.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.010+0000 7f8282985700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c06cec0 0x7f827c06e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.010+0000 7f8282985700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f827c06e7b0 0x7f827c0ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.010+0000 7f8282985700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827c06ec00 con 0x7f827c15b070 2023-08-27T11:27:30.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.010+0000 7f8282985700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c15d8e0 con 0x7f827c06e7b0 2023-08-27T11:27:30.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.010+0000 7f8282985700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c15dad0 con 0x7f827c06cec0 2023-08-27T11:27:30.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.010+0000 7f8280f22700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f827c06e7b0 0x7f827c0ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.010+0000 7f8280f22700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f827c06e7b0 0x7f827c0ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57776/0 (socket says 172.21.15.45:57776) 2023-08-27T11:27:30.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.010+0000 7f8280f22700 1 -- 172.21.15.45:0/2290275623 learned_addr learned my addr 172.21.15.45:0/2290275623 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f827b7fe700 1 --2- 172.21.15.45:0/2290275623 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c06cec0 0x7f827c06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f8280f22700 1 -- 172.21.15.45:0/2290275623 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c06cec0 msgr2=0x7f827c06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f8280f22700 1 --2- 172.21.15.45:0/2290275623 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c06cec0 0x7f827c06e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f8280f22700 1 -- 172.21.15.45:0/2290275623 >> v1:172.21.15.135:6789/0 conn(0x7f827c15b070 legacy=0x7f827c0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f8280f22700 1 -- 172.21.15.45:0/2290275623 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c15ddb0 con 0x7f827c06e7b0 2023-08-27T11:27:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f827affd700 1 -- 172.21.15.45:0/2290275623 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315702382 0 0) 0x7f826c001110 con 0x7f827c15b070 2023-08-27T11:27:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f8280f22700 1 --2- 172.21.15.45:0/2290275623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 0x7f827c0ae8d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8274009e50 tx=0x7f827400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9bb50648a722f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f827affd700 1 -- 172.21.15.45:0/2290275623 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400e070 con 0x7f827c06e7b0 2023-08-27T11:27:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f827affd700 1 -- 172.21.15.45:0/2290275623 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827400b910 con 0x7f827c06e7b0 2023-08-27T11:27:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.011+0000 7f827affd700 1 -- 172.21.15.45:0/2290275623 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400bcc0 con 0x7f827c06e7b0 2023-08-27T11:27:30.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.012+0000 7f8282985700 1 -- 172.21.15.45:0/2290275623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 msgr2=0x7f827c0ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.012+0000 7f8282985700 1 --2- 172.21.15.45:0/2290275623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 0x7f827c0ae8d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8274009e50 tx=0x7f827400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:30.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.012+0000 7f8282985700 1 -- 172.21.15.45:0/2290275623 shutdown_connections 2023-08-27T11:27:30.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.012+0000 7f8282985700 1 --2- 172.21.15.45:0/2290275623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 0x7f827c0ae8d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.012+0000 7f8282985700 1 --2- 172.21.15.45:0/2290275623 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c06cec0 0x7f827c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.012+0000 7f8282985700 1 -- 172.21.15.45:0/2290275623 >> 172.21.15.45:0/2290275623 conn(0x7f827c0cfe50 msgr2=0x7f827c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:30.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.012+0000 7f8282985700 1 -- 172.21.15.45:0/2290275623 shutdown_connections 2023-08-27T11:27:30.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.012+0000 7f8282985700 1 -- 172.21.15.45:0/2290275623 wait complete. 2023-08-27T11:27:30.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.013+0000 7f8282985700 1 Processor -- start 2023-08-27T11:27:30.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.014+0000 7f8282985700 1 -- start start 2023-08-27T11:27:30.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.014+0000 7f8282985700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827c06cec0 0x7f827c1e6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8282985700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 0x7f827c14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8282985700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c15b070 0x7f827c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8282985700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c1509d0 con 0x7f827c06e7b0 2023-08-27T11:27:30.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8282985700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0c5520 con 0x7f827c06cec0 2023-08-27T11:27:30.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8282985700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0c4910 con 0x7f827c15b070 2023-08-27T11:27:30.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 0x7f827c14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 0x7f827c14fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57780/0 (socket says 172.21.15.45:57780) 2023-08-27T11:27:30.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827b7fe700 1 -- 172.21.15.45:0/3122164085 learned_addr learned my addr 172.21.15.45:0/3122164085 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:30.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8280f22700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c15b070 0x7f827c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827bfff700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827c06cec0 0x7f827c1e6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827b7fe700 1 -- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c15b070 msgr2=0x7f827c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827b7fe700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c15b070 0x7f827c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827b7fe700 1 -- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827c06cec0 msgr2=0x7f827c1e6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827b7fe700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827c06cec0 0x7f827c1e6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827b7fe700 1 -- 172.21.15.45:0/3122164085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274009b30 con 0x7f827c06e7b0 2023-08-27T11:27:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8280f22700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c15b070 0x7f827c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827bfff700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827c06cec0 0x7f827c1e6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f827b7fe700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 0x7f827c14fea0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f826400e9b0 tx=0x7f826400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8278ff9700 1 -- 172.21.15.45:0/3122164085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826400c7f0 con 0x7f827c06e7b0 2023-08-27T11:27:30.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8278ff9700 1 -- 172.21.15.45:0/3122164085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826400c950 con 0x7f827c06e7b0 2023-08-27T11:27:30.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c0c4bf0 con 0x7f827c06e7b0 2023-08-27T11:27:30.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8278ff9700 1 -- 172.21.15.45:0/3122164085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826400ccc0 con 0x7f827c06e7b0 2023-08-27T11:27:30.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.015+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c0c51b0 con 0x7f827c06e7b0 2023-08-27T11:27:30.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.016+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c0c5800 con 0x7f827c06e7b0 2023-08-27T11:27:30.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.017+0000 7f8278ff9700 1 -- 172.21.15.45:0/3122164085 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8264013470 con 0x7f827c06e7b0 2023-08-27T11:27:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.017+0000 7f8278ff9700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f825c06a200 0x7f825c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.017+0000 7f8278ff9700 1 -- 172.21.15.45:0/3122164085 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f82640889c0 con 0x7f827c06e7b0 2023-08-27T11:27:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.018+0000 7f827bfff700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f825c06a200 0x7f825c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.018+0000 7f827bfff700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f825c06a200 0x7f825c06c6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f826c000f60 tx=0x7f826c009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.022+0000 7f8278ff9700 1 -- 172.21.15.45:0/3122164085 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8264053d90 con 0x7f827c06e7b0 2023-08-27T11:27:30.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.218+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 --> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f827c15e370 con 0x7f825c06a200 2023-08-27T11:27:30.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.228+0000 7f8278ff9700 1 -- 172.21.15.45:0/3122164085 <== mgr.6247 v2:172.21.15.135:6832/3430028400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306448 (secure 0 0 0) 0x7f826c0050d0 con 0x7f825c06a200 2023-08-27T11:27:30.232 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:30.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f825c06a200 msgr2=0x7f825c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f825c06a200 0x7f825c06c6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f826c000f60 tx=0x7f826c009040 comp rx=0 tx=0).stop 2023-08-27T11:27:30.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 msgr2=0x7f827c14fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 0x7f827c14fea0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f826400e9b0 tx=0x7f826400ed70 comp rx=0 tx=0).stop 2023-08-27T11:27:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 shutdown_connections 2023-08-27T11:27:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f825c06a200 0x7f825c06c6c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f827c15b070 0x7f827c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f827c06cec0 0x7f827c1e6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 --2- 172.21.15.45:0/3122164085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f827c06e7b0 0x7f827c14fea0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 >> 172.21.15.45:0/3122164085 conn(0x7f827c0cfe50 msgr2=0x7f827c159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 shutdown_connections 2023-08-27T11:27:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.231+0000 7f8282985700 1 -- 172.21.15.45:0/3122164085 wait complete. 2023-08-27T11:27:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:27:30.245 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:27:29.350988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266824,"kb_used_data":9056,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754707896,"statfs":{"total":773094113280,"available":772820885504,"internally_reserved":0,"allocated":9273344,"data_stored":5375776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.304515"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065740+0000","last_change":"2023-08-27T11:27:25.065740+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065740+0000","last_undegraded":"2023-08-27T11:27:25.065740+0000","last_fullsized":"2023-08-27T11:27:25.065740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062228+0000","last_change":"2023-08-27T11:27:25.062228+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062228+0000","last_undegraded":"2023-08-27T11:27:25.062228+0000","last_fullsized":"2023-08-27T11:27:25.062228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060843+0000","last_change":"2023-08-27T11:27:25.060843+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060843+0000","last_undegraded":"2023-08-27T11:27:25.060843+0000","last_fullsized":"2023-08-27T11:27:25.060843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060288+0000","last_change":"2023-08-27T11:27:25.060288+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060288+0000","last_undegraded":"2023-08-27T11:27:25.060288+0000","last_fullsized":"2023-08-27T11:27:25.060288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059387+0000","last_change":"2023-08-27T11:27:25.059387+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059387+0000","last_undegraded":"2023-08-27T11:27:25.059387+0000","last_fullsized":"2023-08-27T11:27:25.059387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062065+0000","last_change":"2023-08-27T11:27:25.062065+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062065+0000","last_undegraded":"2023-08-27T11:27:25.062065+0000","last_fullsized":"2023-08-27T11:27:25.062065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063011+0000","last_change":"2023-08-27T11:27:25.063011+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063011+0000","last_undegraded":"2023-08-27T11:27:25.063011+0000","last_fullsized":"2023-08-27T11:27:25.063011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062097+0000","last_change":"2023-08-27T11:27:25.062097+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062097+0000","last_undegraded":"2023-08-27T11:27:25.062097+0000","last_fullsized":"2023-08-27T11:27:25.062097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058937+0000","last_change":"2023-08-27T11:27:25.058937+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058937+0000","last_undegraded":"2023-08-27T11:27:25.058937+0000","last_fullsized":"2023-08-27T11:27:25.058937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060848+0000","last_change":"2023-08-27T11:27:25.060848+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060848+0000","last_undegraded":"2023-08-27T11:27:25.060848+0000","last_fullsized":"2023-08-27T11:27:25.060848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065903+0000","last_change":"2023-08-27T11:27:25.065903+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065903+0000","last_undegraded":"2023-08-27T11:27:25.065903+0000","last_fullsized":"2023-08-27T11:27:25.065903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059898+0000","last_change":"2023-08-27T11:27:25.059898+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059898+0000","last_undegraded":"2023-08-27T11:27:25.059898+0000","last_fullsized":"2023-08-27T11:27:25.059898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063681+0000","last_change":"2023-08-27T11:27:25.063681+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063681+0000","last_undegraded":"2023-08-27T11:27:25.063681+0000","last_fullsized":"2023-08-27T11:27:25.063681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058054+0000","last_change":"2023-08-27T11:27:25.058054+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058054+0000","last_undegraded":"2023-08-27T11:27:25.058054+0000","last_fullsized":"2023-08-27T11:27:25.058054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066424+0000","last_change":"2023-08-27T11:27:25.066424+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066424+0000","last_undegraded":"2023-08-27T11:27:25.066424+0000","last_fullsized":"2023-08-27T11:27:25.066424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059416+0000","last_change":"2023-08-27T11:27:25.059416+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059416+0000","last_undegraded":"2023-08-27T11:27:25.059416+0000","last_fullsized":"2023-08-27T11:27:25.059416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058352+0000","last_change":"2023-08-27T11:27:25.058352+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058352+0000","last_undegraded":"2023-08-27T11:27:25.058352+0000","last_fullsized":"2023-08-27T11:27:25.058352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059190+0000","last_change":"2023-08-27T11:27:25.059190+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059190+0000","last_undegraded":"2023-08-27T11:27:25.059190+0000","last_fullsized":"2023-08-27T11:27:25.059190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066698+0000","last_change":"2023-08-27T11:27:25.066698+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066698+0000","last_undegraded":"2023-08-27T11:27:25.066698+0000","last_fullsized":"2023-08-27T11:27:25.066698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063155+0000","last_change":"2023-08-27T11:27:25.063155+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063155+0000","last_undegraded":"2023-08-27T11:27:25.063155+0000","last_fullsized":"2023-08-27T11:27:25.063155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061527+0000","last_change":"2023-08-27T11:27:25.061527+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061527+0000","last_undegraded":"2023-08-27T11:27:25.061527+0000","last_fullsized":"2023-08-27T11:27:25.061527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059917+0000","last_change":"2023-08-27T11:27:25.059917+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059917+0000","last_undegraded":"2023-08-27T11:27:25.059917+0000","last_fullsized":"2023-08-27T11:27:25.059917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064806+0000","last_change":"2023-08-27T11:27:25.064806+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064806+0000","last_undegraded":"2023-08-27T11:27:25.064806+0000","last_fullsized":"2023-08-27T11:27:25.064806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064119+0000","last_change":"2023-08-27T11:27:25.064119+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064119+0000","last_undegraded":"2023-08-27T11:27:25.064119+0000","last_fullsized":"2023-08-27T11:27:25.064119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060052+0000","last_change":"2023-08-27T11:27:25.060052+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060052+0000","last_undegraded":"2023-08-27T11:27:25.060052+0000","last_fullsized":"2023-08-27T11:27:25.060052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065794+0000","last_change":"2023-08-27T11:27:25.065794+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065794+0000","last_undegraded":"2023-08-27T11:27:25.065794+0000","last_fullsized":"2023-08-27T11:27:25.065794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060404+0000","last_change":"2023-08-27T11:27:25.060404+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060404+0000","last_undegraded":"2023-08-27T11:27:25.060404+0000","last_fullsized":"2023-08-27T11:27:25.060404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060723+0000","last_change":"2023-08-27T11:27:25.060723+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060723+0000","last_undegraded":"2023-08-27T11:27:25.060723+0000","last_fullsized":"2023-08-27T11:27:25.060723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060613+0000","last_change":"2023-08-27T11:27:25.060613+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060613+0000","last_undegraded":"2023-08-27T11:27:25.060613+0000","last_fullsized":"2023-08-27T11:27:25.060613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065228+0000","last_change":"2023-08-27T11:27:25.065228+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065228+0000","last_undegraded":"2023-08-27T11:27:25.065228+0000","last_fullsized":"2023-08-27T11:27:25.065228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062163+0000","last_change":"2023-08-27T11:27:25.062163+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062163+0000","last_undegraded":"2023-08-27T11:27:25.062163+0000","last_fullsized":"2023-08-27T11:27:25.062163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061179+0000","last_change":"2023-08-27T11:27:25.061179+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061179+0000","last_undegraded":"2023-08-27T11:27:25.061179+0000","last_fullsized":"2023-08-27T11:27:25.061179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064503+0000","last_change":"2023-08-27T11:27:25.064503+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064503+0000","last_undegraded":"2023-08-27T11:27:25.064503+0000","last_fullsized":"2023-08-27T11:27:25.064503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075864+0000","last_change":"2023-08-27T11:27:25.074467+0000","last_active":"2023-08-27T11:27:25.075864+0000","last_peered":"2023-08-27T11:27:25.075864+0000","last_clean":"2023-08-27T11:27:25.075864+0000","last_became_active":"2023-08-27T11:27:25.074201+0000","last_became_peered":"2023-08-27T11:27:25.074201+0000","last_unstale":"2023-08-27T11:27:25.075864+0000","last_undegraded":"2023-08-27T11:27:25.075864+0000","last_fullsized":"2023-08-27T11:27:25.075864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:36:13.473546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057922+0000","last_change":"2023-08-27T11:27:25.057922+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057922+0000","last_undegraded":"2023-08-27T11:27:25.057922+0000","last_fullsized":"2023-08-27T11:27:25.057922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075576+0000","last_change":"2023-08-27T11:27:25.072826+0000","last_active":"2023-08-27T11:27:25.075576+0000","last_peered":"2023-08-27T11:27:25.075576+0000","last_clean":"2023-08-27T11:27:25.075576+0000","last_became_active":"2023-08-27T11:27:25.072493+0000","last_became_peered":"2023-08-27T11:27:25.072493+0000","last_unstale":"2023-08-27T11:27:25.075576+0000","last_undegraded":"2023-08-27T11:27:25.075576+0000","last_fullsized":"2023-08-27T11:27:25.075576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:08.223558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.067551+0000","last_change":"2023-08-27T11:27:25.067551+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.067551+0000","last_undegraded":"2023-08-27T11:27:25.067551+0000","last_fullsized":"2023-08-27T11:27:25.067551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075945+0000","last_change":"2023-08-27T11:27:25.075211+0000","last_active":"2023-08-27T11:27:25.075945+0000","last_peered":"2023-08-27T11:27:25.075945+0000","last_clean":"2023-08-27T11:27:25.075945+0000","last_became_active":"2023-08-27T11:27:25.074962+0000","last_became_peered":"2023-08-27T11:27:25.074962+0000","last_unstale":"2023-08-27T11:27:25.075945+0000","last_undegraded":"2023-08-27T11:27:25.075945+0000","last_fullsized":"2023-08-27T11:27:25.075945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:48.066087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065441+0000","last_change":"2023-08-27T11:27:25.065441+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065441+0000","last_undegraded":"2023-08-27T11:27:25.065441+0000","last_fullsized":"2023-08-27T11:27:25.065441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078193+0000","last_change":"2023-08-27T11:27:25.070402+0000","last_active":"2023-08-27T11:27:25.078193+0000","last_peered":"2023-08-27T11:27:25.078193+0000","last_clean":"2023-08-27T11:27:25.078193+0000","last_became_active":"2023-08-27T11:27:25.070112+0000","last_became_peered":"2023-08-27T11:27:25.070112+0000","last_unstale":"2023-08-27T11:27:25.078193+0000","last_undegraded":"2023-08-27T11:27:25.078193+0000","last_fullsized":"2023-08-27T11:27:25.078193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:46.302771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062102+0000","last_change":"2023-08-27T11:27:25.062102+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062102+0000","last_undegraded":"2023-08-27T11:27:25.062102+0000","last_fullsized":"2023-08-27T11:27:25.062102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073084+0000","last_change":"2023-08-27T11:27:25.071009+0000","last_active":"2023-08-27T11:27:25.073084+0000","last_peered":"2023-08-27T11:27:25.073084+0000","last_clean":"2023-08-27T11:27:25.073084+0000","last_became_active":"2023-08-27T11:27:25.070686+0000","last_became_peered":"2023-08-27T11:27:25.070686+0000","last_unstale":"2023-08-27T11:27:25.073084+0000","last_undegraded":"2023-08-27T11:27:25.073084+0000","last_fullsized":"2023-08-27T11:27:25.073084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:34:52.384052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061142+0000","last_change":"2023-08-27T11:27:25.061142+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061142+0000","last_undegraded":"2023-08-27T11:27:25.061142+0000","last_fullsized":"2023-08-27T11:27:25.061142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078714+0000","last_change":"2023-08-27T11:27:25.076385+0000","last_active":"2023-08-27T11:27:25.078714+0000","last_peered":"2023-08-27T11:27:25.078714+0000","last_clean":"2023-08-27T11:27:25.078714+0000","last_became_active":"2023-08-27T11:27:25.076062+0000","last_became_peered":"2023-08-27T11:27:25.076062+0000","last_unstale":"2023-08-27T11:27:25.078714+0000","last_undegraded":"2023-08-27T11:27:25.078714+0000","last_fullsized":"2023-08-27T11:27:25.078714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:59:06.875963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057837+0000","last_change":"2023-08-27T11:27:25.057837+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057837+0000","last_undegraded":"2023-08-27T11:27:25.057837+0000","last_fullsized":"2023-08-27T11:27:25.057837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.071644+0000","last_change":"2023-08-27T11:27:25.068965+0000","last_active":"2023-08-27T11:27:25.071644+0000","last_peered":"2023-08-27T11:27:25.071644+0000","last_clean":"2023-08-27T11:27:25.071644+0000","last_became_active":"2023-08-27T11:27:25.068679+0000","last_became_peered":"2023-08-27T11:27:25.068679+0000","last_unstale":"2023-08-27T11:27:25.071644+0000","last_undegraded":"2023-08-27T11:27:25.071644+0000","last_fullsized":"2023-08-27T11:27:25.071644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:59.810868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064147+0000","last_change":"2023-08-27T11:27:25.064147+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064147+0000","last_undegraded":"2023-08-27T11:27:25.064147+0000","last_fullsized":"2023-08-27T11:27:25.064147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.066727+0000","last_change":"2023-08-27T11:27:25.065522+0000","last_active":"2023-08-27T11:27:25.066727+0000","last_peered":"2023-08-27T11:27:25.066727+0000","last_clean":"2023-08-27T11:27:25.066727+0000","last_became_active":"2023-08-27T11:27:25.065251+0000","last_became_peered":"2023-08-27T11:27:25.065251+0000","last_unstale":"2023-08-27T11:27:25.066727+0000","last_undegraded":"2023-08-27T11:27:25.066727+0000","last_fullsized":"2023-08-27T11:27:25.066727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:05.350508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063105+0000","last_change":"2023-08-27T11:27:25.063105+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063105+0000","last_undegraded":"2023-08-27T11:27:25.063105+0000","last_fullsized":"2023-08-27T11:27:25.063105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075138+0000","last_change":"2023-08-27T11:27:25.068373+0000","last_active":"2023-08-27T11:27:25.075138+0000","last_peered":"2023-08-27T11:27:25.075138+0000","last_clean":"2023-08-27T11:27:25.075138+0000","last_became_active":"2023-08-27T11:27:25.068082+0000","last_became_peered":"2023-08-27T11:27:25.068082+0000","last_unstale":"2023-08-27T11:27:25.075138+0000","last_undegraded":"2023-08-27T11:27:25.075138+0000","last_fullsized":"2023-08-27T11:27:25.075138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:07.411357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064080+0000","last_change":"2023-08-27T11:27:25.064080+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064080+0000","last_undegraded":"2023-08-27T11:27:25.064080+0000","last_fullsized":"2023-08-27T11:27:25.064080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078741+0000","last_change":"2023-08-27T11:27:25.076541+0000","last_active":"2023-08-27T11:27:25.078741+0000","last_peered":"2023-08-27T11:27:25.078741+0000","last_clean":"2023-08-27T11:27:25.078741+0000","last_became_active":"2023-08-27T11:27:25.076262+0000","last_became_peered":"2023-08-27T11:27:25.076262+0000","last_unstale":"2023-08-27T11:27:25.078741+0000","last_undegraded":"2023-08-27T11:27:25.078741+0000","last_fullsized":"2023-08-27T11:27:25.078741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:59:00.006614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066919+0000","last_change":"2023-08-27T11:27:25.066919+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066919+0000","last_undegraded":"2023-08-27T11:27:25.066919+0000","last_fullsized":"2023-08-27T11:27:25.066919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.068584+0000","last_change":"2023-08-27T11:27:25.068119+0000","last_active":"2023-08-27T11:27:25.068584+0000","last_peered":"2023-08-27T11:27:25.068584+0000","last_clean":"2023-08-27T11:27:25.068584+0000","last_became_active":"2023-08-27T11:27:25.067846+0000","last_became_peered":"2023-08-27T11:27:25.067846+0000","last_unstale":"2023-08-27T11:27:25.068584+0000","last_undegraded":"2023-08-27T11:27:25.068584+0000","last_fullsized":"2023-08-27T11:27:25.068584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:43.039809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058952+0000","last_change":"2023-08-27T11:27:25.058952+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058952+0000","last_undegraded":"2023-08-27T11:27:25.058952+0000","last_fullsized":"2023-08-27T11:27:25.058952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.067165+0000","last_change":"2023-08-27T11:27:25.065700+0000","last_active":"2023-08-27T11:27:25.067165+0000","last_peered":"2023-08-27T11:27:25.067165+0000","last_clean":"2023-08-27T11:27:25.067165+0000","last_became_active":"2023-08-27T11:27:25.065409+0000","last_became_peered":"2023-08-27T11:27:25.065409+0000","last_unstale":"2023-08-27T11:27:25.067165+0000","last_undegraded":"2023-08-27T11:27:25.067165+0000","last_fullsized":"2023-08-27T11:27:25.067165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:18.780769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066322+0000","last_change":"2023-08-27T11:27:25.066322+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066322+0000","last_undegraded":"2023-08-27T11:27:25.066322+0000","last_fullsized":"2023-08-27T11:27:25.066322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073580+0000","last_change":"2023-08-27T11:27:25.070284+0000","last_active":"2023-08-27T11:27:25.073580+0000","last_peered":"2023-08-27T11:27:25.073580+0000","last_clean":"2023-08-27T11:27:25.073580+0000","last_became_active":"2023-08-27T11:27:25.069996+0000","last_became_peered":"2023-08-27T11:27:25.069996+0000","last_unstale":"2023-08-27T11:27:25.073580+0000","last_undegraded":"2023-08-27T11:27:25.073580+0000","last_fullsized":"2023-08-27T11:27:25.073580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:00.139285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061933+0000","last_change":"2023-08-27T11:27:25.061933+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061933+0000","last_undegraded":"2023-08-27T11:27:25.061933+0000","last_fullsized":"2023-08-27T11:27:25.061933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078214+0000","last_change":"2023-08-27T11:27:25.070398+0000","last_active":"2023-08-27T11:27:25.078214+0000","last_peered":"2023-08-27T11:27:25.078214+0000","last_clean":"2023-08-27T11:27:25.078214+0000","last_became_active":"2023-08-27T11:27:25.070052+0000","last_became_peered":"2023-08-27T11:27:25.070052+0000","last_unstale":"2023-08-27T11:27:25.078214+0000","last_undegraded":"2023-08-27T11:27:25.078214+0000","last_fullsized":"2023-08-27T11:27:25.078214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:39.907863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062446+0000","last_change":"2023-08-27T11:27:25.062446+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062446+0000","last_undegraded":"2023-08-27T11:27:25.062446+0000","last_fullsized":"2023-08-27T11:27:25.062446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073733+0000","last_change":"2023-08-27T11:27:25.072045+0000","last_active":"2023-08-27T11:27:25.073733+0000","last_peered":"2023-08-27T11:27:25.073733+0000","last_clean":"2023-08-27T11:27:25.073733+0000","last_became_active":"2023-08-27T11:27:25.071747+0000","last_became_peered":"2023-08-27T11:27:25.071747+0000","last_unstale":"2023-08-27T11:27:25.073733+0000","last_undegraded":"2023-08-27T11:27:25.073733+0000","last_fullsized":"2023-08-27T11:27:25.073733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:53.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057521+0000","last_change":"2023-08-27T11:27:25.057521+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057521+0000","last_undegraded":"2023-08-27T11:27:25.057521+0000","last_fullsized":"2023-08-27T11:27:25.057521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075545+0000","last_change":"2023-08-27T11:27:25.072388+0000","last_active":"2023-08-27T11:27:25.075545+0000","last_peered":"2023-08-27T11:27:25.075545+0000","last_clean":"2023-08-27T11:27:25.075545+0000","last_became_active":"2023-08-27T11:27:25.072093+0000","last_became_peered":"2023-08-27T11:27:25.072093+0000","last_unstale":"2023-08-27T11:27:25.075545+0000","last_undegraded":"2023-08-27T11:27:25.075545+0000","last_fullsized":"2023-08-27T11:27:25.075545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:59:37.328310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063415+0000","last_change":"2023-08-27T11:27:25.063415+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063415+0000","last_undegraded":"2023-08-27T11:27:25.063415+0000","last_fullsized":"2023-08-27T11:27:25.063415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078786+0000","last_change":"2023-08-27T11:27:25.076200+0000","last_active":"2023-08-27T11:27:25.078786+0000","last_peered":"2023-08-27T11:27:25.078786+0000","last_clean":"2023-08-27T11:27:25.078786+0000","last_became_active":"2023-08-27T11:27:25.074425+0000","last_became_peered":"2023-08-27T11:27:25.074425+0000","last_unstale":"2023-08-27T11:27:25.078786+0000","last_undegraded":"2023-08-27T11:27:25.078786+0000","last_fullsized":"2023-08-27T11:27:25.078786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:14:08.033370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060910+0000","last_change":"2023-08-27T11:27:25.060910+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060910+0000","last_undegraded":"2023-08-27T11:27:25.060910+0000","last_fullsized":"2023-08-27T11:27:25.060910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073798+0000","last_change":"2023-08-27T11:27:25.072101+0000","last_active":"2023-08-27T11:27:25.073798+0000","last_peered":"2023-08-27T11:27:25.073798+0000","last_clean":"2023-08-27T11:27:25.073798+0000","last_became_active":"2023-08-27T11:27:25.071818+0000","last_became_peered":"2023-08-27T11:27:25.071818+0000","last_unstale":"2023-08-27T11:27:25.073798+0000","last_undegraded":"2023-08-27T11:27:25.073798+0000","last_fullsized":"2023-08-27T11:27:25.073798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:09.111047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063757+0000","last_change":"2023-08-27T11:27:25.063757+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063757+0000","last_undegraded":"2023-08-27T11:27:25.063757+0000","last_fullsized":"2023-08-27T11:27:25.063757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075266+0000","last_change":"2023-08-27T11:27:25.072164+0000","last_active":"2023-08-27T11:27:25.075266+0000","last_peered":"2023-08-27T11:27:25.075266+0000","last_clean":"2023-08-27T11:27:25.075266+0000","last_became_active":"2023-08-27T11:27:25.071847+0000","last_became_peered":"2023-08-27T11:27:25.071847+0000","last_unstale":"2023-08-27T11:27:25.075266+0000","last_undegraded":"2023-08-27T11:27:25.075266+0000","last_fullsized":"2023-08-27T11:27:25.075266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:22:02.251506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064766+0000","last_change":"2023-08-27T11:27:25.064766+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064766+0000","last_undegraded":"2023-08-27T11:27:25.064766+0000","last_fullsized":"2023-08-27T11:27:25.064766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.076129+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:25.076129+0000","last_peered":"2023-08-27T11:27:25.076129+0000","last_clean":"2023-08-27T11:27:25.076129+0000","last_became_active":"2023-08-27T11:27:25.074195+0000","last_became_peered":"2023-08-27T11:27:25.074195+0000","last_unstale":"2023-08-27T11:27:25.076129+0000","last_undegraded":"2023-08-27T11:27:25.076129+0000","last_fullsized":"2023-08-27T11:27:25.076129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:43.039809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057420+0000","last_change":"2023-08-27T11:27:25.057420+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057420+0000","last_undegraded":"2023-08-27T11:27:25.057420+0000","last_fullsized":"2023-08-27T11:27:25.057420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.066696+0000","last_change":"2023-08-27T11:27:25.065564+0000","last_active":"2023-08-27T11:27:25.066696+0000","last_peered":"2023-08-27T11:27:25.066696+0000","last_clean":"2023-08-27T11:27:25.066696+0000","last_became_active":"2023-08-27T11:27:25.065291+0000","last_became_peered":"2023-08-27T11:27:25.065291+0000","last_unstale":"2023-08-27T11:27:25.066696+0000","last_undegraded":"2023-08-27T11:27:25.066696+0000","last_fullsized":"2023-08-27T11:27:25.066696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:10:02.253904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066220+0000","last_change":"2023-08-27T11:27:25.066220+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066220+0000","last_undegraded":"2023-08-27T11:27:25.066220+0000","last_fullsized":"2023-08-27T11:27:25.066220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.072825+0000","last_change":"2023-08-27T11:27:25.069284+0000","last_active":"2023-08-27T11:27:25.072825+0000","last_peered":"2023-08-27T11:27:25.072825+0000","last_clean":"2023-08-27T11:27:25.072825+0000","last_became_active":"2023-08-27T11:27:25.068979+0000","last_became_peered":"2023-08-27T11:27:25.068979+0000","last_unstale":"2023-08-27T11:27:25.072825+0000","last_undegraded":"2023-08-27T11:27:25.072825+0000","last_fullsized":"2023-08-27T11:27:25.072825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:54:46.692552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058711+0000","last_change":"2023-08-27T11:27:25.058711+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058711+0000","last_undegraded":"2023-08-27T11:27:25.058711+0000","last_fullsized":"2023-08-27T11:27:25.058711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075539+0000","last_change":"2023-08-27T11:27:25.072420+0000","last_active":"2023-08-27T11:27:25.075539+0000","last_peered":"2023-08-27T11:27:25.075539+0000","last_clean":"2023-08-27T11:27:25.075539+0000","last_became_active":"2023-08-27T11:27:25.072117+0000","last_became_peered":"2023-08-27T11:27:25.072117+0000","last_unstale":"2023-08-27T11:27:25.075539+0000","last_undegraded":"2023-08-27T11:27:25.075539+0000","last_fullsized":"2023-08-27T11:27:25.075539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:42.396071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060737+0000","last_change":"2023-08-27T11:27:25.060737+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060737+0000","last_undegraded":"2023-08-27T11:27:25.060737+0000","last_fullsized":"2023-08-27T11:27:25.060737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073981+0000","last_change":"2023-08-27T11:27:25.069386+0000","last_active":"2023-08-27T11:27:25.073981+0000","last_peered":"2023-08-27T11:27:25.073981+0000","last_clean":"2023-08-27T11:27:25.073981+0000","last_became_active":"2023-08-27T11:27:25.069120+0000","last_became_peered":"2023-08-27T11:27:25.069120+0000","last_unstale":"2023-08-27T11:27:25.073981+0000","last_undegraded":"2023-08-27T11:27:25.073981+0000","last_fullsized":"2023-08-27T11:27:25.073981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:58.548667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057630+0000","last_change":"2023-08-27T11:27:25.057630+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057630+0000","last_undegraded":"2023-08-27T11:27:25.057630+0000","last_fullsized":"2023-08-27T11:27:25.057630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.076857+0000","last_change":"2023-08-27T11:27:25.076447+0000","last_active":"2023-08-27T11:27:25.076857+0000","last_peered":"2023-08-27T11:27:25.076857+0000","last_clean":"2023-08-27T11:27:25.076857+0000","last_became_active":"2023-08-27T11:27:25.076148+0000","last_became_peered":"2023-08-27T11:27:25.076148+0000","last_unstale":"2023-08-27T11:27:25.076857+0000","last_undegraded":"2023-08-27T11:27:25.076857+0000","last_fullsized":"2023-08-27T11:27:25.076857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:27.458908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.062649+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:27:25.062649+0000","last_peered":"2023-08-27T11:27:25.062649+0000","last_clean":"2023-08-27T11:27:25.062649+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:27:25.062649+0000","last_undegraded":"2023-08-27T11:27:25.062649+0000","last_fullsized":"2023-08-27T11:27:25.062649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:41:33.495927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060178+0000","last_change":"2023-08-27T11:27:25.060178+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060178+0000","last_undegraded":"2023-08-27T11:27:25.060178+0000","last_fullsized":"2023-08-27T11:27:25.060178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078664+0000","last_change":"2023-08-27T11:27:25.075707+0000","last_active":"2023-08-27T11:27:25.078664+0000","last_peered":"2023-08-27T11:27:25.078664+0000","last_clean":"2023-08-27T11:27:25.078664+0000","last_became_active":"2023-08-27T11:27:25.074470+0000","last_became_peered":"2023-08-27T11:27:25.074470+0000","last_unstale":"2023-08-27T11:27:25.078664+0000","last_undegraded":"2023-08-27T11:27:25.078664+0000","last_fullsized":"2023-08-27T11:27:25.078664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:23.840169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":140,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.065518+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:27:25.065518+0000","last_peered":"2023-08-27T11:27:25.065518+0000","last_clean":"2023-08-27T11:27:25.065518+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:27:25.065518+0000","last_undegraded":"2023-08-27T11:27:25.065518+0000","last_fullsized":"2023-08-27T11:27:25.065518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:14.402589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058272+0000","last_change":"2023-08-27T11:27:25.058272+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058272+0000","last_undegraded":"2023-08-27T11:27:25.058272+0000","last_fullsized":"2023-08-27T11:27:25.058272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.071038+0000","last_change":"2023-08-27T11:27:25.067095+0000","last_active":"2023-08-27T11:27:25.071038+0000","last_peered":"2023-08-27T11:27:25.071038+0000","last_clean":"2023-08-27T11:27:25.071038+0000","last_became_active":"2023-08-27T11:27:25.066815+0000","last_became_peered":"2023-08-27T11:27:25.066815+0000","last_unstale":"2023-08-27T11:27:25.071038+0000","last_undegraded":"2023-08-27T11:27:25.071038+0000","last_fullsized":"2023-08-27T11:27:25.071038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:09.030179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.055664+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:27:25.055664+0000","last_peered":"2023-08-27T11:27:25.055664+0000","last_clean":"2023-08-27T11:27:25.055664+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:27:25.055664+0000","last_undegraded":"2023-08-27T11:27:25.055664+0000","last_fullsized":"2023-08-27T11:27:25.055664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:58.626537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059736+0000","last_change":"2023-08-27T11:27:25.059736+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059736+0000","last_undegraded":"2023-08-27T11:27:25.059736+0000","last_fullsized":"2023-08-27T11:27:25.059736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078255+0000","last_change":"2023-08-27T11:27:25.071459+0000","last_active":"2023-08-27T11:27:25.078255+0000","last_peered":"2023-08-27T11:27:25.078255+0000","last_clean":"2023-08-27T11:27:25.078255+0000","last_became_active":"2023-08-27T11:27:25.070204+0000","last_became_peered":"2023-08-27T11:27:25.070204+0000","last_unstale":"2023-08-27T11:27:25.078255+0000","last_undegraded":"2023-08-27T11:27:25.078255+0000","last_fullsized":"2023-08-27T11:27:25.078255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:14.619636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.063369+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:27:25.063369+0000","last_peered":"2023-08-27T11:27:25.063369+0000","last_clean":"2023-08-27T11:27:25.063369+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:27:25.063369+0000","last_undegraded":"2023-08-27T11:27:25.063369+0000","last_fullsized":"2023-08-27T11:27:25.063369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:30:05.209195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'161","reported_seq":450,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.068653+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:27:25.068653+0000","last_peered":"2023-08-27T11:27:25.068653+0000","last_clean":"2023-08-27T11:27:25.068653+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:27:25.068653+0000","last_undegraded":"2023-08-27T11:27:25.068653+0000","last_fullsized":"2023-08-27T11:27:25.068653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:13:16.300519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060703+0000","last_change":"2023-08-27T11:27:25.060703+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060703+0000","last_undegraded":"2023-08-27T11:27:25.060703+0000","last_fullsized":"2023-08-27T11:27:25.060703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075763+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:25.075763+0000","last_peered":"2023-08-27T11:27:25.075763+0000","last_clean":"2023-08-27T11:27:25.075763+0000","last_became_active":"2023-08-27T11:27:25.074184+0000","last_became_peered":"2023-08-27T11:27:25.074184+0000","last_unstale":"2023-08-27T11:27:25.075763+0000","last_undegraded":"2023-08-27T11:27:25.075763+0000","last_fullsized":"2023-08-27T11:27:25.075763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:39.493382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.068797+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:27:25.068797+0000","last_peered":"2023-08-27T11:27:25.068797+0000","last_clean":"2023-08-27T11:27:25.068797+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:27:25.068797+0000","last_undegraded":"2023-08-27T11:27:25.068797+0000","last_fullsized":"2023-08-27T11:27:25.068797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:41.138066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059469+0000","last_change":"2023-08-27T11:27:25.059469+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059469+0000","last_undegraded":"2023-08-27T11:27:25.059469+0000","last_fullsized":"2023-08-27T11:27:25.059469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078781+0000","last_change":"2023-08-27T11:27:25.077520+0000","last_active":"2023-08-27T11:27:25.078781+0000","last_peered":"2023-08-27T11:27:25.078781+0000","last_clean":"2023-08-27T11:27:25.078781+0000","last_became_active":"2023-08-27T11:27:25.077249+0000","last_became_peered":"2023-08-27T11:27:25.077249+0000","last_unstale":"2023-08-27T11:27:25.078781+0000","last_undegraded":"2023-08-27T11:27:25.078781+0000","last_fullsized":"2023-08-27T11:27:25.078781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:50:59.249147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.070463+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:27:25.070463+0000","last_peered":"2023-08-27T11:27:25.070463+0000","last_clean":"2023-08-27T11:27:25.070463+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:27:25.070463+0000","last_undegraded":"2023-08-27T11:27:25.070463+0000","last_fullsized":"2023-08-27T11:27:25.070463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:00.142203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064566+0000","last_change":"2023-08-27T11:27:25.064566+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064566+0000","last_undegraded":"2023-08-27T11:27:25.064566+0000","last_fullsized":"2023-08-27T11:27:25.064566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.079974+0000","last_change":"2023-08-27T11:27:25.078799+0000","last_active":"2023-08-27T11:27:25.079974+0000","last_peered":"2023-08-27T11:27:25.079974+0000","last_clean":"2023-08-27T11:27:25.079974+0000","last_became_active":"2023-08-27T11:27:25.078503+0000","last_became_peered":"2023-08-27T11:27:25.078503+0000","last_unstale":"2023-08-27T11:27:25.079974+0000","last_undegraded":"2023-08-27T11:27:25.079974+0000","last_fullsized":"2023-08-27T11:27:25.079974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:04.557286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.062460+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:27:25.062460+0000","last_peered":"2023-08-27T11:27:25.062460+0000","last_clean":"2023-08-27T11:27:25.062460+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:27:25.062460+0000","last_undegraded":"2023-08-27T11:27:25.062460+0000","last_fullsized":"2023-08-27T11:27:25.062460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:07.961836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060427+0000","last_change":"2023-08-27T11:27:25.060427+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060427+0000","last_undegraded":"2023-08-27T11:27:25.060427+0000","last_fullsized":"2023-08-27T11:27:25.060427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078679+0000","last_change":"2023-08-27T11:27:25.074341+0000","last_active":"2023-08-27T11:27:25.078679+0000","last_peered":"2023-08-27T11:27:25.078679+0000","last_clean":"2023-08-27T11:27:25.078679+0000","last_became_active":"2023-08-27T11:27:25.073689+0000","last_became_peered":"2023-08-27T11:27:25.073689+0000","last_unstale":"2023-08-27T11:27:25.078679+0000","last_undegraded":"2023-08-27T11:27:25.078679+0000","last_fullsized":"2023-08-27T11:27:25.078679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:44.085030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073395+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:27:25.073395+0000","last_peered":"2023-08-27T11:27:25.073395+0000","last_clean":"2023-08-27T11:27:25.073395+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:27:25.073395+0000","last_undegraded":"2023-08-27T11:27:25.073395+0000","last_fullsized":"2023-08-27T11:27:25.073395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:10.167081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575034,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1601536,"data_stored":1113000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":1.601,"15min":1.601},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40899999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.72199999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.29099999999999998},"max":{"1min":0.85499999999999998,"5min":1.778,"15min":1.778},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63500000000000001,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63600000000000001,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.67400000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.432,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68400000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56699999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48299999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88300000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.50600000000000001},"min":{"1min":0.29799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575034,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.46000000000000002,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.752,"5min":0.81100000000000005,"15min":1.3540000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.44800000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":3.3780000000000001},"last":0.501},{"interface":"front","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.36899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.77900000000000003,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.438}]},{"osd":2,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.64800000000000002},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.71199999999999997,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72099999999999997,"5min":0.77500000000000002,"15min":1.325},"last":0.33700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57499999999999996,"15min":0.63100000000000001},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.34100000000000003},"max":{"1min":0.79400000000000004,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.48999999999999999,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78200000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.38200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47899999999999998,"15min":0.46400000000000002},"min":{"1min":0.32100000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.71599999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44900000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.502},"min":{"1min":0.34799999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":1.157,"15min":1.157},"last":0.40600000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.77500000000000002,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34899999999999998},"max":{"1min":0.74099999999999999,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.42199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1019904,"data_stored":529624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.38900000000000001},"min":{"1min":0.26500000000000001,"5min":0.22800000000000001,"15min":0.20399999999999999},"max":{"1min":0.53500000000000003,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.29699999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.58999999999999997,"5min":0.61099999999999999,"15min":0.73699999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.49099999999999999,"15min":0.48099999999999998},"min":{"1min":0.36099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.34100000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.48399999999999999},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.33100000000000002,"5min":0.28599999999999998,"15min":0.23400000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.34799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.52700000000000002,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.495},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.55000000000000004,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.88600000000000001,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.40999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.92300000000000004,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.45000000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.56899999999999995,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.60899999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67800000000000005,"15min":0.68100000000000005},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.89400000000000002,"5min":0.96199999999999997,"15min":1.139},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.57199999999999995,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.89800000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.629}]}]},{"osd":0,"up_from":13,"seq":55834575034,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1613824,"data_stored":1119992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.495},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.44700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.59799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.85899999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.92800000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.75},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.84199999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.76700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575035,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.31900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.441}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.59699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.41199999999999998,"15min":0.46000000000000002},"min":{"1min":0.25,"5min":0.23899999999999999,"15min":0.22},"max":{"1min":0.98699999999999999,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.27000000000000002,"5min":0.26900000000000002,"15min":0.224},"max":{"1min":0.61299999999999999,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.38600000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42199999999999999,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.26100000000000001,"15min":0.255},"max":{"1min":0.55800000000000005,"5min":0.64300000000000002,"15min":6.0830000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47399999999999998,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.245},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.25},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":11.939},"last":0.497}]},{"osd":4,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65200000000000002,"15min":0.70599999999999996},"min":{"1min":0.55500000000000005,"5min":0.39700000000000002,"15min":0.376},"max":{"1min":0.99399999999999999,"5min":1.0329999999999999,"15min":11.007},"last":0.752},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":1.113,"15min":1.2929999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70399999999999996,"15min":0.751},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.436},"max":{"1min":0.95099999999999996,"5min":3.0219999999999998,"15min":11.981},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68700000000000006,"15min":0.72999999999999998},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":1.0620000000000001,"5min":1.218,"15min":11.09},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.80400000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.97199999999999998,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72899999999999998,"15min":0.78800000000000003},"min":{"1min":0.54600000000000004,"5min":0.503,"15min":0.47799999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":12.016},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.92100000000000004,"5min":1.006,"15min":11.054},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.51900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0389999999999999,"5min":1.0640000000000001,"15min":1.361},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575034,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.58599999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48699999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.502},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.38800000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40899999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.57499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335516,"statfs":{"total":96636764160,"available":96599568384,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.62},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57399999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.51400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:27:30.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:27:30.371 INFO:tasks.ceph.mds.b.smithi135.stderr:2023-08-27T11:27:30.369+0000 7f10058dc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:27:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.409+0000 7fb7907da700 1 Processor -- start 2023-08-27T11:27:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.409+0000 7fb7907da700 1 -- start start 2023-08-27T11:27:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb7907da700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb78806bda0 0x7fb78806c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb7907da700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7880cff50 con 0x7fb78815d140 2023-08-27T11:27:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb7907da700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7880cc4e0 con 0x7fb7880cc790 2023-08-27T11:27:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb7907da700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880cd330 con 0x7fb78806bda0 2023-08-27T11:27:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78e576700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb78815d140 0x7fb7880ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44562/0 (socket says 172.21.15.45:44562) 2023-08-27T11:27:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78e576700 1 -- 172.21.15.45:0/2601285086 learned_addr learned my addr 172.21.15.45:0/2601285086 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78dd75700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb78806bda0 0x7fb78806c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783608370 0 0) 0x7fb7880cff50 con 0x7fb78815d140 2023-08-27T11:27:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7740035b0 con 0x7fb78815d140 2023-08-27T11:27:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1098422707 0 0) 0x7fb7740035b0 con 0x7fb78815d140 2023-08-27T11:27:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb78806bda0 msgr2=0x7fb78806c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78d574700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb78806bda0 0x7fb78806c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 >> v1:172.21.15.135:6789/0 conn(0x7fb7880cc790 legacy=0x7fb7880ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.410+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7880ce820 con 0x7fb78815d140 2023-08-27T11:27:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fb784001670 con 0x7fb78815d140 2023-08-27T11:27:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 >> v1:172.21.15.45:6789/0 conn(0x7fb78815d140 legacy=0x7fb7880ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb78806bda0 0x7fb774003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774003fb0 0x7fb774004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb774005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880cd330 con 0x7fb774004d90 2023-08-27T11:27:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77400bba0 con 0x7fb774003fb0 2023-08-27T11:27:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77400bd60 con 0x7fb78806bda0 2023-08-27T11:27:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78e576700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb78806bda0 0x7fb774003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78ed77700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb774005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78dd75700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774003fb0 0x7fb774004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78ed77700 1 -- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb78806bda0 msgr2=0x7fb774003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78ed77700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb78806bda0 0x7fb774003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78ed77700 1 -- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774003fb0 msgr2=0x7fb774004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78ed77700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb774003fb0 0x7fb774004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78ed77700 1 -- 172.21.15.45:0/2601285086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740035b0 con 0x7fb774004d90 2023-08-27T11:27:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78ed77700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb774005210 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c190 tx=0x7fb774005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ff63d8e3ee4f9f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780001db0 con 0x7fb774004d90 2023-08-27T11:27:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78000ccd0 con 0x7fb774004d90 2023-08-27T11:27:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.411+0000 7fb78d574700 1 -- 172.21.15.45:0/2601285086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780003c40 con 0x7fb774004d90 2023-08-27T11:27:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.412+0000 7fb7907da700 1 -- 172.21.15.45:0/2601285086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 msgr2=0x7fb774005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.412+0000 7fb7907da700 1 --2- 172.21.15.45:0/2601285086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb774005210 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c190 tx=0x7fb774005880 comp rx=0 tx=0).stop 2023-08-27T11:27:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.412+0000 7fb78ed77700 1 -- 172.21.15.45:0/2601285086 reap_dead start 2023-08-27T11:27:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.412+0000 7fb7907da700 1 -- 172.21.15.45:0/2601285086 shutdown_connections 2023-08-27T11:27:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.412+0000 7fb7907da700 1 -- 172.21.15.45:0/2601285086 >> 172.21.15.45:0/2601285086 conn(0x7fb78809a160 msgr2=0x7fb7880d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.412+0000 7fb7907da700 1 -- 172.21.15.45:0/2601285086 shutdown_connections 2023-08-27T11:27:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.412+0000 7fb7907da700 1 -- 172.21.15.45:0/2601285086 wait complete. 2023-08-27T11:27:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.413+0000 7fb7907da700 1 Processor -- start 2023-08-27T11:27:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.413+0000 7fb7907da700 1 -- start start 2023-08-27T11:27:30.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb7907da700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb7880b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb7907da700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7880b0250 0x7fb78815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb7907da700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7881511f0 0x7fb7880b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb7907da700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880b1370 con 0x7fb774004d90 2023-08-27T11:27:30.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb7907da700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78815dc20 con 0x7fb7880b0250 2023-08-27T11:27:30.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb7907da700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880b40f0 con 0x7fb7881511f0 2023-08-27T11:27:30.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb78e576700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb7880b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb78e576700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb7880b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57798/0 (socket says 172.21.15.45:57798) 2023-08-27T11:27:30.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb78e576700 1 -- 172.21.15.45:0/277578191 learned_addr learned my addr 172.21.15.45:0/277578191 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:30.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb78dd75700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7880b0250 0x7fb78815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb78ed77700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7881511f0 0x7fb7880b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb78e576700 1 -- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7881511f0 msgr2=0x7fb7880b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.414+0000 7fb78e576700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7881511f0 0x7fb7880b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb78e576700 1 -- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7880b0250 msgr2=0x7fb78815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb78e576700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7880b0250 0x7fb78815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb78e576700 1 -- 172.21.15.45:0/277578191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7880b43d0 con 0x7fb774004d90 2023-08-27T11:27:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb78dd75700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7880b0250 0x7fb78815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb78e576700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb7880b39f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb7840072c0 tx=0x7fb784007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb77effd700 1 -- 172.21.15.45:0/277578191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78400f8d0 con 0x7fb774004d90 2023-08-27T11:27:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb77effd700 1 -- 172.21.15.45:0/277578191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78400fa30 con 0x7fb774004d90 2023-08-27T11:27:30.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7880b46b0 con 0x7fb774004d90 2023-08-27T11:27:30.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb77effd700 1 -- 172.21.15.45:0/277578191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78400fde0 con 0x7fb774004d90 2023-08-27T11:27:30.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.415+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7880b4c70 con 0x7fb774004d90 2023-08-27T11:27:30.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.416+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7880b4e90 con 0x7fb774004d90 2023-08-27T11:27:30.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.416+0000 7fb77effd700 1 -- 172.21.15.45:0/277578191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb78401d460 con 0x7fb774004d90 2023-08-27T11:27:30.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.417+0000 7fb77effd700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb76c06a200 0x7fb76c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.417+0000 7fb77effd700 1 -- 172.21.15.45:0/277578191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fb784088a00 con 0x7fb774004d90 2023-08-27T11:27:30.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.417+0000 7fb78dd75700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb76c06a200 0x7fb76c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.417+0000 7fb78dd75700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb76c06a200 0x7fb76c06c6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb7880ae750 tx=0x7fb778008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.421+0000 7fb77effd700 1 -- 172.21.15.45:0/277578191 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb784053de0 con 0x7fb774004d90 2023-08-27T11:27:30.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.621+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 --> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7880cf340 con 0x7fb76c06a200 2023-08-27T11:27:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.630+0000 7fb77effd700 1 -- 172.21.15.45:0/277578191 <== mgr.6247 v2:172.21.15.135:6832/3430028400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306448 (secure 0 0 0) 0x7fb778014950 con 0x7fb76c06a200 2023-08-27T11:27:30.634 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb76c06a200 msgr2=0x7fb76c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb76c06a200 0x7fb76c06c6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb7880ae750 tx=0x7fb778008040 comp rx=0 tx=0).stop 2023-08-27T11:27:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 msgr2=0x7fb7880b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb7880b39f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb7840072c0 tx=0x7fb784007680 comp rx=0 tx=0).stop 2023-08-27T11:27:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 shutdown_connections 2023-08-27T11:27:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fb76c06a200 0x7fb76c06c6c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb774004d90 0x7fb7880b39f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7881511f0 0x7fb7880b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 --2- 172.21.15.45:0/277578191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7880b0250 0x7fb78815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 >> 172.21.15.45:0/277578191 conn(0x7fb78809a160 msgr2=0x7fb78815c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 shutdown_connections 2023-08-27T11:27:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.633+0000 7fb7907da700 1 -- 172.21.15.45:0/277578191 wait complete. 2023-08-27T11:27:30.640 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:27:30.646 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:27:29.350988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266824,"kb_used_data":9056,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754707896,"statfs":{"total":773094113280,"available":772820885504,"internally_reserved":0,"allocated":9273344,"data_stored":5375776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.304515"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065740+0000","last_change":"2023-08-27T11:27:25.065740+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065740+0000","last_undegraded":"2023-08-27T11:27:25.065740+0000","last_fullsized":"2023-08-27T11:27:25.065740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062228+0000","last_change":"2023-08-27T11:27:25.062228+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062228+0000","last_undegraded":"2023-08-27T11:27:25.062228+0000","last_fullsized":"2023-08-27T11:27:25.062228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060843+0000","last_change":"2023-08-27T11:27:25.060843+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060843+0000","last_undegraded":"2023-08-27T11:27:25.060843+0000","last_fullsized":"2023-08-27T11:27:25.060843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060288+0000","last_change":"2023-08-27T11:27:25.060288+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060288+0000","last_undegraded":"2023-08-27T11:27:25.060288+0000","last_fullsized":"2023-08-27T11:27:25.060288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059387+0000","last_change":"2023-08-27T11:27:25.059387+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059387+0000","last_undegraded":"2023-08-27T11:27:25.059387+0000","last_fullsized":"2023-08-27T11:27:25.059387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062065+0000","last_change":"2023-08-27T11:27:25.062065+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062065+0000","last_undegraded":"2023-08-27T11:27:25.062065+0000","last_fullsized":"2023-08-27T11:27:25.062065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063011+0000","last_change":"2023-08-27T11:27:25.063011+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063011+0000","last_undegraded":"2023-08-27T11:27:25.063011+0000","last_fullsized":"2023-08-27T11:27:25.063011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062097+0000","last_change":"2023-08-27T11:27:25.062097+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062097+0000","last_undegraded":"2023-08-27T11:27:25.062097+0000","last_fullsized":"2023-08-27T11:27:25.062097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058937+0000","last_change":"2023-08-27T11:27:25.058937+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058937+0000","last_undegraded":"2023-08-27T11:27:25.058937+0000","last_fullsized":"2023-08-27T11:27:25.058937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060848+0000","last_change":"2023-08-27T11:27:25.060848+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060848+0000","last_undegraded":"2023-08-27T11:27:25.060848+0000","last_fullsized":"2023-08-27T11:27:25.060848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065903+0000","last_change":"2023-08-27T11:27:25.065903+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065903+0000","last_undegraded":"2023-08-27T11:27:25.065903+0000","last_fullsized":"2023-08-27T11:27:25.065903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059898+0000","last_change":"2023-08-27T11:27:25.059898+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059898+0000","last_undegraded":"2023-08-27T11:27:25.059898+0000","last_fullsized":"2023-08-27T11:27:25.059898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063681+0000","last_change":"2023-08-27T11:27:25.063681+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063681+0000","last_undegraded":"2023-08-27T11:27:25.063681+0000","last_fullsized":"2023-08-27T11:27:25.063681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058054+0000","last_change":"2023-08-27T11:27:25.058054+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058054+0000","last_undegraded":"2023-08-27T11:27:25.058054+0000","last_fullsized":"2023-08-27T11:27:25.058054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066424+0000","last_change":"2023-08-27T11:27:25.066424+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066424+0000","last_undegraded":"2023-08-27T11:27:25.066424+0000","last_fullsized":"2023-08-27T11:27:25.066424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059416+0000","last_change":"2023-08-27T11:27:25.059416+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059416+0000","last_undegraded":"2023-08-27T11:27:25.059416+0000","last_fullsized":"2023-08-27T11:27:25.059416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058352+0000","last_change":"2023-08-27T11:27:25.058352+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058352+0000","last_undegraded":"2023-08-27T11:27:25.058352+0000","last_fullsized":"2023-08-27T11:27:25.058352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059190+0000","last_change":"2023-08-27T11:27:25.059190+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059190+0000","last_undegraded":"2023-08-27T11:27:25.059190+0000","last_fullsized":"2023-08-27T11:27:25.059190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066698+0000","last_change":"2023-08-27T11:27:25.066698+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066698+0000","last_undegraded":"2023-08-27T11:27:25.066698+0000","last_fullsized":"2023-08-27T11:27:25.066698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063155+0000","last_change":"2023-08-27T11:27:25.063155+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063155+0000","last_undegraded":"2023-08-27T11:27:25.063155+0000","last_fullsized":"2023-08-27T11:27:25.063155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061527+0000","last_change":"2023-08-27T11:27:25.061527+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061527+0000","last_undegraded":"2023-08-27T11:27:25.061527+0000","last_fullsized":"2023-08-27T11:27:25.061527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059917+0000","last_change":"2023-08-27T11:27:25.059917+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059917+0000","last_undegraded":"2023-08-27T11:27:25.059917+0000","last_fullsized":"2023-08-27T11:27:25.059917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064806+0000","last_change":"2023-08-27T11:27:25.064806+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064806+0000","last_undegraded":"2023-08-27T11:27:25.064806+0000","last_fullsized":"2023-08-27T11:27:25.064806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064119+0000","last_change":"2023-08-27T11:27:25.064119+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064119+0000","last_undegraded":"2023-08-27T11:27:25.064119+0000","last_fullsized":"2023-08-27T11:27:25.064119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060052+0000","last_change":"2023-08-27T11:27:25.060052+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060052+0000","last_undegraded":"2023-08-27T11:27:25.060052+0000","last_fullsized":"2023-08-27T11:27:25.060052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065794+0000","last_change":"2023-08-27T11:27:25.065794+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065794+0000","last_undegraded":"2023-08-27T11:27:25.065794+0000","last_fullsized":"2023-08-27T11:27:25.065794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060404+0000","last_change":"2023-08-27T11:27:25.060404+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060404+0000","last_undegraded":"2023-08-27T11:27:25.060404+0000","last_fullsized":"2023-08-27T11:27:25.060404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060723+0000","last_change":"2023-08-27T11:27:25.060723+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060723+0000","last_undegraded":"2023-08-27T11:27:25.060723+0000","last_fullsized":"2023-08-27T11:27:25.060723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060613+0000","last_change":"2023-08-27T11:27:25.060613+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060613+0000","last_undegraded":"2023-08-27T11:27:25.060613+0000","last_fullsized":"2023-08-27T11:27:25.060613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065228+0000","last_change":"2023-08-27T11:27:25.065228+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065228+0000","last_undegraded":"2023-08-27T11:27:25.065228+0000","last_fullsized":"2023-08-27T11:27:25.065228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062163+0000","last_change":"2023-08-27T11:27:25.062163+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062163+0000","last_undegraded":"2023-08-27T11:27:25.062163+0000","last_fullsized":"2023-08-27T11:27:25.062163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061179+0000","last_change":"2023-08-27T11:27:25.061179+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061179+0000","last_undegraded":"2023-08-27T11:27:25.061179+0000","last_fullsized":"2023-08-27T11:27:25.061179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064503+0000","last_change":"2023-08-27T11:27:25.064503+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064503+0000","last_undegraded":"2023-08-27T11:27:25.064503+0000","last_fullsized":"2023-08-27T11:27:25.064503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075864+0000","last_change":"2023-08-27T11:27:25.074467+0000","last_active":"2023-08-27T11:27:25.075864+0000","last_peered":"2023-08-27T11:27:25.075864+0000","last_clean":"2023-08-27T11:27:25.075864+0000","last_became_active":"2023-08-27T11:27:25.074201+0000","last_became_peered":"2023-08-27T11:27:25.074201+0000","last_unstale":"2023-08-27T11:27:25.075864+0000","last_undegraded":"2023-08-27T11:27:25.075864+0000","last_fullsized":"2023-08-27T11:27:25.075864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:36:13.473546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057922+0000","last_change":"2023-08-27T11:27:25.057922+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057922+0000","last_undegraded":"2023-08-27T11:27:25.057922+0000","last_fullsized":"2023-08-27T11:27:25.057922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075576+0000","last_change":"2023-08-27T11:27:25.072826+0000","last_active":"2023-08-27T11:27:25.075576+0000","last_peered":"2023-08-27T11:27:25.075576+0000","last_clean":"2023-08-27T11:27:25.075576+0000","last_became_active":"2023-08-27T11:27:25.072493+0000","last_became_peered":"2023-08-27T11:27:25.072493+0000","last_unstale":"2023-08-27T11:27:25.075576+0000","last_undegraded":"2023-08-27T11:27:25.075576+0000","last_fullsized":"2023-08-27T11:27:25.075576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:08.223558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.067551+0000","last_change":"2023-08-27T11:27:25.067551+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.067551+0000","last_undegraded":"2023-08-27T11:27:25.067551+0000","last_fullsized":"2023-08-27T11:27:25.067551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075945+0000","last_change":"2023-08-27T11:27:25.075211+0000","last_active":"2023-08-27T11:27:25.075945+0000","last_peered":"2023-08-27T11:27:25.075945+0000","last_clean":"2023-08-27T11:27:25.075945+0000","last_became_active":"2023-08-27T11:27:25.074962+0000","last_became_peered":"2023-08-27T11:27:25.074962+0000","last_unstale":"2023-08-27T11:27:25.075945+0000","last_undegraded":"2023-08-27T11:27:25.075945+0000","last_fullsized":"2023-08-27T11:27:25.075945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:48.066087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065441+0000","last_change":"2023-08-27T11:27:25.065441+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065441+0000","last_undegraded":"2023-08-27T11:27:25.065441+0000","last_fullsized":"2023-08-27T11:27:25.065441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078193+0000","last_change":"2023-08-27T11:27:25.070402+0000","last_active":"2023-08-27T11:27:25.078193+0000","last_peered":"2023-08-27T11:27:25.078193+0000","last_clean":"2023-08-27T11:27:25.078193+0000","last_became_active":"2023-08-27T11:27:25.070112+0000","last_became_peered":"2023-08-27T11:27:25.070112+0000","last_unstale":"2023-08-27T11:27:25.078193+0000","last_undegraded":"2023-08-27T11:27:25.078193+0000","last_fullsized":"2023-08-27T11:27:25.078193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:46.302771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062102+0000","last_change":"2023-08-27T11:27:25.062102+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062102+0000","last_undegraded":"2023-08-27T11:27:25.062102+0000","last_fullsized":"2023-08-27T11:27:25.062102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073084+0000","last_change":"2023-08-27T11:27:25.071009+0000","last_active":"2023-08-27T11:27:25.073084+0000","last_peered":"2023-08-27T11:27:25.073084+0000","last_clean":"2023-08-27T11:27:25.073084+0000","last_became_active":"2023-08-27T11:27:25.070686+0000","last_became_peered":"2023-08-27T11:27:25.070686+0000","last_unstale":"2023-08-27T11:27:25.073084+0000","last_undegraded":"2023-08-27T11:27:25.073084+0000","last_fullsized":"2023-08-27T11:27:25.073084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:34:52.384052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061142+0000","last_change":"2023-08-27T11:27:25.061142+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061142+0000","last_undegraded":"2023-08-27T11:27:25.061142+0000","last_fullsized":"2023-08-27T11:27:25.061142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078714+0000","last_change":"2023-08-27T11:27:25.076385+0000","last_active":"2023-08-27T11:27:25.078714+0000","last_peered":"2023-08-27T11:27:25.078714+0000","last_clean":"2023-08-27T11:27:25.078714+0000","last_became_active":"2023-08-27T11:27:25.076062+0000","last_became_peered":"2023-08-27T11:27:25.076062+0000","last_unstale":"2023-08-27T11:27:25.078714+0000","last_undegraded":"2023-08-27T11:27:25.078714+0000","last_fullsized":"2023-08-27T11:27:25.078714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:59:06.875963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057837+0000","last_change":"2023-08-27T11:27:25.057837+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057837+0000","last_undegraded":"2023-08-27T11:27:25.057837+0000","last_fullsized":"2023-08-27T11:27:25.057837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.071644+0000","last_change":"2023-08-27T11:27:25.068965+0000","last_active":"2023-08-27T11:27:25.071644+0000","last_peered":"2023-08-27T11:27:25.071644+0000","last_clean":"2023-08-27T11:27:25.071644+0000","last_became_active":"2023-08-27T11:27:25.068679+0000","last_became_peered":"2023-08-27T11:27:25.068679+0000","last_unstale":"2023-08-27T11:27:25.071644+0000","last_undegraded":"2023-08-27T11:27:25.071644+0000","last_fullsized":"2023-08-27T11:27:25.071644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:59.810868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064147+0000","last_change":"2023-08-27T11:27:25.064147+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064147+0000","last_undegraded":"2023-08-27T11:27:25.064147+0000","last_fullsized":"2023-08-27T11:27:25.064147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.066727+0000","last_change":"2023-08-27T11:27:25.065522+0000","last_active":"2023-08-27T11:27:25.066727+0000","last_peered":"2023-08-27T11:27:25.066727+0000","last_clean":"2023-08-27T11:27:25.066727+0000","last_became_active":"2023-08-27T11:27:25.065251+0000","last_became_peered":"2023-08-27T11:27:25.065251+0000","last_unstale":"2023-08-27T11:27:25.066727+0000","last_undegraded":"2023-08-27T11:27:25.066727+0000","last_fullsized":"2023-08-27T11:27:25.066727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:05.350508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063105+0000","last_change":"2023-08-27T11:27:25.063105+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063105+0000","last_undegraded":"2023-08-27T11:27:25.063105+0000","last_fullsized":"2023-08-27T11:27:25.063105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075138+0000","last_change":"2023-08-27T11:27:25.068373+0000","last_active":"2023-08-27T11:27:25.075138+0000","last_peered":"2023-08-27T11:27:25.075138+0000","last_clean":"2023-08-27T11:27:25.075138+0000","last_became_active":"2023-08-27T11:27:25.068082+0000","last_became_peered":"2023-08-27T11:27:25.068082+0000","last_unstale":"2023-08-27T11:27:25.075138+0000","last_undegraded":"2023-08-27T11:27:25.075138+0000","last_fullsized":"2023-08-27T11:27:25.075138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:07.411357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064080+0000","last_change":"2023-08-27T11:27:25.064080+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064080+0000","last_undegraded":"2023-08-27T11:27:25.064080+0000","last_fullsized":"2023-08-27T11:27:25.064080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078741+0000","last_change":"2023-08-27T11:27:25.076541+0000","last_active":"2023-08-27T11:27:25.078741+0000","last_peered":"2023-08-27T11:27:25.078741+0000","last_clean":"2023-08-27T11:27:25.078741+0000","last_became_active":"2023-08-27T11:27:25.076262+0000","last_became_peered":"2023-08-27T11:27:25.076262+0000","last_unstale":"2023-08-27T11:27:25.078741+0000","last_undegraded":"2023-08-27T11:27:25.078741+0000","last_fullsized":"2023-08-27T11:27:25.078741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:59:00.006614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066919+0000","last_change":"2023-08-27T11:27:25.066919+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066919+0000","last_undegraded":"2023-08-27T11:27:25.066919+0000","last_fullsized":"2023-08-27T11:27:25.066919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.068584+0000","last_change":"2023-08-27T11:27:25.068119+0000","last_active":"2023-08-27T11:27:25.068584+0000","last_peered":"2023-08-27T11:27:25.068584+0000","last_clean":"2023-08-27T11:27:25.068584+0000","last_became_active":"2023-08-27T11:27:25.067846+0000","last_became_peered":"2023-08-27T11:27:25.067846+0000","last_unstale":"2023-08-27T11:27:25.068584+0000","last_undegraded":"2023-08-27T11:27:25.068584+0000","last_fullsized":"2023-08-27T11:27:25.068584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:43.039809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058952+0000","last_change":"2023-08-27T11:27:25.058952+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058952+0000","last_undegraded":"2023-08-27T11:27:25.058952+0000","last_fullsized":"2023-08-27T11:27:25.058952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.067165+0000","last_change":"2023-08-27T11:27:25.065700+0000","last_active":"2023-08-27T11:27:25.067165+0000","last_peered":"2023-08-27T11:27:25.067165+0000","last_clean":"2023-08-27T11:27:25.067165+0000","last_became_active":"2023-08-27T11:27:25.065409+0000","last_became_peered":"2023-08-27T11:27:25.065409+0000","last_unstale":"2023-08-27T11:27:25.067165+0000","last_undegraded":"2023-08-27T11:27:25.067165+0000","last_fullsized":"2023-08-27T11:27:25.067165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:18.780769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066322+0000","last_change":"2023-08-27T11:27:25.066322+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066322+0000","last_undegraded":"2023-08-27T11:27:25.066322+0000","last_fullsized":"2023-08-27T11:27:25.066322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073580+0000","last_change":"2023-08-27T11:27:25.070284+0000","last_active":"2023-08-27T11:27:25.073580+0000","last_peered":"2023-08-27T11:27:25.073580+0000","last_clean":"2023-08-27T11:27:25.073580+0000","last_became_active":"2023-08-27T11:27:25.069996+0000","last_became_peered":"2023-08-27T11:27:25.069996+0000","last_unstale":"2023-08-27T11:27:25.073580+0000","last_undegraded":"2023-08-27T11:27:25.073580+0000","last_fullsized":"2023-08-27T11:27:25.073580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:00.139285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061933+0000","last_change":"2023-08-27T11:27:25.061933+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061933+0000","last_undegraded":"2023-08-27T11:27:25.061933+0000","last_fullsized":"2023-08-27T11:27:25.061933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078214+0000","last_change":"2023-08-27T11:27:25.070398+0000","last_active":"2023-08-27T11:27:25.078214+0000","last_peered":"2023-08-27T11:27:25.078214+0000","last_clean":"2023-08-27T11:27:25.078214+0000","last_became_active":"2023-08-27T11:27:25.070052+0000","last_became_peered":"2023-08-27T11:27:25.070052+0000","last_unstale":"2023-08-27T11:27:25.078214+0000","last_undegraded":"2023-08-27T11:27:25.078214+0000","last_fullsized":"2023-08-27T11:27:25.078214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:39.907863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062446+0000","last_change":"2023-08-27T11:27:25.062446+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062446+0000","last_undegraded":"2023-08-27T11:27:25.062446+0000","last_fullsized":"2023-08-27T11:27:25.062446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073733+0000","last_change":"2023-08-27T11:27:25.072045+0000","last_active":"2023-08-27T11:27:25.073733+0000","last_peered":"2023-08-27T11:27:25.073733+0000","last_clean":"2023-08-27T11:27:25.073733+0000","last_became_active":"2023-08-27T11:27:25.071747+0000","last_became_peered":"2023-08-27T11:27:25.071747+0000","last_unstale":"2023-08-27T11:27:25.073733+0000","last_undegraded":"2023-08-27T11:27:25.073733+0000","last_fullsized":"2023-08-27T11:27:25.073733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:53.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057521+0000","last_change":"2023-08-27T11:27:25.057521+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057521+0000","last_undegraded":"2023-08-27T11:27:25.057521+0000","last_fullsized":"2023-08-27T11:27:25.057521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075545+0000","last_change":"2023-08-27T11:27:25.072388+0000","last_active":"2023-08-27T11:27:25.075545+0000","last_peered":"2023-08-27T11:27:25.075545+0000","last_clean":"2023-08-27T11:27:25.075545+0000","last_became_active":"2023-08-27T11:27:25.072093+0000","last_became_peered":"2023-08-27T11:27:25.072093+0000","last_unstale":"2023-08-27T11:27:25.075545+0000","last_undegraded":"2023-08-27T11:27:25.075545+0000","last_fullsized":"2023-08-27T11:27:25.075545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:59:37.328310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063415+0000","last_change":"2023-08-27T11:27:25.063415+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063415+0000","last_undegraded":"2023-08-27T11:27:25.063415+0000","last_fullsized":"2023-08-27T11:27:25.063415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078786+0000","last_change":"2023-08-27T11:27:25.076200+0000","last_active":"2023-08-27T11:27:25.078786+0000","last_peered":"2023-08-27T11:27:25.078786+0000","last_clean":"2023-08-27T11:27:25.078786+0000","last_became_active":"2023-08-27T11:27:25.074425+0000","last_became_peered":"2023-08-27T11:27:25.074425+0000","last_unstale":"2023-08-27T11:27:25.078786+0000","last_undegraded":"2023-08-27T11:27:25.078786+0000","last_fullsized":"2023-08-27T11:27:25.078786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:14:08.033370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060910+0000","last_change":"2023-08-27T11:27:25.060910+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060910+0000","last_undegraded":"2023-08-27T11:27:25.060910+0000","last_fullsized":"2023-08-27T11:27:25.060910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073798+0000","last_change":"2023-08-27T11:27:25.072101+0000","last_active":"2023-08-27T11:27:25.073798+0000","last_peered":"2023-08-27T11:27:25.073798+0000","last_clean":"2023-08-27T11:27:25.073798+0000","last_became_active":"2023-08-27T11:27:25.071818+0000","last_became_peered":"2023-08-27T11:27:25.071818+0000","last_unstale":"2023-08-27T11:27:25.073798+0000","last_undegraded":"2023-08-27T11:27:25.073798+0000","last_fullsized":"2023-08-27T11:27:25.073798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:09.111047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063757+0000","last_change":"2023-08-27T11:27:25.063757+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063757+0000","last_undegraded":"2023-08-27T11:27:25.063757+0000","last_fullsized":"2023-08-27T11:27:25.063757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075266+0000","last_change":"2023-08-27T11:27:25.072164+0000","last_active":"2023-08-27T11:27:25.075266+0000","last_peered":"2023-08-27T11:27:25.075266+0000","last_clean":"2023-08-27T11:27:25.075266+0000","last_became_active":"2023-08-27T11:27:25.071847+0000","last_became_peered":"2023-08-27T11:27:25.071847+0000","last_unstale":"2023-08-27T11:27:25.075266+0000","last_undegraded":"2023-08-27T11:27:25.075266+0000","last_fullsized":"2023-08-27T11:27:25.075266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:22:02.251506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064766+0000","last_change":"2023-08-27T11:27:25.064766+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064766+0000","last_undegraded":"2023-08-27T11:27:25.064766+0000","last_fullsized":"2023-08-27T11:27:25.064766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.076129+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:25.076129+0000","last_peered":"2023-08-27T11:27:25.076129+0000","last_clean":"2023-08-27T11:27:25.076129+0000","last_became_active":"2023-08-27T11:27:25.074195+0000","last_became_peered":"2023-08-27T11:27:25.074195+0000","last_unstale":"2023-08-27T11:27:25.076129+0000","last_undegraded":"2023-08-27T11:27:25.076129+0000","last_fullsized":"2023-08-27T11:27:25.076129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:43.039809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057420+0000","last_change":"2023-08-27T11:27:25.057420+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057420+0000","last_undegraded":"2023-08-27T11:27:25.057420+0000","last_fullsized":"2023-08-27T11:27:25.057420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.066696+0000","last_change":"2023-08-27T11:27:25.065564+0000","last_active":"2023-08-27T11:27:25.066696+0000","last_peered":"2023-08-27T11:27:25.066696+0000","last_clean":"2023-08-27T11:27:25.066696+0000","last_became_active":"2023-08-27T11:27:25.065291+0000","last_became_peered":"2023-08-27T11:27:25.065291+0000","last_unstale":"2023-08-27T11:27:25.066696+0000","last_undegraded":"2023-08-27T11:27:25.066696+0000","last_fullsized":"2023-08-27T11:27:25.066696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:10:02.253904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066220+0000","last_change":"2023-08-27T11:27:25.066220+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066220+0000","last_undegraded":"2023-08-27T11:27:25.066220+0000","last_fullsized":"2023-08-27T11:27:25.066220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.072825+0000","last_change":"2023-08-27T11:27:25.069284+0000","last_active":"2023-08-27T11:27:25.072825+0000","last_peered":"2023-08-27T11:27:25.072825+0000","last_clean":"2023-08-27T11:27:25.072825+0000","last_became_active":"2023-08-27T11:27:25.068979+0000","last_became_peered":"2023-08-27T11:27:25.068979+0000","last_unstale":"2023-08-27T11:27:25.072825+0000","last_undegraded":"2023-08-27T11:27:25.072825+0000","last_fullsized":"2023-08-27T11:27:25.072825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:54:46.692552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058711+0000","last_change":"2023-08-27T11:27:25.058711+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058711+0000","last_undegraded":"2023-08-27T11:27:25.058711+0000","last_fullsized":"2023-08-27T11:27:25.058711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075539+0000","last_change":"2023-08-27T11:27:25.072420+0000","last_active":"2023-08-27T11:27:25.075539+0000","last_peered":"2023-08-27T11:27:25.075539+0000","last_clean":"2023-08-27T11:27:25.075539+0000","last_became_active":"2023-08-27T11:27:25.072117+0000","last_became_peered":"2023-08-27T11:27:25.072117+0000","last_unstale":"2023-08-27T11:27:25.075539+0000","last_undegraded":"2023-08-27T11:27:25.075539+0000","last_fullsized":"2023-08-27T11:27:25.075539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:42.396071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060737+0000","last_change":"2023-08-27T11:27:25.060737+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060737+0000","last_undegraded":"2023-08-27T11:27:25.060737+0000","last_fullsized":"2023-08-27T11:27:25.060737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073981+0000","last_change":"2023-08-27T11:27:25.069386+0000","last_active":"2023-08-27T11:27:25.073981+0000","last_peered":"2023-08-27T11:27:25.073981+0000","last_clean":"2023-08-27T11:27:25.073981+0000","last_became_active":"2023-08-27T11:27:25.069120+0000","last_became_peered":"2023-08-27T11:27:25.069120+0000","last_unstale":"2023-08-27T11:27:25.073981+0000","last_undegraded":"2023-08-27T11:27:25.073981+0000","last_fullsized":"2023-08-27T11:27:25.073981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:58.548667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057630+0000","last_change":"2023-08-27T11:27:25.057630+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057630+0000","last_undegraded":"2023-08-27T11:27:25.057630+0000","last_fullsized":"2023-08-27T11:27:25.057630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.076857+0000","last_change":"2023-08-27T11:27:25.076447+0000","last_active":"2023-08-27T11:27:25.076857+0000","last_peered":"2023-08-27T11:27:25.076857+0000","last_clean":"2023-08-27T11:27:25.076857+0000","last_became_active":"2023-08-27T11:27:25.076148+0000","last_became_peered":"2023-08-27T11:27:25.076148+0000","last_unstale":"2023-08-27T11:27:25.076857+0000","last_undegraded":"2023-08-27T11:27:25.076857+0000","last_fullsized":"2023-08-27T11:27:25.076857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:27.458908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.062649+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:27:25.062649+0000","last_peered":"2023-08-27T11:27:25.062649+0000","last_clean":"2023-08-27T11:27:25.062649+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:27:25.062649+0000","last_undegraded":"2023-08-27T11:27:25.062649+0000","last_fullsized":"2023-08-27T11:27:25.062649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:41:33.495927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060178+0000","last_change":"2023-08-27T11:27:25.060178+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060178+0000","last_undegraded":"2023-08-27T11:27:25.060178+0000","last_fullsized":"2023-08-27T11:27:25.060178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078664+0000","last_change":"2023-08-27T11:27:25.075707+0000","last_active":"2023-08-27T11:27:25.078664+0000","last_peered":"2023-08-27T11:27:25.078664+0000","last_clean":"2023-08-27T11:27:25.078664+0000","last_became_active":"2023-08-27T11:27:25.074470+0000","last_became_peered":"2023-08-27T11:27:25.074470+0000","last_unstale":"2023-08-27T11:27:25.078664+0000","last_undegraded":"2023-08-27T11:27:25.078664+0000","last_fullsized":"2023-08-27T11:27:25.078664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:23.840169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":140,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.065518+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:27:25.065518+0000","last_peered":"2023-08-27T11:27:25.065518+0000","last_clean":"2023-08-27T11:27:25.065518+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:27:25.065518+0000","last_undegraded":"2023-08-27T11:27:25.065518+0000","last_fullsized":"2023-08-27T11:27:25.065518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:14.402589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058272+0000","last_change":"2023-08-27T11:27:25.058272+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058272+0000","last_undegraded":"2023-08-27T11:27:25.058272+0000","last_fullsized":"2023-08-27T11:27:25.058272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.071038+0000","last_change":"2023-08-27T11:27:25.067095+0000","last_active":"2023-08-27T11:27:25.071038+0000","last_peered":"2023-08-27T11:27:25.071038+0000","last_clean":"2023-08-27T11:27:25.071038+0000","last_became_active":"2023-08-27T11:27:25.066815+0000","last_became_peered":"2023-08-27T11:27:25.066815+0000","last_unstale":"2023-08-27T11:27:25.071038+0000","last_undegraded":"2023-08-27T11:27:25.071038+0000","last_fullsized":"2023-08-27T11:27:25.071038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:09.030179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.055664+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:27:25.055664+0000","last_peered":"2023-08-27T11:27:25.055664+0000","last_clean":"2023-08-27T11:27:25.055664+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:27:25.055664+0000","last_undegraded":"2023-08-27T11:27:25.055664+0000","last_fullsized":"2023-08-27T11:27:25.055664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:58.626537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059736+0000","last_change":"2023-08-27T11:27:25.059736+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059736+0000","last_undegraded":"2023-08-27T11:27:25.059736+0000","last_fullsized":"2023-08-27T11:27:25.059736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078255+0000","last_change":"2023-08-27T11:27:25.071459+0000","last_active":"2023-08-27T11:27:25.078255+0000","last_peered":"2023-08-27T11:27:25.078255+0000","last_clean":"2023-08-27T11:27:25.078255+0000","last_became_active":"2023-08-27T11:27:25.070204+0000","last_became_peered":"2023-08-27T11:27:25.070204+0000","last_unstale":"2023-08-27T11:27:25.078255+0000","last_undegraded":"2023-08-27T11:27:25.078255+0000","last_fullsized":"2023-08-27T11:27:25.078255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:14.619636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.063369+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:27:25.063369+0000","last_peered":"2023-08-27T11:27:25.063369+0000","last_clean":"2023-08-27T11:27:25.063369+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:27:25.063369+0000","last_undegraded":"2023-08-27T11:27:25.063369+0000","last_fullsized":"2023-08-27T11:27:25.063369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:30:05.209195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'161","reported_seq":450,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.068653+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:27:25.068653+0000","last_peered":"2023-08-27T11:27:25.068653+0000","last_clean":"2023-08-27T11:27:25.068653+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:27:25.068653+0000","last_undegraded":"2023-08-27T11:27:25.068653+0000","last_fullsized":"2023-08-27T11:27:25.068653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:13:16.300519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060703+0000","last_change":"2023-08-27T11:27:25.060703+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060703+0000","last_undegraded":"2023-08-27T11:27:25.060703+0000","last_fullsized":"2023-08-27T11:27:25.060703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075763+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:25.075763+0000","last_peered":"2023-08-27T11:27:25.075763+0000","last_clean":"2023-08-27T11:27:25.075763+0000","last_became_active":"2023-08-27T11:27:25.074184+0000","last_became_peered":"2023-08-27T11:27:25.074184+0000","last_unstale":"2023-08-27T11:27:25.075763+0000","last_undegraded":"2023-08-27T11:27:25.075763+0000","last_fullsized":"2023-08-27T11:27:25.075763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:39.493382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.068797+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:27:25.068797+0000","last_peered":"2023-08-27T11:27:25.068797+0000","last_clean":"2023-08-27T11:27:25.068797+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:27:25.068797+0000","last_undegraded":"2023-08-27T11:27:25.068797+0000","last_fullsized":"2023-08-27T11:27:25.068797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:41.138066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059469+0000","last_change":"2023-08-27T11:27:25.059469+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059469+0000","last_undegraded":"2023-08-27T11:27:25.059469+0000","last_fullsized":"2023-08-27T11:27:25.059469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078781+0000","last_change":"2023-08-27T11:27:25.077520+0000","last_active":"2023-08-27T11:27:25.078781+0000","last_peered":"2023-08-27T11:27:25.078781+0000","last_clean":"2023-08-27T11:27:25.078781+0000","last_became_active":"2023-08-27T11:27:25.077249+0000","last_became_peered":"2023-08-27T11:27:25.077249+0000","last_unstale":"2023-08-27T11:27:25.078781+0000","last_undegraded":"2023-08-27T11:27:25.078781+0000","last_fullsized":"2023-08-27T11:27:25.078781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:50:59.249147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.070463+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:27:25.070463+0000","last_peered":"2023-08-27T11:27:25.070463+0000","last_clean":"2023-08-27T11:27:25.070463+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:27:25.070463+0000","last_undegraded":"2023-08-27T11:27:25.070463+0000","last_fullsized":"2023-08-27T11:27:25.070463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:00.142203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064566+0000","last_change":"2023-08-27T11:27:25.064566+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064566+0000","last_undegraded":"2023-08-27T11:27:25.064566+0000","last_fullsized":"2023-08-27T11:27:25.064566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.079974+0000","last_change":"2023-08-27T11:27:25.078799+0000","last_active":"2023-08-27T11:27:25.079974+0000","last_peered":"2023-08-27T11:27:25.079974+0000","last_clean":"2023-08-27T11:27:25.079974+0000","last_became_active":"2023-08-27T11:27:25.078503+0000","last_became_peered":"2023-08-27T11:27:25.078503+0000","last_unstale":"2023-08-27T11:27:25.079974+0000","last_undegraded":"2023-08-27T11:27:25.079974+0000","last_fullsized":"2023-08-27T11:27:25.079974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:04.557286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.062460+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:27:25.062460+0000","last_peered":"2023-08-27T11:27:25.062460+0000","last_clean":"2023-08-27T11:27:25.062460+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:27:25.062460+0000","last_undegraded":"2023-08-27T11:27:25.062460+0000","last_fullsized":"2023-08-27T11:27:25.062460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:07.961836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060427+0000","last_change":"2023-08-27T11:27:25.060427+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060427+0000","last_undegraded":"2023-08-27T11:27:25.060427+0000","last_fullsized":"2023-08-27T11:27:25.060427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078679+0000","last_change":"2023-08-27T11:27:25.074341+0000","last_active":"2023-08-27T11:27:25.078679+0000","last_peered":"2023-08-27T11:27:25.078679+0000","last_clean":"2023-08-27T11:27:25.078679+0000","last_became_active":"2023-08-27T11:27:25.073689+0000","last_became_peered":"2023-08-27T11:27:25.073689+0000","last_unstale":"2023-08-27T11:27:25.078679+0000","last_undegraded":"2023-08-27T11:27:25.078679+0000","last_fullsized":"2023-08-27T11:27:25.078679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:44.085030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073395+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:27:25.073395+0000","last_peered":"2023-08-27T11:27:25.073395+0000","last_clean":"2023-08-27T11:27:25.073395+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:27:25.073395+0000","last_undegraded":"2023-08-27T11:27:25.073395+0000","last_fullsized":"2023-08-27T11:27:25.073395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:10.167081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575034,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1601536,"data_stored":1113000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":1.601,"15min":1.601},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40899999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.72199999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.29099999999999998},"max":{"1min":0.85499999999999998,"5min":1.778,"15min":1.778},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63500000000000001,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63600000000000001,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.67400000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.432,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68400000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56699999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48299999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88300000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.50600000000000001},"min":{"1min":0.29799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575034,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.46000000000000002,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.752,"5min":0.81100000000000005,"15min":1.3540000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.44800000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":3.3780000000000001},"last":0.501},{"interface":"front","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.36899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.77900000000000003,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.438}]},{"osd":2,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.64800000000000002},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.71199999999999997,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72099999999999997,"5min":0.77500000000000002,"15min":1.325},"last":0.33700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57499999999999996,"15min":0.63100000000000001},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.34100000000000003},"max":{"1min":0.79400000000000004,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.48999999999999999,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78200000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.38200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47899999999999998,"15min":0.46400000000000002},"min":{"1min":0.32100000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.71599999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44900000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.502},"min":{"1min":0.34799999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":1.157,"15min":1.157},"last":0.40600000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.77500000000000002,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34899999999999998},"max":{"1min":0.74099999999999999,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.42199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1019904,"data_stored":529624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.38900000000000001},"min":{"1min":0.26500000000000001,"5min":0.22800000000000001,"15min":0.20399999999999999},"max":{"1min":0.53500000000000003,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.29699999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.58999999999999997,"5min":0.61099999999999999,"15min":0.73699999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.49099999999999999,"15min":0.48099999999999998},"min":{"1min":0.36099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.34100000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.48399999999999999},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.33100000000000002,"5min":0.28599999999999998,"15min":0.23400000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.34799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.52700000000000002,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.495},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.55000000000000004,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.88600000000000001,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.40999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.92300000000000004,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.45000000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.56899999999999995,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.60899999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67800000000000005,"15min":0.68100000000000005},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.89400000000000002,"5min":0.96199999999999997,"15min":1.139},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.57199999999999995,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.89800000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.629}]}]},{"osd":0,"up_from":13,"seq":55834575034,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1613824,"data_stored":1119992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.495},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.44700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.59799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.85899999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.92800000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.75},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.84199999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.76700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575035,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.31900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.441}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.59699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.41199999999999998,"15min":0.46000000000000002},"min":{"1min":0.25,"5min":0.23899999999999999,"15min":0.22},"max":{"1min":0.98699999999999999,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.27000000000000002,"5min":0.26900000000000002,"15min":0.224},"max":{"1min":0.61299999999999999,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.38600000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42199999999999999,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.26100000000000001,"15min":0.255},"max":{"1min":0.55800000000000005,"5min":0.64300000000000002,"15min":6.0830000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47399999999999998,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.245},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.25},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":11.939},"last":0.497}]},{"osd":4,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65200000000000002,"15min":0.70599999999999996},"min":{"1min":0.55500000000000005,"5min":0.39700000000000002,"15min":0.376},"max":{"1min":0.99399999999999999,"5min":1.0329999999999999,"15min":11.007},"last":0.752},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":1.113,"15min":1.2929999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70399999999999996,"15min":0.751},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.436},"max":{"1min":0.95099999999999996,"5min":3.0219999999999998,"15min":11.981},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68700000000000006,"15min":0.72999999999999998},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":1.0620000000000001,"5min":1.218,"15min":11.09},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.80400000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.97199999999999998,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72899999999999998,"15min":0.78800000000000003},"min":{"1min":0.54600000000000004,"5min":0.503,"15min":0.47799999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":12.016},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.92100000000000004,"5min":1.006,"15min":11.054},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.51900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0389999999999999,"5min":1.0640000000000001,"15min":1.361},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575034,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.58599999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48699999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.502},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.38800000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40899999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.57499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335516,"statfs":{"total":96636764160,"available":96599568384,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.62},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57399999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.51400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:27:30.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:27:30.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.810+0000 7f5ff7f43700 1 Processor -- start 2023-08-27T11:27:30.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.811+0000 7f5ff7f43700 1 -- start start 2023-08-27T11:27:30.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.811+0000 7f5ff7f43700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ff00720b0 0x7f5ff006a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.811+0000 7f5ff7f43700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff00ccd30 con 0x7f5ff00b2ff0 2023-08-27T11:27:30.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.811+0000 7f5ff7f43700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff00b0800 con 0x7f5ff015d630 2023-08-27T11:27:30.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.811+0000 7f5ff7f43700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff00b0ae0 con 0x7f5ff00720b0 2023-08-27T11:27:30.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.811+0000 7f5ff54de700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ff00720b0 0x7f5ff006a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff64e0700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5ff00b2ff0 0x7f5ff006abc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44586/0 (socket says 172.21.15.45:44586) 2023-08-27T11:27:30.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff54de700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ff00720b0 0x7f5ff006a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57808/0 (socket says 172.21.15.45:57808) 2023-08-27T11:27:30.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff64e0700 1 -- 172.21.15.45:0/4253779476 learned_addr learned my addr 172.21.15.45:0/4253779476 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:30.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff54de700 1 -- 172.21.15.45:0/4253779476 >> v1:172.21.15.135:6789/0 conn(0x7f5ff015d630 legacy=0x7f5ff00719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff54de700 1 -- 172.21.15.45:0/4253779476 >> v1:172.21.15.45:6789/0 conn(0x7f5ff00b2ff0 legacy=0x7f5ff006abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff54de700 1 -- 172.21.15.45:0/4253779476 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00b04b0 con 0x7f5ff00720b0 2023-08-27T11:27:30.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff54de700 1 --2- 172.21.15.45:0/4253779476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 0x7f5ff006a680 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00bae0 tx=0x7f5fec00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea16b79b4a942cf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff4cdd700 1 -- 172.21.15.45:0/4253779476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00dd30 con 0x7f5ff00720b0 2023-08-27T11:27:30.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff4cdd700 1 -- 172.21.15.45:0/4253779476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec00de90 con 0x7f5ff00720b0 2023-08-27T11:27:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.812+0000 7f5ff4cdd700 1 -- 172.21.15.45:0/4253779476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec010690 con 0x7f5ff00720b0 2023-08-27T11:27:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.813+0000 7f5ff7f43700 1 -- 172.21.15.45:0/4253779476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 msgr2=0x7f5ff006a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.813+0000 7f5ff7f43700 1 --2- 172.21.15.45:0/4253779476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 0x7f5ff006a680 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00bae0 tx=0x7f5fec00bdf0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.813+0000 7f5ff7f43700 1 -- 172.21.15.45:0/4253779476 shutdown_connections 2023-08-27T11:27:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.813+0000 7f5ff7f43700 1 --2- 172.21.15.45:0/4253779476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 0x7f5ff006a680 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.813+0000 7f5ff7f43700 1 -- 172.21.15.45:0/4253779476 >> 172.21.15.45:0/4253779476 conn(0x7f5ff00b2460 msgr2=0x7f5ff00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.813+0000 7f5ff7f43700 1 -- 172.21.15.45:0/4253779476 shutdown_connections 2023-08-27T11:27:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.813+0000 7f5ff7f43700 1 -- 172.21.15.45:0/4253779476 wait complete. 2023-08-27T11:27:30.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.814+0000 7f5ff7f43700 1 Processor -- start 2023-08-27T11:27:30.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.814+0000 7f5ff7f43700 1 -- start start 2023-08-27T11:27:30.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff7f43700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 0x7f5ff01f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff7f43700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ff00b2ff0 0x7f5ff01585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff7f43700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ff015d630 0x7f5ff00c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff7f43700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff01f64c0 con 0x7f5ff00720b0 2023-08-27T11:27:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff7f43700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff01f6e20 con 0x7f5ff00b2ff0 2023-08-27T11:27:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff7f43700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00c5a60 con 0x7f5ff015d630 2023-08-27T11:27:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff5cdf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 0x7f5ff01f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff5cdf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 0x7f5ff01f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57812/0 (socket says 172.21.15.45:57812) 2023-08-27T11:27:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff5cdf700 1 -- 172.21.15.45:0/3193691134 learned_addr learned my addr 172.21.15.45:0/3193691134 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff54de700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ff00b2ff0 0x7f5ff01585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff5cdf700 1 -- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ff015d630 msgr2=0x7f5ff00c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff64e0700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ff015d630 0x7f5ff00c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff5cdf700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ff015d630 0x7f5ff00c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff5cdf700 1 -- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ff00b2ff0 msgr2=0x7f5ff01585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff5cdf700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ff00b2ff0 0x7f5ff01585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff5cdf700 1 -- 172.21.15.45:0/3193691134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec00b740 con 0x7f5ff00720b0 2023-08-27T11:27:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff54de700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ff00b2ff0 0x7f5ff01585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff64e0700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ff015d630 0x7f5ff00c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.815+0000 7f5ff5cdf700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 0x7f5ff01f68e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c450 tx=0x7f5fe400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.816+0000 7f5fe27fc700 1 -- 172.21.15.45:0/3193691134 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40052d0 con 0x7f5ff00720b0 2023-08-27T11:27:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.816+0000 7f5fe27fc700 1 -- 172.21.15.45:0/3193691134 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4005430 con 0x7f5ff00720b0 2023-08-27T11:27:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.816+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff00c5d40 con 0x7f5ff00720b0 2023-08-27T11:27:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.816+0000 7f5fe27fc700 1 -- 172.21.15.45:0/3193691134 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40038c0 con 0x7f5ff00720b0 2023-08-27T11:27:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.816+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff0158b20 con 0x7f5ff00720b0 2023-08-27T11:27:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.816+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff00c5f80 con 0x7f5ff00720b0 2023-08-27T11:27:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.817+0000 7f5fe27fc700 1 -- 172.21.15.45:0/3193691134 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5fe4003a20 con 0x7f5ff00720b0 2023-08-27T11:27:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.818+0000 7f5fe27fc700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f5fd406a200 0x7f5fd406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.818+0000 7f5fe27fc700 1 -- 172.21.15.45:0/3193691134 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f5fe4087dd0 con 0x7f5ff00720b0 2023-08-27T11:27:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.818+0000 7f5ff54de700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f5fd406a200 0x7f5fd406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.819+0000 7f5ff54de700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f5fd406a200 0x7f5fd406c6c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5ff01f5ea0 tx=0x7f5fec013000 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:30.822+0000 7f5fe27fc700 1 -- 172.21.15.45:0/3193691134 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5fe40533b0 con 0x7f5ff00720b0 2023-08-27T11:27:31.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.017+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 --> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ff0073ba0 con 0x7f5fd406a200 2023-08-27T11:27:31.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.025+0000 7f5fe27fc700 1 -- 172.21.15.45:0/3193691134 <== mgr.6247 v2:172.21.15.135:6832/3430028400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306448 (secure 0 0 0) 0x7f5fec004040 con 0x7f5fd406a200 2023-08-27T11:27:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:31.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f5fd406a200 msgr2=0x7f5fd406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f5fd406a200 0x7f5fd406c6c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5ff01f5ea0 tx=0x7f5fec013000 comp rx=0 tx=0).stop 2023-08-27T11:27:31.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 msgr2=0x7f5ff01f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 0x7f5ff01f68e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c450 tx=0x7f5fe400c810 comp rx=0 tx=0).stop 2023-08-27T11:27:31.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 shutdown_connections 2023-08-27T11:27:31.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f5fd406a200 0x7f5fd406c6c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ff00720b0 0x7f5ff01f68e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ff015d630 0x7f5ff00c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 --2- 172.21.15.45:0/3193691134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ff00b2ff0 0x7f5ff01585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 >> 172.21.15.45:0/3193691134 conn(0x7f5ff00b2460 msgr2=0x7f5ff006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 shutdown_connections 2023-08-27T11:27:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.029+0000 7f5ff7f43700 1 -- 172.21.15.45:0/3193691134 wait complete. 2023-08-27T11:27:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:27:31.042 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:27:29.350988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266824,"kb_used_data":9056,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754707896,"statfs":{"total":773094113280,"available":772820885504,"internally_reserved":0,"allocated":9273344,"data_stored":5375776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.304515"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065740+0000","last_change":"2023-08-27T11:27:25.065740+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065740+0000","last_undegraded":"2023-08-27T11:27:25.065740+0000","last_fullsized":"2023-08-27T11:27:25.065740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062228+0000","last_change":"2023-08-27T11:27:25.062228+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062228+0000","last_undegraded":"2023-08-27T11:27:25.062228+0000","last_fullsized":"2023-08-27T11:27:25.062228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060843+0000","last_change":"2023-08-27T11:27:25.060843+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060843+0000","last_undegraded":"2023-08-27T11:27:25.060843+0000","last_fullsized":"2023-08-27T11:27:25.060843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060288+0000","last_change":"2023-08-27T11:27:25.060288+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060288+0000","last_undegraded":"2023-08-27T11:27:25.060288+0000","last_fullsized":"2023-08-27T11:27:25.060288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059387+0000","last_change":"2023-08-27T11:27:25.059387+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059387+0000","last_undegraded":"2023-08-27T11:27:25.059387+0000","last_fullsized":"2023-08-27T11:27:25.059387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062065+0000","last_change":"2023-08-27T11:27:25.062065+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062065+0000","last_undegraded":"2023-08-27T11:27:25.062065+0000","last_fullsized":"2023-08-27T11:27:25.062065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063011+0000","last_change":"2023-08-27T11:27:25.063011+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063011+0000","last_undegraded":"2023-08-27T11:27:25.063011+0000","last_fullsized":"2023-08-27T11:27:25.063011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062097+0000","last_change":"2023-08-27T11:27:25.062097+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062097+0000","last_undegraded":"2023-08-27T11:27:25.062097+0000","last_fullsized":"2023-08-27T11:27:25.062097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058937+0000","last_change":"2023-08-27T11:27:25.058937+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058937+0000","last_undegraded":"2023-08-27T11:27:25.058937+0000","last_fullsized":"2023-08-27T11:27:25.058937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060848+0000","last_change":"2023-08-27T11:27:25.060848+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060848+0000","last_undegraded":"2023-08-27T11:27:25.060848+0000","last_fullsized":"2023-08-27T11:27:25.060848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065903+0000","last_change":"2023-08-27T11:27:25.065903+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065903+0000","last_undegraded":"2023-08-27T11:27:25.065903+0000","last_fullsized":"2023-08-27T11:27:25.065903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059898+0000","last_change":"2023-08-27T11:27:25.059898+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059898+0000","last_undegraded":"2023-08-27T11:27:25.059898+0000","last_fullsized":"2023-08-27T11:27:25.059898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063681+0000","last_change":"2023-08-27T11:27:25.063681+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063681+0000","last_undegraded":"2023-08-27T11:27:25.063681+0000","last_fullsized":"2023-08-27T11:27:25.063681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058054+0000","last_change":"2023-08-27T11:27:25.058054+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058054+0000","last_undegraded":"2023-08-27T11:27:25.058054+0000","last_fullsized":"2023-08-27T11:27:25.058054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066424+0000","last_change":"2023-08-27T11:27:25.066424+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066424+0000","last_undegraded":"2023-08-27T11:27:25.066424+0000","last_fullsized":"2023-08-27T11:27:25.066424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059416+0000","last_change":"2023-08-27T11:27:25.059416+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059416+0000","last_undegraded":"2023-08-27T11:27:25.059416+0000","last_fullsized":"2023-08-27T11:27:25.059416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058352+0000","last_change":"2023-08-27T11:27:25.058352+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058352+0000","last_undegraded":"2023-08-27T11:27:25.058352+0000","last_fullsized":"2023-08-27T11:27:25.058352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059190+0000","last_change":"2023-08-27T11:27:25.059190+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059190+0000","last_undegraded":"2023-08-27T11:27:25.059190+0000","last_fullsized":"2023-08-27T11:27:25.059190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066698+0000","last_change":"2023-08-27T11:27:25.066698+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066698+0000","last_undegraded":"2023-08-27T11:27:25.066698+0000","last_fullsized":"2023-08-27T11:27:25.066698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063155+0000","last_change":"2023-08-27T11:27:25.063155+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063155+0000","last_undegraded":"2023-08-27T11:27:25.063155+0000","last_fullsized":"2023-08-27T11:27:25.063155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061527+0000","last_change":"2023-08-27T11:27:25.061527+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061527+0000","last_undegraded":"2023-08-27T11:27:25.061527+0000","last_fullsized":"2023-08-27T11:27:25.061527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059917+0000","last_change":"2023-08-27T11:27:25.059917+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059917+0000","last_undegraded":"2023-08-27T11:27:25.059917+0000","last_fullsized":"2023-08-27T11:27:25.059917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064806+0000","last_change":"2023-08-27T11:27:25.064806+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064806+0000","last_undegraded":"2023-08-27T11:27:25.064806+0000","last_fullsized":"2023-08-27T11:27:25.064806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064119+0000","last_change":"2023-08-27T11:27:25.064119+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064119+0000","last_undegraded":"2023-08-27T11:27:25.064119+0000","last_fullsized":"2023-08-27T11:27:25.064119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060052+0000","last_change":"2023-08-27T11:27:25.060052+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060052+0000","last_undegraded":"2023-08-27T11:27:25.060052+0000","last_fullsized":"2023-08-27T11:27:25.060052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065794+0000","last_change":"2023-08-27T11:27:25.065794+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065794+0000","last_undegraded":"2023-08-27T11:27:25.065794+0000","last_fullsized":"2023-08-27T11:27:25.065794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060404+0000","last_change":"2023-08-27T11:27:25.060404+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060404+0000","last_undegraded":"2023-08-27T11:27:25.060404+0000","last_fullsized":"2023-08-27T11:27:25.060404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060723+0000","last_change":"2023-08-27T11:27:25.060723+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060723+0000","last_undegraded":"2023-08-27T11:27:25.060723+0000","last_fullsized":"2023-08-27T11:27:25.060723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060613+0000","last_change":"2023-08-27T11:27:25.060613+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060613+0000","last_undegraded":"2023-08-27T11:27:25.060613+0000","last_fullsized":"2023-08-27T11:27:25.060613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065228+0000","last_change":"2023-08-27T11:27:25.065228+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065228+0000","last_undegraded":"2023-08-27T11:27:25.065228+0000","last_fullsized":"2023-08-27T11:27:25.065228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062163+0000","last_change":"2023-08-27T11:27:25.062163+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062163+0000","last_undegraded":"2023-08-27T11:27:25.062163+0000","last_fullsized":"2023-08-27T11:27:25.062163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061179+0000","last_change":"2023-08-27T11:27:25.061179+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061179+0000","last_undegraded":"2023-08-27T11:27:25.061179+0000","last_fullsized":"2023-08-27T11:27:25.061179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064503+0000","last_change":"2023-08-27T11:27:25.064503+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064503+0000","last_undegraded":"2023-08-27T11:27:25.064503+0000","last_fullsized":"2023-08-27T11:27:25.064503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075864+0000","last_change":"2023-08-27T11:27:25.074467+0000","last_active":"2023-08-27T11:27:25.075864+0000","last_peered":"2023-08-27T11:27:25.075864+0000","last_clean":"2023-08-27T11:27:25.075864+0000","last_became_active":"2023-08-27T11:27:25.074201+0000","last_became_peered":"2023-08-27T11:27:25.074201+0000","last_unstale":"2023-08-27T11:27:25.075864+0000","last_undegraded":"2023-08-27T11:27:25.075864+0000","last_fullsized":"2023-08-27T11:27:25.075864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:36:13.473546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057922+0000","last_change":"2023-08-27T11:27:25.057922+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057922+0000","last_undegraded":"2023-08-27T11:27:25.057922+0000","last_fullsized":"2023-08-27T11:27:25.057922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075576+0000","last_change":"2023-08-27T11:27:25.072826+0000","last_active":"2023-08-27T11:27:25.075576+0000","last_peered":"2023-08-27T11:27:25.075576+0000","last_clean":"2023-08-27T11:27:25.075576+0000","last_became_active":"2023-08-27T11:27:25.072493+0000","last_became_peered":"2023-08-27T11:27:25.072493+0000","last_unstale":"2023-08-27T11:27:25.075576+0000","last_undegraded":"2023-08-27T11:27:25.075576+0000","last_fullsized":"2023-08-27T11:27:25.075576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:08.223558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.067551+0000","last_change":"2023-08-27T11:27:25.067551+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.067551+0000","last_undegraded":"2023-08-27T11:27:25.067551+0000","last_fullsized":"2023-08-27T11:27:25.067551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075945+0000","last_change":"2023-08-27T11:27:25.075211+0000","last_active":"2023-08-27T11:27:25.075945+0000","last_peered":"2023-08-27T11:27:25.075945+0000","last_clean":"2023-08-27T11:27:25.075945+0000","last_became_active":"2023-08-27T11:27:25.074962+0000","last_became_peered":"2023-08-27T11:27:25.074962+0000","last_unstale":"2023-08-27T11:27:25.075945+0000","last_undegraded":"2023-08-27T11:27:25.075945+0000","last_fullsized":"2023-08-27T11:27:25.075945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:55:48.066087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.065441+0000","last_change":"2023-08-27T11:27:25.065441+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.065441+0000","last_undegraded":"2023-08-27T11:27:25.065441+0000","last_fullsized":"2023-08-27T11:27:25.065441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078193+0000","last_change":"2023-08-27T11:27:25.070402+0000","last_active":"2023-08-27T11:27:25.078193+0000","last_peered":"2023-08-27T11:27:25.078193+0000","last_clean":"2023-08-27T11:27:25.078193+0000","last_became_active":"2023-08-27T11:27:25.070112+0000","last_became_peered":"2023-08-27T11:27:25.070112+0000","last_unstale":"2023-08-27T11:27:25.078193+0000","last_undegraded":"2023-08-27T11:27:25.078193+0000","last_fullsized":"2023-08-27T11:27:25.078193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:29:46.302771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062102+0000","last_change":"2023-08-27T11:27:25.062102+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062102+0000","last_undegraded":"2023-08-27T11:27:25.062102+0000","last_fullsized":"2023-08-27T11:27:25.062102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073084+0000","last_change":"2023-08-27T11:27:25.071009+0000","last_active":"2023-08-27T11:27:25.073084+0000","last_peered":"2023-08-27T11:27:25.073084+0000","last_clean":"2023-08-27T11:27:25.073084+0000","last_became_active":"2023-08-27T11:27:25.070686+0000","last_became_peered":"2023-08-27T11:27:25.070686+0000","last_unstale":"2023-08-27T11:27:25.073084+0000","last_undegraded":"2023-08-27T11:27:25.073084+0000","last_fullsized":"2023-08-27T11:27:25.073084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:34:52.384052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061142+0000","last_change":"2023-08-27T11:27:25.061142+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061142+0000","last_undegraded":"2023-08-27T11:27:25.061142+0000","last_fullsized":"2023-08-27T11:27:25.061142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078714+0000","last_change":"2023-08-27T11:27:25.076385+0000","last_active":"2023-08-27T11:27:25.078714+0000","last_peered":"2023-08-27T11:27:25.078714+0000","last_clean":"2023-08-27T11:27:25.078714+0000","last_became_active":"2023-08-27T11:27:25.076062+0000","last_became_peered":"2023-08-27T11:27:25.076062+0000","last_unstale":"2023-08-27T11:27:25.078714+0000","last_undegraded":"2023-08-27T11:27:25.078714+0000","last_fullsized":"2023-08-27T11:27:25.078714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:59:06.875963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057837+0000","last_change":"2023-08-27T11:27:25.057837+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057837+0000","last_undegraded":"2023-08-27T11:27:25.057837+0000","last_fullsized":"2023-08-27T11:27:25.057837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.071644+0000","last_change":"2023-08-27T11:27:25.068965+0000","last_active":"2023-08-27T11:27:25.071644+0000","last_peered":"2023-08-27T11:27:25.071644+0000","last_clean":"2023-08-27T11:27:25.071644+0000","last_became_active":"2023-08-27T11:27:25.068679+0000","last_became_peered":"2023-08-27T11:27:25.068679+0000","last_unstale":"2023-08-27T11:27:25.071644+0000","last_undegraded":"2023-08-27T11:27:25.071644+0000","last_fullsized":"2023-08-27T11:27:25.071644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:59.810868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064147+0000","last_change":"2023-08-27T11:27:25.064147+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064147+0000","last_undegraded":"2023-08-27T11:27:25.064147+0000","last_fullsized":"2023-08-27T11:27:25.064147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.066727+0000","last_change":"2023-08-27T11:27:25.065522+0000","last_active":"2023-08-27T11:27:25.066727+0000","last_peered":"2023-08-27T11:27:25.066727+0000","last_clean":"2023-08-27T11:27:25.066727+0000","last_became_active":"2023-08-27T11:27:25.065251+0000","last_became_peered":"2023-08-27T11:27:25.065251+0000","last_unstale":"2023-08-27T11:27:25.066727+0000","last_undegraded":"2023-08-27T11:27:25.066727+0000","last_fullsized":"2023-08-27T11:27:25.066727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:18:05.350508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063105+0000","last_change":"2023-08-27T11:27:25.063105+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063105+0000","last_undegraded":"2023-08-27T11:27:25.063105+0000","last_fullsized":"2023-08-27T11:27:25.063105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075138+0000","last_change":"2023-08-27T11:27:25.068373+0000","last_active":"2023-08-27T11:27:25.075138+0000","last_peered":"2023-08-27T11:27:25.075138+0000","last_clean":"2023-08-27T11:27:25.075138+0000","last_became_active":"2023-08-27T11:27:25.068082+0000","last_became_peered":"2023-08-27T11:27:25.068082+0000","last_unstale":"2023-08-27T11:27:25.075138+0000","last_undegraded":"2023-08-27T11:27:25.075138+0000","last_fullsized":"2023-08-27T11:27:25.075138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:07.411357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064080+0000","last_change":"2023-08-27T11:27:25.064080+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064080+0000","last_undegraded":"2023-08-27T11:27:25.064080+0000","last_fullsized":"2023-08-27T11:27:25.064080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078741+0000","last_change":"2023-08-27T11:27:25.076541+0000","last_active":"2023-08-27T11:27:25.078741+0000","last_peered":"2023-08-27T11:27:25.078741+0000","last_clean":"2023-08-27T11:27:25.078741+0000","last_became_active":"2023-08-27T11:27:25.076262+0000","last_became_peered":"2023-08-27T11:27:25.076262+0000","last_unstale":"2023-08-27T11:27:25.078741+0000","last_undegraded":"2023-08-27T11:27:25.078741+0000","last_fullsized":"2023-08-27T11:27:25.078741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:59:00.006614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066919+0000","last_change":"2023-08-27T11:27:25.066919+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066919+0000","last_undegraded":"2023-08-27T11:27:25.066919+0000","last_fullsized":"2023-08-27T11:27:25.066919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.068584+0000","last_change":"2023-08-27T11:27:25.068119+0000","last_active":"2023-08-27T11:27:25.068584+0000","last_peered":"2023-08-27T11:27:25.068584+0000","last_clean":"2023-08-27T11:27:25.068584+0000","last_became_active":"2023-08-27T11:27:25.067846+0000","last_became_peered":"2023-08-27T11:27:25.067846+0000","last_unstale":"2023-08-27T11:27:25.068584+0000","last_undegraded":"2023-08-27T11:27:25.068584+0000","last_fullsized":"2023-08-27T11:27:25.068584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:43.039809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058952+0000","last_change":"2023-08-27T11:27:25.058952+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058952+0000","last_undegraded":"2023-08-27T11:27:25.058952+0000","last_fullsized":"2023-08-27T11:27:25.058952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.067165+0000","last_change":"2023-08-27T11:27:25.065700+0000","last_active":"2023-08-27T11:27:25.067165+0000","last_peered":"2023-08-27T11:27:25.067165+0000","last_clean":"2023-08-27T11:27:25.067165+0000","last_became_active":"2023-08-27T11:27:25.065409+0000","last_became_peered":"2023-08-27T11:27:25.065409+0000","last_unstale":"2023-08-27T11:27:25.067165+0000","last_undegraded":"2023-08-27T11:27:25.067165+0000","last_fullsized":"2023-08-27T11:27:25.067165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:18.780769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066322+0000","last_change":"2023-08-27T11:27:25.066322+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066322+0000","last_undegraded":"2023-08-27T11:27:25.066322+0000","last_fullsized":"2023-08-27T11:27:25.066322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073580+0000","last_change":"2023-08-27T11:27:25.070284+0000","last_active":"2023-08-27T11:27:25.073580+0000","last_peered":"2023-08-27T11:27:25.073580+0000","last_clean":"2023-08-27T11:27:25.073580+0000","last_became_active":"2023-08-27T11:27:25.069996+0000","last_became_peered":"2023-08-27T11:27:25.069996+0000","last_unstale":"2023-08-27T11:27:25.073580+0000","last_undegraded":"2023-08-27T11:27:25.073580+0000","last_fullsized":"2023-08-27T11:27:25.073580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:00.139285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.061933+0000","last_change":"2023-08-27T11:27:25.061933+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.061933+0000","last_undegraded":"2023-08-27T11:27:25.061933+0000","last_fullsized":"2023-08-27T11:27:25.061933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078214+0000","last_change":"2023-08-27T11:27:25.070398+0000","last_active":"2023-08-27T11:27:25.078214+0000","last_peered":"2023-08-27T11:27:25.078214+0000","last_clean":"2023-08-27T11:27:25.078214+0000","last_became_active":"2023-08-27T11:27:25.070052+0000","last_became_peered":"2023-08-27T11:27:25.070052+0000","last_unstale":"2023-08-27T11:27:25.078214+0000","last_undegraded":"2023-08-27T11:27:25.078214+0000","last_fullsized":"2023-08-27T11:27:25.078214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:39.907863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.062446+0000","last_change":"2023-08-27T11:27:25.062446+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.062446+0000","last_undegraded":"2023-08-27T11:27:25.062446+0000","last_fullsized":"2023-08-27T11:27:25.062446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073733+0000","last_change":"2023-08-27T11:27:25.072045+0000","last_active":"2023-08-27T11:27:25.073733+0000","last_peered":"2023-08-27T11:27:25.073733+0000","last_clean":"2023-08-27T11:27:25.073733+0000","last_became_active":"2023-08-27T11:27:25.071747+0000","last_became_peered":"2023-08-27T11:27:25.071747+0000","last_unstale":"2023-08-27T11:27:25.073733+0000","last_undegraded":"2023-08-27T11:27:25.073733+0000","last_fullsized":"2023-08-27T11:27:25.073733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:53.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057521+0000","last_change":"2023-08-27T11:27:25.057521+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057521+0000","last_undegraded":"2023-08-27T11:27:25.057521+0000","last_fullsized":"2023-08-27T11:27:25.057521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075545+0000","last_change":"2023-08-27T11:27:25.072388+0000","last_active":"2023-08-27T11:27:25.075545+0000","last_peered":"2023-08-27T11:27:25.075545+0000","last_clean":"2023-08-27T11:27:25.075545+0000","last_became_active":"2023-08-27T11:27:25.072093+0000","last_became_peered":"2023-08-27T11:27:25.072093+0000","last_unstale":"2023-08-27T11:27:25.075545+0000","last_undegraded":"2023-08-27T11:27:25.075545+0000","last_fullsized":"2023-08-27T11:27:25.075545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:59:37.328310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063415+0000","last_change":"2023-08-27T11:27:25.063415+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063415+0000","last_undegraded":"2023-08-27T11:27:25.063415+0000","last_fullsized":"2023-08-27T11:27:25.063415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078786+0000","last_change":"2023-08-27T11:27:25.076200+0000","last_active":"2023-08-27T11:27:25.078786+0000","last_peered":"2023-08-27T11:27:25.078786+0000","last_clean":"2023-08-27T11:27:25.078786+0000","last_became_active":"2023-08-27T11:27:25.074425+0000","last_became_peered":"2023-08-27T11:27:25.074425+0000","last_unstale":"2023-08-27T11:27:25.078786+0000","last_undegraded":"2023-08-27T11:27:25.078786+0000","last_fullsized":"2023-08-27T11:27:25.078786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:14:08.033370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060910+0000","last_change":"2023-08-27T11:27:25.060910+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060910+0000","last_undegraded":"2023-08-27T11:27:25.060910+0000","last_fullsized":"2023-08-27T11:27:25.060910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073798+0000","last_change":"2023-08-27T11:27:25.072101+0000","last_active":"2023-08-27T11:27:25.073798+0000","last_peered":"2023-08-27T11:27:25.073798+0000","last_clean":"2023-08-27T11:27:25.073798+0000","last_became_active":"2023-08-27T11:27:25.071818+0000","last_became_peered":"2023-08-27T11:27:25.071818+0000","last_unstale":"2023-08-27T11:27:25.073798+0000","last_undegraded":"2023-08-27T11:27:25.073798+0000","last_fullsized":"2023-08-27T11:27:25.073798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:09.111047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.063757+0000","last_change":"2023-08-27T11:27:25.063757+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.063757+0000","last_undegraded":"2023-08-27T11:27:25.063757+0000","last_fullsized":"2023-08-27T11:27:25.063757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075266+0000","last_change":"2023-08-27T11:27:25.072164+0000","last_active":"2023-08-27T11:27:25.075266+0000","last_peered":"2023-08-27T11:27:25.075266+0000","last_clean":"2023-08-27T11:27:25.075266+0000","last_became_active":"2023-08-27T11:27:25.071847+0000","last_became_peered":"2023-08-27T11:27:25.071847+0000","last_unstale":"2023-08-27T11:27:25.075266+0000","last_undegraded":"2023-08-27T11:27:25.075266+0000","last_fullsized":"2023-08-27T11:27:25.075266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:22:02.251506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064766+0000","last_change":"2023-08-27T11:27:25.064766+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064766+0000","last_undegraded":"2023-08-27T11:27:25.064766+0000","last_fullsized":"2023-08-27T11:27:25.064766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.076129+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:25.076129+0000","last_peered":"2023-08-27T11:27:25.076129+0000","last_clean":"2023-08-27T11:27:25.076129+0000","last_became_active":"2023-08-27T11:27:25.074195+0000","last_became_peered":"2023-08-27T11:27:25.074195+0000","last_unstale":"2023-08-27T11:27:25.076129+0000","last_undegraded":"2023-08-27T11:27:25.076129+0000","last_fullsized":"2023-08-27T11:27:25.076129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:43.039809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057420+0000","last_change":"2023-08-27T11:27:25.057420+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057420+0000","last_undegraded":"2023-08-27T11:27:25.057420+0000","last_fullsized":"2023-08-27T11:27:25.057420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.066696+0000","last_change":"2023-08-27T11:27:25.065564+0000","last_active":"2023-08-27T11:27:25.066696+0000","last_peered":"2023-08-27T11:27:25.066696+0000","last_clean":"2023-08-27T11:27:25.066696+0000","last_became_active":"2023-08-27T11:27:25.065291+0000","last_became_peered":"2023-08-27T11:27:25.065291+0000","last_unstale":"2023-08-27T11:27:25.066696+0000","last_undegraded":"2023-08-27T11:27:25.066696+0000","last_fullsized":"2023-08-27T11:27:25.066696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:10:02.253904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.066220+0000","last_change":"2023-08-27T11:27:25.066220+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.066220+0000","last_undegraded":"2023-08-27T11:27:25.066220+0000","last_fullsized":"2023-08-27T11:27:25.066220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.072825+0000","last_change":"2023-08-27T11:27:25.069284+0000","last_active":"2023-08-27T11:27:25.072825+0000","last_peered":"2023-08-27T11:27:25.072825+0000","last_clean":"2023-08-27T11:27:25.072825+0000","last_became_active":"2023-08-27T11:27:25.068979+0000","last_became_peered":"2023-08-27T11:27:25.068979+0000","last_unstale":"2023-08-27T11:27:25.072825+0000","last_undegraded":"2023-08-27T11:27:25.072825+0000","last_fullsized":"2023-08-27T11:27:25.072825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:54:46.692552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058711+0000","last_change":"2023-08-27T11:27:25.058711+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058711+0000","last_undegraded":"2023-08-27T11:27:25.058711+0000","last_fullsized":"2023-08-27T11:27:25.058711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075539+0000","last_change":"2023-08-27T11:27:25.072420+0000","last_active":"2023-08-27T11:27:25.075539+0000","last_peered":"2023-08-27T11:27:25.075539+0000","last_clean":"2023-08-27T11:27:25.075539+0000","last_became_active":"2023-08-27T11:27:25.072117+0000","last_became_peered":"2023-08-27T11:27:25.072117+0000","last_unstale":"2023-08-27T11:27:25.075539+0000","last_undegraded":"2023-08-27T11:27:25.075539+0000","last_fullsized":"2023-08-27T11:27:25.075539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:15:42.396071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060737+0000","last_change":"2023-08-27T11:27:25.060737+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060737+0000","last_undegraded":"2023-08-27T11:27:25.060737+0000","last_fullsized":"2023-08-27T11:27:25.060737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073981+0000","last_change":"2023-08-27T11:27:25.069386+0000","last_active":"2023-08-27T11:27:25.073981+0000","last_peered":"2023-08-27T11:27:25.073981+0000","last_clean":"2023-08-27T11:27:25.073981+0000","last_became_active":"2023-08-27T11:27:25.069120+0000","last_became_peered":"2023-08-27T11:27:25.069120+0000","last_unstale":"2023-08-27T11:27:25.073981+0000","last_undegraded":"2023-08-27T11:27:25.073981+0000","last_fullsized":"2023-08-27T11:27:25.073981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:52:58.548667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.057630+0000","last_change":"2023-08-27T11:27:25.057630+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.057630+0000","last_undegraded":"2023-08-27T11:27:25.057630+0000","last_fullsized":"2023-08-27T11:27:25.057630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.076857+0000","last_change":"2023-08-27T11:27:25.076447+0000","last_active":"2023-08-27T11:27:25.076857+0000","last_peered":"2023-08-27T11:27:25.076857+0000","last_clean":"2023-08-27T11:27:25.076857+0000","last_became_active":"2023-08-27T11:27:25.076148+0000","last_became_peered":"2023-08-27T11:27:25.076148+0000","last_unstale":"2023-08-27T11:27:25.076857+0000","last_undegraded":"2023-08-27T11:27:25.076857+0000","last_fullsized":"2023-08-27T11:27:25.076857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:27.458908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.062649+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:27:25.062649+0000","last_peered":"2023-08-27T11:27:25.062649+0000","last_clean":"2023-08-27T11:27:25.062649+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:27:25.062649+0000","last_undegraded":"2023-08-27T11:27:25.062649+0000","last_fullsized":"2023-08-27T11:27:25.062649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:41:33.495927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060178+0000","last_change":"2023-08-27T11:27:25.060178+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060178+0000","last_undegraded":"2023-08-27T11:27:25.060178+0000","last_fullsized":"2023-08-27T11:27:25.060178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078664+0000","last_change":"2023-08-27T11:27:25.075707+0000","last_active":"2023-08-27T11:27:25.078664+0000","last_peered":"2023-08-27T11:27:25.078664+0000","last_clean":"2023-08-27T11:27:25.078664+0000","last_became_active":"2023-08-27T11:27:25.074470+0000","last_became_peered":"2023-08-27T11:27:25.074470+0000","last_unstale":"2023-08-27T11:27:25.078664+0000","last_undegraded":"2023-08-27T11:27:25.078664+0000","last_fullsized":"2023-08-27T11:27:25.078664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:49:23.840169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":140,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.065518+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:27:25.065518+0000","last_peered":"2023-08-27T11:27:25.065518+0000","last_clean":"2023-08-27T11:27:25.065518+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:27:25.065518+0000","last_undegraded":"2023-08-27T11:27:25.065518+0000","last_fullsized":"2023-08-27T11:27:25.065518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:28:14.402589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.058272+0000","last_change":"2023-08-27T11:27:25.058272+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.058272+0000","last_undegraded":"2023-08-27T11:27:25.058272+0000","last_fullsized":"2023-08-27T11:27:25.058272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.071038+0000","last_change":"2023-08-27T11:27:25.067095+0000","last_active":"2023-08-27T11:27:25.071038+0000","last_peered":"2023-08-27T11:27:25.071038+0000","last_clean":"2023-08-27T11:27:25.071038+0000","last_became_active":"2023-08-27T11:27:25.066815+0000","last_became_peered":"2023-08-27T11:27:25.066815+0000","last_unstale":"2023-08-27T11:27:25.071038+0000","last_undegraded":"2023-08-27T11:27:25.071038+0000","last_fullsized":"2023-08-27T11:27:25.071038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:09.030179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.055664+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:27:25.055664+0000","last_peered":"2023-08-27T11:27:25.055664+0000","last_clean":"2023-08-27T11:27:25.055664+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:27:25.055664+0000","last_undegraded":"2023-08-27T11:27:25.055664+0000","last_fullsized":"2023-08-27T11:27:25.055664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:58.626537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059736+0000","last_change":"2023-08-27T11:27:25.059736+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059736+0000","last_undegraded":"2023-08-27T11:27:25.059736+0000","last_fullsized":"2023-08-27T11:27:25.059736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078255+0000","last_change":"2023-08-27T11:27:25.071459+0000","last_active":"2023-08-27T11:27:25.078255+0000","last_peered":"2023-08-27T11:27:25.078255+0000","last_clean":"2023-08-27T11:27:25.078255+0000","last_became_active":"2023-08-27T11:27:25.070204+0000","last_became_peered":"2023-08-27T11:27:25.070204+0000","last_unstale":"2023-08-27T11:27:25.078255+0000","last_undegraded":"2023-08-27T11:27:25.078255+0000","last_fullsized":"2023-08-27T11:27:25.078255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:57:14.619636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.063369+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:27:25.063369+0000","last_peered":"2023-08-27T11:27:25.063369+0000","last_clean":"2023-08-27T11:27:25.063369+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:27:25.063369+0000","last_undegraded":"2023-08-27T11:27:25.063369+0000","last_fullsized":"2023-08-27T11:27:25.063369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:30:05.209195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'161","reported_seq":450,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.068653+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:27:25.068653+0000","last_peered":"2023-08-27T11:27:25.068653+0000","last_clean":"2023-08-27T11:27:25.068653+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:27:25.068653+0000","last_undegraded":"2023-08-27T11:27:25.068653+0000","last_fullsized":"2023-08-27T11:27:25.068653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:13:16.300519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060703+0000","last_change":"2023-08-27T11:27:25.060703+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060703+0000","last_undegraded":"2023-08-27T11:27:25.060703+0000","last_fullsized":"2023-08-27T11:27:25.060703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.075763+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:25.075763+0000","last_peered":"2023-08-27T11:27:25.075763+0000","last_clean":"2023-08-27T11:27:25.075763+0000","last_became_active":"2023-08-27T11:27:25.074184+0000","last_became_peered":"2023-08-27T11:27:25.074184+0000","last_unstale":"2023-08-27T11:27:25.075763+0000","last_undegraded":"2023-08-27T11:27:25.075763+0000","last_fullsized":"2023-08-27T11:27:25.075763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:11:39.493382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.068797+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:27:25.068797+0000","last_peered":"2023-08-27T11:27:25.068797+0000","last_clean":"2023-08-27T11:27:25.068797+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:27:25.068797+0000","last_undegraded":"2023-08-27T11:27:25.068797+0000","last_fullsized":"2023-08-27T11:27:25.068797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:44:41.138066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.059469+0000","last_change":"2023-08-27T11:27:25.059469+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.059469+0000","last_undegraded":"2023-08-27T11:27:25.059469+0000","last_fullsized":"2023-08-27T11:27:25.059469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078781+0000","last_change":"2023-08-27T11:27:25.077520+0000","last_active":"2023-08-27T11:27:25.078781+0000","last_peered":"2023-08-27T11:27:25.078781+0000","last_clean":"2023-08-27T11:27:25.078781+0000","last_became_active":"2023-08-27T11:27:25.077249+0000","last_became_peered":"2023-08-27T11:27:25.077249+0000","last_unstale":"2023-08-27T11:27:25.078781+0000","last_undegraded":"2023-08-27T11:27:25.078781+0000","last_fullsized":"2023-08-27T11:27:25.078781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:50:59.249147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.070463+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:27:25.070463+0000","last_peered":"2023-08-27T11:27:25.070463+0000","last_clean":"2023-08-27T11:27:25.070463+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:27:25.070463+0000","last_undegraded":"2023-08-27T11:27:25.070463+0000","last_fullsized":"2023-08-27T11:27:25.070463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:00.142203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.064566+0000","last_change":"2023-08-27T11:27:25.064566+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.064566+0000","last_undegraded":"2023-08-27T11:27:25.064566+0000","last_fullsized":"2023-08-27T11:27:25.064566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.079974+0000","last_change":"2023-08-27T11:27:25.078799+0000","last_active":"2023-08-27T11:27:25.079974+0000","last_peered":"2023-08-27T11:27:25.079974+0000","last_clean":"2023-08-27T11:27:25.079974+0000","last_became_active":"2023-08-27T11:27:25.078503+0000","last_became_peered":"2023-08-27T11:27:25.078503+0000","last_unstale":"2023-08-27T11:27:25.079974+0000","last_undegraded":"2023-08-27T11:27:25.079974+0000","last_fullsized":"2023-08-27T11:27:25.079974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:04.557286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.062460+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:27:25.062460+0000","last_peered":"2023-08-27T11:27:25.062460+0000","last_clean":"2023-08-27T11:27:25.062460+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:27:25.062460+0000","last_undegraded":"2023-08-27T11:27:25.062460+0000","last_fullsized":"2023-08-27T11:27:25.062460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:46:07.961836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-08-27T11:27:25.060427+0000","last_change":"2023-08-27T11:27:25.060427+0000","last_active":"2023-08-27T11:27:25.043929+0000","last_peered":"2023-08-27T11:27:25.043929+0000","last_clean":"2023-08-27T11:27:25.043929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:27:25.060427+0000","last_undegraded":"2023-08-27T11:27:25.060427+0000","last_fullsized":"2023-08-27T11:27:25.060427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.078679+0000","last_change":"2023-08-27T11:27:25.074341+0000","last_active":"2023-08-27T11:27:25.078679+0000","last_peered":"2023-08-27T11:27:25.078679+0000","last_clean":"2023-08-27T11:27:25.078679+0000","last_became_active":"2023-08-27T11:27:25.073689+0000","last_became_peered":"2023-08-27T11:27:25.073689+0000","last_unstale":"2023-08-27T11:27:25.078679+0000","last_undegraded":"2023-08-27T11:27:25.078679+0000","last_fullsized":"2023-08-27T11:27:25.078679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:44.085030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-08-27T11:27:25.073395+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:27:25.073395+0000","last_peered":"2023-08-27T11:27:25.073395+0000","last_clean":"2023-08-27T11:27:25.073395+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:27:25.073395+0000","last_undegraded":"2023-08-27T11:27:25.073395+0000","last_fullsized":"2023-08-27T11:27:25.073395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:10.167081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575034,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1601536,"data_stored":1113000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":1.601,"15min":1.601},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40899999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.72199999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.29099999999999998},"max":{"1min":0.85499999999999998,"5min":1.778,"15min":1.778},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63500000000000001,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63600000000000001,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.67400000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.432,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68400000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56699999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48299999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88300000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.50600000000000001},"min":{"1min":0.29799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575034,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.46000000000000002,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.745,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.752,"5min":0.81100000000000005,"15min":1.3540000000000001},"last":0.47599999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.44800000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":3.3780000000000001},"last":0.501},{"interface":"front","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.36899999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.77900000000000003,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.438}]},{"osd":2,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.64800000000000002},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.71199999999999997,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72099999999999997,"5min":0.77500000000000002,"15min":1.325},"last":0.33700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57499999999999996,"15min":0.63100000000000001},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.34100000000000003},"max":{"1min":0.79400000000000004,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.48999999999999999,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78200000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.38200000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47899999999999998,"15min":0.46400000000000002},"min":{"1min":0.32100000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.71599999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.311,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44900000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.502},"min":{"1min":0.34799999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75700000000000001,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.75700000000000001,"5min":1.157,"15min":1.157},"last":0.40600000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.38200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.77500000000000002,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38500000000000001,"5min":0.371,"15min":0.34899999999999998},"max":{"1min":0.74099999999999999,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.42199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1019904,"data_stored":529624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.38900000000000001},"min":{"1min":0.26500000000000001,"5min":0.22800000000000001,"15min":0.20399999999999999},"max":{"1min":0.53500000000000003,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.307,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.65600000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.29699999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.58999999999999997,"5min":0.61099999999999999,"15min":0.73699999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.49099999999999999,"15min":0.48099999999999998},"min":{"1min":0.36099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.34100000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47499999999999998,"15min":0.48399999999999999},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.33100000000000002,"5min":0.28599999999999998,"15min":0.23400000000000001},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.34799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.52700000000000002,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.495},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66600000000000004,"15min":0.67000000000000004},"min":{"1min":0.55000000000000004,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.88600000000000001,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.40999999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.46500000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.92300000000000004,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.45000000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.56899999999999995,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.60899999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67800000000000005,"15min":0.68100000000000005},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.89400000000000002,"5min":0.96199999999999997,"15min":1.139},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.57199999999999995,"5min":0.46500000000000002,"15min":0.441},"max":{"1min":0.89800000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.629}]}]},{"osd":0,"up_from":13,"seq":55834575034,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1613824,"data_stored":1119992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.495},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.44700000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.59799999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.85899999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.92800000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.75},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.84199999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.76700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575035,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.31900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.441}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.59699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.41199999999999998,"15min":0.46000000000000002},"min":{"1min":0.25,"5min":0.23899999999999999,"15min":0.22},"max":{"1min":0.98699999999999999,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.27000000000000002,"5min":0.26900000000000002,"15min":0.224},"max":{"1min":0.61299999999999999,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.38600000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42199999999999999,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.26100000000000001,"15min":0.255},"max":{"1min":0.55800000000000005,"5min":0.64300000000000002,"15min":6.0830000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47399999999999998,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.245},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.46200000000000002},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":1.25},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":11.939},"last":0.497}]},{"osd":4,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65200000000000002,"15min":0.70599999999999996},"min":{"1min":0.55500000000000005,"5min":0.39700000000000002,"15min":0.376},"max":{"1min":0.99399999999999999,"5min":1.0329999999999999,"15min":11.007},"last":0.752},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":1.113,"15min":1.2929999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70399999999999996,"15min":0.751},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.436},"max":{"1min":0.95099999999999996,"5min":3.0219999999999998,"15min":11.981},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68700000000000006,"15min":0.72999999999999998},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":1.0620000000000001,"5min":1.218,"15min":11.09},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.80400000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.97199999999999998,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.72899999999999998,"15min":0.78800000000000003},"min":{"1min":0.54600000000000004,"5min":0.503,"15min":0.47799999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":12.016},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:26:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.92100000000000004,"5min":1.006,"15min":11.054},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.51900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0389999999999999,"5min":1.0640000000000001,"15min":1.361},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575034,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.58599999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48699999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.502},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.38800000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40899999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.57499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335516,"statfs":{"total":96636764160,"available":96599568384,"internally_reserved":0,"allocated":1007616,"data_stored":522632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.62},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57399999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.51400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:27:31.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-08-27T11:27:31.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.065740+0000', 'last_change': '2023-08-27T11:27:25.065740+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.065740+0000', 'last_undegraded': '2023-08-27T11:27:25.065740+0000', 'last_fullsized': '2023-08-27T11:27:25.065740+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-08-27T11:27:31.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.062228+0000', 'last_change': '2023-08-27T11:27:25.062228+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.062228+0000', 'last_undegraded': '2023-08-27T11:27:25.062228+0000', 'last_fullsized': '2023-08-27T11:27:25.062228+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:27:31.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-08-27T11:27:31.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060843+0000', 'last_change': '2023-08-27T11:27:25.060843+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060843+0000', 'last_undegraded': '2023-08-27T11:27:25.060843+0000', 'last_fullsized': '2023-08-27T11:27:25.060843+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:27:31.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-08-27T11:27:31.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060288+0000', 'last_change': '2023-08-27T11:27:25.060288+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060288+0000', 'last_undegraded': '2023-08-27T11:27:25.060288+0000', 'last_fullsized': '2023-08-27T11:27:25.060288+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:27:31.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-08-27T11:27:31.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.059387+0000', 'last_change': '2023-08-27T11:27:25.059387+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.059387+0000', 'last_undegraded': '2023-08-27T11:27:25.059387+0000', 'last_fullsized': '2023-08-27T11:27:25.059387+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-08-27T11:27:31.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.062065+0000', 'last_change': '2023-08-27T11:27:25.062065+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.062065+0000', 'last_undegraded': '2023-08-27T11:27:25.062065+0000', 'last_fullsized': '2023-08-27T11:27:25.062065+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:27:31.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-08-27T11:27:31.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.063011+0000', 'last_change': '2023-08-27T11:27:25.063011+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.063011+0000', 'last_undegraded': '2023-08-27T11:27:25.063011+0000', 'last_fullsized': '2023-08-27T11:27:25.063011+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:27:31.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-08-27T11:27:31.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.062097+0000', 'last_change': '2023-08-27T11:27:25.062097+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.062097+0000', 'last_undegraded': '2023-08-27T11:27:25.062097+0000', 'last_fullsized': '2023-08-27T11:27:25.062097+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:27:31.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-08-27T11:27:31.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.058937+0000', 'last_change': '2023-08-27T11:27:25.058937+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.058937+0000', 'last_undegraded': '2023-08-27T11:27:25.058937+0000', 'last_fullsized': '2023-08-27T11:27:25.058937+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-08-27T11:27:31.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060848+0000', 'last_change': '2023-08-27T11:27:25.060848+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060848+0000', 'last_undegraded': '2023-08-27T11:27:25.060848+0000', 'last_fullsized': '2023-08-27T11:27:25.060848+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:27:31.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-08-27T11:27:31.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.065903+0000', 'last_change': '2023-08-27T11:27:25.065903+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.065903+0000', 'last_undegraded': '2023-08-27T11:27:25.065903+0000', 'last_fullsized': '2023-08-27T11:27:25.065903+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-08-27T11:27:31.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.059898+0000', 'last_change': '2023-08-27T11:27:25.059898+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.059898+0000', 'last_undegraded': '2023-08-27T11:27:25.059898+0000', 'last_fullsized': '2023-08-27T11:27:25.059898+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:27:31.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-08-27T11:27:31.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.063681+0000', 'last_change': '2023-08-27T11:27:25.063681+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.063681+0000', 'last_undegraded': '2023-08-27T11:27:25.063681+0000', 'last_fullsized': '2023-08-27T11:27:25.063681+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:27:31.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-08-27T11:27:31.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.058054+0000', 'last_change': '2023-08-27T11:27:25.058054+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.058054+0000', 'last_undegraded': '2023-08-27T11:27:25.058054+0000', 'last_fullsized': '2023-08-27T11:27:25.058054+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-08-27T11:27:31.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.066424+0000', 'last_change': '2023-08-27T11:27:25.066424+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.066424+0000', 'last_undegraded': '2023-08-27T11:27:25.066424+0000', 'last_fullsized': '2023-08-27T11:27:25.066424+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:27:31.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-08-27T11:27:31.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.059416+0000', 'last_change': '2023-08-27T11:27:25.059416+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.059416+0000', 'last_undegraded': '2023-08-27T11:27:25.059416+0000', 'last_fullsized': '2023-08-27T11:27:25.059416+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:27:31.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-08-27T11:27:31.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.058352+0000', 'last_change': '2023-08-27T11:27:25.058352+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.058352+0000', 'last_undegraded': '2023-08-27T11:27:25.058352+0000', 'last_fullsized': '2023-08-27T11:27:25.058352+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-08-27T11:27:31.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.059190+0000', 'last_change': '2023-08-27T11:27:25.059190+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.059190+0000', 'last_undegraded': '2023-08-27T11:27:25.059190+0000', 'last_fullsized': '2023-08-27T11:27:25.059190+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:27:31.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-08-27T11:27:31.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.066698+0000', 'last_change': '2023-08-27T11:27:25.066698+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.066698+0000', 'last_undegraded': '2023-08-27T11:27:25.066698+0000', 'last_fullsized': '2023-08-27T11:27:25.066698+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:27:31.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-08-27T11:27:31.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.063155+0000', 'last_change': '2023-08-27T11:27:25.063155+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.063155+0000', 'last_undegraded': '2023-08-27T11:27:25.063155+0000', 'last_fullsized': '2023-08-27T11:27:25.063155+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:27:31.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-08-27T11:27:31.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.061527+0000', 'last_change': '2023-08-27T11:27:25.061527+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.061527+0000', 'last_undegraded': '2023-08-27T11:27:25.061527+0000', 'last_fullsized': '2023-08-27T11:27:25.061527+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:27:31.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-08-27T11:27:31.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.059917+0000', 'last_change': '2023-08-27T11:27:25.059917+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.059917+0000', 'last_undegraded': '2023-08-27T11:27:25.059917+0000', 'last_fullsized': '2023-08-27T11:27:25.059917+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:27:31.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-08-27T11:27:31.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.064806+0000', 'last_change': '2023-08-27T11:27:25.064806+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.064806+0000', 'last_undegraded': '2023-08-27T11:27:25.064806+0000', 'last_fullsized': '2023-08-27T11:27:25.064806+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-08-27T11:27:31.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.064119+0000', 'last_change': '2023-08-27T11:27:25.064119+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.064119+0000', 'last_undegraded': '2023-08-27T11:27:25.064119+0000', 'last_fullsized': '2023-08-27T11:27:25.064119+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-08-27T11:27:31.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060052+0000', 'last_change': '2023-08-27T11:27:25.060052+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060052+0000', 'last_undegraded': '2023-08-27T11:27:25.060052+0000', 'last_fullsized': '2023-08-27T11:27:25.060052+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:27:31.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-08-27T11:27:31.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.065794+0000', 'last_change': '2023-08-27T11:27:25.065794+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.065794+0000', 'last_undegraded': '2023-08-27T11:27:25.065794+0000', 'last_fullsized': '2023-08-27T11:27:25.065794+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:27:31.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-08-27T11:27:31.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060404+0000', 'last_change': '2023-08-27T11:27:25.060404+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060404+0000', 'last_undegraded': '2023-08-27T11:27:25.060404+0000', 'last_fullsized': '2023-08-27T11:27:25.060404+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:27:31.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-08-27T11:27:31.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060723+0000', 'last_change': '2023-08-27T11:27:25.060723+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060723+0000', 'last_undegraded': '2023-08-27T11:27:25.060723+0000', 'last_fullsized': '2023-08-27T11:27:25.060723+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:27:31.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-08-27T11:27:31.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060613+0000', 'last_change': '2023-08-27T11:27:25.060613+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060613+0000', 'last_undegraded': '2023-08-27T11:27:25.060613+0000', 'last_fullsized': '2023-08-27T11:27:25.060613+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:27:31.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-08-27T11:27:31.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.065228+0000', 'last_change': '2023-08-27T11:27:25.065228+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.065228+0000', 'last_undegraded': '2023-08-27T11:27:25.065228+0000', 'last_fullsized': '2023-08-27T11:27:25.065228+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:27:31.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-08-27T11:27:31.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.062163+0000', 'last_change': '2023-08-27T11:27:25.062163+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.062163+0000', 'last_undegraded': '2023-08-27T11:27:25.062163+0000', 'last_fullsized': '2023-08-27T11:27:25.062163+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:27:31.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-08-27T11:27:31.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.061179+0000', 'last_change': '2023-08-27T11:27:25.061179+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.061179+0000', 'last_undegraded': '2023-08-27T11:27:25.061179+0000', 'last_fullsized': '2023-08-27T11:27:25.061179+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:27:31.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-08-27T11:27:31.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.064503+0000', 'last_change': '2023-08-27T11:27:25.064503+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.064503+0000', 'last_undegraded': '2023-08-27T11:27:25.064503+0000', 'last_fullsized': '2023-08-27T11:27:25.064503+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-08-27T11:27:31.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.057922+0000', 'last_change': '2023-08-27T11:27:25.057922+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.057922+0000', 'last_undegraded': '2023-08-27T11:27:25.057922+0000', 'last_fullsized': '2023-08-27T11:27:25.057922+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-08-27T11:27:31.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.067551+0000', 'last_change': '2023-08-27T11:27:25.067551+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.067551+0000', 'last_undegraded': '2023-08-27T11:27:25.067551+0000', 'last_fullsized': '2023-08-27T11:27:25.067551+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:27:31.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-08-27T11:27:31.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.065441+0000', 'last_change': '2023-08-27T11:27:25.065441+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.065441+0000', 'last_undegraded': '2023-08-27T11:27:25.065441+0000', 'last_fullsized': '2023-08-27T11:27:25.065441+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-08-27T11:27:31.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.062102+0000', 'last_change': '2023-08-27T11:27:25.062102+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.062102+0000', 'last_undegraded': '2023-08-27T11:27:25.062102+0000', 'last_fullsized': '2023-08-27T11:27:25.062102+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:27:31.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-08-27T11:27:31.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.061142+0000', 'last_change': '2023-08-27T11:27:25.061142+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.061142+0000', 'last_undegraded': '2023-08-27T11:27:25.061142+0000', 'last_fullsized': '2023-08-27T11:27:25.061142+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:27:31.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-08-27T11:27:31.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.057837+0000', 'last_change': '2023-08-27T11:27:25.057837+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.057837+0000', 'last_undegraded': '2023-08-27T11:27:25.057837+0000', 'last_fullsized': '2023-08-27T11:27:25.057837+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-08-27T11:27:31.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.064147+0000', 'last_change': '2023-08-27T11:27:25.064147+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.064147+0000', 'last_undegraded': '2023-08-27T11:27:25.064147+0000', 'last_fullsized': '2023-08-27T11:27:25.064147+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:27:31.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-08-27T11:27:31.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.063105+0000', 'last_change': '2023-08-27T11:27:25.063105+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.063105+0000', 'last_undegraded': '2023-08-27T11:27:25.063105+0000', 'last_fullsized': '2023-08-27T11:27:25.063105+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:27:31.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-08-27T11:27:31.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.064080+0000', 'last_change': '2023-08-27T11:27:25.064080+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.064080+0000', 'last_undegraded': '2023-08-27T11:27:25.064080+0000', 'last_fullsized': '2023-08-27T11:27:25.064080+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-08-27T11:27:31.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.066919+0000', 'last_change': '2023-08-27T11:27:25.066919+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.066919+0000', 'last_undegraded': '2023-08-27T11:27:25.066919+0000', 'last_fullsized': '2023-08-27T11:27:25.066919+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:27:31.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-08-27T11:27:31.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.058952+0000', 'last_change': '2023-08-27T11:27:25.058952+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.058952+0000', 'last_undegraded': '2023-08-27T11:27:25.058952+0000', 'last_fullsized': '2023-08-27T11:27:25.058952+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-08-27T11:27:31.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.066322+0000', 'last_change': '2023-08-27T11:27:25.066322+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.066322+0000', 'last_undegraded': '2023-08-27T11:27:25.066322+0000', 'last_fullsized': '2023-08-27T11:27:25.066322+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-08-27T11:27:31.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.061933+0000', 'last_change': '2023-08-27T11:27:25.061933+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.061933+0000', 'last_undegraded': '2023-08-27T11:27:25.061933+0000', 'last_fullsized': '2023-08-27T11:27:25.061933+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:27:31.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-08-27T11:27:31.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.062446+0000', 'last_change': '2023-08-27T11:27:25.062446+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.062446+0000', 'last_undegraded': '2023-08-27T11:27:25.062446+0000', 'last_fullsized': '2023-08-27T11:27:25.062446+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:27:31.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-08-27T11:27:31.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.057521+0000', 'last_change': '2023-08-27T11:27:25.057521+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.057521+0000', 'last_undegraded': '2023-08-27T11:27:25.057521+0000', 'last_fullsized': '2023-08-27T11:27:25.057521+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-08-27T11:27:31.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.063415+0000', 'last_change': '2023-08-27T11:27:25.063415+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.063415+0000', 'last_undegraded': '2023-08-27T11:27:25.063415+0000', 'last_fullsized': '2023-08-27T11:27:25.063415+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:27:31.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-08-27T11:27:31.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060910+0000', 'last_change': '2023-08-27T11:27:25.060910+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060910+0000', 'last_undegraded': '2023-08-27T11:27:25.060910+0000', 'last_fullsized': '2023-08-27T11:27:25.060910+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:27:31.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-08-27T11:27:31.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.063757+0000', 'last_change': '2023-08-27T11:27:25.063757+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.063757+0000', 'last_undegraded': '2023-08-27T11:27:25.063757+0000', 'last_fullsized': '2023-08-27T11:27:25.063757+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-08-27T11:27:31.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.064766+0000', 'last_change': '2023-08-27T11:27:25.064766+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.064766+0000', 'last_undegraded': '2023-08-27T11:27:25.064766+0000', 'last_fullsized': '2023-08-27T11:27:25.064766+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-08-27T11:27:31.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.057420+0000', 'last_change': '2023-08-27T11:27:25.057420+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.057420+0000', 'last_undegraded': '2023-08-27T11:27:25.057420+0000', 'last_fullsized': '2023-08-27T11:27:25.057420+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-08-27T11:27:31.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.066220+0000', 'last_change': '2023-08-27T11:27:25.066220+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.066220+0000', 'last_undegraded': '2023-08-27T11:27:25.066220+0000', 'last_fullsized': '2023-08-27T11:27:25.066220+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:27:31.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-08-27T11:27:31.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.058711+0000', 'last_change': '2023-08-27T11:27:25.058711+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.058711+0000', 'last_undegraded': '2023-08-27T11:27:25.058711+0000', 'last_fullsized': '2023-08-27T11:27:25.058711+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:27:31.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-08-27T11:27:31.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060737+0000', 'last_change': '2023-08-27T11:27:25.060737+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060737+0000', 'last_undegraded': '2023-08-27T11:27:25.060737+0000', 'last_fullsized': '2023-08-27T11:27:25.060737+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:27:31.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-08-27T11:27:31.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.057630+0000', 'last_change': '2023-08-27T11:27:25.057630+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.057630+0000', 'last_undegraded': '2023-08-27T11:27:25.057630+0000', 'last_fullsized': '2023-08-27T11:27:25.057630+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:27:31.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-08-27T11:27:31.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060178+0000', 'last_change': '2023-08-27T11:27:25.060178+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060178+0000', 'last_undegraded': '2023-08-27T11:27:25.060178+0000', 'last_fullsized': '2023-08-27T11:27:25.060178+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:27:31.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-08-27T11:27:31.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.058272+0000', 'last_change': '2023-08-27T11:27:25.058272+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.058272+0000', 'last_undegraded': '2023-08-27T11:27:25.058272+0000', 'last_fullsized': '2023-08-27T11:27:25.058272+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:27:31.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-08-27T11:27:31.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.059736+0000', 'last_change': '2023-08-27T11:27:25.059736+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.059736+0000', 'last_undegraded': '2023-08-27T11:27:25.059736+0000', 'last_fullsized': '2023-08-27T11:27:25.059736+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:27:31.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-08-27T11:27:31.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060703+0000', 'last_change': '2023-08-27T11:27:25.060703+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060703+0000', 'last_undegraded': '2023-08-27T11:27:25.060703+0000', 'last_fullsized': '2023-08-27T11:27:25.060703+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:27:31.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-08-27T11:27:31.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.059469+0000', 'last_change': '2023-08-27T11:27:25.059469+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.059469+0000', 'last_undegraded': '2023-08-27T11:27:25.059469+0000', 'last_fullsized': '2023-08-27T11:27:25.059469+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:27:31.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-08-27T11:27:31.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.064566+0000', 'last_change': '2023-08-27T11:27:25.064566+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.064566+0000', 'last_undegraded': '2023-08-27T11:27:25.064566+0000', 'last_fullsized': '2023-08-27T11:27:25.064566+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:27:31.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-08-27T11:27:31.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:27:25.060427+0000', 'last_change': '2023-08-27T11:27:25.060427+0000', 'last_active': '2023-08-27T11:27:25.043929+0000', 'last_peered': '2023-08-27T11:27:25.043929+0000', 'last_clean': '2023-08-27T11:27:25.043929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:27:25.060427+0000', 'last_undegraded': '2023-08-27T11:27:25.060427+0000', 'last_fullsized': '2023-08-27T11:27:25.060427+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'last_clean_scrub_stamp': '2023-08-27T11:27:25.043929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:27:31.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-27T11:27:31.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.223+0000 7f19b15a2700 1 Processor -- start 2023-08-27T11:27:31.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.223+0000 7f19b15a2700 1 -- start start 2023-08-27T11:27:31.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.224+0000 7f19b15a2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19ac06cec0 0x7f19ac06e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.224+0000 7f19b15a2700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f19ac06e7b0 0x7f19ac0ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.224+0000 7f19b15a2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ac06ebe0 con 0x7f19ac15b070 2023-08-27T11:27:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.224+0000 7f19b15a2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ac15d8e0 con 0x7f19ac06cec0 2023-08-27T11:27:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.224+0000 7f19b15a2700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ac15dad0 con 0x7f19ac06e7b0 2023-08-27T11:27:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.224+0000 7f19aa7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19ac06cec0 0x7f19ac06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.224+0000 7f19aaffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f19ac15b070 0x7f19ac0cda20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44596/0 (socket says 172.21.15.45:44596) 2023-08-27T11:27:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.224+0000 7f19aa7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19ac06cec0 0x7f19ac06e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57822/0 (socket says 172.21.15.45:57822) 2023-08-27T11:27:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.224+0000 7f19aaffd700 1 -- 172.21.15.45:0/3122518192 learned_addr learned my addr 172.21.15.45:0/3122518192 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19aa7fc700 1 -- 172.21.15.45:0/3122518192 >> v2:172.21.15.135:3300/0 conn(0x7f19ac06e7b0 msgr2=0x7f19ac0ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19ab7fe700 1 --2- 172.21.15.45:0/3122518192 >> v2:172.21.15.135:3300/0 conn(0x7f19ac06e7b0 0x7f19ac0ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19aa7fc700 1 --2- 172.21.15.45:0/3122518192 >> v2:172.21.15.135:3300/0 conn(0x7f19ac06e7b0 0x7f19ac0ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19aa7fc700 1 -- 172.21.15.45:0/3122518192 >> v1:172.21.15.45:6789/0 conn(0x7f19ac15b070 legacy=0x7f19ac0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19a9ffb700 1 -- 172.21.15.45:0/3122518192 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892743962 0 0) 0x7f19ac06ebe0 con 0x7f19ac15b070 2023-08-27T11:27:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19aa7fc700 1 -- 172.21.15.45:0/3122518192 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac15ddb0 con 0x7f19ac06cec0 2023-08-27T11:27:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19ab7fe700 1 --2- 172.21.15.45:0/3122518192 >> v2:172.21.15.135:3300/0 conn(0x7f19ac06e7b0 0x7f19ac0ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:31.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19aa7fc700 1 --2- 172.21.15.45:0/3122518192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac06cec0 0x7f19ac06e270 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0677c0 tx=0x7f1994009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f779cbd2de0722 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19a9ffb700 1 -- 172.21.15.45:0/3122518192 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994011830 con 0x7f19ac06cec0 2023-08-27T11:27:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19a9ffb700 1 -- 172.21.15.45:0/3122518192 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1994011990 con 0x7f19ac06cec0 2023-08-27T11:27:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.225+0000 7f19a9ffb700 1 -- 172.21.15.45:0/3122518192 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994011d20 con 0x7f19ac06cec0 2023-08-27T11:27:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.226+0000 7f19b15a2700 1 -- 172.21.15.45:0/3122518192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac06cec0 msgr2=0x7f19ac06e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.226+0000 7f19b15a2700 1 --2- 172.21.15.45:0/3122518192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac06cec0 0x7f19ac06e270 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0677c0 tx=0x7f1994009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.226+0000 7f19b15a2700 1 -- 172.21.15.45:0/3122518192 shutdown_connections 2023-08-27T11:27:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.226+0000 7f19b15a2700 1 --2- 172.21.15.45:0/3122518192 >> v2:172.21.15.135:3300/0 conn(0x7f19ac06e7b0 0x7f19ac0ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.226+0000 7f19b15a2700 1 --2- 172.21.15.45:0/3122518192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac06cec0 0x7f19ac06e270 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.226+0000 7f19b15a2700 1 -- 172.21.15.45:0/3122518192 >> 172.21.15.45:0/3122518192 conn(0x7f19ac0cfe50 msgr2=0x7f19ac0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:31.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.226+0000 7f19b15a2700 1 -- 172.21.15.45:0/3122518192 shutdown_connections 2023-08-27T11:27:31.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.226+0000 7f19b15a2700 1 -- 172.21.15.45:0/3122518192 wait complete. 2023-08-27T11:27:31.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.227+0000 7f19b15a2700 1 Processor -- start 2023-08-27T11:27:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.227+0000 7f19b15a2700 1 -- start start 2023-08-27T11:27:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19b15a2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19ac06cec0 0x7f19ac0c4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19b15a2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19ac06e7b0 0x7f19ac152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19b15a2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac15b070 0x7f19ac1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19b15a2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac153860 con 0x7f19ac15b070 2023-08-27T11:27:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19b15a2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0c5ef0 con 0x7f19ac06cec0 2023-08-27T11:27:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19b15a2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0c47b0 con 0x7f19ac06e7b0 2023-08-27T11:27:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19ab7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac15b070 0x7f19ac1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19ab7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac15b070 0x7f19ac1594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57830/0 (socket says 172.21.15.45:57830) 2023-08-27T11:27:31.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19ab7fe700 1 -- 172.21.15.45:0/2655954890 learned_addr learned my addr 172.21.15.45:0/2655954890 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:31.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19aaffd700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19ac06cec0 0x7f19ac0c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19aa7fc700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19ac06e7b0 0x7f19ac152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19ab7fe700 1 -- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19ac06e7b0 msgr2=0x7f19ac152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19ab7fe700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19ac06e7b0 0x7f19ac152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19ab7fe700 1 -- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19ac06cec0 msgr2=0x7f19ac0c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19ab7fe700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19ac06cec0 0x7f19ac0c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19ab7fe700 1 -- 172.21.15.45:0/2655954890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19940094a0 con 0x7f19ac15b070 2023-08-27T11:27:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19aaffd700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19ac06cec0 0x7f19ac0c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:31.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.228+0000 7f19ab7fe700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac15b070 0x7f19ac1594f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f19a400cef0 tx=0x7f19a400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.229+0000 7f19a37fe700 1 -- 172.21.15.45:0/2655954890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a400ae50 con 0x7f19ac15b070 2023-08-27T11:27:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.229+0000 7f19a37fe700 1 -- 172.21.15.45:0/2655954890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4015440 con 0x7f19ac15b070 2023-08-27T11:27:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.229+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac0c6120 con 0x7f19ac15b070 2023-08-27T11:27:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.229+0000 7f19a37fe700 1 -- 172.21.15.45:0/2655954890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40157f0 con 0x7f19ac15b070 2023-08-27T11:27:31.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.229+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac152de0 con 0x7f19ac15b070 2023-08-27T11:27:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.230+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac072810 con 0x7f19ac15b070 2023-08-27T11:27:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.230+0000 7f19a37fe700 1 -- 172.21.15.45:0/2655954890 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f19a400d430 con 0x7f19ac15b070 2023-08-27T11:27:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.230+0000 7f19a37fe700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f198c06a200 0x7f198c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.231+0000 7f19a37fe700 1 -- 172.21.15.45:0/2655954890 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f19a4088dd0 con 0x7f19ac15b070 2023-08-27T11:27:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.231+0000 7f19aaffd700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f198c06a200 0x7f198c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.231+0000 7f19aaffd700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f198c06a200 0x7f198c06c6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f199c000f60 tx=0x7f199c009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:31.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.235+0000 7f19a37fe700 1 -- 172.21.15.45:0/2655954890 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f19a40542d0 con 0x7f19ac15b070 2023-08-27T11:27:31.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.519+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f19ac0c6760 con 0x7f19ac15b070 2023-08-27T11:27:31.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.520+0000 7f19a37fe700 1 -- 172.21.15.45:0/2655954890 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1034 (secure 0 0 0) 0x7f19a401a420 con 0x7f19ac15b070 2023-08-27T11:27:31.521 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:31.521 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":948,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":86,"num_osds":8,"num_up_osds":8,"osd_up_since":1693134714,"num_in_osds":8,"osd_in_since":1693134705,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":273227776,"bytes_avail":772820885504,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":34,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":6885}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-08-27T11:26:31.287744+0000","services":{}},"progress_events":{}} 2023-08-27T11:27:31.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f198c06a200 msgr2=0x7f198c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f198c06a200 0x7f198c06c6c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f199c000f60 tx=0x7f199c009040 comp rx=0 tx=0).stop 2023-08-27T11:27:31.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac15b070 msgr2=0x7f19ac1594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac15b070 0x7f19ac1594f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f19a400cef0 tx=0x7f19a400a340 comp rx=0 tx=0).stop 2023-08-27T11:27:31.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 shutdown_connections 2023-08-27T11:27:31.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f198c06a200 0x7f198c06c6c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ac15b070 0x7f19ac1594f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19ac06cec0 0x7f19ac0c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 --2- 172.21.15.45:0/2655954890 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f19ac06e7b0 0x7f19ac152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 >> 172.21.15.45:0/2655954890 conn(0x7f19ac0cfe50 msgr2=0x7f19ac06a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:31.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 shutdown_connections 2023-08-27T11:27:31.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.522+0000 7f19b15a2700 1 -- 172.21.15.45:0/2655954890 wait complete. 2023-08-27T11:27:31.535 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-27T11:27:31.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:27:31.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.660+0000 7f57eee99700 1 Processor -- start 2023-08-27T11:27:31.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.660+0000 7f57eee99700 1 -- start start 2023-08-27T11:27:31.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.661+0000 7f57eee99700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57e815dbc0 0x7f57e815a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.661+0000 7f57eee99700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f57e815a750 0x7f57e80cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.661+0000 7f57eee99700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e80caef0 con 0x7f57e8070e40 2023-08-27T11:27:31.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.661+0000 7f57eee99700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e80cac50 con 0x7f57e815dbc0 2023-08-27T11:27:31.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.661+0000 7f57eee99700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e80ce000 con 0x7f57e815a750 2023-08-27T11:27:31.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.661+0000 7f57e7fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57e815dbc0 0x7f57e815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.661+0000 7f57ecc35700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f57e8070e40 0x7f57e8071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44610/0 (socket says 172.21.15.45:44610) 2023-08-27T11:27:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57ecc35700 1 -- 172.21.15.45:0/2768256692 learned_addr learned my addr 172.21.15.45:0/2768256692 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57ed436700 1 --2- 172.21.15.45:0/2768256692 >> v2:172.21.15.135:3300/0 conn(0x7f57e815a750 0x7f57e80cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57e7fff700 1 -- 172.21.15.45:0/2768256692 >> v2:172.21.15.135:3300/0 conn(0x7f57e815a750 msgr2=0x7f57e80cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57e7fff700 1 --2- 172.21.15.45:0/2768256692 >> v2:172.21.15.135:3300/0 conn(0x7f57e815a750 0x7f57e80cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57e7fff700 1 -- 172.21.15.45:0/2768256692 >> v1:172.21.15.45:6789/0 conn(0x7f57e8070e40 legacy=0x7f57e8071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57e7fff700 1 -- 172.21.15.45:0/2768256692 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e815b5c0 con 0x7f57e815dbc0 2023-08-27T11:27:31.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57e77fe700 1 -- 172.21.15.45:0/2768256692 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370015937 0 0) 0x7f57e80caef0 con 0x7f57e8070e40 2023-08-27T11:27:31.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57e7fff700 1 --2- 172.21.15.45:0/2768256692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e815dbc0 0x7f57e815a210 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009840 tx=0x7f57d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc61f8f356ba036f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:31.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57e77fe700 1 -- 172.21.15.45:0/2768256692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4011830 con 0x7f57e815dbc0 2023-08-27T11:27:31.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57e77fe700 1 -- 172.21.15.45:0/2768256692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4011990 con 0x7f57e815dbc0 2023-08-27T11:27:31.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.662+0000 7f57e77fe700 1 -- 172.21.15.45:0/2768256692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4011d20 con 0x7f57e815dbc0 2023-08-27T11:27:31.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.663+0000 7f57eee99700 1 -- 172.21.15.45:0/2768256692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e815dbc0 msgr2=0x7f57e815a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.663+0000 7f57eee99700 1 --2- 172.21.15.45:0/2768256692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e815dbc0 0x7f57e815a210 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009840 tx=0x7f57d4009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:31.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.663+0000 7f57eee99700 1 -- 172.21.15.45:0/2768256692 shutdown_connections 2023-08-27T11:27:31.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.663+0000 7f57eee99700 1 --2- 172.21.15.45:0/2768256692 >> v2:172.21.15.135:3300/0 conn(0x7f57e815a750 0x7f57e80cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.663+0000 7f57eee99700 1 --2- 172.21.15.45:0/2768256692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e815dbc0 0x7f57e815a210 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.663+0000 7f57eee99700 1 -- 172.21.15.45:0/2768256692 >> 172.21.15.45:0/2768256692 conn(0x7f57e80cfd60 msgr2=0x7f57e80d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:31.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.663+0000 7f57eee99700 1 -- 172.21.15.45:0/2768256692 shutdown_connections 2023-08-27T11:27:31.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.663+0000 7f57eee99700 1 -- 172.21.15.45:0/2768256692 wait complete. 2023-08-27T11:27:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.664+0000 7f57eee99700 1 Processor -- start 2023-08-27T11:27:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.664+0000 7f57eee99700 1 -- start start 2023-08-27T11:27:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57eee99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e8070e40 0x7f57e8158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57eee99700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57e815a750 0x7f57e8158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57eee99700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57e815dbc0 0x7f57e8165f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57eee99700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e814aae0 con 0x7f57e8070e40 2023-08-27T11:27:31.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57eee99700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80c9ce0 con 0x7f57e815a750 2023-08-27T11:27:31.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57eee99700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e80c90d0 con 0x7f57e815dbc0 2023-08-27T11:27:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57ecc35700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e8070e40 0x7f57e8158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57ecc35700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e8070e40 0x7f57e8158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57840/0 (socket says 172.21.15.45:57840) 2023-08-27T11:27:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57ecc35700 1 -- 172.21.15.45:0/1106425551 learned_addr learned my addr 172.21.15.45:0/1106425551 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57e7fff700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57e815a750 0x7f57e8158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57ed436700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57e815dbc0 0x7f57e8165f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57ecc35700 1 -- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57e815dbc0 msgr2=0x7f57e8165f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57ecc35700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57e815dbc0 0x7f57e8165f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57ecc35700 1 -- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57e815a750 msgr2=0x7f57e8158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57ecc35700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57e815a750 0x7f57e8158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.665+0000 7f57ecc35700 1 -- 172.21.15.45:0/1106425551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d40094a0 con 0x7f57e8070e40 2023-08-27T11:27:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.666+0000 7f57ecc35700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e8070e40 0x7f57e8158990 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00c580 tx=0x7f57dc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.666+0000 7f57e57fa700 1 -- 172.21.15.45:0/1106425551 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc004c80 con 0x7f57e8070e40 2023-08-27T11:27:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.666+0000 7f57e57fa700 1 -- 172.21.15.45:0/1106425551 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc004e00 con 0x7f57e8070e40 2023-08-27T11:27:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.666+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e80c93b0 con 0x7f57e8070e40 2023-08-27T11:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.666+0000 7f57e57fa700 1 -- 172.21.15.45:0/1106425551 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc007310 con 0x7f57e8070e40 2023-08-27T11:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.666+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e80c9970 con 0x7f57e8070e40 2023-08-27T11:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.667+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e8072180 con 0x7f57e8070e40 2023-08-27T11:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.667+0000 7f57e57fa700 1 -- 172.21.15.45:0/1106425551 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f57dc007470 con 0x7f57e8070e40 2023-08-27T11:27:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.668+0000 7f57e57fa700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f57cc06a200 0x7f57cc06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.668+0000 7f57e57fa700 1 -- 172.21.15.45:0/1106425551 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f57dc088470 con 0x7f57e8070e40 2023-08-27T11:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.668+0000 7f57e7fff700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f57cc06a200 0x7f57cc06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.668+0000 7f57e7fff700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f57cc06a200 0x7f57cc06c6c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009840 tx=0x7f57d4008030 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.672+0000 7f57e57fa700 1 -- 172.21.15.45:0/1106425551 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f57dc053970 con 0x7f57e8070e40 2023-08-27T11:27:31.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.870+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 --> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57e815b000 con 0x7f57cc06a200 2023-08-27T11:27:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.879+0000 7f57e57fa700 1 -- 172.21.15.45:0/1106425551 <== mgr.6247 v2:172.21.15.135:6832/3430028400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311724 (secure 0 0 0) 0x7f57d4011b80 con 0x7f57cc06a200 2023-08-27T11:27:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:31.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f57cc06a200 msgr2=0x7f57cc06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f57cc06a200 0x7f57cc06c6c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009840 tx=0x7f57d4008030 comp rx=0 tx=0).stop 2023-08-27T11:27:31.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e8070e40 msgr2=0x7f57e8158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:31.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e8070e40 0x7f57e8158990 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00c580 tx=0x7f57dc00c940 comp rx=0 tx=0).stop 2023-08-27T11:27:31.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 shutdown_connections 2023-08-27T11:27:31.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f57cc06a200 0x7f57cc06c6c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57e8070e40 0x7f57e8158990 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f57e815dbc0 0x7f57e8165f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 --2- 172.21.15.45:0/1106425551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57e815a750 0x7f57e8158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:31.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 >> 172.21.15.45:0/1106425551 conn(0x7f57e80cfd60 msgr2=0x7f57e80cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:31.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 shutdown_connections 2023-08-27T11:27:31.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:31.882+0000 7f57eee99700 1 -- 172.21.15.45:0/1106425551 wait complete. 2023-08-27T11:27:31.888 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:27:31.895 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-08-27T11:27:31.351424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":284,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267188,"kb_used_data":9384,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754707532,"statfs":{"total":773094113280,"available":772820512768,"internally_reserved":0,"allocated":9609216,"data_stored":5538620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.293919"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352046+0000","last_change":"2023-08-27T11:27:26.087156+0000","last_active":"2023-08-27T11:27:27.352046+0000","last_peered":"2023-08-27T11:27:27.352046+0000","last_clean":"2023-08-27T11:27:27.352046+0000","last_became_active":"2023-08-27T11:27:26.086775+0000","last_became_peered":"2023-08-27T11:27:26.086775+0000","last_unstale":"2023-08-27T11:27:27.352046+0000","last_undegraded":"2023-08-27T11:27:27.352046+0000","last_fullsized":"2023-08-27T11:27:27.352046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:58:28.845767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063862+0000","last_change":"2023-08-27T11:27:26.090010+0000","last_active":"2023-08-27T11:27:27.063862+0000","last_peered":"2023-08-27T11:27:27.063862+0000","last_clean":"2023-08-27T11:27:27.063862+0000","last_became_active":"2023-08-27T11:27:26.089682+0000","last_became_peered":"2023-08-27T11:27:26.089682+0000","last_unstale":"2023-08-27T11:27:27.063862+0000","last_undegraded":"2023-08-27T11:27:27.063862+0000","last_fullsized":"2023-08-27T11:27:27.063862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:31.151499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351678+0000","last_change":"2023-08-27T11:27:26.074317+0000","last_active":"2023-08-27T11:27:27.351678+0000","last_peered":"2023-08-27T11:27:27.351678+0000","last_clean":"2023-08-27T11:27:27.351678+0000","last_became_active":"2023-08-27T11:27:26.073998+0000","last_became_peered":"2023-08-27T11:27:26.073998+0000","last_unstale":"2023-08-27T11:27:27.351678+0000","last_undegraded":"2023-08-27T11:27:27.351678+0000","last_fullsized":"2023-08-27T11:27:27.351678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:39:00.353719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351718+0000","last_change":"2023-08-27T11:27:26.082333+0000","last_active":"2023-08-27T11:27:27.351718+0000","last_peered":"2023-08-27T11:27:27.351718+0000","last_clean":"2023-08-27T11:27:27.351718+0000","last_became_active":"2023-08-27T11:27:26.080958+0000","last_became_peered":"2023-08-27T11:27:26.080958+0000","last_unstale":"2023-08-27T11:27:27.351718+0000","last_undegraded":"2023-08-27T11:27:27.351718+0000","last_fullsized":"2023-08-27T11:27:27.351718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:44:07.170889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059156+0000","last_change":"2023-08-27T11:27:26.089882+0000","last_active":"2023-08-27T11:27:27.059156+0000","last_peered":"2023-08-27T11:27:27.059156+0000","last_clean":"2023-08-27T11:27:27.059156+0000","last_became_active":"2023-08-27T11:27:26.089507+0000","last_became_peered":"2023-08-27T11:27:26.089507+0000","last_unstale":"2023-08-27T11:27:27.059156+0000","last_undegraded":"2023-08-27T11:27:27.059156+0000","last_fullsized":"2023-08-27T11:27:27.059156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:38:57.428180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063411+0000","last_change":"2023-08-27T11:27:26.090230+0000","last_active":"2023-08-27T11:27:27.063411+0000","last_peered":"2023-08-27T11:27:27.063411+0000","last_clean":"2023-08-27T11:27:27.063411+0000","last_became_active":"2023-08-27T11:27:26.089949+0000","last_became_peered":"2023-08-27T11:27:26.089949+0000","last_unstale":"2023-08-27T11:27:27.063411+0000","last_undegraded":"2023-08-27T11:27:27.063411+0000","last_fullsized":"2023-08-27T11:27:27.063411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:01:30.771214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296026+0000","last_change":"2023-08-27T11:27:26.068219+0000","last_active":"2023-08-27T11:27:27.296026+0000","last_peered":"2023-08-27T11:27:27.296026+0000","last_clean":"2023-08-27T11:27:27.296026+0000","last_became_active":"2023-08-27T11:27:26.067941+0000","last_became_peered":"2023-08-27T11:27:26.067941+0000","last_unstale":"2023-08-27T11:27:27.296026+0000","last_undegraded":"2023-08-27T11:27:27.296026+0000","last_fullsized":"2023-08-27T11:27:27.296026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:41.443314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064000+0000","last_change":"2023-08-27T11:27:26.074858+0000","last_active":"2023-08-27T11:27:27.064000+0000","last_peered":"2023-08-27T11:27:27.064000+0000","last_clean":"2023-08-27T11:27:27.064000+0000","last_became_active":"2023-08-27T11:27:26.074537+0000","last_became_peered":"2023-08-27T11:27:26.074537+0000","last_unstale":"2023-08-27T11:27:27.064000+0000","last_undegraded":"2023-08-27T11:27:27.064000+0000","last_fullsized":"2023-08-27T11:27:27.064000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:52.084026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059876+0000","last_change":"2023-08-27T11:27:26.077521+0000","last_active":"2023-08-27T11:27:27.059876+0000","last_peered":"2023-08-27T11:27:27.059876+0000","last_clean":"2023-08-27T11:27:27.059876+0000","last_became_active":"2023-08-27T11:27:26.077195+0000","last_became_peered":"2023-08-27T11:27:26.077195+0000","last_unstale":"2023-08-27T11:27:27.059876+0000","last_undegraded":"2023-08-27T11:27:27.059876+0000","last_fullsized":"2023-08-27T11:27:27.059876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:16:43.277045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063882+0000","last_change":"2023-08-27T11:27:26.070974+0000","last_active":"2023-08-27T11:27:27.063882+0000","last_peered":"2023-08-27T11:27:27.063882+0000","last_clean":"2023-08-27T11:27:27.063882+0000","last_became_active":"2023-08-27T11:27:26.070657+0000","last_became_peered":"2023-08-27T11:27:26.070657+0000","last_unstale":"2023-08-27T11:27:27.063882+0000","last_undegraded":"2023-08-27T11:27:27.063882+0000","last_fullsized":"2023-08-27T11:27:27.063882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:56.826686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351924+0000","last_change":"2023-08-27T11:27:26.086901+0000","last_active":"2023-08-27T11:27:27.351924+0000","last_peered":"2023-08-27T11:27:27.351924+0000","last_clean":"2023-08-27T11:27:27.351924+0000","last_became_active":"2023-08-27T11:27:26.086284+0000","last_became_peered":"2023-08-27T11:27:26.086284+0000","last_unstale":"2023-08-27T11:27:27.351924+0000","last_undegraded":"2023-08-27T11:27:27.351924+0000","last_fullsized":"2023-08-27T11:27:27.351924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:25:49.561987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351802+0000","last_change":"2023-08-27T11:27:26.082738+0000","last_active":"2023-08-27T11:27:27.351802+0000","last_peered":"2023-08-27T11:27:27.351802+0000","last_clean":"2023-08-27T11:27:27.351802+0000","last_became_active":"2023-08-27T11:27:26.082026+0000","last_became_peered":"2023-08-27T11:27:26.082026+0000","last_unstale":"2023-08-27T11:27:27.351802+0000","last_undegraded":"2023-08-27T11:27:27.351802+0000","last_fullsized":"2023-08-27T11:27:27.351802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:27.057388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161536+0000","last_change":"2023-08-27T11:27:26.078802+0000","last_active":"2023-08-27T11:27:27.161536+0000","last_peered":"2023-08-27T11:27:27.161536+0000","last_clean":"2023-08-27T11:27:27.161536+0000","last_became_active":"2023-08-27T11:27:26.078498+0000","last_became_peered":"2023-08-27T11:27:26.078498+0000","last_unstale":"2023-08-27T11:27:27.161536+0000","last_undegraded":"2023-08-27T11:27:27.161536+0000","last_fullsized":"2023-08-27T11:27:27.161536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:58:42.773675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059353+0000","last_change":"2023-08-27T11:27:26.072841+0000","last_active":"2023-08-27T11:27:27.059353+0000","last_peered":"2023-08-27T11:27:27.059353+0000","last_clean":"2023-08-27T11:27:27.059353+0000","last_became_active":"2023-08-27T11:27:26.072550+0000","last_became_peered":"2023-08-27T11:27:26.072550+0000","last_unstale":"2023-08-27T11:27:27.059353+0000","last_undegraded":"2023-08-27T11:27:27.059353+0000","last_fullsized":"2023-08-27T11:27:27.059353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:49:31.564390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161325+0000","last_change":"2023-08-27T11:27:26.070372+0000","last_active":"2023-08-27T11:27:27.161325+0000","last_peered":"2023-08-27T11:27:27.161325+0000","last_clean":"2023-08-27T11:27:27.161325+0000","last_became_active":"2023-08-27T11:27:26.070096+0000","last_became_peered":"2023-08-27T11:27:26.070096+0000","last_unstale":"2023-08-27T11:27:27.161325+0000","last_undegraded":"2023-08-27T11:27:27.161325+0000","last_fullsized":"2023-08-27T11:27:27.161325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:39:00.353719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.062801+0000","last_change":"2023-08-27T11:27:26.082158+0000","last_active":"2023-08-27T11:27:27.062801+0000","last_peered":"2023-08-27T11:27:27.062801+0000","last_clean":"2023-08-27T11:27:27.062801+0000","last_became_active":"2023-08-27T11:27:26.081820+0000","last_became_peered":"2023-08-27T11:27:26.081820+0000","last_unstale":"2023-08-27T11:27:27.062801+0000","last_undegraded":"2023-08-27T11:27:27.062801+0000","last_fullsized":"2023-08-27T11:27:27.062801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:50:44.033038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059884+0000","last_change":"2023-08-27T11:27:26.084750+0000","last_active":"2023-08-27T11:27:27.059884+0000","last_peered":"2023-08-27T11:27:27.059884+0000","last_clean":"2023-08-27T11:27:27.059884+0000","last_became_active":"2023-08-27T11:27:26.084269+0000","last_became_peered":"2023-08-27T11:27:26.084269+0000","last_unstale":"2023-08-27T11:27:27.059884+0000","last_undegraded":"2023-08-27T11:27:27.059884+0000","last_fullsized":"2023-08-27T11:27:27.059884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:20.978056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351423+0000","last_change":"2023-08-27T11:27:26.081548+0000","last_active":"2023-08-27T11:27:27.351423+0000","last_peered":"2023-08-27T11:27:27.351423+0000","last_clean":"2023-08-27T11:27:27.351423+0000","last_became_active":"2023-08-27T11:27:26.081232+0000","last_became_peered":"2023-08-27T11:27:26.081232+0000","last_unstale":"2023-08-27T11:27:27.351423+0000","last_undegraded":"2023-08-27T11:27:27.351423+0000","last_fullsized":"2023-08-27T11:27:27.351423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:08:11.914069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063591+0000","last_change":"2023-08-27T11:27:26.082791+0000","last_active":"2023-08-27T11:27:27.063591+0000","last_peered":"2023-08-27T11:27:27.063591+0000","last_clean":"2023-08-27T11:27:27.063591+0000","last_became_active":"2023-08-27T11:27:26.082524+0000","last_became_peered":"2023-08-27T11:27:26.082524+0000","last_unstale":"2023-08-27T11:27:27.063591+0000","last_undegraded":"2023-08-27T11:27:27.063591+0000","last_fullsized":"2023-08-27T11:27:27.063591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:19.789028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161481+0000","last_change":"2023-08-27T11:27:26.067318+0000","last_active":"2023-08-27T11:27:27.161481+0000","last_peered":"2023-08-27T11:27:27.161481+0000","last_clean":"2023-08-27T11:27:27.161481+0000","last_became_active":"2023-08-27T11:27:26.067027+0000","last_became_peered":"2023-08-27T11:27:26.067027+0000","last_unstale":"2023-08-27T11:27:27.161481+0000","last_undegraded":"2023-08-27T11:27:27.161481+0000","last_fullsized":"2023-08-27T11:27:27.161481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:37.028982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063164+0000","last_change":"2023-08-27T11:27:26.069926+0000","last_active":"2023-08-27T11:27:27.063164+0000","last_peered":"2023-08-27T11:27:27.063164+0000","last_clean":"2023-08-27T11:27:27.063164+0000","last_became_active":"2023-08-27T11:27:26.069646+0000","last_became_peered":"2023-08-27T11:27:26.069646+0000","last_unstale":"2023-08-27T11:27:27.063164+0000","last_undegraded":"2023-08-27T11:27:27.063164+0000","last_fullsized":"2023-08-27T11:27:27.063164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:10:33.160421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063666+0000","last_change":"2023-08-27T11:27:26.084978+0000","last_active":"2023-08-27T11:27:27.063666+0000","last_peered":"2023-08-27T11:27:27.063666+0000","last_clean":"2023-08-27T11:27:27.063666+0000","last_became_active":"2023-08-27T11:27:26.084702+0000","last_became_peered":"2023-08-27T11:27:26.084702+0000","last_unstale":"2023-08-27T11:27:27.063666+0000","last_undegraded":"2023-08-27T11:27:27.063666+0000","last_fullsized":"2023-08-27T11:27:27.063666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:14.311015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351702+0000","last_change":"2023-08-27T11:27:26.081321+0000","last_active":"2023-08-27T11:27:27.351702+0000","last_peered":"2023-08-27T11:27:27.351702+0000","last_clean":"2023-08-27T11:27:27.351702+0000","last_became_active":"2023-08-27T11:27:26.080918+0000","last_became_peered":"2023-08-27T11:27:26.080918+0000","last_unstale":"2023-08-27T11:27:27.351702+0000","last_undegraded":"2023-08-27T11:27:27.351702+0000","last_fullsized":"2023-08-27T11:27:27.351702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:37:26.812145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351534+0000","last_change":"2023-08-27T11:27:26.090185+0000","last_active":"2023-08-27T11:27:27.351534+0000","last_peered":"2023-08-27T11:27:27.351534+0000","last_clean":"2023-08-27T11:27:27.351534+0000","last_became_active":"2023-08-27T11:27:26.089902+0000","last_became_peered":"2023-08-27T11:27:26.089902+0000","last_unstale":"2023-08-27T11:27:27.351534+0000","last_undegraded":"2023-08-27T11:27:27.351534+0000","last_fullsized":"2023-08-27T11:27:27.351534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:52.550476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063587+0000","last_change":"2023-08-27T11:27:26.092089+0000","last_active":"2023-08-27T11:27:27.063587+0000","last_peered":"2023-08-27T11:27:27.063587+0000","last_clean":"2023-08-27T11:27:27.063587+0000","last_became_active":"2023-08-27T11:27:26.091715+0000","last_became_peered":"2023-08-27T11:27:26.091715+0000","last_unstale":"2023-08-27T11:27:27.063587+0000","last_undegraded":"2023-08-27T11:27:27.063587+0000","last_fullsized":"2023-08-27T11:27:27.063587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:59.404231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296538+0000","last_change":"2023-08-27T11:27:26.081434+0000","last_active":"2023-08-27T11:27:27.296538+0000","last_peered":"2023-08-27T11:27:27.296538+0000","last_clean":"2023-08-27T11:27:27.296538+0000","last_became_active":"2023-08-27T11:27:26.081149+0000","last_became_peered":"2023-08-27T11:27:26.081149+0000","last_unstale":"2023-08-27T11:27:27.296538+0000","last_undegraded":"2023-08-27T11:27:27.296538+0000","last_fullsized":"2023-08-27T11:27:27.296538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:30:44.710288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351760+0000","last_change":"2023-08-27T11:27:26.086596+0000","last_active":"2023-08-27T11:27:27.351760+0000","last_peered":"2023-08-27T11:27:27.351760+0000","last_clean":"2023-08-27T11:27:27.351760+0000","last_became_active":"2023-08-27T11:27:26.086266+0000","last_became_peered":"2023-08-27T11:27:26.086266+0000","last_unstale":"2023-08-27T11:27:27.351760+0000","last_undegraded":"2023-08-27T11:27:27.351760+0000","last_fullsized":"2023-08-27T11:27:27.351760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:37.028982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063724+0000","last_change":"2023-08-27T11:27:26.085017+0000","last_active":"2023-08-27T11:27:27.063724+0000","last_peered":"2023-08-27T11:27:27.063724+0000","last_clean":"2023-08-27T11:27:27.063724+0000","last_became_active":"2023-08-27T11:27:26.084738+0000","last_became_peered":"2023-08-27T11:27:26.084738+0000","last_unstale":"2023-08-27T11:27:27.063724+0000","last_undegraded":"2023-08-27T11:27:27.063724+0000","last_fullsized":"2023-08-27T11:27:27.063724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:57:42.609927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351546+0000","last_change":"2023-08-27T11:27:26.069380+0000","last_active":"2023-08-27T11:27:27.351546+0000","last_peered":"2023-08-27T11:27:27.351546+0000","last_clean":"2023-08-27T11:27:27.351546+0000","last_became_active":"2023-08-27T11:27:26.069085+0000","last_became_peered":"2023-08-27T11:27:26.069085+0000","last_unstale":"2023-08-27T11:27:27.351546+0000","last_undegraded":"2023-08-27T11:27:27.351546+0000","last_fullsized":"2023-08-27T11:27:27.351546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:23:39.851882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296165+0000","last_change":"2023-08-27T11:27:26.079114+0000","last_active":"2023-08-27T11:27:27.296165+0000","last_peered":"2023-08-27T11:27:27.296165+0000","last_clean":"2023-08-27T11:27:27.296165+0000","last_became_active":"2023-08-27T11:27:26.078796+0000","last_became_peered":"2023-08-27T11:27:26.078796+0000","last_unstale":"2023-08-27T11:27:27.296165+0000","last_undegraded":"2023-08-27T11:27:27.296165+0000","last_fullsized":"2023-08-27T11:27:27.296165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:38.585090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296067+0000","last_change":"2023-08-27T11:27:26.081119+0000","last_active":"2023-08-27T11:27:27.296067+0000","last_peered":"2023-08-27T11:27:27.296067+0000","last_clean":"2023-08-27T11:27:27.296067+0000","last_became_active":"2023-08-27T11:27:26.080823+0000","last_became_peered":"2023-08-27T11:27:26.080823+0000","last_unstale":"2023-08-27T11:27:27.296067+0000","last_undegraded":"2023-08-27T11:27:27.296067+0000","last_fullsized":"2023-08-27T11:27:27.296067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:58.336542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063913+0000","last_change":"2023-08-27T11:27:26.079608+0000","last_active":"2023-08-27T11:27:27.063913+0000","last_peered":"2023-08-27T11:27:27.063913+0000","last_clean":"2023-08-27T11:27:27.063913+0000","last_became_active":"2023-08-27T11:27:26.077600+0000","last_became_peered":"2023-08-27T11:27:26.077600+0000","last_unstale":"2023-08-27T11:27:27.063913+0000","last_undegraded":"2023-08-27T11:27:27.063913+0000","last_fullsized":"2023-08-27T11:27:27.063913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:56:56.685590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351863+0000","last_change":"2023-08-27T11:27:26.088721+0000","last_active":"2023-08-27T11:27:27.351863+0000","last_peered":"2023-08-27T11:27:27.351863+0000","last_clean":"2023-08-27T11:27:27.351863+0000","last_became_active":"2023-08-27T11:27:26.085981+0000","last_became_peered":"2023-08-27T11:27:26.085981+0000","last_unstale":"2023-08-27T11:27:27.351863+0000","last_undegraded":"2023-08-27T11:27:27.351863+0000","last_fullsized":"2023-08-27T11:27:27.351863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:07:44.018996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"85'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352333+0000","last_change":"2023-08-27T11:27:25.074467+0000","last_active":"2023-08-27T11:27:27.352333+0000","last_peered":"2023-08-27T11:27:27.352333+0000","last_clean":"2023-08-27T11:27:27.352333+0000","last_became_active":"2023-08-27T11:27:25.074201+0000","last_became_peered":"2023-08-27T11:27:25.074201+0000","last_unstale":"2023-08-27T11:27:27.352333+0000","last_undegraded":"2023-08-27T11:27:27.352333+0000","last_fullsized":"2023-08-27T11:27:27.352333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:02:59.315849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059984+0000","last_change":"2023-08-27T11:27:26.076337+0000","last_active":"2023-08-27T11:27:27.059984+0000","last_peered":"2023-08-27T11:27:27.059984+0000","last_clean":"2023-08-27T11:27:27.059984+0000","last_became_active":"2023-08-27T11:27:26.076067+0000","last_became_peered":"2023-08-27T11:27:26.076067+0000","last_unstale":"2023-08-27T11:27:27.059984+0000","last_undegraded":"2023-08-27T11:27:27.059984+0000","last_fullsized":"2023-08-27T11:27:27.059984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:27:06.554894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161625+0000","last_change":"2023-08-27T11:27:25.072826+0000","last_active":"2023-08-27T11:27:27.161625+0000","last_peered":"2023-08-27T11:27:27.161625+0000","last_clean":"2023-08-27T11:27:27.161625+0000","last_became_active":"2023-08-27T11:27:25.072493+0000","last_became_peered":"2023-08-27T11:27:25.072493+0000","last_unstale":"2023-08-27T11:27:27.161625+0000","last_undegraded":"2023-08-27T11:27:27.161625+0000","last_fullsized":"2023-08-27T11:27:27.161625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:15.504191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063380+0000","last_change":"2023-08-27T11:27:26.087619+0000","last_active":"2023-08-27T11:27:27.063380+0000","last_peered":"2023-08-27T11:27:27.063380+0000","last_clean":"2023-08-27T11:27:27.063380+0000","last_became_active":"2023-08-27T11:27:26.087316+0000","last_became_peered":"2023-08-27T11:27:26.087316+0000","last_unstale":"2023-08-27T11:27:27.063380+0000","last_undegraded":"2023-08-27T11:27:27.063380+0000","last_fullsized":"2023-08-27T11:27:27.063380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:39:24.848203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064208+0000","last_change":"2023-08-27T11:27:25.075211+0000","last_active":"2023-08-27T11:27:27.064208+0000","last_peered":"2023-08-27T11:27:27.064208+0000","last_clean":"2023-08-27T11:27:27.064208+0000","last_became_active":"2023-08-27T11:27:25.074962+0000","last_became_peered":"2023-08-27T11:27:25.074962+0000","last_unstale":"2023-08-27T11:27:27.064208+0000","last_undegraded":"2023-08-27T11:27:27.064208+0000","last_fullsized":"2023-08-27T11:27:27.064208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:45:27.390600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352077+0000","last_change":"2023-08-27T11:27:26.087134+0000","last_active":"2023-08-27T11:27:27.352077+0000","last_peered":"2023-08-27T11:27:27.352077+0000","last_clean":"2023-08-27T11:27:27.352077+0000","last_became_active":"2023-08-27T11:27:26.086650+0000","last_became_peered":"2023-08-27T11:27:26.086650+0000","last_unstale":"2023-08-27T11:27:27.352077+0000","last_undegraded":"2023-08-27T11:27:27.352077+0000","last_fullsized":"2023-08-27T11:27:27.352077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:08:59.725903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064230+0000","last_change":"2023-08-27T11:27:25.070402+0000","last_active":"2023-08-27T11:27:27.064230+0000","last_peered":"2023-08-27T11:27:27.064230+0000","last_clean":"2023-08-27T11:27:27.064230+0000","last_became_active":"2023-08-27T11:27:25.070112+0000","last_became_peered":"2023-08-27T11:27:25.070112+0000","last_unstale":"2023-08-27T11:27:27.064230+0000","last_undegraded":"2023-08-27T11:27:27.064230+0000","last_fullsized":"2023-08-27T11:27:27.064230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:24.871104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296192+0000","last_change":"2023-08-27T11:27:26.075409+0000","last_active":"2023-08-27T11:27:27.296192+0000","last_peered":"2023-08-27T11:27:27.296192+0000","last_clean":"2023-08-27T11:27:27.296192+0000","last_became_active":"2023-08-27T11:27:26.074931+0000","last_became_peered":"2023-08-27T11:27:26.074931+0000","last_unstale":"2023-08-27T11:27:27.296192+0000","last_undegraded":"2023-08-27T11:27:27.296192+0000","last_fullsized":"2023-08-27T11:27:27.296192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:30:20.473386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351699+0000","last_change":"2023-08-27T11:27:25.071009+0000","last_active":"2023-08-27T11:27:27.351699+0000","last_peered":"2023-08-27T11:27:27.351699+0000","last_clean":"2023-08-27T11:27:27.351699+0000","last_became_active":"2023-08-27T11:27:25.070686+0000","last_became_peered":"2023-08-27T11:27:25.070686+0000","last_unstale":"2023-08-27T11:27:27.351699+0000","last_undegraded":"2023-08-27T11:27:27.351699+0000","last_fullsized":"2023-08-27T11:27:27.351699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:20.293053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063300+0000","last_change":"2023-08-27T11:27:26.091137+0000","last_active":"2023-08-27T11:27:27.063300+0000","last_peered":"2023-08-27T11:27:27.063300+0000","last_clean":"2023-08-27T11:27:27.063300+0000","last_became_active":"2023-08-27T11:27:26.090750+0000","last_became_peered":"2023-08-27T11:27:26.090750+0000","last_unstale":"2023-08-27T11:27:27.063300+0000","last_undegraded":"2023-08-27T11:27:27.063300+0000","last_fullsized":"2023-08-27T11:27:27.063300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:31:04.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063552+0000","last_change":"2023-08-27T11:27:25.076385+0000","last_active":"2023-08-27T11:27:27.063552+0000","last_peered":"2023-08-27T11:27:27.063552+0000","last_clean":"2023-08-27T11:27:27.063552+0000","last_became_active":"2023-08-27T11:27:25.076062+0000","last_became_peered":"2023-08-27T11:27:25.076062+0000","last_unstale":"2023-08-27T11:27:27.063552+0000","last_undegraded":"2023-08-27T11:27:27.063552+0000","last_fullsized":"2023-08-27T11:27:27.063552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:59.810868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059009+0000","last_change":"2023-08-27T11:27:26.077799+0000","last_active":"2023-08-27T11:27:27.059009+0000","last_peered":"2023-08-27T11:27:27.059009+0000","last_clean":"2023-08-27T11:27:27.059009+0000","last_became_active":"2023-08-27T11:27:26.077538+0000","last_became_peered":"2023-08-27T11:27:26.077538+0000","last_unstale":"2023-08-27T11:27:27.059009+0000","last_undegraded":"2023-08-27T11:27:27.059009+0000","last_fullsized":"2023-08-27T11:27:27.059009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:49:08.232991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351227+0000","last_change":"2023-08-27T11:27:25.068965+0000","last_active":"2023-08-27T11:27:27.351227+0000","last_peered":"2023-08-27T11:27:27.351227+0000","last_clean":"2023-08-27T11:27:27.351227+0000","last_became_active":"2023-08-27T11:27:25.068679+0000","last_became_peered":"2023-08-27T11:27:25.068679+0000","last_unstale":"2023-08-27T11:27:27.351227+0000","last_undegraded":"2023-08-27T11:27:27.351227+0000","last_fullsized":"2023-08-27T11:27:27.351227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:06:18.029572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161371+0000","last_change":"2023-08-27T11:27:26.081798+0000","last_active":"2023-08-27T11:27:27.161371+0000","last_peered":"2023-08-27T11:27:27.161371+0000","last_clean":"2023-08-27T11:27:27.161371+0000","last_became_active":"2023-08-27T11:27:26.081472+0000","last_became_peered":"2023-08-27T11:27:26.081472+0000","last_unstale":"2023-08-27T11:27:27.161371+0000","last_undegraded":"2023-08-27T11:27:27.161371+0000","last_fullsized":"2023-08-27T11:27:27.161371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:39.646170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059060+0000","last_change":"2023-08-27T11:27:25.065522+0000","last_active":"2023-08-27T11:27:27.059060+0000","last_peered":"2023-08-27T11:27:27.059060+0000","last_clean":"2023-08-27T11:27:27.059060+0000","last_became_active":"2023-08-27T11:27:25.065251+0000","last_became_peered":"2023-08-27T11:27:25.065251+0000","last_unstale":"2023-08-27T11:27:27.059060+0000","last_undegraded":"2023-08-27T11:27:27.059060+0000","last_fullsized":"2023-08-27T11:27:27.059060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:19:22.540095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296547+0000","last_change":"2023-08-27T11:27:26.087358+0000","last_active":"2023-08-27T11:27:27.296547+0000","last_peered":"2023-08-27T11:27:27.296547+0000","last_clean":"2023-08-27T11:27:27.296547+0000","last_became_active":"2023-08-27T11:27:26.087095+0000","last_became_peered":"2023-08-27T11:27:26.087095+0000","last_unstale":"2023-08-27T11:27:27.296547+0000","last_undegraded":"2023-08-27T11:27:27.296547+0000","last_fullsized":"2023-08-27T11:27:27.296547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:43.667380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063732+0000","last_change":"2023-08-27T11:27:25.068373+0000","last_active":"2023-08-27T11:27:27.063732+0000","last_peered":"2023-08-27T11:27:27.063732+0000","last_clean":"2023-08-27T11:27:27.063732+0000","last_became_active":"2023-08-27T11:27:25.068082+0000","last_became_peered":"2023-08-27T11:27:25.068082+0000","last_unstale":"2023-08-27T11:27:27.063732+0000","last_undegraded":"2023-08-27T11:27:27.063732+0000","last_fullsized":"2023-08-27T11:27:27.063732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:01:05.833591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352229+0000","last_change":"2023-08-27T11:27:26.088518+0000","last_active":"2023-08-27T11:27:27.352229+0000","last_peered":"2023-08-27T11:27:27.352229+0000","last_clean":"2023-08-27T11:27:27.352229+0000","last_became_active":"2023-08-27T11:27:26.088009+0000","last_became_peered":"2023-08-27T11:27:26.088009+0000","last_unstale":"2023-08-27T11:27:27.352229+0000","last_undegraded":"2023-08-27T11:27:27.352229+0000","last_fullsized":"2023-08-27T11:27:27.352229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:59:47.337006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063976+0000","last_change":"2023-08-27T11:27:25.076541+0000","last_active":"2023-08-27T11:27:27.063976+0000","last_peered":"2023-08-27T11:27:27.063976+0000","last_clean":"2023-08-27T11:27:27.063976+0000","last_became_active":"2023-08-27T11:27:25.076262+0000","last_became_peered":"2023-08-27T11:27:25.076262+0000","last_unstale":"2023-08-27T11:27:27.063976+0000","last_undegraded":"2023-08-27T11:27:27.063976+0000","last_fullsized":"2023-08-27T11:27:27.063976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:58:56.807275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064132+0000","last_change":"2023-08-27T11:27:26.079831+0000","last_active":"2023-08-27T11:27:27.064132+0000","last_peered":"2023-08-27T11:27:27.064132+0000","last_clean":"2023-08-27T11:27:27.064132+0000","last_became_active":"2023-08-27T11:27:26.076770+0000","last_became_peered":"2023-08-27T11:27:26.076770+0000","last_unstale":"2023-08-27T11:27:27.064132+0000","last_undegraded":"2023-08-27T11:27:27.064132+0000","last_fullsized":"2023-08-27T11:27:27.064132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:09.105615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"85'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351833+0000","last_change":"2023-08-27T11:27:25.068119+0000","last_active":"2023-08-27T11:27:27.351833+0000","last_peered":"2023-08-27T11:27:27.351833+0000","last_clean":"2023-08-27T11:27:27.351833+0000","last_became_active":"2023-08-27T11:27:25.067846+0000","last_became_peered":"2023-08-27T11:27:25.067846+0000","last_unstale":"2023-08-27T11:27:27.351833+0000","last_undegraded":"2023-08-27T11:27:27.351833+0000","last_fullsized":"2023-08-27T11:27:27.351833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:58:41.765417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059528+0000","last_change":"2023-08-27T11:27:26.076330+0000","last_active":"2023-08-27T11:27:27.059528+0000","last_peered":"2023-08-27T11:27:27.059528+0000","last_clean":"2023-08-27T11:27:27.059528+0000","last_became_active":"2023-08-27T11:27:26.076029+0000","last_became_peered":"2023-08-27T11:27:26.076029+0000","last_unstale":"2023-08-27T11:27:27.059528+0000","last_undegraded":"2023-08-27T11:27:27.059528+0000","last_fullsized":"2023-08-27T11:27:27.059528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:22:05.061210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"85'6","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351878+0000","last_change":"2023-08-27T11:27:25.065700+0000","last_active":"2023-08-27T11:27:27.351878+0000","last_peered":"2023-08-27T11:27:27.351878+0000","last_clean":"2023-08-27T11:27:27.351878+0000","last_became_active":"2023-08-27T11:27:25.065409+0000","last_became_peered":"2023-08-27T11:27:25.065409+0000","last_unstale":"2023-08-27T11:27:27.351878+0000","last_undegraded":"2023-08-27T11:27:27.351878+0000","last_fullsized":"2023-08-27T11:27:27.351878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:46:52.594914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352020+0000","last_change":"2023-08-27T11:27:26.085915+0000","last_active":"2023-08-27T11:27:27.352020+0000","last_peered":"2023-08-27T11:27:27.352020+0000","last_clean":"2023-08-27T11:27:27.352020+0000","last_became_active":"2023-08-27T11:27:26.085289+0000","last_became_peered":"2023-08-27T11:27:26.085289+0000","last_unstale":"2023-08-27T11:27:27.352020+0000","last_undegraded":"2023-08-27T11:27:27.352020+0000","last_fullsized":"2023-08-27T11:27:27.352020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:17:11.814504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351518+0000","last_change":"2023-08-27T11:27:25.070284+0000","last_active":"2023-08-27T11:27:27.351518+0000","last_peered":"2023-08-27T11:27:27.351518+0000","last_clean":"2023-08-27T11:27:27.351518+0000","last_became_active":"2023-08-27T11:27:25.069996+0000","last_became_peered":"2023-08-27T11:27:25.069996+0000","last_unstale":"2023-08-27T11:27:27.351518+0000","last_undegraded":"2023-08-27T11:27:27.351518+0000","last_fullsized":"2023-08-27T11:27:27.351518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:05:55.644873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296250+0000","last_change":"2023-08-27T11:27:26.071470+0000","last_active":"2023-08-27T11:27:27.296250+0000","last_peered":"2023-08-27T11:27:27.296250+0000","last_clean":"2023-08-27T11:27:27.296250+0000","last_became_active":"2023-08-27T11:27:26.070855+0000","last_became_peered":"2023-08-27T11:27:26.070855+0000","last_unstale":"2023-08-27T11:27:27.296250+0000","last_undegraded":"2023-08-27T11:27:27.296250+0000","last_fullsized":"2023-08-27T11:27:27.296250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:27.852175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063787+0000","last_change":"2023-08-27T11:27:25.070398+0000","last_active":"2023-08-27T11:27:27.063787+0000","last_peered":"2023-08-27T11:27:27.063787+0000","last_clean":"2023-08-27T11:27:27.063787+0000","last_became_active":"2023-08-27T11:27:25.070052+0000","last_became_peered":"2023-08-27T11:27:25.070052+0000","last_unstale":"2023-08-27T11:27:27.063787+0000","last_undegraded":"2023-08-27T11:27:27.063787+0000","last_fullsized":"2023-08-27T11:27:27.063787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:00.139285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161283+0000","last_change":"2023-08-27T11:27:26.080433+0000","last_active":"2023-08-27T11:27:27.161283+0000","last_peered":"2023-08-27T11:27:27.161283+0000","last_clean":"2023-08-27T11:27:27.161283+0000","last_became_active":"2023-08-27T11:27:26.080152+0000","last_became_peered":"2023-08-27T11:27:26.080152+0000","last_unstale":"2023-08-27T11:27:27.161283+0000","last_undegraded":"2023-08-27T11:27:27.161283+0000","last_fullsized":"2023-08-27T11:27:27.161283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:23:39.851882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296238+0000","last_change":"2023-08-27T11:27:25.072045+0000","last_active":"2023-08-27T11:27:27.296238+0000","last_peered":"2023-08-27T11:27:27.296238+0000","last_clean":"2023-08-27T11:27:27.296238+0000","last_became_active":"2023-08-27T11:27:25.071747+0000","last_became_peered":"2023-08-27T11:27:25.071747+0000","last_unstale":"2023-08-27T11:27:27.296238+0000","last_undegraded":"2023-08-27T11:27:27.296238+0000","last_fullsized":"2023-08-27T11:27:27.296238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:25:31.688585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059135+0000","last_change":"2023-08-27T11:27:26.078049+0000","last_active":"2023-08-27T11:27:27.059135+0000","last_peered":"2023-08-27T11:27:27.059135+0000","last_clean":"2023-08-27T11:27:27.059135+0000","last_became_active":"2023-08-27T11:27:26.077817+0000","last_became_peered":"2023-08-27T11:27:26.077817+0000","last_unstale":"2023-08-27T11:27:27.059135+0000","last_undegraded":"2023-08-27T11:27:27.059135+0000","last_fullsized":"2023-08-27T11:27:27.059135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:23:23.072696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064081+0000","last_change":"2023-08-27T11:27:25.072388+0000","last_active":"2023-08-27T11:27:27.064081+0000","last_peered":"2023-08-27T11:27:27.064081+0000","last_clean":"2023-08-27T11:27:27.064081+0000","last_became_active":"2023-08-27T11:27:25.072093+0000","last_became_peered":"2023-08-27T11:27:25.072093+0000","last_unstale":"2023-08-27T11:27:27.064081+0000","last_undegraded":"2023-08-27T11:27:27.064081+0000","last_fullsized":"2023-08-27T11:27:27.064081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:53.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161546+0000","last_change":"2023-08-27T11:27:26.081390+0000","last_active":"2023-08-27T11:27:27.161546+0000","last_peered":"2023-08-27T11:27:27.161546+0000","last_clean":"2023-08-27T11:27:27.161546+0000","last_became_active":"2023-08-27T11:27:26.081091+0000","last_became_peered":"2023-08-27T11:27:26.081091+0000","last_unstale":"2023-08-27T11:27:27.161546+0000","last_undegraded":"2023-08-27T11:27:27.161546+0000","last_fullsized":"2023-08-27T11:27:27.161546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:10.403636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064439+0000","last_change":"2023-08-27T11:27:25.076200+0000","last_active":"2023-08-27T11:27:27.064439+0000","last_peered":"2023-08-27T11:27:27.064439+0000","last_clean":"2023-08-27T11:27:27.064439+0000","last_became_active":"2023-08-27T11:27:25.074425+0000","last_became_peered":"2023-08-27T11:27:25.074425+0000","last_unstale":"2023-08-27T11:27:27.064439+0000","last_undegraded":"2023-08-27T11:27:27.064439+0000","last_fullsized":"2023-08-27T11:27:27.064439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:10:18.953948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296518+0000","last_change":"2023-08-27T11:27:26.068215+0000","last_active":"2023-08-27T11:27:27.296518+0000","last_peered":"2023-08-27T11:27:27.296518+0000","last_clean":"2023-08-27T11:27:27.296518+0000","last_became_active":"2023-08-27T11:27:26.067963+0000","last_became_peered":"2023-08-27T11:27:26.067963+0000","last_unstale":"2023-08-27T11:27:27.296518+0000","last_undegraded":"2023-08-27T11:27:27.296518+0000","last_fullsized":"2023-08-27T11:27:27.296518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:30:23.033851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296671+0000","last_change":"2023-08-27T11:27:25.072101+0000","last_active":"2023-08-27T11:27:27.296671+0000","last_peered":"2023-08-27T11:27:27.296671+0000","last_clean":"2023-08-27T11:27:27.296671+0000","last_became_active":"2023-08-27T11:27:25.071818+0000","last_became_peered":"2023-08-27T11:27:25.071818+0000","last_unstale":"2023-08-27T11:27:27.296671+0000","last_undegraded":"2023-08-27T11:27:27.296671+0000","last_fullsized":"2023-08-27T11:27:27.296671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:48:25.380522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351828+0000","last_change":"2023-08-27T11:27:26.084874+0000","last_active":"2023-08-27T11:27:27.351828+0000","last_peered":"2023-08-27T11:27:27.351828+0000","last_clean":"2023-08-27T11:27:27.351828+0000","last_became_active":"2023-08-27T11:27:26.084599+0000","last_became_peered":"2023-08-27T11:27:26.084599+0000","last_unstale":"2023-08-27T11:27:27.351828+0000","last_undegraded":"2023-08-27T11:27:27.351828+0000","last_fullsized":"2023-08-27T11:27:27.351828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:55:08.337101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059708+0000","last_change":"2023-08-27T11:27:25.072164+0000","last_active":"2023-08-27T11:27:27.059708+0000","last_peered":"2023-08-27T11:27:27.059708+0000","last_clean":"2023-08-27T11:27:27.059708+0000","last_became_active":"2023-08-27T11:27:25.071847+0000","last_became_peered":"2023-08-27T11:27:25.071847+0000","last_unstale":"2023-08-27T11:27:27.059708+0000","last_undegraded":"2023-08-27T11:27:27.059708+0000","last_fullsized":"2023-08-27T11:27:27.059708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:44:27.328403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352107+0000","last_change":"2023-08-27T11:27:26.088845+0000","last_active":"2023-08-27T11:27:27.352107+0000","last_peered":"2023-08-27T11:27:27.352107+0000","last_clean":"2023-08-27T11:27:27.352107+0000","last_became_active":"2023-08-27T11:27:26.085575+0000","last_became_peered":"2023-08-27T11:27:26.085575+0000","last_unstale":"2023-08-27T11:27:27.352107+0000","last_undegraded":"2023-08-27T11:27:27.352107+0000","last_fullsized":"2023-08-27T11:27:27.352107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:01.252359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161472+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:27.161472+0000","last_peered":"2023-08-27T11:27:27.161472+0000","last_clean":"2023-08-27T11:27:27.161472+0000","last_became_active":"2023-08-27T11:27:25.074195+0000","last_became_peered":"2023-08-27T11:27:25.074195+0000","last_unstale":"2023-08-27T11:27:27.161472+0000","last_undegraded":"2023-08-27T11:27:27.161472+0000","last_fullsized":"2023-08-27T11:27:27.161472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:20.293053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059315+0000","last_change":"2023-08-27T11:27:26.077679+0000","last_active":"2023-08-27T11:27:27.059315+0000","last_peered":"2023-08-27T11:27:27.059315+0000","last_clean":"2023-08-27T11:27:27.059315+0000","last_became_active":"2023-08-27T11:27:26.077364+0000","last_became_peered":"2023-08-27T11:27:26.077364+0000","last_unstale":"2023-08-27T11:27:27.059315+0000","last_undegraded":"2023-08-27T11:27:27.059315+0000","last_fullsized":"2023-08-27T11:27:27.059315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:57.026007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059300+0000","last_change":"2023-08-27T11:27:25.065564+0000","last_active":"2023-08-27T11:27:27.059300+0000","last_peered":"2023-08-27T11:27:27.059300+0000","last_clean":"2023-08-27T11:27:27.059300+0000","last_became_active":"2023-08-27T11:27:25.065291+0000","last_became_peered":"2023-08-27T11:27:25.065291+0000","last_unstale":"2023-08-27T11:27:27.059300+0000","last_undegraded":"2023-08-27T11:27:27.059300+0000","last_fullsized":"2023-08-27T11:27:27.059300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:49.745243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063393+0000","last_change":"2023-08-27T11:27:26.088328+0000","last_active":"2023-08-27T11:27:27.063393+0000","last_peered":"2023-08-27T11:27:27.063393+0000","last_clean":"2023-08-27T11:27:27.063393+0000","last_became_active":"2023-08-27T11:27:26.088064+0000","last_became_peered":"2023-08-27T11:27:26.088064+0000","last_unstale":"2023-08-27T11:27:27.063393+0000","last_undegraded":"2023-08-27T11:27:27.063393+0000","last_fullsized":"2023-08-27T11:27:27.063393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:30:11.476233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351326+0000","last_change":"2023-08-27T11:27:25.069284+0000","last_active":"2023-08-27T11:27:27.351326+0000","last_peered":"2023-08-27T11:27:27.351326+0000","last_clean":"2023-08-27T11:27:27.351326+0000","last_became_active":"2023-08-27T11:27:25.068979+0000","last_became_peered":"2023-08-27T11:27:25.068979+0000","last_unstale":"2023-08-27T11:27:27.351326+0000","last_undegraded":"2023-08-27T11:27:27.351326+0000","last_fullsized":"2023-08-27T11:27:27.351326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:39.838299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351206+0000","last_change":"2023-08-27T11:27:26.073103+0000","last_active":"2023-08-27T11:27:27.351206+0000","last_peered":"2023-08-27T11:27:27.351206+0000","last_clean":"2023-08-27T11:27:27.351206+0000","last_became_active":"2023-08-27T11:27:26.072831+0000","last_became_peered":"2023-08-27T11:27:26.072831+0000","last_unstale":"2023-08-27T11:27:27.351206+0000","last_undegraded":"2023-08-27T11:27:27.351206+0000","last_fullsized":"2023-08-27T11:27:27.351206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:15.859017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.062903+0000","last_change":"2023-08-27T11:27:25.072420+0000","last_active":"2023-08-27T11:27:27.062903+0000","last_peered":"2023-08-27T11:27:27.062903+0000","last_clean":"2023-08-27T11:27:27.062903+0000","last_became_active":"2023-08-27T11:27:25.072117+0000","last_became_peered":"2023-08-27T11:27:25.072117+0000","last_unstale":"2023-08-27T11:27:27.062903+0000","last_undegraded":"2023-08-27T11:27:27.062903+0000","last_fullsized":"2023-08-27T11:27:27.062903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:28:47.603167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063190+0000","last_change":"2023-08-27T11:27:26.079290+0000","last_active":"2023-08-27T11:27:27.063190+0000","last_peered":"2023-08-27T11:27:27.063190+0000","last_clean":"2023-08-27T11:27:27.063190+0000","last_became_active":"2023-08-27T11:27:26.078538+0000","last_became_peered":"2023-08-27T11:27:26.078538+0000","last_unstale":"2023-08-27T11:27:27.063190+0000","last_undegraded":"2023-08-27T11:27:27.063190+0000","last_fullsized":"2023-08-27T11:27:27.063190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:58:28.635961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063212+0000","last_change":"2023-08-27T11:27:25.069386+0000","last_active":"2023-08-27T11:27:27.063212+0000","last_peered":"2023-08-27T11:27:27.063212+0000","last_clean":"2023-08-27T11:27:27.063212+0000","last_became_active":"2023-08-27T11:27:25.069120+0000","last_became_peered":"2023-08-27T11:27:25.069120+0000","last_unstale":"2023-08-27T11:27:27.063212+0000","last_undegraded":"2023-08-27T11:27:27.063212+0000","last_fullsized":"2023-08-27T11:27:27.063212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:39.179010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059566+0000","last_change":"2023-08-27T11:27:26.077765+0000","last_active":"2023-08-27T11:27:27.059566+0000","last_peered":"2023-08-27T11:27:27.059566+0000","last_clean":"2023-08-27T11:27:27.059566+0000","last_became_active":"2023-08-27T11:27:26.077447+0000","last_became_peered":"2023-08-27T11:27:26.077447+0000","last_unstale":"2023-08-27T11:27:27.059566+0000","last_undegraded":"2023-08-27T11:27:27.059566+0000","last_fullsized":"2023-08-27T11:27:27.059566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:28:39.663219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351960+0000","last_change":"2023-08-27T11:27:25.076447+0000","last_active":"2023-08-27T11:27:27.351960+0000","last_peered":"2023-08-27T11:27:27.351960+0000","last_clean":"2023-08-27T11:27:27.351960+0000","last_became_active":"2023-08-27T11:27:25.076148+0000","last_became_peered":"2023-08-27T11:27:25.076148+0000","last_unstale":"2023-08-27T11:27:27.351960+0000","last_undegraded":"2023-08-27T11:27:27.351960+0000","last_fullsized":"2023-08-27T11:27:27.351960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:09:55.294388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351972+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:27:27.351972+0000","last_peered":"2023-08-27T11:27:27.351972+0000","last_clean":"2023-08-27T11:27:27.351972+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:27:27.351972+0000","last_undegraded":"2023-08-27T11:27:27.351972+0000","last_fullsized":"2023-08-27T11:27:27.351972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:31:09.595009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352004+0000","last_change":"2023-08-27T11:27:26.079608+0000","last_active":"2023-08-27T11:27:27.352004+0000","last_peered":"2023-08-27T11:27:27.352004+0000","last_clean":"2023-08-27T11:27:27.352004+0000","last_became_active":"2023-08-27T11:27:26.079228+0000","last_became_peered":"2023-08-27T11:27:26.079228+0000","last_unstale":"2023-08-27T11:27:27.352004+0000","last_undegraded":"2023-08-27T11:27:27.352004+0000","last_fullsized":"2023-08-27T11:27:27.352004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:10.403636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063967+0000","last_change":"2023-08-27T11:27:25.075707+0000","last_active":"2023-08-27T11:27:27.063967+0000","last_peered":"2023-08-27T11:27:27.063967+0000","last_clean":"2023-08-27T11:27:27.063967+0000","last_became_active":"2023-08-27T11:27:25.074470+0000","last_became_peered":"2023-08-27T11:27:25.074470+0000","last_unstale":"2023-08-27T11:27:27.063967+0000","last_undegraded":"2023-08-27T11:27:27.063967+0000","last_fullsized":"2023-08-27T11:27:27.063967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:04.434107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":144,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296353+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:27:27.296353+0000","last_peered":"2023-08-27T11:27:27.296353+0000","last_clean":"2023-08-27T11:27:27.296353+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:27:27.296353+0000","last_undegraded":"2023-08-27T11:27:27.296353+0000","last_fullsized":"2023-08-27T11:27:27.296353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:51.305100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351228+0000","last_change":"2023-08-27T11:27:26.080744+0000","last_active":"2023-08-27T11:27:27.351228+0000","last_peered":"2023-08-27T11:27:27.351228+0000","last_clean":"2023-08-27T11:27:27.351228+0000","last_became_active":"2023-08-27T11:27:26.080434+0000","last_became_peered":"2023-08-27T11:27:26.080434+0000","last_unstale":"2023-08-27T11:27:27.351228+0000","last_undegraded":"2023-08-27T11:27:27.351228+0000","last_fullsized":"2023-08-27T11:27:27.351228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:19:37.219616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296287+0000","last_change":"2023-08-27T11:27:25.067095+0000","last_active":"2023-08-27T11:27:27.296287+0000","last_peered":"2023-08-27T11:27:27.296287+0000","last_clean":"2023-08-27T11:27:27.296287+0000","last_became_active":"2023-08-27T11:27:25.066815+0000","last_became_peered":"2023-08-27T11:27:25.066815+0000","last_unstale":"2023-08-27T11:27:27.296287+0000","last_undegraded":"2023-08-27T11:27:27.296287+0000","last_fullsized":"2023-08-27T11:27:27.296287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:00:37.031105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059214+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:27:27.059214+0000","last_peered":"2023-08-27T11:27:27.059214+0000","last_clean":"2023-08-27T11:27:27.059214+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:27:27.059214+0000","last_undegraded":"2023-08-27T11:27:27.059214+0000","last_fullsized":"2023-08-27T11:27:27.059214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:41:39.431379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351556+0000","last_change":"2023-08-27T11:27:26.079582+0000","last_active":"2023-08-27T11:27:27.351556+0000","last_peered":"2023-08-27T11:27:27.351556+0000","last_clean":"2023-08-27T11:27:27.351556+0000","last_became_active":"2023-08-27T11:27:26.079270+0000","last_became_peered":"2023-08-27T11:27:26.079270+0000","last_unstale":"2023-08-27T11:27:27.351556+0000","last_undegraded":"2023-08-27T11:27:27.351556+0000","last_fullsized":"2023-08-27T11:27:27.351556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:39.646170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063827+0000","last_change":"2023-08-27T11:27:25.071459+0000","last_active":"2023-08-27T11:27:27.063827+0000","last_peered":"2023-08-27T11:27:27.063827+0000","last_clean":"2023-08-27T11:27:27.063827+0000","last_became_active":"2023-08-27T11:27:25.070204+0000","last_became_peered":"2023-08-27T11:27:25.070204+0000","last_unstale":"2023-08-27T11:27:27.063827+0000","last_undegraded":"2023-08-27T11:27:27.063827+0000","last_fullsized":"2023-08-27T11:27:27.063827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:02:36.734554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":144,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296424+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:27:27.296424+0000","last_peered":"2023-08-27T11:27:27.296424+0000","last_clean":"2023-08-27T11:27:27.296424+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:27:27.296424+0000","last_undegraded":"2023-08-27T11:27:27.296424+0000","last_fullsized":"2023-08-27T11:27:27.296424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:41:36.445248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'161","reported_seq":454,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351869+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:27:27.351869+0000","last_peered":"2023-08-27T11:27:27.351869+0000","last_clean":"2023-08-27T11:27:27.351869+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:27:27.351869+0000","last_undegraded":"2023-08-27T11:27:27.351869+0000","last_fullsized":"2023-08-27T11:27:27.351869+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:00:20.892106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351320+0000","last_change":"2023-08-27T11:27:26.079726+0000","last_active":"2023-08-27T11:27:27.351320+0000","last_peered":"2023-08-27T11:27:27.351320+0000","last_clean":"2023-08-27T11:27:27.351320+0000","last_became_active":"2023-08-27T11:27:26.079252+0000","last_became_peered":"2023-08-27T11:27:26.079252+0000","last_unstale":"2023-08-27T11:27:27.351320+0000","last_undegraded":"2023-08-27T11:27:27.351320+0000","last_fullsized":"2023-08-27T11:27:27.351320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:31.948081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161267+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:27.161267+0000","last_peered":"2023-08-27T11:27:27.161267+0000","last_clean":"2023-08-27T11:27:27.161267+0000","last_became_active":"2023-08-27T11:27:25.074184+0000","last_became_peered":"2023-08-27T11:27:25.074184+0000","last_unstale":"2023-08-27T11:27:27.161267+0000","last_undegraded":"2023-08-27T11:27:27.161267+0000","last_fullsized":"2023-08-27T11:27:27.161267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:30.939823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351698+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:27:27.351698+0000","last_peered":"2023-08-27T11:27:27.351698+0000","last_clean":"2023-08-27T11:27:27.351698+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:27:27.351698+0000","last_undegraded":"2023-08-27T11:27:27.351698+0000","last_fullsized":"2023-08-27T11:27:27.351698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:11.294773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351857+0000","last_change":"2023-08-27T11:27:26.082541+0000","last_active":"2023-08-27T11:27:27.351857+0000","last_peered":"2023-08-27T11:27:27.351857+0000","last_clean":"2023-08-27T11:27:27.351857+0000","last_became_active":"2023-08-27T11:27:26.081375+0000","last_became_peered":"2023-08-27T11:27:26.081375+0000","last_unstale":"2023-08-27T11:27:27.351857+0000","last_undegraded":"2023-08-27T11:27:27.351857+0000","last_fullsized":"2023-08-27T11:27:27.351857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:17:56.457425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063750+0000","last_change":"2023-08-27T11:27:25.077520+0000","last_active":"2023-08-27T11:27:27.063750+0000","last_peered":"2023-08-27T11:27:27.063750+0000","last_clean":"2023-08-27T11:27:27.063750+0000","last_became_active":"2023-08-27T11:27:25.077249+0000","last_became_peered":"2023-08-27T11:27:25.077249+0000","last_unstale":"2023-08-27T11:27:27.063750+0000","last_undegraded":"2023-08-27T11:27:27.063750+0000","last_fullsized":"2023-08-27T11:27:27.063750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:54:46.692552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352247+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:27:27.352247+0000","last_peered":"2023-08-27T11:27:27.352247+0000","last_clean":"2023-08-27T11:27:27.352247+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:27:27.352247+0000","last_undegraded":"2023-08-27T11:27:27.352247+0000","last_fullsized":"2023-08-27T11:27:27.352247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:46:15.407970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352192+0000","last_change":"2023-08-27T11:27:26.088339+0000","last_active":"2023-08-27T11:27:27.352192+0000","last_peered":"2023-08-27T11:27:27.352192+0000","last_clean":"2023-08-27T11:27:27.352192+0000","last_became_active":"2023-08-27T11:27:26.087691+0000","last_became_peered":"2023-08-27T11:27:26.087691+0000","last_unstale":"2023-08-27T11:27:27.352192+0000","last_undegraded":"2023-08-27T11:27:27.352192+0000","last_fullsized":"2023-08-27T11:27:27.352192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:27.133704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064114+0000","last_change":"2023-08-27T11:27:25.078799+0000","last_active":"2023-08-27T11:27:27.064114+0000","last_peered":"2023-08-27T11:27:27.064114+0000","last_clean":"2023-08-27T11:27:27.064114+0000","last_became_active":"2023-08-27T11:27:25.078503+0000","last_became_peered":"2023-08-27T11:27:25.078503+0000","last_unstale":"2023-08-27T11:27:27.064114+0000","last_undegraded":"2023-08-27T11:27:27.064114+0000","last_fullsized":"2023-08-27T11:27:27.064114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:09.111047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351524+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:27:27.351524+0000","last_peered":"2023-08-27T11:27:27.351524+0000","last_clean":"2023-08-27T11:27:27.351524+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:27:27.351524+0000","last_undegraded":"2023-08-27T11:27:27.351524+0000","last_fullsized":"2023-08-27T11:27:27.351524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:57:40.389214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063905+0000","last_change":"2023-08-27T11:27:26.071818+0000","last_active":"2023-08-27T11:27:27.063905+0000","last_peered":"2023-08-27T11:27:27.063905+0000","last_clean":"2023-08-27T11:27:27.063905+0000","last_became_active":"2023-08-27T11:27:26.071566+0000","last_became_peered":"2023-08-27T11:27:26.071566+0000","last_unstale":"2023-08-27T11:27:27.063905+0000","last_undegraded":"2023-08-27T11:27:27.063905+0000","last_fullsized":"2023-08-27T11:27:27.063905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:10.038438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064458+0000","last_change":"2023-08-27T11:27:25.074341+0000","last_active":"2023-08-27T11:27:27.064458+0000","last_peered":"2023-08-27T11:27:27.064458+0000","last_clean":"2023-08-27T11:27:27.064458+0000","last_became_active":"2023-08-27T11:27:25.073689+0000","last_became_peered":"2023-08-27T11:27:25.073689+0000","last_unstale":"2023-08-27T11:27:27.064458+0000","last_undegraded":"2023-08-27T11:27:27.064458+0000","last_fullsized":"2023-08-27T11:27:27.064458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:59:53.040233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064523+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:27:27.064523+0000","last_peered":"2023-08-27T11:27:27.064523+0000","last_clean":"2023-08-27T11:27:27.064523+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:27:27.064523+0000","last_undegraded":"2023-08-27T11:27:27.064523+0000","last_fullsized":"2023-08-27T11:27:27.064523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:12:37.934346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1646592,"data_stored":1135117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":1.601,"15min":1.601},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40899999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.72199999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.29099999999999998},"max":{"1min":0.85499999999999998,"5min":1.778,"15min":1.778},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63500000000000001,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63600000000000001,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.67400000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.432,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68400000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56699999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48299999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88300000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.50600000000000001},"min":{"1min":0.29799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575035,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":544225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.40799999999999997,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.76700000000000002,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.628},"min":{"1min":0.47399999999999998,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.3540000000000001},"last":0.54100000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":3.3780000000000001},"last":0.439},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.80700000000000005,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.56499999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.61699999999999999},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.73399999999999999,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.504},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":1.325},"last":0.39100000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.60499999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.71699999999999997,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.623}]},{"osd":4,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.46400000000000002},"min":{"1min":0.312,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"min":{"1min":0.35199999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.624,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.58499999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":1.157,"15min":1.157},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.70399999999999996,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.55400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575035,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40548,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331292,"statfs":{"total":96636764160,"available":96595243008,"internally_reserved":0,"allocated":1064960,"data_stored":546361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20399999999999999},"max":{"1min":0.52100000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.628,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40699999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42699999999999999,"15min":0.42399999999999999},"min":{"1min":0.27500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.34100000000000003,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58999999999999997,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.33300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":1.169,"15min":1.169},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44600000000000001,"15min":0.44},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.23400000000000001},"max":{"1min":0.55900000000000005,"5min":1.139,"15min":1.139},"last":0.27200000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.82599999999999996,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81499999999999995,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.46999999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.45900000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88700000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.437}]},{"osd":7,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89400000000000002,"15min":1.139},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1638400,"data_stored":1134501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.32200000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.34300000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.378}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.498},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.437}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.41799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":545745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.216}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.30599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34100000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.748}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.78400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1052672,"data_stored":544245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40699999999999997,"15min":0.42499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.22},"max":{"1min":0.60799999999999998,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.224},"max":{"1min":0.60699999999999998,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.27000000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.64100000000000001,"15min":6.0830000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46200000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.245},"max":{"1min":0.57399999999999995,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.30299999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.68000000000000005,"5min":0.71199999999999997,"15min":1.25},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":0.84499999999999997,"15min":11.939},"last":0.39700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.376},"max":{"1min":0.83099999999999996,"5min":1.0329999999999999,"15min":11.007},"last":0.502},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":1.113,"15min":1.2929999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93300000000000005,"5min":3.0219999999999998,"15min":11.981},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":1.218,"15min":11.09},"last":0.57699999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69099999999999995,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.75600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":1.0269999999999999,"15min":12.016},"last":0.55200000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68999999999999995,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":0.96699999999999997,"15min":11.054},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.53500000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84099999999999997,"5min":1.0640000000000001,"15min":1.361},"last":0.53500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575035,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":544203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.68400000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.71799999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.746},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.5},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.53700000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.70299999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575034,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335468,"statfs":{"total":96636764160,"available":96599519232,"internally_reserved":0,"allocated":1048576,"data_stored":544223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.59199999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.52700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.437}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.623}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:27:34.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:27:35.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.041+0000 7f375eb40700 1 Processor -- start 2023-08-27T11:27:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.041+0000 7f375eb40700 1 -- start start 2023-08-27T11:27:35.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.042+0000 7f375eb40700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f375815a240 0x7f37580cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.042+0000 7f375eb40700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580cbe70 0x7f37580ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.042+0000 7f375eb40700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375806d690 con 0x7f37580ce9d0 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.042+0000 7f375eb40700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37580cee20 con 0x7f375815a240 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.042+0000 7f375eb40700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37580cc440 con 0x7f37580cbe70 2023-08-27T11:27:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.042+0000 7f375d0dd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f37580ce9d0 0x7f3758070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44628/0 (socket says 172.21.15.45:44628) 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.042+0000 7f375d0dd700 1 -- 172.21.15.45:0/1064371124 learned_addr learned my addr 172.21.15.45:0/1064371124 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f375c8dc700 1 --2- 172.21.15.45:0/1064371124 >> v2:172.21.15.135:3300/0 conn(0x7f375815a240 0x7f37580cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f3757fff700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580cbe70 0x7f37580ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232902777 0 0) 0x7f375806d690 con 0x7f37580ce9d0 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37480057c0 con 0x7f37580ce9d0 2023-08-27T11:27:35.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 491630473 0 0) 0x7f37480057c0 con 0x7f37580ce9d0 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580cbe70 msgr2=0x7f37580ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580cbe70 0x7f37580ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 >> v2:172.21.15.135:3300/0 conn(0x7f375815a240 msgr2=0x7f37580cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 --2- 172.21.15.45:0/1064371124 >> v2:172.21.15.135:3300/0 conn(0x7f375815a240 0x7f37580cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375806d040 con 0x7f37580ce9d0 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f37500019f0 con 0x7f37580ce9d0 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 >> v1:172.21.15.45:6789/0 conn(0x7f37580ce9d0 legacy=0x7f3758070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580cbe70 0x7f3748005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748006190 0x7f3748006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006fa0 0x7f3748007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37580cee20 con 0x7f3748006190 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37580cc440 con 0x7f3748006fa0 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374800dc20 con 0x7f37580cbe70 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f3757fff700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748006190 0x7f3748006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f375d0dd700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006fa0 0x7f3748007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.043+0000 7f375c8dc700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580cbe70 0x7f3748005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f3757fff700 1 -- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580cbe70 msgr2=0x7f3748005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f3757fff700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580cbe70 0x7f3748005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f3757fff700 1 -- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006fa0 msgr2=0x7f3748007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f3757fff700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006fa0 0x7f3748007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f3757fff700 1 -- 172.21.15.45:0/1064371124 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37480057c0 con 0x7f3748006190 2023-08-27T11:27:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f375c8dc700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580cbe70 0x7f3748005c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f375d0dd700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006fa0 0x7f3748007420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f3757fff700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748006190 0x7f3748006a60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3744009b60 tx=0x7f3748007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf4c79640e4a82 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400e660 con 0x7f3748006190 2023-08-27T11:27:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400e7c0 con 0x7f3748006190 2023-08-27T11:27:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f37577fe700 1 -- 172.21.15.45:0/1064371124 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400eb30 con 0x7f3748006190 2023-08-27T11:27:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f375eb40700 1 -- 172.21.15.45:0/1064371124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748006190 msgr2=0x7f3748006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f375eb40700 1 --2- 172.21.15.45:0/1064371124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3748006190 0x7f3748006a60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3744009b60 tx=0x7f3748007a90 comp rx=0 tx=0).stop 2023-08-27T11:27:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f375d0dd700 1 -- 172.21.15.45:0/1064371124 reap_dead start 2023-08-27T11:27:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f375eb40700 1 -- 172.21.15.45:0/1064371124 shutdown_connections 2023-08-27T11:27:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.044+0000 7f375eb40700 1 -- 172.21.15.45:0/1064371124 >> 172.21.15.45:0/1064371124 conn(0x7f37580cfe50 msgr2=0x7f37580d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.045+0000 7f375eb40700 1 -- 172.21.15.45:0/1064371124 shutdown_connections 2023-08-27T11:27:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.045+0000 7f375eb40700 1 -- 172.21.15.45:0/1064371124 wait complete. 2023-08-27T11:27:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.046+0000 7f375eb40700 1 Processor -- start 2023-08-27T11:27:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.046+0000 7f375eb40700 1 -- start start 2023-08-27T11:27:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f375eb40700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006190 0x7f37581f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f375eb40700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37581ee1a0 0x7f37580c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f375eb40700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580c83a0 0x7f3758158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f375eb40700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375815c5d0 con 0x7f37581ee1a0 2023-08-27T11:27:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f375eb40700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37581f1490 con 0x7f3748006190 2023-08-27T11:27:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f375eb40700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375814fc10 con 0x7f37580c83a0 2023-08-27T11:27:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f3757fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37581ee1a0 0x7f37580c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f3757fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37581ee1a0 0x7f37580c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57862/0 (socket says 172.21.15.45:57862) 2023-08-27T11:27:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f3757fff700 1 -- 172.21.15.45:0/1136140065 learned_addr learned my addr 172.21.15.45:0/1136140065 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f375d0dd700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580c83a0 0x7f3758158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f375c8dc700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006190 0x7f37581f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f3757fff700 1 -- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580c83a0 msgr2=0x7f3758158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f3757fff700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580c83a0 0x7f3758158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f3757fff700 1 -- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006190 msgr2=0x7f37581f0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f3757fff700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006190 0x7f37581f0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f3757fff700 1 -- 172.21.15.45:0/1136140065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375815b9c0 con 0x7f37581ee1a0 2023-08-27T11:27:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.047+0000 7f3757fff700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37581ee1a0 0x7f37580c7e60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3748006900 tx=0x7f37440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.048+0000 7f37557fa700 1 -- 172.21.15.45:0/1136140065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400e660 con 0x7f37581ee1a0 2023-08-27T11:27:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.048+0000 7f37557fa700 1 -- 172.21.15.45:0/1136140065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37440038a0 con 0x7f37581ee1a0 2023-08-27T11:27:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.048+0000 7f37557fa700 1 -- 172.21.15.45:0/1136140065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744003c50 con 0x7f37581ee1a0 2023-08-27T11:27:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.048+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375815bca0 con 0x7f37581ee1a0 2023-08-27T11:27:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.048+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375815c260 con 0x7f37581ee1a0 2023-08-27T11:27:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.049+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37580c8780 con 0x7f37581ee1a0 2023-08-27T11:27:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.049+0000 7f37557fa700 1 -- 172.21.15.45:0/1136140065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3744013470 con 0x7f37581ee1a0 2023-08-27T11:27:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.050+0000 7f37557fa700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f373c06a200 0x7f373c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.050+0000 7f37557fa700 1 -- 172.21.15.45:0/1136140065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f3744089df0 con 0x7f37581ee1a0 2023-08-27T11:27:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.050+0000 7f375c8dc700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f373c06a200 0x7f373c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.050+0000 7f375c8dc700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f373c06a200 0x7f373c06c6c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f374c009c60 tx=0x7f374c008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.054+0000 7f37557fa700 1 -- 172.21.15.45:0/1136140065 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f37440552f0 con 0x7f37581ee1a0 2023-08-27T11:27:35.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.249+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 --> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37580ce1a0 con 0x7f373c06a200 2023-08-27T11:27:35.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.259+0000 7f37557fa700 1 -- 172.21.15.45:0/1136140065 <== mgr.6247 v2:172.21.15.135:6832/3430028400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311724 (secure 0 0 0) 0x7f374c0098b0 con 0x7f373c06a200 2023-08-27T11:27:35.263 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f373c06a200 msgr2=0x7f373c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f373c06a200 0x7f373c06c6c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f374c009c60 tx=0x7f374c008040 comp rx=0 tx=0).stop 2023-08-27T11:27:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37581ee1a0 msgr2=0x7f37580c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37581ee1a0 0x7f37580c7e60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3748006900 tx=0x7f37440047c0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 shutdown_connections 2023-08-27T11:27:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f373c06a200 0x7f373c06c6c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f37580c83a0 0x7f3758158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3748006190 0x7f37581f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 --2- 172.21.15.45:0/1136140065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37581ee1a0 0x7f37580c7e60 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 >> 172.21.15.45:0/1136140065 conn(0x7f37580cfe50 msgr2=0x7f37580b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 shutdown_connections 2023-08-27T11:27:35.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.262+0000 7f375eb40700 1 -- 172.21.15.45:0/1136140065 wait complete. 2023-08-27T11:27:35.268 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:27:35.276 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-27T11:27:33.352180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":284,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267188,"kb_used_data":9384,"kb_used_omap":13,"kb_used_meta":257650,"kb_avail":754707532,"statfs":{"total":773094113280,"available":772820512768,"internally_reserved":0,"allocated":9609216,"data_stored":5538620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":263833688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002100"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352046+0000","last_change":"2023-08-27T11:27:26.087156+0000","last_active":"2023-08-27T11:27:27.352046+0000","last_peered":"2023-08-27T11:27:27.352046+0000","last_clean":"2023-08-27T11:27:27.352046+0000","last_became_active":"2023-08-27T11:27:26.086775+0000","last_became_peered":"2023-08-27T11:27:26.086775+0000","last_unstale":"2023-08-27T11:27:27.352046+0000","last_undegraded":"2023-08-27T11:27:27.352046+0000","last_fullsized":"2023-08-27T11:27:27.352046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:58:28.845767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063862+0000","last_change":"2023-08-27T11:27:26.090010+0000","last_active":"2023-08-27T11:27:27.063862+0000","last_peered":"2023-08-27T11:27:27.063862+0000","last_clean":"2023-08-27T11:27:27.063862+0000","last_became_active":"2023-08-27T11:27:26.089682+0000","last_became_peered":"2023-08-27T11:27:26.089682+0000","last_unstale":"2023-08-27T11:27:27.063862+0000","last_undegraded":"2023-08-27T11:27:27.063862+0000","last_fullsized":"2023-08-27T11:27:27.063862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:31.151499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351678+0000","last_change":"2023-08-27T11:27:26.074317+0000","last_active":"2023-08-27T11:27:27.351678+0000","last_peered":"2023-08-27T11:27:27.351678+0000","last_clean":"2023-08-27T11:27:27.351678+0000","last_became_active":"2023-08-27T11:27:26.073998+0000","last_became_peered":"2023-08-27T11:27:26.073998+0000","last_unstale":"2023-08-27T11:27:27.351678+0000","last_undegraded":"2023-08-27T11:27:27.351678+0000","last_fullsized":"2023-08-27T11:27:27.351678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:39:00.353719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351718+0000","last_change":"2023-08-27T11:27:26.082333+0000","last_active":"2023-08-27T11:27:27.351718+0000","last_peered":"2023-08-27T11:27:27.351718+0000","last_clean":"2023-08-27T11:27:27.351718+0000","last_became_active":"2023-08-27T11:27:26.080958+0000","last_became_peered":"2023-08-27T11:27:26.080958+0000","last_unstale":"2023-08-27T11:27:27.351718+0000","last_undegraded":"2023-08-27T11:27:27.351718+0000","last_fullsized":"2023-08-27T11:27:27.351718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:44:07.170889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059156+0000","last_change":"2023-08-27T11:27:26.089882+0000","last_active":"2023-08-27T11:27:27.059156+0000","last_peered":"2023-08-27T11:27:27.059156+0000","last_clean":"2023-08-27T11:27:27.059156+0000","last_became_active":"2023-08-27T11:27:26.089507+0000","last_became_peered":"2023-08-27T11:27:26.089507+0000","last_unstale":"2023-08-27T11:27:27.059156+0000","last_undegraded":"2023-08-27T11:27:27.059156+0000","last_fullsized":"2023-08-27T11:27:27.059156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:38:57.428180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063411+0000","last_change":"2023-08-27T11:27:26.090230+0000","last_active":"2023-08-27T11:27:27.063411+0000","last_peered":"2023-08-27T11:27:27.063411+0000","last_clean":"2023-08-27T11:27:27.063411+0000","last_became_active":"2023-08-27T11:27:26.089949+0000","last_became_peered":"2023-08-27T11:27:26.089949+0000","last_unstale":"2023-08-27T11:27:27.063411+0000","last_undegraded":"2023-08-27T11:27:27.063411+0000","last_fullsized":"2023-08-27T11:27:27.063411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:01:30.771214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296026+0000","last_change":"2023-08-27T11:27:26.068219+0000","last_active":"2023-08-27T11:27:27.296026+0000","last_peered":"2023-08-27T11:27:27.296026+0000","last_clean":"2023-08-27T11:27:27.296026+0000","last_became_active":"2023-08-27T11:27:26.067941+0000","last_became_peered":"2023-08-27T11:27:26.067941+0000","last_unstale":"2023-08-27T11:27:27.296026+0000","last_undegraded":"2023-08-27T11:27:27.296026+0000","last_fullsized":"2023-08-27T11:27:27.296026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:41.443314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064000+0000","last_change":"2023-08-27T11:27:26.074858+0000","last_active":"2023-08-27T11:27:27.064000+0000","last_peered":"2023-08-27T11:27:27.064000+0000","last_clean":"2023-08-27T11:27:27.064000+0000","last_became_active":"2023-08-27T11:27:26.074537+0000","last_became_peered":"2023-08-27T11:27:26.074537+0000","last_unstale":"2023-08-27T11:27:27.064000+0000","last_undegraded":"2023-08-27T11:27:27.064000+0000","last_fullsized":"2023-08-27T11:27:27.064000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:52.084026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059876+0000","last_change":"2023-08-27T11:27:26.077521+0000","last_active":"2023-08-27T11:27:27.059876+0000","last_peered":"2023-08-27T11:27:27.059876+0000","last_clean":"2023-08-27T11:27:27.059876+0000","last_became_active":"2023-08-27T11:27:26.077195+0000","last_became_peered":"2023-08-27T11:27:26.077195+0000","last_unstale":"2023-08-27T11:27:27.059876+0000","last_undegraded":"2023-08-27T11:27:27.059876+0000","last_fullsized":"2023-08-27T11:27:27.059876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:16:43.277045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063882+0000","last_change":"2023-08-27T11:27:26.070974+0000","last_active":"2023-08-27T11:27:27.063882+0000","last_peered":"2023-08-27T11:27:27.063882+0000","last_clean":"2023-08-27T11:27:27.063882+0000","last_became_active":"2023-08-27T11:27:26.070657+0000","last_became_peered":"2023-08-27T11:27:26.070657+0000","last_unstale":"2023-08-27T11:27:27.063882+0000","last_undegraded":"2023-08-27T11:27:27.063882+0000","last_fullsized":"2023-08-27T11:27:27.063882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:28:56.826686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351924+0000","last_change":"2023-08-27T11:27:26.086901+0000","last_active":"2023-08-27T11:27:27.351924+0000","last_peered":"2023-08-27T11:27:27.351924+0000","last_clean":"2023-08-27T11:27:27.351924+0000","last_became_active":"2023-08-27T11:27:26.086284+0000","last_became_peered":"2023-08-27T11:27:26.086284+0000","last_unstale":"2023-08-27T11:27:27.351924+0000","last_undegraded":"2023-08-27T11:27:27.351924+0000","last_fullsized":"2023-08-27T11:27:27.351924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:25:49.561987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351802+0000","last_change":"2023-08-27T11:27:26.082738+0000","last_active":"2023-08-27T11:27:27.351802+0000","last_peered":"2023-08-27T11:27:27.351802+0000","last_clean":"2023-08-27T11:27:27.351802+0000","last_became_active":"2023-08-27T11:27:26.082026+0000","last_became_peered":"2023-08-27T11:27:26.082026+0000","last_unstale":"2023-08-27T11:27:27.351802+0000","last_undegraded":"2023-08-27T11:27:27.351802+0000","last_fullsized":"2023-08-27T11:27:27.351802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:12:27.057388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161536+0000","last_change":"2023-08-27T11:27:26.078802+0000","last_active":"2023-08-27T11:27:27.161536+0000","last_peered":"2023-08-27T11:27:27.161536+0000","last_clean":"2023-08-27T11:27:27.161536+0000","last_became_active":"2023-08-27T11:27:26.078498+0000","last_became_peered":"2023-08-27T11:27:26.078498+0000","last_unstale":"2023-08-27T11:27:27.161536+0000","last_undegraded":"2023-08-27T11:27:27.161536+0000","last_fullsized":"2023-08-27T11:27:27.161536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:58:42.773675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059353+0000","last_change":"2023-08-27T11:27:26.072841+0000","last_active":"2023-08-27T11:27:27.059353+0000","last_peered":"2023-08-27T11:27:27.059353+0000","last_clean":"2023-08-27T11:27:27.059353+0000","last_became_active":"2023-08-27T11:27:26.072550+0000","last_became_peered":"2023-08-27T11:27:26.072550+0000","last_unstale":"2023-08-27T11:27:27.059353+0000","last_undegraded":"2023-08-27T11:27:27.059353+0000","last_fullsized":"2023-08-27T11:27:27.059353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:49:31.564390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161325+0000","last_change":"2023-08-27T11:27:26.070372+0000","last_active":"2023-08-27T11:27:27.161325+0000","last_peered":"2023-08-27T11:27:27.161325+0000","last_clean":"2023-08-27T11:27:27.161325+0000","last_became_active":"2023-08-27T11:27:26.070096+0000","last_became_peered":"2023-08-27T11:27:26.070096+0000","last_unstale":"2023-08-27T11:27:27.161325+0000","last_undegraded":"2023-08-27T11:27:27.161325+0000","last_fullsized":"2023-08-27T11:27:27.161325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:39:00.353719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.062801+0000","last_change":"2023-08-27T11:27:26.082158+0000","last_active":"2023-08-27T11:27:27.062801+0000","last_peered":"2023-08-27T11:27:27.062801+0000","last_clean":"2023-08-27T11:27:27.062801+0000","last_became_active":"2023-08-27T11:27:26.081820+0000","last_became_peered":"2023-08-27T11:27:26.081820+0000","last_unstale":"2023-08-27T11:27:27.062801+0000","last_undegraded":"2023-08-27T11:27:27.062801+0000","last_fullsized":"2023-08-27T11:27:27.062801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:50:44.033038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059884+0000","last_change":"2023-08-27T11:27:26.084750+0000","last_active":"2023-08-27T11:27:27.059884+0000","last_peered":"2023-08-27T11:27:27.059884+0000","last_clean":"2023-08-27T11:27:27.059884+0000","last_became_active":"2023-08-27T11:27:26.084269+0000","last_became_peered":"2023-08-27T11:27:26.084269+0000","last_unstale":"2023-08-27T11:27:27.059884+0000","last_undegraded":"2023-08-27T11:27:27.059884+0000","last_fullsized":"2023-08-27T11:27:27.059884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:19:20.978056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351423+0000","last_change":"2023-08-27T11:27:26.081548+0000","last_active":"2023-08-27T11:27:27.351423+0000","last_peered":"2023-08-27T11:27:27.351423+0000","last_clean":"2023-08-27T11:27:27.351423+0000","last_became_active":"2023-08-27T11:27:26.081232+0000","last_became_peered":"2023-08-27T11:27:26.081232+0000","last_unstale":"2023-08-27T11:27:27.351423+0000","last_undegraded":"2023-08-27T11:27:27.351423+0000","last_fullsized":"2023-08-27T11:27:27.351423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:08:11.914069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063591+0000","last_change":"2023-08-27T11:27:26.082791+0000","last_active":"2023-08-27T11:27:27.063591+0000","last_peered":"2023-08-27T11:27:27.063591+0000","last_clean":"2023-08-27T11:27:27.063591+0000","last_became_active":"2023-08-27T11:27:26.082524+0000","last_became_peered":"2023-08-27T11:27:26.082524+0000","last_unstale":"2023-08-27T11:27:27.063591+0000","last_undegraded":"2023-08-27T11:27:27.063591+0000","last_fullsized":"2023-08-27T11:27:27.063591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:19.789028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161481+0000","last_change":"2023-08-27T11:27:26.067318+0000","last_active":"2023-08-27T11:27:27.161481+0000","last_peered":"2023-08-27T11:27:27.161481+0000","last_clean":"2023-08-27T11:27:27.161481+0000","last_became_active":"2023-08-27T11:27:26.067027+0000","last_became_peered":"2023-08-27T11:27:26.067027+0000","last_unstale":"2023-08-27T11:27:27.161481+0000","last_undegraded":"2023-08-27T11:27:27.161481+0000","last_fullsized":"2023-08-27T11:27:27.161481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:37.028982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063164+0000","last_change":"2023-08-27T11:27:26.069926+0000","last_active":"2023-08-27T11:27:27.063164+0000","last_peered":"2023-08-27T11:27:27.063164+0000","last_clean":"2023-08-27T11:27:27.063164+0000","last_became_active":"2023-08-27T11:27:26.069646+0000","last_became_peered":"2023-08-27T11:27:26.069646+0000","last_unstale":"2023-08-27T11:27:27.063164+0000","last_undegraded":"2023-08-27T11:27:27.063164+0000","last_fullsized":"2023-08-27T11:27:27.063164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:10:33.160421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063666+0000","last_change":"2023-08-27T11:27:26.084978+0000","last_active":"2023-08-27T11:27:27.063666+0000","last_peered":"2023-08-27T11:27:27.063666+0000","last_clean":"2023-08-27T11:27:27.063666+0000","last_became_active":"2023-08-27T11:27:26.084702+0000","last_became_peered":"2023-08-27T11:27:26.084702+0000","last_unstale":"2023-08-27T11:27:27.063666+0000","last_undegraded":"2023-08-27T11:27:27.063666+0000","last_fullsized":"2023-08-27T11:27:27.063666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:14.311015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351702+0000","last_change":"2023-08-27T11:27:26.081321+0000","last_active":"2023-08-27T11:27:27.351702+0000","last_peered":"2023-08-27T11:27:27.351702+0000","last_clean":"2023-08-27T11:27:27.351702+0000","last_became_active":"2023-08-27T11:27:26.080918+0000","last_became_peered":"2023-08-27T11:27:26.080918+0000","last_unstale":"2023-08-27T11:27:27.351702+0000","last_undegraded":"2023-08-27T11:27:27.351702+0000","last_fullsized":"2023-08-27T11:27:27.351702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:37:26.812145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351534+0000","last_change":"2023-08-27T11:27:26.090185+0000","last_active":"2023-08-27T11:27:27.351534+0000","last_peered":"2023-08-27T11:27:27.351534+0000","last_clean":"2023-08-27T11:27:27.351534+0000","last_became_active":"2023-08-27T11:27:26.089902+0000","last_became_peered":"2023-08-27T11:27:26.089902+0000","last_unstale":"2023-08-27T11:27:27.351534+0000","last_undegraded":"2023-08-27T11:27:27.351534+0000","last_fullsized":"2023-08-27T11:27:27.351534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:00:52.550476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063587+0000","last_change":"2023-08-27T11:27:26.092089+0000","last_active":"2023-08-27T11:27:27.063587+0000","last_peered":"2023-08-27T11:27:27.063587+0000","last_clean":"2023-08-27T11:27:27.063587+0000","last_became_active":"2023-08-27T11:27:26.091715+0000","last_became_peered":"2023-08-27T11:27:26.091715+0000","last_unstale":"2023-08-27T11:27:27.063587+0000","last_undegraded":"2023-08-27T11:27:27.063587+0000","last_fullsized":"2023-08-27T11:27:27.063587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:59.404231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296538+0000","last_change":"2023-08-27T11:27:26.081434+0000","last_active":"2023-08-27T11:27:27.296538+0000","last_peered":"2023-08-27T11:27:27.296538+0000","last_clean":"2023-08-27T11:27:27.296538+0000","last_became_active":"2023-08-27T11:27:26.081149+0000","last_became_peered":"2023-08-27T11:27:26.081149+0000","last_unstale":"2023-08-27T11:27:27.296538+0000","last_undegraded":"2023-08-27T11:27:27.296538+0000","last_fullsized":"2023-08-27T11:27:27.296538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:30:44.710288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351760+0000","last_change":"2023-08-27T11:27:26.086596+0000","last_active":"2023-08-27T11:27:27.351760+0000","last_peered":"2023-08-27T11:27:27.351760+0000","last_clean":"2023-08-27T11:27:27.351760+0000","last_became_active":"2023-08-27T11:27:26.086266+0000","last_became_peered":"2023-08-27T11:27:26.086266+0000","last_unstale":"2023-08-27T11:27:27.351760+0000","last_undegraded":"2023-08-27T11:27:27.351760+0000","last_fullsized":"2023-08-27T11:27:27.351760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:37.028982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063724+0000","last_change":"2023-08-27T11:27:26.085017+0000","last_active":"2023-08-27T11:27:27.063724+0000","last_peered":"2023-08-27T11:27:27.063724+0000","last_clean":"2023-08-27T11:27:27.063724+0000","last_became_active":"2023-08-27T11:27:26.084738+0000","last_became_peered":"2023-08-27T11:27:26.084738+0000","last_unstale":"2023-08-27T11:27:27.063724+0000","last_undegraded":"2023-08-27T11:27:27.063724+0000","last_fullsized":"2023-08-27T11:27:27.063724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:57:42.609927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351546+0000","last_change":"2023-08-27T11:27:26.069380+0000","last_active":"2023-08-27T11:27:27.351546+0000","last_peered":"2023-08-27T11:27:27.351546+0000","last_clean":"2023-08-27T11:27:27.351546+0000","last_became_active":"2023-08-27T11:27:26.069085+0000","last_became_peered":"2023-08-27T11:27:26.069085+0000","last_unstale":"2023-08-27T11:27:27.351546+0000","last_undegraded":"2023-08-27T11:27:27.351546+0000","last_fullsized":"2023-08-27T11:27:27.351546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:23:39.851882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296165+0000","last_change":"2023-08-27T11:27:26.079114+0000","last_active":"2023-08-27T11:27:27.296165+0000","last_peered":"2023-08-27T11:27:27.296165+0000","last_clean":"2023-08-27T11:27:27.296165+0000","last_became_active":"2023-08-27T11:27:26.078796+0000","last_became_peered":"2023-08-27T11:27:26.078796+0000","last_unstale":"2023-08-27T11:27:27.296165+0000","last_undegraded":"2023-08-27T11:27:27.296165+0000","last_fullsized":"2023-08-27T11:27:27.296165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:21:38.585090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296067+0000","last_change":"2023-08-27T11:27:26.081119+0000","last_active":"2023-08-27T11:27:27.296067+0000","last_peered":"2023-08-27T11:27:27.296067+0000","last_clean":"2023-08-27T11:27:27.296067+0000","last_became_active":"2023-08-27T11:27:26.080823+0000","last_became_peered":"2023-08-27T11:27:26.080823+0000","last_unstale":"2023-08-27T11:27:27.296067+0000","last_undegraded":"2023-08-27T11:27:27.296067+0000","last_fullsized":"2023-08-27T11:27:27.296067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:58.336542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063913+0000","last_change":"2023-08-27T11:27:26.079608+0000","last_active":"2023-08-27T11:27:27.063913+0000","last_peered":"2023-08-27T11:27:27.063913+0000","last_clean":"2023-08-27T11:27:27.063913+0000","last_became_active":"2023-08-27T11:27:26.077600+0000","last_became_peered":"2023-08-27T11:27:26.077600+0000","last_unstale":"2023-08-27T11:27:27.063913+0000","last_undegraded":"2023-08-27T11:27:27.063913+0000","last_fullsized":"2023-08-27T11:27:27.063913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:56:56.685590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351863+0000","last_change":"2023-08-27T11:27:26.088721+0000","last_active":"2023-08-27T11:27:27.351863+0000","last_peered":"2023-08-27T11:27:27.351863+0000","last_clean":"2023-08-27T11:27:27.351863+0000","last_became_active":"2023-08-27T11:27:26.085981+0000","last_became_peered":"2023-08-27T11:27:26.085981+0000","last_unstale":"2023-08-27T11:27:27.351863+0000","last_undegraded":"2023-08-27T11:27:27.351863+0000","last_fullsized":"2023-08-27T11:27:27.351863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:07:44.018996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"85'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352333+0000","last_change":"2023-08-27T11:27:25.074467+0000","last_active":"2023-08-27T11:27:27.352333+0000","last_peered":"2023-08-27T11:27:27.352333+0000","last_clean":"2023-08-27T11:27:27.352333+0000","last_became_active":"2023-08-27T11:27:25.074201+0000","last_became_peered":"2023-08-27T11:27:25.074201+0000","last_unstale":"2023-08-27T11:27:27.352333+0000","last_undegraded":"2023-08-27T11:27:27.352333+0000","last_fullsized":"2023-08-27T11:27:27.352333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:02:59.315849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059984+0000","last_change":"2023-08-27T11:27:26.076337+0000","last_active":"2023-08-27T11:27:27.059984+0000","last_peered":"2023-08-27T11:27:27.059984+0000","last_clean":"2023-08-27T11:27:27.059984+0000","last_became_active":"2023-08-27T11:27:26.076067+0000","last_became_peered":"2023-08-27T11:27:26.076067+0000","last_unstale":"2023-08-27T11:27:27.059984+0000","last_undegraded":"2023-08-27T11:27:27.059984+0000","last_fullsized":"2023-08-27T11:27:27.059984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:27:06.554894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161625+0000","last_change":"2023-08-27T11:27:25.072826+0000","last_active":"2023-08-27T11:27:27.161625+0000","last_peered":"2023-08-27T11:27:27.161625+0000","last_clean":"2023-08-27T11:27:27.161625+0000","last_became_active":"2023-08-27T11:27:25.072493+0000","last_became_peered":"2023-08-27T11:27:25.072493+0000","last_unstale":"2023-08-27T11:27:27.161625+0000","last_undegraded":"2023-08-27T11:27:27.161625+0000","last_fullsized":"2023-08-27T11:27:27.161625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:34:15.504191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063380+0000","last_change":"2023-08-27T11:27:26.087619+0000","last_active":"2023-08-27T11:27:27.063380+0000","last_peered":"2023-08-27T11:27:27.063380+0000","last_clean":"2023-08-27T11:27:27.063380+0000","last_became_active":"2023-08-27T11:27:26.087316+0000","last_became_peered":"2023-08-27T11:27:26.087316+0000","last_unstale":"2023-08-27T11:27:27.063380+0000","last_undegraded":"2023-08-27T11:27:27.063380+0000","last_fullsized":"2023-08-27T11:27:27.063380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:39:24.848203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064208+0000","last_change":"2023-08-27T11:27:25.075211+0000","last_active":"2023-08-27T11:27:27.064208+0000","last_peered":"2023-08-27T11:27:27.064208+0000","last_clean":"2023-08-27T11:27:27.064208+0000","last_became_active":"2023-08-27T11:27:25.074962+0000","last_became_peered":"2023-08-27T11:27:25.074962+0000","last_unstale":"2023-08-27T11:27:27.064208+0000","last_undegraded":"2023-08-27T11:27:27.064208+0000","last_fullsized":"2023-08-27T11:27:27.064208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:45:27.390600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352077+0000","last_change":"2023-08-27T11:27:26.087134+0000","last_active":"2023-08-27T11:27:27.352077+0000","last_peered":"2023-08-27T11:27:27.352077+0000","last_clean":"2023-08-27T11:27:27.352077+0000","last_became_active":"2023-08-27T11:27:26.086650+0000","last_became_peered":"2023-08-27T11:27:26.086650+0000","last_unstale":"2023-08-27T11:27:27.352077+0000","last_undegraded":"2023-08-27T11:27:27.352077+0000","last_fullsized":"2023-08-27T11:27:27.352077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:08:59.725903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064230+0000","last_change":"2023-08-27T11:27:25.070402+0000","last_active":"2023-08-27T11:27:27.064230+0000","last_peered":"2023-08-27T11:27:27.064230+0000","last_clean":"2023-08-27T11:27:27.064230+0000","last_became_active":"2023-08-27T11:27:25.070112+0000","last_became_peered":"2023-08-27T11:27:25.070112+0000","last_unstale":"2023-08-27T11:27:27.064230+0000","last_undegraded":"2023-08-27T11:27:27.064230+0000","last_fullsized":"2023-08-27T11:27:27.064230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:52:24.871104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296192+0000","last_change":"2023-08-27T11:27:26.075409+0000","last_active":"2023-08-27T11:27:27.296192+0000","last_peered":"2023-08-27T11:27:27.296192+0000","last_clean":"2023-08-27T11:27:27.296192+0000","last_became_active":"2023-08-27T11:27:26.074931+0000","last_became_peered":"2023-08-27T11:27:26.074931+0000","last_unstale":"2023-08-27T11:27:27.296192+0000","last_undegraded":"2023-08-27T11:27:27.296192+0000","last_fullsized":"2023-08-27T11:27:27.296192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:30:20.473386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351699+0000","last_change":"2023-08-27T11:27:25.071009+0000","last_active":"2023-08-27T11:27:27.351699+0000","last_peered":"2023-08-27T11:27:27.351699+0000","last_clean":"2023-08-27T11:27:27.351699+0000","last_became_active":"2023-08-27T11:27:25.070686+0000","last_became_peered":"2023-08-27T11:27:25.070686+0000","last_unstale":"2023-08-27T11:27:27.351699+0000","last_undegraded":"2023-08-27T11:27:27.351699+0000","last_fullsized":"2023-08-27T11:27:27.351699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:20.293053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063300+0000","last_change":"2023-08-27T11:27:26.091137+0000","last_active":"2023-08-27T11:27:27.063300+0000","last_peered":"2023-08-27T11:27:27.063300+0000","last_clean":"2023-08-27T11:27:27.063300+0000","last_became_active":"2023-08-27T11:27:26.090750+0000","last_became_peered":"2023-08-27T11:27:26.090750+0000","last_unstale":"2023-08-27T11:27:27.063300+0000","last_undegraded":"2023-08-27T11:27:27.063300+0000","last_fullsized":"2023-08-27T11:27:27.063300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:31:04.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063552+0000","last_change":"2023-08-27T11:27:25.076385+0000","last_active":"2023-08-27T11:27:27.063552+0000","last_peered":"2023-08-27T11:27:27.063552+0000","last_clean":"2023-08-27T11:27:27.063552+0000","last_became_active":"2023-08-27T11:27:25.076062+0000","last_became_peered":"2023-08-27T11:27:25.076062+0000","last_unstale":"2023-08-27T11:27:27.063552+0000","last_undegraded":"2023-08-27T11:27:27.063552+0000","last_fullsized":"2023-08-27T11:27:27.063552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:10:59.810868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059009+0000","last_change":"2023-08-27T11:27:26.077799+0000","last_active":"2023-08-27T11:27:27.059009+0000","last_peered":"2023-08-27T11:27:27.059009+0000","last_clean":"2023-08-27T11:27:27.059009+0000","last_became_active":"2023-08-27T11:27:26.077538+0000","last_became_peered":"2023-08-27T11:27:26.077538+0000","last_unstale":"2023-08-27T11:27:27.059009+0000","last_undegraded":"2023-08-27T11:27:27.059009+0000","last_fullsized":"2023-08-27T11:27:27.059009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:49:08.232991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351227+0000","last_change":"2023-08-27T11:27:25.068965+0000","last_active":"2023-08-27T11:27:27.351227+0000","last_peered":"2023-08-27T11:27:27.351227+0000","last_clean":"2023-08-27T11:27:27.351227+0000","last_became_active":"2023-08-27T11:27:25.068679+0000","last_became_peered":"2023-08-27T11:27:25.068679+0000","last_unstale":"2023-08-27T11:27:27.351227+0000","last_undegraded":"2023-08-27T11:27:27.351227+0000","last_fullsized":"2023-08-27T11:27:27.351227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:06:18.029572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161371+0000","last_change":"2023-08-27T11:27:26.081798+0000","last_active":"2023-08-27T11:27:27.161371+0000","last_peered":"2023-08-27T11:27:27.161371+0000","last_clean":"2023-08-27T11:27:27.161371+0000","last_became_active":"2023-08-27T11:27:26.081472+0000","last_became_peered":"2023-08-27T11:27:26.081472+0000","last_unstale":"2023-08-27T11:27:27.161371+0000","last_undegraded":"2023-08-27T11:27:27.161371+0000","last_fullsized":"2023-08-27T11:27:27.161371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:39.646170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059060+0000","last_change":"2023-08-27T11:27:25.065522+0000","last_active":"2023-08-27T11:27:27.059060+0000","last_peered":"2023-08-27T11:27:27.059060+0000","last_clean":"2023-08-27T11:27:27.059060+0000","last_became_active":"2023-08-27T11:27:25.065251+0000","last_became_peered":"2023-08-27T11:27:25.065251+0000","last_unstale":"2023-08-27T11:27:27.059060+0000","last_undegraded":"2023-08-27T11:27:27.059060+0000","last_fullsized":"2023-08-27T11:27:27.059060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:19:22.540095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296547+0000","last_change":"2023-08-27T11:27:26.087358+0000","last_active":"2023-08-27T11:27:27.296547+0000","last_peered":"2023-08-27T11:27:27.296547+0000","last_clean":"2023-08-27T11:27:27.296547+0000","last_became_active":"2023-08-27T11:27:26.087095+0000","last_became_peered":"2023-08-27T11:27:26.087095+0000","last_unstale":"2023-08-27T11:27:27.296547+0000","last_undegraded":"2023-08-27T11:27:27.296547+0000","last_fullsized":"2023-08-27T11:27:27.296547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:55:43.667380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063732+0000","last_change":"2023-08-27T11:27:25.068373+0000","last_active":"2023-08-27T11:27:27.063732+0000","last_peered":"2023-08-27T11:27:27.063732+0000","last_clean":"2023-08-27T11:27:27.063732+0000","last_became_active":"2023-08-27T11:27:25.068082+0000","last_became_peered":"2023-08-27T11:27:25.068082+0000","last_unstale":"2023-08-27T11:27:27.063732+0000","last_undegraded":"2023-08-27T11:27:27.063732+0000","last_fullsized":"2023-08-27T11:27:27.063732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:01:05.833591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352229+0000","last_change":"2023-08-27T11:27:26.088518+0000","last_active":"2023-08-27T11:27:27.352229+0000","last_peered":"2023-08-27T11:27:27.352229+0000","last_clean":"2023-08-27T11:27:27.352229+0000","last_became_active":"2023-08-27T11:27:26.088009+0000","last_became_peered":"2023-08-27T11:27:26.088009+0000","last_unstale":"2023-08-27T11:27:27.352229+0000","last_undegraded":"2023-08-27T11:27:27.352229+0000","last_fullsized":"2023-08-27T11:27:27.352229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:59:47.337006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063976+0000","last_change":"2023-08-27T11:27:25.076541+0000","last_active":"2023-08-27T11:27:27.063976+0000","last_peered":"2023-08-27T11:27:27.063976+0000","last_clean":"2023-08-27T11:27:27.063976+0000","last_became_active":"2023-08-27T11:27:25.076262+0000","last_became_peered":"2023-08-27T11:27:25.076262+0000","last_unstale":"2023-08-27T11:27:27.063976+0000","last_undegraded":"2023-08-27T11:27:27.063976+0000","last_fullsized":"2023-08-27T11:27:27.063976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:58:56.807275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064132+0000","last_change":"2023-08-27T11:27:26.079831+0000","last_active":"2023-08-27T11:27:27.064132+0000","last_peered":"2023-08-27T11:27:27.064132+0000","last_clean":"2023-08-27T11:27:27.064132+0000","last_became_active":"2023-08-27T11:27:26.076770+0000","last_became_peered":"2023-08-27T11:27:26.076770+0000","last_unstale":"2023-08-27T11:27:27.064132+0000","last_undegraded":"2023-08-27T11:27:27.064132+0000","last_fullsized":"2023-08-27T11:27:27.064132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:01:09.105615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"85'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351833+0000","last_change":"2023-08-27T11:27:25.068119+0000","last_active":"2023-08-27T11:27:27.351833+0000","last_peered":"2023-08-27T11:27:27.351833+0000","last_clean":"2023-08-27T11:27:27.351833+0000","last_became_active":"2023-08-27T11:27:25.067846+0000","last_became_peered":"2023-08-27T11:27:25.067846+0000","last_unstale":"2023-08-27T11:27:27.351833+0000","last_undegraded":"2023-08-27T11:27:27.351833+0000","last_fullsized":"2023-08-27T11:27:27.351833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:58:41.765417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059528+0000","last_change":"2023-08-27T11:27:26.076330+0000","last_active":"2023-08-27T11:27:27.059528+0000","last_peered":"2023-08-27T11:27:27.059528+0000","last_clean":"2023-08-27T11:27:27.059528+0000","last_became_active":"2023-08-27T11:27:26.076029+0000","last_became_peered":"2023-08-27T11:27:26.076029+0000","last_unstale":"2023-08-27T11:27:27.059528+0000","last_undegraded":"2023-08-27T11:27:27.059528+0000","last_fullsized":"2023-08-27T11:27:27.059528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:22:05.061210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"85'6","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351878+0000","last_change":"2023-08-27T11:27:25.065700+0000","last_active":"2023-08-27T11:27:27.351878+0000","last_peered":"2023-08-27T11:27:27.351878+0000","last_clean":"2023-08-27T11:27:27.351878+0000","last_became_active":"2023-08-27T11:27:25.065409+0000","last_became_peered":"2023-08-27T11:27:25.065409+0000","last_unstale":"2023-08-27T11:27:27.351878+0000","last_undegraded":"2023-08-27T11:27:27.351878+0000","last_fullsized":"2023-08-27T11:27:27.351878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:46:52.594914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352020+0000","last_change":"2023-08-27T11:27:26.085915+0000","last_active":"2023-08-27T11:27:27.352020+0000","last_peered":"2023-08-27T11:27:27.352020+0000","last_clean":"2023-08-27T11:27:27.352020+0000","last_became_active":"2023-08-27T11:27:26.085289+0000","last_became_peered":"2023-08-27T11:27:26.085289+0000","last_unstale":"2023-08-27T11:27:27.352020+0000","last_undegraded":"2023-08-27T11:27:27.352020+0000","last_fullsized":"2023-08-27T11:27:27.352020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:17:11.814504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351518+0000","last_change":"2023-08-27T11:27:25.070284+0000","last_active":"2023-08-27T11:27:27.351518+0000","last_peered":"2023-08-27T11:27:27.351518+0000","last_clean":"2023-08-27T11:27:27.351518+0000","last_became_active":"2023-08-27T11:27:25.069996+0000","last_became_peered":"2023-08-27T11:27:25.069996+0000","last_unstale":"2023-08-27T11:27:27.351518+0000","last_undegraded":"2023-08-27T11:27:27.351518+0000","last_fullsized":"2023-08-27T11:27:27.351518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:05:55.644873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296250+0000","last_change":"2023-08-27T11:27:26.071470+0000","last_active":"2023-08-27T11:27:27.296250+0000","last_peered":"2023-08-27T11:27:27.296250+0000","last_clean":"2023-08-27T11:27:27.296250+0000","last_became_active":"2023-08-27T11:27:26.070855+0000","last_became_peered":"2023-08-27T11:27:26.070855+0000","last_unstale":"2023-08-27T11:27:27.296250+0000","last_undegraded":"2023-08-27T11:27:27.296250+0000","last_fullsized":"2023-08-27T11:27:27.296250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:55:27.852175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063787+0000","last_change":"2023-08-27T11:27:25.070398+0000","last_active":"2023-08-27T11:27:27.063787+0000","last_peered":"2023-08-27T11:27:27.063787+0000","last_clean":"2023-08-27T11:27:27.063787+0000","last_became_active":"2023-08-27T11:27:25.070052+0000","last_became_peered":"2023-08-27T11:27:25.070052+0000","last_unstale":"2023-08-27T11:27:27.063787+0000","last_undegraded":"2023-08-27T11:27:27.063787+0000","last_fullsized":"2023-08-27T11:27:27.063787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:00.139285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161283+0000","last_change":"2023-08-27T11:27:26.080433+0000","last_active":"2023-08-27T11:27:27.161283+0000","last_peered":"2023-08-27T11:27:27.161283+0000","last_clean":"2023-08-27T11:27:27.161283+0000","last_became_active":"2023-08-27T11:27:26.080152+0000","last_became_peered":"2023-08-27T11:27:26.080152+0000","last_unstale":"2023-08-27T11:27:27.161283+0000","last_undegraded":"2023-08-27T11:27:27.161283+0000","last_fullsized":"2023-08-27T11:27:27.161283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:23:39.851882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296238+0000","last_change":"2023-08-27T11:27:25.072045+0000","last_active":"2023-08-27T11:27:27.296238+0000","last_peered":"2023-08-27T11:27:27.296238+0000","last_clean":"2023-08-27T11:27:27.296238+0000","last_became_active":"2023-08-27T11:27:25.071747+0000","last_became_peered":"2023-08-27T11:27:25.071747+0000","last_unstale":"2023-08-27T11:27:27.296238+0000","last_undegraded":"2023-08-27T11:27:27.296238+0000","last_fullsized":"2023-08-27T11:27:27.296238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:25:31.688585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059135+0000","last_change":"2023-08-27T11:27:26.078049+0000","last_active":"2023-08-27T11:27:27.059135+0000","last_peered":"2023-08-27T11:27:27.059135+0000","last_clean":"2023-08-27T11:27:27.059135+0000","last_became_active":"2023-08-27T11:27:26.077817+0000","last_became_peered":"2023-08-27T11:27:26.077817+0000","last_unstale":"2023-08-27T11:27:27.059135+0000","last_undegraded":"2023-08-27T11:27:27.059135+0000","last_fullsized":"2023-08-27T11:27:27.059135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:23:23.072696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064081+0000","last_change":"2023-08-27T11:27:25.072388+0000","last_active":"2023-08-27T11:27:27.064081+0000","last_peered":"2023-08-27T11:27:27.064081+0000","last_clean":"2023-08-27T11:27:27.064081+0000","last_became_active":"2023-08-27T11:27:25.072093+0000","last_became_peered":"2023-08-27T11:27:25.072093+0000","last_unstale":"2023-08-27T11:27:27.064081+0000","last_undegraded":"2023-08-27T11:27:27.064081+0000","last_fullsized":"2023-08-27T11:27:27.064081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:53.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161546+0000","last_change":"2023-08-27T11:27:26.081390+0000","last_active":"2023-08-27T11:27:27.161546+0000","last_peered":"2023-08-27T11:27:27.161546+0000","last_clean":"2023-08-27T11:27:27.161546+0000","last_became_active":"2023-08-27T11:27:26.081091+0000","last_became_peered":"2023-08-27T11:27:26.081091+0000","last_unstale":"2023-08-27T11:27:27.161546+0000","last_undegraded":"2023-08-27T11:27:27.161546+0000","last_fullsized":"2023-08-27T11:27:27.161546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:10.403636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064439+0000","last_change":"2023-08-27T11:27:25.076200+0000","last_active":"2023-08-27T11:27:27.064439+0000","last_peered":"2023-08-27T11:27:27.064439+0000","last_clean":"2023-08-27T11:27:27.064439+0000","last_became_active":"2023-08-27T11:27:25.074425+0000","last_became_peered":"2023-08-27T11:27:25.074425+0000","last_unstale":"2023-08-27T11:27:27.064439+0000","last_undegraded":"2023-08-27T11:27:27.064439+0000","last_fullsized":"2023-08-27T11:27:27.064439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:10:18.953948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296518+0000","last_change":"2023-08-27T11:27:26.068215+0000","last_active":"2023-08-27T11:27:27.296518+0000","last_peered":"2023-08-27T11:27:27.296518+0000","last_clean":"2023-08-27T11:27:27.296518+0000","last_became_active":"2023-08-27T11:27:26.067963+0000","last_became_peered":"2023-08-27T11:27:26.067963+0000","last_unstale":"2023-08-27T11:27:27.296518+0000","last_undegraded":"2023-08-27T11:27:27.296518+0000","last_fullsized":"2023-08-27T11:27:27.296518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:30:23.033851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296671+0000","last_change":"2023-08-27T11:27:25.072101+0000","last_active":"2023-08-27T11:27:27.296671+0000","last_peered":"2023-08-27T11:27:27.296671+0000","last_clean":"2023-08-27T11:27:27.296671+0000","last_became_active":"2023-08-27T11:27:25.071818+0000","last_became_peered":"2023-08-27T11:27:25.071818+0000","last_unstale":"2023-08-27T11:27:27.296671+0000","last_undegraded":"2023-08-27T11:27:27.296671+0000","last_fullsized":"2023-08-27T11:27:27.296671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:48:25.380522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351828+0000","last_change":"2023-08-27T11:27:26.084874+0000","last_active":"2023-08-27T11:27:27.351828+0000","last_peered":"2023-08-27T11:27:27.351828+0000","last_clean":"2023-08-27T11:27:27.351828+0000","last_became_active":"2023-08-27T11:27:26.084599+0000","last_became_peered":"2023-08-27T11:27:26.084599+0000","last_unstale":"2023-08-27T11:27:27.351828+0000","last_undegraded":"2023-08-27T11:27:27.351828+0000","last_fullsized":"2023-08-27T11:27:27.351828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:55:08.337101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059708+0000","last_change":"2023-08-27T11:27:25.072164+0000","last_active":"2023-08-27T11:27:27.059708+0000","last_peered":"2023-08-27T11:27:27.059708+0000","last_clean":"2023-08-27T11:27:27.059708+0000","last_became_active":"2023-08-27T11:27:25.071847+0000","last_became_peered":"2023-08-27T11:27:25.071847+0000","last_unstale":"2023-08-27T11:27:27.059708+0000","last_undegraded":"2023-08-27T11:27:27.059708+0000","last_fullsized":"2023-08-27T11:27:27.059708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:44:27.328403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352107+0000","last_change":"2023-08-27T11:27:26.088845+0000","last_active":"2023-08-27T11:27:27.352107+0000","last_peered":"2023-08-27T11:27:27.352107+0000","last_clean":"2023-08-27T11:27:27.352107+0000","last_became_active":"2023-08-27T11:27:26.085575+0000","last_became_peered":"2023-08-27T11:27:26.085575+0000","last_unstale":"2023-08-27T11:27:27.352107+0000","last_undegraded":"2023-08-27T11:27:27.352107+0000","last_fullsized":"2023-08-27T11:27:27.352107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:01.252359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161472+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:27.161472+0000","last_peered":"2023-08-27T11:27:27.161472+0000","last_clean":"2023-08-27T11:27:27.161472+0000","last_became_active":"2023-08-27T11:27:25.074195+0000","last_became_peered":"2023-08-27T11:27:25.074195+0000","last_unstale":"2023-08-27T11:27:27.161472+0000","last_undegraded":"2023-08-27T11:27:27.161472+0000","last_fullsized":"2023-08-27T11:27:27.161472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:20.293053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059315+0000","last_change":"2023-08-27T11:27:26.077679+0000","last_active":"2023-08-27T11:27:27.059315+0000","last_peered":"2023-08-27T11:27:27.059315+0000","last_clean":"2023-08-27T11:27:27.059315+0000","last_became_active":"2023-08-27T11:27:26.077364+0000","last_became_peered":"2023-08-27T11:27:26.077364+0000","last_unstale":"2023-08-27T11:27:27.059315+0000","last_undegraded":"2023-08-27T11:27:27.059315+0000","last_fullsized":"2023-08-27T11:27:27.059315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:22:57.026007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059300+0000","last_change":"2023-08-27T11:27:25.065564+0000","last_active":"2023-08-27T11:27:27.059300+0000","last_peered":"2023-08-27T11:27:27.059300+0000","last_clean":"2023-08-27T11:27:27.059300+0000","last_became_active":"2023-08-27T11:27:25.065291+0000","last_became_peered":"2023-08-27T11:27:25.065291+0000","last_unstale":"2023-08-27T11:27:27.059300+0000","last_undegraded":"2023-08-27T11:27:27.059300+0000","last_fullsized":"2023-08-27T11:27:27.059300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:49.745243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063393+0000","last_change":"2023-08-27T11:27:26.088328+0000","last_active":"2023-08-27T11:27:27.063393+0000","last_peered":"2023-08-27T11:27:27.063393+0000","last_clean":"2023-08-27T11:27:27.063393+0000","last_became_active":"2023-08-27T11:27:26.088064+0000","last_became_peered":"2023-08-27T11:27:26.088064+0000","last_unstale":"2023-08-27T11:27:27.063393+0000","last_undegraded":"2023-08-27T11:27:27.063393+0000","last_fullsized":"2023-08-27T11:27:27.063393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:30:11.476233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351326+0000","last_change":"2023-08-27T11:27:25.069284+0000","last_active":"2023-08-27T11:27:27.351326+0000","last_peered":"2023-08-27T11:27:27.351326+0000","last_clean":"2023-08-27T11:27:27.351326+0000","last_became_active":"2023-08-27T11:27:25.068979+0000","last_became_peered":"2023-08-27T11:27:25.068979+0000","last_unstale":"2023-08-27T11:27:27.351326+0000","last_undegraded":"2023-08-27T11:27:27.351326+0000","last_fullsized":"2023-08-27T11:27:27.351326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:39.838299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351206+0000","last_change":"2023-08-27T11:27:26.073103+0000","last_active":"2023-08-27T11:27:27.351206+0000","last_peered":"2023-08-27T11:27:27.351206+0000","last_clean":"2023-08-27T11:27:27.351206+0000","last_became_active":"2023-08-27T11:27:26.072831+0000","last_became_peered":"2023-08-27T11:27:26.072831+0000","last_unstale":"2023-08-27T11:27:27.351206+0000","last_undegraded":"2023-08-27T11:27:27.351206+0000","last_fullsized":"2023-08-27T11:27:27.351206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:15.859017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.062903+0000","last_change":"2023-08-27T11:27:25.072420+0000","last_active":"2023-08-27T11:27:27.062903+0000","last_peered":"2023-08-27T11:27:27.062903+0000","last_clean":"2023-08-27T11:27:27.062903+0000","last_became_active":"2023-08-27T11:27:25.072117+0000","last_became_peered":"2023-08-27T11:27:25.072117+0000","last_unstale":"2023-08-27T11:27:27.062903+0000","last_undegraded":"2023-08-27T11:27:27.062903+0000","last_fullsized":"2023-08-27T11:27:27.062903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:28:47.603167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063190+0000","last_change":"2023-08-27T11:27:26.079290+0000","last_active":"2023-08-27T11:27:27.063190+0000","last_peered":"2023-08-27T11:27:27.063190+0000","last_clean":"2023-08-27T11:27:27.063190+0000","last_became_active":"2023-08-27T11:27:26.078538+0000","last_became_peered":"2023-08-27T11:27:26.078538+0000","last_unstale":"2023-08-27T11:27:27.063190+0000","last_undegraded":"2023-08-27T11:27:27.063190+0000","last_fullsized":"2023-08-27T11:27:27.063190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:58:28.635961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063212+0000","last_change":"2023-08-27T11:27:25.069386+0000","last_active":"2023-08-27T11:27:27.063212+0000","last_peered":"2023-08-27T11:27:27.063212+0000","last_clean":"2023-08-27T11:27:27.063212+0000","last_became_active":"2023-08-27T11:27:25.069120+0000","last_became_peered":"2023-08-27T11:27:25.069120+0000","last_unstale":"2023-08-27T11:27:27.063212+0000","last_undegraded":"2023-08-27T11:27:27.063212+0000","last_fullsized":"2023-08-27T11:27:27.063212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:39.179010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059566+0000","last_change":"2023-08-27T11:27:26.077765+0000","last_active":"2023-08-27T11:27:27.059566+0000","last_peered":"2023-08-27T11:27:27.059566+0000","last_clean":"2023-08-27T11:27:27.059566+0000","last_became_active":"2023-08-27T11:27:26.077447+0000","last_became_peered":"2023-08-27T11:27:26.077447+0000","last_unstale":"2023-08-27T11:27:27.059566+0000","last_undegraded":"2023-08-27T11:27:27.059566+0000","last_fullsized":"2023-08-27T11:27:27.059566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:28:39.663219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351960+0000","last_change":"2023-08-27T11:27:25.076447+0000","last_active":"2023-08-27T11:27:27.351960+0000","last_peered":"2023-08-27T11:27:27.351960+0000","last_clean":"2023-08-27T11:27:27.351960+0000","last_became_active":"2023-08-27T11:27:25.076148+0000","last_became_peered":"2023-08-27T11:27:25.076148+0000","last_unstale":"2023-08-27T11:27:27.351960+0000","last_undegraded":"2023-08-27T11:27:27.351960+0000","last_fullsized":"2023-08-27T11:27:27.351960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:09:55.294388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351972+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:27:27.351972+0000","last_peered":"2023-08-27T11:27:27.351972+0000","last_clean":"2023-08-27T11:27:27.351972+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:27:27.351972+0000","last_undegraded":"2023-08-27T11:27:27.351972+0000","last_fullsized":"2023-08-27T11:27:27.351972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:31:09.595009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352004+0000","last_change":"2023-08-27T11:27:26.079608+0000","last_active":"2023-08-27T11:27:27.352004+0000","last_peered":"2023-08-27T11:27:27.352004+0000","last_clean":"2023-08-27T11:27:27.352004+0000","last_became_active":"2023-08-27T11:27:26.079228+0000","last_became_peered":"2023-08-27T11:27:26.079228+0000","last_unstale":"2023-08-27T11:27:27.352004+0000","last_undegraded":"2023-08-27T11:27:27.352004+0000","last_fullsized":"2023-08-27T11:27:27.352004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:11:10.403636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063967+0000","last_change":"2023-08-27T11:27:25.075707+0000","last_active":"2023-08-27T11:27:27.063967+0000","last_peered":"2023-08-27T11:27:27.063967+0000","last_clean":"2023-08-27T11:27:27.063967+0000","last_became_active":"2023-08-27T11:27:25.074470+0000","last_became_peered":"2023-08-27T11:27:25.074470+0000","last_unstale":"2023-08-27T11:27:27.063967+0000","last_undegraded":"2023-08-27T11:27:27.063967+0000","last_fullsized":"2023-08-27T11:27:27.063967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:12:04.434107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":144,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296353+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:27:27.296353+0000","last_peered":"2023-08-27T11:27:27.296353+0000","last_clean":"2023-08-27T11:27:27.296353+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:27:27.296353+0000","last_undegraded":"2023-08-27T11:27:27.296353+0000","last_fullsized":"2023-08-27T11:27:27.296353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:51.305100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351228+0000","last_change":"2023-08-27T11:27:26.080744+0000","last_active":"2023-08-27T11:27:27.351228+0000","last_peered":"2023-08-27T11:27:27.351228+0000","last_clean":"2023-08-27T11:27:27.351228+0000","last_became_active":"2023-08-27T11:27:26.080434+0000","last_became_peered":"2023-08-27T11:27:26.080434+0000","last_unstale":"2023-08-27T11:27:27.351228+0000","last_undegraded":"2023-08-27T11:27:27.351228+0000","last_fullsized":"2023-08-27T11:27:27.351228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:19:37.219616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296287+0000","last_change":"2023-08-27T11:27:25.067095+0000","last_active":"2023-08-27T11:27:27.296287+0000","last_peered":"2023-08-27T11:27:27.296287+0000","last_clean":"2023-08-27T11:27:27.296287+0000","last_became_active":"2023-08-27T11:27:25.066815+0000","last_became_peered":"2023-08-27T11:27:25.066815+0000","last_unstale":"2023-08-27T11:27:27.296287+0000","last_undegraded":"2023-08-27T11:27:27.296287+0000","last_fullsized":"2023-08-27T11:27:27.296287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:00:37.031105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.059214+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:27:27.059214+0000","last_peered":"2023-08-27T11:27:27.059214+0000","last_clean":"2023-08-27T11:27:27.059214+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:27:27.059214+0000","last_undegraded":"2023-08-27T11:27:27.059214+0000","last_fullsized":"2023-08-27T11:27:27.059214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:41:39.431379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351556+0000","last_change":"2023-08-27T11:27:26.079582+0000","last_active":"2023-08-27T11:27:27.351556+0000","last_peered":"2023-08-27T11:27:27.351556+0000","last_clean":"2023-08-27T11:27:27.351556+0000","last_became_active":"2023-08-27T11:27:26.079270+0000","last_became_peered":"2023-08-27T11:27:26.079270+0000","last_unstale":"2023-08-27T11:27:27.351556+0000","last_undegraded":"2023-08-27T11:27:27.351556+0000","last_fullsized":"2023-08-27T11:27:27.351556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:11:39.646170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063827+0000","last_change":"2023-08-27T11:27:25.071459+0000","last_active":"2023-08-27T11:27:27.063827+0000","last_peered":"2023-08-27T11:27:27.063827+0000","last_clean":"2023-08-27T11:27:27.063827+0000","last_became_active":"2023-08-27T11:27:25.070204+0000","last_became_peered":"2023-08-27T11:27:25.070204+0000","last_unstale":"2023-08-27T11:27:27.063827+0000","last_undegraded":"2023-08-27T11:27:27.063827+0000","last_fullsized":"2023-08-27T11:27:27.063827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:02:36.734554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":144,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.296424+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:27:27.296424+0000","last_peered":"2023-08-27T11:27:27.296424+0000","last_clean":"2023-08-27T11:27:27.296424+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:27:27.296424+0000","last_undegraded":"2023-08-27T11:27:27.296424+0000","last_fullsized":"2023-08-27T11:27:27.296424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:41:36.445248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'161","reported_seq":454,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351869+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:27:27.351869+0000","last_peered":"2023-08-27T11:27:27.351869+0000","last_clean":"2023-08-27T11:27:27.351869+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:27:27.351869+0000","last_undegraded":"2023-08-27T11:27:27.351869+0000","last_fullsized":"2023-08-27T11:27:27.351869+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:00:20.892106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351320+0000","last_change":"2023-08-27T11:27:26.079726+0000","last_active":"2023-08-27T11:27:27.351320+0000","last_peered":"2023-08-27T11:27:27.351320+0000","last_clean":"2023-08-27T11:27:27.351320+0000","last_became_active":"2023-08-27T11:27:26.079252+0000","last_became_peered":"2023-08-27T11:27:26.079252+0000","last_unstale":"2023-08-27T11:27:27.351320+0000","last_undegraded":"2023-08-27T11:27:27.351320+0000","last_fullsized":"2023-08-27T11:27:27.351320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:31.948081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.161267+0000","last_change":"2023-08-27T11:27:25.074495+0000","last_active":"2023-08-27T11:27:27.161267+0000","last_peered":"2023-08-27T11:27:27.161267+0000","last_clean":"2023-08-27T11:27:27.161267+0000","last_became_active":"2023-08-27T11:27:25.074184+0000","last_became_peered":"2023-08-27T11:27:25.074184+0000","last_unstale":"2023-08-27T11:27:27.161267+0000","last_undegraded":"2023-08-27T11:27:27.161267+0000","last_fullsized":"2023-08-27T11:27:27.161267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:30.939823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351698+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:27:27.351698+0000","last_peered":"2023-08-27T11:27:27.351698+0000","last_clean":"2023-08-27T11:27:27.351698+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:27:27.351698+0000","last_undegraded":"2023-08-27T11:27:27.351698+0000","last_fullsized":"2023-08-27T11:27:27.351698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:11.294773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351857+0000","last_change":"2023-08-27T11:27:26.082541+0000","last_active":"2023-08-27T11:27:27.351857+0000","last_peered":"2023-08-27T11:27:27.351857+0000","last_clean":"2023-08-27T11:27:27.351857+0000","last_became_active":"2023-08-27T11:27:26.081375+0000","last_became_peered":"2023-08-27T11:27:26.081375+0000","last_unstale":"2023-08-27T11:27:27.351857+0000","last_undegraded":"2023-08-27T11:27:27.351857+0000","last_fullsized":"2023-08-27T11:27:27.351857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:17:56.457425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063750+0000","last_change":"2023-08-27T11:27:25.077520+0000","last_active":"2023-08-27T11:27:27.063750+0000","last_peered":"2023-08-27T11:27:27.063750+0000","last_clean":"2023-08-27T11:27:27.063750+0000","last_became_active":"2023-08-27T11:27:25.077249+0000","last_became_peered":"2023-08-27T11:27:25.077249+0000","last_unstale":"2023-08-27T11:27:27.063750+0000","last_undegraded":"2023-08-27T11:27:27.063750+0000","last_fullsized":"2023-08-27T11:27:27.063750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:54:46.692552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352247+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:27:27.352247+0000","last_peered":"2023-08-27T11:27:27.352247+0000","last_clean":"2023-08-27T11:27:27.352247+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:27:27.352247+0000","last_undegraded":"2023-08-27T11:27:27.352247+0000","last_fullsized":"2023-08-27T11:27:27.352247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:46:15.407970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.352192+0000","last_change":"2023-08-27T11:27:26.088339+0000","last_active":"2023-08-27T11:27:27.352192+0000","last_peered":"2023-08-27T11:27:27.352192+0000","last_clean":"2023-08-27T11:27:27.352192+0000","last_became_active":"2023-08-27T11:27:26.087691+0000","last_became_peered":"2023-08-27T11:27:26.087691+0000","last_unstale":"2023-08-27T11:27:27.352192+0000","last_undegraded":"2023-08-27T11:27:27.352192+0000","last_fullsized":"2023-08-27T11:27:27.352192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:52:27.133704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"85'2","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064114+0000","last_change":"2023-08-27T11:27:25.078799+0000","last_active":"2023-08-27T11:27:27.064114+0000","last_peered":"2023-08-27T11:27:27.064114+0000","last_clean":"2023-08-27T11:27:27.064114+0000","last_became_active":"2023-08-27T11:27:25.078503+0000","last_became_peered":"2023-08-27T11:27:25.078503+0000","last_unstale":"2023-08-27T11:27:27.064114+0000","last_undegraded":"2023-08-27T11:27:27.064114+0000","last_fullsized":"2023-08-27T11:27:27.064114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:01:09.111047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":146,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.351524+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:27:27.351524+0000","last_peered":"2023-08-27T11:27:27.351524+0000","last_clean":"2023-08-27T11:27:27.351524+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:27:27.351524+0000","last_undegraded":"2023-08-27T11:27:27.351524+0000","last_fullsized":"2023-08-27T11:27:27.351524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:57:40.389214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.063905+0000","last_change":"2023-08-27T11:27:26.071818+0000","last_active":"2023-08-27T11:27:27.063905+0000","last_peered":"2023-08-27T11:27:27.063905+0000","last_clean":"2023-08-27T11:27:27.063905+0000","last_became_active":"2023-08-27T11:27:26.071566+0000","last_became_peered":"2023-08-27T11:27:26.071566+0000","last_unstale":"2023-08-27T11:27:27.063905+0000","last_undegraded":"2023-08-27T11:27:27.063905+0000","last_fullsized":"2023-08-27T11:27:27.063905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:25.043929+0000","last_clean_scrub_stamp":"2023-08-27T11:27:25.043929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:57:10.038438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"85'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064458+0000","last_change":"2023-08-27T11:27:25.074341+0000","last_active":"2023-08-27T11:27:27.064458+0000","last_peered":"2023-08-27T11:27:27.064458+0000","last_clean":"2023-08-27T11:27:27.064458+0000","last_became_active":"2023-08-27T11:27:25.073689+0000","last_became_peered":"2023-08-27T11:27:25.073689+0000","last_unstale":"2023-08-27T11:27:27.064458+0000","last_undegraded":"2023-08-27T11:27:27.064458+0000","last_fullsized":"2023-08-27T11:27:27.064458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:27:24.035671+0000","last_clean_scrub_stamp":"2023-08-27T11:27:24.035671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:59:53.040233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-08-27T11:27:27.064523+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:27:27.064523+0000","last_peered":"2023-08-27T11:27:27.064523+0000","last_clean":"2023-08-27T11:27:27.064523+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:27:27.064523+0000","last_undegraded":"2023-08-27T11:27:27.064523+0000","last_fullsized":"2023-08-27T11:27:27.064523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:12:37.934346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1646592,"data_stored":1135117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.755,"5min":1.601,"15min":1.601},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40899999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.72199999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.38400000000000001,"15min":0.29099999999999998},"max":{"1min":0.85499999999999998,"5min":1.778,"15min":1.778},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63500000000000001,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63600000000000001,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.67400000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.52900000000000003},"min":{"1min":0.432,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68400000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56699999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48299999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.88300000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.501,"15min":0.50600000000000001},"min":{"1min":0.29799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:26:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575035,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":544225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.40799999999999997,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.76700000000000002,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.628},"min":{"1min":0.47399999999999998,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.3540000000000001},"last":0.54100000000000004}]},{"osd":1,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":3.3780000000000001},"last":0.439},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.80700000000000005,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.56499999999999995}]},{"osd":2,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.61699999999999999},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.73399999999999999,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.504},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":1.325},"last":0.39100000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.60499999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.71699999999999997,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.623}]},{"osd":4,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.46400000000000002},"min":{"1min":0.312,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"min":{"1min":0.35199999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.624,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.58499999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":1.157,"15min":1.157},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.70399999999999996,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.55400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575035,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40548,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331292,"statfs":{"total":96636764160,"available":96595243008,"internally_reserved":0,"allocated":1064960,"data_stored":546361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20399999999999999},"max":{"1min":0.52100000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.628,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.40699999999999997}]},{"osd":2,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42699999999999999,"15min":0.42399999999999999},"min":{"1min":0.27500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.34100000000000003,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58999999999999997,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.33300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":1.169,"15min":1.169},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44600000000000001,"15min":0.44},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.23400000000000001},"max":{"1min":0.55900000000000005,"5min":1.139,"15min":1.139},"last":0.27200000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.82599999999999996,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81499999999999995,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.46999999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.45900000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88700000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.437}]},{"osd":7,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89400000000000002,"15min":1.139},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1638400,"data_stored":1134501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.32200000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.34300000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.378}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.498},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.437}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.41799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1056768,"data_stored":545745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.216}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.30599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34100000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.748}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.78400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1052672,"data_stored":544245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40699999999999997,"15min":0.42499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.22},"max":{"1min":0.60799999999999998,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.224},"max":{"1min":0.60699999999999998,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.27000000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.64100000000000001,"15min":6.0830000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46200000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.245},"max":{"1min":0.57399999999999995,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.30299999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.68000000000000005,"5min":0.71199999999999997,"15min":1.25},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":0.84499999999999997,"15min":11.939},"last":0.39700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.376},"max":{"1min":0.83099999999999996,"5min":1.0329999999999999,"15min":11.007},"last":0.502},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":1.113,"15min":1.2929999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93300000000000005,"5min":3.0219999999999998,"15min":11.981},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":1.218,"15min":11.09},"last":0.57699999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69099999999999995,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.75600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":1.0269999999999999,"15min":12.016},"last":0.55200000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68999999999999995,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":0.96699999999999997,"15min":11.054},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.53500000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84099999999999997,"5min":1.0640000000000001,"15min":1.361},"last":0.53500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575035,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":544203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.68400000000000005}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.71799999999999997}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.746},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.5},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.53700000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.70299999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575034,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335468,"statfs":{"total":96636764160,"available":96599519232,"internally_reserved":0,"allocated":1048576,"data_stored":544223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.59199999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.52700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.437}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.623}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:27:35.285 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:27:35.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:27:35.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.448+0000 7f912b5b2700 1 Processor -- start 2023-08-27T11:27:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.448+0000 7f912b5b2700 1 -- start start 2023-08-27T11:27:35.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f912b5b2700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f912406c260 0x7f91240cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f912b5b2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91240cdd40 0x7f912415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f912b5b2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91240d04f0 con 0x7f91240b02c0 2023-08-27T11:27:35.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f912b5b2700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91240ce1d0 con 0x7f912406c260 2023-08-27T11:27:35.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f912b5b2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91240b06a0 con 0x7f91240cdd40 2023-08-27T11:27:35.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f912934e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f91240b02c0 0x7f91240cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44644/0 (socket says 172.21.15.45:44644) 2023-08-27T11:27:35.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f912934e700 1 -- 172.21.15.45:0/917154811 learned_addr learned my addr 172.21.15.45:0/917154811 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:35.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f9129b4f700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91240cdd40 0x7f912415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f9128b4d700 1 --2- 172.21.15.45:0/917154811 >> v2:172.21.15.135:3300/0 conn(0x7f912406c260 0x7f91240cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708927914 0 0) 0x7f91240d04f0 con 0x7f91240b02c0 2023-08-27T11:27:35.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.449+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91100035b0 con 0x7f91240b02c0 2023-08-27T11:27:35.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1930779121 0 0) 0x7f91100035b0 con 0x7f91240b02c0 2023-08-27T11:27:35.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91240cdd40 msgr2=0x7f912415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91240cdd40 0x7f912415ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 >> v2:172.21.15.135:3300/0 conn(0x7f912406c260 msgr2=0x7f91240cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 --2- 172.21.15.45:0/917154811 >> v2:172.21.15.135:3300/0 conn(0x7f912406c260 0x7f91240cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91240cf8e0 con 0x7f91240b02c0 2023-08-27T11:27:35.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f9114001670 con 0x7f91240b02c0 2023-08-27T11:27:35.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 >> v1:172.21.15.45:6789/0 conn(0x7f91240b02c0 legacy=0x7f91240cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f9110003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91240cdd40 0x7f9110004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9110004d90 0x7f9110005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240ce1d0 con 0x7f9110003a00 2023-08-27T11:27:35.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240b06a0 con 0x7f9110004d90 2023-08-27T11:27:35.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911000ba10 con 0x7f91240cdd40 2023-08-27T11:27:35.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f912934e700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f9110003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f9128b4d700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91240cdd40 0x7f9110004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f9129b4f700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9110004d90 0x7f9110005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f912934e700 1 -- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91240cdd40 msgr2=0x7f9110004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f912934e700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91240cdd40 0x7f9110004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f912934e700 1 -- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9110004d90 msgr2=0x7f9110005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f912934e700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9110004d90 0x7f9110005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f912934e700 1 -- 172.21.15.45:0/917154811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100035b0 con 0x7f9110003a00 2023-08-27T11:27:35.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f9129b4f700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9110004d90 0x7f9110005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:35.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f912934e700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f9110003e20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f911400c3a0 tx=0x7f9110005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f6355a847b6df5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114007c60 con 0x7f9110003a00 2023-08-27T11:27:35.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114005960 con 0x7f9110003a00 2023-08-27T11:27:35.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.450+0000 7f911bfff700 1 -- 172.21.15.45:0/917154811 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400cea0 con 0x7f9110003a00 2023-08-27T11:27:35.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.451+0000 7f912b5b2700 1 -- 172.21.15.45:0/917154811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 msgr2=0x7f9110003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.451+0000 7f912b5b2700 1 --2- 172.21.15.45:0/917154811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f9110003e20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f911400c3a0 tx=0x7f9110005880 comp rx=0 tx=0).stop 2023-08-27T11:27:35.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.451+0000 7f9129b4f700 1 -- 172.21.15.45:0/917154811 reap_dead start 2023-08-27T11:27:35.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.451+0000 7f912b5b2700 1 -- 172.21.15.45:0/917154811 shutdown_connections 2023-08-27T11:27:35.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.451+0000 7f912b5b2700 1 -- 172.21.15.45:0/917154811 >> 172.21.15.45:0/917154811 conn(0x7f912409a1a0 msgr2=0x7f91240d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:35.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.451+0000 7f912b5b2700 1 -- 172.21.15.45:0/917154811 shutdown_connections 2023-08-27T11:27:35.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.451+0000 7f912b5b2700 1 -- 172.21.15.45:0/917154811 wait complete. 2023-08-27T11:27:35.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.452+0000 7f912b5b2700 1 Processor -- start 2023-08-27T11:27:35.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.452+0000 7f912b5b2700 1 -- start start 2023-08-27T11:27:35.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912b5b2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f912415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912b5b2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91241531a0 0x7f912409d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912b5b2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91240cc770 0x7f91240ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912b5b2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240cd140 con 0x7f9110003a00 2023-08-27T11:27:35.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912b5b2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912409c3e0 con 0x7f91240cc770 2023-08-27T11:27:35.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912b5b2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912409c6c0 con 0x7f91241531a0 2023-08-27T11:27:35.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912934e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f912415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912934e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f912415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57880/0 (socket says 172.21.15.45:57880) 2023-08-27T11:27:35.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912934e700 1 -- 172.21.15.45:0/2168477678 learned_addr learned my addr 172.21.15.45:0/2168477678 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:35.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f9129b4f700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91240cc770 0x7f91240ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f9128b4d700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91241531a0 0x7f912409d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.453+0000 7f912934e700 1 -- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91241531a0 msgr2=0x7f912409d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f912934e700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91241531a0 0x7f912409d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f912934e700 1 -- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91240cc770 msgr2=0x7f91240ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f912934e700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91240cc770 0x7f91240ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f912934e700 1 -- 172.21.15.45:0/2168477678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912409c9a0 con 0x7f9110003a00 2023-08-27T11:27:35.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f9129b4f700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91240cc770 0x7f91240ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:35.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f9128b4d700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91241531a0 0x7f912409d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:35.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f912934e700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f912415f3c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f911400c990 tx=0x7f9114000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f9119ffb700 1 -- 172.21.15.45:0/2168477678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114004870 con 0x7f9110003a00 2023-08-27T11:27:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f9119ffb700 1 -- 172.21.15.45:0/2168477678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91140049f0 con 0x7f9110003a00 2023-08-27T11:27:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912409cc80 con 0x7f9110003a00 2023-08-27T11:27:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f9119ffb700 1 -- 172.21.15.45:0/2168477678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114004da0 con 0x7f9110003a00 2023-08-27T11:27:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.454+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912409d9c0 con 0x7f9110003a00 2023-08-27T11:27:35.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.455+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912409eda0 con 0x7f9110003a00 2023-08-27T11:27:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.456+0000 7f9119ffb700 1 -- 172.21.15.45:0/2168477678 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f91140070d0 con 0x7f9110003a00 2023-08-27T11:27:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.456+0000 7f9119ffb700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f910806a200 0x7f910806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.456+0000 7f9119ffb700 1 -- 172.21.15.45:0/2168477678 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9114089f90 con 0x7f9110003a00 2023-08-27T11:27:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.456+0000 7f9128b4d700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f910806a200 0x7f910806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.457+0000 7f9128b4d700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f910806a200 0x7f910806c6c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9120009c00 tx=0x7f9120008040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.460+0000 7f9119ffb700 1 -- 172.21.15.45:0/2168477678 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91140bc990 con 0x7f9110003a00 2023-08-27T11:27:35.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.746+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f91240734a0 con 0x7f9110003a00 2023-08-27T11:27:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.750+0000 7f9119ffb700 1 -- 172.21.15.45:0/2168477678 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f911405e0b0 con 0x7f9110003a00 2023-08-27T11:27:35.751 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-08-27T11:27:35.751 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ== 2023-08-27T11:27:35.752 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2023-08-27T11:27:35.752 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2023-08-27T11:27:35.752 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:27:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f910806a200 msgr2=0x7f910806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f910806a200 0x7f910806c6c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9120009c00 tx=0x7f9120008040 comp rx=0 tx=0).stop 2023-08-27T11:27:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 msgr2=0x7f912415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f912415f3c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f911400c990 tx=0x7f9114000fe0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 shutdown_connections 2023-08-27T11:27:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f910806a200 0x7f910806c6c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9110003a00 0x7f912415f3c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91240cc770 0x7f91240ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 --2- 172.21.15.45:0/2168477678 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f91241531a0 0x7f912409d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 >> 172.21.15.45:0/2168477678 conn(0x7f912409a1a0 msgr2=0x7f91240d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 shutdown_connections 2023-08-27T11:27:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.752+0000 7f912b5b2700 1 -- 172.21.15.45:0/2168477678 wait complete. 2023-08-27T11:27:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-08-27T11:27:35.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:35.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.890+0000 7f9ff1e9d700 1 Processor -- start 2023-08-27T11:27:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.890+0000 7f9ff1e9d700 1 -- start start 2023-08-27T11:27:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.890+0000 7f9ff1e9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9fec0cc790 0x7f9fec0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.890+0000 7f9ff1e9d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec0cff50 con 0x7f9fec06bda0 2023-08-27T11:27:35.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.890+0000 7f9ff1e9d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec0cc4e0 con 0x7f9fec15d140 2023-08-27T11:27:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9ff1e9d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fec0ce820 con 0x7f9fec0cc790 2023-08-27T11:27:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9febfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9fec0cc790 0x7f9fec0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9feaffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9fec06bda0 0x7f9fec06c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44664/0 (socket says 172.21.15.45:44664) 2023-08-27T11:27:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9feaffd700 1 -- 172.21.15.45:0/248447848 learned_addr learned my addr 172.21.15.45:0/248447848 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9febfff700 1 -- 172.21.15.45:0/248447848 >> v1:172.21.15.135:6789/0 conn(0x7f9fec15d140 legacy=0x7f9fec0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9febfff700 1 -- 172.21.15.45:0/248447848 >> v1:172.21.15.45:6789/0 conn(0x7f9fec06bda0 legacy=0x7f9fec06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9fea7fc700 1 -- 172.21.15.45:0/248447848 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586300458 0 0) 0x7f9fec0cff50 con 0x7f9fec06bda0 2023-08-27T11:27:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9febfff700 1 -- 172.21.15.45:0/248447848 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec0cf0a0 con 0x7f9fec0cc790 2023-08-27T11:27:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9febfff700 1 --2- 172.21.15.45:0/248447848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0cc790 0x7f9fec0ccc20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009d90 tx=0x7f9fe400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a11dbc3cca0460e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9fea7fc700 1 -- 172.21.15.45:0/248447848 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe400e070 con 0x7f9fec0cc790 2023-08-27T11:27:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9fea7fc700 1 -- 172.21.15.45:0/248447848 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe400bb90 con 0x7f9fec0cc790 2023-08-27T11:27:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.891+0000 7f9fea7fc700 1 -- 172.21.15.45:0/248447848 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4003a20 con 0x7f9fec0cc790 2023-08-27T11:27:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.892+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/248447848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0cc790 msgr2=0x7f9fec0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.892+0000 7f9ff1e9d700 1 --2- 172.21.15.45:0/248447848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0cc790 0x7f9fec0ccc20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009d90 tx=0x7f9fe400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.892+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/248447848 shutdown_connections 2023-08-27T11:27:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.892+0000 7f9ff1e9d700 1 --2- 172.21.15.45:0/248447848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec0cc790 0x7f9fec0ccc20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.892+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/248447848 >> 172.21.15.45:0/248447848 conn(0x7f9fec09a160 msgr2=0x7f9fec0d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.892+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/248447848 shutdown_connections 2023-08-27T11:27:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.892+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/248447848 wait complete. 2023-08-27T11:27:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.893+0000 7f9ff1e9d700 1 Processor -- start 2023-08-27T11:27:35.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.893+0000 7f9ff1e9d700 1 -- start start 2023-08-27T11:27:35.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9ff1e9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec06bda0 0x7f9fec153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9ff1e9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fec0cc790 0x7f9fec155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9ff1e9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec15d140 0x7f9fec152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9ff1e9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec154420 con 0x7f9fec06bda0 2023-08-27T11:27:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9ff1e9d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec156420 con 0x7f9fec15d140 2023-08-27T11:27:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9ff1e9d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec153810 con 0x7f9fec0cc790 2023-08-27T11:27:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9feb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec06bda0 0x7f9fec153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9feb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec06bda0 0x7f9fec153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57894/0 (socket says 172.21.15.45:57894) 2023-08-27T11:27:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9feb7fe700 1 -- 172.21.15.45:0/3561017805 learned_addr learned my addr 172.21.15.45:0/3561017805 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9febfff700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec15d140 0x7f9fec152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.894+0000 7f9feaffd700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fec0cc790 0x7f9fec155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9feb7fe700 1 -- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fec0cc790 msgr2=0x7f9fec155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9feb7fe700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fec0cc790 0x7f9fec155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9feb7fe700 1 -- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec15d140 msgr2=0x7f9fec152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9feb7fe700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec15d140 0x7f9fec152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9feb7fe700 1 -- 172.21.15.45:0/3561017805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe4009a70 con 0x7f9fec06bda0 2023-08-27T11:27:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9febfff700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec15d140 0x7f9fec152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:35.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9feb7fe700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec06bda0 0x7f9fec153110 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c4d0 tx=0x7f9fdc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9fd3fff700 1 -- 172.21.15.45:0/3561017805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011790 con 0x7f9fec06bda0 2023-08-27T11:27:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9fd3fff700 1 -- 172.21.15.45:0/3561017805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc0118f0 con 0x7f9fec06bda0 2023-08-27T11:27:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9fd3fff700 1 -- 172.21.15.45:0/3561017805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011ca0 con 0x7f9fec06bda0 2023-08-27T11:27:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec153af0 con 0x7f9fec06bda0 2023-08-27T11:27:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.895+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec1540b0 con 0x7f9fec06bda0 2023-08-27T11:27:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.896+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fec154700 con 0x7f9fec06bda0 2023-08-27T11:27:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.897+0000 7f9fd3fff700 1 -- 172.21.15.45:0/3561017805 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9fdc01c460 con 0x7f9fec06bda0 2023-08-27T11:27:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.897+0000 7f9fd3fff700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fd806a200 0x7f9fd806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.897+0000 7f9fd3fff700 1 -- 172.21.15.45:0/3561017805 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9fdc088620 con 0x7f9fec06bda0 2023-08-27T11:27:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.897+0000 7f9feaffd700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fd806a200 0x7f9fd806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:35.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.898+0000 7f9feaffd700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fd806a200 0x7f9fd806c6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9fec159aa0 tx=0x7f9fe0007c90 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:35.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:35.901+0000 7f9fd3fff700 1 -- 172.21.15.45:0/3561017805 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9fdc053b20 con 0x7f9fec06bda0 2023-08-27T11:27:36.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.144+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fec154270 con 0x7f9fec06bda0 2023-08-27T11:27:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.145+0000 7f9fd3fff700 1 -- 172.21.15.45:0/3561017805 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f9fdc057140 con 0x7f9fec06bda0 2023-08-27T11:27:36.146 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:36.146 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2178366980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2178366980},{"type":"v1","addr":"172.21.15.135:6836","nonce":2178366980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2939076938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2939076938},{"type":"v1","addr":"172.21.15.45:6833","nonce":2939076938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/148627485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":148627485},{"type":"v1","addr":"172.21.15.45:6835","nonce":148627485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:25.479170+0000","modified":"2023-08-27T11:27:29.096440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1457694749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1457694749},{"type":"v1","addr":"172.21.15.135:6837","nonce":1457694749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:27:36.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fd806a200 msgr2=0x7f9fd806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fd806a200 0x7f9fd806c6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9fec159aa0 tx=0x7f9fe0007c90 comp rx=0 tx=0).stop 2023-08-27T11:27:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec06bda0 msgr2=0x7f9fec153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec06bda0 0x7f9fec153110 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c4d0 tx=0x7f9fdc00c890 comp rx=0 tx=0).stop 2023-08-27T11:27:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 shutdown_connections 2023-08-27T11:27:36.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f9fd806a200 0x7f9fd806c6c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fec06bda0 0x7f9fec153110 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fec15d140 0x7f9fec152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 --2- 172.21.15.45:0/3561017805 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9fec0cc790 0x7f9fec155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 >> 172.21.15.45:0/3561017805 conn(0x7f9fec09a160 msgr2=0x7f9fec0c8690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 shutdown_connections 2023-08-27T11:27:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.147+0000 7f9ff1e9d700 1 -- 172.21.15.45:0/3561017805 wait complete. 2023-08-27T11:27:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2023-08-27T11:27:36.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:27:25.479170+0000', 'modified': '2023-08-27T11:27:29.096440+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6885}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6885': {'gid': 6885, 'name': 'b', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6837/1457694749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6835', 'nonce': 1457694749}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 1457694749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:27:36.564 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:27:36.564 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-27T11:27:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.697+0000 7f010748b700 1 Processor -- start 2023-08-27T11:27:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.697+0000 7f010748b700 1 -- start start 2023-08-27T11:27:36.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.698+0000 7f010748b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 0x7f01000cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.698+0000 7f010748b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01000caa00 0x7f01000acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.698+0000 7f010748b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01000ce340 con 0x7f01000ca620 2023-08-27T11:27:36.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.698+0000 7f010748b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01000ceec0 con 0x7f01000caa00 2023-08-27T11:27:36.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.698+0000 7f010748b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010015d7c0 con 0x7f01000af200 2023-08-27T11:27:36.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.698+0000 7f0105a28700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01000caa00 0x7f01000acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.698+0000 7f0105a28700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01000caa00 0x7f01000acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57906/0 (socket says 172.21.15.45:57906) 2023-08-27T11:27:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.698+0000 7f0105a28700 1 -- 172.21.15.45:0/1740320590 learned_addr learned my addr 172.21.15.45:0/1740320590 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.698+0000 7f0104a26700 1 --2- 172.21.15.45:0/1740320590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 0x7f01000cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f0105a28700 1 -- 172.21.15.45:0/1740320590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 msgr2=0x7f01000cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f0105a28700 1 --2- 172.21.15.45:0/1740320590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 0x7f01000cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f0105a28700 1 -- 172.21.15.45:0/1740320590 >> v1:172.21.15.135:6789/0 conn(0x7f01000ca620 legacy=0x7f01000ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f0105a28700 1 -- 172.21.15.45:0/1740320590 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000cc570 con 0x7f01000caa00 2023-08-27T11:27:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f0104a26700 1 --2- 172.21.15.45:0/1740320590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 0x7f01000cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f0105a28700 1 --2- 172.21.15.45:0/1740320590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000caa00 0x7f01000acfc0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009e50 tx=0x7f00f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523e93324b6abf0f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f00f7fff700 1 -- 172.21.15.45:0/1740320590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800e070 con 0x7f01000caa00 2023-08-27T11:27:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f00f7fff700 1 -- 172.21.15.45:0/1740320590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f800b9c0 con 0x7f01000caa00 2023-08-27T11:27:36.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f00f7fff700 1 -- 172.21.15.45:0/1740320590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800bd70 con 0x7f01000caa00 2023-08-27T11:27:36.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f010748b700 1 -- 172.21.15.45:0/1740320590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000caa00 msgr2=0x7f01000acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f010748b700 1 --2- 172.21.15.45:0/1740320590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000caa00 0x7f01000acfc0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009e50 tx=0x7f00f800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:36.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.699+0000 7f010748b700 1 -- 172.21.15.45:0/1740320590 shutdown_connections 2023-08-27T11:27:36.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.700+0000 7f010748b700 1 --2- 172.21.15.45:0/1740320590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000caa00 0x7f01000acfc0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.700+0000 7f010748b700 1 --2- 172.21.15.45:0/1740320590 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 0x7f01000cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.700+0000 7f010748b700 1 -- 172.21.15.45:0/1740320590 >> 172.21.15.45:0/1740320590 conn(0x7f01000cc7e0 msgr2=0x7f01000ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:36.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.700+0000 7f010748b700 1 -- 172.21.15.45:0/1740320590 shutdown_connections 2023-08-27T11:27:36.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.700+0000 7f010748b700 1 -- 172.21.15.45:0/1740320590 wait complete. 2023-08-27T11:27:36.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.701+0000 7f010748b700 1 Processor -- start 2023-08-27T11:27:36.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.701+0000 7f010748b700 1 -- start start 2023-08-27T11:27:36.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f010748b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 0x7f01000c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f010748b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000ca620 0x7f01000c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f010748b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01000caa00 0x7f010015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f010748b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c8010 con 0x7f01000ca620 2023-08-27T11:27:36.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f010748b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c7d30 con 0x7f01000caa00 2023-08-27T11:27:36.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f010748b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c50f0 con 0x7f01000af200 2023-08-27T11:27:36.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0104a26700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000ca620 0x7f01000c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0104a26700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000ca620 0x7f01000c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57910/0 (socket says 172.21.15.45:57910) 2023-08-27T11:27:36.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0104a26700 1 -- 172.21.15.45:0/3475465369 learned_addr learned my addr 172.21.15.45:0/3475465369 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:36.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0105a28700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01000caa00 0x7f010015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0105227700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 0x7f01000c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0104a26700 1 -- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 msgr2=0x7f01000c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0104a26700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 0x7f01000c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0104a26700 1 -- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01000caa00 msgr2=0x7f010015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0104a26700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01000caa00 0x7f010015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0104a26700 1 -- 172.21.15.45:0/3475465369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f8009b30 con 0x7f01000ca620 2023-08-27T11:27:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0105a28700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01000caa00 0x7f010015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f0104a26700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000ca620 0x7f01000c97b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00e810 tx=0x7f00fc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f00f5ffb700 1 -- 172.21.15.45:0/3475465369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc013610 con 0x7f01000ca620 2023-08-27T11:27:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f00f5ffb700 1 -- 172.21.15.45:0/3475465369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc013770 con 0x7f01000ca620 2023-08-27T11:27:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f010748b700 1 -- 172.21.15.45:0/3475465369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01000c53d0 con 0x7f01000ca620 2023-08-27T11:27:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.702+0000 7f00f5ffb700 1 -- 172.21.15.45:0/3475465369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc013b20 con 0x7f01000ca620 2023-08-27T11:27:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.703+0000 7f010748b700 1 -- 172.21.15.45:0/3475465369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01000c7370 con 0x7f01000ca620 2023-08-27T11:27:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.705+0000 7f00f5ffb700 1 -- 172.21.15.45:0/3475465369 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f00fc013c80 con 0x7f01000ca620 2023-08-27T11:27:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.705+0000 7f00f5ffb700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f00e406a200 0x7f00e406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.705+0000 7f00f5ffb700 1 -- 172.21.15.45:0/3475465369 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f00fc088590 con 0x7f01000ca620 2023-08-27T11:27:36.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.705+0000 7f0105227700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f00e406a200 0x7f00e406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.706+0000 7f0105227700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f00e406a200 0x7f00e406c6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01000b5bd0 tx=0x7f00f0009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.706+0000 7f011364db80 1 Processor -- start 2023-08-27T11:27:36.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.706+0000 7f011364db80 1 -- start start 2023-08-27T11:27:36.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f011364db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5110320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f011364db80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c510cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f011364db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 0x55b0c510d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f011364db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c503e730 con 0x55b0c510ff40 2023-08-27T11:27:36.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f011364db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c503c050 con 0x55b0c50e33d0 2023-08-27T11:27:36.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f011364db80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0c5037940 con 0x55b0c5121960 2023-08-27T11:27:36.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f0105227700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5110320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f0105227700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5110320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57916/0 (socket says 172.21.15.45:57916) 2023-08-27T11:27:36.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f0105227700 1 -- 172.21.15.45:0/1921543864 learned_addr learned my addr 172.21.15.45:0/1921543864 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:36.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f0104a26700 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c510cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f0105a28700 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 0x55b0c510d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f0105227700 1 -- 172.21.15.45:0/1921543864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 msgr2=0x55b0c510cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f0105227700 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c510cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f0105227700 1 -- 172.21.15.45:0/1921543864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 msgr2=0x55b0c510d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.707+0000 7f0105227700 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 0x55b0c510d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f0105227700 1 -- 172.21.15.45:0/1921543864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0c4eb75e0 con 0x55b0c510ff40 2023-08-27T11:27:36.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f0105a28700 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 0x55b0c510d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:36.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f0105227700 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5110320 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f00f00053b0 tx=0x7f00f0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b7c7f93a78d04 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f00eaffd700 1 -- 172.21.15.45:0/1921543864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0003d20 con 0x55b0c510ff40 2023-08-27T11:27:36.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f00eaffd700 1 -- 172.21.15.45:0/1921543864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f0003e80 con 0x55b0c510ff40 2023-08-27T11:27:36.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f00eaffd700 1 -- 172.21.15.45:0/1921543864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0021650 con 0x55b0c510ff40 2023-08-27T11:27:36.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f011364db80 1 -- 172.21.15.45:0/1921543864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 msgr2=0x55b0c5110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f011364db80 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5110320 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f00f00053b0 tx=0x7f00f0011000 comp rx=0 tx=0).stop 2023-08-27T11:27:36.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f011364db80 1 -- 172.21.15.45:0/1921543864 shutdown_connections 2023-08-27T11:27:36.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f011364db80 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5110320 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f011364db80 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 0x55b0c510d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f011364db80 1 --2- 172.21.15.45:0/1921543864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c510cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f011364db80 1 -- 172.21.15.45:0/1921543864 >> 172.21.15.45:0/1921543864 conn(0x55b0c5043c30 msgr2=0x55b0c512ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:36.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f011364db80 1 -- 172.21.15.45:0/1921543864 shutdown_connections 2023-08-27T11:27:36.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.708+0000 7f011364db80 1 -- 172.21.15.45:0/1921543864 wait complete. 2023-08-27T11:27:36.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.709+0000 7f011364db80 1 -- start start 2023-08-27T11:27:36.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.709+0000 7f011364db80 1 Processor -- start 2023-08-27T11:27:36.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f011364db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c50e33d0 0x55b0c5113b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f011364db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 0x55b0c510dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f011364db80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c512c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f011364db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c4fd5b60 con 0x55b0c50e33d0 2023-08-27T11:27:36.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f011364db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c4fd5860 con 0x55b0c510ff40 2023-08-27T11:27:36.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f011364db80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0c4eb98d0 con 0x55b0c5121960 2023-08-27T11:27:36.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f011364db80 10 client.0 _ll_register_callbacks cb 0x55b0c4fd3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:27:36.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f0105227700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c50e33d0 0x55b0c5113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f0105227700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c50e33d0 0x55b0c5113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57922/0 (socket says 172.21.15.45:57922) 2023-08-27T11:27:36.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f0105227700 1 -- 172.21.15.45:0/3632645767 learned_addr learned my addr 172.21.15.45:0/3632645767 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:36.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f0104a26700 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 0x55b0c510dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f0105a28700 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c512c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f0105227700 1 -- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 msgr2=0x55b0c512c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f0105227700 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c512c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.710+0000 7f0105227700 1 -- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 msgr2=0x55b0c510dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f0105227700 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 0x55b0c510dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f0105227700 1 -- 172.21.15.45:0/3632645767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f000f040 con 0x55b0c50e33d0 2023-08-27T11:27:36.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f0104a26700 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 0x55b0c510dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:36.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f0105227700 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c50e33d0 0x55b0c5113b70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f00f0003fd0 tx=0x7f00f0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e91d2c935391e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f00dffff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:27:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f00dffff700 1 -- 172.21.15.45:0/3632645767 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0021970 con 0x55b0c50e33d0 2023-08-27T11:27:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f00dffff700 1 -- 172.21.15.45:0/3632645767 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f00f0021ad0 con 0x55b0c50e33d0 2023-08-27T11:27:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f011364db80 1 -- 172.21.15.45:0/3632645767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0c5030920 con 0x55b0c50e33d0 2023-08-27T11:27:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f00dffff700 1 -- 172.21.15.45:0/3632645767 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f0021cd0 con 0x55b0c50e33d0 2023-08-27T11:27:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f011364db80 10 client.7065 fetch_fsmap learned FSMap version 34 2023-08-27T11:27:36.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f011364db80 1 -- 172.21.15.45:0/3632645767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55b0c504f750 con 0x55b0c50e33d0 2023-08-27T11:27:36.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f00dffff700 1 -- 172.21.15.45:0/3632645767 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0c5030920 con 0x55b0c50e33d0 2023-08-27T11:27:36.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f00dffff700 1 -- 172.21.15.45:0/3632645767 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f00f002a030 con 0x55b0c50e33d0 2023-08-27T11:27:36.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f011364db80 10 client.7065 fetch_fsmap finished waiting for FSMap version 34 2023-08-27T11:27:36.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f011364db80 10 client.7065 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-27T11:27:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f011364db80 20 client.7065 populate_metadata read hostname 'smithi045' 2023-08-27T11:27:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f011364db80 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] conn(0x55b0c51113f0 0x55b0c51117d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f011364db80 4 client.7065 mds_command: new command op to 6885 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:27:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.711+0000 7f011364db80 1 -- 172.21.15.45:0/3632645767 --> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0c4ff1000 con 0x55b0c51113f0 2023-08-27T11:27:36.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.712+0000 7f0104a26700 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] conn(0x55b0c51113f0 0x55b0c51117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.712+0000 7f00dffff700 10 client.7065 ms_handle_connect on v2:172.21.15.135:6835/1457694749 2023-08-27T11:27:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.712+0000 7f0104a26700 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] conn(0x55b0c51113f0 0x55b0c51117d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548898d855b45564 server_cookie=339772a3e971c7a0 in_seq=0 out_seq=0 2023-08-27T11:27:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f00dffff700 1 -- 172.21.15.45:0/3632645767 <== mds.0 v2:172.21.15.135:6835/1457694749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b0c4ff1000 con 0x55b0c51113f0 2023-08-27T11:27:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f00dffff700 10 client.7065 handle_command_reply: tid=0 2023-08-27T11:27:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 client.7065 shutdown 2023-08-27T11:27:36.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 -- 172.21.15.45:0/3632645767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c50e33d0 msgr2=0x55b0c5113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c50e33d0 0x55b0c5113b70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f00f0003fd0 tx=0x7f00f0000ed0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 -- 172.21.15.45:0/3632645767 shutdown_connections 2023-08-27T11:27:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] conn(0x55b0c51113f0 0x55b0c51117d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c50e33d0 0x55b0c5113b70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c512c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f00dffff700 0 client.7065 ms_handle_reset on v2:172.21.15.135:6835/1457694749 2023-08-27T11:27:36.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 --2- 172.21.15.45:0/3632645767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 0x55b0c510dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 -- 172.21.15.45:0/3632645767 >> 172.21.15.45:0/3632645767 conn(0x55b0c5043c30 msgr2=0x55b0c5109780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:36.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 -- 172.21.15.45:0/3632645767 shutdown_connections 2023-08-27T11:27:36.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 1 -- 172.21.15.45:0/3632645767 wait complete. 2023-08-27T11:27:36.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.713+0000 7f011364db80 20 client.7065 trim_cache size 0 max 16384 2023-08-27T11:27:36.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.731+0000 7f011364db80 1 Processor -- start 2023-08-27T11:27:36.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.731+0000 7f011364db80 1 -- start start 2023-08-27T11:27:36.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f011364db80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c50e33d0 0x55b0c5143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f011364db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 0x55b0c50dc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f011364db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c5121960 0x55b0c512c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f011364db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c50e2fa0 con 0x55b0c5121960 2023-08-27T11:27:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f011364db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c5109560 con 0x55b0c510ff40 2023-08-27T11:27:36.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f011364db80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0c50e47c0 con 0x55b0c50e33d0 2023-08-27T11:27:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105a28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c5121960 0x55b0c512c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105a28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c5121960 0x55b0c512c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57934/0 (socket says 172.21.15.45:57934) 2023-08-27T11:27:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105a28700 1 -- 172.21.15.45:0/3995132985 learned_addr learned my addr 172.21.15.45:0/3995132985 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105227700 1 --2- 172.21.15.45:0/3995132985 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c50e33d0 0x55b0c5143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0104a26700 1 --2- 172.21.15.45:0/3995132985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 0x55b0c50dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105a28700 1 -- 172.21.15.45:0/3995132985 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c50e33d0 msgr2=0x55b0c5143cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105a28700 1 --2- 172.21.15.45:0/3995132985 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c50e33d0 0x55b0c5143cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105a28700 1 -- 172.21.15.45:0/3995132985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 msgr2=0x55b0c50dc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105a28700 1 --2- 172.21.15.45:0/3995132985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 0x55b0c50dc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105a28700 1 -- 172.21.15.45:0/3995132985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0c4e63d90 con 0x55b0c5121960 2023-08-27T11:27:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f0105a28700 1 --2- 172.21.15.45:0/3995132985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c5121960 0x55b0c512c3c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0100074300 tx=0x7f00f8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7920a89239b57657 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f00eb7fe700 1 -- 172.21.15.45:0/3995132985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8003a20 con 0x55b0c5121960 2023-08-27T11:27:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f00eb7fe700 1 -- 172.21.15.45:0/3995132985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f8003b80 con 0x55b0c5121960 2023-08-27T11:27:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.732+0000 7f00eb7fe700 1 -- 172.21.15.45:0/3995132985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8004040 con 0x55b0c5121960 2023-08-27T11:27:36.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 -- 172.21.15.45:0/3995132985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c5121960 msgr2=0x55b0c512c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 --2- 172.21.15.45:0/3995132985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c5121960 0x55b0c512c3c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0100074300 tx=0x7f00f8010000 comp rx=0 tx=0).stop 2023-08-27T11:27:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 -- 172.21.15.45:0/3995132985 shutdown_connections 2023-08-27T11:27:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 --2- 172.21.15.45:0/3995132985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c5121960 0x55b0c512c3c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 --2- 172.21.15.45:0/3995132985 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c50e33d0 0x55b0c5143cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 --2- 172.21.15.45:0/3995132985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c510ff40 0x55b0c50dc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 -- 172.21.15.45:0/3995132985 >> 172.21.15.45:0/3995132985 conn(0x55b0c5043c30 msgr2=0x55b0c50e6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 -- 172.21.15.45:0/3995132985 shutdown_connections 2023-08-27T11:27:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 -- 172.21.15.45:0/3995132985 wait complete. 2023-08-27T11:27:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.733+0000 7f011364db80 1 -- start start 2023-08-27T11:27:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.734+0000 7f011364db80 1 Processor -- start 2023-08-27T11:27:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.734+0000 7f011364db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 0x55b0c5176f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.734+0000 7f011364db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5195400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.734+0000 7f011364db80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c5191480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.734+0000 7f011364db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c51920f0 con 0x55b0c510ff40 2023-08-27T11:27:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.734+0000 7f011364db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0c512ab50 con 0x55b0c50e33d0 2023-08-27T11:27:36.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.734+0000 7f011364db80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0c51774d0 con 0x55b0c5121960 2023-08-27T11:27:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.734+0000 7f011364db80 10 client.0 _ll_register_callbacks cb 0x55b0c4fd3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:27:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0104a26700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0104a26700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5195400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57938/0 (socket says 172.21.15.45:57938) 2023-08-27T11:27:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0104a26700 1 -- 172.21.15.45:0/163453328 learned_addr learned my addr 172.21.15.45:0/163453328 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:36.748 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0105a28700 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c5191480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0105227700 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 0x55b0c5176f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0104a26700 1 -- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 msgr2=0x55b0c5191480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0104a26700 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c5191480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0104a26700 1 -- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 msgr2=0x55b0c5176f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0104a26700 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 0x55b0c5176f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0104a26700 1 -- 172.21.15.45:0/163453328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01000cc570 con 0x55b0c510ff40 2023-08-27T11:27:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f0104a26700 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5195400 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00efd0 tx=0x7f00fc088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a82e406c485ec48 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f00dffff700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:27:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f00dffff700 1 -- 172.21.15.45:0/163453328 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc056ca0 con 0x55b0c510ff40 2023-08-27T11:27:36.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f00dffff700 1 -- 172.21.15.45:0/163453328 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f00fc0862c0 con 0x55b0c510ff40 2023-08-27T11:27:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f011364db80 1 -- 172.21.15.45:0/163453328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0c50d84f0 con 0x55b0c510ff40 2023-08-27T11:27:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f00dffff700 1 -- 172.21.15.45:0/163453328 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00fc05ee30 con 0x55b0c510ff40 2023-08-27T11:27:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f011364db80 10 client.7071 fetch_fsmap learned FSMap version 34 2023-08-27T11:27:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f011364db80 1 -- 172.21.15.45:0/163453328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55b0c4e4cc10 con 0x55b0c510ff40 2023-08-27T11:27:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.735+0000 7f00dffff700 1 -- 172.21.15.45:0/163453328 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0c50d84f0 con 0x55b0c510ff40 2023-08-27T11:27:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f00dffff700 1 -- 172.21.15.45:0/163453328 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f00fc06ab70 con 0x55b0c510ff40 2023-08-27T11:27:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f011364db80 10 client.7071 fetch_fsmap finished waiting for FSMap version 34 2023-08-27T11:27:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f011364db80 10 client.7071 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-27T11:27:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f011364db80 20 client.7071 populate_metadata read hostname 'smithi045' 2023-08-27T11:27:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f011364db80 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] conn(0x55b0c5143630 0x55b0c5143a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f011364db80 4 client.7071 mds_command: new command op to 6885 tid=0[{"prefix": "status"}] 2023-08-27T11:27:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f011364db80 1 -- 172.21.15.45:0/163453328 --> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b0c4e84f20 con 0x55b0c5143630 2023-08-27T11:27:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f0105227700 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] conn(0x55b0c5143630 0x55b0c5143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f0105227700 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] conn(0x55b0c5143630 0x55b0c5143a10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e9a8aaa6dbc1fa server_cookie=7755a80653bfa808 in_seq=0 out_seq=0 2023-08-27T11:27:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f00dffff700 10 client.7071 ms_handle_connect on v2:172.21.15.135:6835/1457694749 2023-08-27T11:27:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f00dffff700 1 -- 172.21.15.45:0/163453328 <== mds.0 v2:172.21.15.135:6835/1457694749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b0c4e84f20 con 0x55b0c5143630 2023-08-27T11:27:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f00dffff700 10 client.7071 handle_command_reply: tid=0 2023-08-27T11:27:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.736+0000 7f011364db80 1 client.7071 shutdown 2023-08-27T11:27:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 -- 172.21.15.45:0/163453328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 msgr2=0x55b0c5195400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5195400 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00efd0 tx=0x7f00fc088040 comp rx=0 tx=0).stop 2023-08-27T11:27:36.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 -- 172.21.15.45:0/163453328 shutdown_connections 2023-08-27T11:27:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:6835/1457694749,v1:172.21.15.135:6837/1457694749] conn(0x55b0c5143630 0x55b0c5143a10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55b0c5121960 0x55b0c5191480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f00dffff700 0 client.7071 ms_handle_reset on v2:172.21.15.135:6835/1457694749 2023-08-27T11:27:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c50e33d0 0x55b0c5176f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 --2- 172.21.15.45:0/163453328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0c510ff40 0x55b0c5195400 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 -- 172.21.15.45:0/163453328 >> 172.21.15.45:0/163453328 conn(0x55b0c5043c30 msgr2=0x55b0c50e03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 -- 172.21.15.45:0/163453328 shutdown_connections 2023-08-27T11:27:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 1 -- 172.21.15.45:0/163453328 wait complete. 2023-08-27T11:27:36.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f011364db80 20 client.7071 trim_cache size 0 max 16384 2023-08-27T11:27:36.756 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:27:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "8ea1ae34-1697-4b79-b2ce-3f741cd3c828", 2023-08-27T11:27:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-08-27T11:27:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6885, 2023-08-27T11:27:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-08-27T11:27:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-08-27T11:27:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-08-27T11:27:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.680982253, 2023-08-27T11:27:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 34, 2023-08-27T11:27:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 86, 2023-08-27T11:27:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 86, 2023-08-27T11:27:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 17.170417657000002 2023-08-27T11:27:36.759 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:27:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 -- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f00e406a200 msgr2=0x7f00e406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f00e406a200 0x7f00e406c6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01000b5bd0 tx=0x7f00f0009040 comp rx=0 tx=0).stop 2023-08-27T11:27:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 -- 172.21.15.45:0/3475465369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000ca620 msgr2=0x7f01000c97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000ca620 0x7f01000c97b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00e810 tx=0x7f00fc00ebd0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 -- 172.21.15.45:0/3475465369 shutdown_connections 2023-08-27T11:27:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f00e406a200 0x7f00e406c6c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01000caa00 0x7f010015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01000af200 0x7f01000c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 --2- 172.21.15.45:0/3475465369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000ca620 0x7f01000c97b0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.737+0000 7f010748b700 1 -- 172.21.15.45:0/3475465369 >> 172.21.15.45:0/3475465369 conn(0x7f01000cc7e0 msgr2=0x7f0100070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.738+0000 7f010748b700 1 -- 172.21.15.45:0/3475465369 shutdown_connections 2023-08-27T11:27:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.738+0000 7f010748b700 1 -- 172.21.15.45:0/3475465369 wait complete. 2023-08-27T11:27:36.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:27:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.888+0000 7f577495f700 1 Processor -- start 2023-08-27T11:27:36.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.888+0000 7f577495f700 1 -- start start 2023-08-27T11:27:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f577495f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577006cec0 0x7f577006e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f577495f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f577006e7b0 0x7f57700ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f577495f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577006ec00 con 0x7f577015b070 2023-08-27T11:27:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f577495f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577015d8e0 con 0x7f577006e7b0 2023-08-27T11:27:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f577495f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577015dad0 con 0x7f577006cec0 2023-08-27T11:27:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576ed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f577006e7b0 0x7f57700ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576e59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f577015b070 0x7f57700cda20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44720/0 (socket says 172.21.15.45:44720) 2023-08-27T11:27:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576e59c700 1 -- 172.21.15.45:0/696211252 learned_addr learned my addr 172.21.15.45:0/696211252 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576ed9d700 1 --2- 172.21.15.45:0/696211252 >> v2:172.21.15.45:3300/0 conn(0x7f577006e7b0 0x7f57700ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57948/0 (socket says 172.21.15.45:57948) 2023-08-27T11:27:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576dd9b700 1 --2- 172.21.15.45:0/696211252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577006cec0 0x7f577006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576ed9d700 1 -- 172.21.15.45:0/696211252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577006cec0 msgr2=0x7f577006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576ed9d700 1 --2- 172.21.15.45:0/696211252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577006cec0 0x7f577006e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576ed9d700 1 -- 172.21.15.45:0/696211252 >> v1:172.21.15.45:6789/0 conn(0x7f577015b070 legacy=0x7f57700cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576d59a700 1 -- 172.21.15.45:0/696211252 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408975924 0 0) 0x7f5758001110 con 0x7f577015b070 2023-08-27T11:27:36.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576ed9d700 1 -- 172.21.15.45:0/696211252 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577015ddb0 con 0x7f577006e7b0 2023-08-27T11:27:36.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576ed9d700 1 --2- 172.21.15.45:0/696211252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 0x7f57700ae8d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5768009d90 tx=0x7f576800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88daaf3df6e5b9d8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576d59a700 1 -- 172.21.15.45:0/696211252 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576800e070 con 0x7f577006e7b0 2023-08-27T11:27:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576d59a700 1 -- 172.21.15.45:0/696211252 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576800bbd0 con 0x7f577006e7b0 2023-08-27T11:27:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.889+0000 7f576d59a700 1 -- 172.21.15.45:0/696211252 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768012430 con 0x7f577006e7b0 2023-08-27T11:27:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.890+0000 7f577495f700 1 -- 172.21.15.45:0/696211252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 msgr2=0x7f57700ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.890+0000 7f577495f700 1 --2- 172.21.15.45:0/696211252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 0x7f57700ae8d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5768009d90 tx=0x7f576800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.890+0000 7f577495f700 1 -- 172.21.15.45:0/696211252 shutdown_connections 2023-08-27T11:27:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.890+0000 7f577495f700 1 --2- 172.21.15.45:0/696211252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 0x7f57700ae8d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.890+0000 7f577495f700 1 --2- 172.21.15.45:0/696211252 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577006cec0 0x7f577006e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.890+0000 7f577495f700 1 -- 172.21.15.45:0/696211252 >> 172.21.15.45:0/696211252 conn(0x7f57700cfe50 msgr2=0x7f57700d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.890+0000 7f577495f700 1 -- 172.21.15.45:0/696211252 shutdown_connections 2023-08-27T11:27:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.890+0000 7f577495f700 1 -- 172.21.15.45:0/696211252 wait complete. 2023-08-27T11:27:36.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.891+0000 7f577495f700 1 Processor -- start 2023-08-27T11:27:36.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.891+0000 7f577495f700 1 -- start start 2023-08-27T11:27:36.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.892+0000 7f577495f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f577006cec0 0x7f5770151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.892+0000 7f577495f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 0x7f5770150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.892+0000 7f577495f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577015b070 0x7f5770154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.892+0000 7f577495f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770151560 con 0x7f577006e7b0 2023-08-27T11:27:36.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.892+0000 7f577495f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770150c50 con 0x7f577006cec0 2023-08-27T11:27:36.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.892+0000 7f577495f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57700c3ae0 con 0x7f577015b070 2023-08-27T11:27:36.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.892+0000 7f576dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 0x7f5770150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.892+0000 7f576dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 0x7f5770150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57952/0 (socket says 172.21.15.45:57952) 2023-08-27T11:27:36.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.892+0000 7f576dd9b700 1 -- 172.21.15.45:0/3972062087 learned_addr learned my addr 172.21.15.45:0/3972062087 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:36.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f576e59c700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f577006cec0 0x7f5770151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f576ed9d700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577015b070 0x7f5770154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f576dd9b700 1 -- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577015b070 msgr2=0x7f5770154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f576dd9b700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577015b070 0x7f5770154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f576dd9b700 1 -- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f577006cec0 msgr2=0x7f5770151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f576dd9b700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f577006cec0 0x7f5770151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f576dd9b700 1 -- 172.21.15.45:0/3972062087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768009a70 con 0x7f577006e7b0 2023-08-27T11:27:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f576e59c700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f577006cec0 0x7f5770151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f576dd9b700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 0x7f5770150710 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f576000e810 tx=0x7f576000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f5766ffd700 1 -- 172.21.15.45:0/3972062087 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760013610 con 0x7f577006e7b0 2023-08-27T11:27:36.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f5766ffd700 1 -- 172.21.15.45:0/3972062087 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5760013770 con 0x7f577006e7b0 2023-08-27T11:27:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57701f2c00 con 0x7f577006e7b0 2023-08-27T11:27:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f5766ffd700 1 -- 172.21.15.45:0/3972062087 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760013b20 con 0x7f577006e7b0 2023-08-27T11:27:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.893+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5770153fb0 con 0x7f577006e7b0 2023-08-27T11:27:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.894+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57700c4380 con 0x7f577006e7b0 2023-08-27T11:27:36.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.894+0000 7f5766ffd700 1 -- 172.21.15.45:0/3972062087 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5760013c80 con 0x7f577006e7b0 2023-08-27T11:27:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.895+0000 7f5766ffd700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f575006a200 0x7f575006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.895+0000 7f5766ffd700 1 -- 172.21.15.45:0/3972062087 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f57600885a0 con 0x7f577006e7b0 2023-08-27T11:27:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.895+0000 7f576e59c700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f575006a200 0x7f575006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:36.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.896+0000 7f576e59c700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f575006a200 0x7f575006c6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5758007fd0 tx=0x7f5758009040 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:36.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:36.899+0000 7f5766ffd700 1 -- 172.21.15.45:0/3972062087 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5760053990 con 0x7f577006e7b0 2023-08-27T11:27:36.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:27:36.933 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:27:37.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.166+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5770072810 con 0x7f577006e7b0 2023-08-27T11:27:37.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.166+0000 7f5766ffd700 1 -- 172.21.15.45:0/3972062087 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5760056fb0 con 0x7f577006e7b0 2023-08-27T11:27:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:27:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f575006a200 msgr2=0x7f575006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f575006a200 0x7f575006c6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5758007fd0 tx=0x7f5758009040 comp rx=0 tx=0).stop 2023-08-27T11:27:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 msgr2=0x7f5770150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 0x7f5770150710 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f576000e810 tx=0x7f576000ebd0 comp rx=0 tx=0).stop 2023-08-27T11:27:37.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 shutdown_connections 2023-08-27T11:27:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f575006a200 0x7f575006c6c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f577015b070 0x7f5770154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f577006cec0 0x7f5770151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:37.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 --2- 172.21.15.45:0/3972062087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577006e7b0 0x7f5770150710 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.168+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 >> 172.21.15.45:0/3972062087 conn(0x7f57700cfe50 msgr2=0x7f577015f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.169+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 shutdown_connections 2023-08-27T11:27:37.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:37.169+0000 7f577495f700 1 -- 172.21.15.45:0/3972062087 wait complete. 2023-08-27T11:27:37.182 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:27:37.182 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:27:37.182 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-08-27T11:27:37.183 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:27:37.183 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:37.183 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:27:37.184 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:27:37.184 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:27:37.184 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:27:37.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:37.185 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-08-27T11:27:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:27:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:27:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:27:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:27:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:37.204 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:37.205 INFO:teuthology.orchestra.run.smithi045.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:27:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-27T11:27:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41484sec preferred_lft 41484sec 2023-08-27T11:27:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-08-27T11:27:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:37.206 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:27:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:37.207 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:27:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:27:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:27:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a41d:a1ff:fed9:3bf2/64 scope link 2023-08-27T11:27:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:37.208 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether a6:e4:29:97:ce:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:27:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a4e4:29ff:fe97:ced1/64 scope link 2023-08-27T11:27:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:37.209 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:27:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:27:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c84:f3ff:fe1a:a84/64 scope link 2023-08-27T11:27:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:27:37.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:37.210 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:27:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-08-27T11:27:37.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:37.272 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-08-27T11:27:37.330 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-08-27T11:27:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:27:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-27T11:27:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-27T11:27:37.331 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:27:37.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:37.332 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:27:37.333 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:27:37.333 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-27T11:27:37.333 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-27T11:27:37.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-27T11:27:37.334 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:27:37.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:37.437 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:27:37.454 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:27:37.455 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:27:37.455 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:27:37.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:37.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-08-27T11:27:37.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:37.523 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:27:37.581 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:27:37.581 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:37.637 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:27:37.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:37.638 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:27:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:27:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:27:37.733 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:27:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:27:37.734 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:37.807 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:27:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:27:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:27:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:27:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:27:37.808 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790" 2023-08-27T11:27:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2023-08-27T11:27:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8ea1ae34-1697-4b79-b2ce-3f741cd3c828" will pass to kernel 2023-08-27T11:27:37.810 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:27:37.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:37.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:37.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:37.840 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:27:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:client6610 2023-08-27T11:27:37.928 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:27:37.928 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:37.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:27:38.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.110+0000 7fefa19ed700 1 Processor -- start 2023-08-27T11:27:38.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.110+0000 7fefa19ed700 1 -- start start 2023-08-27T11:27:38.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.111+0000 7fefa19ed700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fef9c0d0520 0x7fef9c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.111+0000 7fefa19ed700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0cb5c0 0x7fef9c15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.111+0000 7fefa19ed700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c15ec50 con 0x7fef9c0b65d0 2023-08-27T11:27:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.111+0000 7fefa19ed700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c15c620 con 0x7fef9c0d0520 2023-08-27T11:27:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.111+0000 7fefa19ed700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c15eea0 con 0x7fef9c0cb5c0 2023-08-27T11:27:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.111+0000 7fef9affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef9c0b65d0 0x7fef9c0cfe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44738/0 (socket says 172.21.15.45:44738) 2023-08-27T11:27:38.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.111+0000 7fef9affd700 1 -- 172.21.15.45:0/1829877989 learned_addr learned my addr 172.21.15.45:0/1829877989 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef9a7fc700 1 --2- 172.21.15.45:0/1829877989 >> v2:172.21.15.135:3300/0 conn(0x7fef9c0d0520 0x7fef9c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef9b7fe700 1 --2- 172.21.15.45:0/1829877989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0cb5c0 0x7fef9c15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef99ffb700 1 -- 172.21.15.45:0/1829877989 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335727780 0 0) 0x7fef9c15ec50 con 0x7fef9c0b65d0 2023-08-27T11:27:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef99ffb700 1 -- 172.21.15.45:0/1829877989 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef840035b0 con 0x7fef9c0b65d0 2023-08-27T11:27:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef9a7fc700 1 -- 172.21.15.45:0/1829877989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0cb5c0 msgr2=0x7fef9c15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef9a7fc700 1 --2- 172.21.15.45:0/1829877989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0cb5c0 0x7fef9c15c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef9a7fc700 1 -- 172.21.15.45:0/1829877989 >> v1:172.21.15.45:6789/0 conn(0x7fef9c0b65d0 legacy=0x7fef9c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef99ffb700 1 -- 172.21.15.45:0/1829877989 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3268952605 0 0) 0x7fef840035b0 con 0x7fef9c0b65d0 2023-08-27T11:27:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef9a7fc700 1 -- 172.21.15.45:0/1829877989 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c15f180 con 0x7fef9c0d0520 2023-08-27T11:27:38.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef9b7fe700 1 --2- 172.21.15.45:0/1829877989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0cb5c0 0x7fef9c15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:38.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef9a7fc700 1 --2- 172.21.15.45:0/1829877989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c0d0520 0x7fef9c0ccdb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009840 tx=0x7fef8c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb5b511cc2eacb4a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef99ffb700 1 -- 172.21.15.45:0/1829877989 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90001670 con 0x7fef9c0d0520 2023-08-27T11:27:38.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef99ffb700 1 -- 172.21.15.45:0/1829877989 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c004030 con 0x7fef9c0d0520 2023-08-27T11:27:38.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.112+0000 7fef99ffb700 1 -- 172.21.15.45:0/1829877989 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0043c0 con 0x7fef9c0d0520 2023-08-27T11:27:38.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.113+0000 7fefa19ed700 1 -- 172.21.15.45:0/1829877989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c0d0520 msgr2=0x7fef9c0ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.113+0000 7fefa19ed700 1 --2- 172.21.15.45:0/1829877989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c0d0520 0x7fef9c0ccdb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009840 tx=0x7fef8c009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:38.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.113+0000 7fefa19ed700 1 -- 172.21.15.45:0/1829877989 shutdown_connections 2023-08-27T11:27:38.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.113+0000 7fefa19ed700 1 --2- 172.21.15.45:0/1829877989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0cb5c0 0x7fef9c15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.113+0000 7fefa19ed700 1 --2- 172.21.15.45:0/1829877989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c0d0520 0x7fef9c0ccdb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.113+0000 7fefa19ed700 1 -- 172.21.15.45:0/1829877989 >> 172.21.15.45:0/1829877989 conn(0x7fef9c06f960 msgr2=0x7fef9c06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:38.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.113+0000 7fefa19ed700 1 -- 172.21.15.45:0/1829877989 shutdown_connections 2023-08-27T11:27:38.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.113+0000 7fefa19ed700 1 -- 172.21.15.45:0/1829877989 wait complete. 2023-08-27T11:27:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.114+0000 7fefa19ed700 1 Processor -- start 2023-08-27T11:27:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.114+0000 7fefa19ed700 1 -- start start 2023-08-27T11:27:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fefa19ed700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0b65d0 0x7fef9c0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fefa19ed700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c0cb5c0 0x7fef9c06b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fefa19ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9c0d0520 0x7fef9c06b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fefa19ed700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1f3a70 con 0x7fef9c0d0520 2023-08-27T11:27:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fefa19ed700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1f34b0 con 0x7fef9c0cb5c0 2023-08-27T11:27:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fefa19ed700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1f3790 con 0x7fef9c0b65d0 2023-08-27T11:27:38.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fef9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9c0d0520 0x7fef9c06b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fef9b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9c0d0520 0x7fef9c06b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57972/0 (socket says 172.21.15.45:57972) 2023-08-27T11:27:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fef9b7fe700 1 -- 172.21.15.45:0/1932501748 learned_addr learned my addr 172.21.15.45:0/1932501748 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fef9affd700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0b65d0 0x7fef9c0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.115+0000 7fef9a7fc700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c0cb5c0 0x7fef9c06b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fef9b7fe700 1 -- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0b65d0 msgr2=0x7fef9c0c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fef9b7fe700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0b65d0 0x7fef9c0c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fef9b7fe700 1 -- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c0cb5c0 msgr2=0x7fef9c06b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fef9b7fe700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c0cb5c0 0x7fef9c06b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fef9b7fe700 1 -- 172.21.15.45:0/1932501748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94009a70 con 0x7fef9c0d0520 2023-08-27T11:27:38.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fef9b7fe700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9c0d0520 0x7fef9c06b8d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fef9400efb0 tx=0x7fef9400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fef83fff700 1 -- 172.21.15.45:0/1932501748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9400cc90 con 0x7fef9c0d0520 2023-08-27T11:27:38.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fef83fff700 1 -- 172.21.15.45:0/1932501748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9400f040 con 0x7fef9c0d0520 2023-08-27T11:27:38.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c0094a0 con 0x7fef9c0d0520 2023-08-27T11:27:38.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fef83fff700 1 -- 172.21.15.45:0/1932501748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94014480 con 0x7fef9c0d0520 2023-08-27T11:27:38.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.116+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c0c8350 con 0x7fef9c0d0520 2023-08-27T11:27:38.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.117+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c0c8540 con 0x7fef9c0d0520 2023-08-27T11:27:38.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.117+0000 7fef83fff700 1 -- 172.21.15.45:0/1932501748 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fef940145e0 con 0x7fef9c0d0520 2023-08-27T11:27:38.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.118+0000 7fef83fff700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fef8806a200 0x7fef8806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.118+0000 7fef83fff700 1 -- 172.21.15.45:0/1932501748 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fef940894f0 con 0x7fef9c0d0520 2023-08-27T11:27:38.129 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.118+0000 7fef9affd700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fef8806a200 0x7fef8806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.119+0000 7fef9affd700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fef8806a200 0x7fef8806c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef90001820 tx=0x7fef90001230 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.130 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.122+0000 7fef83fff700 1 -- 172.21.15.45:0/1932501748 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fef94058370 con 0x7fef9c0d0520 2023-08-27T11:27:38.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.350+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 --> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fef9c15f740 con 0x7fef8806a200 2023-08-27T11:27:38.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.352+0000 7fef83fff700 1 -- 172.21.15.45:0/1932501748 <== mgr.6247 v2:172.21.15.135:6832/3430028400 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fef900014c0 con 0x7fef8806a200 2023-08-27T11:27:38.353 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:27:38.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:27:38.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephfs" 2023-08-27T11:27:38.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:27:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:27:38.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fef8806a200 msgr2=0x7fef8806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fef8806a200 0x7fef8806c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef90001820 tx=0x7fef90001230 comp rx=0 tx=0).stop 2023-08-27T11:27:38.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9c0d0520 msgr2=0x7fef9c06b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9c0d0520 0x7fef9c06b8d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fef9400efb0 tx=0x7fef9400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 shutdown_connections 2023-08-27T11:27:38.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7fef8806a200 0x7fef8806c6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9c0d0520 0x7fef9c06b8d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fef9c0b65d0 0x7fef9c0c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 --2- 172.21.15.45:0/1932501748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c0cb5c0 0x7fef9c06b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 >> 172.21.15.45:0/1932501748 conn(0x7fef9c06f960 msgr2=0x7fef9c15cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:38.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 shutdown_connections 2023-08-27T11:27:38.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.354+0000 7fefa19ed700 1 -- 172.21.15.45:0/1932501748 wait complete. 2023-08-27T11:27:38.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-27T11:27:38.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.494+0000 7f271dad5700 1 Processor -- start 2023-08-27T11:27:38.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.494+0000 7f271dad5700 1 -- start start 2023-08-27T11:27:38.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.494+0000 7f271dad5700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f271815b2e0 0x7f27180cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.494+0000 7f271dad5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27180cb9b0 con 0x7f27180cd540 2023-08-27T11:27:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.494+0000 7f271dad5700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27180ce2e0 con 0x7f27180cf950 2023-08-27T11:27:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.494+0000 7f271dad5700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27180cc230 con 0x7f271815b2e0 2023-08-27T11:27:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f2716ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f27180cd540 0x7f27180cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44754/0 (socket says 172.21.15.45:44754) 2023-08-27T11:27:38.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f2716ffd700 1 -- 172.21.15.45:0/1389318138 learned_addr learned my addr 172.21.15.45:0/1389318138 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:38.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27177fe700 1 --2- 172.21.15.45:0/1389318138 >> v2:172.21.15.135:3300/0 conn(0x7f271815b2e0 0x7f27180cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696623605 0 0) 0x7f27180cb9b0 con 0x7f27180cd540 2023-08-27T11:27:38.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27040035b0 con 0x7f27180cd540 2023-08-27T11:27:38.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302981559 0 0) 0x7f27180ce2e0 con 0x7f27180cf950 2023-08-27T11:27:38.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27180cb9b0 con 0x7f27180cf950 2023-08-27T11:27:38.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4167050233 0 0) 0x7f27040035b0 con 0x7f27180cd540 2023-08-27T11:27:38.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 >> v2:172.21.15.135:3300/0 conn(0x7f271815b2e0 msgr2=0x7f27180cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:38.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 --2- 172.21.15.45:0/1389318138 >> v2:172.21.15.135:3300/0 conn(0x7f271815b2e0 0x7f27180cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 >> v1:172.21.15.135:6789/0 conn(0x7f27180cf950 legacy=0x7f271806da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27180ce000 con 0x7f27180cd540 2023-08-27T11:27:38.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f2708001670 con 0x7f27180cd540 2023-08-27T11:27:38.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 >> v1:172.21.15.45:6789/0 conn(0x7f27180cd540 legacy=0x7f27180cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f2704003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2704004850 0x7f2704004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2704005210 0x7f270400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27180cc230 con 0x7f2704003be0 2023-08-27T11:27:38.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270400c0e0 con 0x7f2704005210 2023-08-27T11:27:38.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270400c2a0 con 0x7f2704004850 2023-08-27T11:27:38.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f27177fe700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f2704003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f2717fff700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2704005210 0x7f270400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.495+0000 7f2716ffd700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2704004850 0x7f2704004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f27177fe700 1 -- 172.21.15.45:0/1389318138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2704004850 msgr2=0x7f2704004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f27177fe700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2704004850 0x7f2704004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f27177fe700 1 -- 172.21.15.45:0/1389318138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2704005210 msgr2=0x7f270400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f27177fe700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2704005210 0x7f270400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f27177fe700 1 -- 172.21.15.45:0/1389318138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27040035b0 con 0x7f2704003be0 2023-08-27T11:27:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f2717fff700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2704005210 0x7f270400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f2717fff700 1 -- 172.21.15.45:0/1389318138 reap_dead start 2023-08-27T11:27:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f27177fe700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f2704003fe0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2700009b60 tx=0x7f2704005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bfe8a14985f439 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270000e6c0 con 0x7f2704003be0 2023-08-27T11:27:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270000e820 con 0x7f2704003be0 2023-08-27T11:27:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f27167fc700 1 -- 172.21.15.45:0/1389318138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270000eb90 con 0x7f2704003be0 2023-08-27T11:27:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f271dad5700 1 -- 172.21.15.45:0/1389318138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 msgr2=0x7f2704003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f271dad5700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f2704003fe0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2700009b60 tx=0x7f2704005880 comp rx=0 tx=0).stop 2023-08-27T11:27:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f271dad5700 1 -- 172.21.15.45:0/1389318138 shutdown_connections 2023-08-27T11:27:38.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f271dad5700 1 --2- 172.21.15.45:0/1389318138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f2704003fe0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.496+0000 7f271dad5700 1 -- 172.21.15.45:0/1389318138 >> 172.21.15.45:0/1389318138 conn(0x7f271809a1a0 msgr2=0x7f27180d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:38.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.497+0000 7f271dad5700 1 -- 172.21.15.45:0/1389318138 shutdown_connections 2023-08-27T11:27:38.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.497+0000 7f271dad5700 1 -- 172.21.15.45:0/1389318138 wait complete. 2023-08-27T11:27:38.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.498+0000 7f271dad5700 1 Processor -- start 2023-08-27T11:27:38.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.498+0000 7f271dad5700 1 -- start start 2023-08-27T11:27:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f271dad5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f27180b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f271dad5700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27180ae1f0 0x7f27180ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f271dad5700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27180c8fa0 0x7f27180c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f271dad5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27180aebb0 con 0x7f2704003be0 2023-08-27T11:27:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f271dad5700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27180b1110 con 0x7f27180c8fa0 2023-08-27T11:27:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f271dad5700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27180b13d0 con 0x7f27180ae1f0 2023-08-27T11:27:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f27177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f27180b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f27177fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f27180b5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57988/0 (socket says 172.21.15.45:57988) 2023-08-27T11:27:38.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f27177fe700 1 -- 172.21.15.45:0/3161148444 learned_addr learned my addr 172.21.15.45:0/3161148444 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f27177fe700 1 -- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27180ae1f0 msgr2=0x7f27180ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f27177fe700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27180ae1f0 0x7f27180ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f2717fff700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27180c8fa0 0x7f27180c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f27177fe700 1 -- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27180c8fa0 msgr2=0x7f27180c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:38.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f27177fe700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27180c8fa0 0x7f27180c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:38.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f27177fe700 1 -- 172.21.15.45:0/3161148444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27180ca570 con 0x7f2704003be0 2023-08-27T11:27:38.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f27177fe700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f27180b5830 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f27040047d0 tx=0x7f27000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f270ffff700 1 -- 172.21.15.45:0/3161148444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270000e6c0 con 0x7f2704003be0 2023-08-27T11:27:38.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f270ffff700 1 -- 172.21.15.45:0/3161148444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2700003ad0 con 0x7f2704003be0 2023-08-27T11:27:38.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27180c9cd0 con 0x7f2704003be0 2023-08-27T11:27:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.499+0000 7f270ffff700 1 -- 172.21.15.45:0/3161148444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700003e80 con 0x7f2704003be0 2023-08-27T11:27:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.500+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27180ca290 con 0x7f2704003be0 2023-08-27T11:27:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.500+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27180b16b0 con 0x7f2704003be0 2023-08-27T11:27:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.501+0000 7f270ffff700 1 -- 172.21.15.45:0/3161148444 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 30) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2700013470 con 0x7f2704003be0 2023-08-27T11:27:38.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.502+0000 7f270ffff700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f26f806a200 0x7f26f806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:38.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.502+0000 7f270ffff700 1 -- 172.21.15.45:0/3161148444 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f270008a430 con 0x7f2704003be0 2023-08-27T11:27:38.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.502+0000 7f2716ffd700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f26f806a200 0x7f26f806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:38.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.502+0000 7f2716ffd700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f26f806a200 0x7f26f806c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2708001eb0 tx=0x7f2708001190 comp rx=0 tx=0).ready entity=mgr.6247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:38.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.506+0000 7f270ffff700 1 -- 172.21.15.45:0/3161148444 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2700055990 con 0x7f2704003be0 2023-08-27T11:27:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:38.763+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f27180aebb0 con 0x7f2704003be0 2023-08-27T11:27:39.136 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.135+0000 7f270ffff700 1 -- 172.21.15.45:0/3161148444 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 31) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f27000096f0 con 0x7f2704003be0 2023-08-27T11:27:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.137+0000 7f270ffff700 1 -- 172.21.15.45:0/3161148444 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v31) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2700055990 con 0x7f2704003be0 2023-08-27T11:27:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f26f806a200 msgr2=0x7f26f806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f26f806a200 0x7f26f806c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2708001eb0 tx=0x7f2708001190 comp rx=0 tx=0).stop 2023-08-27T11:27:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 msgr2=0x7f27180b5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f27180b5830 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f27040047d0 tx=0x7f27000047c0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 shutdown_connections 2023-08-27T11:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f26f806a200 0x7f26f806c6c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2704003be0 0x7f27180b5830 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27180c8fa0 0x7f27180c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 --2- 172.21.15.45:0/3161148444 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f27180ae1f0 0x7f27180ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.139+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 >> 172.21.15.45:0/3161148444 conn(0x7f271809a1a0 msgr2=0x7f27180aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:39.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.140+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 shutdown_connections 2023-08-27T11:27:39.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.140+0000 7f271dad5700 1 -- 172.21.15.45:0/3161148444 wait complete. 2023-08-27T11:27:39.155 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-27T11:27:39.281 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:39.280+0000 7f2d6d810200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:27:39.285 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:39.285+0000 7ff3236f0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:27:39.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.290+0000 7f7915f6b700 1 Processor -- start 2023-08-27T11:27:39.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.290+0000 7f7915f6b700 1 -- start start 2023-08-27T11:27:39.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f7915f6b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f791015af30 0x7f79100cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f7915f6b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 0x7f79100cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f7915f6b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f791015b310 0x7f79100cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f7915f6b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791006f9c0 con 0x7f791015af30 2023-08-27T11:27:39.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f7915f6b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79100cb8b0 con 0x7f791015b310 2023-08-27T11:27:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f7915f6b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79100cecd0 con 0x7f79100cf960 2023-08-27T11:27:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f790f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f791015af30 0x7f79100cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f790f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f791015af30 0x7f79100cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58024/0 (socket says 172.21.15.45:58024) 2023-08-27T11:27:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f790f7fe700 1 -- 172.21.15.45:0/2320725449 learned_addr learned my addr 172.21.15.45:0/2320725449 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f790effd700 1 --2- 172.21.15.45:0/2320725449 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 0x7f79100cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.291+0000 7f790ffff700 1 --2- 172.21.15.45:0/2320725449 >> v2:172.21.15.135:3300/0 conn(0x7f791015b310 0x7f79100cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790f7fe700 1 -- 172.21.15.45:0/2320725449 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 msgr2=0x7f79100cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790f7fe700 1 --2- 172.21.15.45:0/2320725449 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 0x7f79100cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790f7fe700 1 -- 172.21.15.45:0/2320725449 >> v2:172.21.15.135:3300/0 conn(0x7f791015b310 msgr2=0x7f79100cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790f7fe700 1 --2- 172.21.15.45:0/2320725449 >> v2:172.21.15.135:3300/0 conn(0x7f791015b310 0x7f79100cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790f7fe700 1 -- 172.21.15.45:0/2320725449 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79100cbae0 con 0x7f791015af30 2023-08-27T11:27:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790ffff700 1 --2- 172.21.15.45:0/2320725449 >> v2:172.21.15.135:3300/0 conn(0x7f791015b310 0x7f79100cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790f7fe700 1 --2- 172.21.15.45:0/2320725449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 0x7f79100cf420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f791015fd50 tx=0x7f7900009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875519252a798a0e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790e7fc700 1 -- 172.21.15.45:0/2320725449 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900011830 con 0x7f791015af30 2023-08-27T11:27:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790e7fc700 1 -- 172.21.15.45:0/2320725449 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7900011990 con 0x7f791015af30 2023-08-27T11:27:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.292+0000 7f790e7fc700 1 -- 172.21.15.45:0/2320725449 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900011d20 con 0x7f791015af30 2023-08-27T11:27:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.293+0000 7f7915f6b700 1 -- 172.21.15.45:0/2320725449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 msgr2=0x7f79100cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.293+0000 7f7915f6b700 1 --2- 172.21.15.45:0/2320725449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 0x7f79100cf420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f791015fd50 tx=0x7f7900009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.293+0000 7f7915f6b700 1 -- 172.21.15.45:0/2320725449 shutdown_connections 2023-08-27T11:27:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.293+0000 7f7915f6b700 1 --2- 172.21.15.45:0/2320725449 >> v2:172.21.15.135:3300/0 conn(0x7f791015b310 0x7f79100cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.293+0000 7f7915f6b700 1 --2- 172.21.15.45:0/2320725449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 0x7f79100cf420 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.293+0000 7f7915f6b700 1 --2- 172.21.15.45:0/2320725449 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 0x7f79100cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.293+0000 7f7915f6b700 1 -- 172.21.15.45:0/2320725449 >> 172.21.15.45:0/2320725449 conn(0x7f791009a1a0 msgr2=0x7f79100d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:39.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.293+0000 7f7915f6b700 1 -- 172.21.15.45:0/2320725449 shutdown_connections 2023-08-27T11:27:39.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.293+0000 7f7915f6b700 1 -- 172.21.15.45:0/2320725449 wait complete. 2023-08-27T11:27:39.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.294+0000 7f7915f6b700 1 Processor -- start 2023-08-27T11:27:39.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.294+0000 7f7915f6b700 1 -- start start 2023-08-27T11:27:39.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f7915f6b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 0x7f79100c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f7915f6b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 0x7f791006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f7915f6b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791015b310 0x7f79101f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f7915f6b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791006b530 con 0x7f791015af30 2023-08-27T11:27:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f7915f6b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791006b810 con 0x7f791015b310 2023-08-27T11:27:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f7915f6b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791006baf0 con 0x7f79100cf960 2023-08-27T11:27:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 0x7f791006cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 0x7f791006cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58032/0 (socket says 172.21.15.45:58032) 2023-08-27T11:27:39.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790ffff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791015b310 0x7f79101f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790f7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 0x7f79100c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790effd700 1 -- 172.21.15.45:0/84575739 learned_addr learned my addr 172.21.15.45:0/84575739 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:39.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790f7fe700 1 -- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791015b310 msgr2=0x7f79101f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790f7fe700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791015b310 0x7f79101f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790f7fe700 1 -- 172.21.15.45:0/84575739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 msgr2=0x7f791006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790f7fe700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 0x7f791006cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790f7fe700 1 -- 172.21.15.45:0/84575739 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7900009550 con 0x7f79100cf960 2023-08-27T11:27:39.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.295+0000 7f790effd700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 0x7f791006cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:39.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.296+0000 7f790f7fe700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 0x7f79100c6f60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7900016000 tx=0x7f7900004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.296+0000 7f78f7fff700 1 -- 172.21.15.45:0/84575739 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900004240 con 0x7f79100cf960 2023-08-27T11:27:39.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.296+0000 7f78f7fff700 1 -- 172.21.15.45:0/84575739 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790001f070 con 0x7f79100cf960 2023-08-27T11:27:39.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.296+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791006bdd0 con 0x7f79100cf960 2023-08-27T11:27:39.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.296+0000 7f78f7fff700 1 -- 172.21.15.45:0/84575739 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79000036a0 con 0x7f79100cf960 2023-08-27T11:27:39.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.296+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791006c5d0 con 0x7f79100cf960 2023-08-27T11:27:39.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.297+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791009ee10 con 0x7f79100cf960 2023-08-27T11:27:39.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.297+0000 7f78f7fff700 1 -- 172.21.15.45:0/84575739 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 31) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f7900021030 con 0x7f79100cf960 2023-08-27T11:27:39.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.298+0000 7f78f7fff700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f78fc06a280 0x7f78fc06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.298+0000 7f78f7fff700 1 -- 172.21.15.45:0/84575739 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f7900089680 con 0x7f79100cf960 2023-08-27T11:27:39.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.298+0000 7f790effd700 1 -- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f78fc06a280 msgr2=0x7f78fc06c740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3430028400 2023-08-27T11:27:39.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.298+0000 7f790effd700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f78fc06a280 0x7f78fc06c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:39.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.302+0000 7f78f7fff700 1 -- 172.21.15.45:0/84575739 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7900054a60 con 0x7f79100cf960 2023-08-27T11:27:39.349 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:39.347+0000 7f2d6d810200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:39.356 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:39.356+0000 7ff3236f0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:39.420 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:39.419+0000 7f2d6d810200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:27:39.431 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:39.430+0000 7ff3236f0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:27:39.497 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:39.496+0000 7f2d6d810200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:27:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.499+0000 7f790effd700 1 -- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f78fc06a280 msgr2=0x7f78fc06c740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3430028400 2023-08-27T11:27:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.499+0000 7f790effd700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f78fc06a280 0x7f78fc06c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:39.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.505+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f79100734a0 con 0x7f79100cf960 2023-08-27T11:27:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.509+0000 7f78f7fff700 1 -- 172.21.15.45:0/84575739 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7900015020 con 0x7f79100cf960 2023-08-27T11:27:39.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f78fc06a280 msgr2=0x7f78fc06c740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f78fc06a280 0x7f78fc06c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 msgr2=0x7f79100c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 0x7f79100c6f60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7900016000 tx=0x7f7900004750 comp rx=0 tx=0).stop 2023-08-27T11:27:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 shutdown_connections 2023-08-27T11:27:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f78fc06a280 0x7f78fc06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791015b310 0x7f79101f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f79100cf960 0x7f79100c6f60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 --2- 172.21.15.45:0/84575739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f791015af30 0x7f791006cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 >> 172.21.15.45:0/84575739 conn(0x7f791009a1a0 msgr2=0x7f791015a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:39.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 shutdown_connections 2023-08-27T11:27:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.511+0000 7f7915f6b700 1 -- 172.21.15.45:0/84575739 wait complete. 2023-08-27T11:27:39.515 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:39.512+0000 7ff3236f0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:27:39.525 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-27T11:27:39.640 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:39.639+0000 7f2d6d810200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:27:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.654+0000 7f21ef501700 1 Processor -- start 2023-08-27T11:27:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.654+0000 7f21ef501700 1 -- start start 2023-08-27T11:27:39.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21ef501700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e815b310 0x7f21e80cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21ef501700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e80cb8b0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21ef501700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e80cecd0 con 0x7f21e815af30 2023-08-27T11:27:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21ef501700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e806f9c0 con 0x7f21e815b310 2023-08-27T11:27:39.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21eda9e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f21e80cdcb0 0x7f21e8164fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44842/0 (socket says 172.21.15.45:44842) 2023-08-27T11:27:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21eda9e700 1 -- 172.21.15.45:0/3149516473 learned_addr learned my addr 172.21.15.45:0/3149516473 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21eca9c700 1 --2- 172.21.15.45:0/3149516473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e815b310 0x7f21e80cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21dffff700 1 -- 172.21.15.45:0/3149516473 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028867218 0 0) 0x7f21e80cb8b0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21dffff700 1 -- 172.21.15.45:0/3149516473 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d40035b0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21dffff700 1 -- 172.21.15.45:0/3149516473 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894996015 0 0) 0x7f21e80cecd0 con 0x7f21e815af30 2023-08-27T11:27:39.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21dffff700 1 -- 172.21.15.45:0/3149516473 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e80cb8b0 con 0x7f21e815af30 2023-08-27T11:27:39.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21eca9c700 1 -- 172.21.15.45:0/3149516473 >> v1:172.21.15.135:6789/0 conn(0x7f21e815af30 legacy=0x7f21e80cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21eca9c700 1 -- 172.21.15.45:0/3149516473 >> v1:172.21.15.45:6789/0 conn(0x7f21e80cdcb0 legacy=0x7f21e8164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.655+0000 7f21eca9c700 1 -- 172.21.15.45:0/3149516473 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80cbae0 con 0x7f21e815b310 2023-08-27T11:27:39.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21eca9c700 1 --2- 172.21.15.45:0/3149516473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e815b310 0x7f21e80cd770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009870 tx=0x7f21d8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cd94cec692e974a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21dffff700 1 -- 172.21.15.45:0/3149516473 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8011830 con 0x7f21e815b310 2023-08-27T11:27:39.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21dffff700 1 -- 172.21.15.45:0/3149516473 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8011990 con 0x7f21e815b310 2023-08-27T11:27:39.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21dffff700 1 -- 172.21.15.45:0/3149516473 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8011d20 con 0x7f21e815b310 2023-08-27T11:27:39.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21ef501700 1 -- 172.21.15.45:0/3149516473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e815b310 msgr2=0x7f21e80cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21ef501700 1 --2- 172.21.15.45:0/3149516473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e815b310 0x7f21e80cd770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009870 tx=0x7f21d8009b80 comp rx=0 tx=0).stop 2023-08-27T11:27:39.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21ef501700 1 -- 172.21.15.45:0/3149516473 shutdown_connections 2023-08-27T11:27:39.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21ef501700 1 --2- 172.21.15.45:0/3149516473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e815b310 0x7f21e80cd770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21ef501700 1 -- 172.21.15.45:0/3149516473 >> 172.21.15.45:0/3149516473 conn(0x7f21e809a1a0 msgr2=0x7f21e80d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:39.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21ef501700 1 -- 172.21.15.45:0/3149516473 shutdown_connections 2023-08-27T11:27:39.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.656+0000 7f21ef501700 1 -- 172.21.15.45:0/3149516473 wait complete. 2023-08-27T11:27:39.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.658+0000 7f21ef501700 1 Processor -- start 2023-08-27T11:27:39.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.658+0000 7f21ef501700 1 -- start start 2023-08-27T11:27:39.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.658+0000 7f21ef501700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e80cdcb0 0x7f21e81f0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.658+0000 7f21ef501700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e815af30 0x7f21e815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.658+0000 7f21ef501700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e815b310 0x7f21e815f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.658+0000 7f21ef501700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80b50e0 con 0x7f21e815b310 2023-08-27T11:27:39.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.658+0000 7f21ef501700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80b4510 con 0x7f21e815af30 2023-08-27T11:27:39.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.658+0000 7f21ef501700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80b47b0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21eda9e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e815b310 0x7f21e815f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ed29d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e80cdcb0 0x7f21e81f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21eca9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e815af30 0x7f21e815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21eda9e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e815b310 0x7f21e815f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58072/0 (socket says 172.21.15.45:58072) 2023-08-27T11:27:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21eda9e700 1 -- 172.21.15.45:0/2828898209 learned_addr learned my addr 172.21.15.45:0/2828898209 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:39.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ed29d700 1 -- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e815af30 msgr2=0x7f21e815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ed29d700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e815af30 0x7f21e815ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ed29d700 1 -- 172.21.15.45:0/2828898209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e815b310 msgr2=0x7f21e815f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ed29d700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e815b310 0x7f21e815f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ed29d700 1 -- 172.21.15.45:0/2828898209 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80094a0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ed29d700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e80cdcb0 0x7f21e81f0310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21e400c450 tx=0x7f21e400c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:39.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ddffb700 1 -- 172.21.15.45:0/2828898209 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40052d0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ddffb700 1 -- 172.21.15.45:0/2828898209 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e4005430 con 0x7f21e80cdcb0 2023-08-27T11:27:39.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ddffb700 1 -- 172.21.15.45:0/2828898209 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40038c0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.659+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e80b4a90 con 0x7f21e80cdcb0 2023-08-27T11:27:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.660+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e81f0850 con 0x7f21e80cdcb0 2023-08-27T11:27:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.660+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e80c9030 con 0x7f21e80cdcb0 2023-08-27T11:27:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.661+0000 7f21ddffb700 1 -- 172.21.15.45:0/2828898209 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 31) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f21e4003a20 con 0x7f21e80cdcb0 2023-08-27T11:27:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.661+0000 7f21ddffb700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f21cc06a250 0x7f21cc06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.661+0000 7f21eca9c700 1 -- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f21cc06a250 msgr2=0x7f21cc06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3430028400 2023-08-27T11:27:39.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.661+0000 7f21ddffb700 1 -- 172.21.15.45:0/2828898209 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f21e4087bf0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.661+0000 7f21eca9c700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f21cc06a250 0x7f21cc06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:39.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.665+0000 7f21ddffb700 1 -- 172.21.15.45:0/2828898209 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f21e4053120 con 0x7f21e80cdcb0 2023-08-27T11:27:39.675 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:39.667+0000 7ff3236f0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:27:39.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.862+0000 7f21eca9c700 1 -- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f21cc06a250 msgr2=0x7f21cc06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3430028400 2023-08-27T11:27:39.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.862+0000 7f21eca9c700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f21cc06a250 0x7f21cc06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:39.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.868+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f21e80b48f0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.873+0000 7f21ddffb700 1 -- 172.21.15.45:0/2828898209 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f21e80b48f0 con 0x7f21e80cdcb0 2023-08-27T11:27:39.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f21cc06a250 msgr2=0x7f21cc06c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:39.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f21cc06a250 0x7f21cc06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e80cdcb0 msgr2=0x7f21e81f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:39.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e80cdcb0 0x7f21e81f0310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21e400c450 tx=0x7f21e400c760 comp rx=0 tx=0).stop 2023-08-27T11:27:39.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 shutdown_connections 2023-08-27T11:27:39.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f21cc06a250 0x7f21cc06c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e815b310 0x7f21e815f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f21e80cdcb0 0x7f21e81f0310 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 --2- 172.21.15.45:0/2828898209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f21e815af30 0x7f21e815ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 >> 172.21.15.45:0/2828898209 conn(0x7f21e809a1a0 msgr2=0x7f21e806f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 shutdown_connections 2023-08-27T11:27:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:39.875+0000 7f21ef501700 1 -- 172.21.15.45:0/2828898209 wait complete. 2023-08-27T11:27:39.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:39.889 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-27T11:27:39.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-08-27T11:27:39.967 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:39.965+0000 7f2d6d810200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:27:40.015 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:40.015+0000 7ff3236f0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:27:40.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.079+0000 7f4a50db9700 1 Processor -- start 2023-08-27T11:27:40.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.079+0000 7f4a50db9700 1 -- start start 2023-08-27T11:27:40.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a50db9700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4a4c15a0f0 0x7f4a4c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:40.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a50db9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a4c0cd300 0x7f4a4c15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:40.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a50db9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c0cfae0 con 0x7f4a4c15e030 2023-08-27T11:27:40.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a50db9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c15e900 con 0x7f4a4c0cd300 2023-08-27T11:27:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a50db9700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c0cae60 con 0x7f4a4c15a0f0 2023-08-27T11:27:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a49d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a4c0cd300 0x7f4a4c15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a49d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a4c0cd300 0x7f4a4c15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58124/0 (socket says 172.21.15.45:58124) 2023-08-27T11:27:40.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a4ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4a4c15e030 0x7f4a4c0cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44902/0 (socket says 172.21.15.45:44902) 2023-08-27T11:27:40.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a49d9b700 1 -- 172.21.15.45:0/4105684354 learned_addr learned my addr 172.21.15.45:0/4105684354 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:40.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/4105684354 >> v2:172.21.15.135:3300/0 conn(0x7f4a4c15a0f0 0x7f4a4c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:40.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a49d9b700 1 -- 172.21.15.45:0/4105684354 >> v2:172.21.15.135:3300/0 conn(0x7f4a4c15a0f0 msgr2=0x7f4a4c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:40.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a49d9b700 1 --2- 172.21.15.45:0/4105684354 >> v2:172.21.15.135:3300/0 conn(0x7f4a4c15a0f0 0x7f4a4c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:40.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a49d9b700 1 -- 172.21.15.45:0/4105684354 >> v1:172.21.15.45:6789/0 conn(0x7f4a4c15e030 legacy=0x7f4a4c0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:40.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a4959a700 1 -- 172.21.15.45:0/4105684354 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511530112 0 0) 0x7f4a4c0cfae0 con 0x7f4a4c15e030 2023-08-27T11:27:40.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.081+0000 7f4a49d9b700 1 -- 172.21.15.45:0/4105684354 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0cb140 con 0x7f4a4c0cd300 2023-08-27T11:27:40.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.082+0000 7f4a49d9b700 1 --2- 172.21.15.45:0/4105684354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4c0cd300 0x7f4a4c15daf0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0730c0 tx=0x7f4a40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d8c6d9206d4a5e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:40.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.082+0000 7f4a4959a700 1 -- 172.21.15.45:0/4105684354 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011830 con 0x7f4a4c0cd300 2023-08-27T11:27:40.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.082+0000 7f4a4959a700 1 -- 172.21.15.45:0/4105684354 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40011990 con 0x7f4a4c0cd300 2023-08-27T11:27:40.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.082+0000 7f4a4959a700 1 -- 172.21.15.45:0/4105684354 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011d20 con 0x7f4a4c0cd300 2023-08-27T11:27:40.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.083+0000 7f4a50db9700 1 -- 172.21.15.45:0/4105684354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4c0cd300 msgr2=0x7f4a4c15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:40.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.083+0000 7f4a50db9700 1 --2- 172.21.15.45:0/4105684354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4c0cd300 0x7f4a4c15daf0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0730c0 tx=0x7f4a40009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:40.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.083+0000 7f4a50db9700 1 -- 172.21.15.45:0/4105684354 shutdown_connections 2023-08-27T11:27:40.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.083+0000 7f4a50db9700 1 --2- 172.21.15.45:0/4105684354 >> v2:172.21.15.135:3300/0 conn(0x7f4a4c15a0f0 0x7f4a4c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:40.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.083+0000 7f4a50db9700 1 --2- 172.21.15.45:0/4105684354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4c0cd300 0x7f4a4c15daf0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:40.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.083+0000 7f4a50db9700 1 -- 172.21.15.45:0/4105684354 >> 172.21.15.45:0/4105684354 conn(0x7f4a4c0cc7e0 msgr2=0x7f4a4c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:40.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.083+0000 7f4a50db9700 1 -- 172.21.15.45:0/4105684354 shutdown_connections 2023-08-27T11:27:40.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.083+0000 7f4a50db9700 1 -- 172.21.15.45:0/4105684354 wait complete. 2023-08-27T11:27:40.094 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:40.081+0000 7f2d6d810200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:27:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.085+0000 7f4a50db9700 1 Processor -- start 2023-08-27T11:27:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.085+0000 7f4a50db9700 1 -- start start 2023-08-27T11:27:40.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a50db9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a4c0cd300 0x7f4a4c1f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a50db9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a4c15a0f0 0x7f4a4c1f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a50db9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4c15e030 0x7f4a4c14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a50db9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c1f5f80 con 0x7f4a4c15e030 2023-08-27T11:27:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a50db9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0aec20 con 0x7f4a4c0cd300 2023-08-27T11:27:40.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a50db9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0aeec0 con 0x7f4a4c15a0f0 2023-08-27T11:27:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4c15e030 0x7f4a4c14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4c15e030 0x7f4a4c14f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58132/0 (socket says 172.21.15.45:58132) 2023-08-27T11:27:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4ad9d700 1 -- 172.21.15.45:0/2389839150 learned_addr learned my addr 172.21.15.45:0/2389839150 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a49d9b700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a4c15a0f0 0x7f4a4c1f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:40.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a4c0cd300 0x7f4a4c1f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:40.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4ad9d700 1 -- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a4c15a0f0 msgr2=0x7f4a4c1f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:40.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4ad9d700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4a4c15a0f0 0x7f4a4c1f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:40.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4ad9d700 1 -- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a4c0cd300 msgr2=0x7f4a4c1f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:40.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4ad9d700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a4c0cd300 0x7f4a4c1f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:40.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4ad9d700 1 -- 172.21.15.45:0/2389839150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40009550 con 0x7f4a4c15e030 2023-08-27T11:27:40.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a4c0cd300 0x7f4a4c1f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:40.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a4ad9d700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a4c15e030 0x7f4a4c14f0b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4a44002230 tx=0x7f4a4400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:40.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4400aac0 con 0x7f4a4c15e030 2023-08-27T11:27:40.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.087+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4400ac40 con 0x7f4a4c15e030 2023-08-27T11:27:40.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.088+0000 7f4a50db9700 1 -- 172.21.15.45:0/2389839150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c0af1a0 con 0x7f4a4c15e030 2023-08-27T11:27:40.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.088+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440070d0 con 0x7f4a4c15e030 2023-08-27T11:27:40.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.088+0000 7f4a50db9700 1 -- 172.21.15.45:0/2389839150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c1f64b0 con 0x7f4a4c15e030 2023-08-27T11:27:40.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.089+0000 7f4a50db9700 1 -- 172.21.15.45:0/2389839150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c0726d0 con 0x7f4a4c15e030 2023-08-27T11:27:40.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.089+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 31) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f4a44007250 con 0x7f4a4c15e030 2023-08-27T11:27:40.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.090+0000 7f4a3affd700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 0x7f4a2c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:40.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.090+0000 7f4a4a59c700 1 -- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 msgr2=0x7f4a2c06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3430028400 2023-08-27T11:27:40.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.090+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4a44088620 con 0x7f4a4c15e030 2023-08-27T11:27:40.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.090+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 0x7f4a2c06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:40.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.094+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4a44053b20 con 0x7f4a4c15e030 2023-08-27T11:27:40.140 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:40.140+0000 7ff3236f0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:27:40.156 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:40.155+0000 7f2d6d810200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:27:40.218 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:40.217+0000 7ff3236f0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:27:40.237 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:40.236+0000 7f2d6d810200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:27:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.291+0000 7f4a4a59c700 1 -- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 msgr2=0x7f4a2c06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3430028400 2023-08-27T11:27:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.291+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 0x7f4a2c06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:40.302 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:40.302+0000 7ff3236f0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:27:40.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.322+0000 7f4a50db9700 1 -- 172.21.15.45:0/2389839150 --> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a4c0cb700 con 0x7f4a2c06a250 2023-08-27T11:27:40.607 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:40.606+0000 7f2d6d810200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:27:40.690 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:40.689+0000 7ff3236f0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:27:40.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.692+0000 7f4a4a59c700 1 -- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 msgr2=0x7f4a2c06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3430028400 2023-08-27T11:27:40.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:40.692+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 0x7f4a2c06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:27:41.179 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:41.177+0000 7f2d6d810200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:27:41.248 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:41.247+0000 7f2d6d810200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:27:41.299 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:41.298+0000 7ff3236f0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:27:41.372 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:41.371+0000 7ff3236f0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:27:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:41.493+0000 7f4a4a59c700 1 -- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 msgr2=0x7f4a2c06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3430028400 2023-08-27T11:27:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:41.493+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 0x7f4a2c06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:27:41.880 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:41.878+0000 7f2d6d810200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:41.957 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:41.956+0000 7f2d6d810200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:27:42.037 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:42.035+0000 7ff3236f0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:42.103 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:42.102+0000 7f2d6d810200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:42.119 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:42.119+0000 7ff3236f0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:27:42.271 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:42.270+0000 7ff3236f0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:42.331 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:42.330+0000 7f2d6d810200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:27:42.404 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:42.403+0000 7f2d6d810200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:27:42.508 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:42.508+0000 7ff3236f0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:27:42.587 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:42.586+0000 7ff3236f0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:27:42.670 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:42.669+0000 7f2d6d810200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:27:42.864 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:42.863+0000 7ff3236f0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:27:42.892 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:42.891+0000 7f2d6d810200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:27:43.040 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:43.039+0000 7f2d6d810200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:27:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:43.094+0000 7f4a4a59c700 1 -- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 msgr2=0x7f4a2c06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3430028400 2023-08-27T11:27:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:43.095+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 0x7f4a2c06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-27T11:27:43.096 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:43.095+0000 7ff3236f0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:27:43.154 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:43.153+0000 7f2d6d810200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:27:43.252 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:43.251+0000 7ff3236f0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:27:43.309 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:43.308+0000 7f2d6d810200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:27:43.372 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:43.371+0000 7ff3236f0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:27:43.533 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:43.532+0000 7ff3236f0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:27:43.572 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:43.570+0000 7f2d6d810200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:43.732 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:43.730+0000 7f2d6d810200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:27:43.805 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:43.804+0000 7ff3236f0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:43.973 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:43.972+0000 7ff3236f0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:27:44.257 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.256+0000 7f2d6d810200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:27:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:44.278+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 32) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f4a44053ef0 con 0x7f4a4c15e030 2023-08-27T11:27:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:44.278+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 msgr2=0x7f4a2c06c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:44.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:44.278+0000 7f4a3affd700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/3430028400,v1:172.21.15.135:6833/3430028400] conn(0x7f4a2c06a250 0x7f4a2c06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:44.390 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce3874700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.390 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce3874700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.390 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce3874700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.391 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce3874700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.391 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce3874700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.391 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce086e700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.391 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce086e700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.391 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce086e700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.392 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce086e700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.392 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:44.389+0000 7f2ce086e700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:44.541 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:44.541+0000 7ff3236f0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:27:45.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.285+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 33) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f4a44056cd0 con 0x7f4a4c15e030 2023-08-27T11:27:45.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.286+0000 7f4a3affd700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f4a2c072d40 0x7f4a2c075130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.286+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 --> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a2c036300 con 0x7f4a2c072d40 2023-08-27T11:27:45.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.286+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f4a2c072d40 0x7f4a2c075130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.288+0000 7f4a4a59c700 1 --2- 172.21.15.45:0/2389839150 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f4a2c072d40 0x7f4a2c075130 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c157230 tx=0x7f4a3c009210 comp rx=0 tx=0).ready entity=mgr.6637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.298 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:45.295+0000 7f2cea8c2700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-08-27T11:27:45.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.297+0000 7f4a3affd700 1 -- 172.21.15.45:0/2389839150 <== mgr.6637 v2:172.21.15.135:6832/294910128 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f4a3c011e90 con 0x7f4a2c072d40 2023-08-27T11:27:45.300 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-08-27T11:27:45.304 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-08-27T11:27:45.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:45.306 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-08-27T11:27:45.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-27T11:27:45.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.503+0000 7f8c6db3f700 1 Processor -- start 2023-08-27T11:27:45.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.503+0000 7f8c6db3f700 1 -- start start 2023-08-27T11:27:45.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.503+0000 7f8c6db3f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c6815b2e0 0x7f8c680cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.503+0000 7f8c6db3f700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c680cf950 0x7f8c680cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.503+0000 7f8c6db3f700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c680cb9b0 con 0x7f8c680cd540 2023-08-27T11:27:45.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.503+0000 7f8c6db3f700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c680ce2e0 con 0x7f8c680cf950 2023-08-27T11:27:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.503+0000 7f8c6db3f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680cc230 con 0x7f8c6815b2e0 2023-08-27T11:27:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c677fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c6815b2e0 0x7f8c680cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c67fff700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c680cf950 0x7f8c680cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c66ffd700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8c680cd540 0x7f8c680cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:40658/0 (socket says 172.21.15.45:40658) 2023-08-27T11:27:45.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c66ffd700 1 -- 172.21.15.45:0/3760833001 learned_addr learned my addr 172.21.15.45:0/3760833001 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:45.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c667fc700 1 -- 172.21.15.45:0/3760833001 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265754464 0 0) 0x7f8c680cb9b0 con 0x7f8c680cd540 2023-08-27T11:27:45.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c667fc700 1 -- 172.21.15.45:0/3760833001 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c540035b0 con 0x7f8c680cd540 2023-08-27T11:27:45.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c67fff700 1 -- 172.21.15.45:0/3760833001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c6815b2e0 msgr2=0x7f8c680cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:45.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c67fff700 1 --2- 172.21.15.45:0/3760833001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c6815b2e0 0x7f8c680cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c67fff700 1 -- 172.21.15.45:0/3760833001 >> v1:172.21.15.135:6789/0 conn(0x7f8c680cd540 legacy=0x7f8c680cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c67fff700 1 -- 172.21.15.45:0/3760833001 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680ce000 con 0x7f8c680cf950 2023-08-27T11:27:45.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c677fe700 1 --2- 172.21.15.45:0/3760833001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c6815b2e0 0x7f8c680cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:45.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.504+0000 7f8c67fff700 1 --2- 172.21.15.45:0/3760833001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c680cf950 0x7f8c680cd9c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009d90 tx=0x7f8c5800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e8e58b2ec8a0035 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c667fc700 1 -- 172.21.15.45:0/3760833001 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5800e070 con 0x7f8c680cf950 2023-08-27T11:27:45.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c667fc700 1 -- 172.21.15.45:0/3760833001 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5800bb20 con 0x7f8c680cf950 2023-08-27T11:27:45.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c667fc700 1 -- 172.21.15.45:0/3760833001 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58012430 con 0x7f8c680cf950 2023-08-27T11:27:45.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3760833001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c680cf950 msgr2=0x7f8c680cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c6db3f700 1 --2- 172.21.15.45:0/3760833001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c680cf950 0x7f8c680cd9c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009d90 tx=0x7f8c5800b330 comp rx=0 tx=0).stop 2023-08-27T11:27:45.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3760833001 shutdown_connections 2023-08-27T11:27:45.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c6db3f700 1 --2- 172.21.15.45:0/3760833001 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c6815b2e0 0x7f8c680cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c6db3f700 1 --2- 172.21.15.45:0/3760833001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c680cf950 0x7f8c680cd9c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3760833001 >> 172.21.15.45:0/3760833001 conn(0x7f8c6809a1a0 msgr2=0x7f8c680d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:45.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3760833001 shutdown_connections 2023-08-27T11:27:45.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.505+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3760833001 wait complete. 2023-08-27T11:27:45.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.506+0000 7f8c6db3f700 1 Processor -- start 2023-08-27T11:27:45.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c6db3f700 1 -- start start 2023-08-27T11:27:45.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c6db3f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c680cd540 0x7f8c680c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c6db3f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c680cf950 0x7f8c680c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c6db3f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c6815b2e0 0x7f8c681e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c6db3f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680c9910 con 0x7f8c6815b2e0 2023-08-27T11:27:45.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c6db3f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680c63c0 con 0x7f8c680cf950 2023-08-27T11:27:45.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c6db3f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680c65f0 con 0x7f8c680cd540 2023-08-27T11:27:45.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c67fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c6815b2e0 0x7f8c681e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c67fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c6815b2e0 0x7f8c681e4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58226/0 (socket says 172.21.15.45:58226) 2023-08-27T11:27:45.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.507+0000 7f8c67fff700 1 -- 172.21.15.45:0/3049206522 learned_addr learned my addr 172.21.15.45:0/3049206522 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:45.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c66ffd700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c680cf950 0x7f8c680c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c677fe700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c680cd540 0x7f8c680c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c67fff700 1 -- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c680cd540 msgr2=0x7f8c680c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c67fff700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c680cd540 0x7f8c680c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c67fff700 1 -- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c680cf950 msgr2=0x7f8c680c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:45.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c67fff700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c680cf950 0x7f8c680c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:45.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c67fff700 1 -- 172.21.15.45:0/3049206522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c600094a0 con 0x7f8c6815b2e0 2023-08-27T11:27:45.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c67fff700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c6815b2e0 0x7f8c681e4b70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8c58014040 tx=0x7f8c58004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c5ffff700 1 -- 172.21.15.45:0/3049206522 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c580129f0 con 0x7f8c6815b2e0 2023-08-27T11:27:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c5ffff700 1 -- 172.21.15.45:0/3049206522 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58020070 con 0x7f8c6815b2e0 2023-08-27T11:27:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c58009a70 con 0x7f8c6815b2e0 2023-08-27T11:27:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c5ffff700 1 -- 172.21.15.45:0/3049206522 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58012db0 con 0x7f8c6815b2e0 2023-08-27T11:27:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.508+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c680c6e90 con 0x7f8c6815b2e0 2023-08-27T11:27:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.509+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c680b0cc0 con 0x7f8c6815b2e0 2023-08-27T11:27:45.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.510+0000 7f8c5ffff700 1 -- 172.21.15.45:0/3049206522 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 33) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f8c58004040 con 0x7f8c6815b2e0 2023-08-27T11:27:45.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.510+0000 7f8c5ffff700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c48069f80 0x7f8c4806c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:45.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.510+0000 7f8c5ffff700 1 -- 172.21.15.45:0/3049206522 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f8c58022070 con 0x7f8c6815b2e0 2023-08-27T11:27:45.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.510+0000 7f8c677fe700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c48069f80 0x7f8c4806c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:45.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.511+0000 7f8c677fe700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c48069f80 0x7f8c4806c440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009d00 tx=0x7f8c60008040 comp rx=0 tx=0).ready entity=mgr.6637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:45.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.515+0000 7f8c5ffff700 1 -- 172.21.15.45:0/3049206522 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8c58059800 con 0x7f8c6815b2e0 2023-08-27T11:27:45.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:45.775+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8c6809eda0 con 0x7f8c6815b2e0 2023-08-27T11:27:46.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.335+0000 7f8c5ffff700 1 -- 172.21.15.45:0/3049206522 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f8c58056140 con 0x7f8c6815b2e0 2023-08-27T11:27:46.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.336+0000 7f8c5ffff700 1 -- 172.21.15.45:0/3049206522 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v34) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8c5806cff0 con 0x7f8c6815b2e0 2023-08-27T11:27:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c48069f80 msgr2=0x7f8c4806c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:46.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c48069f80 0x7f8c4806c440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009d00 tx=0x7f8c60008040 comp rx=0 tx=0).stop 2023-08-27T11:27:46.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c6815b2e0 msgr2=0x7f8c681e4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:46.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c6815b2e0 0x7f8c681e4b70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8c58014040 tx=0x7f8c58004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 shutdown_connections 2023-08-27T11:27:46.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c48069f80 0x7f8c4806c440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c6815b2e0 0x7f8c681e4b70 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c680cd540 0x7f8c680c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 --2- 172.21.15.45:0/3049206522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c680cf950 0x7f8c680c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 >> 172.21.15.45:0/3049206522 conn(0x7f8c6809a1a0 msgr2=0x7f8c680c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:46.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.339+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 shutdown_connections 2023-08-27T11:27:46.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.340+0000 7f8c6db3f700 1 -- 172.21.15.45:0/3049206522 wait complete. 2023-08-27T11:27:46.357 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:27:46.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:46.404 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:27:46.447 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:27:46.447 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:27:46.448 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:46.466 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:27:46.466 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:46.484 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:46.482+0000 7fe8df8c2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:27:46.488 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:46.488+0000 7fe70ddea200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:27:46.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:46.550 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:46.548+0000 7fe8df8c2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:46.558 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:46.558+0000 7fe70ddea200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:46.622 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:46.620+0000 7fe8df8c2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:27:46.634 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:46.633+0000 7fe70ddea200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:27:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.672+0000 7f7d1caf0700 1 Processor -- start 2023-08-27T11:27:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.672+0000 7f7d1caf0700 1 -- start start 2023-08-27T11:27:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.673+0000 7f7d1caf0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7d18158cb0 0x7f7d180ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.673+0000 7f7d1caf0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d1806ac60 0x7f7d1806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.673+0000 7f7d1caf0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1806d3a0 0x7f7d1815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.673+0000 7f7d1caf0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1806b230 con 0x7f7d1806ac60 2023-08-27T11:27:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.673+0000 7f7d1caf0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1815b780 con 0x7f7d18158cb0 2023-08-27T11:27:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.673+0000 7f7d1caf0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1815ba20 con 0x7f7d1806d3a0 2023-08-27T11:27:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d15d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d1806ac60 0x7f7d1806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d15d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d1806ac60 0x7f7d1806ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58302/0 (socket says 172.21.15.45:58302) 2023-08-27T11:27:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d15d9b700 1 -- 172.21.15.45:0/708187148 learned_addr learned my addr 172.21.15.45:0/708187148 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d16d9d700 1 --2- 172.21.15.45:0/708187148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1806d3a0 0x7f7d1815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:46.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d1659c700 1 --2- 172.21.15.45:0/708187148 >> v2:172.21.15.135:3300/0 conn(0x7f7d18158cb0 0x7f7d180ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:46.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d15d9b700 1 -- 172.21.15.45:0/708187148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1806d3a0 msgr2=0x7f7d1815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:46.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d15d9b700 1 --2- 172.21.15.45:0/708187148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1806d3a0 0x7f7d1815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d15d9b700 1 -- 172.21.15.45:0/708187148 >> v2:172.21.15.135:3300/0 conn(0x7f7d18158cb0 msgr2=0x7f7d180ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:46.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d15d9b700 1 --2- 172.21.15.45:0/708187148 >> v2:172.21.15.135:3300/0 conn(0x7f7d18158cb0 0x7f7d180ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d15d9b700 1 -- 172.21.15.45:0/708187148 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d180cc5c0 con 0x7f7d1806ac60 2023-08-27T11:27:46.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d15d9b700 1 --2- 172.21.15.45:0/708187148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 0x7f7d1806ce60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7d18072a00 tx=0x7f7d0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1969793e51c56ba server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:46.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d1559a700 1 -- 172.21.15.45:0/708187148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c011830 con 0x7f7d1806ac60 2023-08-27T11:27:46.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d1559a700 1 -- 172.21.15.45:0/708187148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c011990 con 0x7f7d1806ac60 2023-08-27T11:27:46.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.674+0000 7f7d1559a700 1 -- 172.21.15.45:0/708187148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c011d20 con 0x7f7d1806ac60 2023-08-27T11:27:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.675+0000 7f7d1caf0700 1 -- 172.21.15.45:0/708187148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 msgr2=0x7f7d1806ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.675+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/708187148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 0x7f7d1806ce60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7d18072a00 tx=0x7f7d0c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.675+0000 7f7d1caf0700 1 -- 172.21.15.45:0/708187148 shutdown_connections 2023-08-27T11:27:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.675+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/708187148 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d1806d3a0 0x7f7d1815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.675+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/708187148 >> v2:172.21.15.135:3300/0 conn(0x7f7d18158cb0 0x7f7d180ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.675+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/708187148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 0x7f7d1806ce60 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.675+0000 7f7d1caf0700 1 -- 172.21.15.45:0/708187148 >> 172.21.15.45:0/708187148 conn(0x7f7d180cfd60 msgr2=0x7f7d180d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.675+0000 7f7d1caf0700 1 -- 172.21.15.45:0/708187148 shutdown_connections 2023-08-27T11:27:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.675+0000 7f7d1caf0700 1 -- 172.21.15.45:0/708187148 wait complete. 2023-08-27T11:27:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.677+0000 7f7d1caf0700 1 Processor -- start 2023-08-27T11:27:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.677+0000 7f7d1caf0700 1 -- start start 2023-08-27T11:27:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1caf0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 0x7f7d180c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1caf0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1806d3a0 0x7f7d180c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1caf0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d18158cb0 0x7f7d181f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1caf0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180c55e0 con 0x7f7d1806ac60 2023-08-27T11:27:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1caf0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180c4c10 con 0x7f7d1806d3a0 2023-08-27T11:27:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1caf0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180c48e0 con 0x7f7d18158cb0 2023-08-27T11:27:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 0x7f7d180c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 0x7f7d180c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58308/0 (socket says 172.21.15.45:58308) 2023-08-27T11:27:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1659c700 1 -- 172.21.15.45:0/3188559963 learned_addr learned my addr 172.21.15.45:0/3188559963 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d15d9b700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1806d3a0 0x7f7d180c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d16d9d700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d18158cb0 0x7f7d181f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1659c700 1 -- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d18158cb0 msgr2=0x7f7d181f3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1659c700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d18158cb0 0x7f7d181f3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1659c700 1 -- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1806d3a0 msgr2=0x7f7d180c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1659c700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1806d3a0 0x7f7d180c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1659c700 1 -- 172.21.15.45:0/3188559963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c009550 con 0x7f7d1806ac60 2023-08-27T11:27:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1659c700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 0x7f7d180c50a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800e870 tx=0x7f7d0800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d06ffd700 1 -- 172.21.15.45:0/3188559963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08009590 con 0x7f7d1806ac60 2023-08-27T11:27:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d06ffd700 1 -- 172.21.15.45:0/3188559963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08013680 con 0x7f7d1806ac60 2023-08-27T11:27:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d180b30c0 con 0x7f7d1806ac60 2023-08-27T11:27:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.678+0000 7f7d06ffd700 1 -- 172.21.15.45:0/3188559963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d080139f0 con 0x7f7d1806ac60 2023-08-27T11:27:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.679+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d180b2de0 con 0x7f7d1806ac60 2023-08-27T11:27:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.679+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d180c4ef0 con 0x7f7d1806ac60 2023-08-27T11:27:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.680+0000 7f7d06ffd700 1 -- 172.21.15.45:0/3188559963 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f7d08013b50 con 0x7f7d1806ac60 2023-08-27T11:27:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.680+0000 7f7d06ffd700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f7cf806a200 0x7f7cf806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.681+0000 7f7d15d9b700 1 -- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f7cf806a200 msgr2=0x7f7cf806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.681+0000 7f7d06ffd700 1 -- 172.21.15.45:0/3188559963 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f7d080887b0 con 0x7f7d1806ac60 2023-08-27T11:27:46.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.681+0000 7f7d15d9b700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f7cf806a200 0x7f7cf806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:46.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.685+0000 7f7d06ffd700 1 -- 172.21.15.45:0/3188559963 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7d08057550 con 0x7f7d1806ac60 2023-08-27T11:27:46.700 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:46.698+0000 7fe8df8c2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:27:46.720 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:46.719+0000 7fe70ddea200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:27:46.842 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:46.839+0000 7fe8df8c2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:27:46.869 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:46.868+0000 7fe70ddea200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:27:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.882+0000 7f7d15d9b700 1 -- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f7cf806a200 msgr2=0x7f7cf806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.882+0000 7f7d15d9b700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f7cf806a200 0x7f7cf806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:46.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.887+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d180c4740 con 0x7f7d1806ac60 2023-08-27T11:27:46.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.888+0000 7f7d06ffd700 1 -- 172.21.15.45:0/3188559963 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+19144 (secure 0 0 0) 0x7f7d180c4740 con 0x7f7d1806ac60 2023-08-27T11:27:46.889 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:46.890 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":87,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:44.271663+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:23.082458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:24.453679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/790246089":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2253320845":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2129685370":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/289717412":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6833/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6832/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2380106040":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/4054422326":"2023-08-28T11:27:44.271608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:46.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f7cf806a200 msgr2=0x7f7cf806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:46.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f7cf806a200 0x7f7cf806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 msgr2=0x7f7d180c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:46.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 0x7f7d180c50a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800e870 tx=0x7f7d0800eb80 comp rx=0 tx=0).stop 2023-08-27T11:27:46.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 shutdown_connections 2023-08-27T11:27:46.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f7cf806a200 0x7f7cf806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1806ac60 0x7f7d180c50a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7d18158cb0 0x7f7d181f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 --2- 172.21.15.45:0/3188559963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d1806d3a0 0x7f7d180c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:46.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.890+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 >> 172.21.15.45:0/3188559963 conn(0x7f7d180cfd60 msgr2=0x7f7d18157aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:46.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.891+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 shutdown_connections 2023-08-27T11:27:46.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:46.891+0000 7f7d1caf0700 1 -- 172.21.15.45:0/3188559963 wait complete. 2023-08-27T11:27:46.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:27:23.082458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:27:24.453679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:46.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:47.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.075+0000 7f273cedf700 1 Processor -- start 2023-08-27T11:27:47.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.075+0000 7f273cedf700 1 -- start start 2023-08-27T11:27:47.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.076+0000 7f273cedf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27380ae630 0x7f2738073a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.076+0000 7f273cedf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27380723a0 con 0x7f273806a280 2023-08-27T11:27:47.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.076+0000 7f273cedf700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27380cbdf0 con 0x7f273815d560 2023-08-27T11:27:47.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.076+0000 7f273cedf700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273815d940 con 0x7f27380ae630 2023-08-27T11:27:47.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.076+0000 7f2735d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27380ae630 0x7f2738073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.076+0000 7f2736d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f273806a280 0x7f273806bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45132/0 (socket says 172.21.15.45:45132) 2023-08-27T11:27:47.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.076+0000 7f2735d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27380ae630 0x7f2738073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58354/0 (socket says 172.21.15.45:58354) 2023-08-27T11:27:47.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.076+0000 7f2736d9d700 1 -- 172.21.15.45:0/3465778544 learned_addr learned my addr 172.21.15.45:0/3465778544 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:47.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.077+0000 7f2735d9b700 1 -- 172.21.15.45:0/3465778544 >> v1:172.21.15.135:6789/0 conn(0x7f273815d560 legacy=0x7f27380cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.077+0000 7f2735d9b700 1 -- 172.21.15.45:0/3465778544 >> v1:172.21.15.45:6789/0 conn(0x7f273806a280 legacy=0x7f273806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.077+0000 7f2735d9b700 1 -- 172.21.15.45:0/3465778544 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27380cccc0 con 0x7f27380ae630 2023-08-27T11:27:47.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.077+0000 7f2735d9b700 1 --2- 172.21.15.45:0/3465778544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27380ae630 0x7f2738073a10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f272c009840 tx=0x7f272c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb984d46a9947924 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:47.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.077+0000 7f273559a700 1 -- 172.21.15.45:0/3465778544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c011830 con 0x7f27380ae630 2023-08-27T11:27:47.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.077+0000 7f273559a700 1 -- 172.21.15.45:0/3465778544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c011990 con 0x7f27380ae630 2023-08-27T11:27:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.077+0000 7f273559a700 1 -- 172.21.15.45:0/3465778544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c011d20 con 0x7f27380ae630 2023-08-27T11:27:47.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.078+0000 7f273cedf700 1 -- 172.21.15.45:0/3465778544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27380ae630 msgr2=0x7f2738073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.078+0000 7f273cedf700 1 --2- 172.21.15.45:0/3465778544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27380ae630 0x7f2738073a10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f272c009840 tx=0x7f272c009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:47.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.078+0000 7f273cedf700 1 -- 172.21.15.45:0/3465778544 shutdown_connections 2023-08-27T11:27:47.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.078+0000 7f273cedf700 1 --2- 172.21.15.45:0/3465778544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27380ae630 0x7f2738073a10 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.078+0000 7f273cedf700 1 -- 172.21.15.45:0/3465778544 >> 172.21.15.45:0/3465778544 conn(0x7f27380b2420 msgr2=0x7f27380b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:47.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.078+0000 7f273cedf700 1 -- 172.21.15.45:0/3465778544 shutdown_connections 2023-08-27T11:27:47.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.078+0000 7f273cedf700 1 -- 172.21.15.45:0/3465778544 wait complete. 2023-08-27T11:27:47.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.079+0000 7f273cedf700 1 Processor -- start 2023-08-27T11:27:47.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.079+0000 7f273cedf700 1 -- start start 2023-08-27T11:27:47.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273cedf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806a280 0x7f27381f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273cedf700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27380ae630 0x7f273815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273cedf700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f273815d560 0x7f27381604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273cedf700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27380b0ff0 con 0x7f273806a280 2023-08-27T11:27:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273cedf700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2738159f90 con 0x7f27380ae630 2023-08-27T11:27:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273cedf700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27380b0930 con 0x7f273815d560 2023-08-27T11:27:47.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806a280 0x7f27381f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806a280 0x7f27381f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58358/0 (socket says 172.21.15.45:58358) 2023-08-27T11:27:47.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273659c700 1 -- 172.21.15.45:0/2658973932 learned_addr learned my addr 172.21.15.45:0/2658973932 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:47.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f2735d9b700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27380ae630 0x7f273815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f2736d9d700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f273815d560 0x7f27381604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273659c700 1 -- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f273815d560 msgr2=0x7f27381604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273659c700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f273815d560 0x7f27381604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273659c700 1 -- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27380ae630 msgr2=0x7f273815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273659c700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27380ae630 0x7f273815ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.080+0000 7f273659c700 1 -- 172.21.15.45:0/2658973932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c0094a0 con 0x7f273806a280 2023-08-27T11:27:47.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.081+0000 7f273659c700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806a280 0x7f27381f2940 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f272800c430 tx=0x7f272800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:47.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.081+0000 7f2726ffd700 1 -- 172.21.15.45:0/2658973932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27280052d0 con 0x7f273806a280 2023-08-27T11:27:47.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.081+0000 7f2726ffd700 1 -- 172.21.15.45:0/2658973932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2728005450 con 0x7f273806a280 2023-08-27T11:27:47.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.081+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27380b0c10 con 0x7f273806a280 2023-08-27T11:27:47.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.081+0000 7f2726ffd700 1 -- 172.21.15.45:0/2658973932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728006af0 con 0x7f273806a280 2023-08-27T11:27:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.081+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27381f2e80 con 0x7f273806a280 2023-08-27T11:27:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.082+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273804e620 con 0x7f273806a280 2023-08-27T11:27:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.082+0000 7f2726ffd700 1 -- 172.21.15.45:0/2658973932 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f27280036d0 con 0x7f273806a280 2023-08-27T11:27:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.083+0000 7f2726ffd700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f271806a200 0x7f271806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.083+0000 7f2735d9b700 1 -- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f271806a200 msgr2=0x7f271806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.083+0000 7f2726ffd700 1 -- 172.21.15.45:0/2658973932 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f2728087a20 con 0x7f273806a280 2023-08-27T11:27:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.083+0000 7f2735d9b700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f271806a200 0x7f271806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.087+0000 7f2726ffd700 1 -- 172.21.15.45:0/2658973932 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2728052dc0 con 0x7f273806a280 2023-08-27T11:27:47.166 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:47.164+0000 7fe8df8c2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:27:47.219 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:47.218+0000 7fe70ddea200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:27:47.281 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:47.279+0000 7fe8df8c2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:27:47.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.284+0000 7f2735d9b700 1 -- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f271806a200 msgr2=0x7f271806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:47.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.284+0000 7f2735d9b700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f271806a200 0x7f271806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:47.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.292+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27380c5f60 con 0x7f273806a280 2023-08-27T11:27:47.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.293+0000 7f2726ffd700 1 -- 172.21.15.45:0/2658973932 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27280563e0 con 0x7f273806a280 2023-08-27T11:27:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:27:47.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f271806a200 msgr2=0x7f271806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:47.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f271806a200 0x7f271806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806a280 msgr2=0x7f27381f2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806a280 0x7f27381f2940 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f272800c430 tx=0x7f272800c740 comp rx=0 tx=0).stop 2023-08-27T11:27:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 shutdown_connections 2023-08-27T11:27:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f271806a200 0x7f271806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273806a280 0x7f27381f2940 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f273815d560 0x7f27381604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 --2- 172.21.15.45:0/2658973932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27380ae630 0x7f273815ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 >> 172.21.15.45:0/2658973932 conn(0x7f27380b2420 msgr2=0x7f27380cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:47.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 shutdown_connections 2023-08-27T11:27:47.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.295+0000 7f273cedf700 1 -- 172.21.15.45:0/2658973932 wait complete. 2023-08-27T11:27:47.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:47.342 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:47.341+0000 7fe70ddea200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:27:47.356 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:47.354+0000 7fe8df8c2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:27:47.421 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:47.420+0000 7fe70ddea200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:27:47.437 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:47.435+0000 7fe8df8c2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:27:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.439+0000 7fe12cfba700 1 Processor -- start 2023-08-27T11:27:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.439+0000 7fe12cfba700 1 -- start start 2023-08-27T11:27:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe12cfba700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe128160890 0x7fe1280cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe12cfba700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1280b5220 0x7fe1280d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe12cfba700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1280cdf40 con 0x7fe1280b62c0 2023-08-27T11:27:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe12cfba700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1280d0fd0 con 0x7fe1280b5220 2023-08-27T11:27:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe12cfba700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1280ce190 con 0x7fe128160890 2023-08-27T11:27:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe126d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1280b5220 0x7fe1280d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe126d9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe1280b5220 0x7fe1280d1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58372/0 (socket says 172.21.15.45:58372) 2023-08-27T11:27:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe126d9d700 1 -- 172.21.15.45:0/725760774 learned_addr learned my addr 172.21.15.45:0/725760774 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe12659c700 1 --2- 172.21.15.45:0/725760774 >> v2:172.21.15.135:3300/0 conn(0x7fe128160890 0x7fe1280cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe126d9d700 1 -- 172.21.15.45:0/725760774 >> v2:172.21.15.135:3300/0 conn(0x7fe128160890 msgr2=0x7fe1280cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe126d9d700 1 --2- 172.21.15.45:0/725760774 >> v2:172.21.15.135:3300/0 conn(0x7fe128160890 0x7fe1280cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe126d9d700 1 -- 172.21.15.45:0/725760774 >> v1:172.21.15.135:6789/0 conn(0x7fe1280b62c0 legacy=0x7fe1280b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.440+0000 7fe126d9d700 1 -- 172.21.15.45:0/725760774 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280ce470 con 0x7fe1280b5220 2023-08-27T11:27:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.441+0000 7fe126d9d700 1 --2- 172.21.15.45:0/725760774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1280b5220 0x7fe1280d1460 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe12808bf00 tx=0x7fe12000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787a4fc36a2ddcd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.441+0000 7fe12559a700 1 -- 172.21.15.45:0/725760774 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000e070 con 0x7fe1280b5220 2023-08-27T11:27:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.441+0000 7fe12559a700 1 -- 172.21.15.45:0/725760774 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12000b910 con 0x7fe1280b5220 2023-08-27T11:27:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.441+0000 7fe12559a700 1 -- 172.21.15.45:0/725760774 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000bcc0 con 0x7fe1280b5220 2023-08-27T11:27:47.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.441+0000 7fe12cfba700 1 -- 172.21.15.45:0/725760774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1280b5220 msgr2=0x7fe1280d1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.441+0000 7fe12cfba700 1 --2- 172.21.15.45:0/725760774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1280b5220 0x7fe1280d1460 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe12808bf00 tx=0x7fe12000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:47.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.441+0000 7fe12cfba700 1 -- 172.21.15.45:0/725760774 shutdown_connections 2023-08-27T11:27:47.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.441+0000 7fe12cfba700 1 --2- 172.21.15.45:0/725760774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1280b5220 0x7fe1280d1460 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.442+0000 7fe12cfba700 1 --2- 172.21.15.45:0/725760774 >> v2:172.21.15.135:3300/0 conn(0x7fe128160890 0x7fe1280cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.442+0000 7fe12cfba700 1 -- 172.21.15.45:0/725760774 >> 172.21.15.45:0/725760774 conn(0x7fe1280cfb10 msgr2=0x7fe1280cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.442+0000 7fe12cfba700 1 -- 172.21.15.45:0/725760774 shutdown_connections 2023-08-27T11:27:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.442+0000 7fe12cfba700 1 -- 172.21.15.45:0/725760774 wait complete. 2023-08-27T11:27:47.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.443+0000 7fe12cfba700 1 Processor -- start 2023-08-27T11:27:47.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.443+0000 7fe12cfba700 1 -- start start 2023-08-27T11:27:47.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe12cfba700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1280b5220 0x7fe12814bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe12cfba700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1280b62c0 0x7fe12814b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe12cfba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe128160890 0x7fe12814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe12cfba700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12814c240 con 0x7fe128160890 2023-08-27T11:27:47.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe12cfba700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12814c520 con 0x7fe1280b62c0 2023-08-27T11:27:47.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe12cfba700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12814c800 con 0x7fe1280b5220 2023-08-27T11:27:47.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe126d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe128160890 0x7fe12814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe126d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe128160890 0x7fe12814d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58378/0 (socket says 172.21.15.45:58378) 2023-08-27T11:27:47.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe126d9d700 1 -- 172.21.15.45:0/3643764537 learned_addr learned my addr 172.21.15.45:0/3643764537 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:47.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe125d9b700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1280b62c0 0x7fe12814b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe12659c700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1280b5220 0x7fe12814bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe126d9d700 1 -- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1280b5220 msgr2=0x7fe12814bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe126d9d700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1280b5220 0x7fe12814bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe126d9d700 1 -- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1280b62c0 msgr2=0x7fe12814b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe126d9d700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1280b62c0 0x7fe12814b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe126d9d700 1 -- 172.21.15.45:0/3643764537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe120009b30 con 0x7fe128160890 2023-08-27T11:27:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe126d9d700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe128160890 0x7fe12814d2e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe120012000 tx=0x7fe120004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.444+0000 7fe116ffd700 1 -- 172.21.15.45:0/3643764537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120004880 con 0x7fe128160890 2023-08-27T11:27:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.445+0000 7fe116ffd700 1 -- 172.21.15.45:0/3643764537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1200099b0 con 0x7fe128160890 2023-08-27T11:27:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.445+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12814cae0 con 0x7fe128160890 2023-08-27T11:27:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.445+0000 7fe116ffd700 1 -- 172.21.15.45:0/3643764537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12001c680 con 0x7fe128160890 2023-08-27T11:27:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.445+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12814b830 con 0x7fe128160890 2023-08-27T11:27:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.446+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12814cfa0 con 0x7fe128160890 2023-08-27T11:27:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.446+0000 7fe116ffd700 1 -- 172.21.15.45:0/3643764537 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fe12001c7e0 con 0x7fe128160890 2023-08-27T11:27:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.447+0000 7fe116ffd700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fe10806a200 0x7fe10806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.447+0000 7fe116ffd700 1 -- 172.21.15.45:0/3643764537 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fe12008a2c0 con 0x7fe128160890 2023-08-27T11:27:47.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.447+0000 7fe12659c700 1 -- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fe10806a200 msgr2=0x7fe10806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:47.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.447+0000 7fe12659c700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fe10806a200 0x7fe10806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:47.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.451+0000 7fe116ffd700 1 -- 172.21.15.45:0/3643764537 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe120055550 con 0x7fe128160890 2023-08-27T11:27:47.508 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:47.507+0000 7fe70ddea200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:27:47.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.648+0000 7fe12659c700 1 -- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fe10806a200 msgr2=0x7fe10806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:47.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.648+0000 7fe12659c700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fe10806a200 0x7fe10806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:47.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.654+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe12806c120 con 0x7fe128160890 2023-08-27T11:27:47.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.655+0000 7fe116ffd700 1 -- 172.21.15.45:0/3643764537 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe120058b70 con 0x7fe128160890 2023-08-27T11:27:47.656 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:27:47.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fe10806a200 msgr2=0x7fe10806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:47.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fe10806a200 0x7fe10806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe128160890 msgr2=0x7fe12814d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe128160890 0x7fe12814d2e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe120012000 tx=0x7fe120004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 shutdown_connections 2023-08-27T11:27:47.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fe10806a200 0x7fe10806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe128160890 0x7fe12814d2e0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe1280b5220 0x7fe12814bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 --2- 172.21.15.45:0/3643764537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1280b62c0 0x7fe12814b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 >> 172.21.15.45:0/3643764537 conn(0x7fe1280cfb10 msgr2=0x7fe1280d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:47.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 shutdown_connections 2023-08-27T11:27:47.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.657+0000 7fe12cfba700 1 -- 172.21.15.45:0/3643764537 wait complete. 2023-08-27T11:27:47.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:47.809 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:47.807+0000 7fe8df8c2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:27:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.840+0000 7f689d0b4700 1 Processor -- start 2023-08-27T11:27:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.840+0000 7f689d0b4700 1 -- start start 2023-08-27T11:27:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f689d0b4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689815af30 0x7f68980cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f689d0b4700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68980cde80 0x7f6898164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f689d0b4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68980cb8b0 con 0x7f68980cf960 2023-08-27T11:27:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f689d0b4700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689806f9c0 con 0x7f68980cde80 2023-08-27T11:27:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f689d0b4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980cecd0 con 0x7f689815af30 2023-08-27T11:27:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f689659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f68980cf960 0x7f68980cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45184/0 (socket says 172.21.15.45:45184) 2023-08-27T11:27:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f689659c700 1 -- 172.21.15.45:0/701886337 learned_addr learned my addr 172.21.15.45:0/701886337 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f6896d9d700 1 --2- 172.21.15.45:0/701886337 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689815af30 0x7f68980cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f689759e700 1 --2- 172.21.15.45:0/701886337 >> v2:172.21.15.135:3300/0 conn(0x7f68980cde80 0x7f6898164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f6895d9b700 1 -- 172.21.15.45:0/701886337 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785153879 0 0) 0x7f68980cb8b0 con 0x7f68980cf960 2023-08-27T11:27:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.841+0000 7f6895d9b700 1 -- 172.21.15.45:0/701886337 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68800035b0 con 0x7f68980cf960 2023-08-27T11:27:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.842+0000 7f689759e700 1 -- 172.21.15.45:0/701886337 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689815af30 msgr2=0x7f68980cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.842+0000 7f689759e700 1 --2- 172.21.15.45:0/701886337 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689815af30 0x7f68980cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.842+0000 7f6895d9b700 1 -- 172.21.15.45:0/701886337 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4280868043 0 0) 0x7f68800035b0 con 0x7f68980cf960 2023-08-27T11:27:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.842+0000 7f689759e700 1 -- 172.21.15.45:0/701886337 >> v1:172.21.15.45:6789/0 conn(0x7f68980cf960 legacy=0x7f68980cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.842+0000 7f689759e700 1 -- 172.21.15.45:0/701886337 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68980cbae0 con 0x7f68980cde80 2023-08-27T11:27:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.842+0000 7f689759e700 1 --2- 172.21.15.45:0/701886337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68980cde80 0x7f6898164fe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6890009d90 tx=0x7f689000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bcdeac248a8766e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.842+0000 7f6895d9b700 1 -- 172.21.15.45:0/701886337 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689000e070 con 0x7f68980cde80 2023-08-27T11:27:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.842+0000 7f6895d9b700 1 -- 172.21.15.45:0/701886337 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6890003a20 con 0x7f68980cde80 2023-08-27T11:27:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.842+0000 7f6895d9b700 1 -- 172.21.15.45:0/701886337 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c001670 con 0x7f68980cde80 2023-08-27T11:27:47.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.843+0000 7f689d0b4700 1 -- 172.21.15.45:0/701886337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68980cde80 msgr2=0x7f6898164fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.843+0000 7f689d0b4700 1 --2- 172.21.15.45:0/701886337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68980cde80 0x7f6898164fe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6890009d90 tx=0x7f689000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.843+0000 7f689d0b4700 1 -- 172.21.15.45:0/701886337 shutdown_connections 2023-08-27T11:27:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.843+0000 7f689d0b4700 1 --2- 172.21.15.45:0/701886337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68980cde80 0x7f6898164fe0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.843+0000 7f689d0b4700 1 --2- 172.21.15.45:0/701886337 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f689815af30 0x7f68980cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.843+0000 7f689d0b4700 1 -- 172.21.15.45:0/701886337 >> 172.21.15.45:0/701886337 conn(0x7f689809a1a0 msgr2=0x7f68980d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.843+0000 7f689d0b4700 1 -- 172.21.15.45:0/701886337 shutdown_connections 2023-08-27T11:27:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.843+0000 7f689d0b4700 1 -- 172.21.15.45:0/701886337 wait complete. 2023-08-27T11:27:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.844+0000 7f689d0b4700 1 Processor -- start 2023-08-27T11:27:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.844+0000 7f689d0b4700 1 -- start start 2023-08-27T11:27:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689d0b4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980cde80 0x7f68981f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689d0b4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68980cf960 0x7f689815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689d0b4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f689815af30 0x7f68980cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689d0b4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980b68d0 con 0x7f68980cf960 2023-08-27T11:27:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689d0b4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980b5cc0 con 0x7f689815af30 2023-08-27T11:27:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689d0b4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980b5fa0 con 0x7f68980cde80 2023-08-27T11:27:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68980cf960 0x7f689815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68980cf960 0x7f689815cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58414/0 (socket says 172.21.15.45:58414) 2023-08-27T11:27:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689659c700 1 -- 172.21.15.45:0/3176336284 learned_addr learned my addr 172.21.15.45:0/3176336284 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f689759e700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f689815af30 0x7f68980cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.845+0000 7f6896d9d700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980cde80 0x7f68981f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f689659c700 1 -- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980cde80 msgr2=0x7f68981f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f689659c700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980cde80 0x7f68981f00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f689659c700 1 -- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f689815af30 msgr2=0x7f68980cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f689659c700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f689815af30 0x7f68980cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f689659c700 1 -- 172.21.15.45:0/3176336284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890009a70 con 0x7f68980cf960 2023-08-27T11:27:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f689659c700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68980cf960 0x7f689815cd70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c780 tx=0x7f688c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f687f7fe700 1 -- 172.21.15.45:0/3176336284 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c004c80 con 0x7f68980cf960 2023-08-27T11:27:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f687f7fe700 1 -- 172.21.15.45:0/3176336284 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c004de0 con 0x7f68980cf960 2023-08-27T11:27:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68980b6280 con 0x7f68980cf960 2023-08-27T11:27:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f687f7fe700 1 -- 172.21.15.45:0/3176336284 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c0072d0 con 0x7f68980cf960 2023-08-27T11:27:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.846+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68980b1bc0 con 0x7f68980cf960 2023-08-27T11:27:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.847+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68980734a0 con 0x7f68980cf960 2023-08-27T11:27:47.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.848+0000 7f687f7fe700 1 -- 172.21.15.45:0/3176336284 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f688c007430 con 0x7f68980cf960 2023-08-27T11:27:47.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.848+0000 7f687f7fe700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f688406a200 0x7f688406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:47.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.848+0000 7f687f7fe700 1 -- 172.21.15.45:0/3176336284 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f688c088700 con 0x7f68980cf960 2023-08-27T11:27:47.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.848+0000 7f6896d9d700 1 -- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f688406a200 msgr2=0x7f688406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:47.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.848+0000 7f6896d9d700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f688406a200 0x7f688406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:47.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:47.857+0000 7f687f7fe700 1 -- 172.21.15.45:0/3176336284 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f688c053a10 con 0x7f68980cf960 2023-08-27T11:27:47.908 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:47.907+0000 7fe70ddea200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:27:48.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.049+0000 7f6896d9d700 1 -- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f688406a200 msgr2=0x7f688406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.049+0000 7f6896d9d700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f688406a200 0x7f688406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:48.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.063+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68980ce4c0 con 0x7f68980cf960 2023-08-27T11:27:48.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.063+0000 7f687f7fe700 1 -- 172.21.15.45:0/3176336284 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f688c057030 con 0x7f68980cf960 2023-08-27T11:27:48.064 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:27:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f688406a200 msgr2=0x7f688406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f688406a200 0x7f688406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68980cf960 msgr2=0x7f689815cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68980cf960 0x7f689815cd70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c780 tx=0x7f688c00ca90 comp rx=0 tx=0).stop 2023-08-27T11:27:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 shutdown_connections 2023-08-27T11:27:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f689815af30 0x7f68980cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68980cde80 0x7f68981f00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f688406a200 0x7f688406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 --2- 172.21.15.45:0/3176336284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68980cf960 0x7f689815cd70 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 >> 172.21.15.45:0/3176336284 conn(0x7f689809a1a0 msgr2=0x7f68980af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:48.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 shutdown_connections 2023-08-27T11:27:48.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.066+0000 7f689d0b4700 1 -- 172.21.15.45:0/3176336284 wait complete. 2023-08-27T11:27:48.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.208+0000 7f137e786700 1 Processor -- start 2023-08-27T11:27:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.208+0000 7f137e786700 1 -- start start 2023-08-27T11:27:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f137e786700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b5220 0x7f13780cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f137e786700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f13780b62c0 0x7f13780d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f137e786700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13780cdf40 con 0x7f1378160890 2023-08-27T11:27:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f137e786700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13780d0fd0 con 0x7f13780b62c0 2023-08-27T11:27:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f137e786700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13780ce190 con 0x7f13780b5220 2023-08-27T11:27:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f1377fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1378160890 0x7f13780b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45198/0 (socket says 172.21.15.45:45198) 2023-08-27T11:27:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f1377fff700 1 -- 172.21.15.45:0/3197669165 learned_addr learned my addr 172.21.15.45:0/3197669165 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f13777fe700 1 --2- 172.21.15.45:0/3197669165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b5220 0x7f13780cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f137cd23700 1 --2- 172.21.15.45:0/3197669165 >> v2:172.21.15.135:3300/0 conn(0x7f13780b62c0 0x7f13780d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f1376ffd700 1 -- 172.21.15.45:0/3197669165 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185484000 0 0) 0x7f13780cdf40 con 0x7f1378160890 2023-08-27T11:27:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.209+0000 7f1376ffd700 1 -- 172.21.15.45:0/3197669165 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13600035b0 con 0x7f1378160890 2023-08-27T11:27:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.210+0000 7f13777fe700 1 -- 172.21.15.45:0/3197669165 >> v2:172.21.15.135:3300/0 conn(0x7f13780b62c0 msgr2=0x7f13780d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.210+0000 7f13777fe700 1 --2- 172.21.15.45:0/3197669165 >> v2:172.21.15.135:3300/0 conn(0x7f13780b62c0 0x7f13780d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.210+0000 7f13777fe700 1 -- 172.21.15.45:0/3197669165 >> v1:172.21.15.45:6789/0 conn(0x7f1378160890 legacy=0x7f13780b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.210+0000 7f13777fe700 1 -- 172.21.15.45:0/3197669165 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13780ce470 con 0x7f13780b5220 2023-08-27T11:27:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.210+0000 7f137cd23700 1 --2- 172.21.15.45:0/3197669165 >> v2:172.21.15.135:3300/0 conn(0x7f13780b62c0 0x7f13780d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.210+0000 7f13777fe700 1 --2- 172.21.15.45:0/3197669165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b5220 0x7f13780cd8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f136c009840 tx=0x7f136c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=223e20a1b0a8b99b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.210+0000 7f1376ffd700 1 -- 172.21.15.45:0/3197669165 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c011830 con 0x7f13780b5220 2023-08-27T11:27:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.210+0000 7f1376ffd700 1 -- 172.21.15.45:0/3197669165 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c011990 con 0x7f13780b5220 2023-08-27T11:27:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.210+0000 7f1376ffd700 1 -- 172.21.15.45:0/3197669165 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c011d20 con 0x7f13780b5220 2023-08-27T11:27:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.211+0000 7f137e786700 1 -- 172.21.15.45:0/3197669165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b5220 msgr2=0x7f13780cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.211+0000 7f137e786700 1 --2- 172.21.15.45:0/3197669165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b5220 0x7f13780cd8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f136c009840 tx=0x7f136c009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.211+0000 7f137e786700 1 -- 172.21.15.45:0/3197669165 shutdown_connections 2023-08-27T11:27:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.211+0000 7f137e786700 1 --2- 172.21.15.45:0/3197669165 >> v2:172.21.15.135:3300/0 conn(0x7f13780b62c0 0x7f13780d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.211+0000 7f137e786700 1 --2- 172.21.15.45:0/3197669165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b5220 0x7f13780cd8f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.211+0000 7f137e786700 1 -- 172.21.15.45:0/3197669165 >> 172.21.15.45:0/3197669165 conn(0x7f13780cfb10 msgr2=0x7f13780cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.211+0000 7f137e786700 1 -- 172.21.15.45:0/3197669165 shutdown_connections 2023-08-27T11:27:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.211+0000 7f137e786700 1 -- 172.21.15.45:0/3197669165 wait complete. 2023-08-27T11:27:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.212+0000 7f137e786700 1 Processor -- start 2023-08-27T11:27:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.213+0000 7f137e786700 1 -- start start 2023-08-27T11:27:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f137e786700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13780b5220 0x7f13780cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f137e786700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b62c0 0x7f137814b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f137e786700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1378160890 0x7f137814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f137e786700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137815d020 con 0x7f13780b5220 2023-08-27T11:27:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f137e786700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137814bb00 con 0x7f1378160890 2023-08-27T11:27:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f137e786700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137815c450 con 0x7f13780b62c0 2023-08-27T11:27:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f1377fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13780b5220 0x7f13780cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f13777fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b62c0 0x7f137814b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f13777fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b62c0 0x7f137814b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:56340/0 (socket says 172.21.15.45:56340) 2023-08-27T11:27:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f1377fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13780b5220 0x7f13780cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58428/0 (socket says 172.21.15.45:58428) 2023-08-27T11:27:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f13777fe700 1 -- 172.21.15.45:0/2441462665 learned_addr learned my addr 172.21.15.45:0/2441462665 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f137cd23700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1378160890 0x7f137814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f13777fe700 1 -- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1378160890 msgr2=0x7f137814b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f13777fe700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1378160890 0x7f137814b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f13777fe700 1 -- 172.21.15.45:0/2441462665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13780b5220 msgr2=0x7f13780cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f13777fe700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13780b5220 0x7f13780cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f13777fe700 1 -- 172.21.15.45:0/2441462665 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1370009a70 con 0x7f13780b62c0 2023-08-27T11:27:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.214+0000 7f13777fe700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b62c0 0x7f137814b080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f136c000c00 tx=0x7f136c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.215+0000 7f1374ff9700 1 -- 172.21.15.45:0/2441462665 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c004550 con 0x7f13780b62c0 2023-08-27T11:27:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.215+0000 7f1374ff9700 1 -- 172.21.15.45:0/2441462665 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c01f070 con 0x7f13780b62c0 2023-08-27T11:27:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.215+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c0094a0 con 0x7f13780b62c0 2023-08-27T11:27:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.215+0000 7f1374ff9700 1 -- 172.21.15.45:0/2441462665 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0038c0 con 0x7f13780b62c0 2023-08-27T11:27:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.215+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137815c9d0 con 0x7f13780b62c0 2023-08-27T11:27:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.216+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137806c120 con 0x7f13780b62c0 2023-08-27T11:27:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.217+0000 7f1374ff9700 1 -- 172.21.15.45:0/2441462665 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f136c003a20 con 0x7f13780b62c0 2023-08-27T11:27:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.217+0000 7f1374ff9700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f136406a200 0x7f136406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.217+0000 7f1374ff9700 1 -- 172.21.15.45:0/2441462665 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f136c021070 con 0x7f13780b62c0 2023-08-27T11:27:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.217+0000 7f1377fff700 1 -- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f136406a200 msgr2=0x7f136406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.217+0000 7f1377fff700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f136406a200 0x7f136406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.224+0000 7f1374ff9700 1 -- 172.21.15.45:0/2441462665 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f136c0555f0 con 0x7f13780b62c0 2023-08-27T11:27:48.379 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:48.378+0000 7fe8df8c2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:27:48.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.418+0000 7f1377fff700 1 -- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f136406a200 msgr2=0x7f136406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.418+0000 7f1377fff700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f136406a200 0x7f136406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:48.449 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:48.448+0000 7fe8df8c2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:27:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.470+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13780734a0 con 0x7f13780b62c0 2023-08-27T11:27:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.471+0000 7f1374ff9700 1 -- 172.21.15.45:0/2441462665 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f136c015020 con 0x7f13780b62c0 2023-08-27T11:27:48.472 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:27:48.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f136406a200 msgr2=0x7f136406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f136406a200 0x7f136406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b62c0 msgr2=0x7f137814b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b62c0 0x7f137814b080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f136c000c00 tx=0x7f136c004750 comp rx=0 tx=0).stop 2023-08-27T11:27:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 shutdown_connections 2023-08-27T11:27:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f136406a200 0x7f136406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13780b5220 0x7f13780cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1378160890 0x7f137814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 --2- 172.21.15.45:0/2441462665 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13780b62c0 0x7f137814b080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.473+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 >> 172.21.15.45:0/2441462665 conn(0x7f13780cfb10 msgr2=0x7f13780c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.474+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 shutdown_connections 2023-08-27T11:27:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.474+0000 7f137e786700 1 -- 172.21.15.45:0/2441462665 wait complete. 2023-08-27T11:27:48.488 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:27:48.523 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:48.523+0000 7fe70ddea200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:27:48.598 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:48.598+0000 7fe70ddea200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:27:48.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.615+0000 7fa5929b3700 1 Processor -- start 2023-08-27T11:27:48.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.615+0000 7fa5929b3700 1 -- start start 2023-08-27T11:27:48.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.616+0000 7fa5929b3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c15a0f0 0x7fa58c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.616+0000 7fa5929b3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa58c06ed90 0x7fa58c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.616+0000 7fa5929b3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58c0cfae0 con 0x7fa58c0cd300 2023-08-27T11:27:48.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.616+0000 7fa5929b3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa58c15e900 con 0x7fa58c06ed90 2023-08-27T11:27:48.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.616+0000 7fa5929b3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0cae60 con 0x7fa58c15a0f0 2023-08-27T11:27:48.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.616+0000 7fa58b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa58c0cd300 0x7fa58c15daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45214/0 (socket says 172.21.15.45:45214) 2023-08-27T11:27:48.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.616+0000 7fa58b7fe700 1 -- 172.21.15.45:0/459590388 learned_addr learned my addr 172.21.15.45:0/459590388 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:48.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa590f50700 1 --2- 172.21.15.45:0/459590388 >> v2:172.21.15.135:3300/0 conn(0x7fa58c06ed90 0x7fa58c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58bfff700 1 --2- 172.21.15.45:0/459590388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c15a0f0 0x7fa58c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58affd700 1 -- 172.21.15.45:0/459590388 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356706950 0 0) 0x7fa58c0cfae0 con 0x7fa58c0cd300 2023-08-27T11:27:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58affd700 1 -- 172.21.15.45:0/459590388 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5780057c0 con 0x7fa58c0cd300 2023-08-27T11:27:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58bfff700 1 -- 172.21.15.45:0/459590388 >> v2:172.21.15.135:3300/0 conn(0x7fa58c06ed90 msgr2=0x7fa58c15e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58bfff700 1 --2- 172.21.15.45:0/459590388 >> v2:172.21.15.135:3300/0 conn(0x7fa58c06ed90 0x7fa58c15e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58bfff700 1 -- 172.21.15.45:0/459590388 >> v1:172.21.15.45:6789/0 conn(0x7fa58c0cd300 legacy=0x7fa58c15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58bfff700 1 -- 172.21.15.45:0/459590388 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c0cb140 con 0x7fa58c15a0f0 2023-08-27T11:27:48.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58bfff700 1 --2- 172.21.15.45:0/459590388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c15a0f0 0x7fa58c0cbc20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa574009840 tx=0x7fa574009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abf6302aaff9225 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:48.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58affd700 1 -- 172.21.15.45:0/459590388 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574011830 con 0x7fa58c15a0f0 2023-08-27T11:27:48.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58affd700 1 -- 172.21.15.45:0/459590388 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa574011990 con 0x7fa58c15a0f0 2023-08-27T11:27:48.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.617+0000 7fa58affd700 1 -- 172.21.15.45:0/459590388 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574011d20 con 0x7fa58c15a0f0 2023-08-27T11:27:48.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.618+0000 7fa5929b3700 1 -- 172.21.15.45:0/459590388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c15a0f0 msgr2=0x7fa58c0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.618+0000 7fa5929b3700 1 --2- 172.21.15.45:0/459590388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c15a0f0 0x7fa58c0cbc20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa574009840 tx=0x7fa574009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:48.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.618+0000 7fa5929b3700 1 -- 172.21.15.45:0/459590388 shutdown_connections 2023-08-27T11:27:48.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.618+0000 7fa5929b3700 1 --2- 172.21.15.45:0/459590388 >> v2:172.21.15.135:3300/0 conn(0x7fa58c06ed90 0x7fa58c15e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.618+0000 7fa5929b3700 1 --2- 172.21.15.45:0/459590388 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c15a0f0 0x7fa58c0cbc20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.618+0000 7fa5929b3700 1 -- 172.21.15.45:0/459590388 >> 172.21.15.45:0/459590388 conn(0x7fa58c0cc7e0 msgr2=0x7fa58c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:48.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.618+0000 7fa5929b3700 1 -- 172.21.15.45:0/459590388 shutdown_connections 2023-08-27T11:27:48.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.618+0000 7fa5929b3700 1 -- 172.21.15.45:0/459590388 wait complete. 2023-08-27T11:27:48.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.619+0000 7fa5929b3700 1 Processor -- start 2023-08-27T11:27:48.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.619+0000 7fa5929b3700 1 -- start start 2023-08-27T11:27:48.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.620+0000 7fa5929b3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c06ed90 0x7fa58c0c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.620+0000 7fa5929b3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa58c0cd300 0x7fa58c0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.620+0000 7fa5929b3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa58c15a0f0 0x7fa58c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.620+0000 7fa5929b3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1f6830 con 0x7fa58c0cd300 2023-08-27T11:27:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.620+0000 7fa5929b3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c14cff0 con 0x7fa58c15a0f0 2023-08-27T11:27:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.620+0000 7fa5929b3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c14a130 con 0x7fa58c06ed90 2023-08-27T11:27:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa58c0cd300 0x7fa58c0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58bfff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c06ed90 0x7fa58c0c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa58c0cd300 0x7fa58c0b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58444/0 (socket says 172.21.15.45:58444) 2023-08-27T11:27:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58b7fe700 1 -- 172.21.15.45:0/911507440 learned_addr learned my addr 172.21.15.45:0/911507440 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa590f50700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa58c15a0f0 0x7fa58c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58b7fe700 1 -- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c06ed90 msgr2=0x7fa58c0c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58b7fe700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c06ed90 0x7fa58c0c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58b7fe700 1 -- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa58c15a0f0 msgr2=0x7fa58c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58b7fe700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa58c15a0f0 0x7fa58c0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58b7fe700 1 -- 172.21.15.45:0/911507440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740094a0 con 0x7fa58c0cd300 2023-08-27T11:27:48.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58bfff700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c06ed90 0x7fa58c0c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:48.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa58b7fe700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa58c0cd300 0x7fa58c0b0960 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00c730 tx=0x7fa57c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:48.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa588ff9700 1 -- 172.21.15.45:0/911507440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c005d70 con 0x7fa58c0cd300 2023-08-27T11:27:48.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa588ff9700 1 -- 172.21.15.45:0/911507440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c0070d0 con 0x7fa58c0cd300 2023-08-27T11:27:48.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58c0c65b0 con 0x7fa58c0cd300 2023-08-27T11:27:48.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa588ff9700 1 -- 172.21.15.45:0/911507440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c007440 con 0x7fa58c0cd300 2023-08-27T11:27:48.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.621+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c1f6170 con 0x7fa58c0cd300 2023-08-27T11:27:48.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.622+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58c0726d0 con 0x7fa58c0cd300 2023-08-27T11:27:48.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.623+0000 7fa588ff9700 1 -- 172.21.15.45:0/911507440 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fa57c0075a0 con 0x7fa58c0cd300 2023-08-27T11:27:48.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.623+0000 7fa588ff9700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fa56c06a200 0x7fa56c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:48.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.623+0000 7fa58bfff700 1 -- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fa56c06a200 msgr2=0x7fa56c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:48.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.623+0000 7fa588ff9700 1 -- 172.21.15.45:0/911507440 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa57c087c20 con 0x7fa58c0cd300 2023-08-27T11:27:48.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.623+0000 7fa58bfff700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fa56c06a200 0x7fa56c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:48.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.628+0000 7fa588ff9700 1 -- 172.21.15.45:0/911507440 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa57c053070 con 0x7fa58c0cd300 2023-08-27T11:27:48.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.824+0000 7fa58bfff700 1 -- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fa56c06a200 msgr2=0x7fa56c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:48.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.824+0000 7fa58bfff700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fa56c06a200 0x7fa56c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:48.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.844+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa58c0cf840 con 0x7fa58c0cd300 2023-08-27T11:27:48.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.844+0000 7fa588ff9700 1 -- 172.21.15.45:0/911507440 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v87) v1 ==== 80+0+476 (secure 0 0 0) 0x7fa57c056690 con 0x7fa58c0cd300 2023-08-27T11:27:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/790246089 2023-08-28T11:27:44.271608+0000 2023-08-27T11:27:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/2253320845 2023-08-28T11:27:44.271608+0000 2023-08-27T11:27:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/2129685370 2023-08-28T11:27:44.271608+0000 2023-08-27T11:27:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/289717412 2023-08-28T11:27:44.271608+0000 2023-08-27T11:27:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6833/3430028400 2023-08-28T11:27:44.271608+0000 2023-08-27T11:27:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6832/3430028400 2023-08-28T11:27:44.271608+0000 2023-08-27T11:27:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/2380106040 2023-08-28T11:27:44.271608+0000 2023-08-27T11:27:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/4054422326 2023-08-28T11:27:44.271608+0000 2023-08-27T11:27:48.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fa56c06a200 msgr2=0x7fa56c06c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fa56c06a200 0x7fa56c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa58c0cd300 msgr2=0x7fa58c0b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa58c0cd300 0x7fa58c0b0960 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00c730 tx=0x7fa57c00ca40 comp rx=0 tx=0).stop 2023-08-27T11:27:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 shutdown_connections 2023-08-27T11:27:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa58c15a0f0 0x7fa58c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa58c06ed90 0x7fa58c0c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7fa56c06a200 0x7fa56c06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 --2- 172.21.15.45:0/911507440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa58c0cd300 0x7fa58c0b0960 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 >> 172.21.15.45:0/911507440 conn(0x7fa58c0cc7e0 msgr2=0x7fa58c0afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 shutdown_connections 2023-08-27T11:27:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:48.847+0000 7fa5929b3700 1 -- 172.21.15.45:0/911507440 wait complete. 2023-08-27T11:27:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:listed 8 entries 2023-08-27T11:27:48.860 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:27:48.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:48.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:27:48.949 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:27:48.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:27:48.950 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:27:48.967 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:27:48.968 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:48.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:48.986 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:27:48.987 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:49.024 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:27:49.024 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:27:49.025 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:27:49.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:27:49.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:49.087 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:49.085+0000 7fe8df8c2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:49.165 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:49.164+0000 7fe8df8c2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:27:49.195 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.194+0000 7f8c0e3df700 1 Processor -- start 2023-08-27T11:27:49.196 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.194+0000 7f8c0e3df700 1 -- start start 2023-08-27T11:27:49.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c0e3df700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c080ceba0 0x7f8c080cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c0e3df700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c080cbe70 0x7f8c0806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c0e3df700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0806d690 con 0x7f8c0815a240 2023-08-27T11:27:49.198 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c0e3df700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c080cc440 con 0x7f8c080ceba0 2023-08-27T11:27:49.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c0e3df700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0806d040 con 0x7f8c080cbe70 2023-08-27T11:27:49.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c077fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c080ceba0 0x7f8c080cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c07fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8c0815a240 0x7f8c080ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45228/0 (socket says 172.21.15.45:45228) 2023-08-27T11:27:49.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c077fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c080ceba0 0x7f8c080cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58454/0 (socket says 172.21.15.45:58454) 2023-08-27T11:27:49.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c07fff700 1 -- 172.21.15.45:0/2666721905 learned_addr learned my addr 172.21.15.45:0/2666721905 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:49.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.195+0000 7f8c0c97c700 1 --2- 172.21.15.45:0/2666721905 >> v2:172.21.15.135:3300/0 conn(0x7f8c080cbe70 0x7f8c0806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c077fe700 1 -- 172.21.15.45:0/2666721905 >> v2:172.21.15.135:3300/0 conn(0x7f8c080cbe70 msgr2=0x7f8c0806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:49.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c077fe700 1 --2- 172.21.15.45:0/2666721905 >> v2:172.21.15.135:3300/0 conn(0x7f8c080cbe70 0x7f8c0806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c077fe700 1 -- 172.21.15.45:0/2666721905 >> v1:172.21.15.45:6789/0 conn(0x7f8c0815a240 legacy=0x7f8c080ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c077fe700 1 -- 172.21.15.45:0/2666721905 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0806d320 con 0x7f8c080ceba0 2023-08-27T11:27:49.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c06ffd700 1 -- 172.21.15.45:0/2666721905 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137996418 0 0) 0x7f8c0806d690 con 0x7f8c0815a240 2023-08-27T11:27:49.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c0c97c700 1 --2- 172.21.15.45:0/2666721905 >> v2:172.21.15.135:3300/0 conn(0x7f8c080cbe70 0x7f8c0806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:49.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c077fe700 1 --2- 172.21.15.45:0/2666721905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080ceba0 0x7f8c080cae80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8c080af860 tx=0x7f8bfc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c38e474855b6ed7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:49.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c06ffd700 1 -- 172.21.15.45:0/2666721905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc011830 con 0x7f8c080ceba0 2023-08-27T11:27:49.201 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c06ffd700 1 -- 172.21.15.45:0/2666721905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc011990 con 0x7f8c080ceba0 2023-08-27T11:27:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.196+0000 7f8c06ffd700 1 -- 172.21.15.45:0/2666721905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc011d20 con 0x7f8c080ceba0 2023-08-27T11:27:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.197+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2666721905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080ceba0 msgr2=0x7f8c080cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.197+0000 7f8c0e3df700 1 --2- 172.21.15.45:0/2666721905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080ceba0 0x7f8c080cae80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8c080af860 tx=0x7f8bfc009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.197+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2666721905 shutdown_connections 2023-08-27T11:27:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.197+0000 7f8c0e3df700 1 --2- 172.21.15.45:0/2666721905 >> v2:172.21.15.135:3300/0 conn(0x7f8c080cbe70 0x7f8c0806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.197+0000 7f8c0e3df700 1 --2- 172.21.15.45:0/2666721905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080ceba0 0x7f8c080cae80 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.197+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2666721905 >> 172.21.15.45:0/2666721905 conn(0x7f8c080cfe50 msgr2=0x7f8c080d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.197+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2666721905 shutdown_connections 2023-08-27T11:27:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.197+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2666721905 wait complete. 2023-08-27T11:27:49.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.198+0000 7f8c0e3df700 1 Processor -- start 2023-08-27T11:27:49.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.198+0000 7f8c0e3df700 1 -- start start 2023-08-27T11:27:49.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c0e3df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080cbe70 0x7f8c0815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c0e3df700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080ceba0 0x7f8c0814c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c0e3df700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c0815a240 0x7f8c080b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c0e3df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081f60c0 con 0x7f8c080cbe70 2023-08-27T11:27:49.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c0e3df700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081f63a0 con 0x7f8c080ceba0 2023-08-27T11:27:49.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c0e3df700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081f6680 con 0x7f8c0815a240 2023-08-27T11:27:49.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080cbe70 0x7f8c0815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080cbe70 0x7f8c0815a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58458/0 (socket says 172.21.15.45:58458) 2023-08-27T11:27:49.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c07fff700 1 -- 172.21.15.45:0/2913029591 learned_addr learned my addr 172.21.15.45:0/2913029591 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:49.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c077fe700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080ceba0 0x7f8c0814c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c0c97c700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c0815a240 0x7f8c080b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c07fff700 1 -- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c0815a240 msgr2=0x7f8c080b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c07fff700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c0815a240 0x7f8c080b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c07fff700 1 -- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080ceba0 msgr2=0x7f8c0814c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c07fff700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080ceba0 0x7f8c0814c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.199+0000 7f8c07fff700 1 -- 172.21.15.45:0/2913029591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc0094a0 con 0x7f8c080cbe70 2023-08-27T11:27:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.200+0000 7f8c07fff700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080cbe70 0x7f8c0815a620 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800c550 tx=0x7f8bf800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.200+0000 7f8c04ff9700 1 -- 172.21.15.45:0/2913029591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8004c80 con 0x7f8c080cbe70 2023-08-27T11:27:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.200+0000 7f8c04ff9700 1 -- 172.21.15.45:0/2913029591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf8004e00 con 0x7f8c080cbe70 2023-08-27T11:27:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.200+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c081f6960 con 0x7f8c080cbe70 2023-08-27T11:27:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.200+0000 7f8c04ff9700 1 -- 172.21.15.45:0/2913029591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8007310 con 0x7f8c080cbe70 2023-08-27T11:27:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.200+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0815ab60 con 0x7f8c080cbe70 2023-08-27T11:27:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.201+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c081f6e80 con 0x7f8c080cbe70 2023-08-27T11:27:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.201+0000 7f8c04ff9700 1 -- 172.21.15.45:0/2913029591 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f8bf8007470 con 0x7f8c080cbe70 2023-08-27T11:27:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.202+0000 7f8c04ff9700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8bf406a200 0x7f8bf406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.202+0000 7f8c04ff9700 1 -- 172.21.15.45:0/2913029591 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f8bf8088490 con 0x7f8c080cbe70 2023-08-27T11:27:49.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.202+0000 7f8c077fe700 1 -- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8bf406a200 msgr2=0x7f8bf406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:49.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.202+0000 7f8c077fe700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8bf406a200 0x7f8bf406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:49.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.206+0000 7f8c04ff9700 1 -- 172.21.15.45:0/2913029591 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8bf8053830 con 0x7f8c080cbe70 2023-08-27T11:27:49.280 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:49.279+0000 7fe70ddea200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:49.310 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:49.309+0000 7fe8df8c2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:49.364 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:49.363+0000 7fe70ddea200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:27:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.403+0000 7f8c077fe700 1 -- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8bf406a200 msgr2=0x7f8bf406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.403+0000 7f8c077fe700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8bf406a200 0x7f8bf406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.453+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c08072810 con 0x7f8c080cbe70 2023-08-27T11:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.454+0000 7f8c04ff9700 1 -- 172.21.15.45:0/2913029591 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f8bf8056e50 con 0x7f8c080cbe70 2023-08-27T11:27:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:49.455 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2178366980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2178366980},{"type":"v1","addr":"172.21.15.135:6836","nonce":2178366980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2939076938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2939076938},{"type":"v1","addr":"172.21.15.45:6833","nonce":2939076938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/148627485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":148627485},{"type":"v1","addr":"172.21.15.45:6835","nonce":148627485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:25.479170+0000","modified":"2023-08-27T11:27:29.096440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1457694749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1457694749},{"type":"v1","addr":"172.21.15.135:6837","nonce":1457694749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:27:49.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8bf406a200 msgr2=0x7f8bf406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:49.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8bf406a200 0x7f8bf406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080cbe70 msgr2=0x7f8c0815a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080cbe70 0x7f8c0815a620 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800c550 tx=0x7f8bf800c860 comp rx=0 tx=0).stop 2023-08-27T11:27:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 shutdown_connections 2023-08-27T11:27:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8bf406a200 0x7f8bf406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c080cbe70 0x7f8c0815a620 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c0815a240 0x7f8c080b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 --2- 172.21.15.45:0/2913029591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c080ceba0 0x7f8c0814c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 >> 172.21.15.45:0/2913029591 conn(0x7f8c080cfe50 msgr2=0x7f8c080cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 shutdown_connections 2023-08-27T11:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.456+0000 7f8c0e3df700 1 -- 172.21.15.45:0/2913029591 wait complete. 2023-08-27T11:27:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2023-08-27T11:27:49.469 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:49.518 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:49.518+0000 7fe70ddea200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:27:49.539 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:49.538+0000 7fe8df8c2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:27:49.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.596+0000 7ff52c00a700 1 Processor -- start 2023-08-27T11:27:49.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.596+0000 7ff52c00a700 1 -- start start 2023-08-27T11:27:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff52c00a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 0x7ff5240cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff52c00a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5240d1a20 0x7ff5240cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff52c00a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5240ada50 con 0x7ff5240cd9b0 2023-08-27T11:27:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff52c00a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5240cde40 con 0x7ff5240d1a20 2023-08-27T11:27:49.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff52c00a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5240ccd60 con 0x7ff52415b120 2023-08-27T11:27:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff52a5a7700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff5240cd9b0 0x7ff5240cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45274/0 (socket says 172.21.15.45:45274) 2023-08-27T11:27:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff52a5a7700 1 -- 172.21.15.45:0/1787499372 learned_addr learned my addr 172.21.15.45:0/1787499372 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff5295a5700 1 --2- 172.21.15.45:0/1787499372 >> v2:172.21.15.135:3300/0 conn(0x7ff5240d1a20 0x7ff5240cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff529da6700 1 --2- 172.21.15.45:0/1787499372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 0x7ff5240cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff528da4700 1 -- 172.21.15.45:0/1787499372 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975674030 0 0) 0x7ff5240ada50 con 0x7ff5240cd9b0 2023-08-27T11:27:49.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff528da4700 1 -- 172.21.15.45:0/1787499372 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5100035b0 con 0x7ff5240cd9b0 2023-08-27T11:27:49.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff529da6700 1 -- 172.21.15.45:0/1787499372 >> v2:172.21.15.135:3300/0 conn(0x7ff5240d1a20 msgr2=0x7ff5240cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:49.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff529da6700 1 --2- 172.21.15.45:0/1787499372 >> v2:172.21.15.135:3300/0 conn(0x7ff5240d1a20 0x7ff5240cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff529da6700 1 -- 172.21.15.45:0/1787499372 >> v1:172.21.15.45:6789/0 conn(0x7ff5240cd9b0 legacy=0x7ff5240cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff529da6700 1 -- 172.21.15.45:0/1787499372 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5240b0860 con 0x7ff52415b120 2023-08-27T11:27:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.597+0000 7ff5295a5700 1 --2- 172.21.15.45:0/1787499372 >> v2:172.21.15.135:3300/0 conn(0x7ff5240d1a20 0x7ff5240cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff529da6700 1 --2- 172.21.15.45:0/1787499372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 0x7ff5240cc430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff520009840 tx=0x7ff520009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f585948bd3a46273 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff528da4700 1 -- 172.21.15.45:0/1787499372 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520011830 con 0x7ff52415b120 2023-08-27T11:27:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff528da4700 1 -- 172.21.15.45:0/1787499372 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff520011990 con 0x7ff52415b120 2023-08-27T11:27:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff528da4700 1 -- 172.21.15.45:0/1787499372 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520011d20 con 0x7ff52415b120 2023-08-27T11:27:49.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff52c00a700 1 -- 172.21.15.45:0/1787499372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 msgr2=0x7ff5240cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff52c00a700 1 --2- 172.21.15.45:0/1787499372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 0x7ff5240cc430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff520009840 tx=0x7ff520009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:49.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff52c00a700 1 -- 172.21.15.45:0/1787499372 shutdown_connections 2023-08-27T11:27:49.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff52c00a700 1 --2- 172.21.15.45:0/1787499372 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 0x7ff5240cc430 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff52c00a700 1 --2- 172.21.15.45:0/1787499372 >> v2:172.21.15.135:3300/0 conn(0x7ff5240d1a20 0x7ff5240cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff52c00a700 1 -- 172.21.15.45:0/1787499372 >> 172.21.15.45:0/1787499372 conn(0x7ff52409a160 msgr2=0x7ff5240d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:49.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff52c00a700 1 -- 172.21.15.45:0/1787499372 shutdown_connections 2023-08-27T11:27:49.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.598+0000 7ff52c00a700 1 -- 172.21.15.45:0/1787499372 wait complete. 2023-08-27T11:27:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.600+0000 7ff52c00a700 1 Processor -- start 2023-08-27T11:27:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.600+0000 7ff52c00a700 1 -- start start 2023-08-27T11:27:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.600+0000 7ff52c00a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5240cd9b0 0x7ff5241f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.600+0000 7ff52c00a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5240d1a20 0x7ff524151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.600+0000 7ff52c00a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 0x7ff52406bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.600+0000 7ff52c00a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52406c620 con 0x7ff5240cd9b0 2023-08-27T11:27:49.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.600+0000 7ff52c00a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5240c80d0 con 0x7ff5240d1a20 2023-08-27T11:27:49.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.600+0000 7ff52c00a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5240c83b0 con 0x7ff52415b120 2023-08-27T11:27:49.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff529da6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5240cd9b0 0x7ff5241f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff529da6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5240cd9b0 0x7ff5241f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58500/0 (socket says 172.21.15.45:58500) 2023-08-27T11:27:49.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff529da6700 1 -- 172.21.15.45:0/794241721 learned_addr learned my addr 172.21.15.45:0/794241721 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:49.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff5295a5700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5240d1a20 0x7ff524151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff52a5a7700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 0x7ff52406bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff529da6700 1 -- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 msgr2=0x7ff52406bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff529da6700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 0x7ff52406bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff529da6700 1 -- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5240d1a20 msgr2=0x7ff524151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff529da6700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5240d1a20 0x7ff524151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff529da6700 1 -- 172.21.15.45:0/794241721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5140094a0 con 0x7ff5240cd9b0 2023-08-27T11:27:49.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff529da6700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5240cd9b0 0x7ff5241f6f80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff520009fd0 tx=0x7ff520004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff51a7fc700 1 -- 172.21.15.45:0/794241721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5200036a0 con 0x7ff5240cd9b0 2023-08-27T11:27:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff51a7fc700 1 -- 172.21.15.45:0/794241721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52001e070 con 0x7ff5240cd9b0 2023-08-27T11:27:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5200094a0 con 0x7ff5240cd9b0 2023-08-27T11:27:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff51a7fc700 1 -- 172.21.15.45:0/794241721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520003a60 con 0x7ff5240cd9b0 2023-08-27T11:27:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.601+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5240c8970 con 0x7ff5240cd9b0 2023-08-27T11:27:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.602+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5240c8b60 con 0x7ff5240cd9b0 2023-08-27T11:27:49.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.603+0000 7ff51a7fc700 1 -- 172.21.15.45:0/794241721 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7ff520003bc0 con 0x7ff5240cd9b0 2023-08-27T11:27:49.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.603+0000 7ff51a7fc700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7ff50806a200 0x7ff50806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.603+0000 7ff5295a5700 1 -- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7ff50806a200 msgr2=0x7ff50806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:49.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.603+0000 7ff51a7fc700 1 -- 172.21.15.45:0/794241721 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7ff520089840 con 0x7ff5240cd9b0 2023-08-27T11:27:49.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.603+0000 7ff5295a5700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7ff50806a200 0x7ff50806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:49.614 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:49.613+0000 7fe8df8c2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:27:49.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.608+0000 7ff51a7fc700 1 -- 172.21.15.45:0/794241721 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff520054b50 con 0x7ff5240cd9b0 2023-08-27T11:27:49.763 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:49.762+0000 7fe70ddea200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:27:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.804+0000 7ff5295a5700 1 -- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7ff50806a200 msgr2=0x7ff50806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:49.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.804+0000 7ff5295a5700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7ff50806a200 0x7ff50806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:49.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.814+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff52409edd0 con 0x7ff5240cd9b0 2023-08-27T11:27:49.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.816+0000 7ff51a7fc700 1 -- 172.21.15.45:0/794241721 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+19144 (secure 0 0 0) 0x7ff520058170 con 0x7ff5240cd9b0 2023-08-27T11:27:49.817 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:49.820 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":87,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:44.271663+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:23.082458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:24.453679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/790246089":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2253320845":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2129685370":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/289717412":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6833/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6832/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2380106040":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/4054422326":"2023-08-28T11:27:44.271608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:49.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7ff50806a200 msgr2=0x7ff50806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:49.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7ff50806a200 0x7ff50806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5240cd9b0 msgr2=0x7ff5241f6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5240cd9b0 0x7ff5241f6f80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff520009fd0 tx=0x7ff520004750 comp rx=0 tx=0).stop 2023-08-27T11:27:49.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 shutdown_connections 2023-08-27T11:27:49.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7ff50806a200 0x7ff50806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5240cd9b0 0x7ff5241f6f80 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff52415b120 0x7ff52406bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 --2- 172.21.15.45:0/794241721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5240d1a20 0x7ff524151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 >> 172.21.15.45:0/794241721 conn(0x7ff52409a160 msgr2=0x7ff524159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:49.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 shutdown_connections 2023-08-27T11:27:49.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.818+0000 7ff52c00a700 1 -- 172.21.15.45:0/794241721 wait complete. 2023-08-27T11:27:49.832 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:27:23.082458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:27:24.453679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:27:49.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:27:49.843 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:49.842+0000 7fe70ddea200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:27:49.879 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:49.878+0000 7fe8df8c2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:27:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.966+0000 7f8c340cb700 1 Processor -- start 2023-08-27T11:27:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.966+0000 7f8c340cb700 1 -- start start 2023-08-27T11:27:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c340cb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 0x7f8c2c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c340cb700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c2c06c4b0 0x7f8c2c0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c340cb700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2c15d780 con 0x7f8c2c0d02c0 2023-08-27T11:27:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c340cb700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2c0cfe30 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c340cb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0cf220 con 0x7f8c2c15d3a0 2023-08-27T11:27:49.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c31666700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8c2c0d02c0 0x7f8c2c06bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45288/0 (socket says 172.21.15.45:45288) 2023-08-27T11:27:49.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c31666700 1 -- 172.21.15.45:0/3245865631 learned_addr learned my addr 172.21.15.45:0/3245865631 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:49.970 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c32668700 1 --2- 172.21.15.45:0/3245865631 >> v2:172.21.15.135:3300/0 conn(0x7f8c2c06c4b0 0x7f8c2c0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c31e67700 1 --2- 172.21.15.45:0/3245865631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 0x7f8c2c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c30e65700 1 -- 172.21.15.45:0/3245865631 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479903735 0 0) 0x7f8c1c001110 con 0x7f8c2c0d02c0 2023-08-27T11:27:49.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c30e65700 1 -- 172.21.15.45:0/3245865631 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c180035b0 con 0x7f8c2c0d02c0 2023-08-27T11:27:49.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c32668700 1 -- 172.21.15.45:0/3245865631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 msgr2=0x7f8c2c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:49.971 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.967+0000 7f8c32668700 1 --2- 172.21.15.45:0/3245865631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 0x7f8c2c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.968+0000 7f8c32668700 1 -- 172.21.15.45:0/3245865631 >> v1:172.21.15.45:6789/0 conn(0x7f8c2c0d02c0 legacy=0x7f8c2c06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.968+0000 7f8c32668700 1 -- 172.21.15.45:0/3245865631 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0cf500 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.968+0000 7f8c32668700 1 --2- 172.21.15.45:0/3245865631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c2c06c4b0 0x7f8c2c0cb490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009d90 tx=0x7f8c2400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=799a71dc2eea8438 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:49.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.968+0000 7f8c30e65700 1 -- 172.21.15.45:0/3245865631 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400e070 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.968+0000 7f8c30e65700 1 -- 172.21.15.45:0/3245865631 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2400bb20 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.968+0000 7f8c30e65700 1 -- 172.21.15.45:0/3245865631 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24012430 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.969+0000 7f8c340cb700 1 -- 172.21.15.45:0/3245865631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c2c06c4b0 msgr2=0x7f8c2c0cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.969+0000 7f8c340cb700 1 --2- 172.21.15.45:0/3245865631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c2c06c4b0 0x7f8c2c0cb490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009d90 tx=0x7f8c2400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:49.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.969+0000 7f8c340cb700 1 -- 172.21.15.45:0/3245865631 shutdown_connections 2023-08-27T11:27:49.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.969+0000 7f8c340cb700 1 --2- 172.21.15.45:0/3245865631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c2c06c4b0 0x7f8c2c0cb490 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.969+0000 7f8c340cb700 1 --2- 172.21.15.45:0/3245865631 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 0x7f8c2c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.969+0000 7f8c340cb700 1 -- 172.21.15.45:0/3245865631 >> 172.21.15.45:0/3245865631 conn(0x7f8c2c09a1a0 msgr2=0x7f8c2c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:49.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.969+0000 7f8c340cb700 1 -- 172.21.15.45:0/3245865631 shutdown_connections 2023-08-27T11:27:49.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.969+0000 7f8c340cb700 1 -- 172.21.15.45:0/3245865631 wait complete. 2023-08-27T11:27:49.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.970+0000 7f8c340cb700 1 Processor -- start 2023-08-27T11:27:49.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.970+0000 7f8c340cb700 1 -- start start 2023-08-27T11:27:49.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c340cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c2c06c4b0 0x7f8c2c0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c340cb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c2c0d02c0 0x7f8c2c0c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c340cb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 0x7f8c2c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c340cb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0b3550 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c340cb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0b3fb0 con 0x7f8c2c0d02c0 2023-08-27T11:27:49.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c340cb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0b4290 con 0x7f8c2c15d3a0 2023-08-27T11:27:49.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c31e67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c2c06c4b0 0x7f8c2c0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c31e67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c2c06c4b0 0x7f8c2c0b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58516/0 (socket says 172.21.15.45:58516) 2023-08-27T11:27:49.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c31e67700 1 -- 172.21.15.45:0/3391864529 learned_addr learned my addr 172.21.15.45:0/3391864529 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:49.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c31666700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c2c0d02c0 0x7f8c2c0c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c32668700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 0x7f8c2c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:49.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c31e67700 1 -- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 msgr2=0x7f8c2c0b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c31e67700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 0x7f8c2c0b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c31e67700 1 -- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c2c0d02c0 msgr2=0x7f8c2c0c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:49.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c31e67700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c2c0d02c0 0x7f8c2c0c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:49.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.971+0000 7f8c31e67700 1 -- 172.21.15.45:0/3391864529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c24009a70 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.972+0000 7f8c31e67700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c2c06c4b0 0x7f8c2c0b6350 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800e870 tx=0x7f8c2800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:49.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.972+0000 7f8c227fc700 1 -- 172.21.15.45:0/3391864529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28009590 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.972+0000 7f8c227fc700 1 -- 172.21.15.45:0/3391864529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c28013680 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.972+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2c0b4570 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.972+0000 7f8c227fc700 1 -- 172.21.15.45:0/3391864529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c280139f0 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.972+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c0b6890 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.973+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c0b4a90 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.973+0000 7f8c227fc700 1 -- 172.21.15.45:0/3391864529 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f8c28013b50 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.974+0000 7f8c227fc700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c1006a200 0x7f8c1006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:49.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.974+0000 7f8c227fc700 1 -- 172.21.15.45:0/3391864529 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f8c280887f0 con 0x7f8c2c06c4b0 2023-08-27T11:27:49.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.974+0000 7f8c31666700 1 -- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c1006a200 msgr2=0x7f8c1006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:49.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.974+0000 7f8c31666700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c1006a200 0x7f8c1006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:49.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:49.978+0000 7f8c227fc700 1 -- 172.21.15.45:0/3391864529 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8c2c0b4a90 con 0x7f8c2c06c4b0 2023-08-27T11:27:50.102 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:50.101+0000 7fe8df8c2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:27:50.128 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:50.127+0000 7fe70ddea200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:27:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.175+0000 7f8c31666700 1 -- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c1006a200 msgr2=0x7f8c1006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.175+0000 7f8c31666700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c1006a200 0x7f8c1006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:50.181 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.181+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c2c0b4a90 con 0x7f8c2c06c4b0 2023-08-27T11:27:50.182 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.181+0000 7f8c227fc700 1 -- 172.21.15.45:0/3391864529 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c2c0b4a90 con 0x7f8c2c06c4b0 2023-08-27T11:27:50.182 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:27:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c1006a200 msgr2=0x7f8c1006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c1006a200 0x7f8c1006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c2c06c4b0 msgr2=0x7f8c2c0b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c2c06c4b0 0x7f8c2c0b6350 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800e870 tx=0x7f8c2800eb80 comp rx=0 tx=0).stop 2023-08-27T11:27:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 shutdown_connections 2023-08-27T11:27:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f8c1006a200 0x7f8c1006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c2c06c4b0 0x7f8c2c0b6350 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8c2c15d3a0 0x7f8c2c0b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 --2- 172.21.15.45:0/3391864529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c2c0d02c0 0x7f8c2c0c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 >> 172.21.15.45:0/3391864529 conn(0x7f8c2c09a1a0 msgr2=0x7f8c2c0af940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 shutdown_connections 2023-08-27T11:27:50.187 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.184+0000 7f8c340cb700 1 -- 172.21.15.45:0/3391864529 wait complete. 2023-08-27T11:27:50.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:27:50.251 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:50.250+0000 7fe8df8c2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:27:50.366 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:50.364+0000 7fe8df8c2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:27:50.367 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:50.366+0000 7fe70ddea200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:27:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.369+0000 7f733088c700 1 Processor -- start 2023-08-27T11:27:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.369+0000 7f733088c700 1 -- start start 2023-08-27T11:27:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.370+0000 7f733088c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f732c15b310 0x7f732c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.370+0000 7f733088c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c0cb8b0 con 0x7f732c0cdcb0 2023-08-27T11:27:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.370+0000 7f733088c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c0cecd0 con 0x7f732c15af30 2023-08-27T11:27:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.370+0000 7f733088c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c06f9c0 con 0x7f732c15b310 2023-08-27T11:27:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f7329d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f732c15b310 0x7f732c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f7329d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f732c15b310 0x7f732c0cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58528/0 (socket says 172.21.15.45:58528) 2023-08-27T11:27:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f732ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f732c0cdcb0 0x7f732c164fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45306/0 (socket says 172.21.15.45:45306) 2023-08-27T11:27:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f7329d9b700 1 -- 172.21.15.45:0/2758250143 learned_addr learned my addr 172.21.15.45:0/2758250143 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f7329d9b700 1 -- 172.21.15.45:0/2758250143 >> v1:172.21.15.135:6789/0 conn(0x7f732c15af30 legacy=0x7f732c0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f7329d9b700 1 -- 172.21.15.45:0/2758250143 >> v1:172.21.15.45:6789/0 conn(0x7f732c0cdcb0 legacy=0x7f732c164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f7329d9b700 1 -- 172.21.15.45:0/2758250143 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0cbae0 con 0x7f732c15b310 2023-08-27T11:27:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f732959a700 1 -- 172.21.15.45:0/2758250143 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570049576 0 0) 0x7f732c0cb8b0 con 0x7f732c0cdcb0 2023-08-27T11:27:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f7329d9b700 1 --2- 172.21.15.45:0/2758250143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15b310 0x7f732c0cd770 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f73200098f0 tx=0x7f7320009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4b0635b1dddcb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f732959a700 1 -- 172.21.15.45:0/2758250143 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000b8a0 con 0x7f732c15b310 2023-08-27T11:27:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f732959a700 1 -- 172.21.15.45:0/2758250143 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732000ba00 con 0x7f732c15b310 2023-08-27T11:27:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.371+0000 7f732959a700 1 -- 172.21.15.45:0/2758250143 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000bd90 con 0x7f732c15b310 2023-08-27T11:27:50.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.372+0000 7f733088c700 1 -- 172.21.15.45:0/2758250143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15b310 msgr2=0x7f732c0cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.372+0000 7f733088c700 1 --2- 172.21.15.45:0/2758250143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15b310 0x7f732c0cd770 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f73200098f0 tx=0x7f7320009c00 comp rx=0 tx=0).stop 2023-08-27T11:27:50.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.372+0000 7f733088c700 1 -- 172.21.15.45:0/2758250143 shutdown_connections 2023-08-27T11:27:50.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.372+0000 7f733088c700 1 --2- 172.21.15.45:0/2758250143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15b310 0x7f732c0cd770 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.372+0000 7f733088c700 1 -- 172.21.15.45:0/2758250143 >> 172.21.15.45:0/2758250143 conn(0x7f732c09a1a0 msgr2=0x7f732c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:50.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.372+0000 7f733088c700 1 -- 172.21.15.45:0/2758250143 shutdown_connections 2023-08-27T11:27:50.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.372+0000 7f733088c700 1 -- 172.21.15.45:0/2758250143 wait complete. 2023-08-27T11:27:50.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.374+0000 7f733088c700 1 Processor -- start 2023-08-27T11:27:50.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.375+0000 7f733088c700 1 -- start start 2023-08-27T11:27:50.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f733088c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f732c0cdcb0 0x7f732c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f733088c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15af30 0x7f732c159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f733088c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732c15b310 0x7f732c14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f733088c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c15e7f0 con 0x7f732c15af30 2023-08-27T11:27:50.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f733088c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c15dc70 con 0x7f732c15b310 2023-08-27T11:27:50.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f733088c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c15df50 con 0x7f732c0cdcb0 2023-08-27T11:27:50.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f7329d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15af30 0x7f732c159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f7329d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15af30 0x7f732c159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58536/0 (socket says 172.21.15.45:58536) 2023-08-27T11:27:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f7329d9b700 1 -- 172.21.15.45:0/3617120386 learned_addr learned my addr 172.21.15.45:0/3617120386 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f732ad9d700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732c15b310 0x7f732c14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f732a59c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f732c0cdcb0 0x7f732c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f7329d9b700 1 -- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f732c0cdcb0 msgr2=0x7f732c15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f7329d9b700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f732c0cdcb0 0x7f732c15d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f7329d9b700 1 -- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732c15b310 msgr2=0x7f732c14f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f7329d9b700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732c15b310 0x7f732c14f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f7329d9b700 1 -- 172.21.15.45:0/3617120386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7320009550 con 0x7f732c15af30 2023-08-27T11:27:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f732a59c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f732c0cdcb0 0x7f732c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.376+0000 7f732ad9d700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732c15b310 0x7f732c14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.377+0000 7f7329d9b700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15af30 0x7f732c159240 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7320014040 tx=0x7f7320004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.377+0000 7f731affd700 1 -- 172.21.15.45:0/3617120386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732001a900 con 0x7f732c15af30 2023-08-27T11:27:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.377+0000 7f731affd700 1 -- 172.21.15.45:0/3617120386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732001f070 con 0x7f732c15af30 2023-08-27T11:27:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.377+0000 7f731affd700 1 -- 172.21.15.45:0/3617120386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732001ac60 con 0x7f732c15af30 2023-08-27T11:27:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.377+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c15e230 con 0x7f732c15af30 2023-08-27T11:27:50.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.377+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c15d8b0 con 0x7f732c15af30 2023-08-27T11:27:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.378+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c15da40 con 0x7f732c15af30 2023-08-27T11:27:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.378+0000 7f731affd700 1 -- 172.21.15.45:0/3617120386 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f7320026480 con 0x7f732c15af30 2023-08-27T11:27:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.379+0000 7f731affd700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f730c06a200 0x7f730c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.379+0000 7f732a59c700 1 -- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f730c06a200 msgr2=0x7f730c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:50.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.379+0000 7f731affd700 1 -- 172.21.15.45:0/3617120386 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f7320021070 con 0x7f732c15af30 2023-08-27T11:27:50.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.379+0000 7f732a59c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f730c06a200 0x7f730c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:50.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.383+0000 7f731affd700 1 -- 172.21.15.45:0/3617120386 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7320055580 con 0x7f732c15af30 2023-08-27T11:27:50.521 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:50.520+0000 7fe8df8c2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:27:50.529 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:50.528+0000 7fe70ddea200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:27:50.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.580+0000 7f732a59c700 1 -- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f730c06a200 msgr2=0x7f730c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:50.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.580+0000 7f732a59c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f730c06a200 0x7f730c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.585+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f732c0734a0 con 0x7f732c15af30 2023-08-27T11:27:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.585+0000 7f731affd700 1 -- 172.21.15.45:0/3617120386 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7320058ba0 con 0x7f732c15af30 2023-08-27T11:27:50.586 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:27:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f730c06a200 msgr2=0x7f730c06c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f730c06a200 0x7f730c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15af30 msgr2=0x7f732c159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15af30 0x7f732c159240 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7320014040 tx=0x7f7320004750 comp rx=0 tx=0).stop 2023-08-27T11:27:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 shutdown_connections 2023-08-27T11:27:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732c15b310 0x7f732c14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f732c0cdcb0 0x7f732c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f730c06a200 0x7f730c06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 --2- 172.21.15.45:0/3617120386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732c15af30 0x7f732c159240 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 >> 172.21.15.45:0/3617120386 conn(0x7f732c09a1a0 msgr2=0x7f732c0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 shutdown_connections 2023-08-27T11:27:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.588+0000 7f733088c700 1 -- 172.21.15.45:0/3617120386 wait complete. 2023-08-27T11:27:50.601 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:27:50.652 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:50.651+0000 7fe70ddea200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:27:50.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.730+0000 7facf22c1700 1 Processor -- start 2023-08-27T11:27:50.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.730+0000 7facf22c1700 1 -- start start 2023-08-27T11:27:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.731+0000 7facf22c1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7facec0cde80 0x7facec165ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.731+0000 7facf22c1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec0cb8b0 con 0x7facec15af30 2023-08-27T11:27:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.731+0000 7facf22c1700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec0cecd0 con 0x7facec15b310 2023-08-27T11:27:50.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.731+0000 7facf22c1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec06f9c0 con 0x7facec0cde80 2023-08-27T11:27:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.731+0000 7facebfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7facec15af30 0x7facec0cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45318/0 (socket says 172.21.15.45:45318) 2023-08-27T11:27:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.731+0000 7facebfff700 1 -- 172.21.15.45:0/4001405766 learned_addr learned my addr 172.21.15.45:0/4001405766 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.731+0000 7facf085e700 1 --2- 172.21.15.45:0/4001405766 >> v2:172.21.15.45:3300/0 conn(0x7facec0cde80 0x7facec165ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.732+0000 7faceaffd700 1 -- 172.21.15.45:0/4001405766 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401142351 0 0) 0x7facec0cb8b0 con 0x7facec15af30 2023-08-27T11:27:50.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.732+0000 7faceaffd700 1 -- 172.21.15.45:0/4001405766 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd40035b0 con 0x7facec15af30 2023-08-27T11:27:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.732+0000 7facf085e700 1 -- 172.21.15.45:0/4001405766 >> v1:172.21.15.135:6789/0 conn(0x7facec15b310 legacy=0x7facec0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.732+0000 7facf085e700 1 -- 172.21.15.45:0/4001405766 >> v1:172.21.15.45:6789/0 conn(0x7facec15af30 legacy=0x7facec0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.732+0000 7facf085e700 1 -- 172.21.15.45:0/4001405766 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec0cbae0 con 0x7facec0cde80 2023-08-27T11:27:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.732+0000 7facf085e700 1 --2- 172.21.15.45:0/4001405766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 0x7facec165ff0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7face4009d90 tx=0x7face400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582560bc1b4f8b5b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:50.736 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.732+0000 7faceaffd700 1 -- 172.21.15.45:0/4001405766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face400e070 con 0x7facec0cde80 2023-08-27T11:27:50.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.732+0000 7faceaffd700 1 -- 172.21.15.45:0/4001405766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face400bb20 con 0x7facec0cde80 2023-08-27T11:27:50.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.732+0000 7faceaffd700 1 -- 172.21.15.45:0/4001405766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4012430 con 0x7facec0cde80 2023-08-27T11:27:50.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.733+0000 7facf22c1700 1 -- 172.21.15.45:0/4001405766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 msgr2=0x7facec165ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.733+0000 7facf22c1700 1 --2- 172.21.15.45:0/4001405766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 0x7facec165ff0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7face4009d90 tx=0x7face400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.733+0000 7facf22c1700 1 -- 172.21.15.45:0/4001405766 shutdown_connections 2023-08-27T11:27:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.733+0000 7facf22c1700 1 --2- 172.21.15.45:0/4001405766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 0x7facec165ff0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.733+0000 7facf22c1700 1 -- 172.21.15.45:0/4001405766 >> 172.21.15.45:0/4001405766 conn(0x7facec09a1a0 msgr2=0x7facec0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.733+0000 7facf22c1700 1 -- 172.21.15.45:0/4001405766 shutdown_connections 2023-08-27T11:27:50.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.733+0000 7facf22c1700 1 -- 172.21.15.45:0/4001405766 wait complete. 2023-08-27T11:27:50.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.734+0000 7facf22c1700 1 Processor -- start 2023-08-27T11:27:50.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.734+0000 7facf22c1700 1 -- start start 2023-08-27T11:27:50.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facf22c1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 0x7facec0b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facf22c1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facec15af30 0x7facec06c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facf22c1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facec15b310 0x7facec154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facf22c1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec1f2bb0 con 0x7facec0cde80 2023-08-27T11:27:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facf22c1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec15c800 con 0x7facec15b310 2023-08-27T11:27:50.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facf22c1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec06cb30 con 0x7facec15af30 2023-08-27T11:27:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facebfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 0x7facec0b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facebfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 0x7facec0b6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58548/0 (socket says 172.21.15.45:58548) 2023-08-27T11:27:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facebfff700 1 -- 172.21.15.45:0/1839330977 learned_addr learned my addr 172.21.15.45:0/1839330977 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facf085e700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facec15b310 0x7facec154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:50.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7faceb7fe700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facec15af30 0x7facec06c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facebfff700 1 -- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facec15af30 msgr2=0x7facec06c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facebfff700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facec15af30 0x7facec06c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facebfff700 1 -- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facec15b310 msgr2=0x7facec154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.735+0000 7facebfff700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facec15b310 0x7facec154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.736+0000 7facebfff700 1 -- 172.21.15.45:0/1839330977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4009a70 con 0x7facec0cde80 2023-08-27T11:27:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.736+0000 7faceb7fe700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facec15af30 0x7facec06c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.736+0000 7facebfff700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 0x7facec0b6710 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7facdc00c700 tx=0x7facdc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.736+0000 7face8ff9700 1 -- 172.21.15.45:0/1839330977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc005d70 con 0x7facec0cde80 2023-08-27T11:27:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.736+0000 7face8ff9700 1 -- 172.21.15.45:0/1839330977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc0070d0 con 0x7facec0cde80 2023-08-27T11:27:50.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.736+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec06ce10 con 0x7facec0cde80 2023-08-27T11:27:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.736+0000 7face8ff9700 1 -- 172.21.15.45:0/1839330977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc007460 con 0x7facec0cde80 2023-08-27T11:27:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.736+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec15cac0 con 0x7facec0cde80 2023-08-27T11:27:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.737+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec0734a0 con 0x7facec0cde80 2023-08-27T11:27:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.737+0000 7face8ff9700 1 -- 172.21.15.45:0/1839330977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7facdc0075c0 con 0x7facec0cde80 2023-08-27T11:27:50.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.738+0000 7face8ff9700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7facd806a200 0x7facd806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.738+0000 7face8ff9700 1 -- 172.21.15.45:0/1839330977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7facdc087ea0 con 0x7facec0cde80 2023-08-27T11:27:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.738+0000 7faceb7fe700 1 -- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7facd806a200 msgr2=0x7facd806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.738+0000 7faceb7fe700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7facd806a200 0x7facd806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.742+0000 7face8ff9700 1 -- 172.21.15.45:0/1839330977 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7facdc0531b0 con 0x7facec0cde80 2023-08-27T11:27:50.783 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:50.782+0000 7fe8df8c2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:50.820 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:50.820+0000 7fe70ddea200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:27:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.939+0000 7faceb7fe700 1 -- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7facd806a200 msgr2=0x7facd806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.939+0000 7faceb7fe700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7facd806a200 0x7facd806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:50.943 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:50.942+0000 7fe8df8c2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:27:50.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.946+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7facec0ce4c0 con 0x7facec0cde80 2023-08-27T11:27:50.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.946+0000 7face8ff9700 1 -- 172.21.15.45:0/1839330977 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7facdc0567d0 con 0x7facec0cde80 2023-08-27T11:27:50.947 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:27:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7facd806a200 msgr2=0x7facd806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7facd806a200 0x7facd806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 msgr2=0x7facec0b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 0x7facec0b6710 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7facdc00c700 tx=0x7facdc00ca10 comp rx=0 tx=0).stop 2023-08-27T11:27:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 shutdown_connections 2023-08-27T11:27:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7facd806a200 0x7facd806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facec0cde80 0x7facec0b6710 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7facec15b310 0x7facec154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 --2- 172.21.15.45:0/1839330977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7facec15af30 0x7facec06c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 >> 172.21.15.45:0/1839330977 conn(0x7facec09a1a0 msgr2=0x7facec06e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 shutdown_connections 2023-08-27T11:27:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:50.949+0000 7facf22c1700 1 -- 172.21.15.45:0/1839330977 wait complete. 2023-08-27T11:27:50.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:27:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.093+0000 7f017b335700 1 Processor -- start 2023-08-27T11:27:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.093+0000 7f017b335700 1 -- start start 2023-08-27T11:27:51.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.094+0000 7f017b335700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f01740b5220 0x7f01740cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.094+0000 7f017b335700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b62c0 0x7f01740d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.094+0000 7f017b335700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01740cdf40 con 0x7f0174160890 2023-08-27T11:27:51.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.094+0000 7f017b335700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01740d0fd0 con 0x7f01740b5220 2023-08-27T11:27:51.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.094+0000 7f017b335700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01740ce190 con 0x7f01740b62c0 2023-08-27T11:27:51.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.094+0000 7f01788d0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f01740b5220 0x7f01740cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.094+0000 7f01790d1700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f0174160890 0x7f01740b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:40998/0 (socket says 172.21.15.45:40998) 2023-08-27T11:27:51.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.094+0000 7f01790d1700 1 -- 172.21.15.45:0/3435860041 learned_addr learned my addr 172.21.15.45:0/3435860041 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:51.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.094+0000 7f01798d2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b62c0 0x7f01740d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f016bfff700 1 -- 172.21.15.45:0/3435860041 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871100976 0 0) 0x7f01740cdf40 con 0x7f0174160890 2023-08-27T11:27:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f016bfff700 1 -- 172.21.15.45:0/3435860041 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01600035b0 con 0x7f0174160890 2023-08-27T11:27:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f01788d0700 1 -- 172.21.15.45:0/3435860041 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b62c0 msgr2=0x7f01740d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f01788d0700 1 --2- 172.21.15.45:0/3435860041 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b62c0 0x7f01740d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f01788d0700 1 -- 172.21.15.45:0/3435860041 >> v1:172.21.15.135:6789/0 conn(0x7f0174160890 legacy=0x7f01740b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f01788d0700 1 -- 172.21.15.45:0/3435860041 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01740ce470 con 0x7f01740b5220 2023-08-27T11:27:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f01788d0700 1 --2- 172.21.15.45:0/3435860041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01740b5220 0x7f01740cd8f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0164009840 tx=0x7f0164009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc68844db1ed9b1f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f016bfff700 1 -- 172.21.15.45:0/3435860041 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164011830 con 0x7f01740b5220 2023-08-27T11:27:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f016bfff700 1 -- 172.21.15.45:0/3435860041 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0164011990 con 0x7f01740b5220 2023-08-27T11:27:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.095+0000 7f016bfff700 1 -- 172.21.15.45:0/3435860041 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164011d20 con 0x7f01740b5220 2023-08-27T11:27:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.096+0000 7f017b335700 1 -- 172.21.15.45:0/3435860041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01740b5220 msgr2=0x7f01740cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.096+0000 7f017b335700 1 --2- 172.21.15.45:0/3435860041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01740b5220 0x7f01740cd8f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0164009840 tx=0x7f0164009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.096+0000 7f017b335700 1 -- 172.21.15.45:0/3435860041 shutdown_connections 2023-08-27T11:27:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.096+0000 7f017b335700 1 --2- 172.21.15.45:0/3435860041 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b62c0 0x7f01740d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.096+0000 7f017b335700 1 --2- 172.21.15.45:0/3435860041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01740b5220 0x7f01740cd8f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.096+0000 7f017b335700 1 -- 172.21.15.45:0/3435860041 >> 172.21.15.45:0/3435860041 conn(0x7f01740cfb10 msgr2=0x7f01740cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.096+0000 7f017b335700 1 -- 172.21.15.45:0/3435860041 shutdown_connections 2023-08-27T11:27:51.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.096+0000 7f017b335700 1 -- 172.21.15.45:0/3435860041 wait complete. 2023-08-27T11:27:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.097+0000 7f017b335700 1 Processor -- start 2023-08-27T11:27:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.097+0000 7f017b335700 1 -- start start 2023-08-27T11:27:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f017b335700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b5220 0x7f017415b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f017b335700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01740b62c0 0x7f0174146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f017b335700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0174160890 0x7f01740ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f017b335700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0174146550 con 0x7f01740b62c0 2023-08-27T11:27:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f017b335700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01740cd050 con 0x7f0174160890 2023-08-27T11:27:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f017b335700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0174152e00 con 0x7f01740b5220 2023-08-27T11:27:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01788d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01740b62c0 0x7f0174146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01788d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01740b62c0 0x7f0174146010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58566/0 (socket says 172.21.15.45:58566) 2023-08-27T11:27:51.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01788d0700 1 -- 172.21.15.45:0/641636477 learned_addr learned my addr 172.21.15.45:0/641636477 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:51.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01790d1700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b5220 0x7f017415b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01798d2700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0174160890 0x7f01740ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01788d0700 1 -- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b5220 msgr2=0x7f017415b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01788d0700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b5220 0x7f017415b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01788d0700 1 -- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0174160890 msgr2=0x7f01740ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01788d0700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0174160890 0x7f01740ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01788d0700 1 -- 172.21.15.45:0/641636477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01640094a0 con 0x7f01740b62c0 2023-08-27T11:27:51.109 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:51.098+0000 7fe70ddea200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:27:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.098+0000 7f01788d0700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01740b62c0 0x7f0174146010 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f01640090b0 tx=0x7f0164004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.099+0000 7f0169ffb700 1 -- 172.21.15.45:0/641636477 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164004590 con 0x7f01740b62c0 2023-08-27T11:27:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.099+0000 7f0169ffb700 1 -- 172.21.15.45:0/641636477 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016401f070 con 0x7f01740b62c0 2023-08-27T11:27:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.099+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01741530e0 con 0x7f01740b62c0 2023-08-27T11:27:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.099+0000 7f0169ffb700 1 -- 172.21.15.45:0/641636477 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640038e0 con 0x7f01740b62c0 2023-08-27T11:27:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.099+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01741536a0 con 0x7f01740b62c0 2023-08-27T11:27:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.100+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017406c120 con 0x7f01740b62c0 2023-08-27T11:27:51.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.101+0000 7f0169ffb700 1 -- 172.21.15.45:0/641636477 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f0164003a40 con 0x7f01740b62c0 2023-08-27T11:27:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.101+0000 7f0169ffb700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f015806a200 0x7f015806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.102+0000 7f0169ffb700 1 -- 172.21.15.45:0/641636477 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f0164021070 con 0x7f01740b62c0 2023-08-27T11:27:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.102+0000 7f01790d1700 1 -- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f015806a200 msgr2=0x7f015806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.102+0000 7f01790d1700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f015806a200 0x7f015806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.105+0000 7f0169ffb700 1 -- 172.21.15.45:0/641636477 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0164055a30 con 0x7f01740b62c0 2023-08-27T11:27:51.271 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:51.269+0000 7fe70ddea200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:27:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.302+0000 7f01790d1700 1 -- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f015806a200 msgr2=0x7f015806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.302+0000 7f01790d1700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f015806a200 0x7f015806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:27:51.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.305+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01740734a0 con 0x7f01740b62c0 2023-08-27T11:27:51.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.305+0000 7f0169ffb700 1 -- 172.21.15.45:0/641636477 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0164024380 con 0x7f01740b62c0 2023-08-27T11:27:51.307 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:27:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.307+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f015806a200 msgr2=0x7f015806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.307+0000 7f017b335700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f015806a200 0x7f015806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.307+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01740b62c0 msgr2=0x7f0174146010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.307+0000 7f017b335700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01740b62c0 0x7f0174146010 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f01640090b0 tx=0x7f0164004750 comp rx=0 tx=0).stop 2023-08-27T11:27:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.308+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 shutdown_connections 2023-08-27T11:27:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.308+0000 7f017b335700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0174160890 0x7f01740ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.308+0000 7f017b335700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01740b5220 0x7f017415b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.308+0000 7f017b335700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f015806a200 0x7f015806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.308+0000 7f017b335700 1 --2- 172.21.15.45:0/641636477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01740b62c0 0x7f0174146010 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.308+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 >> 172.21.15.45:0/641636477 conn(0x7f01740cfb10 msgr2=0x7f01740cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.308+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 shutdown_connections 2023-08-27T11:27:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.308+0000 7f017b335700 1 -- 172.21.15.45:0/641636477 wait complete. 2023-08-27T11:27:51.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:51.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.447+0000 7f6db3fca700 1 Processor -- start 2023-08-27T11:27:51.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.447+0000 7f6db3fca700 1 -- start start 2023-08-27T11:27:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.448+0000 7f6db3fca700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6dac06bda0 0x7f6dac06c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.448+0000 7f6db3fca700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 0x7f6dac0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.448+0000 7f6db3fca700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dac0cff50 con 0x7f6dac15d140 2023-08-27T11:27:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.448+0000 7f6db3fca700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dac0cc4e0 con 0x7f6dac06bda0 2023-08-27T11:27:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.448+0000 7f6db3fca700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac0ce820 con 0x7f6dac0cc790 2023-08-27T11:27:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.448+0000 7f6db1d66700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6dac15d140 0x7f6dac0ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45350/0 (socket says 172.21.15.45:45350) 2023-08-27T11:27:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.448+0000 7f6db1d66700 1 -- 172.21.15.45:0/1380169714 learned_addr learned my addr 172.21.15.45:0/1380169714 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.448+0000 7f6db2567700 1 --2- 172.21.15.45:0/1380169714 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 0x7f6dac0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.448+0000 7f6db1565700 1 --2- 172.21.15.45:0/1380169714 >> v2:172.21.15.135:3300/0 conn(0x7f6dac06bda0 0x7f6dac06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db0d64700 1 -- 172.21.15.45:0/1380169714 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576552759 0 0) 0x7f6dac0cff50 con 0x7f6dac15d140 2023-08-27T11:27:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db0d64700 1 -- 172.21.15.45:0/1380169714 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d980035b0 con 0x7f6dac15d140 2023-08-27T11:27:51.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db2567700 1 -- 172.21.15.45:0/1380169714 >> v2:172.21.15.135:3300/0 conn(0x7f6dac06bda0 msgr2=0x7f6dac06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:51.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db2567700 1 --2- 172.21.15.45:0/1380169714 >> v2:172.21.15.135:3300/0 conn(0x7f6dac06bda0 0x7f6dac06c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db2567700 1 -- 172.21.15.45:0/1380169714 >> v1:172.21.15.45:6789/0 conn(0x7f6dac15d140 legacy=0x7f6dac0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db2567700 1 -- 172.21.15.45:0/1380169714 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac0cf0a0 con 0x7f6dac0cc790 2023-08-27T11:27:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db1565700 1 --2- 172.21.15.45:0/1380169714 >> v2:172.21.15.135:3300/0 conn(0x7f6dac06bda0 0x7f6dac06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db2567700 1 --2- 172.21.15.45:0/1380169714 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 0x7f6dac0ccc20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009d90 tx=0x7f6da400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40916aa3139b84f2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db0d64700 1 -- 172.21.15.45:0/1380169714 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da400e070 con 0x7f6dac0cc790 2023-08-27T11:27:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db0d64700 1 -- 172.21.15.45:0/1380169714 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da400bb20 con 0x7f6dac0cc790 2023-08-27T11:27:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.449+0000 7f6db0d64700 1 -- 172.21.15.45:0/1380169714 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da400be90 con 0x7f6dac0cc790 2023-08-27T11:27:51.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.450+0000 7f6db3fca700 1 -- 172.21.15.45:0/1380169714 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 msgr2=0x7f6dac0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.450+0000 7f6db3fca700 1 --2- 172.21.15.45:0/1380169714 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 0x7f6dac0ccc20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009d90 tx=0x7f6da400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:51.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.450+0000 7f6db3fca700 1 -- 172.21.15.45:0/1380169714 shutdown_connections 2023-08-27T11:27:51.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.450+0000 7f6db3fca700 1 --2- 172.21.15.45:0/1380169714 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 0x7f6dac0ccc20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.450+0000 7f6db3fca700 1 --2- 172.21.15.45:0/1380169714 >> v2:172.21.15.135:3300/0 conn(0x7f6dac06bda0 0x7f6dac06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.450+0000 7f6db3fca700 1 -- 172.21.15.45:0/1380169714 >> 172.21.15.45:0/1380169714 conn(0x7f6dac09a160 msgr2=0x7f6dac0d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.450+0000 7f6db3fca700 1 -- 172.21.15.45:0/1380169714 shutdown_connections 2023-08-27T11:27:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.450+0000 7f6db3fca700 1 -- 172.21.15.45:0/1380169714 wait complete. 2023-08-27T11:27:51.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.451+0000 7f6db3fca700 1 Processor -- start 2023-08-27T11:27:51.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.451+0000 7f6db3fca700 1 -- start start 2023-08-27T11:27:51.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db3fca700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac06bda0 0x7f6dac1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db3fca700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 0x7f6dac0c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db3fca700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dac15d140 0x7f6dac0c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db3fca700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac15f020 con 0x7f6dac15d140 2023-08-27T11:27:51.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db3fca700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac09e9a0 con 0x7f6dac06bda0 2023-08-27T11:27:51.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db3fca700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac15e6f0 con 0x7f6dac0cc790 2023-08-27T11:27:51.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db2567700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dac15d140 0x7f6dac0c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db2567700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dac15d140 0x7f6dac0c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58584/0 (socket says 172.21.15.45:58584) 2023-08-27T11:27:51.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db2567700 1 -- 172.21.15.45:0/1926647026 learned_addr learned my addr 172.21.15.45:0/1926647026 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:51.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db1565700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 0x7f6dac0c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db1d66700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac06bda0 0x7f6dac1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db2567700 1 -- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 msgr2=0x7f6dac0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db2567700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 0x7f6dac0c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db2567700 1 -- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac06bda0 msgr2=0x7f6dac1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db2567700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac06bda0 0x7f6dac1593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.452+0000 7f6db2567700 1 -- 172.21.15.45:0/1926647026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0094a0 con 0x7f6dac15d140 2023-08-27T11:27:51.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.453+0000 7f6db2567700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dac15d140 0x7f6dac0c94e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6da4014040 tx=0x7f6da4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:51.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.453+0000 7f6da27fc700 1 -- 172.21.15.45:0/1926647026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4004800 con 0x7f6dac15d140 2023-08-27T11:27:51.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.453+0000 7f6da27fc700 1 -- 172.21.15.45:0/1926647026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da4020070 con 0x7f6dac15d140 2023-08-27T11:27:51.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.453+0000 7f6da27fc700 1 -- 172.21.15.45:0/1926647026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da40276c0 con 0x7f6dac15d140 2023-08-27T11:27:51.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.453+0000 7f6db3fca700 1 -- 172.21.15.45:0/1926647026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da4009a70 con 0x7f6dac15d140 2023-08-27T11:27:51.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.453+0000 7f6db3fca700 1 -- 172.21.15.45:0/1926647026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dac15ecb0 con 0x7f6dac15d140 2023-08-27T11:27:51.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.454+0000 7f6db3fca700 1 -- 172.21.15.45:0/1926647026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dac15f190 con 0x7f6dac15d140 2023-08-27T11:27:51.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.455+0000 7f6da27fc700 1 -- 172.21.15.45:0/1926647026 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 34) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f6da4012430 con 0x7f6dac15d140 2023-08-27T11:27:51.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.455+0000 7f6da27fc700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f6d9006a200 0x7f6d9006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.455+0000 7f6db1d66700 1 -- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f6d9006a200 msgr2=0x7f6d9006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/294910128 2023-08-27T11:27:51.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.455+0000 7f6da27fc700 1 -- 172.21.15.45:0/1926647026 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f6da4022070 con 0x7f6dac15d140 2023-08-27T11:27:51.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.455+0000 7f6db1d66700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f6d9006a200 0x7f6d9006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:27:51.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.460+0000 7f6da27fc700 1 -- 172.21.15.45:0/1926647026 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6da40558c0 con 0x7f6dac15d140 2023-08-27T11:27:51.470 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.469+0000 7fe8df8c2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:27:51.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.490+0000 7f6da27fc700 1 -- 172.21.15.45:0/1926647026 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 35) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f6da4055c90 con 0x7f6dac15d140 2023-08-27T11:27:51.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.490+0000 7f6da27fc700 1 -- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f6d9006a200 msgr2=0x7f6d9006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:27:51.492 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.490+0000 7f6da27fc700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f6d9006a200 0x7f6d9006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.550 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85812b700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.550 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85812b700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.550 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85812b700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.551 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85812b700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.551 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85812b700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.551 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85b932700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.551 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85b932700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.551 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85b932700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.551 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85b932700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.552 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:51.548+0000 7fe85b932700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:27:51.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.713+0000 7f6db3fca700 1 -- 172.21.15.45:0/1926647026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dac0afe20 con 0x7f6dac15d140 2023-08-27T11:27:51.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.714+0000 7f6da27fc700 1 -- 172.21.15.45:0/1926647026 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f6da405c000 con 0x7f6dac15d140 2023-08-27T11:27:51.717 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:51.717 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2178366980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2178366980},{"type":"v1","addr":"172.21.15.135:6836","nonce":2178366980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2939076938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2939076938},{"type":"v1","addr":"172.21.15.45:6833","nonce":2939076938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/148627485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":148627485},{"type":"v1","addr":"172.21.15.45:6835","nonce":148627485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:25.479170+0000","modified":"2023-08-27T11:27:29.096440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1457694749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1457694749},{"type":"v1","addr":"172.21.15.135:6837","nonce":1457694749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:27:51.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 -- 172.21.15.45:0/1926647026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dac15d140 msgr2=0x7f6dac0c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dac15d140 0x7f6dac0c94e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6da4014040 tx=0x7f6da4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 -- 172.21.15.45:0/1926647026 shutdown_connections 2023-08-27T11:27:51.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:6832/294910128,v1:172.21.15.135:6833/294910128] conn(0x7f6d9006a200 0x7f6d9006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dac15d140 0x7f6dac0c94e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dac06bda0 0x7f6dac1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 --2- 172.21.15.45:0/1926647026 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6dac0cc790 0x7f6dac0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 -- 172.21.15.45:0/1926647026 >> 172.21.15.45:0/1926647026 conn(0x7f6dac09a160 msgr2=0x7f6dac0d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:51.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 -- 172.21.15.45:0/1926647026 shutdown_connections 2023-08-27T11:27:51.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.716+0000 7f6db3fca700 1 -- 172.21.15.45:0/1926647026 wait complete. 2023-08-27T11:27:51.722 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2023-08-27T11:27:51.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:51.854 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:27:51.852+0000 7fe70ddea200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:27:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.855+0000 7f284d06d700 1 Processor -- start 2023-08-27T11:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.855+0000 7f284d06d700 1 -- start start 2023-08-27T11:27:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f284d06d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f284815b120 0x7f28480cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f284d06d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28480ada50 con 0x7f28480cdb80 2023-08-27T11:27:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f284d06d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28480ccd60 con 0x7f28480d1a20 2023-08-27T11:27:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f284d06d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28480b0860 con 0x7f284815b120 2023-08-27T11:27:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f284759e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f28480cdb80 0x7f28480b1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45368/0 (socket says 172.21.15.45:45368) 2023-08-27T11:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f284759e700 1 -- 172.21.15.45:0/3399313320 learned_addr learned my addr 172.21.15.45:0/3399313320 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2846d9d700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f284815b120 0x7f28480cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172356287 0 0) 0x7f28480ada50 con 0x7f28480cdb80 2023-08-27T11:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28300035b0 con 0x7f28480cdb80 2023-08-27T11:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251969638 0 0) 0x7f28480ccd60 con 0x7f28480d1a20 2023-08-27T11:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28480ada50 con 0x7f28480d1a20 2023-08-27T11:27:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1383074311 0 0) 0x7f28300035b0 con 0x7f28480cdb80 2023-08-27T11:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f284815b120 msgr2=0x7f28480cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f284815b120 0x7f28480cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 >> v1:172.21.15.135:6789/0 conn(0x7f28480d1a20 legacy=0x7f28480cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480b0b40 con 0x7f28480cdb80 2023-08-27T11:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f28400019f0 con 0x7f28480cdb80 2023-08-27T11:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2846d9d700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f284815b120 0x7f28480cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:51.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.856+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 >> v1:172.21.15.45:6789/0 conn(0x7f28480cdb80 legacy=0x7f28480b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2845d9b700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2830003bc0 0x7f2830003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2845d9b700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f284815b120 0x7f2830004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2845d9b700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2830004d90 0x7f2830005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28480b0860 con 0x7f2830004d90 2023-08-27T11:27:51.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283000bac0 con 0x7f2830003bc0 2023-08-27T11:27:51.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283000bc00 con 0x7f284815b120 2023-08-27T11:27:51.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f284759e700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2830004d90 0x7f2830005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2846d9d700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2830003bc0 0x7f2830003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f284659c700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f284815b120 0x7f2830004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2846d9d700 1 -- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f284815b120 msgr2=0x7f2830004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:51.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2846d9d700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f284815b120 0x7f2830004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2846d9d700 1 -- 172.21.15.45:0/3399313320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2830004d90 msgr2=0x7f2830005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:51.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2846d9d700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2830004d90 0x7f2830005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2846d9d700 1 -- 172.21.15.45:0/3399313320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28380094a0 con 0x7f2830003bc0 2023-08-27T11:27:51.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f284759e700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2830004d90 0x7f2830005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:51.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2846d9d700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2830003bc0 0x7f2830003fc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2838009d30 tx=0x7f2830005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea0851ffe884981 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:51.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838009e80 con 0x7f2830003bc0 2023-08-27T11:27:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283800e9a0 con 0x7f2830003bc0 2023-08-27T11:27:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.857+0000 7f2845d9b700 1 -- 172.21.15.45:0/3399313320 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283800ed10 con 0x7f2830003bc0 2023-08-27T11:27:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.858+0000 7f284d06d700 1 -- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2830003bc0 msgr2=0x7f2830003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.858+0000 7f284d06d700 1 --2- 172.21.15.45:0/3399313320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2830003bc0 0x7f2830003fc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2838009d30 tx=0x7f2830005880 comp rx=0 tx=0).stop 2023-08-27T11:27:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.858+0000 7f284759e700 1 -- 172.21.15.45:0/3399313320 reap_dead start 2023-08-27T11:27:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.858+0000 7f284d06d700 1 -- 172.21.15.45:0/3399313320 shutdown_connections 2023-08-27T11:27:51.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.858+0000 7f284d06d700 1 -- 172.21.15.45:0/3399313320 >> 172.21.15.45:0/3399313320 conn(0x7f284809a160 msgr2=0x7f28480d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:51.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.858+0000 7f284d06d700 1 -- 172.21.15.45:0/3399313320 shutdown_connections 2023-08-27T11:27:51.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.858+0000 7f284d06d700 1 -- 172.21.15.45:0/3399313320 wait complete. 2023-08-27T11:27:51.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.860+0000 7f284d06d700 1 Processor -- start 2023-08-27T11:27:51.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.860+0000 7f284d06d700 1 -- start start 2023-08-27T11:27:51.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284d06d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2830003bc0 0x7f284814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284d06d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28480b3aa0 0x7f284814ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284d06d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f284806b390 0x7f284806ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:51.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284d06d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28480b5ab0 con 0x7f284806b390 2023-08-27T11:27:51.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284d06d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284806d3d0 con 0x7f28480b3aa0 2023-08-27T11:27:51.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284d06d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284814f640 con 0x7f2830003bc0 2023-08-27T11:27:51.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f284806b390 0x7f284806ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f284806b390 0x7f284806ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58604/0 (socket says 172.21.15.45:58604) 2023-08-27T11:27:51.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284759e700 1 -- 172.21.15.45:0/2386350049 learned_addr learned my addr 172.21.15.45:0/2386350049 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:51.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284659c700 1 --2- 172.21.15.45:0/2386350049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28480b3aa0 0x7f284814ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f2846d9d700 1 --2- 172.21.15.45:0/2386350049 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2830003bc0 0x7f284814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:51.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284759e700 1 -- 172.21.15.45:0/2386350049 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2830003bc0 msgr2=0x7f284814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284759e700 1 --2- 172.21.15.45:0/2386350049 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2830003bc0 0x7f284814c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284759e700 1 -- 172.21.15.45:0/2386350049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28480b3aa0 msgr2=0x7f284814ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:51.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284759e700 1 --2- 172.21.15.45:0/2386350049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28480b3aa0 0x7f284814ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:51.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284759e700 1 -- 172.21.15.45:0/2386350049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284814f920 con 0x7f284806b390 2023-08-27T11:27:51.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.861+0000 7f284759e700 1 --2- 172.21.15.45:0/2386350049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f284806b390 0x7f284806ee90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f284806d9f0 tx=0x7f2840007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:51.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.862+0000 7f282f7fe700 1 -- 172.21.15.45:0/2386350049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400133f0 con 0x7f284806b390 2023-08-27T11:27:51.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.862+0000 7f282f7fe700 1 -- 172.21.15.45:0/2386350049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840013550 con 0x7f284806b390 2023-08-27T11:27:51.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.862+0000 7f284d06d700 1 -- 172.21.15.45:0/2386350049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284814fc00 con 0x7f284806b390 2023-08-27T11:27:51.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.862+0000 7f282f7fe700 1 -- 172.21.15.45:0/2386350049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840013900 con 0x7f284806b390 2023-08-27T11:27:51.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.862+0000 7f284d06d700 1 -- 172.21.15.45:0/2386350049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284806b810 con 0x7f284806b390 2023-08-27T11:27:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.863+0000 7f284d06d700 1 -- 172.21.15.45:0/2386350049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284806ba30 con 0x7f284806b390 2023-08-27T11:27:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.863+0000 7f282f7fe700 1 -- 172.21.15.45:0/2386350049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 35) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f2840013a60 con 0x7f284806b390 2023-08-27T11:27:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.864+0000 7f282f7fe700 1 -- 172.21.15.45:0/2386350049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f2840088ce0 con 0x7f284806b390 2023-08-27T11:27:51.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:51.869+0000 7f282f7fe700 1 -- 172.21.15.45:0/2386350049 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2840054400 con 0x7f284806b390 2023-08-27T11:27:52.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.069+0000 7f284d06d700 1 -- 172.21.15.45:0/2386350049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f284809edd0 con 0x7f284806b390 2023-08-27T11:27:52.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.070+0000 7f282f7fe700 1 -- 172.21.15.45:0/2386350049 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f284000a3f0 con 0x7f284806b390 2023-08-27T11:27:52.073 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:52.073 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:51.483650+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:23.082458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:24.453679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/3810104053":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/3438903793":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/4068027882":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/1613956798":"2023-08-28T11:27:51.483615+0000","172.21.15.135:6833/294910128":"2023-08-28T11:27:51.483615+0000","172.21.15.135:6832/294910128":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/3221283236":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/3835899160":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/790246089":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2253320845":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2129685370":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/289717412":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6833/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6832/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2380106040":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/4054422326":"2023-08-28T11:27:44.271608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:52.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.072+0000 7f284d06d700 1 -- 172.21.15.45:0/2386350049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f284806b390 msgr2=0x7f284806ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.072+0000 7f284d06d700 1 --2- 172.21.15.45:0/2386350049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f284806b390 0x7f284806ee90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f284806d9f0 tx=0x7f2840007570 comp rx=0 tx=0).stop 2023-08-27T11:27:52.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.072+0000 7f284d06d700 1 -- 172.21.15.45:0/2386350049 shutdown_connections 2023-08-27T11:27:52.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.072+0000 7f284d06d700 1 --2- 172.21.15.45:0/2386350049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f284806b390 0x7f284806ee90 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.072+0000 7f284d06d700 1 --2- 172.21.15.45:0/2386350049 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2830003bc0 0x7f284814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.072+0000 7f284d06d700 1 --2- 172.21.15.45:0/2386350049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28480b3aa0 0x7f284814ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.072+0000 7f284d06d700 1 -- 172.21.15.45:0/2386350049 >> 172.21.15.45:0/2386350049 conn(0x7f284809a160 msgr2=0x7f28480b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:52.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.072+0000 7f284d06d700 1 -- 172.21.15.45:0/2386350049 shutdown_connections 2023-08-27T11:27:52.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.072+0000 7f284d06d700 1 -- 172.21.15.45:0/2386350049 wait complete. 2023-08-27T11:27:52.087 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:27:52.087 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.213+0000 7ffb02d5b700 1 Processor -- start 2023-08-27T11:27:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.213+0000 7ffb02d5b700 1 -- start start 2023-08-27T11:27:52.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb02d5b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffafc15a1f0 0x7ffafc0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb02d5b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffafc0cb9d0 0x7ffafc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb02d5b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc0cc9a0 con 0x7ffafc071280 2023-08-27T11:27:52.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb02d5b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc072080 con 0x7ffafc15a1f0 2023-08-27T11:27:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb02d5b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc0cbe80 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb00af7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffafc15a1f0 0x7ffafc0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb012f8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffafc071280 0x7ffafc15dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45388/0 (socket says 172.21.15.45:45388) 2023-08-27T11:27:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb012f8700 1 -- 172.21.15.45:0/1647841230 learned_addr learned my addr 172.21.15.45:0/1647841230 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffafbfff700 1 --2- 172.21.15.45:0/1647841230 >> v2:172.21.15.135:3300/0 conn(0x7ffafc0cb9d0 0x7ffafc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb00af7700 1 -- 172.21.15.45:0/1647841230 >> v2:172.21.15.135:3300/0 conn(0x7ffafc0cb9d0 msgr2=0x7ffafc070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:52.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb00af7700 1 --2- 172.21.15.45:0/1647841230 >> v2:172.21.15.135:3300/0 conn(0x7ffafc0cb9d0 0x7ffafc070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb00af7700 1 -- 172.21.15.45:0/1647841230 >> v1:172.21.15.45:6789/0 conn(0x7ffafc071280 legacy=0x7ffafc15dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb00af7700 1 -- 172.21.15.45:0/1647841230 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc071ac0 con 0x7ffafc15a1f0 2023-08-27T11:27:52.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffafb7fe700 1 -- 172.21.15.45:0/1647841230 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463236294 0 0) 0x7ffafc0cc9a0 con 0x7ffafc071280 2023-08-27T11:27:52.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffafbfff700 1 --2- 172.21.15.45:0/1647841230 >> v2:172.21.15.135:3300/0 conn(0x7ffafc0cb9d0 0x7ffafc070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:52.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffb00af7700 1 --2- 172.21.15.45:0/1647841230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc15a1f0 0x7ffafc0cb490 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009840 tx=0x7ffaf0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6facc211afe902 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:52.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffafb7fe700 1 -- 172.21.15.45:0/1647841230 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0011830 con 0x7ffafc15a1f0 2023-08-27T11:27:52.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffafb7fe700 1 -- 172.21.15.45:0/1647841230 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf0011990 con 0x7ffafc15a1f0 2023-08-27T11:27:52.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.214+0000 7ffafb7fe700 1 -- 172.21.15.45:0/1647841230 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0011d20 con 0x7ffafc15a1f0 2023-08-27T11:27:52.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.215+0000 7ffb02d5b700 1 -- 172.21.15.45:0/1647841230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc15a1f0 msgr2=0x7ffafc0cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.215+0000 7ffb02d5b700 1 --2- 172.21.15.45:0/1647841230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc15a1f0 0x7ffafc0cb490 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009840 tx=0x7ffaf0009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:52.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.215+0000 7ffb02d5b700 1 -- 172.21.15.45:0/1647841230 shutdown_connections 2023-08-27T11:27:52.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.215+0000 7ffb02d5b700 1 --2- 172.21.15.45:0/1647841230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc15a1f0 0x7ffafc0cb490 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.215+0000 7ffb02d5b700 1 --2- 172.21.15.45:0/1647841230 >> v2:172.21.15.135:3300/0 conn(0x7ffafc0cb9d0 0x7ffafc070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.215+0000 7ffb02d5b700 1 -- 172.21.15.45:0/1647841230 >> 172.21.15.45:0/1647841230 conn(0x7ffafc0b07c0 msgr2=0x7ffafc0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.215+0000 7ffb02d5b700 1 -- 172.21.15.45:0/1647841230 shutdown_connections 2023-08-27T11:27:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.215+0000 7ffb02d5b700 1 -- 172.21.15.45:0/1647841230 wait complete. 2023-08-27T11:27:52.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.217+0000 7ffb02d5b700 1 Processor -- start 2023-08-27T11:27:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.217+0000 7ffb02d5b700 1 -- start start 2023-08-27T11:27:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffb02d5b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffafc071280 0x7ffafc0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffb02d5b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc0cb9d0 0x7ffafc0c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffb02d5b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc15a1f0 0x7ffafc15e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffb02d5b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc14b890 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffb02d5b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc14c6b0 con 0x7ffafc15a1f0 2023-08-27T11:27:52.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffb02d5b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc14baa0 con 0x7ffafc071280 2023-08-27T11:27:52.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffafbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc0cb9d0 0x7ffafc0c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffafbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc0cb9d0 0x7ffafc0c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58616/0 (socket says 172.21.15.45:58616) 2023-08-27T11:27:52.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffafbfff700 1 -- 172.21.15.45:0/2643923031 learned_addr learned my addr 172.21.15.45:0/2643923031 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:52.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffb012f8700 1 --2- 172.21.15.45:0/2643923031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc15a1f0 0x7ffafc15e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffb00af7700 1 --2- 172.21.15.45:0/2643923031 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffafc071280 0x7ffafc0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffafbfff700 1 -- 172.21.15.45:0/2643923031 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffafc071280 msgr2=0x7ffafc0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffafbfff700 1 --2- 172.21.15.45:0/2643923031 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffafc071280 0x7ffafc0c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffafbfff700 1 -- 172.21.15.45:0/2643923031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc15a1f0 msgr2=0x7ffafc15e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffafbfff700 1 --2- 172.21.15.45:0/2643923031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc15a1f0 0x7ffafc15e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffafbfff700 1 -- 172.21.15.45:0/2643923031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf00094a0 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffafbfff700 1 --2- 172.21.15.45:0/2643923031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc0cb9d0 0x7ffafc0c49a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ffae800c750 tx=0x7ffae800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:52.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffaf97fa700 1 -- 172.21.15.45:0/2643923031 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8011800 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffaf97fa700 1 -- 172.21.15.45:0/2643923031 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae8011960 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.218+0000 7ffaf97fa700 1 -- 172.21.15.45:0/2643923031 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8011cd0 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.219+0000 7ffb02d5b700 1 -- 172.21.15.45:0/2643923031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc14bd80 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.219+0000 7ffb02d5b700 1 -- 172.21.15.45:0/2643923031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc14c340 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.219+0000 7ffb02d5b700 1 -- 172.21.15.45:0/2643923031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc04e620 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.220+0000 7ffaf97fa700 1 -- 172.21.15.45:0/2643923031 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 35) v1 ==== 88052+0+0 (secure 0 0 0) 0x7ffae8021350 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.220+0000 7ffaf97fa700 1 -- 172.21.15.45:0/2643923031 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7114+0+0 (secure 0 0 0) 0x7ffae80883a0 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.225+0000 7ffaf97fa700 1 -- 172.21.15.45:0/2643923031 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffae80538f0 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.475 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.473+0000 7ffb02d5b700 1 -- 172.21.15.45:0/2643923031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffafc14c7f0 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.476 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.474+0000 7ffaf97fa700 1 -- 172.21.15.45:0/2643923031 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4880 (secure 0 0 0) 0x7ffae80570e0 con 0x7ffafc0cb9d0 2023-08-27T11:27:52.476 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:52.476 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2178366980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2178366980},{"type":"v1","addr":"172.21.15.135:6836","nonce":2178366980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2939076938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2939076938},{"type":"v1","addr":"172.21.15.45:6833","nonce":2939076938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/148627485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":148627485},{"type":"v1","addr":"172.21.15.45:6835","nonce":148627485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:25.479170+0000","modified":"2023-08-27T11:27:29.096440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1457694749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1457694749},{"type":"v1","addr":"172.21.15.135:6837","nonce":1457694749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:27:52.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.476+0000 7ffb02d5b700 1 -- 172.21.15.45:0/2643923031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc0cb9d0 msgr2=0x7ffafc0c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.476+0000 7ffb02d5b700 1 --2- 172.21.15.45:0/2643923031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc0cb9d0 0x7ffafc0c49a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ffae800c750 tx=0x7ffae800cb10 comp rx=0 tx=0).stop 2023-08-27T11:27:52.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.476+0000 7ffb02d5b700 1 -- 172.21.15.45:0/2643923031 shutdown_connections 2023-08-27T11:27:52.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.476+0000 7ffb02d5b700 1 --2- 172.21.15.45:0/2643923031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc15a1f0 0x7ffafc15e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.476+0000 7ffb02d5b700 1 --2- 172.21.15.45:0/2643923031 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffafc071280 0x7ffafc0c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.476+0000 7ffb02d5b700 1 --2- 172.21.15.45:0/2643923031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffafc0cb9d0 0x7ffafc0c49a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.476+0000 7ffb02d5b700 1 -- 172.21.15.45:0/2643923031 >> 172.21.15.45:0/2643923031 conn(0x7ffafc0b07c0 msgr2=0x7ffafc0c8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:52.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.476+0000 7ffb02d5b700 1 -- 172.21.15.45:0/2643923031 shutdown_connections 2023-08-27T11:27:52.480 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.476+0000 7ffb02d5b700 1 -- 172.21.15.45:0/2643923031 wait complete. 2023-08-27T11:27:52.480 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2023-08-27T11:27:52.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:27:52.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.616+0000 7fd0ffc73700 1 Processor -- start 2023-08-27T11:27:52.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.616+0000 7fd0ffc73700 1 -- start start 2023-08-27T11:27:52.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0ffc73700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f815b070 0x7fd0f80cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0ffc73700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0f806e980 0x7fd0f80ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0ffc73700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f815dad0 con 0x7fd0f80cdf60 2023-08-27T11:27:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0ffc73700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f815d8e0 con 0x7fd0f806e980 2023-08-27T11:27:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0ffc73700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f815ddb0 con 0x7fd0f815b070 2023-08-27T11:27:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0fe210700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0f806e980 0x7fd0f80ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0fe210700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0f806e980 0x7fd0f80ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58636/0 (socket says 172.21.15.45:58636) 2023-08-27T11:27:52.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0fe210700 1 -- 172.21.15.45:0/855695812 learned_addr learned my addr 172.21.15.45:0/855695812 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:52.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0fe210700 1 -- 172.21.15.45:0/855695812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f815b070 msgr2=0x7fd0f80cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:52.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0fda0f700 1 --2- 172.21.15.45:0/855695812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f815b070 0x7fd0f80cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0fe210700 1 --2- 172.21.15.45:0/855695812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f815b070 0x7fd0f80cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0fe210700 1 -- 172.21.15.45:0/855695812 >> v1:172.21.15.135:6789/0 conn(0x7fd0f80cdf60 legacy=0x7fd0f806e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:52.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0fe210700 1 -- 172.21.15.45:0/855695812 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f815e090 con 0x7fd0f806e980 2023-08-27T11:27:52.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.617+0000 7fd0fe210700 1 --2- 172.21.15.45:0/855695812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f806e980 0x7fd0f80ae8d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009e50 tx=0x7fd0f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754b5947f679675f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:52.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0fca0d700 1 -- 172.21.15.45:0/855695812 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f000e070 con 0x7fd0f806e980 2023-08-27T11:27:52.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0fca0d700 1 -- 172.21.15.45:0/855695812 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f0003a20 con 0x7fd0f806e980 2023-08-27T11:27:52.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0fca0d700 1 -- 172.21.15.45:0/855695812 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0003dd0 con 0x7fd0f806e980 2023-08-27T11:27:52.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0ffc73700 1 -- 172.21.15.45:0/855695812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f806e980 msgr2=0x7fd0f80ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0ffc73700 1 --2- 172.21.15.45:0/855695812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f806e980 0x7fd0f80ae8d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009e50 tx=0x7fd0f000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0ffc73700 1 -- 172.21.15.45:0/855695812 shutdown_connections 2023-08-27T11:27:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0ffc73700 1 --2- 172.21.15.45:0/855695812 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f815b070 0x7fd0f80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0ffc73700 1 --2- 172.21.15.45:0/855695812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f806e980 0x7fd0f80ae8d0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0ffc73700 1 -- 172.21.15.45:0/855695812 >> 172.21.15.45:0/855695812 conn(0x7fd0f80cfe50 msgr2=0x7fd0f80d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:52.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0ffc73700 1 -- 172.21.15.45:0/855695812 shutdown_connections 2023-08-27T11:27:52.627 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.618+0000 7fd0ffc73700 1 -- 172.21.15.45:0/855695812 wait complete. 2023-08-27T11:27:52.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.619+0000 7fd0ffc73700 1 Processor -- start 2023-08-27T11:27:52.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0ffc73700 1 -- start start 2023-08-27T11:27:52.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0ffc73700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f806e980 0x7fd0f80b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0ffc73700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f80cdf60 0x7fd0f80c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0ffc73700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f815b070 0x7fd0f80c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0ffc73700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80c6290 con 0x7fd0f815b070 2023-08-27T11:27:52.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0ffc73700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80c6570 con 0x7fd0f80cdf60 2023-08-27T11:27:52.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0ffc73700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80c6850 con 0x7fd0f806e980 2023-08-27T11:27:52.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0fe210700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f815b070 0x7fd0f80c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0fe210700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f815b070 0x7fd0f80c4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58642/0 (socket says 172.21.15.45:58642) 2023-08-27T11:27:52.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.620+0000 7fd0fe210700 1 -- 172.21.15.45:0/4089520533 learned_addr learned my addr 172.21.15.45:0/4089520533 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:52.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0fda0f700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f806e980 0x7fd0f80b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0fd20e700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f80cdf60 0x7fd0f80c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0fe210700 1 -- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f806e980 msgr2=0x7fd0f80b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0fe210700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f806e980 0x7fd0f80b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0fe210700 1 -- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f80cdf60 msgr2=0x7fd0f80c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0fe210700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f80cdf60 0x7fd0f80c45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0fe210700 1 -- 172.21.15.45:0/4089520533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f0009b30 con 0x7fd0f815b070 2023-08-27T11:27:52.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0fe210700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f815b070 0x7fd0f80c4b20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80cc7b0 tx=0x7fd0f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0ee7fc700 1 -- 172.21.15.45:0/4089520533 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0004440 con 0x7fd0f815b070 2023-08-27T11:27:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0ee7fc700 1 -- 172.21.15.45:0/4089520533 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f00045a0 con 0x7fd0f815b070 2023-08-27T11:27:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0ee7fc700 1 -- 172.21.15.45:0/4089520533 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0004950 con 0x7fd0f815b070 2023-08-27T11:27:52.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f8154020 con 0x7fd0f815b070 2023-08-27T11:27:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.621+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f8146600 con 0x7fd0f815b070 2023-08-27T11:27:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.622+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8146790 con 0x7fd0f815b070 2023-08-27T11:27:52.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.623+0000 7fd0ee7fc700 1 -- 172.21.15.45:0/4089520533 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 36) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fd0f0019030 con 0x7fd0f815b070 2023-08-27T11:27:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.623+0000 7fd0ee7fc700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd0dc069f30 0x7fd0dc06c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.623+0000 7fd0ee7fc700 1 -- 172.21.15.45:0/4089520533 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fd0f0089790 con 0x7fd0f815b070 2023-08-27T11:27:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.623+0000 7fd0fda0f700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd0dc069f30 0x7fd0dc06c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:52.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.624+0000 7fd0fda0f700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd0dc069f30 0x7fd0dc06c3f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8156440 tx=0x7fd0f4008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.628+0000 7fd0ee7fc700 1 -- 172.21.15.45:0/4089520533 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd0f0058830 con 0x7fd0f815b070 2023-08-27T11:27:52.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.870+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0f81544e0 con 0x7fd0f815b070 2023-08-27T11:27:52.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.871+0000 7fd0ee7fc700 1 -- 172.21.15.45:0/4089520533 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fd0f00583c0 con 0x7fd0f815b070 2023-08-27T11:27:52.873 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2178366980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2178366980},{"type":"v1","addr":"172.21.15.135:6836","nonce":2178366980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2939076938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2939076938},{"type":"v1","addr":"172.21.15.45:6833","nonce":2939076938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/148627485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":148627485},{"type":"v1","addr":"172.21.15.45:6835","nonce":148627485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:27:25.479170+0000","modified":"2023-08-27T11:27:29.096440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/1457694749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1457694749},{"type":"v1","addr":"172.21.15.135:6837","nonce":1457694749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-27T11:27:52.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd0dc069f30 msgr2=0x7fd0dc06c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd0dc069f30 0x7fd0dc06c3f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8156440 tx=0x7fd0f4008040 comp rx=0 tx=0).stop 2023-08-27T11:27:52.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f815b070 msgr2=0x7fd0f80c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:52.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f815b070 0x7fd0f80c4b20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80cc7b0 tx=0x7fd0f0004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 shutdown_connections 2023-08-27T11:27:52.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd0dc069f30 0x7fd0dc06c3f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f815b070 0x7fd0f80c4b20 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd0f806e980 0x7fd0f80b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 --2- 172.21.15.45:0/4089520533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0f80cdf60 0x7fd0f80c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:52.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.872+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 >> 172.21.15.45:0/4089520533 conn(0x7fd0f80cfe50 msgr2=0x7fd0f80d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:52.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.873+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 shutdown_connections 2023-08-27T11:27:52.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:52.873+0000 7fd0ffc73700 1 -- 172.21.15.45:0/4089520533 wait complete. 2023-08-27T11:27:52.878 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2023-08-27T11:27:52.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:53.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.010+0000 7fb7c19d0700 1 Processor -- start 2023-08-27T11:27:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.010+0000 7fb7c19d0700 1 -- start start 2023-08-27T11:27:53.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7c19d0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc15d2c0 0x7fb7bc0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7c19d0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb7bc0b8c20 0x7fb7bc0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7c19d0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7bc0ce2e0 0x7fb7bc06c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7c19d0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7bc0ce770 con 0x7fb7bc0ce2e0 2023-08-27T11:27:53.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7c19d0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7bc15d6a0 con 0x7fb7bc0b8c20 2023-08-27T11:27:53.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7c19d0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0d2c90 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7bb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7bc0ce2e0 0x7fb7bc06c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7bb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7bc0ce2e0 0x7fb7bc06c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58652/0 (socket says 172.21.15.45:58652) 2023-08-27T11:27:53.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7bb7fe700 1 -- 172.21.15.45:0/439688526 learned_addr learned my addr 172.21.15.45:0/439688526 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:53.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7baffd700 1 --2- 172.21.15.45:0/439688526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc15d2c0 0x7fb7bc0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7ba7fc700 1 --2- 172.21.15.45:0/439688526 >> v2:172.21.15.135:3300/0 conn(0x7fb7bc0b8c20 0x7fb7bc0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7bb7fe700 1 -- 172.21.15.45:0/439688526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc15d2c0 msgr2=0x7fb7bc0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7bb7fe700 1 --2- 172.21.15.45:0/439688526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc15d2c0 0x7fb7bc0d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7bb7fe700 1 -- 172.21.15.45:0/439688526 >> v2:172.21.15.135:3300/0 conn(0x7fb7bc0b8c20 msgr2=0x7fb7bc0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7bb7fe700 1 --2- 172.21.15.45:0/439688526 >> v2:172.21.15.135:3300/0 conn(0x7fb7bc0b8c20 0x7fb7bc0cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.011+0000 7fb7bb7fe700 1 -- 172.21.15.45:0/439688526 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc0d1bf0 con 0x7fb7bc0ce2e0 2023-08-27T11:27:53.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7bb7fe700 1 --2- 172.21.15.45:0/439688526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc0ce2e0 0x7fb7bc06c260 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc072920 tx=0x7fb7b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a5a367c483422c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7b9ffb700 1 -- 172.21.15.45:0/439688526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400e070 con 0x7fb7bc0ce2e0 2023-08-27T11:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7b9ffb700 1 -- 172.21.15.45:0/439688526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b400b9c0 con 0x7fb7bc0ce2e0 2023-08-27T11:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7b9ffb700 1 -- 172.21.15.45:0/439688526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400bd70 con 0x7fb7bc0ce2e0 2023-08-27T11:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7c19d0700 1 -- 172.21.15.45:0/439688526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc0ce2e0 msgr2=0x7fb7bc06c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/439688526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc0ce2e0 0x7fb7bc06c260 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc072920 tx=0x7fb7b400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7c19d0700 1 -- 172.21.15.45:0/439688526 shutdown_connections 2023-08-27T11:27:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/439688526 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc15d2c0 0x7fb7bc0d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/439688526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc0ce2e0 0x7fb7bc06c260 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/439688526 >> v2:172.21.15.135:3300/0 conn(0x7fb7bc0b8c20 0x7fb7bc0cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7c19d0700 1 -- 172.21.15.45:0/439688526 >> 172.21.15.45:0/439688526 conn(0x7fb7bc0d2fd0 msgr2=0x7fb7bc0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7c19d0700 1 -- 172.21.15.45:0/439688526 shutdown_connections 2023-08-27T11:27:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.012+0000 7fb7c19d0700 1 -- 172.21.15.45:0/439688526 wait complete. 2023-08-27T11:27:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.014+0000 7fb7c19d0700 1 Processor -- start 2023-08-27T11:27:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.014+0000 7fb7c19d0700 1 -- start start 2023-08-27T11:27:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7c19d0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc0b8c20 0x7fb7bc1557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7c19d0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7bc0ce2e0 0x7fb7bc153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7c19d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc15d2c0 0x7fb7bc0c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7c19d0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc156a30 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7c19d0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0c8a60 con 0x7fb7bc0ce2e0 2023-08-27T11:27:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7c19d0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc155db0 con 0x7fb7bc0b8c20 2023-08-27T11:27:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc15d2c0 0x7fb7bc0c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7bb7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc15d2c0 0x7fb7bc0c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58658/0 (socket says 172.21.15.45:58658) 2023-08-27T11:27:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7bb7fe700 1 -- 172.21.15.45:0/1284375453 learned_addr learned my addr 172.21.15.45:0/1284375453 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7baffd700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc0b8c20 0x7fb7bc1557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7ba7fc700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7bc0ce2e0 0x7fb7bc153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7bb7fe700 1 -- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc0b8c20 msgr2=0x7fb7bc1557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7bb7fe700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc0b8c20 0x7fb7bc1557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7bb7fe700 1 -- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7bc0ce2e0 msgr2=0x7fb7bc153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.015+0000 7fb7bb7fe700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7bc0ce2e0 0x7fb7bc153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.016+0000 7fb7bb7fe700 1 -- 172.21.15.45:0/1284375453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4009b30 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.016+0000 7fb7bb7fe700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc15d2c0 0x7fb7bc0c8520 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc06d270 tx=0x7fb7b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.016+0000 7fb7b3fff700 1 -- 172.21.15.45:0/1284375453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4004880 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.016+0000 7fb7b3fff700 1 -- 172.21.15.45:0/1284375453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b40099b0 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.016+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc156090 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.016+0000 7fb7b3fff700 1 -- 172.21.15.45:0/1284375453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b401c680 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.016+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc156650 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.017+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7bc156b70 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.017+0000 7fb7b3fff700 1 -- 172.21.15.45:0/1284375453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 36) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fb7b401c7e0 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.018+0000 7fb7b3fff700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb79c069f30 0x7fb79c06c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.018+0000 7fb7b3fff700 1 -- 172.21.15.45:0/1284375453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fb7b4089d80 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.018+0000 7fb7baffd700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb79c069f30 0x7fb79c06c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.019+0000 7fb7baffd700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb79c069f30 0x7fb79c06c3f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac006b20 tx=0x7fb7ac008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.023+0000 7fb7b3fff700 1 -- 172.21.15.45:0/1284375453 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb7b40554b0 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.223+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7bc04e620 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.224+0000 7fb7b3fff700 1 -- 172.21.15.45:0/1284375453 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+19652 (secure 0 0 0) 0x7fb7b4058ad0 con 0x7fb7bc15d2c0 2023-08-27T11:27:53.226 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:53.227 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:51.483650+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:27:23.082458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-27T11:27:24.453679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/3810104053":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/3438903793":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/4068027882":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/1613956798":"2023-08-28T11:27:51.483615+0000","172.21.15.135:6833/294910128":"2023-08-28T11:27:51.483615+0000","172.21.15.135:6832/294910128":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/3221283236":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/3835899160":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/790246089":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2253320845":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2129685370":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/289717412":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6833/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6832/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2380106040":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/4054422326":"2023-08-28T11:27:44.271608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:27:53.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb79c069f30 msgr2=0x7fb79c06c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb79c069f30 0x7fb79c06c3f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac006b20 tx=0x7fb7ac008040 comp rx=0 tx=0).stop 2023-08-27T11:27:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc15d2c0 msgr2=0x7fb7bc0c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc15d2c0 0x7fb7bc0c8520 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc06d270 tx=0x7fb7b4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 shutdown_connections 2023-08-27T11:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb79c069f30 0x7fb79c06c3f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7bc15d2c0 0x7fb7bc0c8520 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb7bc0b8c20 0x7fb7bc1557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 --2- 172.21.15.45:0/1284375453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7bc0ce2e0 0x7fb7bc153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 >> 172.21.15.45:0/1284375453 conn(0x7fb7bc0d2fd0 msgr2=0x7fb7bc15c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 shutdown_connections 2023-08-27T11:27:53.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.226+0000 7fb7c19d0700 1 -- 172.21.15.45:0/1284375453 wait complete. 2023-08-27T11:27:53.241 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:27:53.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.366+0000 7fa32cf67700 1 Processor -- start 2023-08-27T11:27:53.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.366+0000 7fa32cf67700 1 -- start start 2023-08-27T11:27:53.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32cf67700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa3280cbba0 0x7fa328070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32cf67700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa328071280 0x7fa3280cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32cf67700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3280cc9a0 con 0x7fa32815a1f0 2023-08-27T11:27:53.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32cf67700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa328072080 con 0x7fa3280cbba0 2023-08-27T11:27:53.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32cf67700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa328071ac0 con 0x7fa328071280 2023-08-27T11:27:53.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa32815a1f0 0x7fa3280cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45438/0 (socket says 172.21.15.45:45438) 2023-08-27T11:27:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32659c700 1 -- 172.21.15.45:0/1621796037 learned_addr learned my addr 172.21.15.45:0/1621796037 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa326d9d700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa328071280 0x7fa3280cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa325d9b700 1 --2- 172.21.15.45:0/1621796037 >> v2:172.21.15.135:3300/0 conn(0x7fa3280cbba0 0x7fa328070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025910425 0 0) 0x7fa3280cc9a0 con 0x7fa32815a1f0 2023-08-27T11:27:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3100035b0 con 0x7fa32815a1f0 2023-08-27T11:27:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4144234977 0 0) 0x7fa3100035b0 con 0x7fa32815a1f0 2023-08-27T11:27:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa328071280 msgr2=0x7fa3280cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa328071280 0x7fa3280cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 >> v2:172.21.15.135:3300/0 conn(0x7fa3280cbba0 msgr2=0x7fa328070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 --2- 172.21.15.45:0/1621796037 >> v2:172.21.15.135:3300/0 conn(0x7fa3280cbba0 0x7fa328070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa328071da0 con 0x7fa32815a1f0 2023-08-27T11:27:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fa31c001670 con 0x7fa32815a1f0 2023-08-27T11:27:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 >> v1:172.21.15.45:6789/0 conn(0x7fa32815a1f0 legacy=0x7fa3280cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.367+0000 7fa32559a700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa310003a00 0x7fa310003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32559a700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310004850 0x7fa310004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32559a700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa310005210 0x7fa31000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa328072080 con 0x7fa310003a00 2023-08-27T11:27:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa328071ac0 con 0x7fa310005210 2023-08-27T11:27:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31000bf50 con 0x7fa310004850 2023-08-27T11:27:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa326d9d700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa328071280 0x7fa3280cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa325d9b700 1 --2- 172.21.15.45:0/1621796037 >> v2:172.21.15.135:3300/0 conn(0x7fa3280cbba0 0x7fa328070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32659c700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa310003a00 0x7fa310003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa325d9b700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310004850 0x7fa310004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa326d9d700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa310005210 0x7fa31000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32659c700 1 -- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310004850 msgr2=0x7fa310004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32659c700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310004850 0x7fa310004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32659c700 1 -- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa310005210 msgr2=0x7fa31000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32659c700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa310005210 0x7fa31000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32659c700 1 -- 172.21.15.45:0/1621796037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3100035b0 con 0x7fa310003a00 2023-08-27T11:27:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32659c700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa310003a00 0x7fa310003e20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00c2a0 tx=0x7fa310005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170a90be911fabda server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c007c10 con 0x7fa310003a00 2023-08-27T11:27:53.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c00cde0 con 0x7fa310003a00 2023-08-27T11:27:53.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.368+0000 7fa32559a700 1 -- 172.21.15.45:0/1621796037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c005ba0 con 0x7fa310003a00 2023-08-27T11:27:53.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.369+0000 7fa32cf67700 1 -- 172.21.15.45:0/1621796037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa310003a00 msgr2=0x7fa310003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.369+0000 7fa32cf67700 1 --2- 172.21.15.45:0/1621796037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa310003a00 0x7fa310003e20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00c2a0 tx=0x7fa310005880 comp rx=0 tx=0).stop 2023-08-27T11:27:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.369+0000 7fa326d9d700 1 -- 172.21.15.45:0/1621796037 reap_dead start 2023-08-27T11:27:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.369+0000 7fa32cf67700 1 -- 172.21.15.45:0/1621796037 shutdown_connections 2023-08-27T11:27:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.369+0000 7fa32cf67700 1 -- 172.21.15.45:0/1621796037 >> 172.21.15.45:0/1621796037 conn(0x7fa3280b07c0 msgr2=0x7fa3280b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.369+0000 7fa32cf67700 1 -- 172.21.15.45:0/1621796037 shutdown_connections 2023-08-27T11:27:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.369+0000 7fa32cf67700 1 -- 172.21.15.45:0/1621796037 wait complete. 2023-08-27T11:27:53.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.371+0000 7fa32cf67700 1 Processor -- start 2023-08-27T11:27:53.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.371+0000 7fa32cf67700 1 -- start start 2023-08-27T11:27:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.371+0000 7fa32cf67700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310003a00 0x7fa3281f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa32cf67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3281458c0 0x7fa3280c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa32cf67700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32815aec0 0x7fa32815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa32cf67700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32815aaa0 con 0x7fa3281458c0 2023-08-27T11:27:53.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa32cf67700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32815a750 con 0x7fa32815aec0 2023-08-27T11:27:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa32cf67700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3281460d0 con 0x7fa310003a00 2023-08-27T11:27:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa325d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3281458c0 0x7fa3280c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa325d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3281458c0 0x7fa3280c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58676/0 (socket says 172.21.15.45:58676) 2023-08-27T11:27:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa325d9b700 1 -- 172.21.15.45:0/583181689 learned_addr learned my addr 172.21.15.45:0/583181689 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa32659c700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310003a00 0x7fa3281f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa326d9d700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32815aec0 0x7fa32815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa325d9b700 1 -- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310003a00 msgr2=0x7fa3281f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa325d9b700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310003a00 0x7fa3281f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa325d9b700 1 -- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32815aec0 msgr2=0x7fa32815b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa325d9b700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32815aec0 0x7fa32815b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa325d9b700 1 -- 172.21.15.45:0/583181689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3281463b0 con 0x7fa3281458c0 2023-08-27T11:27:53.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa32659c700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310003a00 0x7fa3281f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:53.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa326d9d700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32815aec0 0x7fa32815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:53.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa325d9b700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3281458c0 0x7fa3280c7c40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa318012c40 tx=0x7fa3180103e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa316ffd700 1 -- 172.21.15.45:0/583181689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318010cb0 con 0x7fa3281458c0 2023-08-27T11:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa316ffd700 1 -- 172.21.15.45:0/583181689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa318010e10 con 0x7fa3281458c0 2023-08-27T11:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa328146690 con 0x7fa3281458c0 2023-08-27T11:27:53.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa316ffd700 1 -- 172.21.15.45:0/583181689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31801c620 con 0x7fa3281458c0 2023-08-27T11:27:53.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.372+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3280c8180 con 0x7fa3281458c0 2023-08-27T11:27:53.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.373+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32804e620 con 0x7fa3281458c0 2023-08-27T11:27:53.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.374+0000 7fa316ffd700 1 -- 172.21.15.45:0/583181689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 36) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fa31801a070 con 0x7fa3281458c0 2023-08-27T11:27:53.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.374+0000 7fa316ffd700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa308069f30 0x7fa30806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.374+0000 7fa316ffd700 1 -- 172.21.15.45:0/583181689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fa31808a4b0 con 0x7fa3281458c0 2023-08-27T11:27:53.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.375+0000 7fa32659c700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa308069f30 0x7fa30806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.375+0000 7fa32659c700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa308069f30 0x7fa30806c3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa31c015040 tx=0x7fa31c007000 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.379+0000 7fa316ffd700 1 -- 172.21.15.45:0/583181689 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa318055ac0 con 0x7fa3281458c0 2023-08-27T11:27:53.529 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:53.528+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.622+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa32806d370 con 0x7fa3281458c0 2023-08-27T11:27:53.625 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-08-27T11:27:53.623+0000 7f87a1c48700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:27:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.627+0000 7fa316ffd700 1 -- 172.21.15.45:0/583181689 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa318009300 con 0x7fa3281458c0 2023-08-27T11:27:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa308069f30 msgr2=0x7fa30806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa308069f30 0x7fa30806c3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa31c015040 tx=0x7fa31c007000 comp rx=0 tx=0).stop 2023-08-27T11:27:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3281458c0 msgr2=0x7fa3280c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3281458c0 0x7fa3280c7c40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa318012c40 tx=0x7fa3180103e0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 shutdown_connections 2023-08-27T11:27:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa308069f30 0x7fa30806c3f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32815aec0 0x7fa32815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa310003a00 0x7fa3281f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 --2- 172.21.15.45:0/583181689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3281458c0 0x7fa3280c7c40 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 >> 172.21.15.45:0/583181689 conn(0x7fa3280b07c0 msgr2=0x7fa3280cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 shutdown_connections 2023-08-27T11:27:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.629+0000 7fa32cf67700 1 -- 172.21.15.45:0/583181689 wait complete. 2023-08-27T11:27:53.634 INFO:tasks.ceph.mds.b.smithi135.stderr: -4888> 2023-08-27T11:27:30.369+0000 7f10058dc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:27:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:27:53.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:27:53.666 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2023-08-27T11:27:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.771+0000 7f007e0e7700 1 Processor -- start 2023-08-27T11:27:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.771+0000 7f007e0e7700 1 -- start start 2023-08-27T11:27:53.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.772+0000 7f007e0e7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f007815d4f0 0x7f00780ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.772+0000 7f007e0e7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aefb0 0x7f00780ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.772+0000 7f007e0e7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007806b010 con 0x7f00780aea60 2023-08-27T11:27:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.772+0000 7f007e0e7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007815d8d0 con 0x7f007815d4f0 2023-08-27T11:27:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.772+0000 7f007e0e7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780ad9f0 con 0x7f00780aefb0 2023-08-27T11:27:53.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.772+0000 7f00777fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f007815d4f0 0x7f00780ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.772+0000 7f0077fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f00780aea60 0x7f00780cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45462/0 (socket says 172.21.15.45:45462) 2023-08-27T11:27:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.772+0000 7f0077fff700 1 -- 172.21.15.45:0/2420093946 learned_addr learned my addr 172.21.15.45:0/2420093946 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.772+0000 7f006effd700 1 --2- 172.21.15.45:0/2420093946 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aefb0 0x7f00780ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f00777fe700 1 -- 172.21.15.45:0/2420093946 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aefb0 msgr2=0x7f00780ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f00777fe700 1 --2- 172.21.15.45:0/2420093946 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aefb0 0x7f00780ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f00777fe700 1 -- 172.21.15.45:0/2420093946 >> v1:172.21.15.45:6789/0 conn(0x7f00780aea60 legacy=0x7f00780cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f00777fe700 1 -- 172.21.15.45:0/2420093946 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007806ad30 con 0x7f007815d4f0 2023-08-27T11:27:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f0076ffd700 1 -- 172.21.15.45:0/2420093946 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1490326074 0 0) 0x7f007806b010 con 0x7f00780aea60 2023-08-27T11:27:53.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f00777fe700 1 --2- 172.21.15.45:0/2420093946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f007815d4f0 0x7f00780ae490 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f0068009840 tx=0x7f0068009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45a48c7b55adc89 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f0076ffd700 1 -- 172.21.15.45:0/2420093946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011830 con 0x7f007815d4f0 2023-08-27T11:27:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f0076ffd700 1 -- 172.21.15.45:0/2420093946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0068011990 con 0x7f007815d4f0 2023-08-27T11:27:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f0076ffd700 1 -- 172.21.15.45:0/2420093946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011d20 con 0x7f007815d4f0 2023-08-27T11:27:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.773+0000 7f007e0e7700 1 -- 172.21.15.45:0/2420093946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f007815d4f0 msgr2=0x7f00780ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.774+0000 7f007e0e7700 1 --2- 172.21.15.45:0/2420093946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f007815d4f0 0x7f00780ae490 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f0068009840 tx=0x7f0068009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:53.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.774+0000 7f007e0e7700 1 -- 172.21.15.45:0/2420093946 shutdown_connections 2023-08-27T11:27:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.774+0000 7f007e0e7700 1 --2- 172.21.15.45:0/2420093946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f007815d4f0 0x7f00780ae490 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.774+0000 7f007e0e7700 1 --2- 172.21.15.45:0/2420093946 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aefb0 0x7f00780ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.774+0000 7f007e0e7700 1 -- 172.21.15.45:0/2420093946 >> 172.21.15.45:0/2420093946 conn(0x7f00780cc7e0 msgr2=0x7f00780ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.774+0000 7f007e0e7700 1 -- 172.21.15.45:0/2420093946 shutdown_connections 2023-08-27T11:27:53.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.774+0000 7f007e0e7700 1 -- 172.21.15.45:0/2420093946 wait complete. 2023-08-27T11:27:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.775+0000 7f007e0e7700 1 Processor -- start 2023-08-27T11:27:53.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.775+0000 7f007e0e7700 1 -- start start 2023-08-27T11:27:53.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f007e0e7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aea60 0x7f007815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f007e0e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00780aefb0 0x7f00780b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f007e0e7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007815d4f0 0x7f00780b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f007e0e7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007815b760 con 0x7f00780aefb0 2023-08-27T11:27:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f007e0e7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007815f050 con 0x7f007815d4f0 2023-08-27T11:27:53.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f007e0e7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c98f0 con 0x7f00780aea60 2023-08-27T11:27:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f006effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00780aefb0 0x7f00780b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f006effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00780aefb0 0x7f00780b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58690/0 (socket says 172.21.15.45:58690) 2023-08-27T11:27:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f006effd700 1 -- 172.21.15.45:0/2770326344 learned_addr learned my addr 172.21.15.45:0/2770326344 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f00777fe700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aea60 0x7f007815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f0077fff700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007815d4f0 0x7f00780b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f006effd700 1 -- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aea60 msgr2=0x7f007815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f006effd700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aea60 0x7f007815b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f006effd700 1 -- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007815d4f0 msgr2=0x7f00780b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f006effd700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007815d4f0 0x7f00780b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f006effd700 1 -- 172.21.15.45:0/2770326344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680094a0 con 0x7f00780aefb0 2023-08-27T11:27:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f0077fff700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007815d4f0 0x7f00780b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.776+0000 7f006effd700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00780aefb0 0x7f00780b33b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f006000e870 tx=0x7f006000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.777+0000 7f0074ff9700 1 -- 172.21.15.45:0/2770326344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060009590 con 0x7f00780aefb0 2023-08-27T11:27:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.777+0000 7f0074ff9700 1 -- 172.21.15.45:0/2770326344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006000ee70 con 0x7f00780aefb0 2023-08-27T11:27:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.777+0000 7f0074ff9700 1 -- 172.21.15.45:0/2770326344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000ca50 con 0x7f00780aefb0 2023-08-27T11:27:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.777+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00780c9610 con 0x7f00780aefb0 2023-08-27T11:27:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.777+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007815ece0 con 0x7f00780aefb0 2023-08-27T11:27:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.778+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007815f1e0 con 0x7f00780aefb0 2023-08-27T11:27:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.778+0000 7f0074ff9700 1 -- 172.21.15.45:0/2770326344 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 36) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f006000cbb0 con 0x7f00780aefb0 2023-08-27T11:27:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.779+0000 7f0074ff9700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f0058069f30 0x7f005806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.779+0000 7f0074ff9700 1 -- 172.21.15.45:0/2770326344 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f00600885d0 con 0x7f00780aefb0 2023-08-27T11:27:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.779+0000 7f00777fe700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f0058069f30 0x7f005806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.779+0000 7f00777fe700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f0058069f30 0x7f005806c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f00680090b0 tx=0x7f0068008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:53.783+0000 7f0074ff9700 1 -- 172.21.15.45:0/2770326344 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0060057620 con 0x7f00780aefb0 2023-08-27T11:27:54.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.034+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f00780726d0 con 0x7f00780aefb0 2023-08-27T11:27:54.187 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:54.185+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:54.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.499+0000 7f0074ff9700 1 -- 172.21.15.45:0/2770326344 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f0060053f60 con 0x7f00780aefb0 2023-08-27T11:27:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.627+0000 7f0074ff9700 1 -- 172.21.15.45:0/2770326344 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0060057620 con 0x7f00780aefb0 2023-08-27T11:27:54.631 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f0058069f30 msgr2=0x7f005806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.632 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f0058069f30 0x7f005806c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f00680090b0 tx=0x7f0068008030 comp rx=0 tx=0).stop 2023-08-27T11:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00780aefb0 msgr2=0x7f00780b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00780aefb0 0x7f00780b33b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f006000e870 tx=0x7f006000ec30 comp rx=0 tx=0).stop 2023-08-27T11:27:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 shutdown_connections 2023-08-27T11:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f0058069f30 0x7f005806c3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007815d4f0 0x7f00780b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00780aea60 0x7f007815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 --2- 172.21.15.45:0/2770326344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00780aefb0 0x7f00780b33b0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.629+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 >> 172.21.15.45:0/2770326344 conn(0x7f00780cc7e0 msgr2=0x7f007806ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.630+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 shutdown_connections 2023-08-27T11:27:54.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.630+0000 7f007e0e7700 1 -- 172.21.15.45:0/2770326344 wait complete. 2023-08-27T11:27:54.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:27:54.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.772+0000 7fad0a6b7700 1 Processor -- start 2023-08-27T11:27:54.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.772+0000 7fad0a6b7700 1 -- start start 2023-08-27T11:27:54.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad0a6b7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b5220 0x7fad040cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad0a6b7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fad040b62c0 0x7fad040d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad0a6b7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad040cdf40 con 0x7fad04160890 2023-08-27T11:27:54.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad0a6b7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad040d0fd0 con 0x7fad040b62c0 2023-08-27T11:27:54.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad0a6b7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad040ce190 con 0x7fad040b5220 2023-08-27T11:27:54.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad08c54700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fad040b62c0 0x7fad040d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad03fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fad04160890 0x7fad040b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45472/0 (socket says 172.21.15.45:45472) 2023-08-27T11:27:54.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad03fff700 1 -- 172.21.15.45:0/4243294401 learned_addr learned my addr 172.21.15.45:0/4243294401 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:54.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad037fe700 1 --2- 172.21.15.45:0/4243294401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b5220 0x7fad040cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad08c54700 1 -- 172.21.15.45:0/4243294401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b5220 msgr2=0x7fad040cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:54.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad08c54700 1 --2- 172.21.15.45:0/4243294401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b5220 0x7fad040cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad08c54700 1 -- 172.21.15.45:0/4243294401 >> v1:172.21.15.45:6789/0 conn(0x7fad04160890 legacy=0x7fad040b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad08c54700 1 -- 172.21.15.45:0/4243294401 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad040ce470 con 0x7fad040b62c0 2023-08-27T11:27:54.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad02ffd700 1 -- 172.21.15.45:0/4243294401 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331461415 0 0) 0x7fad040cdf40 con 0x7fad04160890 2023-08-27T11:27:54.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad037fe700 1 --2- 172.21.15.45:0/4243294401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b5220 0x7fad040cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:54.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad08c54700 1 --2- 172.21.15.45:0/4243294401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b62c0 0x7fad040d1460 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7facfc009d90 tx=0x7facfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62d2a2fc5a89087 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:54.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad02ffd700 1 -- 172.21.15.45:0/4243294401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc00e070 con 0x7fad040b62c0 2023-08-27T11:27:54.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.773+0000 7fad02ffd700 1 -- 172.21.15.45:0/4243294401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc00bbd0 con 0x7fad040b62c0 2023-08-27T11:27:54.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.774+0000 7fad02ffd700 1 -- 172.21.15.45:0/4243294401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc012430 con 0x7fad040b62c0 2023-08-27T11:27:54.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.774+0000 7fad0a6b7700 1 -- 172.21.15.45:0/4243294401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b62c0 msgr2=0x7fad040d1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.774+0000 7fad0a6b7700 1 --2- 172.21.15.45:0/4243294401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b62c0 0x7fad040d1460 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7facfc009d90 tx=0x7facfc00b330 comp rx=0 tx=0).stop 2023-08-27T11:27:54.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.774+0000 7fad0a6b7700 1 -- 172.21.15.45:0/4243294401 shutdown_connections 2023-08-27T11:27:54.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.774+0000 7fad0a6b7700 1 --2- 172.21.15.45:0/4243294401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b62c0 0x7fad040d1460 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.774+0000 7fad0a6b7700 1 --2- 172.21.15.45:0/4243294401 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b5220 0x7fad040cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.774+0000 7fad0a6b7700 1 -- 172.21.15.45:0/4243294401 >> 172.21.15.45:0/4243294401 conn(0x7fad040cfb10 msgr2=0x7fad040cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.774+0000 7fad0a6b7700 1 -- 172.21.15.45:0/4243294401 shutdown_connections 2023-08-27T11:27:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.774+0000 7fad0a6b7700 1 -- 172.21.15.45:0/4243294401 wait complete. 2023-08-27T11:27:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.775+0000 7fad0a6b7700 1 Processor -- start 2023-08-27T11:27:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad0a6b7700 1 -- start start 2023-08-27T11:27:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad0a6b7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b5220 0x7fad04150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad0a6b7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b62c0 0x7fad040b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad0a6b7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad04160890 0x7fad040b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad0a6b7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0414dfb0 con 0x7fad040b5220 2023-08-27T11:27:54.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad0a6b7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad04150e60 con 0x7fad04160890 2023-08-27T11:27:54.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad0a6b7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad041f7080 con 0x7fad040b62c0 2023-08-27T11:27:54.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b5220 0x7fad04150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad03fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b5220 0x7fad04150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58702/0 (socket says 172.21.15.45:58702) 2023-08-27T11:27:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad03fff700 1 -- 172.21.15.45:0/3702963993 learned_addr learned my addr 172.21.15.45:0/3702963993 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.776+0000 7fad037fe700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b62c0 0x7fad040b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad08c54700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad04160890 0x7fad040b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad03fff700 1 -- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b62c0 msgr2=0x7fad040b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad03fff700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b62c0 0x7fad040b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad03fff700 1 -- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad04160890 msgr2=0x7fad040b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:54.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad03fff700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad04160890 0x7fad040b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:54.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad03fff700 1 -- 172.21.15.45:0/3702963993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc009a70 con 0x7fad040b5220 2023-08-27T11:27:54.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad03fff700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b5220 0x7fad04150920 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7facf400c550 tx=0x7facf400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:54.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad00ff9700 1 -- 172.21.15.45:0/3702963993 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4004c80 con 0x7fad040b5220 2023-08-27T11:27:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad00ff9700 1 -- 172.21.15.45:0/3702963993 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf4004e00 con 0x7fad040b5220 2023-08-27T11:27:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad00ff9700 1 -- 172.21.15.45:0/3702963993 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4007310 con 0x7fad040b5220 2023-08-27T11:27:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad040cb720 con 0x7fad040b5220 2023-08-27T11:27:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.777+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad040cbce0 con 0x7fad040b5220 2023-08-27T11:27:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.778+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad040b8230 con 0x7fad040b5220 2023-08-27T11:27:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.779+0000 7fad00ff9700 1 -- 172.21.15.45:0/3702963993 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7facf4007470 con 0x7fad040b5220 2023-08-27T11:27:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.779+0000 7fad00ff9700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7facf006a170 0x7facf006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.779+0000 7fad00ff9700 1 -- 172.21.15.45:0/3702963993 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7200+0+0 (secure 0 0 0) 0x7facf40885f0 con 0x7fad040b5220 2023-08-27T11:27:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.779+0000 7fad037fe700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7facf006a170 0x7facf006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.780+0000 7fad037fe700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7facf006a170 0x7facf006c630 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fad040cb400 tx=0x7facf8008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.783+0000 7fad00ff9700 1 -- 172.21.15.45:0/3702963993 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7facf40538a0 con 0x7fad040b5220 2023-08-27T11:27:54.844 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:54.842+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:54.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:54.992+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad0406c120 con 0x7fad040b5220 2023-08-27T11:27:55.500 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:55.498+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:55.630 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.629+0000 7fad00ff9700 1 -- 172.21.15.45:0/3702963993 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 181+0+0 (secure 0 0 0) 0x7facf4056ec0 con 0x7fad040b5220 2023-08-27T11:27:55.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7facf006a170 msgr2=0x7facf006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7facf006a170 0x7facf006c630 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fad040cb400 tx=0x7facf8008040 comp rx=0 tx=0).stop 2023-08-27T11:27:55.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b5220 msgr2=0x7fad04150920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b5220 0x7fad04150920 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7facf400c550 tx=0x7facf400c860 comp rx=0 tx=0).stop 2023-08-27T11:27:55.634 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 shutdown_connections 2023-08-27T11:27:55.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7facf006a170 0x7facf006c630 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad040b5220 0x7fad04150920 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad04160890 0x7fad040b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 --2- 172.21.15.45:0/3702963993 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fad040b62c0 0x7fad040b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 >> 172.21.15.45:0/3702963993 conn(0x7fad040cfb10 msgr2=0x7fad040d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 shutdown_connections 2023-08-27T11:27:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.632+0000 7fad0a6b7700 1 -- 172.21.15.45:0/3702963993 wait complete. 2023-08-27T11:27:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:27:55.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:27:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.781+0000 7f2591912700 1 Processor -- start 2023-08-27T11:27:55.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.781+0000 7f2591912700 1 -- start start 2023-08-27T11:27:55.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.782+0000 7f2591912700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f258c0ce4b0 0x7f258c167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.782+0000 7f2591912700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c15d6a0 con 0x7f258c0b8c20 2023-08-27T11:27:55.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.782+0000 7f2591912700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c0d2c90 con 0x7f258c15d2c0 2023-08-27T11:27:55.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.782+0000 7f2591912700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c0d1bf0 con 0x7f258c0ce4b0 2023-08-27T11:27:55.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.782+0000 7f258affd700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f258c15d2c0 0x7f258c0d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:41150/0 (socket says 172.21.15.45:41150) 2023-08-27T11:27:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.782+0000 7f258affd700 1 -- 172.21.15.45:0/1780950995 learned_addr learned my addr 172.21.15.45:0/1780950995 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.782+0000 7f258b7fe700 1 --2- 172.21.15.45:0/1780950995 >> v2:172.21.15.45:3300/0 conn(0x7f258c0ce4b0 0x7f258c167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.782+0000 7f2589ffb700 1 -- 172.21.15.45:0/1780950995 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340461947 0 0) 0x7f258c0d2c90 con 0x7f258c15d2c0 2023-08-27T11:27:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.782+0000 7f2589ffb700 1 -- 172.21.15.45:0/1780950995 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25740035b0 con 0x7f258c15d2c0 2023-08-27T11:27:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.783+0000 7f258b7fe700 1 -- 172.21.15.45:0/1780950995 >> v1:172.21.15.135:6789/0 conn(0x7f258c15d2c0 legacy=0x7f258c0d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.783+0000 7f258b7fe700 1 -- 172.21.15.45:0/1780950995 >> v1:172.21.15.45:6789/0 conn(0x7f258c0b8c20 legacy=0x7f258c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.783+0000 7f258b7fe700 1 -- 172.21.15.45:0/1780950995 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c0d1ed0 con 0x7f258c0ce4b0 2023-08-27T11:27:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.783+0000 7f258b7fe700 1 --2- 172.21.15.45:0/1780950995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c0ce4b0 0x7f258c167d80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2584009dc0 tx=0x7f258400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e10683e562cacb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.783+0000 7f2589ffb700 1 -- 172.21.15.45:0/1780950995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258400e070 con 0x7f258c0ce4b0 2023-08-27T11:27:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.783+0000 7f2589ffb700 1 -- 172.21.15.45:0/1780950995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258400bae0 con 0x7f258c0ce4b0 2023-08-27T11:27:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.783+0000 7f2589ffb700 1 -- 172.21.15.45:0/1780950995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258400be90 con 0x7f258c0ce4b0 2023-08-27T11:27:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.783+0000 7f2591912700 1 -- 172.21.15.45:0/1780950995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c0ce4b0 msgr2=0x7f258c167d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.783+0000 7f2591912700 1 --2- 172.21.15.45:0/1780950995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c0ce4b0 0x7f258c167d80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2584009dc0 tx=0x7f258400b330 comp rx=0 tx=0).stop 2023-08-27T11:27:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.784+0000 7f2591912700 1 -- 172.21.15.45:0/1780950995 shutdown_connections 2023-08-27T11:27:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.784+0000 7f2591912700 1 --2- 172.21.15.45:0/1780950995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c0ce4b0 0x7f258c167d80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.784+0000 7f2591912700 1 -- 172.21.15.45:0/1780950995 >> 172.21.15.45:0/1780950995 conn(0x7f258c0d2fd0 msgr2=0x7f258c0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.784+0000 7f2591912700 1 -- 172.21.15.45:0/1780950995 shutdown_connections 2023-08-27T11:27:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.784+0000 7f2591912700 1 -- 172.21.15.45:0/1780950995 wait complete. 2023-08-27T11:27:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.785+0000 7f2591912700 1 Processor -- start 2023-08-27T11:27:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.785+0000 7f2591912700 1 -- start start 2023-08-27T11:27:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f2591912700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f258c0b8c20 0x7f258c1f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f2591912700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f258c0ce4b0 0x7f258c15fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f2591912700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c15d2c0 0x7f258c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f2591912700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c15dde0 con 0x7f258c15d2c0 2023-08-27T11:27:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f2591912700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c15e0c0 con 0x7f258c0ce4b0 2023-08-27T11:27:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f2591912700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c15e3a0 con 0x7f258c0b8c20 2023-08-27T11:27:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c15d2c0 0x7f258c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c15d2c0 0x7f258c159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58720/0 (socket says 172.21.15.45:58720) 2023-08-27T11:27:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258b7fe700 1 -- 172.21.15.45:0/169396672 learned_addr learned my addr 172.21.15.45:0/169396672 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258a7fc700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f258c0ce4b0 0x7f258c15fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258affd700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f258c0b8c20 0x7f258c1f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258b7fe700 1 -- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f258c0b8c20 msgr2=0x7f258c1f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258b7fe700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f258c0b8c20 0x7f258c1f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258b7fe700 1 -- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f258c0ce4b0 msgr2=0x7f258c15fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258b7fe700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f258c0ce4b0 0x7f258c15fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258b7fe700 1 -- 172.21.15.45:0/169396672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2584009a70 con 0x7f258c15d2c0 2023-08-27T11:27:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258affd700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f258c0b8c20 0x7f258c1f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.786+0000 7f258b7fe700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c15d2c0 0x7f258c159390 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2584014040 tx=0x7f2584004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.787+0000 7f2573fff700 1 -- 172.21.15.45:0/169396672 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840045d0 con 0x7f258c15d2c0 2023-08-27T11:27:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.787+0000 7f2573fff700 1 -- 172.21.15.45:0/169396672 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584020070 con 0x7f258c15d2c0 2023-08-27T11:27:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.787+0000 7f2573fff700 1 -- 172.21.15.45:0/169396672 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584027480 con 0x7f258c15d2c0 2023-08-27T11:27:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.787+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c15e680 con 0x7f258c15d2c0 2023-08-27T11:27:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.787+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c1f25d0 con 0x7f258c15d2c0 2023-08-27T11:27:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.788+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c15eba0 con 0x7f258c15d2c0 2023-08-27T11:27:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.788+0000 7f2573fff700 1 -- 172.21.15.45:0/169396672 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f25840275e0 con 0x7f258c15d2c0 2023-08-27T11:27:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.788+0000 7f2573fff700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f257806a1a0 0x7f257806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.789+0000 7f2573fff700 1 -- 172.21.15.45:0/169396672 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6762+0+0 (secure 0 0 0) 0x7f2584022070 con 0x7f258c15d2c0 2023-08-27T11:27:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.789+0000 7f258affd700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f257806a1a0 0x7f257806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.789+0000 7f258affd700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f257806a1a0 0x7f257806c660 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25800011e0 tx=0x7f2580009040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:55.793+0000 7f2573fff700 1 -- 172.21.15.45:0/169396672 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f258c15eba0 con 0x7f258c15d2c0 2023-08-27T11:27:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.004+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f258c15eba0 con 0x7f258c15d2c0 2023-08-27T11:27:56.167 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:56.165+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:56.635 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.633+0000 7f2573fff700 1 -- 172.21.15.45:0/169396672 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7f258c15eba0 con 0x7f258c15d2c0 2023-08-27T11:27:56.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f257806a1a0 msgr2=0x7f257806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f257806a1a0 0x7f257806c660 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25800011e0 tx=0x7f2580009040 comp rx=0 tx=0).stop 2023-08-27T11:27:56.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c15d2c0 msgr2=0x7f258c159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c15d2c0 0x7f258c159390 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2584014040 tx=0x7f2584004ad0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 shutdown_connections 2023-08-27T11:27:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f257806a1a0 0x7f257806c660 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f258c15d2c0 0x7f258c159390 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f258c0b8c20 0x7f258c1f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 --2- 172.21.15.45:0/169396672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f258c0ce4b0 0x7f258c15fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 >> 172.21.15.45:0/169396672 conn(0x7f258c0d2fd0 msgr2=0x7f258c15c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:56.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 shutdown_connections 2023-08-27T11:27:56.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.636+0000 7f2591912700 1 -- 172.21.15.45:0/169396672 wait complete. 2023-08-27T11:27:56.641 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-08-27T11:27:56.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:27:56.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.780+0000 7f13f10ff700 1 Processor -- start 2023-08-27T11:27:56.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.780+0000 7f13f10ff700 1 -- start start 2023-08-27T11:27:56.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.781+0000 7f13f10ff700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f13ec06a280 0x7f13ec06a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.781+0000 7f13f10ff700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0b2290 con 0x7f13ec15b5b0 2023-08-27T11:27:56.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.781+0000 7f13f10ff700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec06aba0 con 0x7f13ec071400 2023-08-27T11:27:56.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.781+0000 7f13f10ff700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec071fa0 con 0x7f13ec06a280 2023-08-27T11:27:56.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.781+0000 7f13eb59e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f13ec15b5b0 0x7f13ec15ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45500/0 (socket says 172.21.15.45:45500) 2023-08-27T11:27:56.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.781+0000 7f13eb59e700 1 -- 172.21.15.45:0/1576120859 learned_addr learned my addr 172.21.15.45:0/1576120859 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:56.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.781+0000 7f13ead9d700 1 --2- 172.21.15.45:0/1576120859 >> v2:172.21.15.135:3300/0 conn(0x7f13ec06a280 0x7f13ec06a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.781+0000 7f13e9d9b700 1 -- 172.21.15.45:0/1576120859 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968164762 0 0) 0x7f13ec0b2290 con 0x7f13ec15b5b0 2023-08-27T11:27:56.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.781+0000 7f13e9d9b700 1 -- 172.21.15.45:0/1576120859 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d40035b0 con 0x7f13ec15b5b0 2023-08-27T11:27:56.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13e9d9b700 1 -- 172.21.15.45:0/1576120859 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365385246 0 0) 0x7f13ec06aba0 con 0x7f13ec071400 2023-08-27T11:27:56.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13e9d9b700 1 -- 172.21.15.45:0/1576120859 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13ec0b2290 con 0x7f13ec071400 2023-08-27T11:27:56.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13ead9d700 1 -- 172.21.15.45:0/1576120859 >> v1:172.21.15.135:6789/0 conn(0x7f13ec071400 legacy=0x7f13ec071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13ead9d700 1 -- 172.21.15.45:0/1576120859 >> v1:172.21.15.45:6789/0 conn(0x7f13ec15b5b0 legacy=0x7f13ec15ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13e9d9b700 1 -- 172.21.15.45:0/1576120859 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2468860108 0 0) 0x7f13d40035b0 con 0x7f13ec15b5b0 2023-08-27T11:27:56.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13ead9d700 1 -- 172.21.15.45:0/1576120859 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0b1960 con 0x7f13ec06a280 2023-08-27T11:27:56.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13ead9d700 1 --2- 172.21.15.45:0/1576120859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 0x7f13ec06a660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009840 tx=0x7f13e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2e80e40166236ea server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13e9d9b700 1 -- 172.21.15.45:0/1576120859 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40019f0 con 0x7f13ec06a280 2023-08-27T11:27:56.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13e9d9b700 1 -- 172.21.15.45:0/1576120859 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e0004030 con 0x7f13ec06a280 2023-08-27T11:27:56.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13e9d9b700 1 -- 172.21.15.45:0/1576120859 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e00043c0 con 0x7f13ec06a280 2023-08-27T11:27:56.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13f10ff700 1 -- 172.21.15.45:0/1576120859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 msgr2=0x7f13ec06a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13f10ff700 1 --2- 172.21.15.45:0/1576120859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 0x7f13ec06a660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009840 tx=0x7f13e0009b50 comp rx=0 tx=0).stop 2023-08-27T11:27:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13f10ff700 1 -- 172.21.15.45:0/1576120859 shutdown_connections 2023-08-27T11:27:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13f10ff700 1 --2- 172.21.15.45:0/1576120859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 0x7f13ec06a660 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.782+0000 7f13f10ff700 1 -- 172.21.15.45:0/1576120859 >> 172.21.15.45:0/1576120859 conn(0x7f13ec0cc7c0 msgr2=0x7f13ec0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.783+0000 7f13f10ff700 1 -- 172.21.15.45:0/1576120859 shutdown_connections 2023-08-27T11:27:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.783+0000 7f13f10ff700 1 -- 172.21.15.45:0/1576120859 wait complete. 2023-08-27T11:27:56.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.784+0000 7f13f10ff700 1 Processor -- start 2023-08-27T11:27:56.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.784+0000 7f13f10ff700 1 -- start start 2023-08-27T11:27:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13f10ff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 0x7f13ec0c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13f10ff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13ec071400 0x7f13ec1491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13f10ff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13ec15b5b0 0x7f13ec0ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13f10ff700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0c6d80 con 0x7f13ec15b5b0 2023-08-27T11:27:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13f10ff700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0c7060 con 0x7f13ec06a280 2023-08-27T11:27:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13f10ff700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0c7340 con 0x7f13ec071400 2023-08-27T11:27:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13eb59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13ec15b5b0 0x7f13ec0ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13eb59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13ec15b5b0 0x7f13ec0ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58730/0 (socket says 172.21.15.45:58730) 2023-08-27T11:27:56.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13eb59e700 1 -- 172.21.15.45:0/1461409160 learned_addr learned my addr 172.21.15.45:0/1461409160 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13ead9d700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 0x7f13ec0c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13ea59c700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13ec071400 0x7f13ec1491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13eb59e700 1 -- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13ec071400 msgr2=0x7f13ec1491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13eb59e700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13ec071400 0x7f13ec1491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13eb59e700 1 -- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 msgr2=0x7f13ec0c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:56.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13eb59e700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 0x7f13ec0c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13eb59e700 1 -- 172.21.15.45:0/1461409160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e00094a0 con 0x7f13ec15b5b0 2023-08-27T11:27:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.785+0000 7f13ead9d700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 0x7f13ec0c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.786+0000 7f13eb59e700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13ec15b5b0 0x7f13ec0ca5c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f13e400cc50 tx=0x7f13e400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.786+0000 7f13d37fe700 1 -- 172.21.15.45:0/1461409160 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e400ade0 con 0x7f13ec15b5b0 2023-08-27T11:27:56.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.786+0000 7f13d37fe700 1 -- 172.21.15.45:0/1461409160 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e400ce50 con 0x7f13ec15b5b0 2023-08-27T11:27:56.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.786+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec0c7620 con 0x7f13ec15b5b0 2023-08-27T11:27:56.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.786+0000 7f13d37fe700 1 -- 172.21.15.45:0/1461409160 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4007310 con 0x7f13ec15b5b0 2023-08-27T11:27:56.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.786+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec149700 con 0x7f13ec15b5b0 2023-08-27T11:27:56.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.787+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec0c7b40 con 0x7f13ec15b5b0 2023-08-27T11:27:56.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.787+0000 7f13d37fe700 1 -- 172.21.15.45:0/1461409160 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f13e4007470 con 0x7f13ec15b5b0 2023-08-27T11:27:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.788+0000 7f13d37fe700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f13d806a170 0x7f13d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.788+0000 7f13d37fe700 1 -- 172.21.15.45:0/1461409160 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f13e4088560 con 0x7f13ec15b5b0 2023-08-27T11:27:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.788+0000 7f13ead9d700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f13d806a170 0x7f13d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:56.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.788+0000 7f13ead9d700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f13d806a170 0x7f13d806c630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f13e00090b0 tx=0x7f13e0013040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:56.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.792+0000 7f13d37fe700 1 -- 172.21.15.45:0/1461409160 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f13e4053bd0 con 0x7f13ec15b5b0 2023-08-27T11:27:56.840 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:56.838+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:56.997+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f13ec072690 con 0x7f13ec15b5b0 2023-08-27T11:27:57.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.000+0000 7f13d37fe700 1 -- 172.21.15.45:0/1461409160 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f13e40571f0 con 0x7f13ec15b5b0 2023-08-27T11:27:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f13d806a170 msgr2=0x7f13d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f13d806a170 0x7f13d806c630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f13e00090b0 tx=0x7f13e0013040 comp rx=0 tx=0).stop 2023-08-27T11:27:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13ec15b5b0 msgr2=0x7f13ec0ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13ec15b5b0 0x7f13ec0ca5c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f13e400cc50 tx=0x7f13e400a340 comp rx=0 tx=0).stop 2023-08-27T11:27:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 shutdown_connections 2023-08-27T11:27:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f13d806a170 0x7f13d806c630 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13ec15b5b0 0x7f13ec0ca5c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13ec06a280 0x7f13ec0c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 --2- 172.21.15.45:0/1461409160 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13ec071400 0x7f13ec1491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.002+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 >> 172.21.15.45:0/1461409160 conn(0x7f13ec0cc7c0 msgr2=0x7f13ec06ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.003+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 shutdown_connections 2023-08-27T11:27:57.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.003+0000 7f13f10ff700 1 -- 172.21.15.45:0/1461409160 wait complete. 2023-08-27T11:27:57.017 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-27T11:27:57.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.145+0000 7f021f59e700 1 Processor -- start 2023-08-27T11:27:57.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.145+0000 7f021f59e700 1 -- start start 2023-08-27T11:27:57.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021f59e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f022015a010 0x7f022015e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021f59e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0220071cd0 0x7f0220070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021f59e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02200ce9e0 con 0x7f0220071440 2023-08-27T11:27:57.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021f59e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02200caa10 con 0x7f022015a010 2023-08-27T11:27:57.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021f59e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0220071850 con 0x7f0220071cd0 2023-08-27T11:27:57.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021e59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f022015a010 0x7f022015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021ed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0220071440 0x7f022015a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45514/0 (socket says 172.21.15.45:45514) 2023-08-27T11:27:57.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021ed9d700 1 -- 172.21.15.45:0/3065145669 learned_addr learned my addr 172.21.15.45:0/3065145669 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:57.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021dd9b700 1 --2- 172.21.15.45:0/3065145669 >> v2:172.21.15.135:3300/0 conn(0x7f0220071cd0 0x7f0220070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021e59c700 1 -- 172.21.15.45:0/3065145669 >> v2:172.21.15.135:3300/0 conn(0x7f0220071cd0 msgr2=0x7f0220070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:57.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021e59c700 1 --2- 172.21.15.45:0/3065145669 >> v2:172.21.15.135:3300/0 conn(0x7f0220071cd0 0x7f0220070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021e59c700 1 -- 172.21.15.45:0/3065145669 >> v1:172.21.15.45:6789/0 conn(0x7f0220071440 legacy=0x7f022015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021d59a700 1 -- 172.21.15.45:0/3065145669 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620828982 0 0) 0x7f02200ce9e0 con 0x7f0220071440 2023-08-27T11:27:57.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.146+0000 7f021e59c700 1 -- 172.21.15.45:0/3065145669 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022015e8c0 con 0x7f022015a010 2023-08-27T11:27:57.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021e59c700 1 --2- 172.21.15.45:0/3065145669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022015a010 0x7f022015e380 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f02200c6e60 tx=0x7f0210009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123ed79cee7ffec0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021d59a700 1 -- 172.21.15.45:0/3065145669 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210011830 con 0x7f022015a010 2023-08-27T11:27:57.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021d59a700 1 -- 172.21.15.45:0/3065145669 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0210011990 con 0x7f022015a010 2023-08-27T11:27:57.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021d59a700 1 -- 172.21.15.45:0/3065145669 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210011d20 con 0x7f022015a010 2023-08-27T11:27:57.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021f59e700 1 -- 172.21.15.45:0/3065145669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022015a010 msgr2=0x7f022015e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021f59e700 1 --2- 172.21.15.45:0/3065145669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022015a010 0x7f022015e380 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f02200c6e60 tx=0x7f0210009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:57.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021f59e700 1 -- 172.21.15.45:0/3065145669 shutdown_connections 2023-08-27T11:27:57.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021f59e700 1 --2- 172.21.15.45:0/3065145669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f022015a010 0x7f022015e380 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021f59e700 1 --2- 172.21.15.45:0/3065145669 >> v2:172.21.15.135:3300/0 conn(0x7f0220071cd0 0x7f0220070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.147+0000 7f021f59e700 1 -- 172.21.15.45:0/3065145669 >> 172.21.15.45:0/3065145669 conn(0x7f02200cfe00 msgr2=0x7f02200d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.148+0000 7f021f59e700 1 -- 172.21.15.45:0/3065145669 shutdown_connections 2023-08-27T11:27:57.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.148+0000 7f021f59e700 1 -- 172.21.15.45:0/3065145669 wait complete. 2023-08-27T11:27:57.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.149+0000 7f021f59e700 1 Processor -- start 2023-08-27T11:27:57.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.149+0000 7f021f59e700 1 -- start start 2023-08-27T11:27:57.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.150+0000 7f021f59e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220071440 0x7f02200b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.150+0000 7f021f59e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0220071cd0 0x7f02200c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.150+0000 7f021f59e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f022015a010 0x7f02200c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.150+0000 7f021f59e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200c9870 con 0x7f0220071440 2023-08-27T11:27:57.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.150+0000 7f021f59e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200c8290 con 0x7f0220071cd0 2023-08-27T11:27:57.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.150+0000 7f021f59e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02200c8570 con 0x7f022015a010 2023-08-27T11:27:57.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.150+0000 7f021e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220071440 0x7f02200b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.150+0000 7f021e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220071440 0x7f02200b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58742/0 (socket says 172.21.15.45:58742) 2023-08-27T11:27:57.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.150+0000 7f021e59c700 1 -- 172.21.15.45:0/570852130 learned_addr learned my addr 172.21.15.45:0/570852130 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:57.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021dd9b700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0220071cd0 0x7f02200c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021ed9d700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f022015a010 0x7f02200c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021e59c700 1 -- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f022015a010 msgr2=0x7f02200c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021e59c700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f022015a010 0x7f02200c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021e59c700 1 -- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0220071cd0 msgr2=0x7f02200c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021e59c700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0220071cd0 0x7f02200c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021e59c700 1 -- 172.21.15.45:0/570852130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02100094a0 con 0x7f0220071440 2023-08-27T11:27:57.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021e59c700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220071440 0x7f02200b1eb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0210014040 tx=0x7f0210004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f020effd700 1 -- 172.21.15.45:0/570852130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02100036a0 con 0x7f0220071440 2023-08-27T11:27:57.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f020effd700 1 -- 172.21.15.45:0/570852130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021001e070 con 0x7f0220071440 2023-08-27T11:27:57.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02200c8850 con 0x7f0220071440 2023-08-27T11:27:57.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f020effd700 1 -- 172.21.15.45:0/570852130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210003a60 con 0x7f0220071440 2023-08-27T11:27:57.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.151+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02200c8e10 con 0x7f0220071440 2023-08-27T11:27:57.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.152+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0220072780 con 0x7f0220071440 2023-08-27T11:27:57.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.153+0000 7f020effd700 1 -- 172.21.15.45:0/570852130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f0210003bc0 con 0x7f0220071440 2023-08-27T11:27:57.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.153+0000 7f020effd700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f020006a170 0x7f020006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.153+0000 7f020effd700 1 -- 172.21.15.45:0/570852130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f0210089480 con 0x7f0220071440 2023-08-27T11:27:57.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.153+0000 7f021dd9b700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f020006a170 0x7f020006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.154+0000 7f021dd9b700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f020006a170 0x7f020006c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0214009be0 tx=0x7f0214009210 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.157+0000 7f020effd700 1 -- 172.21.15.45:0/570852130 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0210054af0 con 0x7f0220071440 2023-08-27T11:27:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.358+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f02200cc310 con 0x7f0220071440 2023-08-27T11:27:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.362+0000 7f020effd700 1 -- 172.21.15.45:0/570852130 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v26) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0210058110 con 0x7f0220071440 2023-08-27T11:27:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f020006a170 msgr2=0x7f020006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f020006a170 0x7f020006c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0214009be0 tx=0x7f0214009210 comp rx=0 tx=0).stop 2023-08-27T11:27:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220071440 msgr2=0x7f02200b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220071440 0x7f02200b1eb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0210014040 tx=0x7f0210004750 comp rx=0 tx=0).stop 2023-08-27T11:27:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 shutdown_connections 2023-08-27T11:27:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f020006a170 0x7f020006c630 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0220071440 0x7f02200b1eb0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f022015a010 0x7f02200c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 --2- 172.21.15.45:0/570852130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0220071cd0 0x7f02200c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 >> 172.21.15.45:0/570852130 conn(0x7f02200cfe00 msgr2=0x7f022015b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 shutdown_connections 2023-08-27T11:27:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.364+0000 7f021f59e700 1 -- 172.21.15.45:0/570852130 wait complete. 2023-08-27T11:27:57.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-27T11:27:57.502 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:57.501+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:57.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.506+0000 7f73203db700 1 Processor -- start 2023-08-27T11:27:57.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.506+0000 7f73203db700 1 -- start start 2023-08-27T11:27:57.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f73203db700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73180cdf10 0x7f731806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f73203db700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73180d04f0 con 0x7f731806c260 2023-08-27T11:27:57.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f73203db700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73180b06a0 con 0x7f73180b02c0 2023-08-27T11:27:57.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f73203db700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73180cf8e0 con 0x7f73180cdf10 2023-08-27T11:27:57.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f731e978700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73180cdf10 0x7f731806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f731d976700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f731806c260 0x7f73180cd800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45526/0 (socket says 172.21.15.45:45526) 2023-08-27T11:27:57.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f731d976700 1 -- 172.21.15.45:0/614159612 learned_addr learned my addr 172.21.15.45:0/614159612 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:57.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f731e978700 1 -- 172.21.15.45:0/614159612 >> v1:172.21.15.135:6789/0 conn(0x7f73180b02c0 legacy=0x7f73180cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f731e978700 1 -- 172.21.15.45:0/614159612 >> v1:172.21.15.45:6789/0 conn(0x7f731806c260 legacy=0x7f73180cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f731d175700 1 -- 172.21.15.45:0/614159612 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666094107 0 0) 0x7f73180b06a0 con 0x7f73180b02c0 2023-08-27T11:27:57.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.507+0000 7f731e978700 1 -- 172.21.15.45:0/614159612 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180cfbc0 con 0x7f73180cdf10 2023-08-27T11:27:57.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f731e978700 1 --2- 172.21.15.45:0/614159612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73180cdf10 0x7f731806dd20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7310009d90 tx=0x7f731000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5944906663b7934b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f731d175700 1 -- 172.21.15.45:0/614159612 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731000e070 con 0x7f73180cdf10 2023-08-27T11:27:57.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f731d175700 1 -- 172.21.15.45:0/614159612 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731000bad0 con 0x7f73180cdf10 2023-08-27T11:27:57.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f731d175700 1 -- 172.21.15.45:0/614159612 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731000be80 con 0x7f73180cdf10 2023-08-27T11:27:57.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f73203db700 1 -- 172.21.15.45:0/614159612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73180cdf10 msgr2=0x7f731806dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f73203db700 1 --2- 172.21.15.45:0/614159612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73180cdf10 0x7f731806dd20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7310009d90 tx=0x7f731000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:57.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f73203db700 1 -- 172.21.15.45:0/614159612 shutdown_connections 2023-08-27T11:27:57.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f73203db700 1 --2- 172.21.15.45:0/614159612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73180cdf10 0x7f731806dd20 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f73203db700 1 -- 172.21.15.45:0/614159612 >> 172.21.15.45:0/614159612 conn(0x7f731809a1a0 msgr2=0x7f73180d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f73203db700 1 -- 172.21.15.45:0/614159612 shutdown_connections 2023-08-27T11:27:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.508+0000 7f73203db700 1 -- 172.21.15.45:0/614159612 wait complete. 2023-08-27T11:27:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f73203db700 1 Processor -- start 2023-08-27T11:27:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f73203db700 1 -- start start 2023-08-27T11:27:57.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f73203db700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f731806c260 0x7f73180b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f73203db700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73180b02c0 0x7f73181f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f73203db700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73180cdf10 0x7f731806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f73203db700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73180b5c10 con 0x7f731806c260 2023-08-27T11:27:57.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f73203db700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73180b5ef0 con 0x7f73180cdf10 2023-08-27T11:27:57.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f73203db700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73180b61d0 con 0x7f73180b02c0 2023-08-27T11:27:57.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f731e177700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f731806c260 0x7f73180b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f731e177700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f731806c260 0x7f73180b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58754/0 (socket says 172.21.15.45:58754) 2023-08-27T11:27:57.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.510+0000 7f731e177700 1 -- 172.21.15.45:0/4280420138 learned_addr learned my addr 172.21.15.45:0/4280420138 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:57.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f731d976700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73180b02c0 0x7f73181f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f731e978700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73180cdf10 0x7f731806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f731e177700 1 -- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73180b02c0 msgr2=0x7f73181f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f731e177700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73180b02c0 0x7f73181f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f731e177700 1 -- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73180cdf10 msgr2=0x7f731806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f731e177700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73180cdf10 0x7f731806ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f731e177700 1 -- 172.21.15.45:0/4280420138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7310009a70 con 0x7f731806c260 2023-08-27T11:27:57.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f731e978700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73180cdf10 0x7f731806ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:57.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f731e177700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f731806c260 0x7f73180b4360 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f730800c550 tx=0x7f730800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f730effd700 1 -- 172.21.15.45:0/4280420138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080054e0 con 0x7f731806c260 2023-08-27T11:27:57.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f730effd700 1 -- 172.21.15.45:0/4280420138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7308003680 con 0x7f731806c260 2023-08-27T11:27:57.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73180b64b0 con 0x7f731806c260 2023-08-27T11:27:57.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f730effd700 1 -- 172.21.15.45:0/4280420138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308003a30 con 0x7f731806c260 2023-08-27T11:27:57.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.511+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73180b48a0 con 0x7f731806c260 2023-08-27T11:27:57.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.512+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731809eda0 con 0x7f731806c260 2023-08-27T11:27:57.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.513+0000 7f730effd700 1 -- 172.21.15.45:0/4280420138 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f7308003b90 con 0x7f731806c260 2023-08-27T11:27:57.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.513+0000 7f730effd700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f72fc06a170 0x7f72fc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.513+0000 7f730effd700 1 -- 172.21.15.45:0/4280420138 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f7308087b40 con 0x7f731806c260 2023-08-27T11:27:57.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.513+0000 7f731d976700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f72fc06a170 0x7f72fc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.514+0000 7f731d976700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f72fc06a170 0x7f72fc06c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7314007c20 tx=0x7f7314007d70 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.518+0000 7f730effd700 1 -- 172.21.15.45:0/4280420138 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7308053120 con 0x7f731806c260 2023-08-27T11:27:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.725+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f73180b5d50 con 0x7f731806c260 2023-08-27T11:27:57.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.728+0000 7f730effd700 1 -- 172.21.15.45:0/4280420138 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v27) v1 ==== 115+0+0 (secure 0 0 0) 0x7f73180b5d50 con 0x7f731806c260 2023-08-27T11:27:57.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f72fc06a170 msgr2=0x7f72fc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f72fc06a170 0x7f72fc06c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7314007c20 tx=0x7f7314007d70 comp rx=0 tx=0).stop 2023-08-27T11:27:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f731806c260 msgr2=0x7f73180b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f731806c260 0x7f73180b4360 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f730800c550 tx=0x7f730800c910 comp rx=0 tx=0).stop 2023-08-27T11:27:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 shutdown_connections 2023-08-27T11:27:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f72fc06a170 0x7f72fc06c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f731806c260 0x7f73180b4360 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73180cdf10 0x7f731806ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 --2- 172.21.15.45:0/4280420138 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f73180b02c0 0x7f73181f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 >> 172.21.15.45:0/4280420138 conn(0x7f731809a1a0 msgr2=0x7f73180ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 shutdown_connections 2023-08-27T11:27:57.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.730+0000 7f73203db700 1 -- 172.21.15.45:0/4280420138 wait complete. 2023-08-27T11:27:57.744 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-08-27T11:27:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.869+0000 7fa9f1c9c700 1 Processor -- start 2023-08-27T11:27:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.869+0000 7fa9f1c9c700 1 -- start start 2023-08-27T11:27:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9f1c9c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9ec15d450 0x7fa9ec0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9f1c9c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa9ec0cbaf0 0x7fa9ec0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9f1c9c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ec0ce1c0 con 0x7fa9ec0b2140 2023-08-27T11:27:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9f1c9c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ec0cf290 con 0x7fa9ec15d450 2023-08-27T11:27:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9f1c9c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ec0ce440 con 0x7fa9ec0cbaf0 2023-08-27T11:27:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9eb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9ec15d450 0x7fa9ec0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9ebfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa9ec0b2140 0x7fa9ec06b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45538/0 (socket says 172.21.15.45:45538) 2023-08-27T11:27:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9eb7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9ec15d450 0x7fa9ec0cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58762/0 (socket says 172.21.15.45:58762) 2023-08-27T11:27:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9ebfff700 1 -- 172.21.15.45:0/3843406022 learned_addr learned my addr 172.21.15.45:0/3843406022 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9eaffd700 1 --2- 172.21.15.45:0/3843406022 >> v2:172.21.15.135:3300/0 conn(0x7fa9ec0cbaf0 0x7fa9ec0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9eb7fe700 1 -- 172.21.15.45:0/3843406022 >> v2:172.21.15.135:3300/0 conn(0x7fa9ec0cbaf0 msgr2=0x7fa9ec0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9eb7fe700 1 --2- 172.21.15.45:0/3843406022 >> v2:172.21.15.135:3300/0 conn(0x7fa9ec0cbaf0 0x7fa9ec0cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9eb7fe700 1 -- 172.21.15.45:0/3843406022 >> v1:172.21.15.45:6789/0 conn(0x7fa9ec0b2140 legacy=0x7fa9ec06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9eb7fe700 1 -- 172.21.15.45:0/3843406022 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec0ce720 con 0x7fa9ec15d450 2023-08-27T11:27:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9ea7fc700 1 -- 172.21.15.45:0/3843406022 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982453690 0 0) 0x7fa9ec0ce1c0 con 0x7fa9ec0b2140 2023-08-27T11:27:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.870+0000 7fa9eaffd700 1 --2- 172.21.15.45:0/3843406022 >> v2:172.21.15.135:3300/0 conn(0x7fa9ec0cbaf0 0x7fa9ec0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:27:57.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9eb7fe700 1 --2- 172.21.15.45:0/3843406022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 0x7fa9ec0cb5b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0b2930 tx=0x7fa9e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fef137592116485 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9ea7fc700 1 -- 172.21.15.45:0/3843406022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0011830 con 0x7fa9ec15d450 2023-08-27T11:27:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9ea7fc700 1 -- 172.21.15.45:0/3843406022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e0011990 con 0x7fa9ec15d450 2023-08-27T11:27:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9ea7fc700 1 -- 172.21.15.45:0/3843406022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0011d20 con 0x7fa9ec15d450 2023-08-27T11:27:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/3843406022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 msgr2=0x7fa9ec0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9f1c9c700 1 --2- 172.21.15.45:0/3843406022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 0x7fa9ec0cb5b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0b2930 tx=0x7fa9e0009b20 comp rx=0 tx=0).stop 2023-08-27T11:27:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/3843406022 shutdown_connections 2023-08-27T11:27:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9f1c9c700 1 --2- 172.21.15.45:0/3843406022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 0x7fa9ec0cb5b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9f1c9c700 1 --2- 172.21.15.45:0/3843406022 >> v2:172.21.15.135:3300/0 conn(0x7fa9ec0cbaf0 0x7fa9ec0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/3843406022 >> 172.21.15.45:0/3843406022 conn(0x7fa9ec0d0f60 msgr2=0x7fa9ec0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/3843406022 shutdown_connections 2023-08-27T11:27:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.871+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/3843406022 wait complete. 2023-08-27T11:27:57.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.873+0000 7fa9f1c9c700 1 Processor -- start 2023-08-27T11:27:57.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.873+0000 7fa9f1c9c700 1 -- start start 2023-08-27T11:27:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9f1c9c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9ec0b2140 0x7fa9ec151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9f1c9c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9ec0cbaf0 0x7fa9ec0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9f1c9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 0x7fa9ec0d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9f1c9c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec1553d0 con 0x7fa9ec15d450 2023-08-27T11:27:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9f1c9c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec1556b0 con 0x7fa9ec0b2140 2023-08-27T11:27:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9f1c9c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec155990 con 0x7fa9ec0cbaf0 2023-08-27T11:27:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9ebfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 0x7fa9ec0d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9ebfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 0x7fa9ec0d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58772/0 (socket says 172.21.15.45:58772) 2023-08-27T11:27:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9ebfff700 1 -- 172.21.15.45:0/1148470791 learned_addr learned my addr 172.21.15.45:0/1148470791 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9eb7fe700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9ec0b2140 0x7fa9ec151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9eaffd700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9ec0cbaf0 0x7fa9ec0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9ebfff700 1 -- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9ec0cbaf0 msgr2=0x7fa9ec0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9ebfff700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9ec0cbaf0 0x7fa9ec0cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9ebfff700 1 -- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9ec0b2140 msgr2=0x7fa9ec151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9ebfff700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9ec0b2140 0x7fa9ec151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9ebfff700 1 -- 172.21.15.45:0/1148470791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00094a0 con 0x7fa9ec15d450 2023-08-27T11:27:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9ebfff700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 0x7fa9ec0d0270 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400cca0 tx=0x7fa9e400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9d3fff700 1 -- 172.21.15.45:0/1148470791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400ab70 con 0x7fa9ec15d450 2023-08-27T11:27:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9d3fff700 1 -- 172.21.15.45:0/1148470791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e400acd0 con 0x7fa9ec15d450 2023-08-27T11:27:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ec155c70 con 0x7fa9ec15d450 2023-08-27T11:27:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.874+0000 7fa9d3fff700 1 -- 172.21.15.45:0/1148470791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4007310 con 0x7fa9ec15d450 2023-08-27T11:27:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.875+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec15e1a0 con 0x7fa9ec15d450 2023-08-27T11:27:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.875+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ec155510 con 0x7fa9ec15d450 2023-08-27T11:27:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.876+0000 7fa9d3fff700 1 -- 172.21.15.45:0/1148470791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fa9e4007470 con 0x7fa9ec15d450 2023-08-27T11:27:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.876+0000 7fa9d3fff700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa9d806a170 0x7fa9d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.876+0000 7fa9d3fff700 1 -- 172.21.15.45:0/1148470791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fa9e4088420 con 0x7fa9ec15d450 2023-08-27T11:27:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.877+0000 7fa9eb7fe700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa9d806a170 0x7fa9d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.877+0000 7fa9eb7fe700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa9d806a170 0x7fa9d806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00090b0 tx=0x7fa9e001c040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:57.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:57.881+0000 7fa9d3fff700 1 -- 172.21.15.45:0/1148470791 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa9ec155510 con 0x7fa9ec15d450 2023-08-27T11:27:58.170 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:58.168+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:58.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.177+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fa9ec155510 con 0x7fa9ec15d450 2023-08-27T11:27:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.635+0000 7fa9d3fff700 1 -- 172.21.15.45:0/1148470791 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v598) v1 ==== 156+0+0 (secure 0 0 0) 0x7fa9ec155510 con 0x7fa9ec15d450 2023-08-27T11:27:58.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.637+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa9d806a170 msgr2=0x7fa9d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.637+0000 7fa9f1c9c700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa9d806a170 0x7fa9d806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00090b0 tx=0x7fa9e001c040 comp rx=0 tx=0).stop 2023-08-27T11:27:58.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.637+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 msgr2=0x7fa9ec0d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.640 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.637+0000 7fa9f1c9c700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 0x7fa9ec0d0270 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400cca0 tx=0x7fa9e400a060 comp rx=0 tx=0).stop 2023-08-27T11:27:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.637+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 shutdown_connections 2023-08-27T11:27:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.637+0000 7fa9f1c9c700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa9d806a170 0x7fa9d806c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.637+0000 7fa9f1c9c700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9ec15d450 0x7fa9ec0d0270 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.637+0000 7fa9f1c9c700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9ec0b2140 0x7fa9ec151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.637+0000 7fa9f1c9c700 1 --2- 172.21.15.45:0/1148470791 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9ec0cbaf0 0x7fa9ec0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.638+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 >> 172.21.15.45:0/1148470791 conn(0x7fa9ec0d0f60 msgr2=0x7fa9ec0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.638+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 shutdown_connections 2023-08-27T11:27:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.638+0000 7fa9f1c9c700 1 -- 172.21.15.45:0/1148470791 wait complete. 2023-08-27T11:27:58.652 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-08-27T11:27:58.652 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-08-27T11:27:58.653 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-27T11:27:58.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-08-27T11:27:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.777+0000 7feef8f1e700 1 Processor -- start 2023-08-27T11:27:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.777+0000 7feef8f1e700 1 -- start start 2023-08-27T11:27:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef8f1e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef415d2d0 0x7feef40cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef8f1e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feef40d0910 0x7feef406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef8f1e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7feef406e920 0x7feef406a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef8f1e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef40cfb90 con 0x7feef40d0910 2023-08-27T11:27:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef8f1e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef40cf5c0 con 0x7feef406e920 2023-08-27T11:27:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef8f1e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef4161880 con 0x7feef415d2d0 2023-08-27T11:27:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef1d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feef40d0910 0x7feef406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef1d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feef40d0910 0x7feef406a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58776/0 (socket says 172.21.15.45:58776) 2023-08-27T11:27:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef1d9b700 1 -- 172.21.15.45:0/3563833326 learned_addr learned my addr 172.21.15.45:0/3563833326 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef2d9d700 1 --2- 172.21.15.45:0/3563833326 >> v2:172.21.15.135:3300/0 conn(0x7feef406e920 0x7feef406a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.778+0000 7feef259c700 1 --2- 172.21.15.45:0/3563833326 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef415d2d0 0x7feef40cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef1d9b700 1 -- 172.21.15.45:0/3563833326 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef415d2d0 msgr2=0x7feef40cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef1d9b700 1 --2- 172.21.15.45:0/3563833326 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef415d2d0 0x7feef40cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef1d9b700 1 -- 172.21.15.45:0/3563833326 >> v2:172.21.15.135:3300/0 conn(0x7feef406e920 msgr2=0x7feef406a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef1d9b700 1 --2- 172.21.15.45:0/3563833326 >> v2:172.21.15.135:3300/0 conn(0x7feef406e920 0x7feef406a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef1d9b700 1 -- 172.21.15.45:0/3563833326 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef4161db0 con 0x7feef40d0910 2023-08-27T11:27:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef259c700 1 --2- 172.21.15.45:0/3563833326 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef415d2d0 0x7feef40cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:27:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef1d9b700 1 --2- 172.21.15.45:0/3563833326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 0x7feef406a280 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7feef406ae40 tx=0x7feee4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf28a96f30782c11 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef159a700 1 -- 172.21.15.45:0/3563833326 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4011830 con 0x7feef40d0910 2023-08-27T11:27:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef159a700 1 -- 172.21.15.45:0/3563833326 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4011990 con 0x7feef40d0910 2023-08-27T11:27:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef159a700 1 -- 172.21.15.45:0/3563833326 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4011d20 con 0x7feef40d0910 2023-08-27T11:27:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef8f1e700 1 -- 172.21.15.45:0/3563833326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 msgr2=0x7feef406a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.779+0000 7feef8f1e700 1 --2- 172.21.15.45:0/3563833326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 0x7feef406a280 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7feef406ae40 tx=0x7feee4009bd0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.780+0000 7feef8f1e700 1 -- 172.21.15.45:0/3563833326 shutdown_connections 2023-08-27T11:27:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.780+0000 7feef8f1e700 1 --2- 172.21.15.45:0/3563833326 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef415d2d0 0x7feef40cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.780+0000 7feef8f1e700 1 --2- 172.21.15.45:0/3563833326 >> v2:172.21.15.135:3300/0 conn(0x7feef406e920 0x7feef406a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.780+0000 7feef8f1e700 1 --2- 172.21.15.45:0/3563833326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 0x7feef406a280 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.780+0000 7feef8f1e700 1 -- 172.21.15.45:0/3563833326 >> 172.21.15.45:0/3563833326 conn(0x7feef40cfdf0 msgr2=0x7feef40d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:58.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.780+0000 7feef8f1e700 1 -- 172.21.15.45:0/3563833326 shutdown_connections 2023-08-27T11:27:58.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.780+0000 7feef8f1e700 1 -- 172.21.15.45:0/3563833326 wait complete. 2023-08-27T11:27:58.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.781+0000 7feef8f1e700 1 Processor -- start 2023-08-27T11:27:58.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.781+0000 7feef8f1e700 1 -- start start 2023-08-27T11:27:58.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.782+0000 7feef8f1e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef406e920 0x7feef4158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.782+0000 7feef8f1e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 0x7feef41595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.782+0000 7feef8f1e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef415d2d0 0x7feef40b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.782+0000 7feef8f1e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef40affd0 con 0x7feef40d0910 2023-08-27T11:27:58.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.782+0000 7feef8f1e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef4159af0 con 0x7feef415d2d0 2023-08-27T11:27:58.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.782+0000 7feef8f1e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef41592d0 con 0x7feef406e920 2023-08-27T11:27:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 0x7feef41595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef259c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef406e920 0x7feef4158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef259c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef406e920 0x7feef4158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:56692/0 (socket says 172.21.15.45:56692) 2023-08-27T11:27:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 0x7feef41595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58784/0 (socket says 172.21.15.45:58784) 2023-08-27T11:27:58.792 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef259c700 1 -- 172.21.15.45:0/638309070 learned_addr learned my addr 172.21.15.45:0/638309070 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef2d9d700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef415d2d0 0x7feef40b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef1d9b700 1 -- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef406e920 msgr2=0x7feef4158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef1d9b700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef406e920 0x7feef4158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef1d9b700 1 -- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef415d2d0 msgr2=0x7feef40b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef1d9b700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef415d2d0 0x7feef40b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef1d9b700 1 -- 172.21.15.45:0/638309070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4009550 con 0x7feef40d0910 2023-08-27T11:27:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef2d9d700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef415d2d0 0x7feef40b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef1d9b700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 0x7feef41595b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7feee4016000 tx=0x7feee4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feee2ffd700 1 -- 172.21.15.45:0/638309070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4004510 con 0x7feef40d0910 2023-08-27T11:27:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feee2ffd700 1 -- 172.21.15.45:0/638309070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee401e070 con 0x7feef40d0910 2023-08-27T11:27:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef40af980 con 0x7feef40d0910 2023-08-27T11:27:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feee2ffd700 1 -- 172.21.15.45:0/638309070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40038e0 con 0x7feef40d0910 2023-08-27T11:27:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.783+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef41ea950 con 0x7feef40d0910 2023-08-27T11:27:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.784+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef41eab40 con 0x7feef40d0910 2023-08-27T11:27:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.785+0000 7feee2ffd700 1 -- 172.21.15.45:0/638309070 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7feee4003a40 con 0x7feef40d0910 2023-08-27T11:27:58.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.785+0000 7feee2ffd700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7feed406a170 0x7feed406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:58.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.785+0000 7feee2ffd700 1 -- 172.21.15.45:0/638309070 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7feee4089340 con 0x7feef40d0910 2023-08-27T11:27:58.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.785+0000 7feef259c700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7feed406a170 0x7feed406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:58.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.786+0000 7feef259c700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7feed406a170 0x7feed406c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feee800ca00 tx=0x7feee8008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:58.790+0000 7feee2ffd700 1 -- 172.21.15.45:0/638309070 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feee40549b0 con 0x7feef40d0910 2023-08-27T11:27:58.844 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:58.842+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.089+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7feef415cb00 con 0x7feef40d0910 2023-08-27T11:27:59.517 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:27:59.515+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:27:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.639+0000 7feee2ffd700 1 -- 172.21.15.45:0/638309070 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v599) v1 ==== 169+0+0 (secure 0 0 0) 0x7feee4057fd0 con 0x7feef40d0910 2023-08-27T11:27:59.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7feed406a170 msgr2=0x7feed406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7feed406a170 0x7feed406c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feee800ca00 tx=0x7feee8008040 comp rx=0 tx=0).stop 2023-08-27T11:27:59.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 msgr2=0x7feef41595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 0x7feef41595b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7feee4016000 tx=0x7feee4004750 comp rx=0 tx=0).stop 2023-08-27T11:27:59.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 shutdown_connections 2023-08-27T11:27:59.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7feed406a170 0x7feed406c630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef415d2d0 0x7feef40b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feef406e920 0x7feef4158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 --2- 172.21.15.45:0/638309070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef40d0910 0x7feef41595b0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 >> 172.21.15.45:0/638309070 conn(0x7feef40cfdf0 msgr2=0x7feef40d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 shutdown_connections 2023-08-27T11:27:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.641+0000 7feef8f1e700 1 -- 172.21.15.45:0/638309070 wait complete. 2023-08-27T11:27:59.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:27:59.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.781+0000 7f292e397700 1 Processor -- start 2023-08-27T11:27:59.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.781+0000 7f292e397700 1 -- start start 2023-08-27T11:27:59.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292e397700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f292815b120 0x7f29280cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292e397700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 0x7f29280b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292e397700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29280ada50 con 0x7f29280d1a20 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292e397700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29280ccd60 con 0x7f292815b120 2023-08-27T11:27:59.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292e397700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29280b0860 con 0x7f29280cdb80 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f2927fff700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f292815b120 0x7f29280cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292c934700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 0x7f29280b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f29277fe700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f29280d1a20 0x7f29280cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:41230/0 (socket says 172.21.15.45:41230) 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f29277fe700 1 -- 172.21.15.45:0/3398267165 learned_addr learned my addr 172.21.15.45:0/3398267165 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292c934700 1 -- 172.21.15.45:0/3398267165 >> v2:172.21.15.135:3300/0 conn(0x7f292815b120 msgr2=0x7f29280cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:27:59.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292c934700 1 --2- 172.21.15.45:0/3398267165 >> v2:172.21.15.135:3300/0 conn(0x7f292815b120 0x7f29280cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292c934700 1 -- 172.21.15.45:0/3398267165 >> v1:172.21.15.135:6789/0 conn(0x7f29280d1a20 legacy=0x7f29280cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f292c934700 1 -- 172.21.15.45:0/3398267165 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29280b0b40 con 0x7f29280cdb80 2023-08-27T11:27:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f2927fff700 1 --2- 172.21.15.45:0/3398267165 >> v2:172.21.15.135:3300/0 conn(0x7f292815b120 0x7f29280cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.782+0000 7f2926ffd700 1 -- 172.21.15.45:0/3398267165 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755751917 0 0) 0x7f29280ada50 con 0x7f29280d1a20 2023-08-27T11:27:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f292c934700 1 --2- 172.21.15.45:0/3398267165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 0x7f29280b1620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2920009d90 tx=0x7f292000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d4543860794cf10 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f2926ffd700 1 -- 172.21.15.45:0/3398267165 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292000e070 con 0x7f29280cdb80 2023-08-27T11:27:59.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f2926ffd700 1 -- 172.21.15.45:0/3398267165 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292000bb20 con 0x7f29280cdb80 2023-08-27T11:27:59.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f2926ffd700 1 -- 172.21.15.45:0/3398267165 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920012430 con 0x7f29280cdb80 2023-08-27T11:27:59.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f292e397700 1 -- 172.21.15.45:0/3398267165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 msgr2=0x7f29280b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f292e397700 1 --2- 172.21.15.45:0/3398267165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 0x7f29280b1620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2920009d90 tx=0x7f292000b330 comp rx=0 tx=0).stop 2023-08-27T11:27:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f292e397700 1 -- 172.21.15.45:0/3398267165 shutdown_connections 2023-08-27T11:27:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f292e397700 1 --2- 172.21.15.45:0/3398267165 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 0x7f29280b1620 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f292e397700 1 --2- 172.21.15.45:0/3398267165 >> v2:172.21.15.135:3300/0 conn(0x7f292815b120 0x7f29280cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f292e397700 1 -- 172.21.15.45:0/3398267165 >> 172.21.15.45:0/3398267165 conn(0x7f292809a160 msgr2=0x7f29280d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:27:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f292e397700 1 -- 172.21.15.45:0/3398267165 shutdown_connections 2023-08-27T11:27:59.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.783+0000 7f292e397700 1 -- 172.21.15.45:0/3398267165 wait complete. 2023-08-27T11:27:59.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.785+0000 7f292e397700 1 Processor -- start 2023-08-27T11:27:59.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.785+0000 7f292e397700 1 -- start start 2023-08-27T11:27:59.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f292e397700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 0x7f292806df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f292e397700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29280d1a20 0x7f29280c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f292e397700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292815b120 0x7f29280c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f292e397700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292806e9c0 con 0x7f29280d1a20 2023-08-27T11:27:59.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f292e397700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292806eca0 con 0x7f292815b120 2023-08-27T11:27:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f292e397700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292806ef80 con 0x7f29280cdb80 2023-08-27T11:27:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f29277fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29280d1a20 0x7f29280c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f29277fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29280d1a20 0x7f29280c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58796/0 (socket says 172.21.15.45:58796) 2023-08-27T11:27:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f29277fe700 1 -- 172.21.15.45:0/1151379018 learned_addr learned my addr 172.21.15.45:0/1151379018 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:27:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f292c934700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292815b120 0x7f29280c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.786+0000 7f2927fff700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 0x7f292806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f29277fe700 1 -- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 msgr2=0x7f292806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f29277fe700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 0x7f292806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f29277fe700 1 -- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292815b120 msgr2=0x7f29280c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:27:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f29277fe700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292815b120 0x7f29280c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:27:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f29277fe700 1 -- 172.21.15.45:0/1151379018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180094a0 con 0x7f29280d1a20 2023-08-27T11:27:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f292c934700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292815b120 0x7f29280c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:27:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f29277fe700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29280d1a20 0x7f29280c9420 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f291c0011e0 tx=0x7f291c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f2924ff9700 1 -- 172.21.15.45:0/1151379018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c004c80 con 0x7f29280d1a20 2023-08-27T11:27:59.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f2924ff9700 1 -- 172.21.15.45:0/1151379018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c004e00 con 0x7f29280d1a20 2023-08-27T11:27:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f2924ff9700 1 -- 172.21.15.45:0/1151379018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c007310 con 0x7f29280d1a20 2023-08-27T11:27:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2920009a70 con 0x7f29280d1a20 2023-08-27T11:27:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.787+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292806f540 con 0x7f29280d1a20 2023-08-27T11:27:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.788+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292806f760 con 0x7f29280d1a20 2023-08-27T11:27:59.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.789+0000 7f2924ff9700 1 -- 172.21.15.45:0/1151379018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f291c007470 con 0x7f29280d1a20 2023-08-27T11:27:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.789+0000 7f2924ff9700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f291406a170 0x7f291406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:27:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.789+0000 7f2924ff9700 1 -- 172.21.15.45:0/1151379018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f291c088350 con 0x7f29280d1a20 2023-08-27T11:27:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.789+0000 7f2927fff700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f291406a170 0x7f291406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:27:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.790+0000 7f2927fff700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f291406a170 0x7f291406c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f292809eda0 tx=0x7f2918009210 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:27:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.794+0000 7f2924ff9700 1 -- 172.21.15.45:0/1151379018 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f291c053930 con 0x7f29280d1a20 2023-08-27T11:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.996+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f292809edd0 con 0x7f29280d1a20 2023-08-27T11:27:59.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.997+0000 7f2924ff9700 1 -- 172.21.15.45:0/1151379018 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16074 (secure 0 0 0) 0x7f291c056f50 con 0x7f29280d1a20 2023-08-27T11:27:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:27:59.999 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:27:56.628547+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:6835/1457694749":"2023-08-28T11:27:53.624785+0000","172.21.15.135:0/3810104053":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/3438903793":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/4068027882":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/1613956798":"2023-08-28T11:27:51.483615+0000","172.21.15.135:6833/294910128":"2023-08-28T11:27:51.483615+0000","172.21.15.135:6832/294910128":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/3221283236":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/3835899160":"2023-08-28T11:27:51.483615+0000","172.21.15.135:0/790246089":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2253320845":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6837/1457694749":"2023-08-28T11:27:53.624785+0000","172.21.15.135:0/2129685370":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/289717412":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6833/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:6832/3430028400":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/2380106040":"2023-08-28T11:27:44.271608+0000","172.21.15.135:0/4054422326":"2023-08-28T11:27:44.271608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:00.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f291406a170 msgr2=0x7f291406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f291406a170 0x7f291406c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f292809eda0 tx=0x7f2918009210 comp rx=0 tx=0).stop 2023-08-27T11:28:00.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29280d1a20 msgr2=0x7f29280c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29280d1a20 0x7f29280c9420 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f291c0011e0 tx=0x7f291c00ca30 comp rx=0 tx=0).stop 2023-08-27T11:28:00.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 shutdown_connections 2023-08-27T11:28:00.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f291406a170 0x7f291406c630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292815b120 0x7f29280c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f29280cdb80 0x7f292806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 --2- 172.21.15.45:0/1151379018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29280d1a20 0x7f29280c9420 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 >> 172.21.15.45:0/1151379018 conn(0x7f292809a160 msgr2=0x7f29280cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 shutdown_connections 2023-08-27T11:28:00.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:27:59.999+0000 7f292e397700 1 -- 172.21.15.45:0/1151379018 wait complete. 2023-08-27T11:28:00.013 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:00.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.141+0000 7f53e31f3700 1 Processor -- start 2023-08-27T11:28:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.141+0000 7f53e31f3700 1 -- start start 2023-08-27T11:28:00.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e31f3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f53dc06a280 0x7f53dc06a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e31f3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e31f3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53dc0b2290 con 0x7f53dc06ced0 2023-08-27T11:28:00.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e31f3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53dc15c5c0 con 0x7f53dc06a280 2023-08-27T11:28:00.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e31f3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc06aba0 con 0x7f53dc071400 2023-08-27T11:28:00.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e1790700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f53dc06ced0 0x7f53dc06d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45584/0 (socket says 172.21.15.45:45584) 2023-08-27T11:28:00.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e1790700 1 -- 172.21.15.45:0/98618412 learned_addr learned my addr 172.21.15.45:0/98618412 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:00.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e0f8f700 1 --2- 172.21.15.45:0/98618412 >> v2:172.21.15.135:3300/0 conn(0x7f53dc06a280 0x7f53dc06a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53dbfff700 1 --2- 172.21.15.45:0/98618412 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53db7fe700 1 -- 172.21.15.45:0/98618412 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344394920 0 0) 0x7f53dc0b2290 con 0x7f53dc06ced0 2023-08-27T11:28:00.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53db7fe700 1 -- 172.21.15.45:0/98618412 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53cc0035b0 con 0x7f53dc06ced0 2023-08-27T11:28:00.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e0f8f700 1 -- 172.21.15.45:0/98618412 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 msgr2=0x7f53dc071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:00.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e0f8f700 1 --2- 172.21.15.45:0/98618412 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.142+0000 7f53e0f8f700 1 -- 172.21.15.45:0/98618412 >> v1:172.21.15.45:6789/0 conn(0x7f53dc06ced0 legacy=0x7f53dc06d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.143+0000 7f53e0f8f700 1 -- 172.21.15.45:0/98618412 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc0b1960 con 0x7f53dc06a280 2023-08-27T11:28:00.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.143+0000 7f53dbfff700 1 --2- 172.21.15.45:0/98618412 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:00.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.143+0000 7f53e0f8f700 1 --2- 172.21.15.45:0/98618412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06a280 0x7f53dc06a660 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009840 tx=0x7f53d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7dde8a5147069d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.143+0000 7f53db7fe700 1 -- 172.21.15.45:0/98618412 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0011830 con 0x7f53dc06a280 2023-08-27T11:28:00.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.143+0000 7f53db7fe700 1 -- 172.21.15.45:0/98618412 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d0011990 con 0x7f53dc06a280 2023-08-27T11:28:00.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.143+0000 7f53db7fe700 1 -- 172.21.15.45:0/98618412 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0011d00 con 0x7f53dc06a280 2023-08-27T11:28:00.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.144+0000 7f53e31f3700 1 -- 172.21.15.45:0/98618412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06a280 msgr2=0x7f53dc06a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.144+0000 7f53e31f3700 1 --2- 172.21.15.45:0/98618412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06a280 0x7f53dc06a660 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009840 tx=0x7f53d0009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:00.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.144+0000 7f53e31f3700 1 -- 172.21.15.45:0/98618412 shutdown_connections 2023-08-27T11:28:00.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.144+0000 7f53e31f3700 1 --2- 172.21.15.45:0/98618412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06a280 0x7f53dc06a660 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.144+0000 7f53e31f3700 1 --2- 172.21.15.45:0/98618412 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.144+0000 7f53e31f3700 1 -- 172.21.15.45:0/98618412 >> 172.21.15.45:0/98618412 conn(0x7f53dc0cc7c0 msgr2=0x7f53dc0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.144+0000 7f53e31f3700 1 -- 172.21.15.45:0/98618412 shutdown_connections 2023-08-27T11:28:00.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.144+0000 7f53e31f3700 1 -- 172.21.15.45:0/98618412 wait complete. 2023-08-27T11:28:00.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.145+0000 7f53e31f3700 1 Processor -- start 2023-08-27T11:28:00.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.145+0000 7f53e31f3700 1 -- start start 2023-08-27T11:28:00.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e31f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53dc06a280 0x7f53dc1f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e31f3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06ced0 0x7f53dc0a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e31f3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e31f3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0c8950 con 0x7f53dc06a280 2023-08-27T11:28:00.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e31f3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0c8c30 con 0x7f53dc06ced0 2023-08-27T11:28:00.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e31f3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc0c8f10 con 0x7f53dc071400 2023-08-27T11:28:00.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e0f8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53dc06a280 0x7f53dc1f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e0f8f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53dc06a280 0x7f53dc1f0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58810/0 (socket says 172.21.15.45:58810) 2023-08-27T11:28:00.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e0f8f700 1 -- 172.21.15.45:0/323740352 learned_addr learned my addr 172.21.15.45:0/323740352 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:00.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53dbfff700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06ced0 0x7f53dc0a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e1790700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e0f8f700 1 -- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 msgr2=0x7f53dc0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e0f8f700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e0f8f700 1 -- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06ced0 msgr2=0x7f53dc0a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e0f8f700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06ced0 0x7f53dc0a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e0f8f700 1 -- 172.21.15.45:0/323740352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80094a0 con 0x7f53dc06a280 2023-08-27T11:28:00.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e1790700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:00.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53dbfff700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06ced0 0x7f53dc0a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:00.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53e0f8f700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53dc06a280 0x7f53dc1f0420 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f53d00090b0 tx=0x7f53d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.146+0000 7f53d97fa700 1 -- 172.21.15.45:0/323740352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00036a0 con 0x7f53dc06a280 2023-08-27T11:28:00.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.147+0000 7f53d97fa700 1 -- 172.21.15.45:0/323740352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d001e070 con 0x7f53dc06a280 2023-08-27T11:28:00.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.147+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d00094a0 con 0x7f53dc06a280 2023-08-27T11:28:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.147+0000 7f53d97fa700 1 -- 172.21.15.45:0/323740352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0003a40 con 0x7f53dc06a280 2023-08-27T11:28:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.147+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53dc0c94d0 con 0x7f53dc06a280 2023-08-27T11:28:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.148+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53dc0c96f0 con 0x7f53dc06a280 2023-08-27T11:28:00.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.148+0000 7f53d97fa700 1 -- 172.21.15.45:0/323740352 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f53d0003ba0 con 0x7f53dc06a280 2023-08-27T11:28:00.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.148+0000 7f53d97fa700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f53c006a170 0x7f53c006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.148+0000 7f53d97fa700 1 -- 172.21.15.45:0/323740352 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f53d00895e0 con 0x7f53dc06a280 2023-08-27T11:28:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.149+0000 7f53dbfff700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f53c006a170 0x7f53c006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.149+0000 7f53dbfff700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f53c006a170 0x7f53c006c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f53dc154310 tx=0x7f53c8008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.153+0000 7f53d97fa700 1 -- 172.21.15.45:0/323740352 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f53d0054c50 con 0x7f53dc06a280 2023-08-27T11:28:00.194 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:00.192+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.354+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53dc072690 con 0x7f53dc06a280 2023-08-27T11:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.354+0000 7f53d97fa700 1 -- 172.21.15.45:0/323740352 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53d0058270 con 0x7f53dc06a280 2023-08-27T11:28:00.356 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:28:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f53c006a170 msgr2=0x7f53c006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f53c006a170 0x7f53c006c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f53dc154310 tx=0x7f53c8008040 comp rx=0 tx=0).stop 2023-08-27T11:28:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53dc06a280 msgr2=0x7f53dc1f0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53dc06a280 0x7f53dc1f0420 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f53d00090b0 tx=0x7f53d0004750 comp rx=0 tx=0).stop 2023-08-27T11:28:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 shutdown_connections 2023-08-27T11:28:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f53c006a170 0x7f53c006c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53dc06a280 0x7f53dc1f0420 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f53dc071400 0x7f53dc0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 --2- 172.21.15.45:0/323740352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53dc06ced0 0x7f53dc0a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 >> 172.21.15.45:0/323740352 conn(0x7f53dc0cc7c0 msgr2=0x7f53dc1591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 shutdown_connections 2023-08-27T11:28:00.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.356+0000 7f53e31f3700 1 -- 172.21.15.45:0/323740352 wait complete. 2023-08-27T11:28:00.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.498+0000 7ffbd9290700 1 Processor -- start 2023-08-27T11:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.498+0000 7ffbd9290700 1 -- start start 2023-08-27T11:28:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd9290700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffbd415d2c0 0x7ffbd40d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd9290700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffbd40ce4b0 0x7ffbd40b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd9290700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd415d6a0 con 0x7ffbd40b8c20 2023-08-27T11:28:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd9290700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd40d2c90 con 0x7ffbd415d2c0 2023-08-27T11:28:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd9290700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd40d1bf0 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd2ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffbd415d2c0 0x7ffbd40d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd2ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffbd415d2c0 0x7ffbd40d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58818/0 (socket says 172.21.15.45:58818) 2023-08-27T11:28:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd2ffd700 1 -- 172.21.15.45:0/4020106232 learned_addr learned my addr 172.21.15.45:0/4020106232 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd37fe700 1 --2- 172.21.15.45:0/4020106232 >> v2:172.21.15.135:3300/0 conn(0x7ffbd40ce4b0 0x7ffbd40b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd2ffd700 1 -- 172.21.15.45:0/4020106232 >> v2:172.21.15.135:3300/0 conn(0x7ffbd40ce4b0 msgr2=0x7ffbd40b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd2ffd700 1 --2- 172.21.15.45:0/4020106232 >> v2:172.21.15.135:3300/0 conn(0x7ffbd40ce4b0 0x7ffbd40b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd2ffd700 1 -- 172.21.15.45:0/4020106232 >> v1:172.21.15.135:6789/0 conn(0x7ffbd40b8c20 legacy=0x7ffbd40cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd2ffd700 1 -- 172.21.15.45:0/4020106232 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40d1ed0 con 0x7ffbd415d2c0 2023-08-27T11:28:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd37fe700 1 --2- 172.21.15.45:0/4020106232 >> v2:172.21.15.135:3300/0 conn(0x7ffbd40ce4b0 0x7ffbd40b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.499+0000 7ffbd2ffd700 1 --2- 172.21.15.45:0/4020106232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd415d2c0 0x7ffbd40d13a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098f0 tx=0x7ffbc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6856ed6ca4d7baf server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.500+0000 7ffbd1ffb700 1 -- 172.21.15.45:0/4020106232 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc800b860 con 0x7ffbd415d2c0 2023-08-27T11:28:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.500+0000 7ffbd1ffb700 1 -- 172.21.15.45:0/4020106232 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc800b9c0 con 0x7ffbd415d2c0 2023-08-27T11:28:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.500+0000 7ffbd1ffb700 1 -- 172.21.15.45:0/4020106232 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc800bd50 con 0x7ffbd415d2c0 2023-08-27T11:28:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.500+0000 7ffbd9290700 1 -- 172.21.15.45:0/4020106232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd415d2c0 msgr2=0x7ffbd40d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.500+0000 7ffbd9290700 1 --2- 172.21.15.45:0/4020106232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd415d2c0 0x7ffbd40d13a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098f0 tx=0x7ffbc8009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.500+0000 7ffbd9290700 1 -- 172.21.15.45:0/4020106232 shutdown_connections 2023-08-27T11:28:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.500+0000 7ffbd9290700 1 --2- 172.21.15.45:0/4020106232 >> v2:172.21.15.135:3300/0 conn(0x7ffbd40ce4b0 0x7ffbd40b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.500+0000 7ffbd9290700 1 --2- 172.21.15.45:0/4020106232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd415d2c0 0x7ffbd40d13a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.501+0000 7ffbd9290700 1 -- 172.21.15.45:0/4020106232 >> 172.21.15.45:0/4020106232 conn(0x7ffbd40d2fd0 msgr2=0x7ffbd40d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.501+0000 7ffbd9290700 1 -- 172.21.15.45:0/4020106232 shutdown_connections 2023-08-27T11:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.501+0000 7ffbd9290700 1 -- 172.21.15.45:0/4020106232 wait complete. 2023-08-27T11:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.502+0000 7ffbd9290700 1 Processor -- start 2023-08-27T11:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.502+0000 7ffbd9290700 1 -- start start 2023-08-27T11:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd9290700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffbd40b8c20 0x7ffbd40cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd9290700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd40ce4b0 0x7ffbd40c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd9290700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd415d2c0 0x7ffbd40b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd9290700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40cc2b0 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd9290700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40b4790 con 0x7ffbd415d2c0 2023-08-27T11:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd9290700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40c8070 con 0x7ffbd40b8c20 2023-08-27T11:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd40ce4b0 0x7ffbd40c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd27fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd40ce4b0 0x7ffbd40c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58824/0 (socket says 172.21.15.45:58824) 2023-08-27T11:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd27fc700 1 -- 172.21.15.45:0/3869355884 learned_addr learned my addr 172.21.15.45:0/3869355884 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd37fe700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd415d2c0 0x7ffbd40b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd2ffd700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffbd40b8c20 0x7ffbd40cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd27fc700 1 -- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffbd40b8c20 msgr2=0x7ffbd40cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd27fc700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffbd40b8c20 0x7ffbd40cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd27fc700 1 -- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd415d2c0 msgr2=0x7ffbd40b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd27fc700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd415d2c0 0x7ffbd40b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd27fc700 1 -- 172.21.15.45:0/3869355884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8009550 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.503+0000 7ffbd27fc700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd40ce4b0 0x7ffbd40c7b30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400c510 tx=0x7ffbc400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.504+0000 7ffbbb7fe700 1 -- 172.21.15.45:0/3869355884 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40052d0 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.504+0000 7ffbbb7fe700 1 -- 172.21.15.45:0/3869355884 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc4005430 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.504+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd415fef0 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.504+0000 7ffbbb7fe700 1 -- 172.21.15.45:0/3869355884 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40038c0 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.504+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd40cc780 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.505+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd40b4940 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.505+0000 7ffbbb7fe700 1 -- 172.21.15.45:0/3869355884 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7ffbc4003a20 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.506+0000 7ffbbb7fe700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ffbc006a170 0x7ffbc006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.506+0000 7ffbbb7fe700 1 -- 172.21.15.45:0/3869355884 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7ffbc40878c0 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.506+0000 7ffbd2ffd700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ffbc006a170 0x7ffbc006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.506+0000 7ffbd2ffd700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ffbc006a170 0x7ffbc006c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8014040 tx=0x7ffbc8008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.510+0000 7ffbbb7fe700 1 -- 172.21.15.45:0/3869355884 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffbd40b4940 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.716+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffbd415d6a0 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.717+0000 7ffbbb7fe700 1 -- 172.21.15.45:0/3869355884 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v91) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffbd415d6a0 con 0x7ffbd40ce4b0 2023-08-27T11:28:00.719 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:28:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ffbc006a170 msgr2=0x7ffbc006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ffbc006a170 0x7ffbc006c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8014040 tx=0x7ffbc8008030 comp rx=0 tx=0).stop 2023-08-27T11:28:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd40ce4b0 msgr2=0x7ffbd40c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd40ce4b0 0x7ffbd40c7b30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400c510 tx=0x7ffbc400c820 comp rx=0 tx=0).stop 2023-08-27T11:28:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 shutdown_connections 2023-08-27T11:28:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ffbc006a170 0x7ffbc006c630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbd415d2c0 0x7ffbd40b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ffbd40b8c20 0x7ffbd40cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 --2- 172.21.15.45:0/3869355884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd40ce4b0 0x7ffbd40c7b30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 >> 172.21.15.45:0/3869355884 conn(0x7ffbd40d2fd0 msgr2=0x7ffbd40b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 shutdown_connections 2023-08-27T11:28:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.719+0000 7ffbd9290700 1 -- 172.21.15.45:0/3869355884 wait complete. 2023-08-27T11:28:00.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:28:00.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.860+0000 7f07d0809700 1 Processor -- start 2023-08-27T11:28:00.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.860+0000 7f07d0809700 1 -- start start 2023-08-27T11:28:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07d0809700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07cc06c260 0x7f07cc0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07d0809700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07cc0d04f0 con 0x7f07cc0b02c0 2023-08-27T11:28:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07d0809700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07cc0b06a0 con 0x7f07cc0cdd40 2023-08-27T11:28:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07d0809700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07cc0ce1d0 con 0x7f07cc06c260 2023-08-27T11:28:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07c9d9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07cc06c260 0x7f07cc0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07ca59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f07cc0b02c0 0x7f07cc0cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45606/0 (socket says 172.21.15.45:45606) 2023-08-27T11:28:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07ca59c700 1 -- 172.21.15.45:0/794862670 learned_addr learned my addr 172.21.15.45:0/794862670 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07c9d9b700 1 -- 172.21.15.45:0/794862670 >> v1:172.21.15.135:6789/0 conn(0x7f07cc0cdd40 legacy=0x7f07cc15ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07c9d9b700 1 -- 172.21.15.45:0/794862670 >> v1:172.21.15.45:6789/0 conn(0x7f07cc0b02c0 legacy=0x7f07cc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.861+0000 7f07c9d9b700 1 -- 172.21.15.45:0/794862670 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0cf8e0 con 0x7f07cc06c260 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07c9d9b700 1 --2- 172.21.15.45:0/794862670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc06c260 0x7f07cc0cd800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009840 tx=0x7f07bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68827dd6fc1f1898 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07c959a700 1 -- 172.21.15.45:0/794862670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc004230 con 0x7f07cc06c260 2023-08-27T11:28:00.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07c959a700 1 -- 172.21.15.45:0/794862670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc004390 con 0x7f07cc06c260 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07c959a700 1 -- 172.21.15.45:0/794862670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0036a0 con 0x7f07cc06c260 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07d0809700 1 -- 172.21.15.45:0/794862670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc06c260 msgr2=0x7f07cc0cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07d0809700 1 --2- 172.21.15.45:0/794862670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc06c260 0x7f07cc0cd800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009840 tx=0x7f07bc009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07d0809700 1 -- 172.21.15.45:0/794862670 shutdown_connections 2023-08-27T11:28:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07d0809700 1 --2- 172.21.15.45:0/794862670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc06c260 0x7f07cc0cd800 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07d0809700 1 -- 172.21.15.45:0/794862670 >> 172.21.15.45:0/794862670 conn(0x7f07cc09a1a0 msgr2=0x7f07cc0d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:00.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07d0809700 1 -- 172.21.15.45:0/794862670 shutdown_connections 2023-08-27T11:28:00.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.862+0000 7f07d0809700 1 -- 172.21.15.45:0/794862670 wait complete. 2023-08-27T11:28:00.870 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:00.865+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.864+0000 7f07d0809700 1 Processor -- start 2023-08-27T11:28:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.864+0000 7f07d0809700 1 -- start start 2023-08-27T11:28:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07d0809700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07cc06c260 0x7f07cc1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07d0809700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc0b02c0 0x7f07cc0b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07d0809700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f07cc0cdd40 0x7f07cc0b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07d0809700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0c56e0 con 0x7f07cc0b02c0 2023-08-27T11:28:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07d0809700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0c59c0 con 0x7f07cc06c260 2023-08-27T11:28:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07d0809700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0c5ca0 con 0x7f07cc0cdd40 2023-08-27T11:28:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07c9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc0b02c0 0x7f07cc0b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07c9d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc0b02c0 0x7f07cc0b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58838/0 (socket says 172.21.15.45:58838) 2023-08-27T11:28:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07c9d9b700 1 -- 172.21.15.45:0/1238920079 learned_addr learned my addr 172.21.15.45:0/1238920079 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07ca59c700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07cc06c260 0x7f07cc1553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07cad9d700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f07cc0cdd40 0x7f07cc0b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07c9d9b700 1 -- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f07cc0cdd40 msgr2=0x7f07cc0b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07c9d9b700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f07cc0cdd40 0x7f07cc0b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07c9d9b700 1 -- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07cc06c260 msgr2=0x7f07cc1553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:00.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07c9d9b700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07cc06c260 0x7f07cc1553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:00.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07c9d9b700 1 -- 172.21.15.45:0/1238920079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc0094a0 con 0x7f07cc0b02c0 2023-08-27T11:28:00.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07c9d9b700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc0b02c0 0x7f07cc0b4a40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0090b0 tx=0x7f07bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07baffd700 1 -- 172.21.15.45:0/1238920079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc003ea0 con 0x7f07cc0b02c0 2023-08-27T11:28:00.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07baffd700 1 -- 172.21.15.45:0/1238920079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc011630 con 0x7f07cc0b02c0 2023-08-27T11:28:00.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07baffd700 1 -- 172.21.15.45:0/1238920079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0119e0 con 0x7f07cc0b02c0 2023-08-27T11:28:00.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc0c5f80 con 0x7f07cc0b02c0 2023-08-27T11:28:00.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.865+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc155910 con 0x7f07cc0b02c0 2023-08-27T11:28:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.866+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc09eda0 con 0x7f07cc0b02c0 2023-08-27T11:28:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.867+0000 7f07baffd700 1 -- 172.21.15.45:0/1238920079 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f07bc011b40 con 0x7f07cc0b02c0 2023-08-27T11:28:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.867+0000 7f07baffd700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f07ac06a170 0x7f07ac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:00.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.867+0000 7f07baffd700 1 -- 172.21.15.45:0/1238920079 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f07bc089e50 con 0x7f07cc0b02c0 2023-08-27T11:28:00.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.867+0000 7f07ca59c700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f07ac06a170 0x7f07ac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:00.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.868+0000 7f07ca59c700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f07ac06a170 0x7f07ac06c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07c0000f60 tx=0x7f07c0007f10 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:00.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:00.872+0000 7f07baffd700 1 -- 172.21.15.45:0/1238920079 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f07bc0554c0 con 0x7f07cc0b02c0 2023-08-27T11:28:01.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.080+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f07cc0cd260 con 0x7f07cc0b02c0 2023-08-27T11:28:01.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.083+0000 7f07baffd700 1 -- 172.21.15.45:0/1238920079 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f07bc058ae0 con 0x7f07cc0b02c0 2023-08-27T11:28:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f07ac06a170 msgr2=0x7f07ac06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f07ac06a170 0x7f07ac06c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07c0000f60 tx=0x7f07c0007f10 comp rx=0 tx=0).stop 2023-08-27T11:28:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc0b02c0 msgr2=0x7f07cc0b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc0b02c0 0x7f07cc0b4a40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0090b0 tx=0x7f07bc0047c0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 shutdown_connections 2023-08-27T11:28:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f07ac06a170 0x7f07ac06c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f07cc0cdd40 0x7f07cc0b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07cc06c260 0x7f07cc1553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 --2- 172.21.15.45:0/1238920079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07cc0b02c0 0x7f07cc0b4a40 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 >> 172.21.15.45:0/1238920079 conn(0x7f07cc09a1a0 msgr2=0x7f07cc06b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.085+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 shutdown_connections 2023-08-27T11:28:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.086+0000 7f07d0809700 1 -- 172.21.15.45:0/1238920079 wait complete. 2023-08-27T11:28:01.100 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:28:01.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:01.142 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:01.181 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:28:01.181 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:28:01.182 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:01.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:01.199 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:28:01.200 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:01.238 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:01.238 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:28:01.239 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:01.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:01.258 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:28:01.258 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:01.295 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:01.295 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:28:01.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:01.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.424+0000 7fa64b5f6700 1 Processor -- start 2023-08-27T11:28:01.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.424+0000 7fa64b5f6700 1 -- start start 2023-08-27T11:28:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa64b5f6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa64415a0f0 0x7fa6440cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa64b5f6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6440cd300 0x7fa64415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa64b5f6700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6440cfae0 con 0x7fa64415e030 2023-08-27T11:28:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa64b5f6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64415e900 con 0x7fa64415a0f0 2023-08-27T11:28:01.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa64b5f6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440cae60 con 0x7fa6440cd300 2023-08-27T11:28:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa649392700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa64415a0f0 0x7fa6440cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa649392700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa64415a0f0 0x7fa6440cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58844/0 (socket says 172.21.15.45:58844) 2023-08-27T11:28:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa649392700 1 -- 172.21.15.45:0/1166254129 learned_addr learned my addr 172.21.15.45:0/1166254129 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa648b91700 1 --2- 172.21.15.45:0/1166254129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6440cd300 0x7fa64415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa649392700 1 -- 172.21.15.45:0/1166254129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6440cd300 msgr2=0x7fa64415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:01.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa649392700 1 --2- 172.21.15.45:0/1166254129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6440cd300 0x7fa64415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa649392700 1 -- 172.21.15.45:0/1166254129 >> v1:172.21.15.135:6789/0 conn(0x7fa64415e030 legacy=0x7fa6440cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa63bfff700 1 -- 172.21.15.45:0/1166254129 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563267842 0 0) 0x7fa6440cfae0 con 0x7fa64415e030 2023-08-27T11:28:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.425+0000 7fa649392700 1 -- 172.21.15.45:0/1166254129 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440cb140 con 0x7fa64415a0f0 2023-08-27T11:28:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa649392700 1 --2- 172.21.15.45:0/1166254129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 0x7fa6440cbc20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098f0 tx=0x7fa640009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529c7bbdf48f5a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa63bfff700 1 -- 172.21.15.45:0/1166254129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640011830 con 0x7fa64415a0f0 2023-08-27T11:28:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa63bfff700 1 -- 172.21.15.45:0/1166254129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640011990 con 0x7fa64415a0f0 2023-08-27T11:28:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa63bfff700 1 -- 172.21.15.45:0/1166254129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640011d20 con 0x7fa64415a0f0 2023-08-27T11:28:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa64b5f6700 1 -- 172.21.15.45:0/1166254129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 msgr2=0x7fa6440cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa64b5f6700 1 --2- 172.21.15.45:0/1166254129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 0x7fa6440cbc20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098f0 tx=0x7fa640009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:01.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa64b5f6700 1 -- 172.21.15.45:0/1166254129 shutdown_connections 2023-08-27T11:28:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa64b5f6700 1 --2- 172.21.15.45:0/1166254129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 0x7fa6440cbc20 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa64b5f6700 1 --2- 172.21.15.45:0/1166254129 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa6440cd300 0x7fa64415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa64b5f6700 1 -- 172.21.15.45:0/1166254129 >> 172.21.15.45:0/1166254129 conn(0x7fa6440cc7e0 msgr2=0x7fa6440ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.426+0000 7fa64b5f6700 1 -- 172.21.15.45:0/1166254129 shutdown_connections 2023-08-27T11:28:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.427+0000 7fa64b5f6700 1 -- 172.21.15.45:0/1166254129 wait complete. 2023-08-27T11:28:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.428+0000 7fa64b5f6700 1 Processor -- start 2023-08-27T11:28:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.428+0000 7fa64b5f6700 1 -- start start 2023-08-27T11:28:01.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.428+0000 7fa64b5f6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6440cd300 0x7fa64406de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.428+0000 7fa64b5f6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 0x7fa6440b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.428+0000 7fa64b5f6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa64415e030 0x7fa64415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.428+0000 7fa64b5f6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b46b0 con 0x7fa64415a0f0 2023-08-27T11:28:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.428+0000 7fa64b5f6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b4990 con 0x7fa6440cd300 2023-08-27T11:28:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.428+0000 7fa64b5f6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b4c70 con 0x7fa64415e030 2023-08-27T11:28:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa648b91700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 0x7fa6440b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa648b91700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 0x7fa6440b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58850/0 (socket says 172.21.15.45:58850) 2023-08-27T11:28:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa648b91700 1 -- 172.21.15.45:0/271240584 learned_addr learned my addr 172.21.15.45:0/271240584 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa649392700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6440cd300 0x7fa64406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa649b93700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa64415e030 0x7fa64415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa648b91700 1 -- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa64415e030 msgr2=0x7fa64415a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa648b91700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa64415e030 0x7fa64415a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa648b91700 1 -- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6440cd300 msgr2=0x7fa64406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa648b91700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6440cd300 0x7fa64406de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa648b91700 1 -- 172.21.15.45:0/271240584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa640009550 con 0x7fa64415a0f0 2023-08-27T11:28:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa649b93700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa64415e030 0x7fa64415a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa649392700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6440cd300 0x7fa64406de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa648b91700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 0x7fa6440b3c40 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e870 tx=0x7fa63400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa639ffb700 1 -- 172.21.15.45:0/271240584 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634009590 con 0x7fa64415a0f0 2023-08-27T11:28:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa639ffb700 1 -- 172.21.15.45:0/271240584 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63400ee70 con 0x7fa64415a0f0 2023-08-27T11:28:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6440b4f50 con 0x7fa64415a0f0 2023-08-27T11:28:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa639ffb700 1 -- 172.21.15.45:0/271240584 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400ca50 con 0x7fa64415a0f0 2023-08-27T11:28:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.429+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64406e3a0 con 0x7fa64415a0f0 2023-08-27T11:28:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.430+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64406e5c0 con 0x7fa64415a0f0 2023-08-27T11:28:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.431+0000 7fa639ffb700 1 -- 172.21.15.45:0/271240584 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fa63400cbb0 con 0x7fa64415a0f0 2023-08-27T11:28:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.431+0000 7fa639ffb700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa62806a170 0x7fa62806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.431+0000 7fa639ffb700 1 -- 172.21.15.45:0/271240584 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fa634088680 con 0x7fa64415a0f0 2023-08-27T11:28:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.431+0000 7fa649392700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa62806a170 0x7fa62806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.432+0000 7fa649392700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa62806a170 0x7fa62806c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa640014040 tx=0x7fa640008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.436+0000 7fa639ffb700 1 -- 172.21.15.45:0/271240584 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa634057680 con 0x7fa64415a0f0 2023-08-27T11:28:01.559 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:01.557+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:01.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.688+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6440726d0 con 0x7fa64415a0f0 2023-08-27T11:28:01.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.688+0000 7fa639ffb700 1 -- 172.21.15.45:0/271240584 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fa634021e90 con 0x7fa64415a0f0 2023-08-27T11:28:01.691 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:01.692 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/2859687397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":2859687397},{"type":"v1","addr":"172.21.15.135:6837","nonce":2859687397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2178366980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2178366980},{"type":"v1","addr":"172.21.15.135:6836","nonce":2178366980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2939076938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2939076938},{"type":"v1","addr":"172.21.15.45:6833","nonce":2939076938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/148627485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":148627485},{"type":"v1","addr":"172.21.15.45:6835","nonce":148627485}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2023-08-27T11:28:01.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa62806a170 msgr2=0x7fa62806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa62806a170 0x7fa62806c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa640014040 tx=0x7fa640008030 comp rx=0 tx=0).stop 2023-08-27T11:28:01.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 msgr2=0x7fa6440b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 0x7fa6440b3c40 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e870 tx=0x7fa63400ec30 comp rx=0 tx=0).stop 2023-08-27T11:28:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 shutdown_connections 2023-08-27T11:28:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa62806a170 0x7fa62806c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa64415e030 0x7fa64415a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6440cd300 0x7fa64406de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 --2- 172.21.15.45:0/271240584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa64415a0f0 0x7fa6440b3c40 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.690+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 >> 172.21.15.45:0/271240584 conn(0x7fa6440cc7e0 msgr2=0x7fa64406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.691+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 shutdown_connections 2023-08-27T11:28:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.691+0000 7fa64b5f6700 1 -- 172.21.15.45:0/271240584 wait complete. 2023-08-27T11:28:01.695 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 36 2023-08-27T11:28:01.705 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:28:01.706 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:28:01.706 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:28:01.707 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:01.707 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:28:01.707 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:28:01.707 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:28:01.708 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:01.708 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:28:01.708 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:28:01.709 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:28:01.709 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:01.709 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:28:01.710 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:28:01.710 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:28:01.710 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:01.715 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:28:01.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:28:01.716 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:28:01.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:28:01.718 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:28:01.718 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:28:01.719 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:28:01.719 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:28:01.759 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:28:01.760 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:01.762 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:28:01.762 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:01.764 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:28:01.765 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:28:01.765 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:01.766 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:01.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:28:01.842 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2023-08-27T11:28:01.842 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2023-08-27T11:28:01.843 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-08-27T11:28:01.844 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-08-27T11:28:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.900+0000 7f0075df7700 1 Processor -- start 2023-08-27T11:28:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.900+0000 7f0075df7700 1 -- start start 2023-08-27T11:28:01.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.901+0000 7f0075df7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0070071e10 0x7f0070071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.901+0000 7f0075df7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00700c5740 0x7f00700c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.901+0000 7f0075df7700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007015ea60 con 0x7f007015a000 2023-08-27T11:28:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.901+0000 7f0075df7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00700c6130 con 0x7f0070071e10 2023-08-27T11:28:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.901+0000 7f0075df7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007015b3f0 con 0x7f00700c5740 2023-08-27T11:28:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.901+0000 7f006effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0070071e10 0x7f0070071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.901+0000 7f006effd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0070071e10 0x7f0070071820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58880/0 (socket says 172.21.15.45:58880) 2023-08-27T11:28:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.901+0000 7f006effd700 1 -- 172.21.15.45:0/970319335 learned_addr learned my addr 172.21.15.45:0/970319335 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.901+0000 7f006ffff700 1 --2- 172.21.15.45:0/970319335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00700c5740 0x7f00700c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f006effd700 1 -- 172.21.15.45:0/970319335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00700c5740 msgr2=0x7f00700c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f006effd700 1 --2- 172.21.15.45:0/970319335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00700c5740 0x7f00700c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f006e7fc700 1 -- 172.21.15.45:0/970319335 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655614382 0 0) 0x7f0060001110 con 0x7f007015a000 2023-08-27T11:28:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f006effd700 1 -- 172.21.15.45:0/970319335 >> v1:172.21.15.135:6789/0 conn(0x7f007015a000 legacy=0x7f007015e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f006effd700 1 -- 172.21.15.45:0/970319335 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007015dec0 con 0x7f0070071e10 2023-08-27T11:28:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f006effd700 1 --2- 172.21.15.45:0/970319335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070071e10 0x7f0070071820 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f00640098f0 tx=0x7f0064009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9303f401852c33c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f006e7fc700 1 -- 172.21.15.45:0/970319335 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064011830 con 0x7f0070071e10 2023-08-27T11:28:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f006e7fc700 1 -- 172.21.15.45:0/970319335 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0064011990 con 0x7f0070071e10 2023-08-27T11:28:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f006e7fc700 1 -- 172.21.15.45:0/970319335 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064011d20 con 0x7f0070071e10 2023-08-27T11:28:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.902+0000 7f0075df7700 1 -- 172.21.15.45:0/970319335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070071e10 msgr2=0x7f0070071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.903+0000 7f0075df7700 1 --2- 172.21.15.45:0/970319335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070071e10 0x7f0070071820 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f00640098f0 tx=0x7f0064009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.903+0000 7f0075df7700 1 -- 172.21.15.45:0/970319335 shutdown_connections 2023-08-27T11:28:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.903+0000 7f0075df7700 1 --2- 172.21.15.45:0/970319335 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f00700c5740 0x7f00700c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.903+0000 7f0075df7700 1 --2- 172.21.15.45:0/970319335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0070071e10 0x7f0070071820 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.903+0000 7f0075df7700 1 -- 172.21.15.45:0/970319335 >> 172.21.15.45:0/970319335 conn(0x7f00700cfdd0 msgr2=0x7f00700d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.903+0000 7f0075df7700 1 -- 172.21.15.45:0/970319335 shutdown_connections 2023-08-27T11:28:01.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.903+0000 7f0075df7700 1 -- 172.21.15.45:0/970319335 wait complete. 2023-08-27T11:28:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.904+0000 7f0075df7700 1 Processor -- start 2023-08-27T11:28:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.904+0000 7f0075df7700 1 -- start start 2023-08-27T11:28:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f0075df7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0070071e10 0x7f00700b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f0075df7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00700c5740 0x7f00701f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f0075df7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007015a000 0x7f00701578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f0075df7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0070156eb0 con 0x7f00700c5740 2023-08-27T11:28:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f0075df7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00700b49d0 con 0x7f007015a000 2023-08-27T11:28:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f0075df7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0070156230 con 0x7f0070071e10 2023-08-27T11:28:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00700c5740 0x7f00701f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00700c5740 0x7f00701f13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58886/0 (socket says 172.21.15.45:58886) 2023-08-27T11:28:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006effd700 1 -- 172.21.15.45:0/1547601329 learned_addr learned my addr 172.21.15.45:0/1547601329 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006ffff700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007015a000 0x7f00701578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006f7fe700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0070071e10 0x7f00700b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006effd700 1 -- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0070071e10 msgr2=0x7f00700b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006effd700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0070071e10 0x7f00700b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006effd700 1 -- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007015a000 msgr2=0x7f00701578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006effd700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007015a000 0x7f00701578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006effd700 1 -- 172.21.15.45:0/1547601329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064009550 con 0x7f00700c5740 2023-08-27T11:28:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006ffff700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007015a000 0x7f00701578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006f7fe700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0070071e10 0x7f00700b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f006effd700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00700c5740 0x7f00701f13b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0064009c40 tx=0x7f0064004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.905+0000 7f0057fff700 1 -- 172.21.15.45:0/1547601329 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00640036a0 con 0x7f00700c5740 2023-08-27T11:28:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.906+0000 7f0057fff700 1 -- 172.21.15.45:0/1547601329 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006401e070 con 0x7f00700c5740 2023-08-27T11:28:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.906+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0070156510 con 0x7f00700c5740 2023-08-27T11:28:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.906+0000 7f0057fff700 1 -- 172.21.15.45:0/1547601329 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064003a20 con 0x7f00700c5740 2023-08-27T11:28:01.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.906+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0070156ad0 con 0x7f00700c5740 2023-08-27T11:28:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.907+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00700727c0 con 0x7f00700c5740 2023-08-27T11:28:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.907+0000 7f0057fff700 1 -- 172.21.15.45:0/1547601329 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f0064003b80 con 0x7f00700c5740 2023-08-27T11:28:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.908+0000 7f0057fff700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f005c06a170 0x7f005c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.908+0000 7f0057fff700 1 -- 172.21.15.45:0/1547601329 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f0064089590 con 0x7f00700c5740 2023-08-27T11:28:01.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.908+0000 7f006f7fe700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f005c06a170 0x7f005c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.908+0000 7f006f7fe700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f005c06a170 0x7f005c06c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0060000f60 tx=0x7f0060009040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:01.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:01.912+0000 7f0057fff700 1 -- 172.21.15.45:0/1547601329 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0064054c00 con 0x7f00700c5740 2023-08-27T11:28:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:02.136+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f00700d0c70 con 0x7f00700c5740 2023-08-27T11:28:02.265 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:02.264+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:02.965 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:02.964+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:03.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.097+0000 7f0057fff700 1 -- 172.21.15.45:0/1547601329 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v96) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0064058220 con 0x7f00700c5740 2023-08-27T11:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f005c06a170 msgr2=0x7f005c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f005c06a170 0x7f005c06c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0060000f60 tx=0x7f0060009040 comp rx=0 tx=0).stop 2023-08-27T11:28:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00700c5740 msgr2=0x7f00701f13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00700c5740 0x7f00701f13b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0064009c40 tx=0x7f0064004750 comp rx=0 tx=0).stop 2023-08-27T11:28:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 shutdown_connections 2023-08-27T11:28:03.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f005c06a170 0x7f005c06c630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f007015a000 0x7f00701578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0070071e10 0x7f00700b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 --2- 172.21.15.45:0/1547601329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00700c5740 0x7f00701f13b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 >> 172.21.15.45:0/1547601329 conn(0x7f00700cfdd0 msgr2=0x7f007006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:03.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 shutdown_connections 2023-08-27T11:28:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.099+0000 7f0075df7700 1 -- 172.21.15.45:0/1547601329 wait complete. 2023-08-27T11:28:03.104 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-08-27T11:28:03.114 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:28:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.238+0000 7fa1e2c19700 1 Processor -- start 2023-08-27T11:28:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.238+0000 7fa1e2c19700 1 -- start start 2023-08-27T11:28:03.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1e2c19700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 0x7fa1dc0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1e2c19700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1dc06bda0 0x7fa1dc06c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1e2c19700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0cff50 con 0x7fa1dc0cc790 2023-08-27T11:28:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1e2c19700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cc4e0 con 0x7fa1dc06bda0 2023-08-27T11:28:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1e2c19700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cd330 con 0x7fa1dc15d140 2023-08-27T11:28:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1dbfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1dc06bda0 0x7fa1dc06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1e11b6700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa1dc0cc790 0x7fa1dc0ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45676/0 (socket says 172.21.15.45:45676) 2023-08-27T11:28:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1e11b6700 1 -- 172.21.15.45:0/4274548133 learned_addr learned my addr 172.21.15.45:0/4274548133 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1e09b5700 1 --2- 172.21.15.45:0/4274548133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 0x7fa1dc0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1dbfff700 1 -- 172.21.15.45:0/4274548133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 msgr2=0x7fa1dc0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1dbfff700 1 --2- 172.21.15.45:0/4274548133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 0x7fa1dc0ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1dbfff700 1 -- 172.21.15.45:0/4274548133 >> v1:172.21.15.45:6789/0 conn(0x7fa1dc0cc790 legacy=0x7fa1dc0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1dbfff700 1 -- 172.21.15.45:0/4274548133 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0cf0a0 con 0x7fa1dc06bda0 2023-08-27T11:28:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.239+0000 7fa1db7fe700 1 -- 172.21.15.45:0/4274548133 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777380392 0 0) 0x7fa1dc0cff50 con 0x7fa1dc0cc790 2023-08-27T11:28:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1dbfff700 1 --2- 172.21.15.45:0/4274548133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc06bda0 0x7fa1dc06c220 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009840 tx=0x7fa1c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5386b61a172092 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1db7fe700 1 -- 172.21.15.45:0/4274548133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8011830 con 0x7fa1dc06bda0 2023-08-27T11:28:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1db7fe700 1 -- 172.21.15.45:0/4274548133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c8011990 con 0x7fa1dc06bda0 2023-08-27T11:28:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1db7fe700 1 -- 172.21.15.45:0/4274548133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8011d20 con 0x7fa1dc06bda0 2023-08-27T11:28:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1e2c19700 1 -- 172.21.15.45:0/4274548133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc06bda0 msgr2=0x7fa1dc06c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1e2c19700 1 --2- 172.21.15.45:0/4274548133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc06bda0 0x7fa1dc06c220 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009840 tx=0x7fa1c8009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1e2c19700 1 -- 172.21.15.45:0/4274548133 shutdown_connections 2023-08-27T11:28:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1e2c19700 1 --2- 172.21.15.45:0/4274548133 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 0x7fa1dc0ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1e2c19700 1 --2- 172.21.15.45:0/4274548133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc06bda0 0x7fa1dc06c220 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1e2c19700 1 -- 172.21.15.45:0/4274548133 >> 172.21.15.45:0/4274548133 conn(0x7fa1dc09a160 msgr2=0x7fa1dc0d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:03.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1e2c19700 1 -- 172.21.15.45:0/4274548133 shutdown_connections 2023-08-27T11:28:03.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.240+0000 7fa1e2c19700 1 -- 172.21.15.45:0/4274548133 wait complete. 2023-08-27T11:28:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.241+0000 7fa1e2c19700 1 Processor -- start 2023-08-27T11:28:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1e2c19700 1 -- start start 2023-08-27T11:28:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1e2c19700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1dc06bda0 0x7fa1dc157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1e2c19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc0cc790 0x7fa1dc0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1e2c19700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 0x7fa1dc0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1e2c19700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc15e410 con 0x7fa1dc0cc790 2023-08-27T11:28:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1e2c19700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc158250 con 0x7fa1dc06bda0 2023-08-27T11:28:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1e2c19700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0b30a0 con 0x7fa1dc15d140 2023-08-27T11:28:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1dbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc0cc790 0x7fa1dc0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1dbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc0cc790 0x7fa1dc0cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58904/0 (socket says 172.21.15.45:58904) 2023-08-27T11:28:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.242+0000 7fa1dbfff700 1 -- 172.21.15.45:0/2017695999 learned_addr learned my addr 172.21.15.45:0/2017695999 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1e11b6700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 0x7fa1dc0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1e09b5700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1dc06bda0 0x7fa1dc157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1dbfff700 1 -- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 msgr2=0x7fa1dc0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1dbfff700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 0x7fa1dc0ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1dbfff700 1 -- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1dc06bda0 msgr2=0x7fa1dc157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1dbfff700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1dc06bda0 0x7fa1dc157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1dbfff700 1 -- 172.21.15.45:0/2017695999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c80094a0 con 0x7fa1dc0cc790 2023-08-27T11:28:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1e09b5700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1dc06bda0 0x7fa1dc157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1dbfff700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc0cc790 0x7fa1dc0cdb50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009b90 tx=0x7fa1c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1d97fa700 1 -- 172.21.15.45:0/2017695999 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80036a0 con 0x7fa1dc0cc790 2023-08-27T11:28:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1d97fa700 1 -- 172.21.15.45:0/2017695999 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c801e070 con 0x7fa1dc0cc790 2023-08-27T11:28:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1d97fa700 1 -- 172.21.15.45:0/2017695999 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8003a20 con 0x7fa1dc0cc790 2023-08-27T11:28:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc15fb80 con 0x7fa1dc0cc790 2023-08-27T11:28:03.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.243+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc15d930 con 0x7fa1dc0cc790 2023-08-27T11:28:03.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.244+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc0cf8f0 con 0x7fa1dc0cc790 2023-08-27T11:28:03.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.245+0000 7fa1d97fa700 1 -- 172.21.15.45:0/2017695999 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fa1c8003b80 con 0x7fa1dc0cc790 2023-08-27T11:28:03.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.245+0000 7fa1d97fa700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa1c006a170 0x7fa1c006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.245+0000 7fa1d97fa700 1 -- 172.21.15.45:0/2017695999 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fa1c8089280 con 0x7fa1dc0cc790 2023-08-27T11:28:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.245+0000 7fa1e09b5700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa1c006a170 0x7fa1c006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.246+0000 7fa1e09b5700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa1c006a170 0x7fa1c006c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009c30 tx=0x7fa1d0008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.250+0000 7fa1d97fa700 1 -- 172.21.15.45:0/2017695999 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa1c8054bf0 con 0x7fa1dc0cc790 2023-08-27T11:28:03.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.529+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa1dc0b3530 con 0x7fa1dc0cc790 2023-08-27T11:28:03.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.530+0000 7fa1d97fa700 1 -- 172.21.15.45:0/2017695999 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa1c8058210 con 0x7fa1dc0cc790 2023-08-27T11:28:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:03.533 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:28:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-08-27T11:28:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-08-27T11:28:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tknzGlBhAABVGsUvHGyOdTgH9miKO0Cg==", 2023-08-27T11:28:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-08-27T11:28:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkEWbPExAAgt/9BpPD1vK9QoG8l3HeFw==", 2023-08-27T11:28:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.538 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-08-27T11:28:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk6qRCDhAAGaj+NY32IDjdRcuSNbASpw==", 2023-08-27T11:28:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.540 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.541 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-08-27T11:28:03.542 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkE9NxGxAAnWum05rSbYQSLZ9IjjqDpQ==", 2023-08-27T11:28:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:03.543 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-08-27T11:28:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBML+tkOzvBBBAA196QdZiLzcsm5hLBGDl0Qw==", 2023-08-27T11:28:03.545 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.547 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-08-27T11:28:03.547 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBPL+tkIXaxBBAAfHwZ2SsmOENhnNxZsG230Q==", 2023-08-27T11:28:03.547 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.547 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:03.547 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:03.548 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.548 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-08-27T11:28:03.549 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBSL+tkpzphBBAA37UzwlmuLdSWDIrkBu/8Rw==", 2023-08-27T11:28:03.549 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.549 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:03.549 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:03.550 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.550 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-08-27T11:28:03.551 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVL+tkt63hAxAAJReGReJHqyggSbdFEhm3+A==", 2023-08-27T11:28:03.551 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.551 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:03.551 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:03.552 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.553 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-08-27T11:28:03.553 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBbL+tkTOFXBxAAZ1I44e50IrNzgj7mjKyuGA==", 2023-08-27T11:28:03.553 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.554 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:03.554 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:03.554 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.555 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-08-27T11:28:03.555 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBeL+tkdezFBhAANof2C80WiBZ6hWm3oB3kfg==", 2023-08-27T11:28:03.555 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.555 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:03.556 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:03.556 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.556 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.557 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-08-27T11:28:03.557 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBhL+tk7auXBhAAJPwKwTVIUzao5D3t4iXYcw==", 2023-08-27T11:28:03.557 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.557 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:03.558 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:03.558 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.559 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-08-27T11:28:03.559 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBkL+tkY+ztBRAAvM2oxELLN8WV0i0spqfY6Q==", 2023-08-27T11:28:03.559 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.559 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:03.559 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:03.560 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.560 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.560 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-08-27T11:28:03.561 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ==", 2023-08-27T11:28:03.561 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.561 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:03.561 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2023-08-27T11:28:03.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:28:03.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.562 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-08-27T11:28:03.563 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkMCDoJhAAevee0PlTshlWJcPzX8fpYg==", 2023-08-27T11:28:03.563 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.563 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:03.563 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:28:03.563 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:28:03.564 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:28:03.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.565 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-08-27T11:28:03.565 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk1xh+IxAAswoUpd+D70hUwfeKI2gc0w==", 2023-08-27T11:28:03.565 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.566 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:03.566 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:28:03.566 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:28:03.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:28:03.567 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.567 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-08-27T11:28:03.567 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkGWuaLBAAYM2t+TMEs59u635jL9f9kg==", 2023-08-27T11:28:03.568 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.568 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:03.568 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:28:03.568 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:28:03.569 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:28:03.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.569 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-08-27T11:28:03.570 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkkw7pGRAAVim28nKfumQg/LcBfJpivA==", 2023-08-27T11:28:03.570 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.570 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:28:03.570 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:03.570 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:03.571 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.572 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-08-27T11:28:03.572 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkNxfXAhAA/K5r8Mx/C7nRitNuzXXvKg==", 2023-08-27T11:28:03.572 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.573 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:28:03.573 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:28:03.573 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:03.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:03.574 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-08-27T11:28:03.574 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkW6B8NRAAF2hsNVRi3Ji4M8anuYwpDg==", 2023-08-27T11:28:03.575 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:03.575 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:28:03.575 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:28:03.575 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:03.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:03.576 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-08-27T11:28:03.576 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:28:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa1c006a170 msgr2=0x7fa1c006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa1c006a170 0x7fa1c006c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009c30 tx=0x7fa1d0008040 comp rx=0 tx=0).stop 2023-08-27T11:28:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc0cc790 msgr2=0x7fa1dc0cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc0cc790 0x7fa1dc0cdb50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009b90 tx=0x7fa1c8004750 comp rx=0 tx=0).stop 2023-08-27T11:28:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 shutdown_connections 2023-08-27T11:28:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa1c006a170 0x7fa1c006c630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa1dc15d140 0x7fa1dc0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1dc06bda0 0x7fa1dc157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 --2- 172.21.15.45:0/2017695999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1dc0cc790 0x7fa1dc0cdb50 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 >> 172.21.15.45:0/2017695999 conn(0x7fa1dc09a160 msgr2=0x7fa1dc15c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 shutdown_connections 2023-08-27T11:28:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.532+0000 7fa1e2c19700 1 -- 172.21.15.45:0/2017695999 wait complete. 2023-08-27T11:28:03.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:03.664 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:03.662+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:03.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.709+0000 7f13066f4700 1 Processor -- start 2023-08-27T11:28:03.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.709+0000 7f13066f4700 1 -- start start 2023-08-27T11:28:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f13066f4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130006b090 0x7f13000c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f13066f4700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f13000cbda0 0x7f13000b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f13066f4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13000afce0 con 0x7f13000b0690 2023-08-27T11:28:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f13066f4700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13000cc230 con 0x7f13000cbda0 2023-08-27T11:28:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f13066f4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13000cc510 con 0x7f130006b090 2023-08-27T11:28:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f1304c91700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f13000b0690 0x7f13000719c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45690/0 (socket says 172.21.15.45:45690) 2023-08-27T11:28:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f1304c91700 1 -- 172.21.15.45:0/351152439 learned_addr learned my addr 172.21.15.45:0/351152439 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f12fffff700 1 --2- 172.21.15.45:0/351152439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130006b090 0x7f13000c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f12ff7fe700 1 --2- 172.21.15.45:0/351152439 >> v2:172.21.15.135:3300/0 conn(0x7f13000cbda0 0x7f13000b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f12feffd700 1 -- 172.21.15.45:0/351152439 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405497707 0 0) 0x7f13000afce0 con 0x7f13000b0690 2023-08-27T11:28:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f12feffd700 1 -- 172.21.15.45:0/351152439 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12ec0035b0 con 0x7f13000b0690 2023-08-27T11:28:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f12fffff700 1 -- 172.21.15.45:0/351152439 >> v2:172.21.15.135:3300/0 conn(0x7f13000cbda0 msgr2=0x7f13000b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f12fffff700 1 --2- 172.21.15.45:0/351152439 >> v2:172.21.15.135:3300/0 conn(0x7f13000cbda0 0x7f13000b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f12fffff700 1 -- 172.21.15.45:0/351152439 >> v1:172.21.15.45:6789/0 conn(0x7f13000b0690 legacy=0x7f13000719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f12fffff700 1 -- 172.21.15.45:0/351152439 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13000cc7f0 con 0x7f130006b090 2023-08-27T11:28:03.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.710+0000 7f12ff7fe700 1 --2- 172.21.15.45:0/351152439 >> v2:172.21.15.135:3300/0 conn(0x7f13000cbda0 0x7f13000b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:03.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f12fffff700 1 --2- 172.21.15.45:0/351152439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130006b090 0x7f13000c9b70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009840 tx=0x7f12f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf66e01f98731f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:03.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f12feffd700 1 -- 172.21.15.45:0/351152439 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0011830 con 0x7f130006b090 2023-08-27T11:28:03.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f12feffd700 1 -- 172.21.15.45:0/351152439 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0011990 con 0x7f130006b090 2023-08-27T11:28:03.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f12feffd700 1 -- 172.21.15.45:0/351152439 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0011d20 con 0x7f130006b090 2023-08-27T11:28:03.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f13066f4700 1 -- 172.21.15.45:0/351152439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130006b090 msgr2=0x7f13000c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f13066f4700 1 --2- 172.21.15.45:0/351152439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130006b090 0x7f13000c9b70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009840 tx=0x7f12f0009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:03.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f13066f4700 1 -- 172.21.15.45:0/351152439 shutdown_connections 2023-08-27T11:28:03.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f13066f4700 1 --2- 172.21.15.45:0/351152439 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f130006b090 0x7f13000c9b70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f13066f4700 1 --2- 172.21.15.45:0/351152439 >> v2:172.21.15.135:3300/0 conn(0x7f13000cbda0 0x7f13000b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.711+0000 7f13066f4700 1 -- 172.21.15.45:0/351152439 >> 172.21.15.45:0/351152439 conn(0x7f13000b2470 msgr2=0x7f13000b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:03.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.712+0000 7f13066f4700 1 -- 172.21.15.45:0/351152439 shutdown_connections 2023-08-27T11:28:03.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.712+0000 7f13066f4700 1 -- 172.21.15.45:0/351152439 wait complete. 2023-08-27T11:28:03.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.713+0000 7f13066f4700 1 Processor -- start 2023-08-27T11:28:03.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.713+0000 7f13066f4700 1 -- start start 2023-08-27T11:28:03.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f13066f4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130006b090 0x7f13000cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f13066f4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13000b0690 0x7f13000cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f13066f4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13000acd70 0x7f13000cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f13066f4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13000ce4a0 con 0x7f13000b0690 2023-08-27T11:28:03.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f13066f4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13000cdee0 con 0x7f130006b090 2023-08-27T11:28:03.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f13066f4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13000ce1c0 con 0x7f13000acd70 2023-08-27T11:28:03.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12ff7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13000b0690 0x7f13000cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12ff7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13000b0690 0x7f13000cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58918/0 (socket says 172.21.15.45:58918) 2023-08-27T11:28:03.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12ff7fe700 1 -- 172.21.15.45:0/89485935 learned_addr learned my addr 172.21.15.45:0/89485935 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:03.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12fffff700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130006b090 0x7f13000cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f1304c91700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13000acd70 0x7f13000cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12ff7fe700 1 -- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13000acd70 msgr2=0x7f13000cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12ff7fe700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13000acd70 0x7f13000cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12ff7fe700 1 -- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130006b090 msgr2=0x7f13000cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12ff7fe700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130006b090 0x7f13000cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12ff7fe700 1 -- 172.21.15.45:0/89485935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e80094a0 con 0x7f13000b0690 2023-08-27T11:28:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12ff7fe700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13000b0690 0x7f13000cd340 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f12e800ebb0 tx=0x7f12e800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12fcff9700 1 -- 172.21.15.45:0/89485935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e800c850 con 0x7f13000b0690 2023-08-27T11:28:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12fcff9700 1 -- 172.21.15.45:0/89485935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e800c9b0 con 0x7f13000b0690 2023-08-27T11:28:03.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f00094a0 con 0x7f13000b0690 2023-08-27T11:28:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.714+0000 7f12fcff9700 1 -- 172.21.15.45:0/89485935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e800cd40 con 0x7f13000b0690 2023-08-27T11:28:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.715+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13000cfe10 con 0x7f13000b0690 2023-08-27T11:28:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.715+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13000cf1f0 con 0x7f13000b0690 2023-08-27T11:28:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.716+0000 7f12fcff9700 1 -- 172.21.15.45:0/89485935 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f12e8013470 con 0x7f13000b0690 2023-08-27T11:28:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.717+0000 7f12fcff9700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f12e006a170 0x7f12e006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:03.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.717+0000 7f12fcff9700 1 -- 172.21.15.45:0/89485935 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f12e8088730 con 0x7f13000b0690 2023-08-27T11:28:03.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.717+0000 7f12fffff700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f12e006a170 0x7f12e006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:03.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.717+0000 7f12fffff700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f12e006a170 0x7f12e006c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12f0000c00 tx=0x7f12f0008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:03.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.721+0000 7f12fcff9700 1 -- 172.21.15.45:0/89485935 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f12e8054010 con 0x7f13000b0690 2023-08-27T11:28:03.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.923+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13000ad180 con 0x7f13000b0690 2023-08-27T11:28:03.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.924+0000 7f12fcff9700 1 -- 172.21.15.45:0/89485935 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+14927 (secure 0 0 0) 0x7f12e8057630 con 0x7f13000b0690 2023-08-27T11:28:03.926 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:03.926 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":96,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:03.095787+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f12e006a170 msgr2=0x7f12e006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f12e006a170 0x7f12e006c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12f0000c00 tx=0x7f12f0008030 comp rx=0 tx=0).stop 2023-08-27T11:28:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13000b0690 msgr2=0x7f13000cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:03.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13000b0690 0x7f13000cd340 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f12e800ebb0 tx=0x7f12e800eec0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 shutdown_connections 2023-08-27T11:28:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f12e006a170 0x7f12e006c630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f13000acd70 0x7f13000cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130006b090 0x7f13000cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 --2- 172.21.15.45:0/89485935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13000b0690 0x7f13000cd340 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 >> 172.21.15.45:0/89485935 conn(0x7f13000b2470 msgr2=0x7f1300152470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:03.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 shutdown_connections 2023-08-27T11:28:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:03.926+0000 7f13066f4700 1 -- 172.21.15.45:0/89485935 wait complete. 2023-08-27T11:28:03.940 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:03.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.069+0000 7fdb1b8af700 1 Processor -- start 2023-08-27T11:28:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.069+0000 7fdb1b8af700 1 -- start start 2023-08-27T11:28:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb1b8af700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdb1415d3f0 0x7fdb14160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb1b8af700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb14161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb1b8af700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb14071de0 con 0x7fdb140af420 2023-08-27T11:28:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb1b8af700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb140c6ca0 con 0x7fdb1415d3f0 2023-08-27T11:28:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb1b8af700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1415d7d0 con 0x7fdb140aed60 2023-08-27T11:28:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb1964b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdb1415d3f0 0x7fdb14160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb18e4a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb14161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb19e4c700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fdb140af420 0x7fdb1406a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:41366/0 (socket says 172.21.15.45:41366) 2023-08-27T11:28:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb1964b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdb1415d3f0 0x7fdb14160e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:47780/0 (socket says 172.21.15.45:47780) 2023-08-27T11:28:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb19e4c700 1 -- 172.21.15.45:0/1781489411 learned_addr learned my addr 172.21.15.45:0/1781489411 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb18e4a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb14161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:56836/0 (socket says 172.21.15.45:56836) 2023-08-27T11:28:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb18e4a700 1 -- 172.21.15.45:0/1781489411 >> v2:172.21.15.135:3300/0 conn(0x7fdb1415d3f0 msgr2=0x7fdb14160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb18e4a700 1 --2- 172.21.15.45:0/1781489411 >> v2:172.21.15.135:3300/0 conn(0x7fdb1415d3f0 0x7fdb14160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb18e4a700 1 -- 172.21.15.45:0/1781489411 >> v1:172.21.15.135:6789/0 conn(0x7fdb140af420 legacy=0x7fdb1406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb18e4a700 1 -- 172.21.15.45:0/1781489411 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb140c5310 con 0x7fdb140aed60 2023-08-27T11:28:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.070+0000 7fdb1964b700 1 --2- 172.21.15.45:0/1781489411 >> v2:172.21.15.135:3300/0 conn(0x7fdb1415d3f0 0x7fdb14160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb18e4a700 1 --2- 172.21.15.45:0/1781489411 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb14161390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009840 tx=0x7fdb04009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=debd40e4baed6629 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb0bfff700 1 -- 172.21.15.45:0/1781489411 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04011830 con 0x7fdb140aed60 2023-08-27T11:28:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb0bfff700 1 -- 172.21.15.45:0/1781489411 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb04011990 con 0x7fdb140aed60 2023-08-27T11:28:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb0bfff700 1 -- 172.21.15.45:0/1781489411 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04011d20 con 0x7fdb140aed60 2023-08-27T11:28:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb1b8af700 1 -- 172.21.15.45:0/1781489411 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 msgr2=0x7fdb14161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb1b8af700 1 --2- 172.21.15.45:0/1781489411 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb14161390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009840 tx=0x7fdb04009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb1b8af700 1 -- 172.21.15.45:0/1781489411 shutdown_connections 2023-08-27T11:28:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb1b8af700 1 --2- 172.21.15.45:0/1781489411 >> v2:172.21.15.135:3300/0 conn(0x7fdb1415d3f0 0x7fdb14160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb1b8af700 1 --2- 172.21.15.45:0/1781489411 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb14161390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb1b8af700 1 -- 172.21.15.45:0/1781489411 >> 172.21.15.45:0/1781489411 conn(0x7fdb140cfdd0 msgr2=0x7fdb140d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb1b8af700 1 -- 172.21.15.45:0/1781489411 shutdown_connections 2023-08-27T11:28:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.071+0000 7fdb1b8af700 1 -- 172.21.15.45:0/1781489411 wait complete. 2023-08-27T11:28:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.073+0000 7fdb1b8af700 1 Processor -- start 2023-08-27T11:28:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.073+0000 7fdb1b8af700 1 -- start start 2023-08-27T11:28:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.073+0000 7fdb1b8af700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb140cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.073+0000 7fdb1b8af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb140af420 0x7fdb1414eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.073+0000 7fdb1b8af700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb1415d3f0 0x7fdb140ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.073+0000 7fdb1b8af700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140b3a20 con 0x7fdb140af420 2023-08-27T11:28:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.073+0000 7fdb1b8af700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140ad920 con 0x7fdb1415d3f0 2023-08-27T11:28:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.073+0000 7fdb1b8af700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140ad360 con 0x7fdb140aed60 2023-08-27T11:28:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb18e4a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb140af420 0x7fdb1414eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb18e4a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb140af420 0x7fdb1414eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58932/0 (socket says 172.21.15.45:58932) 2023-08-27T11:28:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb18e4a700 1 -- 172.21.15.45:0/3977275090 learned_addr learned my addr 172.21.15.45:0/3977275090 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb19e4c700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb1415d3f0 0x7fdb140ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb1964b700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb140cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb1964b700 1 -- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb1415d3f0 msgr2=0x7fdb140ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb1964b700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb1415d3f0 0x7fdb140ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb1964b700 1 -- 172.21.15.45:0/3977275090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb140af420 msgr2=0x7fdb1414eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb1964b700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb140af420 0x7fdb1414eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb1964b700 1 -- 172.21.15.45:0/3977275090 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb100094a0 con 0x7fdb140aed60 2023-08-27T11:28:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb18e4a700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb140af420 0x7fdb1414eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb1964b700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb140cc770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000e900 tx=0x7fdb1000ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb09ffb700 1 -- 172.21.15.45:0/3977275090 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10013680 con 0x7fdb140aed60 2023-08-27T11:28:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb09ffb700 1 -- 172.21.15.45:0/3977275090 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb100137e0 con 0x7fdb140aed60 2023-08-27T11:28:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb040094a0 con 0x7fdb140aed60 2023-08-27T11:28:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.074+0000 7fdb09ffb700 1 -- 172.21.15.45:0/3977275090 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10013b70 con 0x7fdb140aed60 2023-08-27T11:28:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.075+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb140b3c50 con 0x7fdb140aed60 2023-08-27T11:28:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.076+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb140b42a0 con 0x7fdb140aed60 2023-08-27T11:28:04.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.076+0000 7fdb09ffb700 1 -- 172.21.15.45:0/3977275090 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fdb10013cd0 con 0x7fdb140aed60 2023-08-27T11:28:04.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.076+0000 7fdb09ffb700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdaf806a170 0x7fdaf806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.077+0000 7fdb09ffb700 1 -- 172.21.15.45:0/3977275090 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fdb10088310 con 0x7fdb140aed60 2023-08-27T11:28:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.077+0000 7fdb18e4a700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdaf806a170 0x7fdaf806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.077+0000 7fdb18e4a700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdaf806a170 0x7fdaf806c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb141535d0 tx=0x7fdb04008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.081+0000 7fdb09ffb700 1 -- 172.21.15.45:0/3977275090 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdb10053c00 con 0x7fdb140aed60 2023-08-27T11:28:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.286+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb140727c0 con 0x7fdb140aed60 2023-08-27T11:28:04.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.286+0000 7fdb09ffb700 1 -- 172.21.15.45:0/3977275090 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb100210a0 con 0x7fdb140aed60 2023-08-27T11:28:04.288 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:28:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdaf806a170 msgr2=0x7fdaf806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdaf806a170 0x7fdaf806c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb141535d0 tx=0x7fdb04008030 comp rx=0 tx=0).stop 2023-08-27T11:28:04.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 msgr2=0x7fdb140cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb140cc770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000e900 tx=0x7fdb1000ec10 comp rx=0 tx=0).stop 2023-08-27T11:28:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 shutdown_connections 2023-08-27T11:28:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdaf806a170 0x7fdaf806c630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb1415d3f0 0x7fdb140ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdb140aed60 0x7fdb140cc770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 --2- 172.21.15.45:0/3977275090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb140af420 0x7fdb1414eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 >> 172.21.15.45:0/3977275090 conn(0x7fdb140cfdd0 msgr2=0x7fdb140afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 shutdown_connections 2023-08-27T11:28:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.289+0000 7fdb1b8af700 1 -- 172.21.15.45:0/3977275090 wait complete. 2023-08-27T11:28:04.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:04.355 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:04.353+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:04.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.430+0000 7f607e27a700 1 Processor -- start 2023-08-27T11:28:04.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.431+0000 7f607e27a700 1 -- start start 2023-08-27T11:28:04.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.431+0000 7f607e27a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f607806cf40 0x7f607806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.431+0000 7f607e27a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607806f8b0 con 0x7f607815b270 2023-08-27T11:28:04.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.431+0000 7f607e27a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607806f010 con 0x7f6078071e80 2023-08-27T11:28:04.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.431+0000 7f607e27a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60780ca650 con 0x7f607806cf40 2023-08-27T11:28:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.431+0000 7f6077fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f607806cf40 0x7f607806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.431+0000 7f60777fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f607815b270 0x7f607806d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45714/0 (socket says 172.21.15.45:45714) 2023-08-27T11:28:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.431+0000 7f60777fe700 1 -- 172.21.15.45:0/2915687202 learned_addr learned my addr 172.21.15.45:0/2915687202 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.432+0000 7f6077fff700 1 -- 172.21.15.45:0/2915687202 >> v1:172.21.15.135:6789/0 conn(0x7f6078071e80 legacy=0x7f60780caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.432+0000 7f6077fff700 1 -- 172.21.15.45:0/2915687202 >> v1:172.21.15.45:6789/0 conn(0x7f607815b270 legacy=0x7f607806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.432+0000 7f6077fff700 1 -- 172.21.15.45:0/2915687202 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607806f2f0 con 0x7f607806cf40 2023-08-27T11:28:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.432+0000 7f6076ffd700 1 -- 172.21.15.45:0/2915687202 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216028165 0 0) 0x7f6060001110 con 0x7f607815b270 2023-08-27T11:28:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.432+0000 7f6077fff700 1 --2- 172.21.15.45:0/2915687202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607806cf40 0x7f607806ac90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6068009840 tx=0x7f6068009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d05ad5bab73e2d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.432+0000 7f6076ffd700 1 -- 172.21.15.45:0/2915687202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068011830 con 0x7f607806cf40 2023-08-27T11:28:04.437 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.432+0000 7f6076ffd700 1 -- 172.21.15.45:0/2915687202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6068011990 con 0x7f607806cf40 2023-08-27T11:28:04.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.432+0000 7f6076ffd700 1 -- 172.21.15.45:0/2915687202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068011d20 con 0x7f607806cf40 2023-08-27T11:28:04.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.433+0000 7f607e27a700 1 -- 172.21.15.45:0/2915687202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607806cf40 msgr2=0x7f607806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.433+0000 7f607e27a700 1 --2- 172.21.15.45:0/2915687202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607806cf40 0x7f607806ac90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6068009840 tx=0x7f6068009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:04.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.433+0000 7f607e27a700 1 -- 172.21.15.45:0/2915687202 shutdown_connections 2023-08-27T11:28:04.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.433+0000 7f607e27a700 1 --2- 172.21.15.45:0/2915687202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607806cf40 0x7f607806ac90 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.433+0000 7f607e27a700 1 -- 172.21.15.45:0/2915687202 >> 172.21.15.45:0/2915687202 conn(0x7f60780cc7e0 msgr2=0x7f60780ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:04.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.433+0000 7f607e27a700 1 -- 172.21.15.45:0/2915687202 shutdown_connections 2023-08-27T11:28:04.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.433+0000 7f607e27a700 1 -- 172.21.15.45:0/2915687202 wait complete. 2023-08-27T11:28:04.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.434+0000 7f607e27a700 1 Processor -- start 2023-08-27T11:28:04.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.435+0000 7f607e27a700 1 -- start start 2023-08-27T11:28:04.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.435+0000 7f607e27a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6078071e80 0x7f607814ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.435+0000 7f607e27a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607815b270 0x7f6078152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.435+0000 7f607e27a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6078153310 0x7f6078150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.435+0000 7f607e27a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780c6b50 con 0x7f607815b270 2023-08-27T11:28:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.435+0000 7f607e27a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780c62b0 con 0x7f6078153310 2023-08-27T11:28:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.435+0000 7f607e27a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60780c6590 con 0x7f6078071e80 2023-08-27T11:28:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f60777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607815b270 0x7f6078152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f60777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607815b270 0x7f6078152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58946/0 (socket says 172.21.15.45:58946) 2023-08-27T11:28:04.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f60777fe700 1 -- 172.21.15.45:0/2810661117 learned_addr learned my addr 172.21.15.45:0/2810661117 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f6077fff700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6078071e80 0x7f607814ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f607c817700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6078153310 0x7f6078150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f60777fe700 1 -- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6078071e80 msgr2=0x7f607814ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f60777fe700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6078071e80 0x7f607814ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f60777fe700 1 -- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6078153310 msgr2=0x7f6078150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f60777fe700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6078153310 0x7f6078150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f60777fe700 1 -- 172.21.15.45:0/2810661117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60680094a0 con 0x7f607815b270 2023-08-27T11:28:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f60777fe700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607815b270 0x7f6078152dd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6060001eb0 tx=0x7f606000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f6074ff9700 1 -- 172.21.15.45:0/2810661117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060004c80 con 0x7f607815b270 2023-08-27T11:28:04.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f6074ff9700 1 -- 172.21.15.45:0/2810661117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6060004de0 con 0x7f607815b270 2023-08-27T11:28:04.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60780c6870 con 0x7f607815b270 2023-08-27T11:28:04.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f6074ff9700 1 -- 172.21.15.45:0/2810661117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060007310 con 0x7f607815b270 2023-08-27T11:28:04.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.436+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607814f4d0 con 0x7f607815b270 2023-08-27T11:28:04.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.437+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60780b3dd0 con 0x7f607815b270 2023-08-27T11:28:04.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.438+0000 7f6074ff9700 1 -- 172.21.15.45:0/2810661117 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f6060007470 con 0x7f607815b270 2023-08-27T11:28:04.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.438+0000 7f6074ff9700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f605806a170 0x7f605806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.438+0000 7f6074ff9700 1 -- 172.21.15.45:0/2810661117 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f6060087d10 con 0x7f607815b270 2023-08-27T11:28:04.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.438+0000 7f6077fff700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f605806a170 0x7f605806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.439+0000 7f6077fff700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f605806a170 0x7f605806c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f607806c470 tx=0x7f606801c040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.443+0000 7f6074ff9700 1 -- 172.21.15.45:0/2810661117 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f60780b3dd0 con 0x7f607815b270 2023-08-27T11:28:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.653+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f607806f8b0 con 0x7f607815b270 2023-08-27T11:28:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.653+0000 7f6074ff9700 1 -- 172.21.15.45:0/2810661117 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v96) v1 ==== 92+0+10 (secure 0 0 0) 0x7f607806f8b0 con 0x7f607815b270 2023-08-27T11:28:04.656 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:28:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f605806a170 msgr2=0x7f605806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f605806a170 0x7f605806c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f607806c470 tx=0x7f606801c040 comp rx=0 tx=0).stop 2023-08-27T11:28:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607815b270 msgr2=0x7f6078152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607815b270 0x7f6078152dd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6060001eb0 tx=0x7f606000c730 comp rx=0 tx=0).stop 2023-08-27T11:28:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 shutdown_connections 2023-08-27T11:28:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f605806a170 0x7f605806c630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6078153310 0x7f6078150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6078071e80 0x7f607814ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 --2- 172.21.15.45:0/2810661117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607815b270 0x7f6078152dd0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.655+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 >> 172.21.15.45:0/2810661117 conn(0x7f60780cc7e0 msgr2=0x7f60780c7920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.656+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 shutdown_connections 2023-08-27T11:28:04.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.656+0000 7f607e27a700 1 -- 172.21.15.45:0/2810661117 wait complete. 2023-08-27T11:28:04.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:28:04.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.795+0000 7fbbd0689700 1 Processor -- start 2023-08-27T11:28:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.795+0000 7fbbd0689700 1 -- start start 2023-08-27T11:28:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbd0689700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbbc815d2c0 0x7fbbc80d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbd0689700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbbc80ce4b0 0x7fbbc80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbd0689700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc815d6a0 con 0x7fbbc80b8c20 2023-08-27T11:28:04.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbd0689700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc80d2c90 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbd0689700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc80d1bf0 con 0x7fbbc815d2c0 2023-08-27T11:28:04.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcec26700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbbc80ce4b0 0x7fbbc80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcdc24700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbbc80b8c20 0x7fbbc80cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45730/0 (socket says 172.21.15.45:45730) 2023-08-27T11:28:04.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcec26700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbbc80ce4b0 0x7fbbc80b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58956/0 (socket says 172.21.15.45:58956) 2023-08-27T11:28:04.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbce425700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbbc815d2c0 0x7fbbc80d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcec26700 1 -- 172.21.15.45:0/1067805491 learned_addr learned my addr 172.21.15.45:0/1067805491 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:04.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcec26700 1 -- 172.21.15.45:0/1067805491 >> v2:172.21.15.135:3300/0 conn(0x7fbbc815d2c0 msgr2=0x7fbbc80d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:04.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcec26700 1 --2- 172.21.15.45:0/1067805491 >> v2:172.21.15.135:3300/0 conn(0x7fbbc815d2c0 0x7fbbc80d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcec26700 1 -- 172.21.15.45:0/1067805491 >> v1:172.21.15.45:6789/0 conn(0x7fbbc80b8c20 legacy=0x7fbbc80cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcd423700 1 -- 172.21.15.45:0/1067805491 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743972507 0 0) 0x7fbbb8001110 con 0x7fbbc80b8c20 2023-08-27T11:28:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcec26700 1 -- 172.21.15.45:0/1067805491 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc80d1ed0 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcec26700 1 --2- 172.21.15.45:0/1067805491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 0x7fbbc80b6100 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009e50 tx=0x7fbbc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f6efc658174bc4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcd423700 1 -- 172.21.15.45:0/1067805491 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc000e070 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcd423700 1 -- 172.21.15.45:0/1067805491 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc000b8d0 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.796+0000 7fbbcd423700 1 -- 172.21.15.45:0/1067805491 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc000bc80 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.797+0000 7fbbd0689700 1 -- 172.21.15.45:0/1067805491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 msgr2=0x7fbbc80b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.797+0000 7fbbd0689700 1 --2- 172.21.15.45:0/1067805491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 0x7fbbc80b6100 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009e50 tx=0x7fbbc000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.797+0000 7fbbd0689700 1 -- 172.21.15.45:0/1067805491 shutdown_connections 2023-08-27T11:28:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.797+0000 7fbbd0689700 1 --2- 172.21.15.45:0/1067805491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 0x7fbbc80b6100 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.797+0000 7fbbd0689700 1 --2- 172.21.15.45:0/1067805491 >> v2:172.21.15.135:3300/0 conn(0x7fbbc815d2c0 0x7fbbc80d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.797+0000 7fbbd0689700 1 -- 172.21.15.45:0/1067805491 >> 172.21.15.45:0/1067805491 conn(0x7fbbc80d2fd0 msgr2=0x7fbbc80d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.797+0000 7fbbd0689700 1 -- 172.21.15.45:0/1067805491 shutdown_connections 2023-08-27T11:28:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.797+0000 7fbbd0689700 1 -- 172.21.15.45:0/1067805491 wait complete. 2023-08-27T11:28:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.799+0000 7fbbd0689700 1 Processor -- start 2023-08-27T11:28:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.799+0000 7fbbd0689700 1 -- start start 2023-08-27T11:28:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbd0689700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbc80b8c20 0x7fbbc80cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbd0689700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 0x7fbbc81f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbd0689700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbc815d2c0 0x7fbbc815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbd0689700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80cc220 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbd0689700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80c8bf0 con 0x7fbbc80b8c20 2023-08-27T11:28:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbd0689700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc81f60c0 con 0x7fbbc815d2c0 2023-08-27T11:28:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcdc24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 0x7fbbc81f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcdc24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 0x7fbbc81f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58960/0 (socket says 172.21.15.45:58960) 2023-08-27T11:28:04.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbce425700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbc80b8c20 0x7fbbc80cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcdc24700 1 -- 172.21.15.45:0/3248342680 learned_addr learned my addr 172.21.15.45:0/3248342680 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcec26700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbc815d2c0 0x7fbbc815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcdc24700 1 -- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbc815d2c0 msgr2=0x7fbbc815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcdc24700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbc815d2c0 0x7fbbc815d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcdc24700 1 -- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbc80b8c20 msgr2=0x7fbbc80cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcdc24700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbc80b8c20 0x7fbbc80cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcdc24700 1 -- 172.21.15.45:0/3248342680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc0009b30 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbcdc24700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 0x7fbbc81f5b80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c760 tx=0x7fbbb800ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.800+0000 7fbbbeffd700 1 -- 172.21.15.45:0/3248342680 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8004c80 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.801+0000 7fbbbeffd700 1 -- 172.21.15.45:0/3248342680 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb8004e00 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.801+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc80caeb0 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.801+0000 7fbbbeffd700 1 -- 172.21.15.45:0/3248342680 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8007310 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.801+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc80c85a0 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.801+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc804e620 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.802+0000 7fbbbeffd700 1 -- 172.21.15.45:0/3248342680 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fbbb8007470 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.802+0000 7fbbbeffd700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbbac06a170 0x7fbbac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.803+0000 7fbbbeffd700 1 -- 172.21.15.45:0/3248342680 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fbbb8087e30 con 0x7fbbc80ce4b0 2023-08-27T11:28:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.803+0000 7fbbce425700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbbac06a170 0x7fbbac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.803+0000 7fbbce425700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbbac06a170 0x7fbbac06c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009b40 tx=0x7fbbc4009210 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:04.807+0000 7fbbbeffd700 1 -- 172.21.15.45:0/3248342680 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbbb8053710 con 0x7fbbc80ce4b0 2023-08-27T11:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.010+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbbc815d6a0 con 0x7fbbc80ce4b0 2023-08-27T11:28:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.010+0000 7fbbbeffd700 1 -- 172.21.15.45:0/3248342680 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbbc815d6a0 con 0x7fbbc80ce4b0 2023-08-27T11:28:05.012 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:05.013 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:28:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-08-27T11:28:05.013 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-08-27T11:28:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-08-27T11:28:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-08-27T11:28:05.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:05.015 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:28:05.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbbac06a170 msgr2=0x7fbbac06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbbac06a170 0x7fbbac06c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009b40 tx=0x7fbbc4009210 comp rx=0 tx=0).stop 2023-08-27T11:28:05.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 msgr2=0x7fbbc81f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 0x7fbbc81f5b80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c760 tx=0x7fbbb800ca70 comp rx=0 tx=0).stop 2023-08-27T11:28:05.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 shutdown_connections 2023-08-27T11:28:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbbac06a170 0x7fbbac06c630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbbc815d2c0 0x7fbbc815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbc80b8c20 0x7fbbc80cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 --2- 172.21.15.45:0/3248342680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbc80ce4b0 0x7fbbc81f5b80 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 >> 172.21.15.45:0/3248342680 conn(0x7fbbc80d2fd0 msgr2=0x7fbbc815c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 shutdown_connections 2023-08-27T11:28:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.012+0000 7fbbd0689700 1 -- 172.21.15.45:0/3248342680 wait complete. 2023-08-27T11:28:05.027 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:28:05.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:28:05.055 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:05.054+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.156+0000 7feda769d700 1 Processor -- start 2023-08-27T11:28:05.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.156+0000 7feda769d700 1 -- start start 2023-08-27T11:28:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda769d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 0x7feda00716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda769d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feda00b17c0 0x7feda00b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda769d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda015b460 con 0x7feda006ced0 2023-08-27T11:28:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda769d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda0157fc0 con 0x7feda00b17c0 2023-08-27T11:28:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda769d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda00b2280 con 0x7feda0157be0 2023-08-27T11:28:05.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda4c38700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feda00b17c0 0x7feda00b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda5c3a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feda006ced0 0x7feda00cab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45744/0 (socket says 172.21.15.45:45744) 2023-08-27T11:28:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda5c3a700 1 -- 172.21.15.45:0/1106515797 learned_addr learned my addr 172.21.15.45:0/1106515797 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda5439700 1 --2- 172.21.15.45:0/1106515797 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 0x7feda00716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda4c38700 1 -- 172.21.15.45:0/1106515797 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 msgr2=0x7feda00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:05.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda4c38700 1 --2- 172.21.15.45:0/1106515797 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 0x7feda00716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.157+0000 7feda4c38700 1 -- 172.21.15.45:0/1106515797 >> v1:172.21.15.45:6789/0 conn(0x7feda006ced0 legacy=0x7feda00cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7feda4c38700 1 -- 172.21.15.45:0/1106515797 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda00b2560 con 0x7feda00b17c0 2023-08-27T11:28:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7feda4c38700 1 --2- 172.21.15.45:0/1106515797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 0x7feda00b1c40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009840 tx=0x7fed9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e47839b3906b94a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7fed97fff700 1 -- 172.21.15.45:0/1106515797 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c011830 con 0x7feda00b17c0 2023-08-27T11:28:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7fed97fff700 1 -- 172.21.15.45:0/1106515797 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c011990 con 0x7feda00b17c0 2023-08-27T11:28:05.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7fed97fff700 1 -- 172.21.15.45:0/1106515797 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c011d20 con 0x7feda00b17c0 2023-08-27T11:28:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7feda769d700 1 -- 172.21.15.45:0/1106515797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 msgr2=0x7feda00b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7feda769d700 1 --2- 172.21.15.45:0/1106515797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 0x7feda00b1c40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009840 tx=0x7fed9c009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7feda769d700 1 -- 172.21.15.45:0/1106515797 shutdown_connections 2023-08-27T11:28:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7feda769d700 1 --2- 172.21.15.45:0/1106515797 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 0x7feda00716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7feda769d700 1 --2- 172.21.15.45:0/1106515797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 0x7feda00b1c40 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.158+0000 7feda769d700 1 -- 172.21.15.45:0/1106515797 >> 172.21.15.45:0/1106515797 conn(0x7feda00cc7c0 msgr2=0x7feda00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.159+0000 7feda769d700 1 -- 172.21.15.45:0/1106515797 shutdown_connections 2023-08-27T11:28:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.159+0000 7feda769d700 1 -- 172.21.15.45:0/1106515797 wait complete. 2023-08-27T11:28:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.160+0000 7feda769d700 1 Processor -- start 2023-08-27T11:28:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.160+0000 7feda769d700 1 -- start start 2023-08-27T11:28:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda769d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feda006ced0 0x7feda01460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda769d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 0x7feda006abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda769d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 0x7feda00c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda769d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda0146610 con 0x7feda00b17c0 2023-08-27T11:28:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda769d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda00b5330 con 0x7feda006ced0 2023-08-27T11:28:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda769d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda00b4d70 con 0x7feda0157be0 2023-08-27T11:28:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda4c38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 0x7feda006abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda4c38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 0x7feda006abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58974/0 (socket says 172.21.15.45:58974) 2023-08-27T11:28:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda4c38700 1 -- 172.21.15.45:0/3710002360 learned_addr learned my addr 172.21.15.45:0/3710002360 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda5439700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feda006ced0 0x7feda01460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.161+0000 7feda5c3a700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 0x7feda00c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7feda4c38700 1 -- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 msgr2=0x7feda00c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7feda4c38700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 0x7feda00c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7feda4c38700 1 -- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feda006ced0 msgr2=0x7feda01460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7feda4c38700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feda006ced0 0x7feda01460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7feda4c38700 1 -- 172.21.15.45:0/3710002360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c0094a0 con 0x7feda00b17c0 2023-08-27T11:28:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7feda4c38700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 0x7feda006abc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009fd0 tx=0x7fed9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7fed95ffb700 1 -- 172.21.15.45:0/3710002360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0036a0 con 0x7feda00b17c0 2023-08-27T11:28:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7fed95ffb700 1 -- 172.21.15.45:0/3710002360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c01f070 con 0x7feda00b17c0 2023-08-27T11:28:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda00b5050 con 0x7feda00b17c0 2023-08-27T11:28:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7fed95ffb700 1 -- 172.21.15.45:0/3710002360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c003a60 con 0x7feda00b17c0 2023-08-27T11:28:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.162+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda00c3bf0 con 0x7feda00b17c0 2023-08-27T11:28:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.163+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda0072690 con 0x7feda00b17c0 2023-08-27T11:28:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.163+0000 7fed95ffb700 1 -- 172.21.15.45:0/3710002360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fed9c003bc0 con 0x7feda00b17c0 2023-08-27T11:28:05.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.164+0000 7fed95ffb700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fed8406a170 0x7fed8406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.164+0000 7fed95ffb700 1 -- 172.21.15.45:0/3710002360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fed9c021070 con 0x7feda00b17c0 2023-08-27T11:28:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.164+0000 7feda5439700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fed8406a170 0x7fed8406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.165+0000 7feda5439700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fed8406a170 0x7fed8406c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed90009be0 tx=0x7fed90009210 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.169+0000 7fed95ffb700 1 -- 172.21.15.45:0/3710002360 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fed9c0554d0 con 0x7feda00b17c0 2023-08-27T11:28:05.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.376+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feda00734a0 con 0x7feda00b17c0 2023-08-27T11:28:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.651+0000 7fed95ffb700 1 -- 172.21.15.45:0/3710002360 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) v1 ==== 138+0+0 (secure 0 0 0) 0x7fed9c015020 con 0x7feda00b17c0 2023-08-27T11:28:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fed8406a170 msgr2=0x7fed8406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fed8406a170 0x7fed8406c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed90009be0 tx=0x7fed90009210 comp rx=0 tx=0).stop 2023-08-27T11:28:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 msgr2=0x7feda006abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 0x7feda006abc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009fd0 tx=0x7fed9c004750 comp rx=0 tx=0).stop 2023-08-27T11:28:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 shutdown_connections 2023-08-27T11:28:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fed8406a170 0x7fed8406c630 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feda0157be0 0x7feda00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feda006ced0 0x7feda01460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 --2- 172.21.15.45:0/3710002360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda00b17c0 0x7feda006abc0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.656+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 >> 172.21.15.45:0/3710002360 conn(0x7feda00cc7c0 msgr2=0x7feda00ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.657+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 shutdown_connections 2023-08-27T11:28:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.657+0000 7feda769d700 1 -- 172.21.15.45:0/3710002360 wait complete. 2023-08-27T11:28:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-08-27T11:28:05.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:28:05.765 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:05.764+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.804+0000 7f9683026700 1 Processor -- start 2023-08-27T11:28:05.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.804+0000 7f9683026700 1 -- start start 2023-08-27T11:28:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9683026700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f967c15b2e0 0x7f967c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9683026700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cd540 0x7f967c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9683026700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f967c0cf240 0x7f967c0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9683026700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967c0cfc10 con 0x7f967c15b2e0 2023-08-27T11:28:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9683026700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967c0cb9b0 con 0x7f967c0cf240 2023-08-27T11:28:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9683026700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967c0ce2e0 con 0x7f967c0cd540 2023-08-27T11:28:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9680dc2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f967c15b2e0 0x7f967c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9680dc2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f967c15b2e0 0x7f967c0cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58980/0 (socket says 172.21.15.45:58980) 2023-08-27T11:28:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9680dc2700 1 -- 172.21.15.45:0/3206772636 learned_addr learned my addr 172.21.15.45:0/3206772636 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f967bfff700 1 --2- 172.21.15.45:0/3206772636 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cd540 0x7f967c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f96815c3700 1 --2- 172.21.15.45:0/3206772636 >> v2:172.21.15.135:3300/0 conn(0x7f967c0cf240 0x7f967c0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9680dc2700 1 -- 172.21.15.45:0/3206772636 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cd540 msgr2=0x7f967c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9680dc2700 1 --2- 172.21.15.45:0/3206772636 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cd540 0x7f967c0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9680dc2700 1 -- 172.21.15.45:0/3206772636 >> v2:172.21.15.135:3300/0 conn(0x7f967c0cf240 msgr2=0x7f967c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9680dc2700 1 --2- 172.21.15.45:0/3206772636 >> v2:172.21.15.135:3300/0 conn(0x7f967c0cf240 0x7f967c0cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.805+0000 7f9680dc2700 1 -- 172.21.15.45:0/3206772636 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c0cc230 con 0x7f967c15b2e0 2023-08-27T11:28:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9680dc2700 1 --2- 172.21.15.45:0/3206772636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c15b2e0 0x7f967c0cc6c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f967c09a640 tx=0x7f967400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1dcab02957553a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f967b7fe700 1 -- 172.21.15.45:0/3206772636 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967400e070 con 0x7f967c15b2e0 2023-08-27T11:28:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f967b7fe700 1 -- 172.21.15.45:0/3206772636 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967400b910 con 0x7f967c15b2e0 2023-08-27T11:28:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f967b7fe700 1 -- 172.21.15.45:0/3206772636 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967400bcc0 con 0x7f967c15b2e0 2023-08-27T11:28:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9683026700 1 -- 172.21.15.45:0/3206772636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c15b2e0 msgr2=0x7f967c0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9683026700 1 --2- 172.21.15.45:0/3206772636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c15b2e0 0x7f967c0cc6c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f967c09a640 tx=0x7f967400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9683026700 1 -- 172.21.15.45:0/3206772636 shutdown_connections 2023-08-27T11:28:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9683026700 1 --2- 172.21.15.45:0/3206772636 >> v2:172.21.15.135:3300/0 conn(0x7f967c0cf240 0x7f967c0cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9683026700 1 --2- 172.21.15.45:0/3206772636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c15b2e0 0x7f967c0cc6c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9683026700 1 --2- 172.21.15.45:0/3206772636 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cd540 0x7f967c0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9683026700 1 -- 172.21.15.45:0/3206772636 >> 172.21.15.45:0/3206772636 conn(0x7f967c09a1a0 msgr2=0x7f967c0d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9683026700 1 -- 172.21.15.45:0/3206772636 shutdown_connections 2023-08-27T11:28:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.806+0000 7f9683026700 1 -- 172.21.15.45:0/3206772636 wait complete. 2023-08-27T11:28:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.808+0000 7f9683026700 1 Processor -- start 2023-08-27T11:28:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.808+0000 7f9683026700 1 -- start start 2023-08-27T11:28:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9683026700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0cd540 0x7f967c158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9683026700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cf240 0x7f967c06ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9683026700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f967c15b2e0 0x7f967c06f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9683026700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c15cf60 con 0x7f967c0cd540 2023-08-27T11:28:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9683026700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c0b31c0 con 0x7f967c15b2e0 2023-08-27T11:28:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9683026700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967c158a80 con 0x7f967c0cf240 2023-08-27T11:28:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9680dc2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0cd540 0x7f967c158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9680dc2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0cd540 0x7f967c158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58984/0 (socket says 172.21.15.45:58984) 2023-08-27T11:28:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9680dc2700 1 -- 172.21.15.45:0/3359051068 learned_addr learned my addr 172.21.15.45:0/3359051068 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f967bfff700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cf240 0x7f967c06ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f96815c3700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f967c15b2e0 0x7f967c06f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9680dc2700 1 -- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cf240 msgr2=0x7f967c06ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9680dc2700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cf240 0x7f967c06ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9680dc2700 1 -- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f967c15b2e0 msgr2=0x7f967c06f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9680dc2700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f967c15b2e0 0x7f967c06f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9680dc2700 1 -- 172.21.15.45:0/3359051068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674009b30 con 0x7f967c0cd540 2023-08-27T11:28:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9680dc2700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0cd540 0x7f967c158540 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f967400b370 tx=0x7f9674004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f96797fa700 1 -- 172.21.15.45:0/3359051068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967400ce80 con 0x7f967c0cd540 2023-08-27T11:28:05.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f96797fa700 1 -- 172.21.15.45:0/3359051068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9674004250 con 0x7f967c0cd540 2023-08-27T11:28:05.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c15c890 con 0x7f967c0cd540 2023-08-27T11:28:05.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f96797fa700 1 -- 172.21.15.45:0/3359051068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96740045e0 con 0x7f967c0cd540 2023-08-27T11:28:05.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.809+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c15c5b0 con 0x7f967c0cd540 2023-08-27T11:28:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.810+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967c15d0a0 con 0x7f967c0cd540 2023-08-27T11:28:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.811+0000 7f96797fa700 1 -- 172.21.15.45:0/3359051068 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f9674004740 con 0x7f967c0cd540 2023-08-27T11:28:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.811+0000 7f96797fa700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f965c06a170 0x7f965c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.812+0000 7f96797fa700 1 -- 172.21.15.45:0/3359051068 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f9674089a00 con 0x7f967c0cd540 2023-08-27T11:28:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.812+0000 7f967bfff700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f965c06a170 0x7f965c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.812+0000 7f967bfff700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f965c06a170 0x7f965c06c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f967c15a000 tx=0x7f9664008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:05.817+0000 7f96797fa700 1 -- 172.21.15.45:0/3359051068 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f967c15d0a0 con 0x7f967c0cd540 2023-08-27T11:28:06.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.018+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f967c15eb70 con 0x7f967c0cd540 2023-08-27T11:28:06.472 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:06.470+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.658+0000 7f96797fa700 1 -- 172.21.15.45:0/3359051068 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) v1 ==== 170+0+0 (secure 0 0 0) 0x7f96740550f0 con 0x7f967c0cd540 2023-08-27T11:28:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f965c06a170 msgr2=0x7f965c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f965c06a170 0x7f965c06c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f967c15a000 tx=0x7f9664008040 comp rx=0 tx=0).stop 2023-08-27T11:28:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0cd540 msgr2=0x7f967c158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0cd540 0x7f967c158540 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f967400b370 tx=0x7f9674004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 shutdown_connections 2023-08-27T11:28:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f965c06a170 0x7f965c06c630 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f967c15a000 tx=0x7f9664008040 comp rx=0 tx=0).stop 2023-08-27T11:28:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f967c0cd540 0x7f967c158540 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f967c15b2e0 0x7f967c06f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 --2- 172.21.15.45:0/3359051068 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f967c0cf240 0x7f967c06ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.663+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 >> 172.21.15.45:0/3359051068 conn(0x7f967c09a1a0 msgr2=0x7f967c0c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.665+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 shutdown_connections 2023-08-27T11:28:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.668+0000 7f9683026700 1 -- 172.21.15.45:0/3359051068 wait complete. 2023-08-27T11:28:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-08-27T11:28:06.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:28:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.820+0000 7ff6ca99c700 1 Processor -- start 2023-08-27T11:28:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.820+0000 7ff6ca99c700 1 -- start start 2023-08-27T11:28:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.821+0000 7ff6ca99c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40b65d0 0x7ff6c40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.821+0000 7ff6ca99c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6c40cb5c0 0x7ff6c415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.821+0000 7ff6ca99c700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c415ec50 con 0x7ff6c40cfe10 2023-08-27T11:28:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.821+0000 7ff6ca99c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c415c620 con 0x7ff6c40cb5c0 2023-08-27T11:28:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.821+0000 7ff6ca99c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c415eea0 con 0x7ff6c40b65d0 2023-08-27T11:28:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.821+0000 7ff6c37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6c40cb5c0 0x7ff6c415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.821+0000 7ff6c37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6c40cb5c0 0x7ff6c415c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58994/0 (socket says 172.21.15.45:58994) 2023-08-27T11:28:06.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.821+0000 7ff6c37fe700 1 -- 172.21.15.45:0/3714735209 learned_addr learned my addr 172.21.15.45:0/3714735209 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:06.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.821+0000 7ff6c3fff700 1 --2- 172.21.15.45:0/3714735209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40b65d0 0x7ff6c40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6c37fe700 1 -- 172.21.15.45:0/3714735209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40b65d0 msgr2=0x7ff6c40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6c37fe700 1 --2- 172.21.15.45:0/3714735209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40b65d0 0x7ff6c40ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6c37fe700 1 -- 172.21.15.45:0/3714735209 >> v1:172.21.15.135:6789/0 conn(0x7ff6c40cfe10 legacy=0x7ff6c40cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6c2ffd700 1 -- 172.21.15.45:0/3714735209 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972038438 0 0) 0x7ff6c415ec50 con 0x7ff6c40cfe10 2023-08-27T11:28:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6c37fe700 1 -- 172.21.15.45:0/3714735209 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c415f180 con 0x7ff6c40cb5c0 2023-08-27T11:28:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6c37fe700 1 --2- 172.21.15.45:0/3714735209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40cb5c0 0x7ff6c415c0e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80098f0 tx=0x7ff6b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f044d32f2bf9e12d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6c2ffd700 1 -- 172.21.15.45:0/3714735209 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8011830 con 0x7ff6c40cb5c0 2023-08-27T11:28:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6c2ffd700 1 -- 172.21.15.45:0/3714735209 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b8011990 con 0x7ff6c40cb5c0 2023-08-27T11:28:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6c2ffd700 1 -- 172.21.15.45:0/3714735209 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8011d20 con 0x7ff6c40cb5c0 2023-08-27T11:28:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6ca99c700 1 -- 172.21.15.45:0/3714735209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40cb5c0 msgr2=0x7ff6c415c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6ca99c700 1 --2- 172.21.15.45:0/3714735209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40cb5c0 0x7ff6c415c0e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff6b80098f0 tx=0x7ff6b8009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:06.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6ca99c700 1 -- 172.21.15.45:0/3714735209 shutdown_connections 2023-08-27T11:28:06.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6ca99c700 1 --2- 172.21.15.45:0/3714735209 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40b65d0 0x7ff6c40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6ca99c700 1 --2- 172.21.15.45:0/3714735209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40cb5c0 0x7ff6c415c0e0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.822+0000 7ff6ca99c700 1 -- 172.21.15.45:0/3714735209 >> 172.21.15.45:0/3714735209 conn(0x7ff6c406f960 msgr2=0x7ff6c406fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:06.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.823+0000 7ff6ca99c700 1 -- 172.21.15.45:0/3714735209 shutdown_connections 2023-08-27T11:28:06.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.823+0000 7ff6ca99c700 1 -- 172.21.15.45:0/3714735209 wait complete. 2023-08-27T11:28:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.824+0000 7ff6ca99c700 1 Processor -- start 2023-08-27T11:28:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.824+0000 7ff6ca99c700 1 -- start start 2023-08-27T11:28:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6ca99c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40b65d0 0x7ff6c414bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6ca99c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40cb5c0 0x7ff6c40b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6ca99c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6c40cfe10 0x7ff6c406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6ca99c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40c9120 con 0x7ff6c40b65d0 2023-08-27T11:28:06.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6ca99c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40c96e0 con 0x7ff6c40cfe10 2023-08-27T11:28:06.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6ca99c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c414d380 con 0x7ff6c40cb5c0 2023-08-27T11:28:06.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40b65d0 0x7ff6c414bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40b65d0 0x7ff6c414bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58998/0 (socket says 172.21.15.45:58998) 2023-08-27T11:28:06.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c3fff700 1 -- 172.21.15.45:0/2526389612 learned_addr learned my addr 172.21.15.45:0/2526389612 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:06.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c37fe700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40cb5c0 0x7ff6c40b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c8f39700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6c40cfe10 0x7ff6c406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c3fff700 1 -- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40cb5c0 msgr2=0x7ff6c40b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c3fff700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40cb5c0 0x7ff6c40b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c3fff700 1 -- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6c40cfe10 msgr2=0x7ff6c406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c3fff700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6c40cfe10 0x7ff6c406c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:06.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c3fff700 1 -- 172.21.15.45:0/2526389612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8009550 con 0x7ff6c40b65d0 2023-08-27T11:28:06.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c8f39700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6c40cfe10 0x7ff6c406c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:06.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c3fff700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40b65d0 0x7ff6c414bf70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400e9b0 tx=0x7ff6b400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:06.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c0ff9700 1 -- 172.21.15.45:0/2526389612 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4009710 con 0x7ff6c40b65d0 2023-08-27T11:28:06.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c0ff9700 1 -- 172.21.15.45:0/2526389612 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b400c850 con 0x7ff6c40b65d0 2023-08-27T11:28:06.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c414c950 con 0x7ff6c40b65d0 2023-08-27T11:28:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6c0ff9700 1 -- 172.21.15.45:0/2526389612 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400cbc0 con 0x7ff6c40b65d0 2023-08-27T11:28:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.825+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c40b0e00 con 0x7ff6c40b65d0 2023-08-27T11:28:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.826+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c40c99c0 con 0x7ff6c40b65d0 2023-08-27T11:28:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.827+0000 7ff6c0ff9700 1 -- 172.21.15.45:0/2526389612 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7ff6b400cd20 con 0x7ff6c40b65d0 2023-08-27T11:28:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.828+0000 7ff6c0ff9700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff6b006a170 0x7ff6b006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.828+0000 7ff6c0ff9700 1 -- 172.21.15.45:0/2526389612 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6326+0+0 (secure 0 0 0) 0x7ff6b4088b00 con 0x7ff6c40b65d0 2023-08-27T11:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.828+0000 7ff6c37fe700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff6b006a170 0x7ff6b006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.828+0000 7ff6c37fe700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff6b006a170 0x7ff6b006c630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff6c415c260 tx=0x7ff6b8008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:06.833+0000 7ff6c0ff9700 1 -- 172.21.15.45:0/2526389612 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff6b4057b90 con 0x7ff6c40b65d0 2023-08-27T11:28:07.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:28:07.007 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:28:07.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.090+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff6c406f4a0 con 0x7ff6c40b65d0 2023-08-27T11:28:07.203 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:07.201+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:07.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.679+0000 7ff6c0ff9700 1 -- 172.21.15.45:0/2526389612 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff6b4057720 con 0x7ff6c40b65d0 2023-08-27T11:28:07.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.688+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff6b006a170 msgr2=0x7ff6b006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.688+0000 7ff6ca99c700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff6b006a170 0x7ff6b006c630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff6c415c260 tx=0x7ff6b8008030 comp rx=0 tx=0).stop 2023-08-27T11:28:07.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.688+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40b65d0 msgr2=0x7ff6c414bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.688+0000 7ff6ca99c700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40b65d0 0x7ff6c414bf70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400e9b0 tx=0x7ff6b400ed70 comp rx=0 tx=0).stop 2023-08-27T11:28:07.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.689+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 shutdown_connections 2023-08-27T11:28:07.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.689+0000 7ff6ca99c700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff6b006a170 0x7ff6b006c630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.689+0000 7ff6ca99c700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c40b65d0 0x7ff6c414bf70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.689+0000 7ff6ca99c700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6c40cfe10 0x7ff6c406c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.689+0000 7ff6ca99c700 1 --2- 172.21.15.45:0/2526389612 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff6c40cb5c0 0x7ff6c40b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.689+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 >> 172.21.15.45:0/2526389612 conn(0x7ff6c406f960 msgr2=0x7ff6c415d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:07.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.691+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 shutdown_connections 2023-08-27T11:28:07.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.691+0000 7ff6ca99c700 1 -- 172.21.15.45:0/2526389612 wait complete. 2023-08-27T11:28:07.694 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 12 and data pool 13 2023-08-27T11:28:07.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.870+0000 7f5d5a944700 1 Processor -- start 2023-08-27T11:28:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.870+0000 7f5d5a944700 1 -- start start 2023-08-27T11:28:07.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.870+0000 7f5d5a944700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 0x7f5d540cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.870+0000 7f5d5a944700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5d540cd9b0 0x7f5d540b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.870+0000 7f5d5a944700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d540ada50 con 0x7f5d5415b120 2023-08-27T11:28:07.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.870+0000 7f5d5a944700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540cde40 con 0x7f5d540cd9b0 2023-08-27T11:28:07.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.870+0000 7f5d5a944700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540ccb40 con 0x7f5d540d1a20 2023-08-27T11:28:07.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d58ee1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5d540cd9b0 0x7f5d540b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d58ee1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5d540cd9b0 0x7f5d540b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59024/0 (socket says 172.21.15.45:59024) 2023-08-27T11:28:07.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d58ee1700 1 -- 172.21.15.45:0/812527473 learned_addr learned my addr 172.21.15.45:0/812527473 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:07.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d537fe700 1 --2- 172.21.15.45:0/812527473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 0x7f5d540cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d58ee1700 1 -- 172.21.15.45:0/812527473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 msgr2=0x7f5d540cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:07.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d58ee1700 1 --2- 172.21.15.45:0/812527473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 0x7f5d540cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d58ee1700 1 -- 172.21.15.45:0/812527473 >> v1:172.21.15.135:6789/0 conn(0x7f5d5415b120 legacy=0x7f5d540cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d58ee1700 1 -- 172.21.15.45:0/812527473 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540ccd60 con 0x7f5d540cd9b0 2023-08-27T11:28:07.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d537fe700 1 --2- 172.21.15.45:0/812527473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 0x7f5d540cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:07.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d58ee1700 1 --2- 172.21.15.45:0/812527473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d540cd9b0 0x7f5d540b1620 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009e80 tx=0x7f5d4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e24bb8e4298e5bc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d52ffd700 1 -- 172.21.15.45:0/812527473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00e070 con 0x7f5d540cd9b0 2023-08-27T11:28:07.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d52ffd700 1 -- 172.21.15.45:0/812527473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c00b980 con 0x7f5d540cd9b0 2023-08-27T11:28:07.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.871+0000 7f5d52ffd700 1 -- 172.21.15.45:0/812527473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00bd30 con 0x7f5d540cd9b0 2023-08-27T11:28:07.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.872+0000 7f5d5a944700 1 -- 172.21.15.45:0/812527473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d540cd9b0 msgr2=0x7f5d540b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.872+0000 7f5d5a944700 1 --2- 172.21.15.45:0/812527473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d540cd9b0 0x7f5d540b1620 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009e80 tx=0x7f5d4c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:07.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.872+0000 7f5d5a944700 1 -- 172.21.15.45:0/812527473 shutdown_connections 2023-08-27T11:28:07.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.872+0000 7f5d5a944700 1 --2- 172.21.15.45:0/812527473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d540cd9b0 0x7f5d540b1620 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.872+0000 7f5d5a944700 1 --2- 172.21.15.45:0/812527473 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 0x7f5d540cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.872+0000 7f5d5a944700 1 -- 172.21.15.45:0/812527473 >> 172.21.15.45:0/812527473 conn(0x7f5d5409a160 msgr2=0x7f5d540d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:07.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.872+0000 7f5d5a944700 1 -- 172.21.15.45:0/812527473 shutdown_connections 2023-08-27T11:28:07.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.872+0000 7f5d5a944700 1 -- 172.21.15.45:0/812527473 wait complete. 2023-08-27T11:28:07.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.873+0000 7f5d5a944700 1 Processor -- start 2023-08-27T11:28:07.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.873+0000 7f5d5a944700 1 -- start start 2023-08-27T11:28:07.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d5a944700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d540cd9b0 0x7f5d540ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d5a944700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 0x7f5d540c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d5a944700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d5415b120 0x7f5d541f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d5a944700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541f6340 con 0x7f5d5415b120 2023-08-27T11:28:07.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d5a944700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540cad80 con 0x7f5d540cd9b0 2023-08-27T11:28:07.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d5a944700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c87a0 con 0x7f5d540d1a20 2023-08-27T11:28:07.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d58ee1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d5415b120 0x7f5d541f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d53fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d540cd9b0 0x7f5d540ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d58ee1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d5415b120 0x7f5d541f6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59030/0 (socket says 172.21.15.45:59030) 2023-08-27T11:28:07.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d58ee1700 1 -- 172.21.15.45:0/1014531276 learned_addr learned my addr 172.21.15.45:0/1014531276 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:07.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.874+0000 7f5d537fe700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 0x7f5d540c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d58ee1700 1 -- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 msgr2=0x7f5d540c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d58ee1700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 0x7f5d540c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d58ee1700 1 -- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d540cd9b0 msgr2=0x7f5d540ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:07.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d58ee1700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d540cd9b0 0x7f5d540ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:07.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d58ee1700 1 -- 172.21.15.45:0/1014531276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009b30 con 0x7f5d5415b120 2023-08-27T11:28:07.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d58ee1700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d5415b120 0x7f5d541f6760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5d540708a0 tx=0x7f5d4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d50ff9700 1 -- 172.21.15.45:0/1014531276 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c004800 con 0x7f5d5415b120 2023-08-27T11:28:07.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d50ff9700 1 -- 172.21.15.45:0/1014531276 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c004960 con 0x7f5d5415b120 2023-08-27T11:28:07.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d540c8a80 con 0x7f5d5415b120 2023-08-27T11:28:07.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d50ff9700 1 -- 172.21.15.45:0/1014531276 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c01c680 con 0x7f5d5415b120 2023-08-27T11:28:07.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.875+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540c8490 con 0x7f5d5415b120 2023-08-27T11:28:07.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.876+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5409edd0 con 0x7f5d5415b120 2023-08-27T11:28:07.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.877+0000 7f5d50ff9700 1 -- 172.21.15.45:0/1014531276 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f5d4c01c7e0 con 0x7f5d5415b120 2023-08-27T11:28:07.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.877+0000 7f5d50ff9700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f5d4006a170 0x7f5d4006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:07.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.877+0000 7f5d50ff9700 1 -- 172.21.15.45:0/1014531276 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f5d4c089c90 con 0x7f5d5415b120 2023-08-27T11:28:07.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.877+0000 7f5d53fff700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f5d4006a170 0x7f5d4006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:07.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.878+0000 7f5d53fff700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f5d4006a170 0x7f5d4006c630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d44007fd0 tx=0x7f5d44009040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:07.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:07.881+0000 7f5d50ff9700 1 -- 172.21.15.45:0/1014531276 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5d4c055110 con 0x7f5d5415b120 2023-08-27T11:28:07.936 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:07.934+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:08.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.089+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d54066650 con 0x7f5d5415b120 2023-08-27T11:28:08.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.091+0000 7f5d50ff9700 1 -- 172.21.15.45:0/1014531276 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+18642 (secure 0 0 0) 0x7f5d4c058730 con 0x7f5d5415b120 2023-08-27T11:28:08.093 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:08.093 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":99,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:07.092267+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:05.410178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:06.051204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f5d4006a170 msgr2=0x7f5d4006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f5d4006a170 0x7f5d4006c630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d44007fd0 tx=0x7f5d44009040 comp rx=0 tx=0).stop 2023-08-27T11:28:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d5415b120 msgr2=0x7f5d541f6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d5415b120 0x7f5d541f6760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5d540708a0 tx=0x7f5d4c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 shutdown_connections 2023-08-27T11:28:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f5d4006a170 0x7f5d4006c630 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d5415b120 0x7f5d541f6760 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d540cd9b0 0x7f5d540ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 --2- 172.21.15.45:0/1014531276 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d540d1a20 0x7f5d540c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 >> 172.21.15.45:0/1014531276 conn(0x7f5d5409a160 msgr2=0x7f5d5415a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 shutdown_connections 2023-08-27T11:28:08.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.093+0000 7f5d5a944700 1 -- 172.21.15.45:0/1014531276 wait complete. 2023-08-27T11:28:08.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:08.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.232+0000 7f96a0fd4700 1 Processor -- start 2023-08-27T11:28:08.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.232+0000 7f96a0fd4700 1 -- start start 2023-08-27T11:28:08.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f96a0fd4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f969c158cb0 0x7f969c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f96a0fd4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c0cc5c0 con 0x7f969c15be70 2023-08-27T11:28:08.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f96a0fd4700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c0cbd80 con 0x7f969c06ac60 2023-08-27T11:28:08.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f96a0fd4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c06b230 con 0x7f969c158cb0 2023-08-27T11:28:08.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f969a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f969c158cb0 0x7f969c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f969a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f969c158cb0 0x7f969c0ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59034/0 (socket says 172.21.15.45:59034) 2023-08-27T11:28:08.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f969ad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f969c15be70 0x7f969c06e210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45812/0 (socket says 172.21.15.45:45812) 2023-08-27T11:28:08.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f969a59c700 1 -- 172.21.15.45:0/4008951659 learned_addr learned my addr 172.21.15.45:0/4008951659 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:08.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f969a59c700 1 -- 172.21.15.45:0/4008951659 >> v1:172.21.15.135:6789/0 conn(0x7f969c06ac60 legacy=0x7f969c06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:08.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f969a59c700 1 -- 172.21.15.45:0/4008951659 >> v1:172.21.15.45:6789/0 conn(0x7f969c15be70 legacy=0x7f969c06e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f969a59c700 1 -- 172.21.15.45:0/4008951659 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c0ca8f0 con 0x7f969c158cb0 2023-08-27T11:28:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.233+0000 7f969959a700 1 -- 172.21.15.45:0/4008951659 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186586429 0 0) 0x7f969c0cc5c0 con 0x7f969c15be70 2023-08-27T11:28:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.234+0000 7f969a59c700 1 --2- 172.21.15.45:0/4008951659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c158cb0 0x7f969c0ca3b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f96900098f0 tx=0x7f9690009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9f7d0257158d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.234+0000 7f969959a700 1 -- 172.21.15.45:0/4008951659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690004230 con 0x7f969c158cb0 2023-08-27T11:28:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.234+0000 7f969959a700 1 -- 172.21.15.45:0/4008951659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9690004390 con 0x7f969c158cb0 2023-08-27T11:28:08.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.234+0000 7f969959a700 1 -- 172.21.15.45:0/4008951659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96900036a0 con 0x7f969c158cb0 2023-08-27T11:28:08.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.234+0000 7f96a0fd4700 1 -- 172.21.15.45:0/4008951659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c158cb0 msgr2=0x7f969c0ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.234+0000 7f96a0fd4700 1 --2- 172.21.15.45:0/4008951659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c158cb0 0x7f969c0ca3b0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f96900098f0 tx=0x7f9690009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:08.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.234+0000 7f96a0fd4700 1 -- 172.21.15.45:0/4008951659 shutdown_connections 2023-08-27T11:28:08.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.234+0000 7f96a0fd4700 1 --2- 172.21.15.45:0/4008951659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c158cb0 0x7f969c0ca3b0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.234+0000 7f96a0fd4700 1 -- 172.21.15.45:0/4008951659 >> 172.21.15.45:0/4008951659 conn(0x7f969c0cfd60 msgr2=0x7f969c0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.235+0000 7f96a0fd4700 1 -- 172.21.15.45:0/4008951659 shutdown_connections 2023-08-27T11:28:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.235+0000 7f96a0fd4700 1 -- 172.21.15.45:0/4008951659 wait complete. 2023-08-27T11:28:08.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.236+0000 7f96a0fd4700 1 Processor -- start 2023-08-27T11:28:08.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.236+0000 7f96a0fd4700 1 -- start start 2023-08-27T11:28:08.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f96a0fd4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969c06ac60 0x7f969c0c3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f96a0fd4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f969c158cb0 0x7f969c14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f96a0fd4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c15be70 0x7f969c1e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f96a0fd4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c3db0 con 0x7f969c15be70 2023-08-27T11:28:08.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f96a0fd4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c9590 con 0x7f969c06ac60 2023-08-27T11:28:08.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f96a0fd4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c9c30 con 0x7f969c158cb0 2023-08-27T11:28:08.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c15be70 0x7f969c1e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c15be70 0x7f969c1e53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59042/0 (socket says 172.21.15.45:59042) 2023-08-27T11:28:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969ad9d700 1 -- 172.21.15.45:0/1350612912 learned_addr learned my addr 172.21.15.45:0/1350612912 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f9699d9b700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f969c158cb0 0x7f969c14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969a59c700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969c06ac60 0x7f969c0c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969ad9d700 1 -- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f969c158cb0 msgr2=0x7f969c14cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969ad9d700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f969c158cb0 0x7f969c14cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969ad9d700 1 -- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969c06ac60 msgr2=0x7f969c0c3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969ad9d700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969c06ac60 0x7f969c0c3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969ad9d700 1 -- 172.21.15.45:0/1350612912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009550 con 0x7f969c15be70 2023-08-27T11:28:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.237+0000 7f969ad9d700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c15be70 0x7f969c1e53d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f969400cbd0 tx=0x7f969400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.238+0000 7f968affd700 1 -- 172.21.15.45:0/1350612912 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969400aa60 con 0x7f969c15be70 2023-08-27T11:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.238+0000 7f968affd700 1 -- 172.21.15.45:0/1350612912 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969400abe0 con 0x7f969c15be70 2023-08-27T11:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.238+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c0c81d0 con 0x7f969c15be70 2023-08-27T11:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.238+0000 7f968affd700 1 -- 172.21.15.45:0/1350612912 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96940070d0 con 0x7f969c15be70 2023-08-27T11:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.238+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c0c8560 con 0x7f969c15be70 2023-08-27T11:28:08.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.238+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c072780 con 0x7f969c15be70 2023-08-27T11:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.239+0000 7f968affd700 1 -- 172.21.15.45:0/1350612912 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f9694007230 con 0x7f969c15be70 2023-08-27T11:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.239+0000 7f968affd700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f967c06a1a0 0x7f967c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.240+0000 7f968affd700 1 -- 172.21.15.45:0/1350612912 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9694088440 con 0x7f969c15be70 2023-08-27T11:28:08.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.240+0000 7f969a59c700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f967c06a1a0 0x7f967c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.240+0000 7f969a59c700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f967c06a1a0 0x7f967c06c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9690009130 tx=0x7f9690008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.245+0000 7f968affd700 1 -- 172.21.15.45:0/1350612912 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9694053960 con 0x7f969c15be70 2023-08-27T11:28:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.450+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f969c15b2b0 con 0x7f969c15be70 2023-08-27T11:28:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.451+0000 7f968affd700 1 -- 172.21.15.45:0/1350612912 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+18642 (secure 0 0 0) 0x7f9694056f80 con 0x7f969c15be70 2023-08-27T11:28:08.453 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:08.454 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":99,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:07.092267+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:05.410178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:06.051204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f967c06a1a0 msgr2=0x7f967c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f967c06a1a0 0x7f967c06c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9690009130 tx=0x7f9690008030 comp rx=0 tx=0).stop 2023-08-27T11:28:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c15be70 msgr2=0x7f969c1e53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c15be70 0x7f969c1e53d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f969400cbd0 tx=0x7f969400cee0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 shutdown_connections 2023-08-27T11:28:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f967c06a1a0 0x7f967c06c660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969c15be70 0x7f969c1e53d0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969c06ac60 0x7f969c0c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 --2- 172.21.15.45:0/1350612912 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f969c158cb0 0x7f969c14cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.453+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 >> 172.21.15.45:0/1350612912 conn(0x7f969c0cfd60 msgr2=0x7f969c15a200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:08.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.454+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 shutdown_connections 2023-08-27T11:28:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.454+0000 7f96a0fd4700 1 -- 172.21.15.45:0/1350612912 wait complete. 2023-08-27T11:28:08.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:28:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.598+0000 7fa97f87c700 1 Processor -- start 2023-08-27T11:28:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.598+0000 7fa97f87c700 1 -- start start 2023-08-27T11:28:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97f87c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa978157be0 0x7fa9780716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97f87c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 0x7fa9780b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97f87c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa97806ced0 0x7fa97806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97f87c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa978157fc0 con 0x7fa978157be0 2023-08-27T11:28:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97f87c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97806d860 con 0x7fa97806ced0 2023-08-27T11:28:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97f87c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97815b460 con 0x7fa9780b17c0 2023-08-27T11:28:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97d618700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa978157be0 0x7fa9780716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97d618700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa978157be0 0x7fa9780716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59046/0 (socket says 172.21.15.45:59046) 2023-08-27T11:28:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97d618700 1 -- 172.21.15.45:0/4026666517 learned_addr learned my addr 172.21.15.45:0/4026666517 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97de19700 1 --2- 172.21.15.45:0/4026666517 >> v2:172.21.15.135:3300/0 conn(0x7fa97806ced0 0x7fa97806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97ce17700 1 --2- 172.21.15.45:0/4026666517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 0x7fa9780b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97d618700 1 -- 172.21.15.45:0/4026666517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 msgr2=0x7fa9780b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97d618700 1 --2- 172.21.15.45:0/4026666517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 0x7fa9780b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97d618700 1 -- 172.21.15.45:0/4026666517 >> v2:172.21.15.135:3300/0 conn(0x7fa97806ced0 msgr2=0x7fa97806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97d618700 1 --2- 172.21.15.45:0/4026666517 >> v2:172.21.15.135:3300/0 conn(0x7fa97806ced0 0x7fa97806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97d618700 1 -- 172.21.15.45:0/4026666517 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780b2280 con 0x7fa978157be0 2023-08-27T11:28:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.599+0000 7fa97d618700 1 --2- 172.21.15.45:0/4026666517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 0x7fa9780716b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa9780cd3f0 tx=0x7fa968009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1c88a6d82ebfe7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa96ffff700 1 -- 172.21.15.45:0/4026666517 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968011830 con 0x7fa978157be0 2023-08-27T11:28:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa96ffff700 1 -- 172.21.15.45:0/4026666517 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa968011990 con 0x7fa978157be0 2023-08-27T11:28:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa96ffff700 1 -- 172.21.15.45:0/4026666517 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968011d20 con 0x7fa978157be0 2023-08-27T11:28:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa97f87c700 1 -- 172.21.15.45:0/4026666517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 msgr2=0x7fa9780716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa97f87c700 1 --2- 172.21.15.45:0/4026666517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 0x7fa9780716b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa9780cd3f0 tx=0x7fa968009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa97f87c700 1 -- 172.21.15.45:0/4026666517 shutdown_connections 2023-08-27T11:28:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa97f87c700 1 --2- 172.21.15.45:0/4026666517 >> v2:172.21.15.135:3300/0 conn(0x7fa97806ced0 0x7fa97806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa97f87c700 1 --2- 172.21.15.45:0/4026666517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 0x7fa9780716b0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa97f87c700 1 --2- 172.21.15.45:0/4026666517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 0x7fa9780b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa97f87c700 1 -- 172.21.15.45:0/4026666517 >> 172.21.15.45:0/4026666517 conn(0x7fa9780cc7c0 msgr2=0x7fa9780ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa97f87c700 1 -- 172.21.15.45:0/4026666517 shutdown_connections 2023-08-27T11:28:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.600+0000 7fa97f87c700 1 -- 172.21.15.45:0/4026666517 wait complete. 2023-08-27T11:28:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.602+0000 7fa97f87c700 1 Processor -- start 2023-08-27T11:28:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.602+0000 7fa97f87c700 1 -- start start 2023-08-27T11:28:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.602+0000 7fa97f87c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa97806ced0 0x7fa9780c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.602+0000 7fa97f87c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 0x7fa97815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.602+0000 7fa97f87c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 0x7fa978159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.602+0000 7fa97f87c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780b3460 con 0x7fa978157be0 2023-08-27T11:28:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.602+0000 7fa97f87c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780b2b90 con 0x7fa97806ced0 2023-08-27T11:28:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.602+0000 7fa97f87c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa978159fb0 con 0x7fa9780b17c0 2023-08-27T11:28:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97de19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 0x7fa978159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97de19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 0x7fa978159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59056/0 (socket says 172.21.15.45:59056) 2023-08-27T11:28:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97d618700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa97806ced0 0x7fa9780c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97ce17700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 0x7fa97815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97de19700 1 -- 172.21.15.45:0/1370263570 learned_addr learned my addr 172.21.15.45:0/1370263570 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97de19700 1 -- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 msgr2=0x7fa97815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97de19700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 0x7fa97815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97de19700 1 -- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa97806ced0 msgr2=0x7fa9780c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97de19700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa97806ced0 0x7fa9780c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97de19700 1 -- 172.21.15.45:0/1370263570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa968009550 con 0x7fa978157be0 2023-08-27T11:28:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97d618700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa97806ced0 0x7fa9780c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97de19700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 0x7fa978159a70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa97000eed0 tx=0x7fa97000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa96dffb700 1 -- 172.21.15.45:0/1370263570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970010040 con 0x7fa978157be0 2023-08-27T11:28:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa96dffb700 1 -- 172.21.15.45:0/1370263570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97000bc80 con 0x7fa978157be0 2023-08-27T11:28:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa96dffb700 1 -- 172.21.15.45:0/1370263570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970009e40 con 0x7fa978157be0 2023-08-27T11:28:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa978151aa0 con 0x7fa978157be0 2023-08-27T11:28:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.603+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9780b2da0 con 0x7fa978157be0 2023-08-27T11:28:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.604+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9780b35a0 con 0x7fa978157be0 2023-08-27T11:28:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.605+0000 7fa96dffb700 1 -- 172.21.15.45:0/1370263570 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fa970026420 con 0x7fa978157be0 2023-08-27T11:28:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.606+0000 7fa96dffb700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa95c06a170 0x7fa95c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.606+0000 7fa96dffb700 1 -- 172.21.15.45:0/1370263570 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fa970088f70 con 0x7fa978157be0 2023-08-27T11:28:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.606+0000 7fa97d618700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa95c06a170 0x7fa95c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.606+0000 7fa97d618700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa95c06a170 0x7fa95c06c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa968014040 tx=0x7fa968008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.610+0000 7fa96dffb700 1 -- 172.21.15.45:0/1370263570 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa970054470 con 0x7fa978157be0 2023-08-27T11:28:08.657 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:08.655+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:08.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:08.873+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa978151aa0 con 0x7fa978157be0 2023-08-27T11:28:09.386 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:09.385+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:09.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.679+0000 7fa96dffb700 1 -- 172.21.15.45:0/1370263570 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa978151aa0 con 0x7fa978157be0 2023-08-27T11:28:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa95c06a170 msgr2=0x7fa95c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa95c06a170 0x7fa95c06c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa968014040 tx=0x7fa968008030 comp rx=0 tx=0).stop 2023-08-27T11:28:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 msgr2=0x7fa978159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 0x7fa978159a70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa97000eed0 tx=0x7fa97000b390 comp rx=0 tx=0).stop 2023-08-27T11:28:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 shutdown_connections 2023-08-27T11:28:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa95c06a170 0x7fa95c06c630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa978157be0 0x7fa978159a70 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:09.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa97806ced0 0x7fa9780c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:09.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 --2- 172.21.15.45:0/1370263570 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa9780b17c0 0x7fa97815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:09.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 >> 172.21.15.45:0/1370263570 conn(0x7fa9780cc7c0 msgr2=0x7fa9780ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:09.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 shutdown_connections 2023-08-27T11:28:09.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.681+0000 7fa97f87c700 1 -- 172.21.15.45:0/1370263570 wait complete. 2023-08-27T11:28:09.697 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:28:09.697 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:28:09.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.824+0000 7fdec7733700 1 Processor -- start 2023-08-27T11:28:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.825+0000 7fdec7733700 1 -- start start 2023-08-27T11:28:09.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.825+0000 7fdec7733700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdec015a0f0 0x7fdec00cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:09.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.825+0000 7fdec7733700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdec00cd300 0x7fdec015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:09.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.825+0000 7fdec7733700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec00cfae0 con 0x7fdec015e030 2023-08-27T11:28:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.825+0000 7fdec7733700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec015e900 con 0x7fdec015a0f0 2023-08-27T11:28:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.825+0000 7fdec7733700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec00cae60 con 0x7fdec00cd300 2023-08-27T11:28:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdec54cf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdec015a0f0 0x7fdec00cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdec5cd0700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdec015e030 0x7fdec00cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45840/0 (socket says 172.21.15.45:45840) 2023-08-27T11:28:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdec5cd0700 1 -- 172.21.15.45:0/1329595295 learned_addr learned my addr 172.21.15.45:0/1329595295 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:09.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdec4cce700 1 --2- 172.21.15.45:0/1329595295 >> v2:172.21.15.135:3300/0 conn(0x7fdec00cd300 0x7fdec015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:09.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdec54cf700 1 -- 172.21.15.45:0/1329595295 >> v2:172.21.15.135:3300/0 conn(0x7fdec00cd300 msgr2=0x7fdec015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:09.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdec54cf700 1 --2- 172.21.15.45:0/1329595295 >> v2:172.21.15.135:3300/0 conn(0x7fdec00cd300 0x7fdec015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:09.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdec54cf700 1 -- 172.21.15.45:0/1329595295 >> v1:172.21.15.45:6789/0 conn(0x7fdec015e030 legacy=0x7fdec00cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:09.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdec54cf700 1 -- 172.21.15.45:0/1329595295 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec00cb140 con 0x7fdec015a0f0 2023-08-27T11:28:09.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdeb7fff700 1 -- 172.21.15.45:0/1329595295 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532585908 0 0) 0x7fdec00cfae0 con 0x7fdec015e030 2023-08-27T11:28:09.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdec54cf700 1 --2- 172.21.15.45:0/1329595295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec015a0f0 0x7fdec00cbc20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fdec00cf9e0 tx=0x7fdebc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e87021d691d1c96 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:09.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdeb7fff700 1 -- 172.21.15.45:0/1329595295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc011830 con 0x7fdec015a0f0 2023-08-27T11:28:09.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdeb7fff700 1 -- 172.21.15.45:0/1329595295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc011990 con 0x7fdec015a0f0 2023-08-27T11:28:09.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.826+0000 7fdeb7fff700 1 -- 172.21.15.45:0/1329595295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc011d20 con 0x7fdec015a0f0 2023-08-27T11:28:09.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.827+0000 7fdec7733700 1 -- 172.21.15.45:0/1329595295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec015a0f0 msgr2=0x7fdec00cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:09.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.827+0000 7fdec7733700 1 --2- 172.21.15.45:0/1329595295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec015a0f0 0x7fdec00cbc20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fdec00cf9e0 tx=0x7fdebc009b20 comp rx=0 tx=0).stop 2023-08-27T11:28:09.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.827+0000 7fdec7733700 1 -- 172.21.15.45:0/1329595295 shutdown_connections 2023-08-27T11:28:09.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.827+0000 7fdec7733700 1 --2- 172.21.15.45:0/1329595295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec015a0f0 0x7fdec00cbc20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:09.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.827+0000 7fdec7733700 1 --2- 172.21.15.45:0/1329595295 >> v2:172.21.15.135:3300/0 conn(0x7fdec00cd300 0x7fdec015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:09.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.827+0000 7fdec7733700 1 -- 172.21.15.45:0/1329595295 >> 172.21.15.45:0/1329595295 conn(0x7fdec00cc7e0 msgr2=0x7fdec00ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:09.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.827+0000 7fdec7733700 1 -- 172.21.15.45:0/1329595295 shutdown_connections 2023-08-27T11:28:09.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.827+0000 7fdec7733700 1 -- 172.21.15.45:0/1329595295 wait complete. 2023-08-27T11:28:09.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.828+0000 7fdec7733700 1 Processor -- start 2023-08-27T11:28:09.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.828+0000 7fdec7733700 1 -- start start 2023-08-27T11:28:09.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec7733700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec00cd300 0x7fdec01f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:09.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec7733700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdec015a0f0 0x7fdec00c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:09.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec7733700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec015e030 0x7fdec00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:09.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec7733700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00c6080 con 0x7fdec00cd300 2023-08-27T11:28:09.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec7733700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec01f6fc0 con 0x7fdec015e030 2023-08-27T11:28:09.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec7733700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00c7fb0 con 0x7fdec015a0f0 2023-08-27T11:28:09.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec54cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec00cd300 0x7fdec01f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:09.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec54cf700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec00cd300 0x7fdec01f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59066/0 (socket says 172.21.15.45:59066) 2023-08-27T11:28:09.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec54cf700 1 -- 172.21.15.45:0/1415809 learned_addr learned my addr 172.21.15.45:0/1415809 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:09.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec4cce700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdec015a0f0 0x7fdec00c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:09.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec5cd0700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec015e030 0x7fdec00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:09.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec54cf700 1 -- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdec015a0f0 msgr2=0x7fdec00c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec54cf700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdec015a0f0 0x7fdec00c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec54cf700 1 -- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec015e030 msgr2=0x7fdec00b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec54cf700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec015e030 0x7fdec00b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec54cf700 1 -- 172.21.15.45:0/1415809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0094a0 con 0x7fdec00cd300 2023-08-27T11:28:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec4cce700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdec015a0f0 0x7fdec00c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.829+0000 7fdec54cf700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec00cd300 0x7fdec01f6a80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0090b0 tx=0x7fdebc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.830+0000 7fdeb5ffb700 1 -- 172.21.15.45:0/1415809 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0038a0 con 0x7fdec00cd300 2023-08-27T11:28:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.830+0000 7fdeb5ffb700 1 -- 172.21.15.45:0/1415809 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc01e070 con 0x7fdec00cd300 2023-08-27T11:28:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.830+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec00c73a0 con 0x7fdec00cd300 2023-08-27T11:28:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.830+0000 7fdeb5ffb700 1 -- 172.21.15.45:0/1415809 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc003c20 con 0x7fdec00cd300 2023-08-27T11:28:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.830+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec00c7960 con 0x7fdec00cd300 2023-08-27T11:28:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.831+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec00c8290 con 0x7fdec00cd300 2023-08-27T11:28:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.832+0000 7fdeb5ffb700 1 -- 172.21.15.45:0/1415809 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fdebc01a420 con 0x7fdec00cd300 2023-08-27T11:28:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.832+0000 7fdeb5ffb700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdea406a170 0x7fdea406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.832+0000 7fdeb5ffb700 1 -- 172.21.15.45:0/1415809 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fdebc089700 con 0x7fdec00cd300 2023-08-27T11:28:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.832+0000 7fdec4cce700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdea406a170 0x7fdea406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.833+0000 7fdec4cce700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdea406a170 0x7fdea406c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdec01482a0 tx=0x7fdeb0008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:09.837+0000 7fdeb5ffb700 1 -- 172.21.15.45:0/1415809 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdebc054c90 con 0x7fdec00cd300 2023-08-27T11:28:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.092+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdec00726d0 con 0x7fdec00cd300 2023-08-27T11:28:10.104 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:10.102+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.680+0000 7fdeb5ffb700 1 -- 172.21.15.45:0/1415809 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdebc0582b0 con 0x7fdec00cd300 2023-08-27T11:28:10.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdea406a170 msgr2=0x7fdea406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdea406a170 0x7fdea406c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdec01482a0 tx=0x7fdeb0008040 comp rx=0 tx=0).stop 2023-08-27T11:28:10.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec00cd300 msgr2=0x7fdec01f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec00cd300 0x7fdec01f6a80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0090b0 tx=0x7fdebc004750 comp rx=0 tx=0).stop 2023-08-27T11:28:10.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 shutdown_connections 2023-08-27T11:28:10.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fdea406a170 0x7fdea406c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec00cd300 0x7fdec01f6a80 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdec015e030 0x7fdec00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 --2- 172.21.15.45:0/1415809 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fdec015a0f0 0x7fdec00c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 >> 172.21.15.45:0/1415809 conn(0x7fdec00cc7e0 msgr2=0x7fdec00ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 shutdown_connections 2023-08-27T11:28:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.682+0000 7fdec7733700 1 -- 172.21.15.45:0/1415809 wait complete. 2023-08-27T11:28:10.696 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:28:10.697 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.825+0000 7fbe3be9e700 1 Processor -- start 2023-08-27T11:28:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.826+0000 7fbe3be9e700 1 -- start start 2023-08-27T11:28:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.826+0000 7fbe3be9e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe3406cec0 0x7fbe3406e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.826+0000 7fbe3be9e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbe3406e7b0 0x7fbe340ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.826+0000 7fbe3be9e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3406ebe0 con 0x7fbe3415b070 2023-08-27T11:28:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe3be9e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3415d8e0 con 0x7fbe3406cec0 2023-08-27T11:28:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe3be9e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3415dad0 con 0x7fbe3406e7b0 2023-08-27T11:28:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe39439700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe3406cec0 0x7fbe3406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe39439700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe3406cec0 0x7fbe3406e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59076/0 (socket says 172.21.15.45:59076) 2023-08-27T11:28:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe39439700 1 -- 172.21.15.45:0/1791330310 learned_addr learned my addr 172.21.15.45:0/1791330310 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe3a43b700 1 --2- 172.21.15.45:0/1791330310 >> v2:172.21.15.135:3300/0 conn(0x7fbe3406e7b0 0x7fbe340ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe39439700 1 -- 172.21.15.45:0/1791330310 >> v2:172.21.15.135:3300/0 conn(0x7fbe3406e7b0 msgr2=0x7fbe340ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe39439700 1 --2- 172.21.15.45:0/1791330310 >> v2:172.21.15.135:3300/0 conn(0x7fbe3406e7b0 0x7fbe340ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe39439700 1 -- 172.21.15.45:0/1791330310 >> v1:172.21.15.135:6789/0 conn(0x7fbe3415b070 legacy=0x7fbe340cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe39439700 1 -- 172.21.15.45:0/1791330310 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3415ddb0 con 0x7fbe3406cec0 2023-08-27T11:28:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe39439700 1 --2- 172.21.15.45:0/1791330310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406cec0 0x7fbe3406e270 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fbe340677c0 tx=0x7fbe28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4e73a7f3de1de7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe38c38700 1 -- 172.21.15.45:0/1791330310 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28011830 con 0x7fbe3406cec0 2023-08-27T11:28:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe38c38700 1 -- 172.21.15.45:0/1791330310 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28011990 con 0x7fbe3406cec0 2023-08-27T11:28:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.827+0000 7fbe38c38700 1 -- 172.21.15.45:0/1791330310 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28011d20 con 0x7fbe3406cec0 2023-08-27T11:28:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.828+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1791330310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406cec0 msgr2=0x7fbe3406e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.837 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.828+0000 7fbe3be9e700 1 --2- 172.21.15.45:0/1791330310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406cec0 0x7fbe3406e270 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fbe340677c0 tx=0x7fbe28009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.828+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1791330310 shutdown_connections 2023-08-27T11:28:10.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.828+0000 7fbe3be9e700 1 --2- 172.21.15.45:0/1791330310 >> v2:172.21.15.135:3300/0 conn(0x7fbe3406e7b0 0x7fbe340ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.828+0000 7fbe3be9e700 1 --2- 172.21.15.45:0/1791330310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406cec0 0x7fbe3406e270 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.828+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1791330310 >> 172.21.15.45:0/1791330310 conn(0x7fbe340cfe50 msgr2=0x7fbe340d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:10.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.828+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1791330310 shutdown_connections 2023-08-27T11:28:10.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.828+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1791330310 wait complete. 2023-08-27T11:28:10.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.829+0000 7fbe3be9e700 1 Processor -- start 2023-08-27T11:28:10.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.829+0000 7fbe3be9e700 1 -- start start 2023-08-27T11:28:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe3be9e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe3406cec0 0x7fbe340b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe3be9e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406e7b0 0x7fbe3414e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe3be9e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe3415b070 0x7fbe34160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe3be9e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340b29d0 con 0x7fbe3406e7b0 2023-08-27T11:28:10.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe3be9e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340b3cc0 con 0x7fbe3415b070 2023-08-27T11:28:10.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe3be9e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340b2c20 con 0x7fbe3406cec0 2023-08-27T11:28:10.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe39439700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406e7b0 0x7fbe3414e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe39439700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406e7b0 0x7fbe3414e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59082/0 (socket says 172.21.15.45:59082) 2023-08-27T11:28:10.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe39439700 1 -- 172.21.15.45:0/1050603830 learned_addr learned my addr 172.21.15.45:0/1050603830 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:10.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe3a43b700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe3415b070 0x7fbe34160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe39c3a700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe3406cec0 0x7fbe340b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe39439700 1 -- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe3406cec0 msgr2=0x7fbe340b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe39439700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe3406cec0 0x7fbe340b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe39439700 1 -- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe3415b070 msgr2=0x7fbe34160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:10.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe39439700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe3415b070 0x7fbe34160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:10.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.830+0000 7fbe39439700 1 -- 172.21.15.45:0/1050603830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28009550 con 0x7fbe3406e7b0 2023-08-27T11:28:10.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.831+0000 7fbe39439700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406e7b0 0x7fbe3414e5f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009c10 tx=0x7fbe28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.831+0000 7fbe267fc700 1 -- 172.21.15.45:0/1050603830 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280036a0 con 0x7fbe3406e7b0 2023-08-27T11:28:10.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.831+0000 7fbe267fc700 1 -- 172.21.15.45:0/1050603830 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2801e070 con 0x7fbe3406e7b0 2023-08-27T11:28:10.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.831+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe340b2f00 con 0x7fbe3406e7b0 2023-08-27T11:28:10.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.831+0000 7fbe267fc700 1 -- 172.21.15.45:0/1050603830 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28003a40 con 0x7fbe3406e7b0 2023-08-27T11:28:10.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.831+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe340b34c0 con 0x7fbe3406e7b0 2023-08-27T11:28:10.847 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:10.837+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:10.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.832+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe34072810 con 0x7fbe3406e7b0 2023-08-27T11:28:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.832+0000 7fbe267fc700 1 -- 172.21.15.45:0/1050603830 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fbe28003ba0 con 0x7fbe3406e7b0 2023-08-27T11:28:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.833+0000 7fbe267fc700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbe1806a170 0x7fbe1806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.833+0000 7fbe267fc700 1 -- 172.21.15.45:0/1050603830 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fbe280895b0 con 0x7fbe3406e7b0 2023-08-27T11:28:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.833+0000 7fbe39c3a700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbe1806a170 0x7fbe1806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.833+0000 7fbe39c3a700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbe1806a170 0x7fbe1806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe30000f60 tx=0x7fbe30009040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:10.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:10.837+0000 7fbe267fc700 1 -- 172.21.15.45:0/1050603830 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbe28054bd0 con 0x7fbe3406e7b0 2023-08-27T11:28:11.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.089+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe340734a0 con 0x7fbe3406e7b0 2023-08-27T11:28:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.089+0000 7fbe267fc700 1 -- 172.21.15.45:0/1050603830 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fbe280581f0 con 0x7fbe3406e7b0 2023-08-27T11:28:11.092 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:11.092 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3604468746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3604468746},{"type":"v1","addr":"172.21.15.135:6836","nonce":3604468746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1319491874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1319491874},{"type":"v1","addr":"172.21.15.135:6837","nonce":1319491874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6937,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2036943131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2036943131},{"type":"v1","addr":"172.21.15.45:6835","nonce":2036943131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:07.092667+0000","modified":"2023-08-27T11:28:10.679683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7446":{"gid":7446,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/3257398445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3257398445},{"type":"v1","addr":"172.21.15.45:6833","nonce":3257398445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:28:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbe1806a170 msgr2=0x7fbe1806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbe1806a170 0x7fbe1806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbe30000f60 tx=0x7fbe30009040 comp rx=0 tx=0).stop 2023-08-27T11:28:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406e7b0 msgr2=0x7fbe3414e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406e7b0 0x7fbe3414e5f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009c10 tx=0x7fbe28004750 comp rx=0 tx=0).stop 2023-08-27T11:28:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 shutdown_connections 2023-08-27T11:28:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fbe1806a170 0x7fbe1806c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe3415b070 0x7fbe34160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbe3406cec0 0x7fbe340b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 --2- 172.21.15.45:0/1050603830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe3406e7b0 0x7fbe3414e5f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 >> 172.21.15.45:0/1050603830 conn(0x7fbe340cfe50 msgr2=0x7fbe34160590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 shutdown_connections 2023-08-27T11:28:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.091+0000 7fbe3be9e700 1 -- 172.21.15.45:0/1050603830 wait complete. 2023-08-27T11:28:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 41 2023-08-27T11:28:11.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.232+0000 7ff037c78700 1 Processor -- start 2023-08-27T11:28:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.232+0000 7ff037c78700 1 -- start start 2023-08-27T11:28:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.233+0000 7ff037c78700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff03015b120 0x7ff0300cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.233+0000 7ff037c78700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff0300cdb80 0x7ff0300b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.233+0000 7ff037c78700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0300ada50 con 0x7ff0300d1a20 2023-08-27T11:28:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.233+0000 7ff037c78700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0300ccd60 con 0x7ff03015b120 2023-08-27T11:28:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.233+0000 7ff037c78700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0300b0860 con 0x7ff0300cdb80 2023-08-27T11:28:11.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.233+0000 7ff035a14700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff03015b120 0x7ff0300cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.233+0000 7ff035a14700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff03015b120 0x7ff0300cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59088/0 (socket says 172.21.15.45:59088) 2023-08-27T11:28:11.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.233+0000 7ff035a14700 1 -- 172.21.15.45:0/2843712268 learned_addr learned my addr 172.21.15.45:0/2843712268 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:11.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.233+0000 7ff036215700 1 --2- 172.21.15.45:0/2843712268 >> v2:172.21.15.135:3300/0 conn(0x7ff0300cdb80 0x7ff0300b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.234+0000 7ff035a14700 1 -- 172.21.15.45:0/2843712268 >> v2:172.21.15.135:3300/0 conn(0x7ff0300cdb80 msgr2=0x7ff0300b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:11.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.234+0000 7ff035a14700 1 --2- 172.21.15.45:0/2843712268 >> v2:172.21.15.135:3300/0 conn(0x7ff0300cdb80 0x7ff0300b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.234+0000 7ff035a14700 1 -- 172.21.15.45:0/2843712268 >> v1:172.21.15.135:6789/0 conn(0x7ff0300d1a20 legacy=0x7ff0300cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.234+0000 7ff035a14700 1 -- 172.21.15.45:0/2843712268 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0300b0b40 con 0x7ff03015b120 2023-08-27T11:28:11.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.234+0000 7ff035a14700 1 --2- 172.21.15.45:0/2843712268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03015b120 0x7ff0300cc430 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff03009a870 tx=0x7ff020009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba88ebfdfc840c5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.234+0000 7ff034a12700 1 -- 172.21.15.45:0/2843712268 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020011830 con 0x7ff03015b120 2023-08-27T11:28:11.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.234+0000 7ff034a12700 1 -- 172.21.15.45:0/2843712268 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff020011990 con 0x7ff03015b120 2023-08-27T11:28:11.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.234+0000 7ff034a12700 1 -- 172.21.15.45:0/2843712268 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020011d20 con 0x7ff03015b120 2023-08-27T11:28:11.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.235+0000 7ff037c78700 1 -- 172.21.15.45:0/2843712268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03015b120 msgr2=0x7ff0300cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.235+0000 7ff037c78700 1 --2- 172.21.15.45:0/2843712268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03015b120 0x7ff0300cc430 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff03009a870 tx=0x7ff020009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.235+0000 7ff037c78700 1 -- 172.21.15.45:0/2843712268 shutdown_connections 2023-08-27T11:28:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.235+0000 7ff037c78700 1 --2- 172.21.15.45:0/2843712268 >> v2:172.21.15.135:3300/0 conn(0x7ff0300cdb80 0x7ff0300b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.235+0000 7ff037c78700 1 --2- 172.21.15.45:0/2843712268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff03015b120 0x7ff0300cc430 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.235+0000 7ff037c78700 1 -- 172.21.15.45:0/2843712268 >> 172.21.15.45:0/2843712268 conn(0x7ff03009a160 msgr2=0x7ff0300d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.235+0000 7ff037c78700 1 -- 172.21.15.45:0/2843712268 shutdown_connections 2023-08-27T11:28:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.235+0000 7ff037c78700 1 -- 172.21.15.45:0/2843712268 wait complete. 2023-08-27T11:28:11.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.236+0000 7ff037c78700 1 Processor -- start 2023-08-27T11:28:11.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.236+0000 7ff037c78700 1 -- start start 2023-08-27T11:28:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff037c78700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0300cdb80 0x7ff030158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff037c78700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0300d1a20 0x7ff03015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff037c78700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff03015b120 0x7ff0300b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff037c78700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03015de80 con 0x7ff0300d1a20 2023-08-27T11:28:11.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff037c78700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03009b070 con 0x7ff03015b120 2023-08-27T11:28:11.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff037c78700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff030159320 con 0x7ff0300cdb80 2023-08-27T11:28:11.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035213700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0300d1a20 0x7ff03015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035213700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0300d1a20 0x7ff03015cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59092/0 (socket says 172.21.15.45:59092) 2023-08-27T11:28:11.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035213700 1 -- 172.21.15.45:0/483812413 learned_addr learned my addr 172.21.15.45:0/483812413 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:11.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035a14700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0300cdb80 0x7ff030158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff036215700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff03015b120 0x7ff0300b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035213700 1 -- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0300cdb80 msgr2=0x7ff030158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035213700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0300cdb80 0x7ff030158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035213700 1 -- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff03015b120 msgr2=0x7ff0300b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035213700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff03015b120 0x7ff0300b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035213700 1 -- 172.21.15.45:0/483812413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff020009550 con 0x7ff0300d1a20 2023-08-27T11:28:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.237+0000 7ff035213700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0300d1a20 0x7ff03015cf60 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00c540 tx=0x7ff02c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.238+0000 7ff0267fc700 1 -- 172.21.15.45:0/483812413 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0052d0 con 0x7ff0300d1a20 2023-08-27T11:28:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.238+0000 7ff0267fc700 1 -- 172.21.15.45:0/483812413 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c005430 con 0x7ff0300d1a20 2023-08-27T11:28:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.238+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03015d8c0 con 0x7ff0300d1a20 2023-08-27T11:28:11.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.238+0000 7ff0267fc700 1 -- 172.21.15.45:0/483812413 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0038c0 con 0x7ff0300d1a20 2023-08-27T11:28:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.238+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03009ad00 con 0x7ff0300d1a20 2023-08-27T11:28:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.238+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff03009edd0 con 0x7ff0300d1a20 2023-08-27T11:28:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.239+0000 7ff0267fc700 1 -- 172.21.15.45:0/483812413 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7ff02c003a20 con 0x7ff0300d1a20 2023-08-27T11:28:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.239+0000 7ff0267fc700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff01406a170 0x7ff01406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.240+0000 7ff0267fc700 1 -- 172.21.15.45:0/483812413 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff02c087a30 con 0x7ff0300d1a20 2023-08-27T11:28:11.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.240+0000 7ff035a14700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff01406a170 0x7ff01406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.240+0000 7ff035a14700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff01406a170 0x7ff01406c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff020014040 tx=0x7ff020008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.244+0000 7ff0267fc700 1 -- 172.21.15.45:0/483812413 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff02c053050 con 0x7ff0300d1a20 2023-08-27T11:28:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.455+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff03015e030 con 0x7ff0300d1a20 2023-08-27T11:28:11.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.456+0000 7ff0267fc700 1 -- 172.21.15.45:0/483812413 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+18631 (secure 0 0 0) 0x7ff02c056670 con 0x7ff0300d1a20 2023-08-27T11:28:11.459 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:11.459 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:08.659551+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:05.410178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:06.051204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:11.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff01406a170 msgr2=0x7ff01406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff01406a170 0x7ff01406c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff020014040 tx=0x7ff020008030 comp rx=0 tx=0).stop 2023-08-27T11:28:11.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0300d1a20 msgr2=0x7ff03015cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.461 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0300d1a20 0x7ff03015cf60 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00c540 tx=0x7ff02c00c850 comp rx=0 tx=0).stop 2023-08-27T11:28:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 shutdown_connections 2023-08-27T11:28:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff01406a170 0x7ff01406c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff03015b120 0x7ff0300b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff0300cdb80 0x7ff030158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 --2- 172.21.15.45:0/483812413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0300d1a20 0x7ff03015cf60 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 >> 172.21.15.45:0/483812413 conn(0x7ff03009a160 msgr2=0x7ff0300afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 shutdown_connections 2023-08-27T11:28:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.459+0000 7ff037c78700 1 -- 172.21.15.45:0/483812413 wait complete. 2023-08-27T11:28:11.474 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:28:11.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:11.572 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:11.570+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:11.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.600+0000 7f05a9c60700 1 Processor -- start 2023-08-27T11:28:11.602 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.600+0000 7f05a9c60700 1 -- start start 2023-08-27T11:28:11.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a9c60700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05a40cfa00 0x7f05a40cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a9c60700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a40d1ac0 0x7f05a40d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a9c60700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a40d03a0 con 0x7f05a415d350 2023-08-27T11:28:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a9c60700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a415d730 con 0x7f05a40cfa00 2023-08-27T11:28:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a9c60700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40d1070 con 0x7f05a40d1ac0 2023-08-27T11:28:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f059affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05a40cfa00 0x7f05a40cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f059affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05a40cfa00 0x7f05a40cfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59100/0 (socket says 172.21.15.45:59100) 2023-08-27T11:28:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f059affd700 1 -- 172.21.15.45:0/3497663330 learned_addr learned my addr 172.21.15.45:0/3497663330 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a3fff700 1 --2- 172.21.15.45:0/3497663330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a40d1ac0 0x7f05a40d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f059affd700 1 -- 172.21.15.45:0/3497663330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a40d1ac0 msgr2=0x7f05a40d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f059affd700 1 --2- 172.21.15.45:0/3497663330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a40d1ac0 0x7f05a40d1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f059affd700 1 -- 172.21.15.45:0/3497663330 >> v1:172.21.15.135:6789/0 conn(0x7f05a415d350 legacy=0x7f05a4160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f059affd700 1 -- 172.21.15.45:0/3497663330 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4161890 con 0x7f05a40cfa00 2023-08-27T11:28:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a3fff700 1 --2- 172.21.15.45:0/3497663330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a40d1ac0 0x7f05a40d1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f059affd700 1 --2- 172.21.15.45:0/3497663330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 0x7f05a40cfe60 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f058c0098f0 tx=0x7f058c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de22eb5c14abe0b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a2ffd700 1 -- 172.21.15.45:0/3497663330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c004230 con 0x7f05a40cfa00 2023-08-27T11:28:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a2ffd700 1 -- 172.21.15.45:0/3497663330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c004390 con 0x7f05a40cfa00 2023-08-27T11:28:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.602+0000 7f05a2ffd700 1 -- 172.21.15.45:0/3497663330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c0036a0 con 0x7f05a40cfa00 2023-08-27T11:28:11.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.603+0000 7f05a9c60700 1 -- 172.21.15.45:0/3497663330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 msgr2=0x7f05a40cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.603+0000 7f05a9c60700 1 --2- 172.21.15.45:0/3497663330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 0x7f05a40cfe60 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f058c0098f0 tx=0x7f058c009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:11.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.603+0000 7f05a9c60700 1 -- 172.21.15.45:0/3497663330 shutdown_connections 2023-08-27T11:28:11.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.603+0000 7f05a9c60700 1 --2- 172.21.15.45:0/3497663330 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a40d1ac0 0x7f05a40d1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.603+0000 7f05a9c60700 1 --2- 172.21.15.45:0/3497663330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 0x7f05a40cfe60 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.603+0000 7f05a9c60700 1 -- 172.21.15.45:0/3497663330 >> 172.21.15.45:0/3497663330 conn(0x7f05a40d2ff0 msgr2=0x7f05a40d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.603+0000 7f05a9c60700 1 -- 172.21.15.45:0/3497663330 shutdown_connections 2023-08-27T11:28:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.603+0000 7f05a9c60700 1 -- 172.21.15.45:0/3497663330 wait complete. 2023-08-27T11:28:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.604+0000 7f05a9c60700 1 Processor -- start 2023-08-27T11:28:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a9c60700 1 -- start start 2023-08-27T11:28:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a9c60700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 0x7f05a414eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a9c60700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05a40d1ac0 0x7f05a41f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a9c60700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a415d350 0x7f05a41f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a9c60700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a414f000 con 0x7f05a40cfa00 2023-08-27T11:28:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a9c60700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a414d4a0 con 0x7f05a40d1ac0 2023-08-27T11:28:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a9c60700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a415df90 con 0x7f05a415d350 2023-08-27T11:28:11.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 0x7f05a414eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 0x7f05a414eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59104/0 (socket says 172.21.15.45:59104) 2023-08-27T11:28:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.605+0000 7f05a37fe700 1 -- 172.21.15.45:0/1568607564 learned_addr learned my addr 172.21.15.45:0/1568607564 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a37fe700 1 -- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a415d350 msgr2=0x7f05a41f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f059affd700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05a40d1ac0 0x7f05a41f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a3fff700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a415d350 0x7f05a41f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a37fe700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a415d350 0x7f05a41f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a37fe700 1 -- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05a40d1ac0 msgr2=0x7f05a41f60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a37fe700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05a40d1ac0 0x7f05a41f60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a37fe700 1 -- 172.21.15.45:0/1568607564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c009550 con 0x7f05a40cfa00 2023-08-27T11:28:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f059affd700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05a40d1ac0 0x7f05a41f60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:11.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a3fff700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a415d350 0x7f05a41f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:11.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a37fe700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 0x7f05a414eac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f059400c4d0 tx=0x7f059400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a0ff9700 1 -- 172.21.15.45:0/1568607564 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594011790 con 0x7f05a40cfa00 2023-08-27T11:28:11.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a0ff9700 1 -- 172.21.15.45:0/1568607564 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05940118f0 con 0x7f05a40cfa00 2023-08-27T11:28:11.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a415dcb0 con 0x7f05a40cfa00 2023-08-27T11:28:11.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a0ff9700 1 -- 172.21.15.45:0/1568607564 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594011ca0 con 0x7f05a40cfa00 2023-08-27T11:28:11.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.606+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a414da20 con 0x7f05a40cfa00 2023-08-27T11:28:11.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.607+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a414d300 con 0x7f05a40cfa00 2023-08-27T11:28:11.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.608+0000 7f05a0ff9700 1 -- 172.21.15.45:0/1568607564 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f059401c460 con 0x7f05a40cfa00 2023-08-27T11:28:11.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.608+0000 7f05a0ff9700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f058406a170 0x7f058406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:11.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.608+0000 7f05a0ff9700 1 -- 172.21.15.45:0/1568607564 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0594088480 con 0x7f05a40cfa00 2023-08-27T11:28:11.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.608+0000 7f059affd700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f058406a170 0x7f058406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:11.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.609+0000 7f059affd700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f058406a170 0x7f058406c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05a4148eb0 tx=0x7f058c008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:11.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.613+0000 7f05a0ff9700 1 -- 172.21.15.45:0/1568607564 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f05a414d300 con 0x7f05a40cfa00 2023-08-27T11:28:11.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.812+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 --> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05a4161b70 con 0x7f058406a170 2023-08-27T11:28:11.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.822+0000 7f05a0ff9700 1 -- 172.21.15.45:0/1568607564 <== mgr.6709 v2:172.21.15.135:6832/3812189695 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306377 (secure 0 0 0) 0x7f058c01f070 con 0x7f058406a170 2023-08-27T11:28:11.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f058406a170 msgr2=0x7f058406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f058406a170 0x7f058406c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05a4148eb0 tx=0x7f058c008030 comp rx=0 tx=0).stop 2023-08-27T11:28:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 msgr2=0x7f05a414eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 0x7f05a414eac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f059400c4d0 tx=0x7f059400c890 comp rx=0 tx=0).stop 2023-08-27T11:28:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 shutdown_connections 2023-08-27T11:28:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f058406a170 0x7f058406c630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a40cfa00 0x7f05a414eac0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05a415d350 0x7f05a41f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 --2- 172.21.15.45:0/1568607564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05a40d1ac0 0x7f05a41f60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:11.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 >> 172.21.15.45:0/1568607564 conn(0x7f05a40d2ff0 msgr2=0x7f05a40b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:11.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 shutdown_connections 2023-08-27T11:28:11.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:11.826+0000 7f05a9c60700 1 -- 172.21.15.45:0/1568607564 wait complete. 2023-08-27T11:28:11.833 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:11.840 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:28:11.501465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2213,"ondisk_log_size":2213,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284920,"kb_used_data":10512,"kb_used_omap":13,"kb_used_meta":274290,"kb_avail":754689800,"statfs":{"total":773094113280,"available":772802355200,"internally_reserved":0,"allocated":10764288,"data_stored":6183480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":280873048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.849363"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674262+0000","last_change":"2023-08-27T11:28:06.674262+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674262+0000","last_undegraded":"2023-08-27T11:28:06.674262+0000","last_fullsized":"2023-08-27T11:28:06.674262+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663991+0000","last_change":"2023-08-27T11:28:06.663991+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663991+0000","last_undegraded":"2023-08-27T11:28:06.663991+0000","last_fullsized":"2023-08-27T11:28:06.663991+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671195+0000","last_change":"2023-08-27T11:28:06.671195+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671195+0000","last_undegraded":"2023-08-27T11:28:06.671195+0000","last_fullsized":"2023-08-27T11:28:06.671195+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669142+0000","last_change":"2023-08-27T11:28:06.669142+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669142+0000","last_undegraded":"2023-08-27T11:28:06.669142+0000","last_fullsized":"2023-08-27T11:28:06.669142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668212+0000","last_change":"2023-08-27T11:28:06.668212+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668212+0000","last_undegraded":"2023-08-27T11:28:06.668212+0000","last_fullsized":"2023-08-27T11:28:06.668212+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670471+0000","last_change":"2023-08-27T11:28:06.670471+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670471+0000","last_undegraded":"2023-08-27T11:28:06.670471+0000","last_fullsized":"2023-08-27T11:28:06.670471+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667376+0000","last_change":"2023-08-27T11:28:06.667376+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667376+0000","last_undegraded":"2023-08-27T11:28:06.667376+0000","last_fullsized":"2023-08-27T11:28:06.667376+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670912+0000","last_change":"2023-08-27T11:28:06.670912+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670912+0000","last_undegraded":"2023-08-27T11:28:06.670912+0000","last_fullsized":"2023-08-27T11:28:06.670912+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669900+0000","last_change":"2023-08-27T11:28:06.669900+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669900+0000","last_undegraded":"2023-08-27T11:28:06.669900+0000","last_fullsized":"2023-08-27T11:28:06.669900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674502+0000","last_change":"2023-08-27T11:28:06.674502+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674502+0000","last_undegraded":"2023-08-27T11:28:06.674502+0000","last_fullsized":"2023-08-27T11:28:06.674502+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665233+0000","last_change":"2023-08-27T11:28:06.665233+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665233+0000","last_undegraded":"2023-08-27T11:28:06.665233+0000","last_fullsized":"2023-08-27T11:28:06.665233+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.677732+0000","last_change":"2023-08-27T11:28:06.677732+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.677732+0000","last_undegraded":"2023-08-27T11:28:06.677732+0000","last_fullsized":"2023-08-27T11:28:06.677732+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.672762+0000","last_change":"2023-08-27T11:28:06.672762+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.672762+0000","last_undegraded":"2023-08-27T11:28:06.672762+0000","last_fullsized":"2023-08-27T11:28:06.672762+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667659+0000","last_change":"2023-08-27T11:28:06.667659+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667659+0000","last_undegraded":"2023-08-27T11:28:06.667659+0000","last_fullsized":"2023-08-27T11:28:06.667659+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662713+0000","last_change":"2023-08-27T11:28:06.662713+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662713+0000","last_undegraded":"2023-08-27T11:28:06.662713+0000","last_fullsized":"2023-08-27T11:28:06.662713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667753+0000","last_change":"2023-08-27T11:28:06.667753+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667753+0000","last_undegraded":"2023-08-27T11:28:06.667753+0000","last_fullsized":"2023-08-27T11:28:06.667753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667875+0000","last_change":"2023-08-27T11:28:06.667875+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667875+0000","last_undegraded":"2023-08-27T11:28:06.667875+0000","last_fullsized":"2023-08-27T11:28:06.667875+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.666981+0000","last_change":"2023-08-27T11:28:06.666981+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.666981+0000","last_undegraded":"2023-08-27T11:28:06.666981+0000","last_fullsized":"2023-08-27T11:28:06.666981+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662798+0000","last_change":"2023-08-27T11:28:06.662798+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662798+0000","last_undegraded":"2023-08-27T11:28:06.662798+0000","last_fullsized":"2023-08-27T11:28:06.662798+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.666965+0000","last_change":"2023-08-27T11:28:06.666965+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.666965+0000","last_undegraded":"2023-08-27T11:28:06.666965+0000","last_fullsized":"2023-08-27T11:28:06.666965+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668224+0000","last_change":"2023-08-27T11:28:06.668224+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668224+0000","last_undegraded":"2023-08-27T11:28:06.668224+0000","last_fullsized":"2023-08-27T11:28:06.668224+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663693+0000","last_change":"2023-08-27T11:28:06.663693+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663693+0000","last_undegraded":"2023-08-27T11:28:06.663693+0000","last_fullsized":"2023-08-27T11:28:06.663693+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669584+0000","last_change":"2023-08-27T11:28:06.669584+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669584+0000","last_undegraded":"2023-08-27T11:28:06.669584+0000","last_fullsized":"2023-08-27T11:28:06.669584+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665390+0000","last_change":"2023-08-27T11:28:06.665390+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665390+0000","last_undegraded":"2023-08-27T11:28:06.665390+0000","last_fullsized":"2023-08-27T11:28:06.665390+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663341+0000","last_change":"2023-08-27T11:28:06.663341+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663341+0000","last_undegraded":"2023-08-27T11:28:06.663341+0000","last_fullsized":"2023-08-27T11:28:06.663341+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674565+0000","last_change":"2023-08-27T11:28:06.674565+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674565+0000","last_undegraded":"2023-08-27T11:28:06.674565+0000","last_fullsized":"2023-08-27T11:28:06.674565+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669801+0000","last_change":"2023-08-27T11:28:06.669801+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669801+0000","last_undegraded":"2023-08-27T11:28:06.669801+0000","last_fullsized":"2023-08-27T11:28:06.669801+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.677428+0000","last_change":"2023-08-27T11:28:06.677428+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.677428+0000","last_undegraded":"2023-08-27T11:28:06.677428+0000","last_fullsized":"2023-08-27T11:28:06.677428+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662770+0000","last_change":"2023-08-27T11:28:06.662770+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662770+0000","last_undegraded":"2023-08-27T11:28:06.662770+0000","last_fullsized":"2023-08-27T11:28:06.662770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662917+0000","last_change":"2023-08-27T11:28:06.662917+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662917+0000","last_undegraded":"2023-08-27T11:28:06.662917+0000","last_fullsized":"2023-08-27T11:28:06.662917+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667437+0000","last_change":"2023-08-27T11:28:06.667437+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667437+0000","last_undegraded":"2023-08-27T11:28:06.667437+0000","last_fullsized":"2023-08-27T11:28:06.667437+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674563+0000","last_change":"2023-08-27T11:28:06.674563+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674563+0000","last_undegraded":"2023-08-27T11:28:06.674563+0000","last_fullsized":"2023-08-27T11:28:06.674563+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671268+0000","last_change":"2023-08-27T11:28:06.671268+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671268+0000","last_undegraded":"2023-08-27T11:28:06.671268+0000","last_fullsized":"2023-08-27T11:28:06.671268+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682500+0000","last_change":"2023-08-27T11:28:06.680510+0000","last_active":"2023-08-27T11:28:06.682500+0000","last_peered":"2023-08-27T11:28:06.682500+0000","last_clean":"2023-08-27T11:28:06.682500+0000","last_became_active":"2023-08-27T11:28:06.680268+0000","last_became_peered":"2023-08-27T11:28:06.680268+0000","last_unstale":"2023-08-27T11:28:06.682500+0000","last_undegraded":"2023-08-27T11:28:06.682500+0000","last_fullsized":"2023-08-27T11:28:06.682500+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:53.355949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673997+0000","last_change":"2023-08-27T11:28:06.673997+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673997+0000","last_undegraded":"2023-08-27T11:28:06.673997+0000","last_fullsized":"2023-08-27T11:28:06.673997+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680586+0000","last_change":"2023-08-27T11:28:06.678558+0000","last_active":"2023-08-27T11:28:06.680586+0000","last_peered":"2023-08-27T11:28:06.680586+0000","last_clean":"2023-08-27T11:28:06.680586+0000","last_became_active":"2023-08-27T11:28:06.678323+0000","last_became_peered":"2023-08-27T11:28:06.678323+0000","last_unstale":"2023-08-27T11:28:06.680586+0000","last_undegraded":"2023-08-27T11:28:06.680586+0000","last_fullsized":"2023-08-27T11:28:06.680586+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:32:33.684096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673970+0000","last_change":"2023-08-27T11:28:06.673970+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673970+0000","last_undegraded":"2023-08-27T11:28:06.673970+0000","last_fullsized":"2023-08-27T11:28:06.673970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675092+0000","last_change":"2023-08-27T11:28:06.674106+0000","last_active":"2023-08-27T11:28:06.675092+0000","last_peered":"2023-08-27T11:28:06.675092+0000","last_clean":"2023-08-27T11:28:06.675092+0000","last_became_active":"2023-08-27T11:28:06.673843+0000","last_became_peered":"2023-08-27T11:28:06.673843+0000","last_unstale":"2023-08-27T11:28:06.675092+0000","last_undegraded":"2023-08-27T11:28:06.675092+0000","last_fullsized":"2023-08-27T11:28:06.675092+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:18.098063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670630+0000","last_change":"2023-08-27T11:28:06.670630+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670630+0000","last_undegraded":"2023-08-27T11:28:06.670630+0000","last_fullsized":"2023-08-27T11:28:06.670630+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675059+0000","last_change":"2023-08-27T11:28:06.674658+0000","last_active":"2023-08-27T11:28:06.675059+0000","last_peered":"2023-08-27T11:28:06.675059+0000","last_clean":"2023-08-27T11:28:06.675059+0000","last_became_active":"2023-08-27T11:28:06.674384+0000","last_became_peered":"2023-08-27T11:28:06.674384+0000","last_unstale":"2023-08-27T11:28:06.675059+0000","last_undegraded":"2023-08-27T11:28:06.675059+0000","last_fullsized":"2023-08-27T11:28:06.675059+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:33:38.254409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663309+0000","last_change":"2023-08-27T11:28:06.663309+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663309+0000","last_undegraded":"2023-08-27T11:28:06.663309+0000","last_fullsized":"2023-08-27T11:28:06.663309+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680562+0000","last_change":"2023-08-27T11:28:06.678516+0000","last_active":"2023-08-27T11:28:06.680562+0000","last_peered":"2023-08-27T11:28:06.680562+0000","last_clean":"2023-08-27T11:28:06.680562+0000","last_became_active":"2023-08-27T11:28:06.678268+0000","last_became_peered":"2023-08-27T11:28:06.678268+0000","last_unstale":"2023-08-27T11:28:06.680562+0000","last_undegraded":"2023-08-27T11:28:06.680562+0000","last_fullsized":"2023-08-27T11:28:06.680562+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:29:56.120623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667804+0000","last_change":"2023-08-27T11:28:06.667804+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667804+0000","last_undegraded":"2023-08-27T11:28:06.667804+0000","last_fullsized":"2023-08-27T11:28:06.667804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680480+0000","last_change":"2023-08-27T11:28:06.677993+0000","last_active":"2023-08-27T11:28:06.680480+0000","last_peered":"2023-08-27T11:28:06.680480+0000","last_clean":"2023-08-27T11:28:06.680480+0000","last_became_active":"2023-08-27T11:28:06.677679+0000","last_became_peered":"2023-08-27T11:28:06.677679+0000","last_unstale":"2023-08-27T11:28:06.680480+0000","last_undegraded":"2023-08-27T11:28:06.680480+0000","last_fullsized":"2023-08-27T11:28:06.680480+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:04.340118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670493+0000","last_change":"2023-08-27T11:28:06.670493+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670493+0000","last_undegraded":"2023-08-27T11:28:06.670493+0000","last_fullsized":"2023-08-27T11:28:06.670493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676184+0000","last_change":"2023-08-27T11:28:06.675717+0000","last_active":"2023-08-27T11:28:06.676184+0000","last_peered":"2023-08-27T11:28:06.676184+0000","last_clean":"2023-08-27T11:28:06.676184+0000","last_became_active":"2023-08-27T11:28:06.675161+0000","last_became_peered":"2023-08-27T11:28:06.675161+0000","last_unstale":"2023-08-27T11:28:06.676184+0000","last_undegraded":"2023-08-27T11:28:06.676184+0000","last_fullsized":"2023-08-27T11:28:06.676184+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:30.138559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662304+0000","last_change":"2023-08-27T11:28:06.662304+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662304+0000","last_undegraded":"2023-08-27T11:28:06.662304+0000","last_fullsized":"2023-08-27T11:28:06.662304+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680689+0000","last_change":"2023-08-27T11:28:06.680053+0000","last_active":"2023-08-27T11:28:06.680689+0000","last_peered":"2023-08-27T11:28:06.680689+0000","last_clean":"2023-08-27T11:28:06.680689+0000","last_became_active":"2023-08-27T11:28:06.679714+0000","last_became_peered":"2023-08-27T11:28:06.679714+0000","last_unstale":"2023-08-27T11:28:06.680689+0000","last_undegraded":"2023-08-27T11:28:06.680689+0000","last_fullsized":"2023-08-27T11:28:06.680689+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:13.718165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.661902+0000","last_change":"2023-08-27T11:28:06.661902+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.661902+0000","last_undegraded":"2023-08-27T11:28:06.661902+0000","last_fullsized":"2023-08-27T11:28:06.661902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.683222+0000","last_change":"2023-08-27T11:28:06.682417+0000","last_active":"2023-08-27T11:28:06.683222+0000","last_peered":"2023-08-27T11:28:06.683222+0000","last_clean":"2023-08-27T11:28:06.683222+0000","last_became_active":"2023-08-27T11:28:06.682174+0000","last_became_peered":"2023-08-27T11:28:06.682174+0000","last_unstale":"2023-08-27T11:28:06.683222+0000","last_undegraded":"2023-08-27T11:28:06.683222+0000","last_fullsized":"2023-08-27T11:28:06.683222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:22.047442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669447+0000","last_change":"2023-08-27T11:28:06.669447+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669447+0000","last_undegraded":"2023-08-27T11:28:06.669447+0000","last_fullsized":"2023-08-27T11:28:06.669447+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.683629+0000","last_change":"2023-08-27T11:28:06.682913+0000","last_active":"2023-08-27T11:28:06.683629+0000","last_peered":"2023-08-27T11:28:06.683629+0000","last_clean":"2023-08-27T11:28:06.683629+0000","last_became_active":"2023-08-27T11:28:06.682773+0000","last_became_peered":"2023-08-27T11:28:06.682773+0000","last_unstale":"2023-08-27T11:28:06.683629+0000","last_undegraded":"2023-08-27T11:28:06.683629+0000","last_fullsized":"2023-08-27T11:28:06.683629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:13.142404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662935+0000","last_change":"2023-08-27T11:28:06.662935+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662935+0000","last_undegraded":"2023-08-27T11:28:06.662935+0000","last_fullsized":"2023-08-27T11:28:06.662935+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682449+0000","last_change":"2023-08-27T11:28:06.680796+0000","last_active":"2023-08-27T11:28:06.682449+0000","last_peered":"2023-08-27T11:28:06.682449+0000","last_clean":"2023-08-27T11:28:06.682449+0000","last_became_active":"2023-08-27T11:28:06.680639+0000","last_became_peered":"2023-08-27T11:28:06.680639+0000","last_unstale":"2023-08-27T11:28:06.682449+0000","last_undegraded":"2023-08-27T11:28:06.682449+0000","last_fullsized":"2023-08-27T11:28:06.682449+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:54.607978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669454+0000","last_change":"2023-08-27T11:28:06.669454+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669454+0000","last_undegraded":"2023-08-27T11:28:06.669454+0000","last_fullsized":"2023-08-27T11:28:06.669454+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679081+0000","last_change":"2023-08-27T11:28:06.675117+0000","last_active":"2023-08-27T11:28:06.679081+0000","last_peered":"2023-08-27T11:28:06.679081+0000","last_clean":"2023-08-27T11:28:06.679081+0000","last_became_active":"2023-08-27T11:28:06.674808+0000","last_became_peered":"2023-08-27T11:28:06.674808+0000","last_unstale":"2023-08-27T11:28:06.679081+0000","last_undegraded":"2023-08-27T11:28:06.679081+0000","last_fullsized":"2023-08-27T11:28:06.679081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:13.763908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663233+0000","last_change":"2023-08-27T11:28:06.663233+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663233+0000","last_undegraded":"2023-08-27T11:28:06.663233+0000","last_fullsized":"2023-08-27T11:28:06.663233+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.673619+0000","last_change":"2023-08-27T11:28:06.670804+0000","last_active":"2023-08-27T11:28:06.673619+0000","last_peered":"2023-08-27T11:28:06.673619+0000","last_clean":"2023-08-27T11:28:06.673619+0000","last_became_active":"2023-08-27T11:28:06.669783+0000","last_became_peered":"2023-08-27T11:28:06.669783+0000","last_unstale":"2023-08-27T11:28:06.673619+0000","last_undegraded":"2023-08-27T11:28:06.673619+0000","last_fullsized":"2023-08-27T11:28:06.673619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:54.035323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.672405+0000","last_change":"2023-08-27T11:28:06.672405+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.672405+0000","last_undegraded":"2023-08-27T11:28:06.672405+0000","last_fullsized":"2023-08-27T11:28:06.672405+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672817+0000","last_change":"2023-08-27T11:28:06.668164+0000","last_active":"2023-08-27T11:28:06.672817+0000","last_peered":"2023-08-27T11:28:06.672817+0000","last_clean":"2023-08-27T11:28:06.672817+0000","last_became_active":"2023-08-27T11:28:06.667870+0000","last_became_peered":"2023-08-27T11:28:06.667870+0000","last_unstale":"2023-08-27T11:28:06.672817+0000","last_undegraded":"2023-08-27T11:28:06.672817+0000","last_fullsized":"2023-08-27T11:28:06.672817+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:34.544721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664777+0000","last_change":"2023-08-27T11:28:06.664777+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664777+0000","last_undegraded":"2023-08-27T11:28:06.664777+0000","last_fullsized":"2023-08-27T11:28:06.664777+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.681660+0000","last_change":"2023-08-27T11:28:06.679870+0000","last_active":"2023-08-27T11:28:06.681660+0000","last_peered":"2023-08-27T11:28:06.681660+0000","last_clean":"2023-08-27T11:28:06.681660+0000","last_became_active":"2023-08-27T11:28:06.679627+0000","last_became_peered":"2023-08-27T11:28:06.679627+0000","last_unstale":"2023-08-27T11:28:06.681660+0000","last_undegraded":"2023-08-27T11:28:06.681660+0000","last_fullsized":"2023-08-27T11:28:06.681660+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:35.146679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665638+0000","last_change":"2023-08-27T11:28:06.665638+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665638+0000","last_undegraded":"2023-08-27T11:28:06.665638+0000","last_fullsized":"2023-08-27T11:28:06.665638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682022+0000","last_change":"2023-08-27T11:28:06.681196+0000","last_active":"2023-08-27T11:28:06.682022+0000","last_peered":"2023-08-27T11:28:06.682022+0000","last_clean":"2023-08-27T11:28:06.682022+0000","last_became_active":"2023-08-27T11:28:06.680914+0000","last_became_peered":"2023-08-27T11:28:06.680914+0000","last_unstale":"2023-08-27T11:28:06.682022+0000","last_undegraded":"2023-08-27T11:28:06.682022+0000","last_fullsized":"2023-08-27T11:28:06.682022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:12.051340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670514+0000","last_change":"2023-08-27T11:28:06.670514+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670514+0000","last_undegraded":"2023-08-27T11:28:06.670514+0000","last_fullsized":"2023-08-27T11:28:06.670514+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680425+0000","last_change":"2023-08-27T11:28:06.679947+0000","last_active":"2023-08-27T11:28:06.680425+0000","last_peered":"2023-08-27T11:28:06.680425+0000","last_clean":"2023-08-27T11:28:06.680425+0000","last_became_active":"2023-08-27T11:28:06.679654+0000","last_became_peered":"2023-08-27T11:28:06.679654+0000","last_unstale":"2023-08-27T11:28:06.680425+0000","last_undegraded":"2023-08-27T11:28:06.680425+0000","last_fullsized":"2023-08-27T11:28:06.680425+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:36.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671123+0000","last_change":"2023-08-27T11:28:06.671123+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671123+0000","last_undegraded":"2023-08-27T11:28:06.671123+0000","last_fullsized":"2023-08-27T11:28:06.671123+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672339+0000","last_change":"2023-08-27T11:28:06.668640+0000","last_active":"2023-08-27T11:28:06.672339+0000","last_peered":"2023-08-27T11:28:06.672339+0000","last_clean":"2023-08-27T11:28:06.672339+0000","last_became_active":"2023-08-27T11:28:06.668372+0000","last_became_peered":"2023-08-27T11:28:06.668372+0000","last_unstale":"2023-08-27T11:28:06.672339+0000","last_undegraded":"2023-08-27T11:28:06.672339+0000","last_fullsized":"2023-08-27T11:28:06.672339+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:48:01.533647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673604+0000","last_change":"2023-08-27T11:28:06.673604+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673604+0000","last_undegraded":"2023-08-27T11:28:06.673604+0000","last_fullsized":"2023-08-27T11:28:06.673604+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682683+0000","last_change":"2023-08-27T11:28:06.682044+0000","last_active":"2023-08-27T11:28:06.682683+0000","last_peered":"2023-08-27T11:28:06.682683+0000","last_clean":"2023-08-27T11:28:06.682683+0000","last_became_active":"2023-08-27T11:28:06.681789+0000","last_became_peered":"2023-08-27T11:28:06.681789+0000","last_unstale":"2023-08-27T11:28:06.682683+0000","last_undegraded":"2023-08-27T11:28:06.682683+0000","last_fullsized":"2023-08-27T11:28:06.682683+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:39:11.072746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668771+0000","last_change":"2023-08-27T11:28:06.668771+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668771+0000","last_undegraded":"2023-08-27T11:28:06.668771+0000","last_fullsized":"2023-08-27T11:28:06.668771+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682726+0000","last_change":"2023-08-27T11:28:06.682213+0000","last_active":"2023-08-27T11:28:06.682726+0000","last_peered":"2023-08-27T11:28:06.682726+0000","last_clean":"2023-08-27T11:28:06.682726+0000","last_became_active":"2023-08-27T11:28:06.681772+0000","last_became_peered":"2023-08-27T11:28:06.681772+0000","last_unstale":"2023-08-27T11:28:06.682726+0000","last_undegraded":"2023-08-27T11:28:06.682726+0000","last_fullsized":"2023-08-27T11:28:06.682726+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:34:15.373195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670120+0000","last_change":"2023-08-27T11:28:06.670120+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670120+0000","last_undegraded":"2023-08-27T11:28:06.670120+0000","last_fullsized":"2023-08-27T11:28:06.670120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682020+0000","last_change":"2023-08-27T11:28:06.681163+0000","last_active":"2023-08-27T11:28:06.682020+0000","last_peered":"2023-08-27T11:28:06.682020+0000","last_clean":"2023-08-27T11:28:06.682020+0000","last_became_active":"2023-08-27T11:28:06.680922+0000","last_became_peered":"2023-08-27T11:28:06.680922+0000","last_unstale":"2023-08-27T11:28:06.682020+0000","last_undegraded":"2023-08-27T11:28:06.682020+0000","last_fullsized":"2023-08-27T11:28:06.682020+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:13:20.626473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664201+0000","last_change":"2023-08-27T11:28:06.664201+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664201+0000","last_undegraded":"2023-08-27T11:28:06.664201+0000","last_fullsized":"2023-08-27T11:28:06.664201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680463+0000","last_change":"2023-08-27T11:28:06.677834+0000","last_active":"2023-08-27T11:28:06.680463+0000","last_peered":"2023-08-27T11:28:06.680463+0000","last_clean":"2023-08-27T11:28:06.680463+0000","last_became_active":"2023-08-27T11:28:06.677483+0000","last_became_peered":"2023-08-27T11:28:06.677483+0000","last_unstale":"2023-08-27T11:28:06.680463+0000","last_undegraded":"2023-08-27T11:28:06.680463+0000","last_fullsized":"2023-08-27T11:28:06.680463+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:33:20.246978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670946+0000","last_change":"2023-08-27T11:28:06.670946+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670946+0000","last_undegraded":"2023-08-27T11:28:06.670946+0000","last_fullsized":"2023-08-27T11:28:06.670946+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682708+0000","last_change":"2023-08-27T11:28:06.682127+0000","last_active":"2023-08-27T11:28:06.682708+0000","last_peered":"2023-08-27T11:28:06.682708+0000","last_clean":"2023-08-27T11:28:06.682708+0000","last_became_active":"2023-08-27T11:28:06.681824+0000","last_became_peered":"2023-08-27T11:28:06.681824+0000","last_unstale":"2023-08-27T11:28:06.682708+0000","last_undegraded":"2023-08-27T11:28:06.682708+0000","last_fullsized":"2023-08-27T11:28:06.682708+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:03.058354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669155+0000","last_change":"2023-08-27T11:28:06.669155+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669155+0000","last_undegraded":"2023-08-27T11:28:06.669155+0000","last_fullsized":"2023-08-27T11:28:06.669155+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679086+0000","last_change":"2023-08-27T11:28:06.676211+0000","last_active":"2023-08-27T11:28:06.679086+0000","last_peered":"2023-08-27T11:28:06.679086+0000","last_clean":"2023-08-27T11:28:06.679086+0000","last_became_active":"2023-08-27T11:28:06.675956+0000","last_became_peered":"2023-08-27T11:28:06.675956+0000","last_unstale":"2023-08-27T11:28:06.679086+0000","last_undegraded":"2023-08-27T11:28:06.679086+0000","last_fullsized":"2023-08-27T11:28:06.679086+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:48:39.217804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669642+0000","last_change":"2023-08-27T11:28:06.669642+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669642+0000","last_undegraded":"2023-08-27T11:28:06.669642+0000","last_fullsized":"2023-08-27T11:28:06.669642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676840+0000","last_change":"2023-08-27T11:28:06.673096+0000","last_active":"2023-08-27T11:28:06.676840+0000","last_peered":"2023-08-27T11:28:06.676840+0000","last_clean":"2023-08-27T11:28:06.676840+0000","last_became_active":"2023-08-27T11:28:06.672819+0000","last_became_peered":"2023-08-27T11:28:06.672819+0000","last_unstale":"2023-08-27T11:28:06.676840+0000","last_undegraded":"2023-08-27T11:28:06.676840+0000","last_fullsized":"2023-08-27T11:28:06.676840+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:34.207300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.666042+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:06.666042+0000","last_peered":"2023-08-27T11:28:06.666042+0000","last_clean":"2023-08-27T11:28:06.666042+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:06.666042+0000","last_undegraded":"2023-08-27T11:28:06.666042+0000","last_fullsized":"2023-08-27T11:28:06.666042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:28.335693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669129+0000","last_change":"2023-08-27T11:28:06.669129+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669129+0000","last_undegraded":"2023-08-27T11:28:06.669129+0000","last_fullsized":"2023-08-27T11:28:06.669129+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676890+0000","last_change":"2023-08-27T11:28:06.676306+0000","last_active":"2023-08-27T11:28:06.676890+0000","last_peered":"2023-08-27T11:28:06.676890+0000","last_clean":"2023-08-27T11:28:06.676890+0000","last_became_active":"2023-08-27T11:28:06.676049+0000","last_became_peered":"2023-08-27T11:28:06.676049+0000","last_unstale":"2023-08-27T11:28:06.676890+0000","last_undegraded":"2023-08-27T11:28:06.676890+0000","last_fullsized":"2023-08-27T11:28:06.676890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:52.590584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672445+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:06.672445+0000","last_peered":"2023-08-27T11:28:06.672445+0000","last_clean":"2023-08-27T11:28:06.672445+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:06.672445+0000","last_undegraded":"2023-08-27T11:28:06.672445+0000","last_fullsized":"2023-08-27T11:28:06.672445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:04.382696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670944+0000","last_change":"2023-08-27T11:28:06.670944+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670944+0000","last_undegraded":"2023-08-27T11:28:06.670944+0000","last_fullsized":"2023-08-27T11:28:06.670944+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682490+0000","last_change":"2023-08-27T11:28:06.680009+0000","last_active":"2023-08-27T11:28:06.682490+0000","last_peered":"2023-08-27T11:28:06.682490+0000","last_clean":"2023-08-27T11:28:06.682490+0000","last_became_active":"2023-08-27T11:28:06.679754+0000","last_became_peered":"2023-08-27T11:28:06.679754+0000","last_unstale":"2023-08-27T11:28:06.682490+0000","last_undegraded":"2023-08-27T11:28:06.682490+0000","last_fullsized":"2023-08-27T11:28:06.682490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:06.046171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":154,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.670537+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:06.670537+0000","last_peered":"2023-08-27T11:28:06.670537+0000","last_clean":"2023-08-27T11:28:06.670537+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:06.670537+0000","last_undegraded":"2023-08-27T11:28:06.670537+0000","last_fullsized":"2023-08-27T11:28:06.670537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:46:09.042939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664810+0000","last_change":"2023-08-27T11:28:06.664810+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664810+0000","last_undegraded":"2023-08-27T11:28:06.664810+0000","last_fullsized":"2023-08-27T11:28:06.664810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.681624+0000","last_change":"2023-08-27T11:28:06.680561+0000","last_active":"2023-08-27T11:28:06.681624+0000","last_peered":"2023-08-27T11:28:06.681624+0000","last_clean":"2023-08-27T11:28:06.681624+0000","last_became_active":"2023-08-27T11:28:06.680307+0000","last_became_peered":"2023-08-27T11:28:06.680307+0000","last_unstale":"2023-08-27T11:28:06.681624+0000","last_undegraded":"2023-08-27T11:28:06.681624+0000","last_fullsized":"2023-08-27T11:28:06.681624+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:58.720509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672903+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:06.672903+0000","last_peered":"2023-08-27T11:28:06.672903+0000","last_clean":"2023-08-27T11:28:06.672903+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:06.672903+0000","last_undegraded":"2023-08-27T11:28:06.672903+0000","last_fullsized":"2023-08-27T11:28:06.672903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:40:14.646059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"98'2213","reported_seq":2572,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:07.501955+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:07.501955+0000","last_peered":"2023-08-27T11:28:07.501955+0000","last_clean":"2023-08-27T11:28:07.501955+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:07.501955+0000","last_undegraded":"2023-08-27T11:28:07.501955+0000","last_fullsized":"2023-08-27T11:28:07.501955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":2213,"log_dups_size":0,"ondisk_log_size":2213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:54:12.932795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673646+0000","last_change":"2023-08-27T11:28:06.673646+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673646+0000","last_undegraded":"2023-08-27T11:28:06.673646+0000","last_fullsized":"2023-08-27T11:28:06.673646+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680214+0000","last_change":"2023-08-27T11:28:06.678931+0000","last_active":"2023-08-27T11:28:06.680214+0000","last_peered":"2023-08-27T11:28:06.680214+0000","last_clean":"2023-08-27T11:28:06.680214+0000","last_became_active":"2023-08-27T11:28:06.678661+0000","last_became_peered":"2023-08-27T11:28:06.678661+0000","last_unstale":"2023-08-27T11:28:06.680214+0000","last_undegraded":"2023-08-27T11:28:06.680214+0000","last_fullsized":"2023-08-27T11:28:06.680214+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:00:05.382377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675655+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:06.675655+0000","last_peered":"2023-08-27T11:28:06.675655+0000","last_clean":"2023-08-27T11:28:06.675655+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:06.675655+0000","last_undegraded":"2023-08-27T11:28:06.675655+0000","last_fullsized":"2023-08-27T11:28:06.675655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:10:39.559849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667293+0000","last_change":"2023-08-27T11:28:06.667293+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667293+0000","last_undegraded":"2023-08-27T11:28:06.667293+0000","last_fullsized":"2023-08-27T11:28:06.667293+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.673836+0000","last_change":"2023-08-27T11:28:06.672441+0000","last_active":"2023-08-27T11:28:06.673836+0000","last_peered":"2023-08-27T11:28:06.673836+0000","last_clean":"2023-08-27T11:28:06.673836+0000","last_became_active":"2023-08-27T11:28:06.672183+0000","last_became_peered":"2023-08-27T11:28:06.672183+0000","last_unstale":"2023-08-27T11:28:06.673836+0000","last_undegraded":"2023-08-27T11:28:06.673836+0000","last_fullsized":"2023-08-27T11:28:06.673836+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:23.790945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.677403+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:06.677403+0000","last_peered":"2023-08-27T11:28:06.677403+0000","last_clean":"2023-08-27T11:28:06.677403+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:06.677403+0000","last_undegraded":"2023-08-27T11:28:06.677403+0000","last_fullsized":"2023-08-27T11:28:06.677403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:13:06.409958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671744+0000","last_change":"2023-08-27T11:28:06.671744+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671744+0000","last_undegraded":"2023-08-27T11:28:06.671744+0000","last_fullsized":"2023-08-27T11:28:06.671744+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.670942+0000","last_change":"2023-08-27T11:28:06.666649+0000","last_active":"2023-08-27T11:28:06.670942+0000","last_peered":"2023-08-27T11:28:06.670942+0000","last_clean":"2023-08-27T11:28:06.670942+0000","last_became_active":"2023-08-27T11:28:06.666348+0000","last_became_peered":"2023-08-27T11:28:06.666348+0000","last_unstale":"2023-08-27T11:28:06.670942+0000","last_undegraded":"2023-08-27T11:28:06.670942+0000","last_fullsized":"2023-08-27T11:28:06.670942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:31:23.091962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.666080+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:06.666080+0000","last_peered":"2023-08-27T11:28:06.666080+0000","last_clean":"2023-08-27T11:28:06.666080+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:06.666080+0000","last_undegraded":"2023-08-27T11:28:06.666080+0000","last_fullsized":"2023-08-27T11:28:06.666080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:05.589881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664961+0000","last_change":"2023-08-27T11:28:06.664961+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664961+0000","last_undegraded":"2023-08-27T11:28:06.664961+0000","last_fullsized":"2023-08-27T11:28:06.664961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.684250+0000","last_change":"2023-08-27T11:28:06.683725+0000","last_active":"2023-08-27T11:28:06.684250+0000","last_peered":"2023-08-27T11:28:06.684250+0000","last_clean":"2023-08-27T11:28:06.684250+0000","last_became_active":"2023-08-27T11:28:06.683424+0000","last_became_peered":"2023-08-27T11:28:06.683424+0000","last_unstale":"2023-08-27T11:28:06.684250+0000","last_undegraded":"2023-08-27T11:28:06.684250+0000","last_fullsized":"2023-08-27T11:28:06.684250+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:13.388800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679319+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:06.679319+0000","last_peered":"2023-08-27T11:28:06.679319+0000","last_clean":"2023-08-27T11:28:06.679319+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:06.679319+0000","last_undegraded":"2023-08-27T11:28:06.679319+0000","last_fullsized":"2023-08-27T11:28:06.679319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:19:35.146879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2213,"ondisk_log_size":2213,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575041,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41252,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330588,"statfs":{"total":96636764160,"available":96594522112,"internally_reserved":0,"allocated":1794048,"data_stored":1217459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.38500000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":1.601,"15min":1.601},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.59999999999999998,"15min":0.58299999999999996},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.499}]},{"osd":1,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.71199999999999997,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.77000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.432}]},{"osd":2,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.80100000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.29099999999999998},"max":{"1min":0.749,"5min":1.778,"15min":1.778},"last":0.52100000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.65300000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.221},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68799999999999994,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.30499999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65300000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.25800000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.34499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575041,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.40799999999999997,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.76700000000000002,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.628},"min":{"1min":0.47399999999999998,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.3540000000000001},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":3.3780000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.80700000000000005,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.41699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.61699999999999999},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.73399999999999999,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":1.325},"last":0.44}]},{"osd":3,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.60499999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.71699999999999997,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.46400000000000002},"min":{"1min":0.312,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"min":{"1min":0.35199999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.624,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":1.157,"15min":1.157},"last":0.45200000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.70399999999999996,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.36899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40680,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331160,"statfs":{"total":96636764160,"available":96595107840,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20399999999999999},"max":{"1min":0.52100000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.495},{"interface":"front","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.628,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42699999999999999,"15min":0.42399999999999999},"min":{"1min":0.27500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.34100000000000003,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58999999999999997,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":1.169,"15min":1.169},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44600000000000001,"15min":0.44},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.23400000000000001},"max":{"1min":0.55900000000000005,"5min":1.139,"15min":1.139},"last":0.47299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.82599999999999996,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81499999999999995,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88700000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.59899999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89400000000000002,"15min":1.139},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575041,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41248,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330592,"statfs":{"total":96636764160,"available":96594526208,"internally_reserved":0,"allocated":1794048,"data_stored":1217459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.31},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.29399999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.26700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.374}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.41499999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.48999999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.39400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":620099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.25}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.31}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.433}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.48099999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.504},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.54700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40699999999999997,"15min":0.42499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.22},"max":{"1min":0.60799999999999998,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.224},"max":{"1min":0.60699999999999998,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.30499999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.64100000000000001,"15min":6.0830000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46200000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.245},"max":{"1min":0.57399999999999995,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.68000000000000005,"5min":0.71199999999999997,"15min":1.25},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":0.84499999999999997,"15min":11.939},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.376},"max":{"1min":0.83099999999999996,"5min":1.0329999999999999,"15min":11.007},"last":0.374},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":1.113,"15min":1.2929999999999999},"last":0.45100000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93300000000000005,"5min":3.0219999999999998,"15min":11.981},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":1.218,"15min":11.09},"last":0.498}]},{"osd":6,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69099999999999995,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.75600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":1.0269999999999999,"15min":12.016},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68999999999999995,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":0.96699999999999997,"15min":11.054},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.53500000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84099999999999997,"5min":1.0640000000000001,"15min":1.361},"last":0.47699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575041,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.32600000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.39400000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.504}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.47199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575040,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335340,"statfs":{"total":96636764160,"available":96599388160,"internally_reserved":0,"allocated":1187840,"data_stored":620099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.38},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.26000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.39300000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.34100000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.43099999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.42199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:11.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.007+0000 7fe38b4fc700 1 Processor -- start 2023-08-27T11:28:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.007+0000 7fe38b4fc700 1 -- start start 2023-08-27T11:28:12.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe38b4fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38415d3d0 0x7fe3841619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe38b4fc700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe3840af040 0x7fe3840cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe38b4fc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe384161550 con 0x7fe38406e3a0 2023-08-27T11:28:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe38b4fc700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3840cdf90 con 0x7fe3840af040 2023-08-27T11:28:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe38b4fc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe384161f20 con 0x7fe38415d3d0 2023-08-27T11:28:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe389a99700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe38406e3a0 0x7fe384070e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45892/0 (socket says 172.21.15.45:45892) 2023-08-27T11:28:12.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe389a99700 1 -- 172.21.15.45:0/881438772 learned_addr learned my addr 172.21.15.45:0/881438772 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:12.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe389298700 1 --2- 172.21.15.45:0/881438772 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38415d3d0 0x7fe3841619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe388a97700 1 --2- 172.21.15.45:0/881438772 >> v2:172.21.15.135:3300/0 conn(0x7fe3840af040 0x7fe3840cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe37bfff700 1 -- 172.21.15.45:0/881438772 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371676089 0 0) 0x7fe384161550 con 0x7fe38406e3a0 2023-08-27T11:28:12.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe37bfff700 1 -- 172.21.15.45:0/881438772 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3700035b0 con 0x7fe38406e3a0 2023-08-27T11:28:12.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe388a97700 1 -- 172.21.15.45:0/881438772 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38415d3d0 msgr2=0x7fe3841619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:12.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe388a97700 1 --2- 172.21.15.45:0/881438772 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38415d3d0 0x7fe3841619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe388a97700 1 -- 172.21.15.45:0/881438772 >> v1:172.21.15.45:6789/0 conn(0x7fe38406e3a0 legacy=0x7fe384070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.008+0000 7fe388a97700 1 -- 172.21.15.45:0/881438772 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840cd9d0 con 0x7fe3840af040 2023-08-27T11:28:12.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe389298700 1 --2- 172.21.15.45:0/881438772 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38415d3d0 0x7fe3841619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe388a97700 1 --2- 172.21.15.45:0/881438772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 0x7fe3840cf420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe374009840 tx=0x7fe374009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1087990c2abbdd2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe37bfff700 1 -- 172.21.15.45:0/881438772 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374011830 con 0x7fe3840af040 2023-08-27T11:28:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe37bfff700 1 -- 172.21.15.45:0/881438772 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374011990 con 0x7fe3840af040 2023-08-27T11:28:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe37bfff700 1 -- 172.21.15.45:0/881438772 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374011d20 con 0x7fe3840af040 2023-08-27T11:28:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe38b4fc700 1 -- 172.21.15.45:0/881438772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 msgr2=0x7fe3840cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe38b4fc700 1 --2- 172.21.15.45:0/881438772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 0x7fe3840cf420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe374009840 tx=0x7fe374009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe38b4fc700 1 -- 172.21.15.45:0/881438772 shutdown_connections 2023-08-27T11:28:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe38b4fc700 1 --2- 172.21.15.45:0/881438772 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38415d3d0 0x7fe3841619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe38b4fc700 1 --2- 172.21.15.45:0/881438772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 0x7fe3840cf420 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.009+0000 7fe38b4fc700 1 -- 172.21.15.45:0/881438772 >> 172.21.15.45:0/881438772 conn(0x7fe3840cfd60 msgr2=0x7fe3840d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:12.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.010+0000 7fe38b4fc700 1 -- 172.21.15.45:0/881438772 shutdown_connections 2023-08-27T11:28:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.010+0000 7fe38b4fc700 1 -- 172.21.15.45:0/881438772 wait complete. 2023-08-27T11:28:12.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.011+0000 7fe38b4fc700 1 Processor -- start 2023-08-27T11:28:12.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.011+0000 7fe38b4fc700 1 -- start start 2023-08-27T11:28:12.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.011+0000 7fe38b4fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38406e3a0 0x7fe3840b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.011+0000 7fe38b4fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 0x7fe38414dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.011+0000 7fe38b4fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38415d3d0 0x7fe38414e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.011+0000 7fe38b4fc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840c5f40 con 0x7fe38415d3d0 2023-08-27T11:28:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.011+0000 7fe38b4fc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840c6220 con 0x7fe3840af040 2023-08-27T11:28:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.011+0000 7fe38b4fc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3841489b0 con 0x7fe38406e3a0 2023-08-27T11:28:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389a99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38415d3d0 0x7fe38414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389a99700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38415d3d0 0x7fe38414e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59122/0 (socket says 172.21.15.45:59122) 2023-08-27T11:28:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389a99700 1 -- 172.21.15.45:0/102378221 learned_addr learned my addr 172.21.15.45:0/102378221 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe388a97700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 0x7fe38414dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389298700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38406e3a0 0x7fe3840b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389a99700 1 -- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38406e3a0 msgr2=0x7fe3840b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389a99700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38406e3a0 0x7fe3840b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389a99700 1 -- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 msgr2=0x7fe38414dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389a99700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 0x7fe38414dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389a99700 1 -- 172.21.15.45:0/102378221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3800094a0 con 0x7fe38415d3d0 2023-08-27T11:28:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe388a97700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 0x7fe38414dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe389a99700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38415d3d0 0x7fe38414e260 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00caa0 tx=0x7fe37c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe379ffb700 1 -- 172.21.15.45:0/102378221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00aa80 con 0x7fe38415d3d0 2023-08-27T11:28:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe379ffb700 1 -- 172.21.15.45:0/102378221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37c00abe0 con 0x7fe38415d3d0 2023-08-27T11:28:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3740094a0 con 0x7fe38415d3d0 2023-08-27T11:28:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.012+0000 7fe379ffb700 1 -- 172.21.15.45:0/102378221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c012400 con 0x7fe38415d3d0 2023-08-27T11:28:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.013+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3840b5690 con 0x7fe38415d3d0 2023-08-27T11:28:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.014+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3840b5880 con 0x7fe38415d3d0 2023-08-27T11:28:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.014+0000 7fe379ffb700 1 -- 172.21.15.45:0/102378221 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fe37c01b3b0 con 0x7fe38415d3d0 2023-08-27T11:28:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.014+0000 7fe379ffb700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fe36806a170 0x7fe36806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.014+0000 7fe379ffb700 1 -- 172.21.15.45:0/102378221 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe37c088450 con 0x7fe38415d3d0 2023-08-27T11:28:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.015+0000 7fe389298700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fe36806a170 0x7fe36806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.015+0000 7fe389298700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fe36806a170 0x7fe36806c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe380009c20 tx=0x7fe380008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.019+0000 7fe379ffb700 1 -- 172.21.15.45:0/102378221 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe37c0539e0 con 0x7fe38415d3d0 2023-08-27T11:28:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.223+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 --> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3841611e0 con 0x7fe36806a170 2023-08-27T11:28:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.232+0000 7fe379ffb700 1 -- 172.21.15.45:0/102378221 <== mgr.6709 v2:172.21.15.135:6832/3812189695 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306377 (secure 0 0 0) 0x7fe3800108a0 con 0x7fe36806a170 2023-08-27T11:28:12.238 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fe36806a170 msgr2=0x7fe36806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fe36806a170 0x7fe36806c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe380009c20 tx=0x7fe380008040 comp rx=0 tx=0).stop 2023-08-27T11:28:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38415d3d0 msgr2=0x7fe38414e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38415d3d0 0x7fe38414e260 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00caa0 tx=0x7fe37c00ce60 comp rx=0 tx=0).stop 2023-08-27T11:28:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 shutdown_connections 2023-08-27T11:28:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fe36806a170 0x7fe36806c630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38415d3d0 0x7fe38414e260 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe38406e3a0 0x7fe3840b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 --2- 172.21.15.45:0/102378221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3840af040 0x7fe38414dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 >> 172.21.15.45:0/102378221 conn(0x7fe3840cfd60 msgr2=0x7fe3840d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 shutdown_connections 2023-08-27T11:28:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.236+0000 7fe38b4fc700 1 -- 172.21.15.45:0/102378221 wait complete. 2023-08-27T11:28:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:12.250 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:28:11.501465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2213,"ondisk_log_size":2213,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284920,"kb_used_data":10512,"kb_used_omap":13,"kb_used_meta":274290,"kb_avail":754689800,"statfs":{"total":773094113280,"available":772802355200,"internally_reserved":0,"allocated":10764288,"data_stored":6183480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":280873048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.849363"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674262+0000","last_change":"2023-08-27T11:28:06.674262+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674262+0000","last_undegraded":"2023-08-27T11:28:06.674262+0000","last_fullsized":"2023-08-27T11:28:06.674262+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663991+0000","last_change":"2023-08-27T11:28:06.663991+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663991+0000","last_undegraded":"2023-08-27T11:28:06.663991+0000","last_fullsized":"2023-08-27T11:28:06.663991+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671195+0000","last_change":"2023-08-27T11:28:06.671195+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671195+0000","last_undegraded":"2023-08-27T11:28:06.671195+0000","last_fullsized":"2023-08-27T11:28:06.671195+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669142+0000","last_change":"2023-08-27T11:28:06.669142+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669142+0000","last_undegraded":"2023-08-27T11:28:06.669142+0000","last_fullsized":"2023-08-27T11:28:06.669142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668212+0000","last_change":"2023-08-27T11:28:06.668212+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668212+0000","last_undegraded":"2023-08-27T11:28:06.668212+0000","last_fullsized":"2023-08-27T11:28:06.668212+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670471+0000","last_change":"2023-08-27T11:28:06.670471+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670471+0000","last_undegraded":"2023-08-27T11:28:06.670471+0000","last_fullsized":"2023-08-27T11:28:06.670471+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667376+0000","last_change":"2023-08-27T11:28:06.667376+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667376+0000","last_undegraded":"2023-08-27T11:28:06.667376+0000","last_fullsized":"2023-08-27T11:28:06.667376+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670912+0000","last_change":"2023-08-27T11:28:06.670912+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670912+0000","last_undegraded":"2023-08-27T11:28:06.670912+0000","last_fullsized":"2023-08-27T11:28:06.670912+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669900+0000","last_change":"2023-08-27T11:28:06.669900+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669900+0000","last_undegraded":"2023-08-27T11:28:06.669900+0000","last_fullsized":"2023-08-27T11:28:06.669900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674502+0000","last_change":"2023-08-27T11:28:06.674502+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674502+0000","last_undegraded":"2023-08-27T11:28:06.674502+0000","last_fullsized":"2023-08-27T11:28:06.674502+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665233+0000","last_change":"2023-08-27T11:28:06.665233+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665233+0000","last_undegraded":"2023-08-27T11:28:06.665233+0000","last_fullsized":"2023-08-27T11:28:06.665233+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.677732+0000","last_change":"2023-08-27T11:28:06.677732+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.677732+0000","last_undegraded":"2023-08-27T11:28:06.677732+0000","last_fullsized":"2023-08-27T11:28:06.677732+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.672762+0000","last_change":"2023-08-27T11:28:06.672762+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.672762+0000","last_undegraded":"2023-08-27T11:28:06.672762+0000","last_fullsized":"2023-08-27T11:28:06.672762+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667659+0000","last_change":"2023-08-27T11:28:06.667659+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667659+0000","last_undegraded":"2023-08-27T11:28:06.667659+0000","last_fullsized":"2023-08-27T11:28:06.667659+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662713+0000","last_change":"2023-08-27T11:28:06.662713+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662713+0000","last_undegraded":"2023-08-27T11:28:06.662713+0000","last_fullsized":"2023-08-27T11:28:06.662713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667753+0000","last_change":"2023-08-27T11:28:06.667753+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667753+0000","last_undegraded":"2023-08-27T11:28:06.667753+0000","last_fullsized":"2023-08-27T11:28:06.667753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667875+0000","last_change":"2023-08-27T11:28:06.667875+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667875+0000","last_undegraded":"2023-08-27T11:28:06.667875+0000","last_fullsized":"2023-08-27T11:28:06.667875+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.666981+0000","last_change":"2023-08-27T11:28:06.666981+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.666981+0000","last_undegraded":"2023-08-27T11:28:06.666981+0000","last_fullsized":"2023-08-27T11:28:06.666981+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662798+0000","last_change":"2023-08-27T11:28:06.662798+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662798+0000","last_undegraded":"2023-08-27T11:28:06.662798+0000","last_fullsized":"2023-08-27T11:28:06.662798+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.666965+0000","last_change":"2023-08-27T11:28:06.666965+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.666965+0000","last_undegraded":"2023-08-27T11:28:06.666965+0000","last_fullsized":"2023-08-27T11:28:06.666965+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668224+0000","last_change":"2023-08-27T11:28:06.668224+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668224+0000","last_undegraded":"2023-08-27T11:28:06.668224+0000","last_fullsized":"2023-08-27T11:28:06.668224+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663693+0000","last_change":"2023-08-27T11:28:06.663693+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663693+0000","last_undegraded":"2023-08-27T11:28:06.663693+0000","last_fullsized":"2023-08-27T11:28:06.663693+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669584+0000","last_change":"2023-08-27T11:28:06.669584+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669584+0000","last_undegraded":"2023-08-27T11:28:06.669584+0000","last_fullsized":"2023-08-27T11:28:06.669584+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665390+0000","last_change":"2023-08-27T11:28:06.665390+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665390+0000","last_undegraded":"2023-08-27T11:28:06.665390+0000","last_fullsized":"2023-08-27T11:28:06.665390+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663341+0000","last_change":"2023-08-27T11:28:06.663341+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663341+0000","last_undegraded":"2023-08-27T11:28:06.663341+0000","last_fullsized":"2023-08-27T11:28:06.663341+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674565+0000","last_change":"2023-08-27T11:28:06.674565+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674565+0000","last_undegraded":"2023-08-27T11:28:06.674565+0000","last_fullsized":"2023-08-27T11:28:06.674565+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669801+0000","last_change":"2023-08-27T11:28:06.669801+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669801+0000","last_undegraded":"2023-08-27T11:28:06.669801+0000","last_fullsized":"2023-08-27T11:28:06.669801+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.677428+0000","last_change":"2023-08-27T11:28:06.677428+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.677428+0000","last_undegraded":"2023-08-27T11:28:06.677428+0000","last_fullsized":"2023-08-27T11:28:06.677428+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662770+0000","last_change":"2023-08-27T11:28:06.662770+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662770+0000","last_undegraded":"2023-08-27T11:28:06.662770+0000","last_fullsized":"2023-08-27T11:28:06.662770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662917+0000","last_change":"2023-08-27T11:28:06.662917+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662917+0000","last_undegraded":"2023-08-27T11:28:06.662917+0000","last_fullsized":"2023-08-27T11:28:06.662917+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667437+0000","last_change":"2023-08-27T11:28:06.667437+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667437+0000","last_undegraded":"2023-08-27T11:28:06.667437+0000","last_fullsized":"2023-08-27T11:28:06.667437+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674563+0000","last_change":"2023-08-27T11:28:06.674563+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674563+0000","last_undegraded":"2023-08-27T11:28:06.674563+0000","last_fullsized":"2023-08-27T11:28:06.674563+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671268+0000","last_change":"2023-08-27T11:28:06.671268+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671268+0000","last_undegraded":"2023-08-27T11:28:06.671268+0000","last_fullsized":"2023-08-27T11:28:06.671268+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682500+0000","last_change":"2023-08-27T11:28:06.680510+0000","last_active":"2023-08-27T11:28:06.682500+0000","last_peered":"2023-08-27T11:28:06.682500+0000","last_clean":"2023-08-27T11:28:06.682500+0000","last_became_active":"2023-08-27T11:28:06.680268+0000","last_became_peered":"2023-08-27T11:28:06.680268+0000","last_unstale":"2023-08-27T11:28:06.682500+0000","last_undegraded":"2023-08-27T11:28:06.682500+0000","last_fullsized":"2023-08-27T11:28:06.682500+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:53.355949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673997+0000","last_change":"2023-08-27T11:28:06.673997+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673997+0000","last_undegraded":"2023-08-27T11:28:06.673997+0000","last_fullsized":"2023-08-27T11:28:06.673997+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680586+0000","last_change":"2023-08-27T11:28:06.678558+0000","last_active":"2023-08-27T11:28:06.680586+0000","last_peered":"2023-08-27T11:28:06.680586+0000","last_clean":"2023-08-27T11:28:06.680586+0000","last_became_active":"2023-08-27T11:28:06.678323+0000","last_became_peered":"2023-08-27T11:28:06.678323+0000","last_unstale":"2023-08-27T11:28:06.680586+0000","last_undegraded":"2023-08-27T11:28:06.680586+0000","last_fullsized":"2023-08-27T11:28:06.680586+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:32:33.684096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673970+0000","last_change":"2023-08-27T11:28:06.673970+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673970+0000","last_undegraded":"2023-08-27T11:28:06.673970+0000","last_fullsized":"2023-08-27T11:28:06.673970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675092+0000","last_change":"2023-08-27T11:28:06.674106+0000","last_active":"2023-08-27T11:28:06.675092+0000","last_peered":"2023-08-27T11:28:06.675092+0000","last_clean":"2023-08-27T11:28:06.675092+0000","last_became_active":"2023-08-27T11:28:06.673843+0000","last_became_peered":"2023-08-27T11:28:06.673843+0000","last_unstale":"2023-08-27T11:28:06.675092+0000","last_undegraded":"2023-08-27T11:28:06.675092+0000","last_fullsized":"2023-08-27T11:28:06.675092+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:18.098063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670630+0000","last_change":"2023-08-27T11:28:06.670630+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670630+0000","last_undegraded":"2023-08-27T11:28:06.670630+0000","last_fullsized":"2023-08-27T11:28:06.670630+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675059+0000","last_change":"2023-08-27T11:28:06.674658+0000","last_active":"2023-08-27T11:28:06.675059+0000","last_peered":"2023-08-27T11:28:06.675059+0000","last_clean":"2023-08-27T11:28:06.675059+0000","last_became_active":"2023-08-27T11:28:06.674384+0000","last_became_peered":"2023-08-27T11:28:06.674384+0000","last_unstale":"2023-08-27T11:28:06.675059+0000","last_undegraded":"2023-08-27T11:28:06.675059+0000","last_fullsized":"2023-08-27T11:28:06.675059+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:33:38.254409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663309+0000","last_change":"2023-08-27T11:28:06.663309+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663309+0000","last_undegraded":"2023-08-27T11:28:06.663309+0000","last_fullsized":"2023-08-27T11:28:06.663309+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680562+0000","last_change":"2023-08-27T11:28:06.678516+0000","last_active":"2023-08-27T11:28:06.680562+0000","last_peered":"2023-08-27T11:28:06.680562+0000","last_clean":"2023-08-27T11:28:06.680562+0000","last_became_active":"2023-08-27T11:28:06.678268+0000","last_became_peered":"2023-08-27T11:28:06.678268+0000","last_unstale":"2023-08-27T11:28:06.680562+0000","last_undegraded":"2023-08-27T11:28:06.680562+0000","last_fullsized":"2023-08-27T11:28:06.680562+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:29:56.120623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667804+0000","last_change":"2023-08-27T11:28:06.667804+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667804+0000","last_undegraded":"2023-08-27T11:28:06.667804+0000","last_fullsized":"2023-08-27T11:28:06.667804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680480+0000","last_change":"2023-08-27T11:28:06.677993+0000","last_active":"2023-08-27T11:28:06.680480+0000","last_peered":"2023-08-27T11:28:06.680480+0000","last_clean":"2023-08-27T11:28:06.680480+0000","last_became_active":"2023-08-27T11:28:06.677679+0000","last_became_peered":"2023-08-27T11:28:06.677679+0000","last_unstale":"2023-08-27T11:28:06.680480+0000","last_undegraded":"2023-08-27T11:28:06.680480+0000","last_fullsized":"2023-08-27T11:28:06.680480+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:04.340118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670493+0000","last_change":"2023-08-27T11:28:06.670493+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670493+0000","last_undegraded":"2023-08-27T11:28:06.670493+0000","last_fullsized":"2023-08-27T11:28:06.670493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676184+0000","last_change":"2023-08-27T11:28:06.675717+0000","last_active":"2023-08-27T11:28:06.676184+0000","last_peered":"2023-08-27T11:28:06.676184+0000","last_clean":"2023-08-27T11:28:06.676184+0000","last_became_active":"2023-08-27T11:28:06.675161+0000","last_became_peered":"2023-08-27T11:28:06.675161+0000","last_unstale":"2023-08-27T11:28:06.676184+0000","last_undegraded":"2023-08-27T11:28:06.676184+0000","last_fullsized":"2023-08-27T11:28:06.676184+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:30.138559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662304+0000","last_change":"2023-08-27T11:28:06.662304+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662304+0000","last_undegraded":"2023-08-27T11:28:06.662304+0000","last_fullsized":"2023-08-27T11:28:06.662304+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680689+0000","last_change":"2023-08-27T11:28:06.680053+0000","last_active":"2023-08-27T11:28:06.680689+0000","last_peered":"2023-08-27T11:28:06.680689+0000","last_clean":"2023-08-27T11:28:06.680689+0000","last_became_active":"2023-08-27T11:28:06.679714+0000","last_became_peered":"2023-08-27T11:28:06.679714+0000","last_unstale":"2023-08-27T11:28:06.680689+0000","last_undegraded":"2023-08-27T11:28:06.680689+0000","last_fullsized":"2023-08-27T11:28:06.680689+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:13.718165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.661902+0000","last_change":"2023-08-27T11:28:06.661902+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.661902+0000","last_undegraded":"2023-08-27T11:28:06.661902+0000","last_fullsized":"2023-08-27T11:28:06.661902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.683222+0000","last_change":"2023-08-27T11:28:06.682417+0000","last_active":"2023-08-27T11:28:06.683222+0000","last_peered":"2023-08-27T11:28:06.683222+0000","last_clean":"2023-08-27T11:28:06.683222+0000","last_became_active":"2023-08-27T11:28:06.682174+0000","last_became_peered":"2023-08-27T11:28:06.682174+0000","last_unstale":"2023-08-27T11:28:06.683222+0000","last_undegraded":"2023-08-27T11:28:06.683222+0000","last_fullsized":"2023-08-27T11:28:06.683222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:22.047442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669447+0000","last_change":"2023-08-27T11:28:06.669447+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669447+0000","last_undegraded":"2023-08-27T11:28:06.669447+0000","last_fullsized":"2023-08-27T11:28:06.669447+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.683629+0000","last_change":"2023-08-27T11:28:06.682913+0000","last_active":"2023-08-27T11:28:06.683629+0000","last_peered":"2023-08-27T11:28:06.683629+0000","last_clean":"2023-08-27T11:28:06.683629+0000","last_became_active":"2023-08-27T11:28:06.682773+0000","last_became_peered":"2023-08-27T11:28:06.682773+0000","last_unstale":"2023-08-27T11:28:06.683629+0000","last_undegraded":"2023-08-27T11:28:06.683629+0000","last_fullsized":"2023-08-27T11:28:06.683629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:13.142404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662935+0000","last_change":"2023-08-27T11:28:06.662935+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662935+0000","last_undegraded":"2023-08-27T11:28:06.662935+0000","last_fullsized":"2023-08-27T11:28:06.662935+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682449+0000","last_change":"2023-08-27T11:28:06.680796+0000","last_active":"2023-08-27T11:28:06.682449+0000","last_peered":"2023-08-27T11:28:06.682449+0000","last_clean":"2023-08-27T11:28:06.682449+0000","last_became_active":"2023-08-27T11:28:06.680639+0000","last_became_peered":"2023-08-27T11:28:06.680639+0000","last_unstale":"2023-08-27T11:28:06.682449+0000","last_undegraded":"2023-08-27T11:28:06.682449+0000","last_fullsized":"2023-08-27T11:28:06.682449+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:54.607978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669454+0000","last_change":"2023-08-27T11:28:06.669454+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669454+0000","last_undegraded":"2023-08-27T11:28:06.669454+0000","last_fullsized":"2023-08-27T11:28:06.669454+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679081+0000","last_change":"2023-08-27T11:28:06.675117+0000","last_active":"2023-08-27T11:28:06.679081+0000","last_peered":"2023-08-27T11:28:06.679081+0000","last_clean":"2023-08-27T11:28:06.679081+0000","last_became_active":"2023-08-27T11:28:06.674808+0000","last_became_peered":"2023-08-27T11:28:06.674808+0000","last_unstale":"2023-08-27T11:28:06.679081+0000","last_undegraded":"2023-08-27T11:28:06.679081+0000","last_fullsized":"2023-08-27T11:28:06.679081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:13.763908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663233+0000","last_change":"2023-08-27T11:28:06.663233+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663233+0000","last_undegraded":"2023-08-27T11:28:06.663233+0000","last_fullsized":"2023-08-27T11:28:06.663233+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.673619+0000","last_change":"2023-08-27T11:28:06.670804+0000","last_active":"2023-08-27T11:28:06.673619+0000","last_peered":"2023-08-27T11:28:06.673619+0000","last_clean":"2023-08-27T11:28:06.673619+0000","last_became_active":"2023-08-27T11:28:06.669783+0000","last_became_peered":"2023-08-27T11:28:06.669783+0000","last_unstale":"2023-08-27T11:28:06.673619+0000","last_undegraded":"2023-08-27T11:28:06.673619+0000","last_fullsized":"2023-08-27T11:28:06.673619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:54.035323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.672405+0000","last_change":"2023-08-27T11:28:06.672405+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.672405+0000","last_undegraded":"2023-08-27T11:28:06.672405+0000","last_fullsized":"2023-08-27T11:28:06.672405+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672817+0000","last_change":"2023-08-27T11:28:06.668164+0000","last_active":"2023-08-27T11:28:06.672817+0000","last_peered":"2023-08-27T11:28:06.672817+0000","last_clean":"2023-08-27T11:28:06.672817+0000","last_became_active":"2023-08-27T11:28:06.667870+0000","last_became_peered":"2023-08-27T11:28:06.667870+0000","last_unstale":"2023-08-27T11:28:06.672817+0000","last_undegraded":"2023-08-27T11:28:06.672817+0000","last_fullsized":"2023-08-27T11:28:06.672817+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:34.544721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664777+0000","last_change":"2023-08-27T11:28:06.664777+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664777+0000","last_undegraded":"2023-08-27T11:28:06.664777+0000","last_fullsized":"2023-08-27T11:28:06.664777+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.681660+0000","last_change":"2023-08-27T11:28:06.679870+0000","last_active":"2023-08-27T11:28:06.681660+0000","last_peered":"2023-08-27T11:28:06.681660+0000","last_clean":"2023-08-27T11:28:06.681660+0000","last_became_active":"2023-08-27T11:28:06.679627+0000","last_became_peered":"2023-08-27T11:28:06.679627+0000","last_unstale":"2023-08-27T11:28:06.681660+0000","last_undegraded":"2023-08-27T11:28:06.681660+0000","last_fullsized":"2023-08-27T11:28:06.681660+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:35.146679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665638+0000","last_change":"2023-08-27T11:28:06.665638+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665638+0000","last_undegraded":"2023-08-27T11:28:06.665638+0000","last_fullsized":"2023-08-27T11:28:06.665638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682022+0000","last_change":"2023-08-27T11:28:06.681196+0000","last_active":"2023-08-27T11:28:06.682022+0000","last_peered":"2023-08-27T11:28:06.682022+0000","last_clean":"2023-08-27T11:28:06.682022+0000","last_became_active":"2023-08-27T11:28:06.680914+0000","last_became_peered":"2023-08-27T11:28:06.680914+0000","last_unstale":"2023-08-27T11:28:06.682022+0000","last_undegraded":"2023-08-27T11:28:06.682022+0000","last_fullsized":"2023-08-27T11:28:06.682022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:12.051340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670514+0000","last_change":"2023-08-27T11:28:06.670514+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670514+0000","last_undegraded":"2023-08-27T11:28:06.670514+0000","last_fullsized":"2023-08-27T11:28:06.670514+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680425+0000","last_change":"2023-08-27T11:28:06.679947+0000","last_active":"2023-08-27T11:28:06.680425+0000","last_peered":"2023-08-27T11:28:06.680425+0000","last_clean":"2023-08-27T11:28:06.680425+0000","last_became_active":"2023-08-27T11:28:06.679654+0000","last_became_peered":"2023-08-27T11:28:06.679654+0000","last_unstale":"2023-08-27T11:28:06.680425+0000","last_undegraded":"2023-08-27T11:28:06.680425+0000","last_fullsized":"2023-08-27T11:28:06.680425+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:36.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671123+0000","last_change":"2023-08-27T11:28:06.671123+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671123+0000","last_undegraded":"2023-08-27T11:28:06.671123+0000","last_fullsized":"2023-08-27T11:28:06.671123+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672339+0000","last_change":"2023-08-27T11:28:06.668640+0000","last_active":"2023-08-27T11:28:06.672339+0000","last_peered":"2023-08-27T11:28:06.672339+0000","last_clean":"2023-08-27T11:28:06.672339+0000","last_became_active":"2023-08-27T11:28:06.668372+0000","last_became_peered":"2023-08-27T11:28:06.668372+0000","last_unstale":"2023-08-27T11:28:06.672339+0000","last_undegraded":"2023-08-27T11:28:06.672339+0000","last_fullsized":"2023-08-27T11:28:06.672339+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:48:01.533647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673604+0000","last_change":"2023-08-27T11:28:06.673604+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673604+0000","last_undegraded":"2023-08-27T11:28:06.673604+0000","last_fullsized":"2023-08-27T11:28:06.673604+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682683+0000","last_change":"2023-08-27T11:28:06.682044+0000","last_active":"2023-08-27T11:28:06.682683+0000","last_peered":"2023-08-27T11:28:06.682683+0000","last_clean":"2023-08-27T11:28:06.682683+0000","last_became_active":"2023-08-27T11:28:06.681789+0000","last_became_peered":"2023-08-27T11:28:06.681789+0000","last_unstale":"2023-08-27T11:28:06.682683+0000","last_undegraded":"2023-08-27T11:28:06.682683+0000","last_fullsized":"2023-08-27T11:28:06.682683+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:39:11.072746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668771+0000","last_change":"2023-08-27T11:28:06.668771+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668771+0000","last_undegraded":"2023-08-27T11:28:06.668771+0000","last_fullsized":"2023-08-27T11:28:06.668771+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682726+0000","last_change":"2023-08-27T11:28:06.682213+0000","last_active":"2023-08-27T11:28:06.682726+0000","last_peered":"2023-08-27T11:28:06.682726+0000","last_clean":"2023-08-27T11:28:06.682726+0000","last_became_active":"2023-08-27T11:28:06.681772+0000","last_became_peered":"2023-08-27T11:28:06.681772+0000","last_unstale":"2023-08-27T11:28:06.682726+0000","last_undegraded":"2023-08-27T11:28:06.682726+0000","last_fullsized":"2023-08-27T11:28:06.682726+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:34:15.373195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670120+0000","last_change":"2023-08-27T11:28:06.670120+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670120+0000","last_undegraded":"2023-08-27T11:28:06.670120+0000","last_fullsized":"2023-08-27T11:28:06.670120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682020+0000","last_change":"2023-08-27T11:28:06.681163+0000","last_active":"2023-08-27T11:28:06.682020+0000","last_peered":"2023-08-27T11:28:06.682020+0000","last_clean":"2023-08-27T11:28:06.682020+0000","last_became_active":"2023-08-27T11:28:06.680922+0000","last_became_peered":"2023-08-27T11:28:06.680922+0000","last_unstale":"2023-08-27T11:28:06.682020+0000","last_undegraded":"2023-08-27T11:28:06.682020+0000","last_fullsized":"2023-08-27T11:28:06.682020+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:13:20.626473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664201+0000","last_change":"2023-08-27T11:28:06.664201+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664201+0000","last_undegraded":"2023-08-27T11:28:06.664201+0000","last_fullsized":"2023-08-27T11:28:06.664201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680463+0000","last_change":"2023-08-27T11:28:06.677834+0000","last_active":"2023-08-27T11:28:06.680463+0000","last_peered":"2023-08-27T11:28:06.680463+0000","last_clean":"2023-08-27T11:28:06.680463+0000","last_became_active":"2023-08-27T11:28:06.677483+0000","last_became_peered":"2023-08-27T11:28:06.677483+0000","last_unstale":"2023-08-27T11:28:06.680463+0000","last_undegraded":"2023-08-27T11:28:06.680463+0000","last_fullsized":"2023-08-27T11:28:06.680463+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:33:20.246978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670946+0000","last_change":"2023-08-27T11:28:06.670946+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670946+0000","last_undegraded":"2023-08-27T11:28:06.670946+0000","last_fullsized":"2023-08-27T11:28:06.670946+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682708+0000","last_change":"2023-08-27T11:28:06.682127+0000","last_active":"2023-08-27T11:28:06.682708+0000","last_peered":"2023-08-27T11:28:06.682708+0000","last_clean":"2023-08-27T11:28:06.682708+0000","last_became_active":"2023-08-27T11:28:06.681824+0000","last_became_peered":"2023-08-27T11:28:06.681824+0000","last_unstale":"2023-08-27T11:28:06.682708+0000","last_undegraded":"2023-08-27T11:28:06.682708+0000","last_fullsized":"2023-08-27T11:28:06.682708+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:03.058354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669155+0000","last_change":"2023-08-27T11:28:06.669155+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669155+0000","last_undegraded":"2023-08-27T11:28:06.669155+0000","last_fullsized":"2023-08-27T11:28:06.669155+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679086+0000","last_change":"2023-08-27T11:28:06.676211+0000","last_active":"2023-08-27T11:28:06.679086+0000","last_peered":"2023-08-27T11:28:06.679086+0000","last_clean":"2023-08-27T11:28:06.679086+0000","last_became_active":"2023-08-27T11:28:06.675956+0000","last_became_peered":"2023-08-27T11:28:06.675956+0000","last_unstale":"2023-08-27T11:28:06.679086+0000","last_undegraded":"2023-08-27T11:28:06.679086+0000","last_fullsized":"2023-08-27T11:28:06.679086+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:48:39.217804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669642+0000","last_change":"2023-08-27T11:28:06.669642+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669642+0000","last_undegraded":"2023-08-27T11:28:06.669642+0000","last_fullsized":"2023-08-27T11:28:06.669642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676840+0000","last_change":"2023-08-27T11:28:06.673096+0000","last_active":"2023-08-27T11:28:06.676840+0000","last_peered":"2023-08-27T11:28:06.676840+0000","last_clean":"2023-08-27T11:28:06.676840+0000","last_became_active":"2023-08-27T11:28:06.672819+0000","last_became_peered":"2023-08-27T11:28:06.672819+0000","last_unstale":"2023-08-27T11:28:06.676840+0000","last_undegraded":"2023-08-27T11:28:06.676840+0000","last_fullsized":"2023-08-27T11:28:06.676840+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:34.207300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.666042+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:06.666042+0000","last_peered":"2023-08-27T11:28:06.666042+0000","last_clean":"2023-08-27T11:28:06.666042+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:06.666042+0000","last_undegraded":"2023-08-27T11:28:06.666042+0000","last_fullsized":"2023-08-27T11:28:06.666042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:28.335693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669129+0000","last_change":"2023-08-27T11:28:06.669129+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669129+0000","last_undegraded":"2023-08-27T11:28:06.669129+0000","last_fullsized":"2023-08-27T11:28:06.669129+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676890+0000","last_change":"2023-08-27T11:28:06.676306+0000","last_active":"2023-08-27T11:28:06.676890+0000","last_peered":"2023-08-27T11:28:06.676890+0000","last_clean":"2023-08-27T11:28:06.676890+0000","last_became_active":"2023-08-27T11:28:06.676049+0000","last_became_peered":"2023-08-27T11:28:06.676049+0000","last_unstale":"2023-08-27T11:28:06.676890+0000","last_undegraded":"2023-08-27T11:28:06.676890+0000","last_fullsized":"2023-08-27T11:28:06.676890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:52.590584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672445+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:06.672445+0000","last_peered":"2023-08-27T11:28:06.672445+0000","last_clean":"2023-08-27T11:28:06.672445+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:06.672445+0000","last_undegraded":"2023-08-27T11:28:06.672445+0000","last_fullsized":"2023-08-27T11:28:06.672445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:04.382696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670944+0000","last_change":"2023-08-27T11:28:06.670944+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670944+0000","last_undegraded":"2023-08-27T11:28:06.670944+0000","last_fullsized":"2023-08-27T11:28:06.670944+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682490+0000","last_change":"2023-08-27T11:28:06.680009+0000","last_active":"2023-08-27T11:28:06.682490+0000","last_peered":"2023-08-27T11:28:06.682490+0000","last_clean":"2023-08-27T11:28:06.682490+0000","last_became_active":"2023-08-27T11:28:06.679754+0000","last_became_peered":"2023-08-27T11:28:06.679754+0000","last_unstale":"2023-08-27T11:28:06.682490+0000","last_undegraded":"2023-08-27T11:28:06.682490+0000","last_fullsized":"2023-08-27T11:28:06.682490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:06.046171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":154,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.670537+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:06.670537+0000","last_peered":"2023-08-27T11:28:06.670537+0000","last_clean":"2023-08-27T11:28:06.670537+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:06.670537+0000","last_undegraded":"2023-08-27T11:28:06.670537+0000","last_fullsized":"2023-08-27T11:28:06.670537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:46:09.042939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664810+0000","last_change":"2023-08-27T11:28:06.664810+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664810+0000","last_undegraded":"2023-08-27T11:28:06.664810+0000","last_fullsized":"2023-08-27T11:28:06.664810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.681624+0000","last_change":"2023-08-27T11:28:06.680561+0000","last_active":"2023-08-27T11:28:06.681624+0000","last_peered":"2023-08-27T11:28:06.681624+0000","last_clean":"2023-08-27T11:28:06.681624+0000","last_became_active":"2023-08-27T11:28:06.680307+0000","last_became_peered":"2023-08-27T11:28:06.680307+0000","last_unstale":"2023-08-27T11:28:06.681624+0000","last_undegraded":"2023-08-27T11:28:06.681624+0000","last_fullsized":"2023-08-27T11:28:06.681624+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:58.720509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672903+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:06.672903+0000","last_peered":"2023-08-27T11:28:06.672903+0000","last_clean":"2023-08-27T11:28:06.672903+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:06.672903+0000","last_undegraded":"2023-08-27T11:28:06.672903+0000","last_fullsized":"2023-08-27T11:28:06.672903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:40:14.646059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"98'2213","reported_seq":2572,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:07.501955+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:07.501955+0000","last_peered":"2023-08-27T11:28:07.501955+0000","last_clean":"2023-08-27T11:28:07.501955+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:07.501955+0000","last_undegraded":"2023-08-27T11:28:07.501955+0000","last_fullsized":"2023-08-27T11:28:07.501955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":2213,"log_dups_size":0,"ondisk_log_size":2213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:54:12.932795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673646+0000","last_change":"2023-08-27T11:28:06.673646+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673646+0000","last_undegraded":"2023-08-27T11:28:06.673646+0000","last_fullsized":"2023-08-27T11:28:06.673646+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680214+0000","last_change":"2023-08-27T11:28:06.678931+0000","last_active":"2023-08-27T11:28:06.680214+0000","last_peered":"2023-08-27T11:28:06.680214+0000","last_clean":"2023-08-27T11:28:06.680214+0000","last_became_active":"2023-08-27T11:28:06.678661+0000","last_became_peered":"2023-08-27T11:28:06.678661+0000","last_unstale":"2023-08-27T11:28:06.680214+0000","last_undegraded":"2023-08-27T11:28:06.680214+0000","last_fullsized":"2023-08-27T11:28:06.680214+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:00:05.382377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675655+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:06.675655+0000","last_peered":"2023-08-27T11:28:06.675655+0000","last_clean":"2023-08-27T11:28:06.675655+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:06.675655+0000","last_undegraded":"2023-08-27T11:28:06.675655+0000","last_fullsized":"2023-08-27T11:28:06.675655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:10:39.559849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667293+0000","last_change":"2023-08-27T11:28:06.667293+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667293+0000","last_undegraded":"2023-08-27T11:28:06.667293+0000","last_fullsized":"2023-08-27T11:28:06.667293+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.673836+0000","last_change":"2023-08-27T11:28:06.672441+0000","last_active":"2023-08-27T11:28:06.673836+0000","last_peered":"2023-08-27T11:28:06.673836+0000","last_clean":"2023-08-27T11:28:06.673836+0000","last_became_active":"2023-08-27T11:28:06.672183+0000","last_became_peered":"2023-08-27T11:28:06.672183+0000","last_unstale":"2023-08-27T11:28:06.673836+0000","last_undegraded":"2023-08-27T11:28:06.673836+0000","last_fullsized":"2023-08-27T11:28:06.673836+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:23.790945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.677403+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:06.677403+0000","last_peered":"2023-08-27T11:28:06.677403+0000","last_clean":"2023-08-27T11:28:06.677403+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:06.677403+0000","last_undegraded":"2023-08-27T11:28:06.677403+0000","last_fullsized":"2023-08-27T11:28:06.677403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:13:06.409958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671744+0000","last_change":"2023-08-27T11:28:06.671744+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671744+0000","last_undegraded":"2023-08-27T11:28:06.671744+0000","last_fullsized":"2023-08-27T11:28:06.671744+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.670942+0000","last_change":"2023-08-27T11:28:06.666649+0000","last_active":"2023-08-27T11:28:06.670942+0000","last_peered":"2023-08-27T11:28:06.670942+0000","last_clean":"2023-08-27T11:28:06.670942+0000","last_became_active":"2023-08-27T11:28:06.666348+0000","last_became_peered":"2023-08-27T11:28:06.666348+0000","last_unstale":"2023-08-27T11:28:06.670942+0000","last_undegraded":"2023-08-27T11:28:06.670942+0000","last_fullsized":"2023-08-27T11:28:06.670942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:31:23.091962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.666080+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:06.666080+0000","last_peered":"2023-08-27T11:28:06.666080+0000","last_clean":"2023-08-27T11:28:06.666080+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:06.666080+0000","last_undegraded":"2023-08-27T11:28:06.666080+0000","last_fullsized":"2023-08-27T11:28:06.666080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:05.589881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664961+0000","last_change":"2023-08-27T11:28:06.664961+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664961+0000","last_undegraded":"2023-08-27T11:28:06.664961+0000","last_fullsized":"2023-08-27T11:28:06.664961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.684250+0000","last_change":"2023-08-27T11:28:06.683725+0000","last_active":"2023-08-27T11:28:06.684250+0000","last_peered":"2023-08-27T11:28:06.684250+0000","last_clean":"2023-08-27T11:28:06.684250+0000","last_became_active":"2023-08-27T11:28:06.683424+0000","last_became_peered":"2023-08-27T11:28:06.683424+0000","last_unstale":"2023-08-27T11:28:06.684250+0000","last_undegraded":"2023-08-27T11:28:06.684250+0000","last_fullsized":"2023-08-27T11:28:06.684250+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:13.388800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679319+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:06.679319+0000","last_peered":"2023-08-27T11:28:06.679319+0000","last_clean":"2023-08-27T11:28:06.679319+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:06.679319+0000","last_undegraded":"2023-08-27T11:28:06.679319+0000","last_fullsized":"2023-08-27T11:28:06.679319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:19:35.146879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2213,"ondisk_log_size":2213,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575041,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41252,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330588,"statfs":{"total":96636764160,"available":96594522112,"internally_reserved":0,"allocated":1794048,"data_stored":1217459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.38500000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":1.601,"15min":1.601},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.59999999999999998,"15min":0.58299999999999996},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.499}]},{"osd":1,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.71199999999999997,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.77000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.432}]},{"osd":2,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.80100000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.29099999999999998},"max":{"1min":0.749,"5min":1.778,"15min":1.778},"last":0.52100000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.65300000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.221},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68799999999999994,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.30499999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65300000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.25800000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.34499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575041,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.40799999999999997,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.76700000000000002,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.628},"min":{"1min":0.47399999999999998,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.3540000000000001},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":3.3780000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.80700000000000005,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.41699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.61699999999999999},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.73399999999999999,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":1.325},"last":0.44}]},{"osd":3,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.60499999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.71699999999999997,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.46400000000000002},"min":{"1min":0.312,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"min":{"1min":0.35199999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.624,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":1.157,"15min":1.157},"last":0.45200000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.70399999999999996,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.36899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40680,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331160,"statfs":{"total":96636764160,"available":96595107840,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20399999999999999},"max":{"1min":0.52100000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.495},{"interface":"front","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.628,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42699999999999999,"15min":0.42399999999999999},"min":{"1min":0.27500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.34100000000000003,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58999999999999997,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":1.169,"15min":1.169},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44600000000000001,"15min":0.44},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.23400000000000001},"max":{"1min":0.55900000000000005,"5min":1.139,"15min":1.139},"last":0.47299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.82599999999999996,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81499999999999995,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88700000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.59899999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89400000000000002,"15min":1.139},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575041,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41248,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330592,"statfs":{"total":96636764160,"available":96594526208,"internally_reserved":0,"allocated":1794048,"data_stored":1217459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.31},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.29399999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.26700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.374}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.41499999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.48999999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.39400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":620099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.25}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.31}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.433}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.48099999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.504},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.54700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40699999999999997,"15min":0.42499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.22},"max":{"1min":0.60799999999999998,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.224},"max":{"1min":0.60699999999999998,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.30499999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.64100000000000001,"15min":6.0830000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46200000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.245},"max":{"1min":0.57399999999999995,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.68000000000000005,"5min":0.71199999999999997,"15min":1.25},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":0.84499999999999997,"15min":11.939},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.376},"max":{"1min":0.83099999999999996,"5min":1.0329999999999999,"15min":11.007},"last":0.374},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":1.113,"15min":1.2929999999999999},"last":0.45100000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93300000000000005,"5min":3.0219999999999998,"15min":11.981},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":1.218,"15min":11.09},"last":0.498}]},{"osd":6,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69099999999999995,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.75600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":1.0269999999999999,"15min":12.016},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68999999999999995,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":0.96699999999999997,"15min":11.054},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.53500000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84099999999999997,"5min":1.0640000000000001,"15min":1.361},"last":0.47699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575041,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.32600000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.39400000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.504}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.47199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575040,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335340,"statfs":{"total":96636764160,"available":96599388160,"internally_reserved":0,"allocated":1187840,"data_stored":620099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.38},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.26000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.39300000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.34100000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.43099999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.42199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:12.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:12.303 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:12.301+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:12.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.414+0000 7fb640a5d700 1 Processor -- start 2023-08-27T11:28:12.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.414+0000 7fb640a5d700 1 -- start start 2023-08-27T11:28:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb640a5d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb63c0af040 0x7fb63c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb640a5d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c06e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb640a5d700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c161550 con 0x7fb63c15d3d0 2023-08-27T11:28:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb640a5d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0cdf90 con 0x7fb63c0af040 2023-08-27T11:28:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb640a5d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c06ed70 con 0x7fb63c06e3a0 2023-08-27T11:28:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb639d9b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb63c0af040 0x7fb63c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb63ad9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c06e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb63ad9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c06e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:57040/0 (socket says 172.21.15.45:57040) 2023-08-27T11:28:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb639d9b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb63c0af040 0x7fb63c0cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:47984/0 (socket says 172.21.15.45:47984) 2023-08-27T11:28:12.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb63ad9d700 1 -- 172.21.15.45:0/2507721598 learned_addr learned my addr 172.21.15.45:0/2507721598 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:12.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.415+0000 7fb63a59c700 1 --1- 172.21.15.45:0/2507721598 >> v1:172.21.15.135:6789/0 conn(0x7fb63c15d3d0 0x7fb63c1619e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:41566/0 (socket says 172.21.15.45:41566) 2023-08-27T11:28:12.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb63ad9d700 1 -- 172.21.15.45:0/2507721598 >> v2:172.21.15.135:3300/0 conn(0x7fb63c0af040 msgr2=0x7fb63c0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:12.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb63ad9d700 1 --2- 172.21.15.45:0/2507721598 >> v2:172.21.15.135:3300/0 conn(0x7fb63c0af040 0x7fb63c0cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb63ad9d700 1 -- 172.21.15.45:0/2507721598 >> v1:172.21.15.135:6789/0 conn(0x7fb63c15d3d0 legacy=0x7fb63c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb63959a700 1 -- 172.21.15.45:0/2507721598 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291925939 0 0) 0x7fb63c161550 con 0x7fb63c15d3d0 2023-08-27T11:28:12.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb63ad9d700 1 -- 172.21.15.45:0/2507721598 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0cd9d0 con 0x7fb63c06e3a0 2023-08-27T11:28:12.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb639d9b700 1 --2- 172.21.15.45:0/2507721598 >> v2:172.21.15.135:3300/0 conn(0x7fb63c0af040 0x7fb63c0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:12.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb63ad9d700 1 --2- 172.21.15.45:0/2507721598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c06e830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb634009e50 tx=0x7fb63400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eecb4d690fb4a4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb63959a700 1 -- 172.21.15.45:0/2507721598 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400e070 con 0x7fb63c06e3a0 2023-08-27T11:28:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb63959a700 1 -- 172.21.15.45:0/2507721598 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63400b910 con 0x7fb63c06e3a0 2023-08-27T11:28:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.416+0000 7fb63959a700 1 -- 172.21.15.45:0/2507721598 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400bcc0 con 0x7fb63c06e3a0 2023-08-27T11:28:12.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.417+0000 7fb640a5d700 1 -- 172.21.15.45:0/2507721598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 msgr2=0x7fb63c06e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.417+0000 7fb640a5d700 1 --2- 172.21.15.45:0/2507721598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c06e830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb634009e50 tx=0x7fb63400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.417+0000 7fb640a5d700 1 -- 172.21.15.45:0/2507721598 shutdown_connections 2023-08-27T11:28:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.417+0000 7fb640a5d700 1 --2- 172.21.15.45:0/2507721598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c06e830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.417+0000 7fb640a5d700 1 --2- 172.21.15.45:0/2507721598 >> v2:172.21.15.135:3300/0 conn(0x7fb63c0af040 0x7fb63c0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.417+0000 7fb640a5d700 1 -- 172.21.15.45:0/2507721598 >> 172.21.15.45:0/2507721598 conn(0x7fb63c0cfd60 msgr2=0x7fb63c0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.417+0000 7fb640a5d700 1 -- 172.21.15.45:0/2507721598 shutdown_connections 2023-08-27T11:28:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.417+0000 7fb640a5d700 1 -- 172.21.15.45:0/2507721598 wait complete. 2023-08-27T11:28:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.418+0000 7fb640a5d700 1 Processor -- start 2023-08-27T11:28:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.418+0000 7fb640a5d700 1 -- start start 2023-08-27T11:28:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb640a5d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb640a5d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb63c0af040 0x7fb63c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb640a5d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb63c15d3d0 0x7fb63c0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb640a5d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c15fc10 con 0x7fb63c15d3d0 2023-08-27T11:28:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb640a5d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c15e0a0 con 0x7fb63c0af040 2023-08-27T11:28:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb640a5d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c158160 con 0x7fb63c06e3a0 2023-08-27T11:28:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb639d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb63c0af040 0x7fb63c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb63ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb63c15d3d0 0x7fb63c0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb63a59c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb639d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb63c0af040 0x7fb63c157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:47990/0 (socket says 172.21.15.45:47990) 2023-08-27T11:28:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb639d9b700 1 -- 172.21.15.45:0/1118888266 learned_addr learned my addr 172.21.15.45:0/1118888266 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb639d9b700 1 -- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 msgr2=0x7fb63c153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb639d9b700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb639d9b700 1 -- 172.21.15.45:0/1118888266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb63c15d3d0 msgr2=0x7fb63c0add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb639d9b700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb63c15d3d0 0x7fb63c0add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb639d9b700 1 -- 172.21.15.45:0/1118888266 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6300094a0 con 0x7fb63c0af040 2023-08-27T11:28:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.419+0000 7fb63a59c700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.420+0000 7fb639d9b700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb63c0af040 0x7fb63c157c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb63000ec10 tx=0x7fb63000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.420+0000 7fb62affd700 1 -- 172.21.15.45:0/1118888266 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630009760 con 0x7fb63c0af040 2023-08-27T11:28:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.420+0000 7fb62affd700 1 -- 172.21.15.45:0/1118888266 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63000c850 con 0x7fb63c0af040 2023-08-27T11:28:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.420+0000 7fb62affd700 1 -- 172.21.15.45:0/1118888266 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63000cbe0 con 0x7fb63c0af040 2023-08-27T11:28:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.420+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb634009b30 con 0x7fb63c0af040 2023-08-27T11:28:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.420+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c0cbad0 con 0x7fb63c0af040 2023-08-27T11:28:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.421+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c072780 con 0x7fb63c0af040 2023-08-27T11:28:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.421+0000 7fb62affd700 1 -- 172.21.15.45:0/1118888266 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fb63000cd40 con 0x7fb63c0af040 2023-08-27T11:28:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.422+0000 7fb62affd700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb61c06a170 0x7fb61c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.422+0000 7fb62affd700 1 -- 172.21.15.45:0/1118888266 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fb630088a10 con 0x7fb63c0af040 2023-08-27T11:28:12.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.422+0000 7fb63a59c700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb61c06a170 0x7fb61c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.422+0000 7fb63a59c700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb61c06a170 0x7fb61c06c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb62c000f60 tx=0x7fb62c007e70 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.426+0000 7fb62affd700 1 -- 172.21.15.45:0/1118888266 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb630053fa0 con 0x7fb63c0af040 2023-08-27T11:28:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.627+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 --> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb63c0cdcb0 con 0x7fb61c06a170 2023-08-27T11:28:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.636+0000 7fb62affd700 1 -- 172.21.15.45:0/1118888266 <== mgr.6709 v2:172.21.15.135:6832/3812189695 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306377 (secure 0 0 0) 0x7fb62c001640 con 0x7fb61c06a170 2023-08-27T11:28:12.641 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb61c06a170 msgr2=0x7fb61c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb61c06a170 0x7fb61c06c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb62c000f60 tx=0x7fb62c007e70 comp rx=0 tx=0).stop 2023-08-27T11:28:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb63c0af040 msgr2=0x7fb63c157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb63c0af040 0x7fb63c157c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb63000ec10 tx=0x7fb63000ef20 comp rx=0 tx=0).stop 2023-08-27T11:28:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 shutdown_connections 2023-08-27T11:28:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fb61c06a170 0x7fb61c06c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb63c15d3d0 0x7fb63c0add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb63c06e3a0 0x7fb63c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 --2- 172.21.15.45:0/1118888266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb63c0af040 0x7fb63c157c20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 >> 172.21.15.45:0/1118888266 conn(0x7fb63c0cfd60 msgr2=0x7fb63c0d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 shutdown_connections 2023-08-27T11:28:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.639+0000 7fb640a5d700 1 -- 172.21.15.45:0/1118888266 wait complete. 2023-08-27T11:28:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:12.648 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-08-27T11:28:12.647+0000 7f2bb7a3e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:28:12.653 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:28:11.501465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2213,"ondisk_log_size":2213,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284920,"kb_used_data":10512,"kb_used_omap":13,"kb_used_meta":274290,"kb_avail":754689800,"statfs":{"total":773094113280,"available":772802355200,"internally_reserved":0,"allocated":10764288,"data_stored":6183480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":280873048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.849363"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674262+0000","last_change":"2023-08-27T11:28:06.674262+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674262+0000","last_undegraded":"2023-08-27T11:28:06.674262+0000","last_fullsized":"2023-08-27T11:28:06.674262+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663991+0000","last_change":"2023-08-27T11:28:06.663991+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663991+0000","last_undegraded":"2023-08-27T11:28:06.663991+0000","last_fullsized":"2023-08-27T11:28:06.663991+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671195+0000","last_change":"2023-08-27T11:28:06.671195+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671195+0000","last_undegraded":"2023-08-27T11:28:06.671195+0000","last_fullsized":"2023-08-27T11:28:06.671195+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669142+0000","last_change":"2023-08-27T11:28:06.669142+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669142+0000","last_undegraded":"2023-08-27T11:28:06.669142+0000","last_fullsized":"2023-08-27T11:28:06.669142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668212+0000","last_change":"2023-08-27T11:28:06.668212+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668212+0000","last_undegraded":"2023-08-27T11:28:06.668212+0000","last_fullsized":"2023-08-27T11:28:06.668212+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670471+0000","last_change":"2023-08-27T11:28:06.670471+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670471+0000","last_undegraded":"2023-08-27T11:28:06.670471+0000","last_fullsized":"2023-08-27T11:28:06.670471+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667376+0000","last_change":"2023-08-27T11:28:06.667376+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667376+0000","last_undegraded":"2023-08-27T11:28:06.667376+0000","last_fullsized":"2023-08-27T11:28:06.667376+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670912+0000","last_change":"2023-08-27T11:28:06.670912+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670912+0000","last_undegraded":"2023-08-27T11:28:06.670912+0000","last_fullsized":"2023-08-27T11:28:06.670912+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669900+0000","last_change":"2023-08-27T11:28:06.669900+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669900+0000","last_undegraded":"2023-08-27T11:28:06.669900+0000","last_fullsized":"2023-08-27T11:28:06.669900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674502+0000","last_change":"2023-08-27T11:28:06.674502+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674502+0000","last_undegraded":"2023-08-27T11:28:06.674502+0000","last_fullsized":"2023-08-27T11:28:06.674502+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665233+0000","last_change":"2023-08-27T11:28:06.665233+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665233+0000","last_undegraded":"2023-08-27T11:28:06.665233+0000","last_fullsized":"2023-08-27T11:28:06.665233+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.677732+0000","last_change":"2023-08-27T11:28:06.677732+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.677732+0000","last_undegraded":"2023-08-27T11:28:06.677732+0000","last_fullsized":"2023-08-27T11:28:06.677732+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.672762+0000","last_change":"2023-08-27T11:28:06.672762+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.672762+0000","last_undegraded":"2023-08-27T11:28:06.672762+0000","last_fullsized":"2023-08-27T11:28:06.672762+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667659+0000","last_change":"2023-08-27T11:28:06.667659+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667659+0000","last_undegraded":"2023-08-27T11:28:06.667659+0000","last_fullsized":"2023-08-27T11:28:06.667659+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662713+0000","last_change":"2023-08-27T11:28:06.662713+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662713+0000","last_undegraded":"2023-08-27T11:28:06.662713+0000","last_fullsized":"2023-08-27T11:28:06.662713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667753+0000","last_change":"2023-08-27T11:28:06.667753+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667753+0000","last_undegraded":"2023-08-27T11:28:06.667753+0000","last_fullsized":"2023-08-27T11:28:06.667753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667875+0000","last_change":"2023-08-27T11:28:06.667875+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667875+0000","last_undegraded":"2023-08-27T11:28:06.667875+0000","last_fullsized":"2023-08-27T11:28:06.667875+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.666981+0000","last_change":"2023-08-27T11:28:06.666981+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.666981+0000","last_undegraded":"2023-08-27T11:28:06.666981+0000","last_fullsized":"2023-08-27T11:28:06.666981+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662798+0000","last_change":"2023-08-27T11:28:06.662798+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662798+0000","last_undegraded":"2023-08-27T11:28:06.662798+0000","last_fullsized":"2023-08-27T11:28:06.662798+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.666965+0000","last_change":"2023-08-27T11:28:06.666965+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.666965+0000","last_undegraded":"2023-08-27T11:28:06.666965+0000","last_fullsized":"2023-08-27T11:28:06.666965+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668224+0000","last_change":"2023-08-27T11:28:06.668224+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668224+0000","last_undegraded":"2023-08-27T11:28:06.668224+0000","last_fullsized":"2023-08-27T11:28:06.668224+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663693+0000","last_change":"2023-08-27T11:28:06.663693+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663693+0000","last_undegraded":"2023-08-27T11:28:06.663693+0000","last_fullsized":"2023-08-27T11:28:06.663693+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669584+0000","last_change":"2023-08-27T11:28:06.669584+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669584+0000","last_undegraded":"2023-08-27T11:28:06.669584+0000","last_fullsized":"2023-08-27T11:28:06.669584+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665390+0000","last_change":"2023-08-27T11:28:06.665390+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665390+0000","last_undegraded":"2023-08-27T11:28:06.665390+0000","last_fullsized":"2023-08-27T11:28:06.665390+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663341+0000","last_change":"2023-08-27T11:28:06.663341+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663341+0000","last_undegraded":"2023-08-27T11:28:06.663341+0000","last_fullsized":"2023-08-27T11:28:06.663341+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674565+0000","last_change":"2023-08-27T11:28:06.674565+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674565+0000","last_undegraded":"2023-08-27T11:28:06.674565+0000","last_fullsized":"2023-08-27T11:28:06.674565+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669801+0000","last_change":"2023-08-27T11:28:06.669801+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669801+0000","last_undegraded":"2023-08-27T11:28:06.669801+0000","last_fullsized":"2023-08-27T11:28:06.669801+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.677428+0000","last_change":"2023-08-27T11:28:06.677428+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.677428+0000","last_undegraded":"2023-08-27T11:28:06.677428+0000","last_fullsized":"2023-08-27T11:28:06.677428+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662770+0000","last_change":"2023-08-27T11:28:06.662770+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662770+0000","last_undegraded":"2023-08-27T11:28:06.662770+0000","last_fullsized":"2023-08-27T11:28:06.662770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662917+0000","last_change":"2023-08-27T11:28:06.662917+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662917+0000","last_undegraded":"2023-08-27T11:28:06.662917+0000","last_fullsized":"2023-08-27T11:28:06.662917+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667437+0000","last_change":"2023-08-27T11:28:06.667437+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667437+0000","last_undegraded":"2023-08-27T11:28:06.667437+0000","last_fullsized":"2023-08-27T11:28:06.667437+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674563+0000","last_change":"2023-08-27T11:28:06.674563+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674563+0000","last_undegraded":"2023-08-27T11:28:06.674563+0000","last_fullsized":"2023-08-27T11:28:06.674563+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671268+0000","last_change":"2023-08-27T11:28:06.671268+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671268+0000","last_undegraded":"2023-08-27T11:28:06.671268+0000","last_fullsized":"2023-08-27T11:28:06.671268+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682500+0000","last_change":"2023-08-27T11:28:06.680510+0000","last_active":"2023-08-27T11:28:06.682500+0000","last_peered":"2023-08-27T11:28:06.682500+0000","last_clean":"2023-08-27T11:28:06.682500+0000","last_became_active":"2023-08-27T11:28:06.680268+0000","last_became_peered":"2023-08-27T11:28:06.680268+0000","last_unstale":"2023-08-27T11:28:06.682500+0000","last_undegraded":"2023-08-27T11:28:06.682500+0000","last_fullsized":"2023-08-27T11:28:06.682500+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:53.355949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673997+0000","last_change":"2023-08-27T11:28:06.673997+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673997+0000","last_undegraded":"2023-08-27T11:28:06.673997+0000","last_fullsized":"2023-08-27T11:28:06.673997+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680586+0000","last_change":"2023-08-27T11:28:06.678558+0000","last_active":"2023-08-27T11:28:06.680586+0000","last_peered":"2023-08-27T11:28:06.680586+0000","last_clean":"2023-08-27T11:28:06.680586+0000","last_became_active":"2023-08-27T11:28:06.678323+0000","last_became_peered":"2023-08-27T11:28:06.678323+0000","last_unstale":"2023-08-27T11:28:06.680586+0000","last_undegraded":"2023-08-27T11:28:06.680586+0000","last_fullsized":"2023-08-27T11:28:06.680586+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:32:33.684096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673970+0000","last_change":"2023-08-27T11:28:06.673970+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673970+0000","last_undegraded":"2023-08-27T11:28:06.673970+0000","last_fullsized":"2023-08-27T11:28:06.673970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675092+0000","last_change":"2023-08-27T11:28:06.674106+0000","last_active":"2023-08-27T11:28:06.675092+0000","last_peered":"2023-08-27T11:28:06.675092+0000","last_clean":"2023-08-27T11:28:06.675092+0000","last_became_active":"2023-08-27T11:28:06.673843+0000","last_became_peered":"2023-08-27T11:28:06.673843+0000","last_unstale":"2023-08-27T11:28:06.675092+0000","last_undegraded":"2023-08-27T11:28:06.675092+0000","last_fullsized":"2023-08-27T11:28:06.675092+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:18.098063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670630+0000","last_change":"2023-08-27T11:28:06.670630+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670630+0000","last_undegraded":"2023-08-27T11:28:06.670630+0000","last_fullsized":"2023-08-27T11:28:06.670630+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675059+0000","last_change":"2023-08-27T11:28:06.674658+0000","last_active":"2023-08-27T11:28:06.675059+0000","last_peered":"2023-08-27T11:28:06.675059+0000","last_clean":"2023-08-27T11:28:06.675059+0000","last_became_active":"2023-08-27T11:28:06.674384+0000","last_became_peered":"2023-08-27T11:28:06.674384+0000","last_unstale":"2023-08-27T11:28:06.675059+0000","last_undegraded":"2023-08-27T11:28:06.675059+0000","last_fullsized":"2023-08-27T11:28:06.675059+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:33:38.254409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663309+0000","last_change":"2023-08-27T11:28:06.663309+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663309+0000","last_undegraded":"2023-08-27T11:28:06.663309+0000","last_fullsized":"2023-08-27T11:28:06.663309+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680562+0000","last_change":"2023-08-27T11:28:06.678516+0000","last_active":"2023-08-27T11:28:06.680562+0000","last_peered":"2023-08-27T11:28:06.680562+0000","last_clean":"2023-08-27T11:28:06.680562+0000","last_became_active":"2023-08-27T11:28:06.678268+0000","last_became_peered":"2023-08-27T11:28:06.678268+0000","last_unstale":"2023-08-27T11:28:06.680562+0000","last_undegraded":"2023-08-27T11:28:06.680562+0000","last_fullsized":"2023-08-27T11:28:06.680562+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:29:56.120623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667804+0000","last_change":"2023-08-27T11:28:06.667804+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667804+0000","last_undegraded":"2023-08-27T11:28:06.667804+0000","last_fullsized":"2023-08-27T11:28:06.667804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680480+0000","last_change":"2023-08-27T11:28:06.677993+0000","last_active":"2023-08-27T11:28:06.680480+0000","last_peered":"2023-08-27T11:28:06.680480+0000","last_clean":"2023-08-27T11:28:06.680480+0000","last_became_active":"2023-08-27T11:28:06.677679+0000","last_became_peered":"2023-08-27T11:28:06.677679+0000","last_unstale":"2023-08-27T11:28:06.680480+0000","last_undegraded":"2023-08-27T11:28:06.680480+0000","last_fullsized":"2023-08-27T11:28:06.680480+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:04.340118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670493+0000","last_change":"2023-08-27T11:28:06.670493+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670493+0000","last_undegraded":"2023-08-27T11:28:06.670493+0000","last_fullsized":"2023-08-27T11:28:06.670493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676184+0000","last_change":"2023-08-27T11:28:06.675717+0000","last_active":"2023-08-27T11:28:06.676184+0000","last_peered":"2023-08-27T11:28:06.676184+0000","last_clean":"2023-08-27T11:28:06.676184+0000","last_became_active":"2023-08-27T11:28:06.675161+0000","last_became_peered":"2023-08-27T11:28:06.675161+0000","last_unstale":"2023-08-27T11:28:06.676184+0000","last_undegraded":"2023-08-27T11:28:06.676184+0000","last_fullsized":"2023-08-27T11:28:06.676184+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:30.138559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662304+0000","last_change":"2023-08-27T11:28:06.662304+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662304+0000","last_undegraded":"2023-08-27T11:28:06.662304+0000","last_fullsized":"2023-08-27T11:28:06.662304+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680689+0000","last_change":"2023-08-27T11:28:06.680053+0000","last_active":"2023-08-27T11:28:06.680689+0000","last_peered":"2023-08-27T11:28:06.680689+0000","last_clean":"2023-08-27T11:28:06.680689+0000","last_became_active":"2023-08-27T11:28:06.679714+0000","last_became_peered":"2023-08-27T11:28:06.679714+0000","last_unstale":"2023-08-27T11:28:06.680689+0000","last_undegraded":"2023-08-27T11:28:06.680689+0000","last_fullsized":"2023-08-27T11:28:06.680689+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:13.718165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.661902+0000","last_change":"2023-08-27T11:28:06.661902+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.661902+0000","last_undegraded":"2023-08-27T11:28:06.661902+0000","last_fullsized":"2023-08-27T11:28:06.661902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.683222+0000","last_change":"2023-08-27T11:28:06.682417+0000","last_active":"2023-08-27T11:28:06.683222+0000","last_peered":"2023-08-27T11:28:06.683222+0000","last_clean":"2023-08-27T11:28:06.683222+0000","last_became_active":"2023-08-27T11:28:06.682174+0000","last_became_peered":"2023-08-27T11:28:06.682174+0000","last_unstale":"2023-08-27T11:28:06.683222+0000","last_undegraded":"2023-08-27T11:28:06.683222+0000","last_fullsized":"2023-08-27T11:28:06.683222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:22.047442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669447+0000","last_change":"2023-08-27T11:28:06.669447+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669447+0000","last_undegraded":"2023-08-27T11:28:06.669447+0000","last_fullsized":"2023-08-27T11:28:06.669447+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.683629+0000","last_change":"2023-08-27T11:28:06.682913+0000","last_active":"2023-08-27T11:28:06.683629+0000","last_peered":"2023-08-27T11:28:06.683629+0000","last_clean":"2023-08-27T11:28:06.683629+0000","last_became_active":"2023-08-27T11:28:06.682773+0000","last_became_peered":"2023-08-27T11:28:06.682773+0000","last_unstale":"2023-08-27T11:28:06.683629+0000","last_undegraded":"2023-08-27T11:28:06.683629+0000","last_fullsized":"2023-08-27T11:28:06.683629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:13.142404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662935+0000","last_change":"2023-08-27T11:28:06.662935+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662935+0000","last_undegraded":"2023-08-27T11:28:06.662935+0000","last_fullsized":"2023-08-27T11:28:06.662935+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682449+0000","last_change":"2023-08-27T11:28:06.680796+0000","last_active":"2023-08-27T11:28:06.682449+0000","last_peered":"2023-08-27T11:28:06.682449+0000","last_clean":"2023-08-27T11:28:06.682449+0000","last_became_active":"2023-08-27T11:28:06.680639+0000","last_became_peered":"2023-08-27T11:28:06.680639+0000","last_unstale":"2023-08-27T11:28:06.682449+0000","last_undegraded":"2023-08-27T11:28:06.682449+0000","last_fullsized":"2023-08-27T11:28:06.682449+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:54.607978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669454+0000","last_change":"2023-08-27T11:28:06.669454+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669454+0000","last_undegraded":"2023-08-27T11:28:06.669454+0000","last_fullsized":"2023-08-27T11:28:06.669454+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679081+0000","last_change":"2023-08-27T11:28:06.675117+0000","last_active":"2023-08-27T11:28:06.679081+0000","last_peered":"2023-08-27T11:28:06.679081+0000","last_clean":"2023-08-27T11:28:06.679081+0000","last_became_active":"2023-08-27T11:28:06.674808+0000","last_became_peered":"2023-08-27T11:28:06.674808+0000","last_unstale":"2023-08-27T11:28:06.679081+0000","last_undegraded":"2023-08-27T11:28:06.679081+0000","last_fullsized":"2023-08-27T11:28:06.679081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:13.763908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663233+0000","last_change":"2023-08-27T11:28:06.663233+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663233+0000","last_undegraded":"2023-08-27T11:28:06.663233+0000","last_fullsized":"2023-08-27T11:28:06.663233+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.673619+0000","last_change":"2023-08-27T11:28:06.670804+0000","last_active":"2023-08-27T11:28:06.673619+0000","last_peered":"2023-08-27T11:28:06.673619+0000","last_clean":"2023-08-27T11:28:06.673619+0000","last_became_active":"2023-08-27T11:28:06.669783+0000","last_became_peered":"2023-08-27T11:28:06.669783+0000","last_unstale":"2023-08-27T11:28:06.673619+0000","last_undegraded":"2023-08-27T11:28:06.673619+0000","last_fullsized":"2023-08-27T11:28:06.673619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:54.035323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.672405+0000","last_change":"2023-08-27T11:28:06.672405+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.672405+0000","last_undegraded":"2023-08-27T11:28:06.672405+0000","last_fullsized":"2023-08-27T11:28:06.672405+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672817+0000","last_change":"2023-08-27T11:28:06.668164+0000","last_active":"2023-08-27T11:28:06.672817+0000","last_peered":"2023-08-27T11:28:06.672817+0000","last_clean":"2023-08-27T11:28:06.672817+0000","last_became_active":"2023-08-27T11:28:06.667870+0000","last_became_peered":"2023-08-27T11:28:06.667870+0000","last_unstale":"2023-08-27T11:28:06.672817+0000","last_undegraded":"2023-08-27T11:28:06.672817+0000","last_fullsized":"2023-08-27T11:28:06.672817+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:34.544721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664777+0000","last_change":"2023-08-27T11:28:06.664777+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664777+0000","last_undegraded":"2023-08-27T11:28:06.664777+0000","last_fullsized":"2023-08-27T11:28:06.664777+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.681660+0000","last_change":"2023-08-27T11:28:06.679870+0000","last_active":"2023-08-27T11:28:06.681660+0000","last_peered":"2023-08-27T11:28:06.681660+0000","last_clean":"2023-08-27T11:28:06.681660+0000","last_became_active":"2023-08-27T11:28:06.679627+0000","last_became_peered":"2023-08-27T11:28:06.679627+0000","last_unstale":"2023-08-27T11:28:06.681660+0000","last_undegraded":"2023-08-27T11:28:06.681660+0000","last_fullsized":"2023-08-27T11:28:06.681660+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:35.146679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665638+0000","last_change":"2023-08-27T11:28:06.665638+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665638+0000","last_undegraded":"2023-08-27T11:28:06.665638+0000","last_fullsized":"2023-08-27T11:28:06.665638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682022+0000","last_change":"2023-08-27T11:28:06.681196+0000","last_active":"2023-08-27T11:28:06.682022+0000","last_peered":"2023-08-27T11:28:06.682022+0000","last_clean":"2023-08-27T11:28:06.682022+0000","last_became_active":"2023-08-27T11:28:06.680914+0000","last_became_peered":"2023-08-27T11:28:06.680914+0000","last_unstale":"2023-08-27T11:28:06.682022+0000","last_undegraded":"2023-08-27T11:28:06.682022+0000","last_fullsized":"2023-08-27T11:28:06.682022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:12.051340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670514+0000","last_change":"2023-08-27T11:28:06.670514+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670514+0000","last_undegraded":"2023-08-27T11:28:06.670514+0000","last_fullsized":"2023-08-27T11:28:06.670514+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680425+0000","last_change":"2023-08-27T11:28:06.679947+0000","last_active":"2023-08-27T11:28:06.680425+0000","last_peered":"2023-08-27T11:28:06.680425+0000","last_clean":"2023-08-27T11:28:06.680425+0000","last_became_active":"2023-08-27T11:28:06.679654+0000","last_became_peered":"2023-08-27T11:28:06.679654+0000","last_unstale":"2023-08-27T11:28:06.680425+0000","last_undegraded":"2023-08-27T11:28:06.680425+0000","last_fullsized":"2023-08-27T11:28:06.680425+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:36.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671123+0000","last_change":"2023-08-27T11:28:06.671123+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671123+0000","last_undegraded":"2023-08-27T11:28:06.671123+0000","last_fullsized":"2023-08-27T11:28:06.671123+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672339+0000","last_change":"2023-08-27T11:28:06.668640+0000","last_active":"2023-08-27T11:28:06.672339+0000","last_peered":"2023-08-27T11:28:06.672339+0000","last_clean":"2023-08-27T11:28:06.672339+0000","last_became_active":"2023-08-27T11:28:06.668372+0000","last_became_peered":"2023-08-27T11:28:06.668372+0000","last_unstale":"2023-08-27T11:28:06.672339+0000","last_undegraded":"2023-08-27T11:28:06.672339+0000","last_fullsized":"2023-08-27T11:28:06.672339+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:48:01.533647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673604+0000","last_change":"2023-08-27T11:28:06.673604+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673604+0000","last_undegraded":"2023-08-27T11:28:06.673604+0000","last_fullsized":"2023-08-27T11:28:06.673604+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682683+0000","last_change":"2023-08-27T11:28:06.682044+0000","last_active":"2023-08-27T11:28:06.682683+0000","last_peered":"2023-08-27T11:28:06.682683+0000","last_clean":"2023-08-27T11:28:06.682683+0000","last_became_active":"2023-08-27T11:28:06.681789+0000","last_became_peered":"2023-08-27T11:28:06.681789+0000","last_unstale":"2023-08-27T11:28:06.682683+0000","last_undegraded":"2023-08-27T11:28:06.682683+0000","last_fullsized":"2023-08-27T11:28:06.682683+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:39:11.072746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668771+0000","last_change":"2023-08-27T11:28:06.668771+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668771+0000","last_undegraded":"2023-08-27T11:28:06.668771+0000","last_fullsized":"2023-08-27T11:28:06.668771+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682726+0000","last_change":"2023-08-27T11:28:06.682213+0000","last_active":"2023-08-27T11:28:06.682726+0000","last_peered":"2023-08-27T11:28:06.682726+0000","last_clean":"2023-08-27T11:28:06.682726+0000","last_became_active":"2023-08-27T11:28:06.681772+0000","last_became_peered":"2023-08-27T11:28:06.681772+0000","last_unstale":"2023-08-27T11:28:06.682726+0000","last_undegraded":"2023-08-27T11:28:06.682726+0000","last_fullsized":"2023-08-27T11:28:06.682726+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:34:15.373195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670120+0000","last_change":"2023-08-27T11:28:06.670120+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670120+0000","last_undegraded":"2023-08-27T11:28:06.670120+0000","last_fullsized":"2023-08-27T11:28:06.670120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682020+0000","last_change":"2023-08-27T11:28:06.681163+0000","last_active":"2023-08-27T11:28:06.682020+0000","last_peered":"2023-08-27T11:28:06.682020+0000","last_clean":"2023-08-27T11:28:06.682020+0000","last_became_active":"2023-08-27T11:28:06.680922+0000","last_became_peered":"2023-08-27T11:28:06.680922+0000","last_unstale":"2023-08-27T11:28:06.682020+0000","last_undegraded":"2023-08-27T11:28:06.682020+0000","last_fullsized":"2023-08-27T11:28:06.682020+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:13:20.626473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664201+0000","last_change":"2023-08-27T11:28:06.664201+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664201+0000","last_undegraded":"2023-08-27T11:28:06.664201+0000","last_fullsized":"2023-08-27T11:28:06.664201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680463+0000","last_change":"2023-08-27T11:28:06.677834+0000","last_active":"2023-08-27T11:28:06.680463+0000","last_peered":"2023-08-27T11:28:06.680463+0000","last_clean":"2023-08-27T11:28:06.680463+0000","last_became_active":"2023-08-27T11:28:06.677483+0000","last_became_peered":"2023-08-27T11:28:06.677483+0000","last_unstale":"2023-08-27T11:28:06.680463+0000","last_undegraded":"2023-08-27T11:28:06.680463+0000","last_fullsized":"2023-08-27T11:28:06.680463+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:33:20.246978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670946+0000","last_change":"2023-08-27T11:28:06.670946+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670946+0000","last_undegraded":"2023-08-27T11:28:06.670946+0000","last_fullsized":"2023-08-27T11:28:06.670946+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682708+0000","last_change":"2023-08-27T11:28:06.682127+0000","last_active":"2023-08-27T11:28:06.682708+0000","last_peered":"2023-08-27T11:28:06.682708+0000","last_clean":"2023-08-27T11:28:06.682708+0000","last_became_active":"2023-08-27T11:28:06.681824+0000","last_became_peered":"2023-08-27T11:28:06.681824+0000","last_unstale":"2023-08-27T11:28:06.682708+0000","last_undegraded":"2023-08-27T11:28:06.682708+0000","last_fullsized":"2023-08-27T11:28:06.682708+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:03.058354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669155+0000","last_change":"2023-08-27T11:28:06.669155+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669155+0000","last_undegraded":"2023-08-27T11:28:06.669155+0000","last_fullsized":"2023-08-27T11:28:06.669155+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679086+0000","last_change":"2023-08-27T11:28:06.676211+0000","last_active":"2023-08-27T11:28:06.679086+0000","last_peered":"2023-08-27T11:28:06.679086+0000","last_clean":"2023-08-27T11:28:06.679086+0000","last_became_active":"2023-08-27T11:28:06.675956+0000","last_became_peered":"2023-08-27T11:28:06.675956+0000","last_unstale":"2023-08-27T11:28:06.679086+0000","last_undegraded":"2023-08-27T11:28:06.679086+0000","last_fullsized":"2023-08-27T11:28:06.679086+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:48:39.217804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669642+0000","last_change":"2023-08-27T11:28:06.669642+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669642+0000","last_undegraded":"2023-08-27T11:28:06.669642+0000","last_fullsized":"2023-08-27T11:28:06.669642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676840+0000","last_change":"2023-08-27T11:28:06.673096+0000","last_active":"2023-08-27T11:28:06.676840+0000","last_peered":"2023-08-27T11:28:06.676840+0000","last_clean":"2023-08-27T11:28:06.676840+0000","last_became_active":"2023-08-27T11:28:06.672819+0000","last_became_peered":"2023-08-27T11:28:06.672819+0000","last_unstale":"2023-08-27T11:28:06.676840+0000","last_undegraded":"2023-08-27T11:28:06.676840+0000","last_fullsized":"2023-08-27T11:28:06.676840+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:34.207300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.666042+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:06.666042+0000","last_peered":"2023-08-27T11:28:06.666042+0000","last_clean":"2023-08-27T11:28:06.666042+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:06.666042+0000","last_undegraded":"2023-08-27T11:28:06.666042+0000","last_fullsized":"2023-08-27T11:28:06.666042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:28.335693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669129+0000","last_change":"2023-08-27T11:28:06.669129+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669129+0000","last_undegraded":"2023-08-27T11:28:06.669129+0000","last_fullsized":"2023-08-27T11:28:06.669129+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676890+0000","last_change":"2023-08-27T11:28:06.676306+0000","last_active":"2023-08-27T11:28:06.676890+0000","last_peered":"2023-08-27T11:28:06.676890+0000","last_clean":"2023-08-27T11:28:06.676890+0000","last_became_active":"2023-08-27T11:28:06.676049+0000","last_became_peered":"2023-08-27T11:28:06.676049+0000","last_unstale":"2023-08-27T11:28:06.676890+0000","last_undegraded":"2023-08-27T11:28:06.676890+0000","last_fullsized":"2023-08-27T11:28:06.676890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:52.590584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672445+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:06.672445+0000","last_peered":"2023-08-27T11:28:06.672445+0000","last_clean":"2023-08-27T11:28:06.672445+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:06.672445+0000","last_undegraded":"2023-08-27T11:28:06.672445+0000","last_fullsized":"2023-08-27T11:28:06.672445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:04.382696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670944+0000","last_change":"2023-08-27T11:28:06.670944+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670944+0000","last_undegraded":"2023-08-27T11:28:06.670944+0000","last_fullsized":"2023-08-27T11:28:06.670944+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682490+0000","last_change":"2023-08-27T11:28:06.680009+0000","last_active":"2023-08-27T11:28:06.682490+0000","last_peered":"2023-08-27T11:28:06.682490+0000","last_clean":"2023-08-27T11:28:06.682490+0000","last_became_active":"2023-08-27T11:28:06.679754+0000","last_became_peered":"2023-08-27T11:28:06.679754+0000","last_unstale":"2023-08-27T11:28:06.682490+0000","last_undegraded":"2023-08-27T11:28:06.682490+0000","last_fullsized":"2023-08-27T11:28:06.682490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:06.046171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":154,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.670537+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:06.670537+0000","last_peered":"2023-08-27T11:28:06.670537+0000","last_clean":"2023-08-27T11:28:06.670537+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:06.670537+0000","last_undegraded":"2023-08-27T11:28:06.670537+0000","last_fullsized":"2023-08-27T11:28:06.670537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:46:09.042939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664810+0000","last_change":"2023-08-27T11:28:06.664810+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664810+0000","last_undegraded":"2023-08-27T11:28:06.664810+0000","last_fullsized":"2023-08-27T11:28:06.664810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.681624+0000","last_change":"2023-08-27T11:28:06.680561+0000","last_active":"2023-08-27T11:28:06.681624+0000","last_peered":"2023-08-27T11:28:06.681624+0000","last_clean":"2023-08-27T11:28:06.681624+0000","last_became_active":"2023-08-27T11:28:06.680307+0000","last_became_peered":"2023-08-27T11:28:06.680307+0000","last_unstale":"2023-08-27T11:28:06.681624+0000","last_undegraded":"2023-08-27T11:28:06.681624+0000","last_fullsized":"2023-08-27T11:28:06.681624+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:58.720509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672903+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:06.672903+0000","last_peered":"2023-08-27T11:28:06.672903+0000","last_clean":"2023-08-27T11:28:06.672903+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:06.672903+0000","last_undegraded":"2023-08-27T11:28:06.672903+0000","last_fullsized":"2023-08-27T11:28:06.672903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:40:14.646059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"98'2213","reported_seq":2572,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:07.501955+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:07.501955+0000","last_peered":"2023-08-27T11:28:07.501955+0000","last_clean":"2023-08-27T11:28:07.501955+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:07.501955+0000","last_undegraded":"2023-08-27T11:28:07.501955+0000","last_fullsized":"2023-08-27T11:28:07.501955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":2213,"log_dups_size":0,"ondisk_log_size":2213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:54:12.932795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673646+0000","last_change":"2023-08-27T11:28:06.673646+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673646+0000","last_undegraded":"2023-08-27T11:28:06.673646+0000","last_fullsized":"2023-08-27T11:28:06.673646+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680214+0000","last_change":"2023-08-27T11:28:06.678931+0000","last_active":"2023-08-27T11:28:06.680214+0000","last_peered":"2023-08-27T11:28:06.680214+0000","last_clean":"2023-08-27T11:28:06.680214+0000","last_became_active":"2023-08-27T11:28:06.678661+0000","last_became_peered":"2023-08-27T11:28:06.678661+0000","last_unstale":"2023-08-27T11:28:06.680214+0000","last_undegraded":"2023-08-27T11:28:06.680214+0000","last_fullsized":"2023-08-27T11:28:06.680214+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:00:05.382377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675655+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:06.675655+0000","last_peered":"2023-08-27T11:28:06.675655+0000","last_clean":"2023-08-27T11:28:06.675655+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:06.675655+0000","last_undegraded":"2023-08-27T11:28:06.675655+0000","last_fullsized":"2023-08-27T11:28:06.675655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:10:39.559849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667293+0000","last_change":"2023-08-27T11:28:06.667293+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667293+0000","last_undegraded":"2023-08-27T11:28:06.667293+0000","last_fullsized":"2023-08-27T11:28:06.667293+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.673836+0000","last_change":"2023-08-27T11:28:06.672441+0000","last_active":"2023-08-27T11:28:06.673836+0000","last_peered":"2023-08-27T11:28:06.673836+0000","last_clean":"2023-08-27T11:28:06.673836+0000","last_became_active":"2023-08-27T11:28:06.672183+0000","last_became_peered":"2023-08-27T11:28:06.672183+0000","last_unstale":"2023-08-27T11:28:06.673836+0000","last_undegraded":"2023-08-27T11:28:06.673836+0000","last_fullsized":"2023-08-27T11:28:06.673836+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:23.790945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.677403+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:06.677403+0000","last_peered":"2023-08-27T11:28:06.677403+0000","last_clean":"2023-08-27T11:28:06.677403+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:06.677403+0000","last_undegraded":"2023-08-27T11:28:06.677403+0000","last_fullsized":"2023-08-27T11:28:06.677403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:13:06.409958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671744+0000","last_change":"2023-08-27T11:28:06.671744+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671744+0000","last_undegraded":"2023-08-27T11:28:06.671744+0000","last_fullsized":"2023-08-27T11:28:06.671744+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.670942+0000","last_change":"2023-08-27T11:28:06.666649+0000","last_active":"2023-08-27T11:28:06.670942+0000","last_peered":"2023-08-27T11:28:06.670942+0000","last_clean":"2023-08-27T11:28:06.670942+0000","last_became_active":"2023-08-27T11:28:06.666348+0000","last_became_peered":"2023-08-27T11:28:06.666348+0000","last_unstale":"2023-08-27T11:28:06.670942+0000","last_undegraded":"2023-08-27T11:28:06.670942+0000","last_fullsized":"2023-08-27T11:28:06.670942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:31:23.091962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.666080+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:06.666080+0000","last_peered":"2023-08-27T11:28:06.666080+0000","last_clean":"2023-08-27T11:28:06.666080+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:06.666080+0000","last_undegraded":"2023-08-27T11:28:06.666080+0000","last_fullsized":"2023-08-27T11:28:06.666080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:05.589881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664961+0000","last_change":"2023-08-27T11:28:06.664961+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664961+0000","last_undegraded":"2023-08-27T11:28:06.664961+0000","last_fullsized":"2023-08-27T11:28:06.664961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.684250+0000","last_change":"2023-08-27T11:28:06.683725+0000","last_active":"2023-08-27T11:28:06.684250+0000","last_peered":"2023-08-27T11:28:06.684250+0000","last_clean":"2023-08-27T11:28:06.684250+0000","last_became_active":"2023-08-27T11:28:06.683424+0000","last_became_peered":"2023-08-27T11:28:06.683424+0000","last_unstale":"2023-08-27T11:28:06.684250+0000","last_undegraded":"2023-08-27T11:28:06.684250+0000","last_fullsized":"2023-08-27T11:28:06.684250+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:13.388800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679319+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:06.679319+0000","last_peered":"2023-08-27T11:28:06.679319+0000","last_clean":"2023-08-27T11:28:06.679319+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:06.679319+0000","last_undegraded":"2023-08-27T11:28:06.679319+0000","last_fullsized":"2023-08-27T11:28:06.679319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:19:35.146879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2213,"ondisk_log_size":2213,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575041,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41252,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330588,"statfs":{"total":96636764160,"available":96594522112,"internally_reserved":0,"allocated":1794048,"data_stored":1217459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.38500000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":1.601,"15min":1.601},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.59999999999999998,"15min":0.58299999999999996},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.499}]},{"osd":1,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.71199999999999997,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.77000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.432}]},{"osd":2,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.80100000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.29099999999999998},"max":{"1min":0.749,"5min":1.778,"15min":1.778},"last":0.52100000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.65300000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.221},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68799999999999994,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.30499999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65300000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.25800000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.34499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575041,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.40799999999999997,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.76700000000000002,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.628},"min":{"1min":0.47399999999999998,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.3540000000000001},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":3.3780000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.80700000000000005,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.41699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.61699999999999999},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.73399999999999999,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":1.325},"last":0.44}]},{"osd":3,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.60499999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.71699999999999997,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.46400000000000002},"min":{"1min":0.312,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"min":{"1min":0.35199999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.624,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":1.157,"15min":1.157},"last":0.45200000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.70399999999999996,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.36899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40680,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331160,"statfs":{"total":96636764160,"available":96595107840,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20399999999999999},"max":{"1min":0.52100000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.495},{"interface":"front","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.628,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42699999999999999,"15min":0.42399999999999999},"min":{"1min":0.27500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.34100000000000003,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58999999999999997,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":1.169,"15min":1.169},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44600000000000001,"15min":0.44},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.23400000000000001},"max":{"1min":0.55900000000000005,"5min":1.139,"15min":1.139},"last":0.47299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.82599999999999996,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81499999999999995,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88700000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.59899999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89400000000000002,"15min":1.139},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575041,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41248,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330592,"statfs":{"total":96636764160,"available":96594526208,"internally_reserved":0,"allocated":1794048,"data_stored":1217459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.31},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.29399999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.26700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.374}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.41499999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.48999999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.39400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":620099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.25}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.31}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.433}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.48099999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.504},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.54700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40699999999999997,"15min":0.42499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.22},"max":{"1min":0.60799999999999998,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.224},"max":{"1min":0.60699999999999998,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.30499999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.64100000000000001,"15min":6.0830000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46200000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.245},"max":{"1min":0.57399999999999995,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.68000000000000005,"5min":0.71199999999999997,"15min":1.25},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":0.84499999999999997,"15min":11.939},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.376},"max":{"1min":0.83099999999999996,"5min":1.0329999999999999,"15min":11.007},"last":0.374},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":1.113,"15min":1.2929999999999999},"last":0.45100000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93300000000000005,"5min":3.0219999999999998,"15min":11.981},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":1.218,"15min":11.09},"last":0.498}]},{"osd":6,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69099999999999995,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.75600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":1.0269999999999999,"15min":12.016},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68999999999999995,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":0.96699999999999997,"15min":11.054},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.53500000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84099999999999997,"5min":1.0640000000000001,"15min":1.361},"last":0.47699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575041,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.32600000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.39400000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.504}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.47199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575040,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335340,"statfs":{"total":96636764160,"available":96599388160,"internally_reserved":0,"allocated":1187840,"data_stored":620099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.38},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.26000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.39300000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.34100000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.43099999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.42199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:12.660 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2023-08-27T11:28:12.660 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.674262+0000', 'last_change': '2023-08-27T11:28:06.674262+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.674262+0000', 'last_undegraded': '2023-08-27T11:28:06.674262+0000', 'last_fullsized': '2023-08-27T11:28:06.674262+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.661 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2023-08-27T11:28:12.661 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.663991+0000', 'last_change': '2023-08-27T11:28:06.663991+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.663991+0000', 'last_undegraded': '2023-08-27T11:28:06.663991+0000', 'last_fullsized': '2023-08-27T11:28:06.663991+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.662 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2023-08-27T11:28:12.662 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.671195+0000', 'last_change': '2023-08-27T11:28:06.671195+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.671195+0000', 'last_undegraded': '2023-08-27T11:28:06.671195+0000', 'last_fullsized': '2023-08-27T11:28:06.671195+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:12.662 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2023-08-27T11:28:12.663 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.669142+0000', 'last_change': '2023-08-27T11:28:06.669142+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.669142+0000', 'last_undegraded': '2023-08-27T11:28:06.669142+0000', 'last_fullsized': '2023-08-27T11:28:06.669142+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:12.663 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2023-08-27T11:28:12.664 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.668212+0000', 'last_change': '2023-08-27T11:28:06.668212+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.668212+0000', 'last_undegraded': '2023-08-27T11:28:06.668212+0000', 'last_fullsized': '2023-08-27T11:28:06.668212+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.664 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2023-08-27T11:28:12.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.670471+0000', 'last_change': '2023-08-27T11:28:06.670471+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.670471+0000', 'last_undegraded': '2023-08-27T11:28:06.670471+0000', 'last_fullsized': '2023-08-27T11:28:06.670471+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:28:12.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2023-08-27T11:28:12.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.667376+0000', 'last_change': '2023-08-27T11:28:06.667376+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.667376+0000', 'last_undegraded': '2023-08-27T11:28:06.667376+0000', 'last_fullsized': '2023-08-27T11:28:06.667376+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:12.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2023-08-27T11:28:12.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.670912+0000', 'last_change': '2023-08-27T11:28:06.670912+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.670912+0000', 'last_undegraded': '2023-08-27T11:28:06.670912+0000', 'last_fullsized': '2023-08-27T11:28:06.670912+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:28:12.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2023-08-27T11:28:12.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.669900+0000', 'last_change': '2023-08-27T11:28:06.669900+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.669900+0000', 'last_undegraded': '2023-08-27T11:28:06.669900+0000', 'last_fullsized': '2023-08-27T11:28:06.669900+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:28:12.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2023-08-27T11:28:12.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.674502+0000', 'last_change': '2023-08-27T11:28:06.674502+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.674502+0000', 'last_undegraded': '2023-08-27T11:28:06.674502+0000', 'last_fullsized': '2023-08-27T11:28:06.674502+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2023-08-27T11:28:12.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.665233+0000', 'last_change': '2023-08-27T11:28:06.665233+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.665233+0000', 'last_undegraded': '2023-08-27T11:28:06.665233+0000', 'last_fullsized': '2023-08-27T11:28:06.665233+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2023-08-27T11:28:12.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.677732+0000', 'last_change': '2023-08-27T11:28:06.677732+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.677732+0000', 'last_undegraded': '2023-08-27T11:28:06.677732+0000', 'last_fullsized': '2023-08-27T11:28:06.677732+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2023-08-27T11:28:12.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.672762+0000', 'last_change': '2023-08-27T11:28:06.672762+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.672762+0000', 'last_undegraded': '2023-08-27T11:28:06.672762+0000', 'last_fullsized': '2023-08-27T11:28:06.672762+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:12.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2023-08-27T11:28:12.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.667659+0000', 'last_change': '2023-08-27T11:28:06.667659+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.667659+0000', 'last_undegraded': '2023-08-27T11:28:06.667659+0000', 'last_fullsized': '2023-08-27T11:28:06.667659+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:12.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2023-08-27T11:28:12.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.662713+0000', 'last_change': '2023-08-27T11:28:06.662713+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.662713+0000', 'last_undegraded': '2023-08-27T11:28:06.662713+0000', 'last_fullsized': '2023-08-27T11:28:06.662713+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:12.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2023-08-27T11:28:12.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.667753+0000', 'last_change': '2023-08-27T11:28:06.667753+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.667753+0000', 'last_undegraded': '2023-08-27T11:28:06.667753+0000', 'last_fullsized': '2023-08-27T11:28:06.667753+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2023-08-27T11:28:12.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.667875+0000', 'last_change': '2023-08-27T11:28:06.667875+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.667875+0000', 'last_undegraded': '2023-08-27T11:28:06.667875+0000', 'last_fullsized': '2023-08-27T11:28:06.667875+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2023-08-27T11:28:12.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.666981+0000', 'last_change': '2023-08-27T11:28:06.666981+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.666981+0000', 'last_undegraded': '2023-08-27T11:28:06.666981+0000', 'last_fullsized': '2023-08-27T11:28:06.666981+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:12.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2023-08-27T11:28:12.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.662798+0000', 'last_change': '2023-08-27T11:28:06.662798+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.662798+0000', 'last_undegraded': '2023-08-27T11:28:06.662798+0000', 'last_fullsized': '2023-08-27T11:28:06.662798+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:12.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2023-08-27T11:28:12.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.666965+0000', 'last_change': '2023-08-27T11:28:06.666965+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.666965+0000', 'last_undegraded': '2023-08-27T11:28:06.666965+0000', 'last_fullsized': '2023-08-27T11:28:06.666965+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:12.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2023-08-27T11:28:12.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.668224+0000', 'last_change': '2023-08-27T11:28:06.668224+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.668224+0000', 'last_undegraded': '2023-08-27T11:28:06.668224+0000', 'last_fullsized': '2023-08-27T11:28:06.668224+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:12.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2023-08-27T11:28:12.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.663693+0000', 'last_change': '2023-08-27T11:28:06.663693+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.663693+0000', 'last_undegraded': '2023-08-27T11:28:06.663693+0000', 'last_fullsized': '2023-08-27T11:28:06.663693+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2023-08-27T11:28:12.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.669584+0000', 'last_change': '2023-08-27T11:28:06.669584+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.669584+0000', 'last_undegraded': '2023-08-27T11:28:06.669584+0000', 'last_fullsized': '2023-08-27T11:28:06.669584+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2023-08-27T11:28:12.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.665390+0000', 'last_change': '2023-08-27T11:28:06.665390+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.665390+0000', 'last_undegraded': '2023-08-27T11:28:06.665390+0000', 'last_fullsized': '2023-08-27T11:28:06.665390+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:12.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2023-08-27T11:28:12.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.663341+0000', 'last_change': '2023-08-27T11:28:06.663341+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.663341+0000', 'last_undegraded': '2023-08-27T11:28:06.663341+0000', 'last_fullsized': '2023-08-27T11:28:06.663341+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2023-08-27T11:28:12.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.674565+0000', 'last_change': '2023-08-27T11:28:06.674565+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.674565+0000', 'last_undegraded': '2023-08-27T11:28:06.674565+0000', 'last_fullsized': '2023-08-27T11:28:06.674565+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:12.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2023-08-27T11:28:12.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.669801+0000', 'last_change': '2023-08-27T11:28:06.669801+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.669801+0000', 'last_undegraded': '2023-08-27T11:28:06.669801+0000', 'last_fullsized': '2023-08-27T11:28:06.669801+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2023-08-27T11:28:12.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.677428+0000', 'last_change': '2023-08-27T11:28:06.677428+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.677428+0000', 'last_undegraded': '2023-08-27T11:28:06.677428+0000', 'last_fullsized': '2023-08-27T11:28:06.677428+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2023-08-27T11:28:12.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.662770+0000', 'last_change': '2023-08-27T11:28:06.662770+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.662770+0000', 'last_undegraded': '2023-08-27T11:28:06.662770+0000', 'last_fullsized': '2023-08-27T11:28:06.662770+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2023-08-27T11:28:12.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.662917+0000', 'last_change': '2023-08-27T11:28:06.662917+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.662917+0000', 'last_undegraded': '2023-08-27T11:28:06.662917+0000', 'last_fullsized': '2023-08-27T11:28:06.662917+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2023-08-27T11:28:12.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.667437+0000', 'last_change': '2023-08-27T11:28:06.667437+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.667437+0000', 'last_undegraded': '2023-08-27T11:28:06.667437+0000', 'last_fullsized': '2023-08-27T11:28:06.667437+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2023-08-27T11:28:12.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.674563+0000', 'last_change': '2023-08-27T11:28:06.674563+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.674563+0000', 'last_undegraded': '2023-08-27T11:28:06.674563+0000', 'last_fullsized': '2023-08-27T11:28:06.674563+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2023-08-27T11:28:12.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.671268+0000', 'last_change': '2023-08-27T11:28:06.671268+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.671268+0000', 'last_undegraded': '2023-08-27T11:28:06.671268+0000', 'last_fullsized': '2023-08-27T11:28:06.671268+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2023-08-27T11:28:12.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.673997+0000', 'last_change': '2023-08-27T11:28:06.673997+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.673997+0000', 'last_undegraded': '2023-08-27T11:28:06.673997+0000', 'last_fullsized': '2023-08-27T11:28:06.673997+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2023-08-27T11:28:12.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.673970+0000', 'last_change': '2023-08-27T11:28:06.673970+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.673970+0000', 'last_undegraded': '2023-08-27T11:28:06.673970+0000', 'last_fullsized': '2023-08-27T11:28:06.673970+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2023-08-27T11:28:12.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.670630+0000', 'last_change': '2023-08-27T11:28:06.670630+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.670630+0000', 'last_undegraded': '2023-08-27T11:28:06.670630+0000', 'last_fullsized': '2023-08-27T11:28:06.670630+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:12.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2023-08-27T11:28:12.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.663309+0000', 'last_change': '2023-08-27T11:28:06.663309+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.663309+0000', 'last_undegraded': '2023-08-27T11:28:06.663309+0000', 'last_fullsized': '2023-08-27T11:28:06.663309+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2023-08-27T11:28:12.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.667804+0000', 'last_change': '2023-08-27T11:28:06.667804+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.667804+0000', 'last_undegraded': '2023-08-27T11:28:06.667804+0000', 'last_fullsized': '2023-08-27T11:28:06.667804+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2023-08-27T11:28:12.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.670493+0000', 'last_change': '2023-08-27T11:28:06.670493+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.670493+0000', 'last_undegraded': '2023-08-27T11:28:06.670493+0000', 'last_fullsized': '2023-08-27T11:28:06.670493+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:12.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2023-08-27T11:28:12.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.662304+0000', 'last_change': '2023-08-27T11:28:06.662304+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.662304+0000', 'last_undegraded': '2023-08-27T11:28:06.662304+0000', 'last_fullsized': '2023-08-27T11:28:06.662304+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:12.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2023-08-27T11:28:12.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.661902+0000', 'last_change': '2023-08-27T11:28:06.661902+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.661902+0000', 'last_undegraded': '2023-08-27T11:28:06.661902+0000', 'last_fullsized': '2023-08-27T11:28:06.661902+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:12.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2023-08-27T11:28:12.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.669447+0000', 'last_change': '2023-08-27T11:28:06.669447+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.669447+0000', 'last_undegraded': '2023-08-27T11:28:06.669447+0000', 'last_fullsized': '2023-08-27T11:28:06.669447+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:28:12.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2023-08-27T11:28:12.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.662935+0000', 'last_change': '2023-08-27T11:28:06.662935+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.662935+0000', 'last_undegraded': '2023-08-27T11:28:06.662935+0000', 'last_fullsized': '2023-08-27T11:28:06.662935+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:12.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2023-08-27T11:28:12.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.669454+0000', 'last_change': '2023-08-27T11:28:06.669454+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.669454+0000', 'last_undegraded': '2023-08-27T11:28:06.669454+0000', 'last_fullsized': '2023-08-27T11:28:06.669454+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2023-08-27T11:28:12.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.663233+0000', 'last_change': '2023-08-27T11:28:06.663233+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.663233+0000', 'last_undegraded': '2023-08-27T11:28:06.663233+0000', 'last_fullsized': '2023-08-27T11:28:06.663233+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:12.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2023-08-27T11:28:12.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.672405+0000', 'last_change': '2023-08-27T11:28:06.672405+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.672405+0000', 'last_undegraded': '2023-08-27T11:28:06.672405+0000', 'last_fullsized': '2023-08-27T11:28:06.672405+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:12.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2023-08-27T11:28:12.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.664777+0000', 'last_change': '2023-08-27T11:28:06.664777+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.664777+0000', 'last_undegraded': '2023-08-27T11:28:06.664777+0000', 'last_fullsized': '2023-08-27T11:28:06.664777+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2023-08-27T11:28:12.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.665638+0000', 'last_change': '2023-08-27T11:28:06.665638+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.665638+0000', 'last_undegraded': '2023-08-27T11:28:06.665638+0000', 'last_fullsized': '2023-08-27T11:28:06.665638+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2023-08-27T11:28:12.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.670514+0000', 'last_change': '2023-08-27T11:28:06.670514+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.670514+0000', 'last_undegraded': '2023-08-27T11:28:06.670514+0000', 'last_fullsized': '2023-08-27T11:28:06.670514+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2023-08-27T11:28:12.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.671123+0000', 'last_change': '2023-08-27T11:28:06.671123+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.671123+0000', 'last_undegraded': '2023-08-27T11:28:06.671123+0000', 'last_fullsized': '2023-08-27T11:28:06.671123+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:12.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2023-08-27T11:28:12.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.673604+0000', 'last_change': '2023-08-27T11:28:06.673604+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.673604+0000', 'last_undegraded': '2023-08-27T11:28:06.673604+0000', 'last_fullsized': '2023-08-27T11:28:06.673604+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2023-08-27T11:28:12.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.668771+0000', 'last_change': '2023-08-27T11:28:06.668771+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.668771+0000', 'last_undegraded': '2023-08-27T11:28:06.668771+0000', 'last_fullsized': '2023-08-27T11:28:06.668771+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2023-08-27T11:28:12.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.670120+0000', 'last_change': '2023-08-27T11:28:06.670120+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.670120+0000', 'last_undegraded': '2023-08-27T11:28:06.670120+0000', 'last_fullsized': '2023-08-27T11:28:06.670120+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:28:12.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2023-08-27T11:28:12.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.664201+0000', 'last_change': '2023-08-27T11:28:06.664201+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.664201+0000', 'last_undegraded': '2023-08-27T11:28:06.664201+0000', 'last_fullsized': '2023-08-27T11:28:06.664201+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:12.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2023-08-27T11:28:12.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.670946+0000', 'last_change': '2023-08-27T11:28:06.670946+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.670946+0000', 'last_undegraded': '2023-08-27T11:28:06.670946+0000', 'last_fullsized': '2023-08-27T11:28:06.670946+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:12.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2023-08-27T11:28:12.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.669155+0000', 'last_change': '2023-08-27T11:28:06.669155+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.669155+0000', 'last_undegraded': '2023-08-27T11:28:06.669155+0000', 'last_fullsized': '2023-08-27T11:28:06.669155+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2023-08-27T11:28:12.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.669642+0000', 'last_change': '2023-08-27T11:28:06.669642+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.669642+0000', 'last_undegraded': '2023-08-27T11:28:06.669642+0000', 'last_fullsized': '2023-08-27T11:28:06.669642+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:28:12.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2023-08-27T11:28:12.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.669129+0000', 'last_change': '2023-08-27T11:28:06.669129+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.669129+0000', 'last_undegraded': '2023-08-27T11:28:06.669129+0000', 'last_fullsized': '2023-08-27T11:28:06.669129+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:12.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2023-08-27T11:28:12.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.670944+0000', 'last_change': '2023-08-27T11:28:06.670944+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.670944+0000', 'last_undegraded': '2023-08-27T11:28:06.670944+0000', 'last_fullsized': '2023-08-27T11:28:06.670944+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:12.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2023-08-27T11:28:12.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.664810+0000', 'last_change': '2023-08-27T11:28:06.664810+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.664810+0000', 'last_undegraded': '2023-08-27T11:28:06.664810+0000', 'last_fullsized': '2023-08-27T11:28:06.664810+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:12.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2023-08-27T11:28:12.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.673646+0000', 'last_change': '2023-08-27T11:28:06.673646+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.673646+0000', 'last_undegraded': '2023-08-27T11:28:06.673646+0000', 'last_fullsized': '2023-08-27T11:28:06.673646+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:12.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2023-08-27T11:28:12.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.667293+0000', 'last_change': '2023-08-27T11:28:06.667293+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.667293+0000', 'last_undegraded': '2023-08-27T11:28:06.667293+0000', 'last_fullsized': '2023-08-27T11:28:06.667293+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:12.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2023-08-27T11:28:12.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.671744+0000', 'last_change': '2023-08-27T11:28:06.671744+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.671744+0000', 'last_undegraded': '2023-08-27T11:28:06.671744+0000', 'last_fullsized': '2023-08-27T11:28:06.671744+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:28:12.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2023-08-27T11:28:12.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:06.664961+0000', 'last_change': '2023-08-27T11:28:06.664961+0000', 'last_active': '2023-08-27T11:28:06.651961+0000', 'last_peered': '2023-08-27T11:28:06.651961+0000', 'last_clean': '2023-08-27T11:28:06.651961+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:06.664961+0000', 'last_undegraded': '2023-08-27T11:28:06.664961+0000', 'last_fullsized': '2023-08-27T11:28:06.664961+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:06.651961+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:12.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-27T11:28:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.834+0000 7fa929b0f700 1 Processor -- start 2023-08-27T11:28:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.834+0000 7fa929b0f700 1 -- start start 2023-08-27T11:28:12.838 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa929b0f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa92415b070 0x7fa9240cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa929b0f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92406e980 0x7fa9240ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa929b0f700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92415dad0 con 0x7fa9240cdf60 2023-08-27T11:28:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa929b0f700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92415d8e0 con 0x7fa92415b070 2023-08-27T11:28:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa929b0f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92415ddb0 con 0x7fa92406e980 2023-08-27T11:28:12.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa9237fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa92415b070 0x7fa9240cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.840 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa9237fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa92415b070 0x7fa9240cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59140/0 (socket says 172.21.15.45:59140) 2023-08-27T11:28:12.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa923fff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92406e980 0x7fa9240ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa9237fe700 1 -- 172.21.15.45:0/43812537 learned_addr learned my addr 172.21.15.45:0/43812537 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:12.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa9237fe700 1 -- 172.21.15.45:0/43812537 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92406e980 msgr2=0x7fa9240ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:12.841 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa9237fe700 1 --2- 172.21.15.45:0/43812537 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92406e980 0x7fa9240ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa9237fe700 1 -- 172.21.15.45:0/43812537 >> v1:172.21.15.135:6789/0 conn(0x7fa9240cdf60 legacy=0x7fa92406e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa9237fe700 1 -- 172.21.15.45:0/43812537 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92415e090 con 0x7fa92415b070 2023-08-27T11:28:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.835+0000 7fa9227fc700 1 -- 172.21.15.45:0/43812537 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607182388 0 0) 0x7fa92415dad0 con 0x7fa9240cdf60 2023-08-27T11:28:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa9237fe700 1 --2- 172.21.15.45:0/43812537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92415b070 0x7fa9240cda20 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa9140098f0 tx=0x7fa914009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca867afedbd36af1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa9227fc700 1 -- 172.21.15.45:0/43812537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914011830 con 0x7fa92415b070 2023-08-27T11:28:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa9227fc700 1 -- 172.21.15.45:0/43812537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa914011990 con 0x7fa92415b070 2023-08-27T11:28:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa9227fc700 1 -- 172.21.15.45:0/43812537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914011d20 con 0x7fa92415b070 2023-08-27T11:28:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa929b0f700 1 -- 172.21.15.45:0/43812537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92415b070 msgr2=0x7fa9240cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa929b0f700 1 --2- 172.21.15.45:0/43812537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92415b070 0x7fa9240cda20 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa9140098f0 tx=0x7fa914009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa929b0f700 1 -- 172.21.15.45:0/43812537 shutdown_connections 2023-08-27T11:28:12.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa929b0f700 1 --2- 172.21.15.45:0/43812537 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92406e980 0x7fa9240ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa929b0f700 1 --2- 172.21.15.45:0/43812537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92415b070 0x7fa9240cda20 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa929b0f700 1 -- 172.21.15.45:0/43812537 >> 172.21.15.45:0/43812537 conn(0x7fa9240cfe50 msgr2=0x7fa9240d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:12.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa929b0f700 1 -- 172.21.15.45:0/43812537 shutdown_connections 2023-08-27T11:28:12.844 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.836+0000 7fa929b0f700 1 -- 172.21.15.45:0/43812537 wait complete. 2023-08-27T11:28:12.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.838+0000 7fa929b0f700 1 Processor -- start 2023-08-27T11:28:12.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.838+0000 7fa929b0f700 1 -- start start 2023-08-27T11:28:12.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.838+0000 7fa929b0f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92406e980 0x7fa9241580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.838+0000 7fa929b0f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9240cdf60 0x7fa9240c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.838+0000 7fa929b0f700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92415b070 0x7fa9240b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.838+0000 7fa929b0f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9241f3730 con 0x7fa92406e980 2023-08-27T11:28:12.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.838+0000 7fa929b0f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9241f3a10 con 0x7fa9240cdf60 2023-08-27T11:28:12.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.838+0000 7fa929b0f700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240c9c10 con 0x7fa92415b070 2023-08-27T11:28:12.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa9237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92406e980 0x7fa9241580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa9237fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92406e980 0x7fa9241580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59144/0 (socket says 172.21.15.45:59144) 2023-08-27T11:28:12.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa9237fe700 1 -- 172.21.15.45:0/3441202711 learned_addr learned my addr 172.21.15.45:0/3441202711 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:12.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa923fff700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92415b070 0x7fa9240b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa922ffd700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9240cdf60 0x7fa9240c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa9237fe700 1 -- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92415b070 msgr2=0x7fa9240b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa9237fe700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92415b070 0x7fa9240b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa9237fe700 1 -- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9240cdf60 msgr2=0x7fa9240c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:12.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa9237fe700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9240cdf60 0x7fa9240c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:12.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa9237fe700 1 -- 172.21.15.45:0/3441202711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa914009550 con 0x7fa92406e980 2023-08-27T11:28:12.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa9237fe700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92406e980 0x7fa9241580b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa914014040 tx=0x7fa914004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa91bfff700 1 -- 172.21.15.45:0/3441202711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9140036a0 con 0x7fa92406e980 2023-08-27T11:28:12.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa91bfff700 1 -- 172.21.15.45:0/3441202711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91401e070 con 0x7fa92406e980 2023-08-27T11:28:12.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9240c9ef0 con 0x7fa92406e980 2023-08-27T11:28:12.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa91bfff700 1 -- 172.21.15.45:0/3441202711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914003a20 con 0x7fa92406e980 2023-08-27T11:28:12.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.839+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9241585f0 con 0x7fa92406e980 2023-08-27T11:28:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.840+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9240c9a70 con 0x7fa92406e980 2023-08-27T11:28:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.841+0000 7fa91bfff700 1 -- 172.21.15.45:0/3441202711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fa914003b80 con 0x7fa92406e980 2023-08-27T11:28:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.841+0000 7fa91bfff700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa90406a170 0x7fa90406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.841+0000 7fa91bfff700 1 -- 172.21.15.45:0/3441202711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fa9140898f0 con 0x7fa92406e980 2023-08-27T11:28:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.841+0000 7fa922ffd700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa90406a170 0x7fa90406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:12.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.842+0000 7fa922ffd700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa90406a170 0x7fa90406c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9240c4dc0 tx=0x7fa90c009040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:12.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:12.846+0000 7fa91bfff700 1 -- 172.21.15.45:0/3441202711 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa9240c9a70 con 0x7fa92406e980 2023-08-27T11:28:13.043 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:13.041+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.140+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa9240c9a70 con 0x7fa92406e980 2023-08-27T11:28:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.141+0000 7fa91bfff700 1 -- 172.21.15.45:0/3441202711 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1098 (secure 0 0 0) 0x7fa9240c9a70 con 0x7fa92406e980 2023-08-27T11:28:13.143 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:13.144 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":990,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1693134714,"num_in_osds":8,"osd_in_since":1693134705,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":291758080,"bytes_avail":772802355200,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415,"read_bytes_sec":1225,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":41,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":7446}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-08-27T11:26:31.287744+0000","services":{}},"progress_events":{}} 2023-08-27T11:28:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa90406a170 msgr2=0x7fa90406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa90406a170 0x7fa90406c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9240c4dc0 tx=0x7fa90c009040 comp rx=0 tx=0).stop 2023-08-27T11:28:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92406e980 msgr2=0x7fa9241580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92406e980 0x7fa9241580b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa914014040 tx=0x7fa914004750 comp rx=0 tx=0).stop 2023-08-27T11:28:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 shutdown_connections 2023-08-27T11:28:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fa90406a170 0x7fa90406c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa92406e980 0x7fa9241580b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa92415b070 0x7fa9240b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 --2- 172.21.15.45:0/3441202711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9240cdf60 0x7fa9240c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 >> 172.21.15.45:0/3441202711 conn(0x7fa9240cfe50 msgr2=0x7fa924070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 shutdown_connections 2023-08-27T11:28:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.143+0000 7fa929b0f700 1 -- 172.21.15.45:0/3441202711 wait complete. 2023-08-27T11:28:13.157 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-27T11:28:13.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.283+0000 7f38e95fc700 1 Processor -- start 2023-08-27T11:28:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.283+0000 7f38e95fc700 1 -- start start 2023-08-27T11:28:13.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e95fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38e40ae1e0 0x7f38e40ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e95fc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e40ccb80 con 0x7f38e40cc650 2023-08-27T11:28:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e95fc700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e40cce60 con 0x7f38e415e990 2023-08-27T11:28:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e95fc700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e40cd600 con 0x7f38e40ae1e0 2023-08-27T11:28:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e37fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38e40ae1e0 0x7f38e40ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e27fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f38e40cc650 0x7f38e40adad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45930/0 (socket says 172.21.15.45:45930) 2023-08-27T11:28:13.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e27fc700 1 -- 172.21.15.45:0/1265152818 learned_addr learned my addr 172.21.15.45:0/1265152818 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:13.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e37fe700 1 -- 172.21.15.45:0/1265152818 >> v1:172.21.15.135:6789/0 conn(0x7f38e415e990 legacy=0x7f38e40ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e37fe700 1 -- 172.21.15.45:0/1265152818 >> v1:172.21.15.45:6789/0 conn(0x7f38e40cc650 legacy=0x7f38e40adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e37fe700 1 -- 172.21.15.45:0/1265152818 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e40cd8e0 con 0x7f38e40ae1e0 2023-08-27T11:28:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e37fe700 1 --2- 172.21.15.45:0/1265152818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e40ae1e0 0x7f38e40ced40 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009d90 tx=0x7f38dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb487b9eba3bb87 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e1ffb700 1 -- 172.21.15.45:0/1265152818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00e070 con 0x7f38e40ae1e0 2023-08-27T11:28:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e1ffb700 1 -- 172.21.15.45:0/1265152818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc00bbd0 con 0x7f38e40ae1e0 2023-08-27T11:28:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.284+0000 7f38e1ffb700 1 -- 172.21.15.45:0/1265152818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc012430 con 0x7f38e40ae1e0 2023-08-27T11:28:13.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.285+0000 7f38e95fc700 1 -- 172.21.15.45:0/1265152818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e40ae1e0 msgr2=0x7f38e40ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.285+0000 7f38e95fc700 1 --2- 172.21.15.45:0/1265152818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e40ae1e0 0x7f38e40ced40 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009d90 tx=0x7f38dc00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.285+0000 7f38e95fc700 1 -- 172.21.15.45:0/1265152818 shutdown_connections 2023-08-27T11:28:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.285+0000 7f38e95fc700 1 --2- 172.21.15.45:0/1265152818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e40ae1e0 0x7f38e40ced40 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.285+0000 7f38e95fc700 1 -- 172.21.15.45:0/1265152818 >> 172.21.15.45:0/1265152818 conn(0x7f38e40d0f60 msgr2=0x7f38e40d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:13.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.285+0000 7f38e95fc700 1 -- 172.21.15.45:0/1265152818 shutdown_connections 2023-08-27T11:28:13.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.285+0000 7f38e95fc700 1 -- 172.21.15.45:0/1265152818 wait complete. 2023-08-27T11:28:13.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.286+0000 7f38e95fc700 1 Processor -- start 2023-08-27T11:28:13.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e95fc700 1 -- start start 2023-08-27T11:28:13.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e95fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38e40cc650 0x7f38e40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e95fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38e415e990 0x7f38e40af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e95fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e41f0870 0x7f38e40b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e95fc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e414fd20 con 0x7f38e41f0870 2023-08-27T11:28:13.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e95fc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4157820 con 0x7f38e40cc650 2023-08-27T11:28:13.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e95fc700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4156780 con 0x7f38e415e990 2023-08-27T11:28:13.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e41f0870 0x7f38e40b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e41f0870 0x7f38e40b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59162/0 (socket says 172.21.15.45:59162) 2023-08-27T11:28:13.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e37fe700 1 -- 172.21.15.45:0/3054599096 learned_addr learned my addr 172.21.15.45:0/3054599096 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:13.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e2ffd700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38e40cc650 0x7f38e40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.287+0000 7f38e27fc700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38e415e990 0x7f38e40af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38e37fe700 1 -- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38e415e990 msgr2=0x7f38e40af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38e37fe700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38e415e990 0x7f38e40af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38e37fe700 1 -- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38e40cc650 msgr2=0x7f38e40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38e37fe700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38e40cc650 0x7f38e40aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38e37fe700 1 -- 172.21.15.45:0/3054599096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc009a70 con 0x7f38e41f0870 2023-08-27T11:28:13.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38e37fe700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e41f0870 0x7f38e40b55d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f38dc014040 tx=0x7f38dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:13.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38cb7fe700 1 -- 172.21.15.45:0/3054599096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc012d20 con 0x7f38e41f0870 2023-08-27T11:28:13.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38cb7fe700 1 -- 172.21.15.45:0/3054599096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc020070 con 0x7f38e41f0870 2023-08-27T11:28:13.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38cb7fe700 1 -- 172.21.15.45:0/3054599096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc012ea0 con 0x7f38e41f0870 2023-08-27T11:28:13.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e4156a60 con 0x7f38e41f0870 2023-08-27T11:28:13.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.288+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e4157020 con 0x7f38e41f0870 2023-08-27T11:28:13.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.289+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e4067bf0 con 0x7f38e41f0870 2023-08-27T11:28:13.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.290+0000 7f38cb7fe700 1 -- 172.21.15.45:0/3054599096 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f38dc004280 con 0x7f38e41f0870 2023-08-27T11:28:13.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.290+0000 7f38cb7fe700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f38d006a170 0x7f38d006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:13.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.290+0000 7f38cb7fe700 1 -- 172.21.15.45:0/3054599096 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f38dc022070 con 0x7f38e41f0870 2023-08-27T11:28:13.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.290+0000 7f38e2ffd700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f38d006a170 0x7f38d006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:13.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.291+0000 7f38e2ffd700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f38d006a170 0x7f38d006c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f38d4007fd0 tx=0x7f38d4009040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:13.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.295+0000 7f38cb7fe700 1 -- 172.21.15.45:0/3054599096 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f38dc055a40 con 0x7f38e41f0870 2023-08-27T11:28:13.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.496+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 --> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38e415ed70 con 0x7f38d006a170 2023-08-27T11:28:13.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.505+0000 7f38cb7fe700 1 -- 172.21.15.45:0/3054599096 <== mgr.6709 v2:172.21.15.135:6832/3812189695 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306377 (secure 0 0 0) 0x7f38d4001560 con 0x7f38d006a170 2023-08-27T11:28:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:13.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.508+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f38d006a170 msgr2=0x7f38d006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.508+0000 7f38e95fc700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f38d006a170 0x7f38d006c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f38d4007fd0 tx=0x7f38d4009040 comp rx=0 tx=0).stop 2023-08-27T11:28:13.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.508+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e41f0870 msgr2=0x7f38e40b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:13.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.508+0000 7f38e95fc700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e41f0870 0x7f38e40b55d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f38dc014040 tx=0x7f38dc004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.509+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 shutdown_connections 2023-08-27T11:28:13.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.509+0000 7f38e95fc700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f38d006a170 0x7f38d006c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.509+0000 7f38e95fc700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38e41f0870 0x7f38e40b55d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.509+0000 7f38e95fc700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38e40cc650 0x7f38e40aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.509+0000 7f38e95fc700 1 --2- 172.21.15.45:0/3054599096 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f38e415e990 0x7f38e40af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.509+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 >> 172.21.15.45:0/3054599096 conn(0x7f38e40d0f60 msgr2=0x7f38e40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.509+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 shutdown_connections 2023-08-27T11:28:13.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:13.509+0000 7f38e95fc700 1 -- 172.21.15.45:0/3054599096 wait complete. 2023-08-27T11:28:13.516 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:13.523 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:28:11.501465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2213,"ondisk_log_size":2213,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284920,"kb_used_data":10512,"kb_used_omap":13,"kb_used_meta":274290,"kb_avail":754689800,"statfs":{"total":773094113280,"available":772802355200,"internally_reserved":0,"allocated":10764288,"data_stored":6183480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":280873048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.849363"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674262+0000","last_change":"2023-08-27T11:28:06.674262+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674262+0000","last_undegraded":"2023-08-27T11:28:06.674262+0000","last_fullsized":"2023-08-27T11:28:06.674262+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663991+0000","last_change":"2023-08-27T11:28:06.663991+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663991+0000","last_undegraded":"2023-08-27T11:28:06.663991+0000","last_fullsized":"2023-08-27T11:28:06.663991+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671195+0000","last_change":"2023-08-27T11:28:06.671195+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671195+0000","last_undegraded":"2023-08-27T11:28:06.671195+0000","last_fullsized":"2023-08-27T11:28:06.671195+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669142+0000","last_change":"2023-08-27T11:28:06.669142+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669142+0000","last_undegraded":"2023-08-27T11:28:06.669142+0000","last_fullsized":"2023-08-27T11:28:06.669142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668212+0000","last_change":"2023-08-27T11:28:06.668212+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668212+0000","last_undegraded":"2023-08-27T11:28:06.668212+0000","last_fullsized":"2023-08-27T11:28:06.668212+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670471+0000","last_change":"2023-08-27T11:28:06.670471+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670471+0000","last_undegraded":"2023-08-27T11:28:06.670471+0000","last_fullsized":"2023-08-27T11:28:06.670471+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667376+0000","last_change":"2023-08-27T11:28:06.667376+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667376+0000","last_undegraded":"2023-08-27T11:28:06.667376+0000","last_fullsized":"2023-08-27T11:28:06.667376+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670912+0000","last_change":"2023-08-27T11:28:06.670912+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670912+0000","last_undegraded":"2023-08-27T11:28:06.670912+0000","last_fullsized":"2023-08-27T11:28:06.670912+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669900+0000","last_change":"2023-08-27T11:28:06.669900+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669900+0000","last_undegraded":"2023-08-27T11:28:06.669900+0000","last_fullsized":"2023-08-27T11:28:06.669900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674502+0000","last_change":"2023-08-27T11:28:06.674502+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674502+0000","last_undegraded":"2023-08-27T11:28:06.674502+0000","last_fullsized":"2023-08-27T11:28:06.674502+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665233+0000","last_change":"2023-08-27T11:28:06.665233+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665233+0000","last_undegraded":"2023-08-27T11:28:06.665233+0000","last_fullsized":"2023-08-27T11:28:06.665233+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.677732+0000","last_change":"2023-08-27T11:28:06.677732+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.677732+0000","last_undegraded":"2023-08-27T11:28:06.677732+0000","last_fullsized":"2023-08-27T11:28:06.677732+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.672762+0000","last_change":"2023-08-27T11:28:06.672762+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.672762+0000","last_undegraded":"2023-08-27T11:28:06.672762+0000","last_fullsized":"2023-08-27T11:28:06.672762+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667659+0000","last_change":"2023-08-27T11:28:06.667659+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667659+0000","last_undegraded":"2023-08-27T11:28:06.667659+0000","last_fullsized":"2023-08-27T11:28:06.667659+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662713+0000","last_change":"2023-08-27T11:28:06.662713+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662713+0000","last_undegraded":"2023-08-27T11:28:06.662713+0000","last_fullsized":"2023-08-27T11:28:06.662713+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667753+0000","last_change":"2023-08-27T11:28:06.667753+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667753+0000","last_undegraded":"2023-08-27T11:28:06.667753+0000","last_fullsized":"2023-08-27T11:28:06.667753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667875+0000","last_change":"2023-08-27T11:28:06.667875+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667875+0000","last_undegraded":"2023-08-27T11:28:06.667875+0000","last_fullsized":"2023-08-27T11:28:06.667875+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.666981+0000","last_change":"2023-08-27T11:28:06.666981+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.666981+0000","last_undegraded":"2023-08-27T11:28:06.666981+0000","last_fullsized":"2023-08-27T11:28:06.666981+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662798+0000","last_change":"2023-08-27T11:28:06.662798+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662798+0000","last_undegraded":"2023-08-27T11:28:06.662798+0000","last_fullsized":"2023-08-27T11:28:06.662798+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.666965+0000","last_change":"2023-08-27T11:28:06.666965+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.666965+0000","last_undegraded":"2023-08-27T11:28:06.666965+0000","last_fullsized":"2023-08-27T11:28:06.666965+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668224+0000","last_change":"2023-08-27T11:28:06.668224+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668224+0000","last_undegraded":"2023-08-27T11:28:06.668224+0000","last_fullsized":"2023-08-27T11:28:06.668224+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663693+0000","last_change":"2023-08-27T11:28:06.663693+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663693+0000","last_undegraded":"2023-08-27T11:28:06.663693+0000","last_fullsized":"2023-08-27T11:28:06.663693+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669584+0000","last_change":"2023-08-27T11:28:06.669584+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669584+0000","last_undegraded":"2023-08-27T11:28:06.669584+0000","last_fullsized":"2023-08-27T11:28:06.669584+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665390+0000","last_change":"2023-08-27T11:28:06.665390+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665390+0000","last_undegraded":"2023-08-27T11:28:06.665390+0000","last_fullsized":"2023-08-27T11:28:06.665390+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663341+0000","last_change":"2023-08-27T11:28:06.663341+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663341+0000","last_undegraded":"2023-08-27T11:28:06.663341+0000","last_fullsized":"2023-08-27T11:28:06.663341+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674565+0000","last_change":"2023-08-27T11:28:06.674565+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674565+0000","last_undegraded":"2023-08-27T11:28:06.674565+0000","last_fullsized":"2023-08-27T11:28:06.674565+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669801+0000","last_change":"2023-08-27T11:28:06.669801+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669801+0000","last_undegraded":"2023-08-27T11:28:06.669801+0000","last_fullsized":"2023-08-27T11:28:06.669801+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.677428+0000","last_change":"2023-08-27T11:28:06.677428+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.677428+0000","last_undegraded":"2023-08-27T11:28:06.677428+0000","last_fullsized":"2023-08-27T11:28:06.677428+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662770+0000","last_change":"2023-08-27T11:28:06.662770+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662770+0000","last_undegraded":"2023-08-27T11:28:06.662770+0000","last_fullsized":"2023-08-27T11:28:06.662770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662917+0000","last_change":"2023-08-27T11:28:06.662917+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662917+0000","last_undegraded":"2023-08-27T11:28:06.662917+0000","last_fullsized":"2023-08-27T11:28:06.662917+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667437+0000","last_change":"2023-08-27T11:28:06.667437+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667437+0000","last_undegraded":"2023-08-27T11:28:06.667437+0000","last_fullsized":"2023-08-27T11:28:06.667437+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.674563+0000","last_change":"2023-08-27T11:28:06.674563+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.674563+0000","last_undegraded":"2023-08-27T11:28:06.674563+0000","last_fullsized":"2023-08-27T11:28:06.674563+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671268+0000","last_change":"2023-08-27T11:28:06.671268+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671268+0000","last_undegraded":"2023-08-27T11:28:06.671268+0000","last_fullsized":"2023-08-27T11:28:06.671268+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682500+0000","last_change":"2023-08-27T11:28:06.680510+0000","last_active":"2023-08-27T11:28:06.682500+0000","last_peered":"2023-08-27T11:28:06.682500+0000","last_clean":"2023-08-27T11:28:06.682500+0000","last_became_active":"2023-08-27T11:28:06.680268+0000","last_became_peered":"2023-08-27T11:28:06.680268+0000","last_unstale":"2023-08-27T11:28:06.682500+0000","last_undegraded":"2023-08-27T11:28:06.682500+0000","last_fullsized":"2023-08-27T11:28:06.682500+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:53:53.355949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673997+0000","last_change":"2023-08-27T11:28:06.673997+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673997+0000","last_undegraded":"2023-08-27T11:28:06.673997+0000","last_fullsized":"2023-08-27T11:28:06.673997+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680586+0000","last_change":"2023-08-27T11:28:06.678558+0000","last_active":"2023-08-27T11:28:06.680586+0000","last_peered":"2023-08-27T11:28:06.680586+0000","last_clean":"2023-08-27T11:28:06.680586+0000","last_became_active":"2023-08-27T11:28:06.678323+0000","last_became_peered":"2023-08-27T11:28:06.678323+0000","last_unstale":"2023-08-27T11:28:06.680586+0000","last_undegraded":"2023-08-27T11:28:06.680586+0000","last_fullsized":"2023-08-27T11:28:06.680586+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:32:33.684096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673970+0000","last_change":"2023-08-27T11:28:06.673970+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673970+0000","last_undegraded":"2023-08-27T11:28:06.673970+0000","last_fullsized":"2023-08-27T11:28:06.673970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675092+0000","last_change":"2023-08-27T11:28:06.674106+0000","last_active":"2023-08-27T11:28:06.675092+0000","last_peered":"2023-08-27T11:28:06.675092+0000","last_clean":"2023-08-27T11:28:06.675092+0000","last_became_active":"2023-08-27T11:28:06.673843+0000","last_became_peered":"2023-08-27T11:28:06.673843+0000","last_unstale":"2023-08-27T11:28:06.675092+0000","last_undegraded":"2023-08-27T11:28:06.675092+0000","last_fullsized":"2023-08-27T11:28:06.675092+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:47:18.098063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670630+0000","last_change":"2023-08-27T11:28:06.670630+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670630+0000","last_undegraded":"2023-08-27T11:28:06.670630+0000","last_fullsized":"2023-08-27T11:28:06.670630+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675059+0000","last_change":"2023-08-27T11:28:06.674658+0000","last_active":"2023-08-27T11:28:06.675059+0000","last_peered":"2023-08-27T11:28:06.675059+0000","last_clean":"2023-08-27T11:28:06.675059+0000","last_became_active":"2023-08-27T11:28:06.674384+0000","last_became_peered":"2023-08-27T11:28:06.674384+0000","last_unstale":"2023-08-27T11:28:06.675059+0000","last_undegraded":"2023-08-27T11:28:06.675059+0000","last_fullsized":"2023-08-27T11:28:06.675059+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:33:38.254409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663309+0000","last_change":"2023-08-27T11:28:06.663309+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663309+0000","last_undegraded":"2023-08-27T11:28:06.663309+0000","last_fullsized":"2023-08-27T11:28:06.663309+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680562+0000","last_change":"2023-08-27T11:28:06.678516+0000","last_active":"2023-08-27T11:28:06.680562+0000","last_peered":"2023-08-27T11:28:06.680562+0000","last_clean":"2023-08-27T11:28:06.680562+0000","last_became_active":"2023-08-27T11:28:06.678268+0000","last_became_peered":"2023-08-27T11:28:06.678268+0000","last_unstale":"2023-08-27T11:28:06.680562+0000","last_undegraded":"2023-08-27T11:28:06.680562+0000","last_fullsized":"2023-08-27T11:28:06.680562+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:29:56.120623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667804+0000","last_change":"2023-08-27T11:28:06.667804+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667804+0000","last_undegraded":"2023-08-27T11:28:06.667804+0000","last_fullsized":"2023-08-27T11:28:06.667804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680480+0000","last_change":"2023-08-27T11:28:06.677993+0000","last_active":"2023-08-27T11:28:06.680480+0000","last_peered":"2023-08-27T11:28:06.680480+0000","last_clean":"2023-08-27T11:28:06.680480+0000","last_became_active":"2023-08-27T11:28:06.677679+0000","last_became_peered":"2023-08-27T11:28:06.677679+0000","last_unstale":"2023-08-27T11:28:06.680480+0000","last_undegraded":"2023-08-27T11:28:06.680480+0000","last_fullsized":"2023-08-27T11:28:06.680480+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:58:04.340118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670493+0000","last_change":"2023-08-27T11:28:06.670493+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670493+0000","last_undegraded":"2023-08-27T11:28:06.670493+0000","last_fullsized":"2023-08-27T11:28:06.670493+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676184+0000","last_change":"2023-08-27T11:28:06.675717+0000","last_active":"2023-08-27T11:28:06.676184+0000","last_peered":"2023-08-27T11:28:06.676184+0000","last_clean":"2023-08-27T11:28:06.676184+0000","last_became_active":"2023-08-27T11:28:06.675161+0000","last_became_peered":"2023-08-27T11:28:06.675161+0000","last_unstale":"2023-08-27T11:28:06.676184+0000","last_undegraded":"2023-08-27T11:28:06.676184+0000","last_fullsized":"2023-08-27T11:28:06.676184+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:30.138559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662304+0000","last_change":"2023-08-27T11:28:06.662304+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662304+0000","last_undegraded":"2023-08-27T11:28:06.662304+0000","last_fullsized":"2023-08-27T11:28:06.662304+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680689+0000","last_change":"2023-08-27T11:28:06.680053+0000","last_active":"2023-08-27T11:28:06.680689+0000","last_peered":"2023-08-27T11:28:06.680689+0000","last_clean":"2023-08-27T11:28:06.680689+0000","last_became_active":"2023-08-27T11:28:06.679714+0000","last_became_peered":"2023-08-27T11:28:06.679714+0000","last_unstale":"2023-08-27T11:28:06.680689+0000","last_undegraded":"2023-08-27T11:28:06.680689+0000","last_fullsized":"2023-08-27T11:28:06.680689+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:05:13.718165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.661902+0000","last_change":"2023-08-27T11:28:06.661902+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.661902+0000","last_undegraded":"2023-08-27T11:28:06.661902+0000","last_fullsized":"2023-08-27T11:28:06.661902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.683222+0000","last_change":"2023-08-27T11:28:06.682417+0000","last_active":"2023-08-27T11:28:06.683222+0000","last_peered":"2023-08-27T11:28:06.683222+0000","last_clean":"2023-08-27T11:28:06.683222+0000","last_became_active":"2023-08-27T11:28:06.682174+0000","last_became_peered":"2023-08-27T11:28:06.682174+0000","last_unstale":"2023-08-27T11:28:06.683222+0000","last_undegraded":"2023-08-27T11:28:06.683222+0000","last_fullsized":"2023-08-27T11:28:06.683222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:22.047442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669447+0000","last_change":"2023-08-27T11:28:06.669447+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669447+0000","last_undegraded":"2023-08-27T11:28:06.669447+0000","last_fullsized":"2023-08-27T11:28:06.669447+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.683629+0000","last_change":"2023-08-27T11:28:06.682913+0000","last_active":"2023-08-27T11:28:06.683629+0000","last_peered":"2023-08-27T11:28:06.683629+0000","last_clean":"2023-08-27T11:28:06.683629+0000","last_became_active":"2023-08-27T11:28:06.682773+0000","last_became_peered":"2023-08-27T11:28:06.682773+0000","last_unstale":"2023-08-27T11:28:06.683629+0000","last_undegraded":"2023-08-27T11:28:06.683629+0000","last_fullsized":"2023-08-27T11:28:06.683629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:13.142404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.662935+0000","last_change":"2023-08-27T11:28:06.662935+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.662935+0000","last_undegraded":"2023-08-27T11:28:06.662935+0000","last_fullsized":"2023-08-27T11:28:06.662935+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682449+0000","last_change":"2023-08-27T11:28:06.680796+0000","last_active":"2023-08-27T11:28:06.682449+0000","last_peered":"2023-08-27T11:28:06.682449+0000","last_clean":"2023-08-27T11:28:06.682449+0000","last_became_active":"2023-08-27T11:28:06.680639+0000","last_became_peered":"2023-08-27T11:28:06.680639+0000","last_unstale":"2023-08-27T11:28:06.682449+0000","last_undegraded":"2023-08-27T11:28:06.682449+0000","last_fullsized":"2023-08-27T11:28:06.682449+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:54.607978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669454+0000","last_change":"2023-08-27T11:28:06.669454+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669454+0000","last_undegraded":"2023-08-27T11:28:06.669454+0000","last_fullsized":"2023-08-27T11:28:06.669454+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679081+0000","last_change":"2023-08-27T11:28:06.675117+0000","last_active":"2023-08-27T11:28:06.679081+0000","last_peered":"2023-08-27T11:28:06.679081+0000","last_clean":"2023-08-27T11:28:06.679081+0000","last_became_active":"2023-08-27T11:28:06.674808+0000","last_became_peered":"2023-08-27T11:28:06.674808+0000","last_unstale":"2023-08-27T11:28:06.679081+0000","last_undegraded":"2023-08-27T11:28:06.679081+0000","last_fullsized":"2023-08-27T11:28:06.679081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:26:13.763908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.663233+0000","last_change":"2023-08-27T11:28:06.663233+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.663233+0000","last_undegraded":"2023-08-27T11:28:06.663233+0000","last_fullsized":"2023-08-27T11:28:06.663233+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.673619+0000","last_change":"2023-08-27T11:28:06.670804+0000","last_active":"2023-08-27T11:28:06.673619+0000","last_peered":"2023-08-27T11:28:06.673619+0000","last_clean":"2023-08-27T11:28:06.673619+0000","last_became_active":"2023-08-27T11:28:06.669783+0000","last_became_peered":"2023-08-27T11:28:06.669783+0000","last_unstale":"2023-08-27T11:28:06.673619+0000","last_undegraded":"2023-08-27T11:28:06.673619+0000","last_fullsized":"2023-08-27T11:28:06.673619+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:02:54.035323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.672405+0000","last_change":"2023-08-27T11:28:06.672405+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.672405+0000","last_undegraded":"2023-08-27T11:28:06.672405+0000","last_fullsized":"2023-08-27T11:28:06.672405+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672817+0000","last_change":"2023-08-27T11:28:06.668164+0000","last_active":"2023-08-27T11:28:06.672817+0000","last_peered":"2023-08-27T11:28:06.672817+0000","last_clean":"2023-08-27T11:28:06.672817+0000","last_became_active":"2023-08-27T11:28:06.667870+0000","last_became_peered":"2023-08-27T11:28:06.667870+0000","last_unstale":"2023-08-27T11:28:06.672817+0000","last_undegraded":"2023-08-27T11:28:06.672817+0000","last_fullsized":"2023-08-27T11:28:06.672817+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:01:34.544721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664777+0000","last_change":"2023-08-27T11:28:06.664777+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664777+0000","last_undegraded":"2023-08-27T11:28:06.664777+0000","last_fullsized":"2023-08-27T11:28:06.664777+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.681660+0000","last_change":"2023-08-27T11:28:06.679870+0000","last_active":"2023-08-27T11:28:06.681660+0000","last_peered":"2023-08-27T11:28:06.681660+0000","last_clean":"2023-08-27T11:28:06.681660+0000","last_became_active":"2023-08-27T11:28:06.679627+0000","last_became_peered":"2023-08-27T11:28:06.679627+0000","last_unstale":"2023-08-27T11:28:06.681660+0000","last_undegraded":"2023-08-27T11:28:06.681660+0000","last_fullsized":"2023-08-27T11:28:06.681660+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:28:35.146679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.665638+0000","last_change":"2023-08-27T11:28:06.665638+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.665638+0000","last_undegraded":"2023-08-27T11:28:06.665638+0000","last_fullsized":"2023-08-27T11:28:06.665638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682022+0000","last_change":"2023-08-27T11:28:06.681196+0000","last_active":"2023-08-27T11:28:06.682022+0000","last_peered":"2023-08-27T11:28:06.682022+0000","last_clean":"2023-08-27T11:28:06.682022+0000","last_became_active":"2023-08-27T11:28:06.680914+0000","last_became_peered":"2023-08-27T11:28:06.680914+0000","last_unstale":"2023-08-27T11:28:06.682022+0000","last_undegraded":"2023-08-27T11:28:06.682022+0000","last_fullsized":"2023-08-27T11:28:06.682022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:20:12.051340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670514+0000","last_change":"2023-08-27T11:28:06.670514+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670514+0000","last_undegraded":"2023-08-27T11:28:06.670514+0000","last_fullsized":"2023-08-27T11:28:06.670514+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680425+0000","last_change":"2023-08-27T11:28:06.679947+0000","last_active":"2023-08-27T11:28:06.680425+0000","last_peered":"2023-08-27T11:28:06.680425+0000","last_clean":"2023-08-27T11:28:06.680425+0000","last_became_active":"2023-08-27T11:28:06.679654+0000","last_became_peered":"2023-08-27T11:28:06.679654+0000","last_unstale":"2023-08-27T11:28:06.680425+0000","last_undegraded":"2023-08-27T11:28:06.680425+0000","last_fullsized":"2023-08-27T11:28:06.680425+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:16:36.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671123+0000","last_change":"2023-08-27T11:28:06.671123+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671123+0000","last_undegraded":"2023-08-27T11:28:06.671123+0000","last_fullsized":"2023-08-27T11:28:06.671123+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672339+0000","last_change":"2023-08-27T11:28:06.668640+0000","last_active":"2023-08-27T11:28:06.672339+0000","last_peered":"2023-08-27T11:28:06.672339+0000","last_clean":"2023-08-27T11:28:06.672339+0000","last_became_active":"2023-08-27T11:28:06.668372+0000","last_became_peered":"2023-08-27T11:28:06.668372+0000","last_unstale":"2023-08-27T11:28:06.672339+0000","last_undegraded":"2023-08-27T11:28:06.672339+0000","last_fullsized":"2023-08-27T11:28:06.672339+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:48:01.533647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673604+0000","last_change":"2023-08-27T11:28:06.673604+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673604+0000","last_undegraded":"2023-08-27T11:28:06.673604+0000","last_fullsized":"2023-08-27T11:28:06.673604+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682683+0000","last_change":"2023-08-27T11:28:06.682044+0000","last_active":"2023-08-27T11:28:06.682683+0000","last_peered":"2023-08-27T11:28:06.682683+0000","last_clean":"2023-08-27T11:28:06.682683+0000","last_became_active":"2023-08-27T11:28:06.681789+0000","last_became_peered":"2023-08-27T11:28:06.681789+0000","last_unstale":"2023-08-27T11:28:06.682683+0000","last_undegraded":"2023-08-27T11:28:06.682683+0000","last_fullsized":"2023-08-27T11:28:06.682683+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:39:11.072746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.668771+0000","last_change":"2023-08-27T11:28:06.668771+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.668771+0000","last_undegraded":"2023-08-27T11:28:06.668771+0000","last_fullsized":"2023-08-27T11:28:06.668771+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682726+0000","last_change":"2023-08-27T11:28:06.682213+0000","last_active":"2023-08-27T11:28:06.682726+0000","last_peered":"2023-08-27T11:28:06.682726+0000","last_clean":"2023-08-27T11:28:06.682726+0000","last_became_active":"2023-08-27T11:28:06.681772+0000","last_became_peered":"2023-08-27T11:28:06.681772+0000","last_unstale":"2023-08-27T11:28:06.682726+0000","last_undegraded":"2023-08-27T11:28:06.682726+0000","last_fullsized":"2023-08-27T11:28:06.682726+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:34:15.373195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670120+0000","last_change":"2023-08-27T11:28:06.670120+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670120+0000","last_undegraded":"2023-08-27T11:28:06.670120+0000","last_fullsized":"2023-08-27T11:28:06.670120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682020+0000","last_change":"2023-08-27T11:28:06.681163+0000","last_active":"2023-08-27T11:28:06.682020+0000","last_peered":"2023-08-27T11:28:06.682020+0000","last_clean":"2023-08-27T11:28:06.682020+0000","last_became_active":"2023-08-27T11:28:06.680922+0000","last_became_peered":"2023-08-27T11:28:06.680922+0000","last_unstale":"2023-08-27T11:28:06.682020+0000","last_undegraded":"2023-08-27T11:28:06.682020+0000","last_fullsized":"2023-08-27T11:28:06.682020+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:13:20.626473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664201+0000","last_change":"2023-08-27T11:28:06.664201+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664201+0000","last_undegraded":"2023-08-27T11:28:06.664201+0000","last_fullsized":"2023-08-27T11:28:06.664201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680463+0000","last_change":"2023-08-27T11:28:06.677834+0000","last_active":"2023-08-27T11:28:06.680463+0000","last_peered":"2023-08-27T11:28:06.680463+0000","last_clean":"2023-08-27T11:28:06.680463+0000","last_became_active":"2023-08-27T11:28:06.677483+0000","last_became_peered":"2023-08-27T11:28:06.677483+0000","last_unstale":"2023-08-27T11:28:06.680463+0000","last_undegraded":"2023-08-27T11:28:06.680463+0000","last_fullsized":"2023-08-27T11:28:06.680463+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:33:20.246978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670946+0000","last_change":"2023-08-27T11:28:06.670946+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670946+0000","last_undegraded":"2023-08-27T11:28:06.670946+0000","last_fullsized":"2023-08-27T11:28:06.670946+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682708+0000","last_change":"2023-08-27T11:28:06.682127+0000","last_active":"2023-08-27T11:28:06.682708+0000","last_peered":"2023-08-27T11:28:06.682708+0000","last_clean":"2023-08-27T11:28:06.682708+0000","last_became_active":"2023-08-27T11:28:06.681824+0000","last_became_peered":"2023-08-27T11:28:06.681824+0000","last_unstale":"2023-08-27T11:28:06.682708+0000","last_undegraded":"2023-08-27T11:28:06.682708+0000","last_fullsized":"2023-08-27T11:28:06.682708+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:04:03.058354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669155+0000","last_change":"2023-08-27T11:28:06.669155+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669155+0000","last_undegraded":"2023-08-27T11:28:06.669155+0000","last_fullsized":"2023-08-27T11:28:06.669155+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679086+0000","last_change":"2023-08-27T11:28:06.676211+0000","last_active":"2023-08-27T11:28:06.679086+0000","last_peered":"2023-08-27T11:28:06.679086+0000","last_clean":"2023-08-27T11:28:06.679086+0000","last_became_active":"2023-08-27T11:28:06.675956+0000","last_became_peered":"2023-08-27T11:28:06.675956+0000","last_unstale":"2023-08-27T11:28:06.679086+0000","last_undegraded":"2023-08-27T11:28:06.679086+0000","last_fullsized":"2023-08-27T11:28:06.679086+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:48:39.217804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669642+0000","last_change":"2023-08-27T11:28:06.669642+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669642+0000","last_undegraded":"2023-08-27T11:28:06.669642+0000","last_fullsized":"2023-08-27T11:28:06.669642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676840+0000","last_change":"2023-08-27T11:28:06.673096+0000","last_active":"2023-08-27T11:28:06.676840+0000","last_peered":"2023-08-27T11:28:06.676840+0000","last_clean":"2023-08-27T11:28:06.676840+0000","last_became_active":"2023-08-27T11:28:06.672819+0000","last_became_peered":"2023-08-27T11:28:06.672819+0000","last_unstale":"2023-08-27T11:28:06.676840+0000","last_undegraded":"2023-08-27T11:28:06.676840+0000","last_fullsized":"2023-08-27T11:28:06.676840+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:47:34.207300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.666042+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:06.666042+0000","last_peered":"2023-08-27T11:28:06.666042+0000","last_clean":"2023-08-27T11:28:06.666042+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:06.666042+0000","last_undegraded":"2023-08-27T11:28:06.666042+0000","last_fullsized":"2023-08-27T11:28:06.666042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:51:28.335693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.669129+0000","last_change":"2023-08-27T11:28:06.669129+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.669129+0000","last_undegraded":"2023-08-27T11:28:06.669129+0000","last_fullsized":"2023-08-27T11:28:06.669129+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.676890+0000","last_change":"2023-08-27T11:28:06.676306+0000","last_active":"2023-08-27T11:28:06.676890+0000","last_peered":"2023-08-27T11:28:06.676890+0000","last_clean":"2023-08-27T11:28:06.676890+0000","last_became_active":"2023-08-27T11:28:06.676049+0000","last_became_peered":"2023-08-27T11:28:06.676049+0000","last_unstale":"2023-08-27T11:28:06.676890+0000","last_undegraded":"2023-08-27T11:28:06.676890+0000","last_fullsized":"2023-08-27T11:28:06.676890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:31:52.590584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672445+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:06.672445+0000","last_peered":"2023-08-27T11:28:06.672445+0000","last_clean":"2023-08-27T11:28:06.672445+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:06.672445+0000","last_undegraded":"2023-08-27T11:28:06.672445+0000","last_fullsized":"2023-08-27T11:28:06.672445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:38:04.382696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.670944+0000","last_change":"2023-08-27T11:28:06.670944+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.670944+0000","last_undegraded":"2023-08-27T11:28:06.670944+0000","last_fullsized":"2023-08-27T11:28:06.670944+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.682490+0000","last_change":"2023-08-27T11:28:06.680009+0000","last_active":"2023-08-27T11:28:06.682490+0000","last_peered":"2023-08-27T11:28:06.682490+0000","last_clean":"2023-08-27T11:28:06.682490+0000","last_became_active":"2023-08-27T11:28:06.679754+0000","last_became_peered":"2023-08-27T11:28:06.679754+0000","last_unstale":"2023-08-27T11:28:06.682490+0000","last_undegraded":"2023-08-27T11:28:06.682490+0000","last_fullsized":"2023-08-27T11:28:06.682490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:06.046171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":154,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.670537+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:06.670537+0000","last_peered":"2023-08-27T11:28:06.670537+0000","last_clean":"2023-08-27T11:28:06.670537+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:06.670537+0000","last_undegraded":"2023-08-27T11:28:06.670537+0000","last_fullsized":"2023-08-27T11:28:06.670537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:46:09.042939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664810+0000","last_change":"2023-08-27T11:28:06.664810+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664810+0000","last_undegraded":"2023-08-27T11:28:06.664810+0000","last_fullsized":"2023-08-27T11:28:06.664810+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.681624+0000","last_change":"2023-08-27T11:28:06.680561+0000","last_active":"2023-08-27T11:28:06.681624+0000","last_peered":"2023-08-27T11:28:06.681624+0000","last_clean":"2023-08-27T11:28:06.681624+0000","last_became_active":"2023-08-27T11:28:06.680307+0000","last_became_peered":"2023-08-27T11:28:06.680307+0000","last_unstale":"2023-08-27T11:28:06.681624+0000","last_undegraded":"2023-08-27T11:28:06.681624+0000","last_fullsized":"2023-08-27T11:28:06.681624+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:58.720509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.672903+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:06.672903+0000","last_peered":"2023-08-27T11:28:06.672903+0000","last_clean":"2023-08-27T11:28:06.672903+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:06.672903+0000","last_undegraded":"2023-08-27T11:28:06.672903+0000","last_fullsized":"2023-08-27T11:28:06.672903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:40:14.646059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"98'2213","reported_seq":2572,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:07.501955+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:07.501955+0000","last_peered":"2023-08-27T11:28:07.501955+0000","last_clean":"2023-08-27T11:28:07.501955+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:07.501955+0000","last_undegraded":"2023-08-27T11:28:07.501955+0000","last_fullsized":"2023-08-27T11:28:07.501955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":2213,"log_dups_size":0,"ondisk_log_size":2213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:54:12.932795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.673646+0000","last_change":"2023-08-27T11:28:06.673646+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.673646+0000","last_undegraded":"2023-08-27T11:28:06.673646+0000","last_fullsized":"2023-08-27T11:28:06.673646+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.680214+0000","last_change":"2023-08-27T11:28:06.678931+0000","last_active":"2023-08-27T11:28:06.680214+0000","last_peered":"2023-08-27T11:28:06.680214+0000","last_clean":"2023-08-27T11:28:06.680214+0000","last_became_active":"2023-08-27T11:28:06.678661+0000","last_became_peered":"2023-08-27T11:28:06.678661+0000","last_unstale":"2023-08-27T11:28:06.680214+0000","last_undegraded":"2023-08-27T11:28:06.680214+0000","last_fullsized":"2023-08-27T11:28:06.680214+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:00:05.382377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.675655+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:06.675655+0000","last_peered":"2023-08-27T11:28:06.675655+0000","last_clean":"2023-08-27T11:28:06.675655+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:06.675655+0000","last_undegraded":"2023-08-27T11:28:06.675655+0000","last_fullsized":"2023-08-27T11:28:06.675655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:10:39.559849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.667293+0000","last_change":"2023-08-27T11:28:06.667293+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.667293+0000","last_undegraded":"2023-08-27T11:28:06.667293+0000","last_fullsized":"2023-08-27T11:28:06.667293+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.673836+0000","last_change":"2023-08-27T11:28:06.672441+0000","last_active":"2023-08-27T11:28:06.673836+0000","last_peered":"2023-08-27T11:28:06.673836+0000","last_clean":"2023-08-27T11:28:06.673836+0000","last_became_active":"2023-08-27T11:28:06.672183+0000","last_became_peered":"2023-08-27T11:28:06.672183+0000","last_unstale":"2023-08-27T11:28:06.673836+0000","last_undegraded":"2023-08-27T11:28:06.673836+0000","last_fullsized":"2023-08-27T11:28:06.673836+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:23.790945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.677403+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:06.677403+0000","last_peered":"2023-08-27T11:28:06.677403+0000","last_clean":"2023-08-27T11:28:06.677403+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:06.677403+0000","last_undegraded":"2023-08-27T11:28:06.677403+0000","last_fullsized":"2023-08-27T11:28:06.677403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:13:06.409958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.671744+0000","last_change":"2023-08-27T11:28:06.671744+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.671744+0000","last_undegraded":"2023-08-27T11:28:06.671744+0000","last_fullsized":"2023-08-27T11:28:06.671744+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.670942+0000","last_change":"2023-08-27T11:28:06.666649+0000","last_active":"2023-08-27T11:28:06.670942+0000","last_peered":"2023-08-27T11:28:06.670942+0000","last_clean":"2023-08-27T11:28:06.670942+0000","last_became_active":"2023-08-27T11:28:06.666348+0000","last_became_peered":"2023-08-27T11:28:06.666348+0000","last_unstale":"2023-08-27T11:28:06.670942+0000","last_undegraded":"2023-08-27T11:28:06.670942+0000","last_fullsized":"2023-08-27T11:28:06.670942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:31:23.091962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.666080+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:06.666080+0000","last_peered":"2023-08-27T11:28:06.666080+0000","last_clean":"2023-08-27T11:28:06.666080+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:06.666080+0000","last_undegraded":"2023-08-27T11:28:06.666080+0000","last_fullsized":"2023-08-27T11:28:06.666080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:05.589881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2023-08-27T11:28:06.664961+0000","last_change":"2023-08-27T11:28:06.664961+0000","last_active":"2023-08-27T11:28:06.651961+0000","last_peered":"2023-08-27T11:28:06.651961+0000","last_clean":"2023-08-27T11:28:06.651961+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:06.664961+0000","last_undegraded":"2023-08-27T11:28:06.664961+0000","last_fullsized":"2023-08-27T11:28:06.664961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.684250+0000","last_change":"2023-08-27T11:28:06.683725+0000","last_active":"2023-08-27T11:28:06.684250+0000","last_peered":"2023-08-27T11:28:06.684250+0000","last_clean":"2023-08-27T11:28:06.684250+0000","last_became_active":"2023-08-27T11:28:06.683424+0000","last_became_peered":"2023-08-27T11:28:06.683424+0000","last_unstale":"2023-08-27T11:28:06.684250+0000","last_undegraded":"2023-08-27T11:28:06.684250+0000","last_fullsized":"2023-08-27T11:28:06.684250+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:13.388800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-08-27T11:28:06.679319+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:06.679319+0000","last_peered":"2023-08-27T11:28:06.679319+0000","last_clean":"2023-08-27T11:28:06.679319+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:06.679319+0000","last_undegraded":"2023-08-27T11:28:06.679319+0000","last_fullsized":"2023-08-27T11:28:06.679319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:19:35.146879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":588,"num_read_kb":1127,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2213,"ondisk_log_size":2213,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575041,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41252,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330588,"statfs":{"total":96636764160,"available":96594522112,"internally_reserved":0,"allocated":1794048,"data_stored":1217459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.38500000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":1.601,"15min":1.601},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.59999999999999998,"15min":0.58299999999999996},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.499}]},{"osd":1,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.71199999999999997,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.77000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.432}]},{"osd":2,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.80100000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.29099999999999998},"max":{"1min":0.749,"5min":1.778,"15min":1.778},"last":0.52100000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.65300000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.221},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68799999999999994,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.30499999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65300000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.25800000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.34499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575041,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.40799999999999997,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.76700000000000002,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.628},"min":{"1min":0.47399999999999998,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.3540000000000001},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":3.3780000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.80700000000000005,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.41699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.61699999999999999},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.73399999999999999,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":1.325},"last":0.44}]},{"osd":3,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.60499999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.71699999999999997,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.46400000000000002},"min":{"1min":0.312,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"min":{"1min":0.35199999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.624,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35099999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":1.157,"15min":1.157},"last":0.45200000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.70399999999999996,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.36899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40680,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331160,"statfs":{"total":96636764160,"available":96595107840,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20399999999999999},"max":{"1min":0.52100000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.495},{"interface":"front","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.628,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61199999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42699999999999999,"15min":0.42399999999999999},"min":{"1min":0.27500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.34100000000000003,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58999999999999997,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.63300000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":1.169,"15min":1.169},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44600000000000001,"15min":0.44},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.23400000000000001},"max":{"1min":0.55900000000000005,"5min":1.139,"15min":1.139},"last":0.47299999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.82599999999999996,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81499999999999995,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88700000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.59899999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89400000000000002,"15min":1.139},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575041,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41248,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330592,"statfs":{"total":96636764160,"available":96594526208,"internally_reserved":0,"allocated":1794048,"data_stored":1217459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.31},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.29399999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.26700000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.374}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.41499999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.48999999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.39400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":620099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.25}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.31}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.433}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.48099999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.504},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.54700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40699999999999997,"15min":0.42499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.22},"max":{"1min":0.60799999999999998,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.224},"max":{"1min":0.60699999999999998,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.30499999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.64100000000000001,"15min":6.0830000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46200000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.245},"max":{"1min":0.57399999999999995,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.68000000000000005,"5min":0.71199999999999997,"15min":1.25},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":0.84499999999999997,"15min":11.939},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.376},"max":{"1min":0.83099999999999996,"5min":1.0329999999999999,"15min":11.007},"last":0.374},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":1.113,"15min":1.2929999999999999},"last":0.45100000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93300000000000005,"5min":3.0219999999999998,"15min":11.981},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":1.218,"15min":11.09},"last":0.498}]},{"osd":6,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69099999999999995,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.75600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":1.0269999999999999,"15min":12.016},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68999999999999995,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":0.96699999999999997,"15min":11.054},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.53500000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84099999999999997,"5min":1.0640000000000001,"15min":1.361},"last":0.47699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575041,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1200128,"data_stored":627091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.32600000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.39400000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.504}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.47199999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575040,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335340,"statfs":{"total":96636764160,"available":96599388160,"internally_reserved":0,"allocated":1187840,"data_stored":620099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.38},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.26000000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.39300000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.34100000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.43099999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.42199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:13.784 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:13.783+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:14.526 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:14.524+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:15.274 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:15.272+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:16.019 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:16.017+0000 7fe86f9da700 -1 client.6206: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6089:6230a0bd-c736-410a-b3e6-447eef9893e3:v1:172.21.15.135:0/3438903793] 2023-08-27T11:28:16.530 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:16.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66d019700 1 Processor -- start 2023-08-27T11:28:16.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66d019700 1 -- start start 2023-08-27T11:28:16.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66d019700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd66815d3d0 0x7fd6681619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66d019700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6680af040 0x7fd6680cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66d019700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd668161550 con 0x7fd66806e3a0 2023-08-27T11:28:16.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66d019700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6680cdf90 con 0x7fd6680af040 2023-08-27T11:28:16.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66d019700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd668161f20 con 0x7fd66815d3d0 2023-08-27T11:28:16.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6680af040 0x7fd6680cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6680af040 0x7fd6680cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59166/0 (socket says 172.21.15.45:59166) 2023-08-27T11:28:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66659c700 1 -- 172.21.15.45:0/3492721123 learned_addr learned my addr 172.21.15.45:0/3492721123 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66659c700 1 -- 172.21.15.45:0/3492721123 >> v2:172.21.15.135:3300/0 conn(0x7fd66815d3d0 msgr2=0x7fd6681619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd666d9d700 1 --2- 172.21.15.45:0/3492721123 >> v2:172.21.15.135:3300/0 conn(0x7fd66815d3d0 0x7fd6681619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66659c700 1 --2- 172.21.15.45:0/3492721123 >> v2:172.21.15.135:3300/0 conn(0x7fd66815d3d0 0x7fd6681619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66659c700 1 -- 172.21.15.45:0/3492721123 >> v1:172.21.15.135:6789/0 conn(0x7fd66806e3a0 legacy=0x7fd668070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:16.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66659c700 1 -- 172.21.15.45:0/3492721123 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680cd9d0 con 0x7fd6680af040 2023-08-27T11:28:16.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd66659c700 1 --2- 172.21.15.45:0/3492721123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 0x7fd6680cf420 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd668067880 tx=0x7fd658009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27b885f6e816070 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:16.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.656+0000 7fd665d9b700 1 -- 172.21.15.45:0/3492721123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658004230 con 0x7fd6680af040 2023-08-27T11:28:16.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd665d9b700 1 -- 172.21.15.45:0/3492721123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658004390 con 0x7fd6680af040 2023-08-27T11:28:16.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd665d9b700 1 -- 172.21.15.45:0/3492721123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6580036a0 con 0x7fd6680af040 2023-08-27T11:28:16.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd66d019700 1 -- 172.21.15.45:0/3492721123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 msgr2=0x7fd6680cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd66d019700 1 --2- 172.21.15.45:0/3492721123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 0x7fd6680cf420 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd668067880 tx=0x7fd658009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd66d019700 1 -- 172.21.15.45:0/3492721123 shutdown_connections 2023-08-27T11:28:16.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd66d019700 1 --2- 172.21.15.45:0/3492721123 >> v2:172.21.15.135:3300/0 conn(0x7fd66815d3d0 0x7fd6681619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd66d019700 1 --2- 172.21.15.45:0/3492721123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 0x7fd6680cf420 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd66d019700 1 -- 172.21.15.45:0/3492721123 >> 172.21.15.45:0/3492721123 conn(0x7fd6680cfd60 msgr2=0x7fd6680d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:16.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd66d019700 1 -- 172.21.15.45:0/3492721123 shutdown_connections 2023-08-27T11:28:16.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.657+0000 7fd66d019700 1 -- 172.21.15.45:0/3492721123 wait complete. 2023-08-27T11:28:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.659+0000 7fd66d019700 1 Processor -- start 2023-08-27T11:28:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.659+0000 7fd66d019700 1 -- start start 2023-08-27T11:28:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.659+0000 7fd66d019700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd66806e3a0 0x7fd66815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.659+0000 7fd66d019700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 0x7fd66815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.659+0000 7fd66d019700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd66815d3d0 0x7fd66815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.659+0000 7fd66d019700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66815eac0 con 0x7fd6680af040 2023-08-27T11:28:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.659+0000 7fd66d019700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66815be60 con 0x7fd66815d3d0 2023-08-27T11:28:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.659+0000 7fd66d019700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66815deb0 con 0x7fd66806e3a0 2023-08-27T11:28:16.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.659+0000 7fd66659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 0x7fd66815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 0x7fd66815d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59172/0 (socket says 172.21.15.45:59172) 2023-08-27T11:28:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66659c700 1 -- 172.21.15.45:0/2464738815 learned_addr learned my addr 172.21.15.45:0/2464738815 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66659c700 1 -- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd66806e3a0 msgr2=0x7fd66815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66659c700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd66806e3a0 0x7fd66815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66659c700 1 -- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd66815d3d0 msgr2=0x7fd66815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66659c700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd66815d3d0 0x7fd66815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66659c700 1 -- 172.21.15.45:0/2464738815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd658009550 con 0x7fd6680af040 2023-08-27T11:28:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66659c700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 0x7fd66815d7b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd668072b20 tx=0x7fd658004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd64f7fe700 1 -- 172.21.15.45:0/2464738815 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658011630 con 0x7fd6680af040 2023-08-27T11:28:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd64f7fe700 1 -- 172.21.15.45:0/2464738815 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658011790 con 0x7fd6680af040 2023-08-27T11:28:16.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66815e190 con 0x7fd6680af040 2023-08-27T11:28:16.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd64f7fe700 1 -- 172.21.15.45:0/2464738815 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658011b40 con 0x7fd6680af040 2023-08-27T11:28:16.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.660+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66815e750 con 0x7fd6680af040 2023-08-27T11:28:16.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.661+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd668072780 con 0x7fd6680af040 2023-08-27T11:28:16.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.662+0000 7fd64f7fe700 1 -- 172.21.15.45:0/2464738815 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fd658011ca0 con 0x7fd6680af040 2023-08-27T11:28:16.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.662+0000 7fd64f7fe700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd65406a170 0x7fd65406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:16.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.662+0000 7fd64f7fe700 1 -- 172.21.15.45:0/2464738815 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fd658089d60 con 0x7fd6680af040 2023-08-27T11:28:16.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.662+0000 7fd666d9d700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd65406a170 0x7fd65406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:16.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.663+0000 7fd666d9d700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd65406a170 0x7fd65406c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd6680add90 tx=0x7fd65c008040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:16.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.666+0000 7fd64f7fe700 1 -- 172.21.15.45:0/2464738815 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd658055380 con 0x7fd6680af040 2023-08-27T11:28:16.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.860+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 --> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6681611e0 con 0x7fd65406a170 2023-08-27T11:28:16.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.869+0000 7fd64f7fe700 1 -- 172.21.15.45:0/2464738815 <== mgr.6709 v2:172.21.15.135:6832/3812189695 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311815 (secure 0 0 0) 0x7fd65c00a570 con 0x7fd65406a170 2023-08-27T11:28:16.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:16.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd65406a170 msgr2=0x7fd65406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd65406a170 0x7fd65406c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd6680add90 tx=0x7fd65c008040 comp rx=0 tx=0).stop 2023-08-27T11:28:16.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 msgr2=0x7fd66815d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:16.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 0x7fd66815d7b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd668072b20 tx=0x7fd658004750 comp rx=0 tx=0).stop 2023-08-27T11:28:16.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 shutdown_connections 2023-08-27T11:28:16.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd65406a170 0x7fd65406c630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd66815d3d0 0x7fd66815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd66806e3a0 0x7fd66815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 --2- 172.21.15.45:0/2464738815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6680af040 0x7fd66815d7b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:16.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 >> 172.21.15.45:0/2464738815 conn(0x7fd6680cfd60 msgr2=0x7fd6680afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:16.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 shutdown_connections 2023-08-27T11:28:16.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:16.873+0000 7fd66d019700 1 -- 172.21.15.45:0/2464738815 wait complete. 2023-08-27T11:28:16.882 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:16.887 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-27T11:28:15.506879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":595,"num_read_kb":1134,"num_write":298,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2936,"ondisk_log_size":2936,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260236,"kb_used_data":10792,"kb_used_omap":13,"kb_used_meta":249330,"kb_avail":754714484,"statfs":{"total":773094113280,"available":772827631616,"internally_reserved":0,"allocated":11051008,"data_stored":6318356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":255314008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.006433"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867763+0000","last_change":"2023-08-27T11:28:07.678151+0000","last_active":"2023-08-27T11:28:09.867763+0000","last_peered":"2023-08-27T11:28:09.867763+0000","last_clean":"2023-08-27T11:28:09.867763+0000","last_became_active":"2023-08-27T11:28:07.677783+0000","last_became_peered":"2023-08-27T11:28:07.677783+0000","last_unstale":"2023-08-27T11:28:09.867763+0000","last_undegraded":"2023-08-27T11:28:09.867763+0000","last_fullsized":"2023-08-27T11:28:09.867763+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:22.774900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868530+0000","last_change":"2023-08-27T11:28:07.679177+0000","last_active":"2023-08-27T11:28:09.868530+0000","last_peered":"2023-08-27T11:28:09.868530+0000","last_clean":"2023-08-27T11:28:09.868530+0000","last_became_active":"2023-08-27T11:28:07.678902+0000","last_became_peered":"2023-08-27T11:28:07.678902+0000","last_unstale":"2023-08-27T11:28:09.868530+0000","last_undegraded":"2023-08-27T11:28:09.868530+0000","last_fullsized":"2023-08-27T11:28:09.868530+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:01:23.963262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925353+0000","last_change":"2023-08-27T11:28:07.681521+0000","last_active":"2023-08-27T11:28:08.925353+0000","last_peered":"2023-08-27T11:28:08.925353+0000","last_clean":"2023-08-27T11:28:08.925353+0000","last_became_active":"2023-08-27T11:28:07.681268+0000","last_became_peered":"2023-08-27T11:28:07.681268+0000","last_unstale":"2023-08-27T11:28:08.925353+0000","last_undegraded":"2023-08-27T11:28:08.925353+0000","last_fullsized":"2023-08-27T11:28:08.925353+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:35:22.507889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662684+0000","last_change":"2023-08-27T11:28:07.687737+0000","last_active":"2023-08-27T11:28:08.662684+0000","last_peered":"2023-08-27T11:28:08.662684+0000","last_clean":"2023-08-27T11:28:08.662684+0000","last_became_active":"2023-08-27T11:28:07.686721+0000","last_became_peered":"2023-08-27T11:28:07.686721+0000","last_unstale":"2023-08-27T11:28:08.662684+0000","last_undegraded":"2023-08-27T11:28:08.662684+0000","last_fullsized":"2023-08-27T11:28:08.662684+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:57:24.694606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663228+0000","last_change":"2023-08-27T11:28:07.672660+0000","last_active":"2023-08-27T11:28:08.663228+0000","last_peered":"2023-08-27T11:28:08.663228+0000","last_clean":"2023-08-27T11:28:08.663228+0000","last_became_active":"2023-08-27T11:28:07.671880+0000","last_became_peered":"2023-08-27T11:28:07.671880+0000","last_unstale":"2023-08-27T11:28:08.663228+0000","last_undegraded":"2023-08-27T11:28:08.663228+0000","last_fullsized":"2023-08-27T11:28:08.663228+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:13.087505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664348+0000","last_change":"2023-08-27T11:28:07.674883+0000","last_active":"2023-08-27T11:28:08.664348+0000","last_peered":"2023-08-27T11:28:08.664348+0000","last_clean":"2023-08-27T11:28:08.664348+0000","last_became_active":"2023-08-27T11:28:07.674680+0000","last_became_peered":"2023-08-27T11:28:07.674680+0000","last_unstale":"2023-08-27T11:28:08.664348+0000","last_undegraded":"2023-08-27T11:28:08.664348+0000","last_fullsized":"2023-08-27T11:28:08.664348+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:09:05.067725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662354+0000","last_change":"2023-08-27T11:28:07.676443+0000","last_active":"2023-08-27T11:28:08.662354+0000","last_peered":"2023-08-27T11:28:08.662354+0000","last_clean":"2023-08-27T11:28:08.662354+0000","last_became_active":"2023-08-27T11:28:07.676171+0000","last_became_peered":"2023-08-27T11:28:07.676171+0000","last_unstale":"2023-08-27T11:28:08.662354+0000","last_undegraded":"2023-08-27T11:28:08.662354+0000","last_fullsized":"2023-08-27T11:28:08.662354+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:37.282816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664245+0000","last_change":"2023-08-27T11:28:07.671078+0000","last_active":"2023-08-27T11:28:08.664245+0000","last_peered":"2023-08-27T11:28:08.664245+0000","last_clean":"2023-08-27T11:28:08.664245+0000","last_became_active":"2023-08-27T11:28:07.670656+0000","last_became_peered":"2023-08-27T11:28:07.670656+0000","last_unstale":"2023-08-27T11:28:08.664245+0000","last_undegraded":"2023-08-27T11:28:08.664245+0000","last_fullsized":"2023-08-27T11:28:08.664245+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:01:40.622942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664475+0000","last_change":"2023-08-27T11:28:07.679269+0000","last_active":"2023-08-27T11:28:08.664475+0000","last_peered":"2023-08-27T11:28:08.664475+0000","last_clean":"2023-08-27T11:28:08.664475+0000","last_became_active":"2023-08-27T11:28:07.679013+0000","last_became_peered":"2023-08-27T11:28:07.679013+0000","last_unstale":"2023-08-27T11:28:08.664475+0000","last_undegraded":"2023-08-27T11:28:08.664475+0000","last_fullsized":"2023-08-27T11:28:08.664475+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:25:34.855703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867665+0000","last_change":"2023-08-27T11:28:07.681064+0000","last_active":"2023-08-27T11:28:09.867665+0000","last_peered":"2023-08-27T11:28:09.867665+0000","last_clean":"2023-08-27T11:28:09.867665+0000","last_became_active":"2023-08-27T11:28:07.680728+0000","last_became_peered":"2023-08-27T11:28:07.680728+0000","last_unstale":"2023-08-27T11:28:09.867665+0000","last_undegraded":"2023-08-27T11:28:09.867665+0000","last_fullsized":"2023-08-27T11:28:09.867665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:52:48.101391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868512+0000","last_change":"2023-08-27T11:28:07.674305+0000","last_active":"2023-08-27T11:28:09.868512+0000","last_peered":"2023-08-27T11:28:09.868512+0000","last_clean":"2023-08-27T11:28:09.868512+0000","last_became_active":"2023-08-27T11:28:07.673969+0000","last_became_peered":"2023-08-27T11:28:07.673969+0000","last_unstale":"2023-08-27T11:28:09.868512+0000","last_undegraded":"2023-08-27T11:28:09.868512+0000","last_fullsized":"2023-08-27T11:28:09.868512+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:11:12.832477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867751+0000","last_change":"2023-08-27T11:28:07.676324+0000","last_active":"2023-08-27T11:28:09.867751+0000","last_peered":"2023-08-27T11:28:09.867751+0000","last_clean":"2023-08-27T11:28:09.867751+0000","last_became_active":"2023-08-27T11:28:07.676062+0000","last_became_peered":"2023-08-27T11:28:07.676062+0000","last_unstale":"2023-08-27T11:28:09.867751+0000","last_undegraded":"2023-08-27T11:28:09.867751+0000","last_fullsized":"2023-08-27T11:28:09.867751+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:09:46.135450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925248+0000","last_change":"2023-08-27T11:28:07.679247+0000","last_active":"2023-08-27T11:28:08.925248+0000","last_peered":"2023-08-27T11:28:08.925248+0000","last_clean":"2023-08-27T11:28:08.925248+0000","last_became_active":"2023-08-27T11:28:07.678991+0000","last_became_peered":"2023-08-27T11:28:07.678991+0000","last_unstale":"2023-08-27T11:28:08.925248+0000","last_undegraded":"2023-08-27T11:28:08.925248+0000","last_fullsized":"2023-08-27T11:28:08.925248+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:00.213451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662666+0000","last_change":"2023-08-27T11:28:07.680345+0000","last_active":"2023-08-27T11:28:08.662666+0000","last_peered":"2023-08-27T11:28:08.662666+0000","last_clean":"2023-08-27T11:28:08.662666+0000","last_became_active":"2023-08-27T11:28:07.680079+0000","last_became_peered":"2023-08-27T11:28:07.680079+0000","last_unstale":"2023-08-27T11:28:08.662666+0000","last_undegraded":"2023-08-27T11:28:08.662666+0000","last_fullsized":"2023-08-27T11:28:08.662666+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:10:15.392763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867480+0000","last_change":"2023-08-27T11:28:07.675637+0000","last_active":"2023-08-27T11:28:09.867480+0000","last_peered":"2023-08-27T11:28:09.867480+0000","last_clean":"2023-08-27T11:28:09.867480+0000","last_became_active":"2023-08-27T11:28:07.675200+0000","last_became_peered":"2023-08-27T11:28:07.675200+0000","last_unstale":"2023-08-27T11:28:09.867480+0000","last_undegraded":"2023-08-27T11:28:09.867480+0000","last_fullsized":"2023-08-27T11:28:09.867480+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:03:09.169275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662774+0000","last_change":"2023-08-27T11:28:07.677981+0000","last_active":"2023-08-27T11:28:08.662774+0000","last_peered":"2023-08-27T11:28:08.662774+0000","last_clean":"2023-08-27T11:28:08.662774+0000","last_became_active":"2023-08-27T11:28:07.677706+0000","last_became_peered":"2023-08-27T11:28:07.677706+0000","last_unstale":"2023-08-27T11:28:08.662774+0000","last_undegraded":"2023-08-27T11:28:08.662774+0000","last_fullsized":"2023-08-27T11:28:08.662774+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:42.860390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663117+0000","last_change":"2023-08-27T11:28:07.668904+0000","last_active":"2023-08-27T11:28:08.663117+0000","last_peered":"2023-08-27T11:28:08.663117+0000","last_clean":"2023-08-27T11:28:08.663117+0000","last_became_active":"2023-08-27T11:28:07.668635+0000","last_became_peered":"2023-08-27T11:28:07.668635+0000","last_unstale":"2023-08-27T11:28:08.663117+0000","last_undegraded":"2023-08-27T11:28:08.663117+0000","last_fullsized":"2023-08-27T11:28:08.663117+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:03:41.932139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662487+0000","last_change":"2023-08-27T11:28:07.689137+0000","last_active":"2023-08-27T11:28:08.662487+0000","last_peered":"2023-08-27T11:28:08.662487+0000","last_clean":"2023-08-27T11:28:08.662487+0000","last_became_active":"2023-08-27T11:28:07.684748+0000","last_became_peered":"2023-08-27T11:28:07.684748+0000","last_unstale":"2023-08-27T11:28:08.662487+0000","last_undegraded":"2023-08-27T11:28:08.662487+0000","last_fullsized":"2023-08-27T11:28:08.662487+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:29:28.522643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931850+0000","last_change":"2023-08-27T11:28:07.677364+0000","last_active":"2023-08-27T11:28:08.931850+0000","last_peered":"2023-08-27T11:28:08.931850+0000","last_clean":"2023-08-27T11:28:08.931850+0000","last_became_active":"2023-08-27T11:28:07.676907+0000","last_became_peered":"2023-08-27T11:28:07.676907+0000","last_unstale":"2023-08-27T11:28:08.931850+0000","last_undegraded":"2023-08-27T11:28:08.931850+0000","last_fullsized":"2023-08-27T11:28:08.931850+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:01.271971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662850+0000","last_change":"2023-08-27T11:28:07.687340+0000","last_active":"2023-08-27T11:28:08.662850+0000","last_peered":"2023-08-27T11:28:08.662850+0000","last_clean":"2023-08-27T11:28:08.662850+0000","last_became_active":"2023-08-27T11:28:07.687030+0000","last_became_peered":"2023-08-27T11:28:07.687030+0000","last_unstale":"2023-08-27T11:28:08.662850+0000","last_undegraded":"2023-08-27T11:28:08.662850+0000","last_fullsized":"2023-08-27T11:28:08.662850+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:52:16.153436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662482+0000","last_change":"2023-08-27T11:28:07.673889+0000","last_active":"2023-08-27T11:28:08.662482+0000","last_peered":"2023-08-27T11:28:08.662482+0000","last_clean":"2023-08-27T11:28:08.662482+0000","last_became_active":"2023-08-27T11:28:07.673664+0000","last_became_peered":"2023-08-27T11:28:07.673664+0000","last_unstale":"2023-08-27T11:28:08.662482+0000","last_undegraded":"2023-08-27T11:28:08.662482+0000","last_fullsized":"2023-08-27T11:28:08.662482+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:49:55.491569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868277+0000","last_change":"2023-08-27T11:28:07.678583+0000","last_active":"2023-08-27T11:28:09.868277+0000","last_peered":"2023-08-27T11:28:09.868277+0000","last_clean":"2023-08-27T11:28:09.868277+0000","last_became_active":"2023-08-27T11:28:07.678059+0000","last_became_peered":"2023-08-27T11:28:07.678059+0000","last_unstale":"2023-08-27T11:28:09.868277+0000","last_undegraded":"2023-08-27T11:28:09.868277+0000","last_fullsized":"2023-08-27T11:28:09.868277+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:19:35.003341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867434+0000","last_change":"2023-08-27T11:28:07.679674+0000","last_active":"2023-08-27T11:28:09.867434+0000","last_peered":"2023-08-27T11:28:09.867434+0000","last_clean":"2023-08-27T11:28:09.867434+0000","last_became_active":"2023-08-27T11:28:07.679475+0000","last_became_peered":"2023-08-27T11:28:07.679475+0000","last_unstale":"2023-08-27T11:28:09.867434+0000","last_undegraded":"2023-08-27T11:28:09.867434+0000","last_fullsized":"2023-08-27T11:28:09.867434+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:05.083621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662236+0000","last_change":"2023-08-27T11:28:07.679277+0000","last_active":"2023-08-27T11:28:08.662236+0000","last_peered":"2023-08-27T11:28:08.662236+0000","last_clean":"2023-08-27T11:28:08.662236+0000","last_became_active":"2023-08-27T11:28:07.679005+0000","last_became_peered":"2023-08-27T11:28:07.679005+0000","last_unstale":"2023-08-27T11:28:08.662236+0000","last_undegraded":"2023-08-27T11:28:08.662236+0000","last_fullsized":"2023-08-27T11:28:08.662236+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:54:02.073308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868521+0000","last_change":"2023-08-27T11:28:07.679195+0000","last_active":"2023-08-27T11:28:09.868521+0000","last_peered":"2023-08-27T11:28:09.868521+0000","last_clean":"2023-08-27T11:28:09.868521+0000","last_became_active":"2023-08-27T11:28:07.678925+0000","last_became_peered":"2023-08-27T11:28:07.678925+0000","last_unstale":"2023-08-27T11:28:09.868521+0000","last_undegraded":"2023-08-27T11:28:09.868521+0000","last_fullsized":"2023-08-27T11:28:09.868521+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:04:57.589274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925369+0000","last_change":"2023-08-27T11:28:07.680054+0000","last_active":"2023-08-27T11:28:08.925369+0000","last_peered":"2023-08-27T11:28:08.925369+0000","last_clean":"2023-08-27T11:28:08.925369+0000","last_became_active":"2023-08-27T11:28:07.679775+0000","last_became_peered":"2023-08-27T11:28:07.679775+0000","last_unstale":"2023-08-27T11:28:08.925369+0000","last_undegraded":"2023-08-27T11:28:08.925369+0000","last_fullsized":"2023-08-27T11:28:08.925369+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:46:22.019723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663240+0000","last_change":"2023-08-27T11:28:07.671008+0000","last_active":"2023-08-27T11:28:08.663240+0000","last_peered":"2023-08-27T11:28:08.663240+0000","last_clean":"2023-08-27T11:28:08.663240+0000","last_became_active":"2023-08-27T11:28:07.670448+0000","last_became_peered":"2023-08-27T11:28:07.670448+0000","last_unstale":"2023-08-27T11:28:08.663240+0000","last_undegraded":"2023-08-27T11:28:08.663240+0000","last_fullsized":"2023-08-27T11:28:08.663240+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:32.404704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867510+0000","last_change":"2023-08-27T11:28:07.677274+0000","last_active":"2023-08-27T11:28:09.867510+0000","last_peered":"2023-08-27T11:28:09.867510+0000","last_clean":"2023-08-27T11:28:09.867510+0000","last_became_active":"2023-08-27T11:28:07.677013+0000","last_became_peered":"2023-08-27T11:28:07.677013+0000","last_unstale":"2023-08-27T11:28:09.867510+0000","last_undegraded":"2023-08-27T11:28:09.867510+0000","last_fullsized":"2023-08-27T11:28:09.867510+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:24.361375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867961+0000","last_change":"2023-08-27T11:28:07.670925+0000","last_active":"2023-08-27T11:28:09.867961+0000","last_peered":"2023-08-27T11:28:09.867961+0000","last_clean":"2023-08-27T11:28:09.867961+0000","last_became_active":"2023-08-27T11:28:07.670661+0000","last_became_peered":"2023-08-27T11:28:07.670661+0000","last_unstale":"2023-08-27T11:28:09.867961+0000","last_undegraded":"2023-08-27T11:28:09.867961+0000","last_fullsized":"2023-08-27T11:28:09.867961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:35:35.382117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867990+0000","last_change":"2023-08-27T11:28:07.682464+0000","last_active":"2023-08-27T11:28:09.867990+0000","last_peered":"2023-08-27T11:28:09.867990+0000","last_clean":"2023-08-27T11:28:09.867990+0000","last_became_active":"2023-08-27T11:28:07.682064+0000","last_became_peered":"2023-08-27T11:28:07.682064+0000","last_unstale":"2023-08-27T11:28:09.867990+0000","last_undegraded":"2023-08-27T11:28:09.867990+0000","last_fullsized":"2023-08-27T11:28:09.867990+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:47:26.211000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662793+0000","last_change":"2023-08-27T11:28:07.677892+0000","last_active":"2023-08-27T11:28:08.662793+0000","last_peered":"2023-08-27T11:28:08.662793+0000","last_clean":"2023-08-27T11:28:08.662793+0000","last_became_active":"2023-08-27T11:28:07.677185+0000","last_became_peered":"2023-08-27T11:28:07.677185+0000","last_unstale":"2023-08-27T11:28:08.662793+0000","last_undegraded":"2023-08-27T11:28:08.662793+0000","last_fullsized":"2023-08-27T11:28:08.662793+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:53:08.741736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867416+0000","last_change":"2023-08-27T11:28:07.677266+0000","last_active":"2023-08-27T11:28:09.867416+0000","last_peered":"2023-08-27T11:28:09.867416+0000","last_clean":"2023-08-27T11:28:09.867416+0000","last_became_active":"2023-08-27T11:28:07.677028+0000","last_became_peered":"2023-08-27T11:28:07.677028+0000","last_unstale":"2023-08-27T11:28:09.867416+0000","last_undegraded":"2023-08-27T11:28:09.867416+0000","last_fullsized":"2023-08-27T11:28:09.867416+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:37:00.540369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663522+0000","last_change":"2023-08-27T11:28:07.678671+0000","last_active":"2023-08-27T11:28:08.663522+0000","last_peered":"2023-08-27T11:28:08.663522+0000","last_clean":"2023-08-27T11:28:08.663522+0000","last_became_active":"2023-08-27T11:28:07.678327+0000","last_became_peered":"2023-08-27T11:28:07.678327+0000","last_unstale":"2023-08-27T11:28:08.663522+0000","last_undegraded":"2023-08-27T11:28:08.663522+0000","last_fullsized":"2023-08-27T11:28:08.663522+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:26:57.890774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"99'4","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868152+0000","last_change":"2023-08-27T11:28:06.680510+0000","last_active":"2023-08-27T11:28:09.868152+0000","last_peered":"2023-08-27T11:28:09.868152+0000","last_clean":"2023-08-27T11:28:09.868152+0000","last_became_active":"2023-08-27T11:28:06.680268+0000","last_became_peered":"2023-08-27T11:28:06.680268+0000","last_unstale":"2023-08-27T11:28:09.868152+0000","last_undegraded":"2023-08-27T11:28:09.868152+0000","last_fullsized":"2023-08-27T11:28:09.868152+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:12:21.105769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867690+0000","last_change":"2023-08-27T11:28:07.680035+0000","last_active":"2023-08-27T11:28:09.867690+0000","last_peered":"2023-08-27T11:28:09.867690+0000","last_clean":"2023-08-27T11:28:09.867690+0000","last_became_active":"2023-08-27T11:28:07.679782+0000","last_became_peered":"2023-08-27T11:28:07.679782+0000","last_unstale":"2023-08-27T11:28:09.867690+0000","last_undegraded":"2023-08-27T11:28:09.867690+0000","last_fullsized":"2023-08-27T11:28:09.867690+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:41:43.327224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663844+0000","last_change":"2023-08-27T11:28:06.678558+0000","last_active":"2023-08-27T11:28:08.663844+0000","last_peered":"2023-08-27T11:28:08.663844+0000","last_clean":"2023-08-27T11:28:08.663844+0000","last_became_active":"2023-08-27T11:28:06.678323+0000","last_became_peered":"2023-08-27T11:28:06.678323+0000","last_unstale":"2023-08-27T11:28:08.663844+0000","last_undegraded":"2023-08-27T11:28:08.663844+0000","last_fullsized":"2023-08-27T11:28:08.663844+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:32:06.173379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867573+0000","last_change":"2023-08-27T11:28:07.677896+0000","last_active":"2023-08-27T11:28:09.867573+0000","last_peered":"2023-08-27T11:28:09.867573+0000","last_clean":"2023-08-27T11:28:09.867573+0000","last_became_active":"2023-08-27T11:28:07.677598+0000","last_became_peered":"2023-08-27T11:28:07.677598+0000","last_unstale":"2023-08-27T11:28:09.867573+0000","last_undegraded":"2023-08-27T11:28:09.867573+0000","last_fullsized":"2023-08-27T11:28:09.867573+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:45:14.808778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.932034+0000","last_change":"2023-08-27T11:28:06.674106+0000","last_active":"2023-08-27T11:28:08.932034+0000","last_peered":"2023-08-27T11:28:08.932034+0000","last_clean":"2023-08-27T11:28:08.932034+0000","last_became_active":"2023-08-27T11:28:06.673843+0000","last_became_peered":"2023-08-27T11:28:06.673843+0000","last_unstale":"2023-08-27T11:28:08.932034+0000","last_undegraded":"2023-08-27T11:28:08.932034+0000","last_fullsized":"2023-08-27T11:28:08.932034+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:45:18.157216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925448+0000","last_change":"2023-08-27T11:28:07.688828+0000","last_active":"2023-08-27T11:28:08.925448+0000","last_peered":"2023-08-27T11:28:08.925448+0000","last_clean":"2023-08-27T11:28:08.925448+0000","last_became_active":"2023-08-27T11:28:07.688565+0000","last_became_peered":"2023-08-27T11:28:07.688565+0000","last_unstale":"2023-08-27T11:28:08.925448+0000","last_undegraded":"2023-08-27T11:28:08.925448+0000","last_fullsized":"2023-08-27T11:28:08.925448+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:13:49.060824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931823+0000","last_change":"2023-08-27T11:28:06.674658+0000","last_active":"2023-08-27T11:28:08.931823+0000","last_peered":"2023-08-27T11:28:08.931823+0000","last_clean":"2023-08-27T11:28:08.931823+0000","last_became_active":"2023-08-27T11:28:06.674384+0000","last_became_peered":"2023-08-27T11:28:06.674384+0000","last_unstale":"2023-08-27T11:28:08.931823+0000","last_undegraded":"2023-08-27T11:28:08.931823+0000","last_fullsized":"2023-08-27T11:28:08.931823+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:03:03.327541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867881+0000","last_change":"2023-08-27T11:28:07.688098+0000","last_active":"2023-08-27T11:28:09.867881+0000","last_peered":"2023-08-27T11:28:09.867881+0000","last_clean":"2023-08-27T11:28:09.867881+0000","last_became_active":"2023-08-27T11:28:07.687812+0000","last_became_peered":"2023-08-27T11:28:07.687812+0000","last_unstale":"2023-08-27T11:28:09.867881+0000","last_undegraded":"2023-08-27T11:28:09.867881+0000","last_fullsized":"2023-08-27T11:28:09.867881+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:34:51.915290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663473+0000","last_change":"2023-08-27T11:28:06.678516+0000","last_active":"2023-08-27T11:28:08.663473+0000","last_peered":"2023-08-27T11:28:08.663473+0000","last_clean":"2023-08-27T11:28:08.663473+0000","last_became_active":"2023-08-27T11:28:06.678268+0000","last_became_peered":"2023-08-27T11:28:06.678268+0000","last_unstale":"2023-08-27T11:28:08.663473+0000","last_undegraded":"2023-08-27T11:28:08.663473+0000","last_fullsized":"2023-08-27T11:28:08.663473+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:35:25.961654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663363+0000","last_change":"2023-08-27T11:28:07.669267+0000","last_active":"2023-08-27T11:28:08.663363+0000","last_peered":"2023-08-27T11:28:08.663363+0000","last_clean":"2023-08-27T11:28:08.663363+0000","last_became_active":"2023-08-27T11:28:07.668956+0000","last_became_peered":"2023-08-27T11:28:07.668956+0000","last_unstale":"2023-08-27T11:28:08.663363+0000","last_undegraded":"2023-08-27T11:28:08.663363+0000","last_fullsized":"2023-08-27T11:28:08.663363+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:06:27.686425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662723+0000","last_change":"2023-08-27T11:28:06.677993+0000","last_active":"2023-08-27T11:28:08.662723+0000","last_peered":"2023-08-27T11:28:08.662723+0000","last_clean":"2023-08-27T11:28:08.662723+0000","last_became_active":"2023-08-27T11:28:06.677679+0000","last_became_peered":"2023-08-27T11:28:06.677679+0000","last_unstale":"2023-08-27T11:28:08.662723+0000","last_undegraded":"2023-08-27T11:28:08.662723+0000","last_fullsized":"2023-08-27T11:28:08.662723+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:09:40.330031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925190+0000","last_change":"2023-08-27T11:28:07.681847+0000","last_active":"2023-08-27T11:28:08.925190+0000","last_peered":"2023-08-27T11:28:08.925190+0000","last_clean":"2023-08-27T11:28:08.925190+0000","last_became_active":"2023-08-27T11:28:07.681682+0000","last_became_peered":"2023-08-27T11:28:07.681682+0000","last_unstale":"2023-08-27T11:28:08.925190+0000","last_undegraded":"2023-08-27T11:28:08.925190+0000","last_fullsized":"2023-08-27T11:28:08.925190+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:26:32.295306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931633+0000","last_change":"2023-08-27T11:28:06.675717+0000","last_active":"2023-08-27T11:28:08.931633+0000","last_peered":"2023-08-27T11:28:08.931633+0000","last_clean":"2023-08-27T11:28:08.931633+0000","last_became_active":"2023-08-27T11:28:06.675161+0000","last_became_peered":"2023-08-27T11:28:06.675161+0000","last_unstale":"2023-08-27T11:28:08.931633+0000","last_undegraded":"2023-08-27T11:28:08.931633+0000","last_fullsized":"2023-08-27T11:28:08.931633+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:43:34.289123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867531+0000","last_change":"2023-08-27T11:28:07.675465+0000","last_active":"2023-08-27T11:28:09.867531+0000","last_peered":"2023-08-27T11:28:09.867531+0000","last_clean":"2023-08-27T11:28:09.867531+0000","last_became_active":"2023-08-27T11:28:07.674848+0000","last_became_peered":"2023-08-27T11:28:07.674848+0000","last_unstale":"2023-08-27T11:28:09.867531+0000","last_undegraded":"2023-08-27T11:28:09.867531+0000","last_fullsized":"2023-08-27T11:28:09.867531+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:31:30.398643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867715+0000","last_change":"2023-08-27T11:28:06.680053+0000","last_active":"2023-08-27T11:28:09.867715+0000","last_peered":"2023-08-27T11:28:09.867715+0000","last_clean":"2023-08-27T11:28:09.867715+0000","last_became_active":"2023-08-27T11:28:06.679714+0000","last_became_peered":"2023-08-27T11:28:06.679714+0000","last_unstale":"2023-08-27T11:28:09.867715+0000","last_undegraded":"2023-08-27T11:28:09.867715+0000","last_fullsized":"2023-08-27T11:28:09.867715+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:12:36.819810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868481+0000","last_change":"2023-08-27T11:28:07.668883+0000","last_active":"2023-08-27T11:28:09.868481+0000","last_peered":"2023-08-27T11:28:09.868481+0000","last_clean":"2023-08-27T11:28:09.868481+0000","last_became_active":"2023-08-27T11:28:07.668610+0000","last_became_peered":"2023-08-27T11:28:07.668610+0000","last_unstale":"2023-08-27T11:28:09.868481+0000","last_undegraded":"2023-08-27T11:28:09.868481+0000","last_fullsized":"2023-08-27T11:28:09.868481+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:40:40.728261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868025+0000","last_change":"2023-08-27T11:28:06.682417+0000","last_active":"2023-08-27T11:28:09.868025+0000","last_peered":"2023-08-27T11:28:09.868025+0000","last_clean":"2023-08-27T11:28:09.868025+0000","last_became_active":"2023-08-27T11:28:06.682174+0000","last_became_peered":"2023-08-27T11:28:06.682174+0000","last_unstale":"2023-08-27T11:28:09.868025+0000","last_undegraded":"2023-08-27T11:28:09.868025+0000","last_fullsized":"2023-08-27T11:28:09.868025+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:42:39.388886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664495+0000","last_change":"2023-08-27T11:28:07.669839+0000","last_active":"2023-08-27T11:28:08.664495+0000","last_peered":"2023-08-27T11:28:08.664495+0000","last_clean":"2023-08-27T11:28:08.664495+0000","last_became_active":"2023-08-27T11:28:07.669560+0000","last_became_peered":"2023-08-27T11:28:07.669560+0000","last_unstale":"2023-08-27T11:28:08.664495+0000","last_undegraded":"2023-08-27T11:28:08.664495+0000","last_fullsized":"2023-08-27T11:28:08.664495+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:49:19.501789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867751+0000","last_change":"2023-08-27T11:28:06.682913+0000","last_active":"2023-08-27T11:28:09.867751+0000","last_peered":"2023-08-27T11:28:09.867751+0000","last_clean":"2023-08-27T11:28:09.867751+0000","last_became_active":"2023-08-27T11:28:06.682773+0000","last_became_peered":"2023-08-27T11:28:06.682773+0000","last_unstale":"2023-08-27T11:28:09.867751+0000","last_undegraded":"2023-08-27T11:28:09.867751+0000","last_fullsized":"2023-08-27T11:28:09.867751+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:51:19.760938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931740+0000","last_change":"2023-08-27T11:28:07.680805+0000","last_active":"2023-08-27T11:28:08.931740+0000","last_peered":"2023-08-27T11:28:08.931740+0000","last_clean":"2023-08-27T11:28:08.931740+0000","last_became_active":"2023-08-27T11:28:07.680554+0000","last_became_peered":"2023-08-27T11:28:07.680554+0000","last_unstale":"2023-08-27T11:28:08.931740+0000","last_undegraded":"2023-08-27T11:28:08.931740+0000","last_fullsized":"2023-08-27T11:28:08.931740+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:01:54.263663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"99'3","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867562+0000","last_change":"2023-08-27T11:28:06.680796+0000","last_active":"2023-08-27T11:28:09.867562+0000","last_peered":"2023-08-27T11:28:09.867562+0000","last_clean":"2023-08-27T11:28:09.867562+0000","last_became_active":"2023-08-27T11:28:06.680639+0000","last_became_peered":"2023-08-27T11:28:06.680639+0000","last_unstale":"2023-08-27T11:28:09.867562+0000","last_undegraded":"2023-08-27T11:28:09.867562+0000","last_fullsized":"2023-08-27T11:28:09.867562+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:12:56.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663214+0000","last_change":"2023-08-27T11:28:07.675412+0000","last_active":"2023-08-27T11:28:08.663214+0000","last_peered":"2023-08-27T11:28:08.663214+0000","last_clean":"2023-08-27T11:28:08.663214+0000","last_became_active":"2023-08-27T11:28:07.675144+0000","last_became_peered":"2023-08-27T11:28:07.675144+0000","last_unstale":"2023-08-27T11:28:08.663214+0000","last_undegraded":"2023-08-27T11:28:08.663214+0000","last_fullsized":"2023-08-27T11:28:08.663214+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:10:25.359793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"99'6","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664550+0000","last_change":"2023-08-27T11:28:06.675117+0000","last_active":"2023-08-27T11:28:08.664550+0000","last_peered":"2023-08-27T11:28:08.664550+0000","last_clean":"2023-08-27T11:28:08.664550+0000","last_became_active":"2023-08-27T11:28:06.674808+0000","last_became_peered":"2023-08-27T11:28:06.674808+0000","last_unstale":"2023-08-27T11:28:08.664550+0000","last_undegraded":"2023-08-27T11:28:08.664550+0000","last_fullsized":"2023-08-27T11:28:08.664550+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:23.790945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931693+0000","last_change":"2023-08-27T11:28:07.675482+0000","last_active":"2023-08-27T11:28:08.931693+0000","last_peered":"2023-08-27T11:28:08.931693+0000","last_clean":"2023-08-27T11:28:08.931693+0000","last_became_active":"2023-08-27T11:28:07.675220+0000","last_became_peered":"2023-08-27T11:28:07.675220+0000","last_unstale":"2023-08-27T11:28:08.931693+0000","last_undegraded":"2023-08-27T11:28:08.931693+0000","last_fullsized":"2023-08-27T11:28:08.931693+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:04:33.068250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868103+0000","last_change":"2023-08-27T11:28:06.670804+0000","last_active":"2023-08-27T11:28:09.868103+0000","last_peered":"2023-08-27T11:28:09.868103+0000","last_clean":"2023-08-27T11:28:09.868103+0000","last_became_active":"2023-08-27T11:28:06.669783+0000","last_became_peered":"2023-08-27T11:28:06.669783+0000","last_unstale":"2023-08-27T11:28:09.868103+0000","last_undegraded":"2023-08-27T11:28:09.868103+0000","last_fullsized":"2023-08-27T11:28:09.868103+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:55:30.263823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925236+0000","last_change":"2023-08-27T11:28:07.680648+0000","last_active":"2023-08-27T11:28:08.925236+0000","last_peered":"2023-08-27T11:28:08.925236+0000","last_clean":"2023-08-27T11:28:08.925236+0000","last_became_active":"2023-08-27T11:28:07.680394+0000","last_became_peered":"2023-08-27T11:28:07.680394+0000","last_unstale":"2023-08-27T11:28:08.925236+0000","last_undegraded":"2023-08-27T11:28:08.925236+0000","last_fullsized":"2023-08-27T11:28:08.925236+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:18:11.967022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868129+0000","last_change":"2023-08-27T11:28:06.668164+0000","last_active":"2023-08-27T11:28:09.868129+0000","last_peered":"2023-08-27T11:28:09.868129+0000","last_clean":"2023-08-27T11:28:09.868129+0000","last_became_active":"2023-08-27T11:28:06.667870+0000","last_became_peered":"2023-08-27T11:28:06.667870+0000","last_unstale":"2023-08-27T11:28:09.868129+0000","last_undegraded":"2023-08-27T11:28:09.868129+0000","last_fullsized":"2023-08-27T11:28:09.868129+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:27:49.190931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867668+0000","last_change":"2023-08-27T11:28:07.669960+0000","last_active":"2023-08-27T11:28:09.867668+0000","last_peered":"2023-08-27T11:28:09.867668+0000","last_clean":"2023-08-27T11:28:09.867668+0000","last_became_active":"2023-08-27T11:28:07.669694+0000","last_became_peered":"2023-08-27T11:28:07.669694+0000","last_unstale":"2023-08-27T11:28:09.867668+0000","last_undegraded":"2023-08-27T11:28:09.867668+0000","last_fullsized":"2023-08-27T11:28:09.867668+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:49:07.996812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664320+0000","last_change":"2023-08-27T11:28:06.679870+0000","last_active":"2023-08-27T11:28:08.664320+0000","last_peered":"2023-08-27T11:28:08.664320+0000","last_clean":"2023-08-27T11:28:08.664320+0000","last_became_active":"2023-08-27T11:28:06.679627+0000","last_became_peered":"2023-08-27T11:28:06.679627+0000","last_unstale":"2023-08-27T11:28:08.664320+0000","last_undegraded":"2023-08-27T11:28:08.664320+0000","last_fullsized":"2023-08-27T11:28:08.664320+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:48:43.447662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867990+0000","last_change":"2023-08-27T11:28:07.678143+0000","last_active":"2023-08-27T11:28:09.867990+0000","last_peered":"2023-08-27T11:28:09.867990+0000","last_clean":"2023-08-27T11:28:09.867990+0000","last_became_active":"2023-08-27T11:28:07.677891+0000","last_became_peered":"2023-08-27T11:28:07.677891+0000","last_unstale":"2023-08-27T11:28:09.867990+0000","last_undegraded":"2023-08-27T11:28:09.867990+0000","last_fullsized":"2023-08-27T11:28:09.867990+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:23:22.550290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"99'8","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931515+0000","last_change":"2023-08-27T11:28:06.681196+0000","last_active":"2023-08-27T11:28:08.931515+0000","last_peered":"2023-08-27T11:28:08.931515+0000","last_clean":"2023-08-27T11:28:08.931515+0000","last_became_active":"2023-08-27T11:28:06.680914+0000","last_became_peered":"2023-08-27T11:28:06.680914+0000","last_unstale":"2023-08-27T11:28:08.931515+0000","last_undegraded":"2023-08-27T11:28:08.931515+0000","last_fullsized":"2023-08-27T11:28:08.931515+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:44:40.935233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663721+0000","last_change":"2023-08-27T11:28:07.672326+0000","last_active":"2023-08-27T11:28:08.663721+0000","last_peered":"2023-08-27T11:28:08.663721+0000","last_clean":"2023-08-27T11:28:08.663721+0000","last_became_active":"2023-08-27T11:28:07.672072+0000","last_became_peered":"2023-08-27T11:28:07.672072+0000","last_unstale":"2023-08-27T11:28:08.663721+0000","last_undegraded":"2023-08-27T11:28:08.663721+0000","last_fullsized":"2023-08-27T11:28:08.663721+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:44:36.439939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868661+0000","last_change":"2023-08-27T11:28:06.679947+0000","last_active":"2023-08-27T11:28:09.868661+0000","last_peered":"2023-08-27T11:28:09.868661+0000","last_clean":"2023-08-27T11:28:09.868661+0000","last_became_active":"2023-08-27T11:28:06.679654+0000","last_became_peered":"2023-08-27T11:28:06.679654+0000","last_unstale":"2023-08-27T11:28:09.868661+0000","last_undegraded":"2023-08-27T11:28:09.868661+0000","last_fullsized":"2023-08-27T11:28:09.868661+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:59.536465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925347+0000","last_change":"2023-08-27T11:28:07.681398+0000","last_active":"2023-08-27T11:28:08.925347+0000","last_peered":"2023-08-27T11:28:08.925347+0000","last_clean":"2023-08-27T11:28:08.925347+0000","last_became_active":"2023-08-27T11:28:07.681114+0000","last_became_peered":"2023-08-27T11:28:07.681114+0000","last_unstale":"2023-08-27T11:28:08.925347+0000","last_undegraded":"2023-08-27T11:28:08.925347+0000","last_fullsized":"2023-08-27T11:28:08.925347+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:35:50.595452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"100'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868734+0000","last_change":"2023-08-27T11:28:06.668640+0000","last_active":"2023-08-27T11:28:09.868734+0000","last_peered":"2023-08-27T11:28:09.868734+0000","last_clean":"2023-08-27T11:28:09.868734+0000","last_became_active":"2023-08-27T11:28:06.668372+0000","last_became_peered":"2023-08-27T11:28:06.668372+0000","last_unstale":"2023-08-27T11:28:09.868734+0000","last_undegraded":"2023-08-27T11:28:09.868734+0000","last_fullsized":"2023-08-27T11:28:09.868734+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:04.079717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867719+0000","last_change":"2023-08-27T11:28:07.688916+0000","last_active":"2023-08-27T11:28:09.867719+0000","last_peered":"2023-08-27T11:28:09.867719+0000","last_clean":"2023-08-27T11:28:09.867719+0000","last_became_active":"2023-08-27T11:28:07.688643+0000","last_became_peered":"2023-08-27T11:28:07.688643+0000","last_unstale":"2023-08-27T11:28:09.867719+0000","last_undegraded":"2023-08-27T11:28:09.867719+0000","last_fullsized":"2023-08-27T11:28:09.867719+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:50:55.754306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867615+0000","last_change":"2023-08-27T11:28:06.682044+0000","last_active":"2023-08-27T11:28:09.867615+0000","last_peered":"2023-08-27T11:28:09.867615+0000","last_clean":"2023-08-27T11:28:09.867615+0000","last_became_active":"2023-08-27T11:28:06.681789+0000","last_became_peered":"2023-08-27T11:28:06.681789+0000","last_unstale":"2023-08-27T11:28:09.867615+0000","last_undegraded":"2023-08-27T11:28:09.867615+0000","last_fullsized":"2023-08-27T11:28:09.867615+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:13.346126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663393+0000","last_change":"2023-08-27T11:28:07.677884+0000","last_active":"2023-08-27T11:28:08.663393+0000","last_peered":"2023-08-27T11:28:08.663393+0000","last_clean":"2023-08-27T11:28:08.663393+0000","last_became_active":"2023-08-27T11:28:07.677487+0000","last_became_peered":"2023-08-27T11:28:07.677487+0000","last_unstale":"2023-08-27T11:28:08.663393+0000","last_undegraded":"2023-08-27T11:28:08.663393+0000","last_fullsized":"2023-08-27T11:28:08.663393+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:35:31.096744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867814+0000","last_change":"2023-08-27T11:28:06.682213+0000","last_active":"2023-08-27T11:28:09.867814+0000","last_peered":"2023-08-27T11:28:09.867814+0000","last_clean":"2023-08-27T11:28:09.867814+0000","last_became_active":"2023-08-27T11:28:06.681772+0000","last_became_peered":"2023-08-27T11:28:06.681772+0000","last_unstale":"2023-08-27T11:28:09.867814+0000","last_undegraded":"2023-08-27T11:28:09.867814+0000","last_fullsized":"2023-08-27T11:28:09.867814+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:39:19.722528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664491+0000","last_change":"2023-08-27T11:28:07.669667+0000","last_active":"2023-08-27T11:28:08.664491+0000","last_peered":"2023-08-27T11:28:08.664491+0000","last_clean":"2023-08-27T11:28:08.664491+0000","last_became_active":"2023-08-27T11:28:07.669351+0000","last_became_peered":"2023-08-27T11:28:07.669351+0000","last_unstale":"2023-08-27T11:28:08.664491+0000","last_undegraded":"2023-08-27T11:28:08.664491+0000","last_fullsized":"2023-08-27T11:28:08.664491+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:04.037732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931987+0000","last_change":"2023-08-27T11:28:06.681163+0000","last_active":"2023-08-27T11:28:08.931987+0000","last_peered":"2023-08-27T11:28:08.931987+0000","last_clean":"2023-08-27T11:28:08.931987+0000","last_became_active":"2023-08-27T11:28:06.680922+0000","last_became_peered":"2023-08-27T11:28:06.680922+0000","last_unstale":"2023-08-27T11:28:08.931987+0000","last_undegraded":"2023-08-27T11:28:08.931987+0000","last_fullsized":"2023-08-27T11:28:08.931987+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:56:35.006229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931888+0000","last_change":"2023-08-27T11:28:07.676445+0000","last_active":"2023-08-27T11:28:08.931888+0000","last_peered":"2023-08-27T11:28:08.931888+0000","last_clean":"2023-08-27T11:28:08.931888+0000","last_became_active":"2023-08-27T11:28:07.676177+0000","last_became_peered":"2023-08-27T11:28:07.676177+0000","last_unstale":"2023-08-27T11:28:08.931888+0000","last_undegraded":"2023-08-27T11:28:08.931888+0000","last_fullsized":"2023-08-27T11:28:08.931888+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:23:07.113470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662945+0000","last_change":"2023-08-27T11:28:06.677834+0000","last_active":"2023-08-27T11:28:08.662945+0000","last_peered":"2023-08-27T11:28:08.662945+0000","last_clean":"2023-08-27T11:28:08.662945+0000","last_became_active":"2023-08-27T11:28:06.677483+0000","last_became_peered":"2023-08-27T11:28:06.677483+0000","last_unstale":"2023-08-27T11:28:08.662945+0000","last_undegraded":"2023-08-27T11:28:08.662945+0000","last_fullsized":"2023-08-27T11:28:08.662945+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:02:19.201761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925125+0000","last_change":"2023-08-27T11:28:07.680102+0000","last_active":"2023-08-27T11:28:08.925125+0000","last_peered":"2023-08-27T11:28:08.925125+0000","last_clean":"2023-08-27T11:28:08.925125+0000","last_became_active":"2023-08-27T11:28:07.679763+0000","last_became_peered":"2023-08-27T11:28:07.679763+0000","last_unstale":"2023-08-27T11:28:08.925125+0000","last_undegraded":"2023-08-27T11:28:08.925125+0000","last_fullsized":"2023-08-27T11:28:08.925125+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:08.791817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867686+0000","last_change":"2023-08-27T11:28:06.682127+0000","last_active":"2023-08-27T11:28:09.867686+0000","last_peered":"2023-08-27T11:28:09.867686+0000","last_clean":"2023-08-27T11:28:09.867686+0000","last_became_active":"2023-08-27T11:28:06.681824+0000","last_became_peered":"2023-08-27T11:28:06.681824+0000","last_unstale":"2023-08-27T11:28:09.867686+0000","last_undegraded":"2023-08-27T11:28:09.867686+0000","last_fullsized":"2023-08-27T11:28:09.867686+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:08.775563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867593+0000","last_change":"2023-08-27T11:28:07.688903+0000","last_active":"2023-08-27T11:28:09.867593+0000","last_peered":"2023-08-27T11:28:09.867593+0000","last_clean":"2023-08-27T11:28:09.867593+0000","last_became_active":"2023-08-27T11:28:07.688655+0000","last_became_peered":"2023-08-27T11:28:07.688655+0000","last_unstale":"2023-08-27T11:28:09.867593+0000","last_undegraded":"2023-08-27T11:28:09.867593+0000","last_fullsized":"2023-08-27T11:28:09.867593+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:47:29.786044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662914+0000","last_change":"2023-08-27T11:28:06.676211+0000","last_active":"2023-08-27T11:28:08.662914+0000","last_peered":"2023-08-27T11:28:08.662914+0000","last_clean":"2023-08-27T11:28:08.662914+0000","last_became_active":"2023-08-27T11:28:06.675956+0000","last_became_peered":"2023-08-27T11:28:06.675956+0000","last_unstale":"2023-08-27T11:28:08.662914+0000","last_undegraded":"2023-08-27T11:28:08.662914+0000","last_fullsized":"2023-08-27T11:28:08.662914+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:00:27.941134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664407+0000","last_change":"2023-08-27T11:28:07.669975+0000","last_active":"2023-08-27T11:28:08.664407+0000","last_peered":"2023-08-27T11:28:08.664407+0000","last_clean":"2023-08-27T11:28:08.664407+0000","last_became_active":"2023-08-27T11:28:07.669705+0000","last_became_peered":"2023-08-27T11:28:07.669705+0000","last_unstale":"2023-08-27T11:28:08.664407+0000","last_undegraded":"2023-08-27T11:28:08.664407+0000","last_fullsized":"2023-08-27T11:28:08.664407+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:45:18.976487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662805+0000","last_change":"2023-08-27T11:28:06.673096+0000","last_active":"2023-08-27T11:28:08.662805+0000","last_peered":"2023-08-27T11:28:08.662805+0000","last_clean":"2023-08-27T11:28:08.662805+0000","last_became_active":"2023-08-27T11:28:06.672819+0000","last_became_peered":"2023-08-27T11:28:06.672819+0000","last_unstale":"2023-08-27T11:28:08.662805+0000","last_undegraded":"2023-08-27T11:28:08.662805+0000","last_fullsized":"2023-08-27T11:28:08.662805+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:55:16.402317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868259+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:09.868259+0000","last_peered":"2023-08-27T11:28:09.868259+0000","last_clean":"2023-08-27T11:28:09.868259+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:09.868259+0000","last_undegraded":"2023-08-27T11:28:09.868259+0000","last_fullsized":"2023-08-27T11:28:09.868259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:46:44.234002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.663320+0000","last_change":"2023-08-27T11:28:07.671184+0000","last_active":"2023-08-27T11:28:08.663320+0000","last_peered":"2023-08-27T11:28:08.663320+0000","last_clean":"2023-08-27T11:28:08.663320+0000","last_became_active":"2023-08-27T11:28:07.670765+0000","last_became_peered":"2023-08-27T11:28:07.670765+0000","last_unstale":"2023-08-27T11:28:08.663320+0000","last_undegraded":"2023-08-27T11:28:08.663320+0000","last_fullsized":"2023-08-27T11:28:08.663320+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:30:58.929519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931902+0000","last_change":"2023-08-27T11:28:06.676306+0000","last_active":"2023-08-27T11:28:08.931902+0000","last_peered":"2023-08-27T11:28:08.931902+0000","last_clean":"2023-08-27T11:28:08.931902+0000","last_became_active":"2023-08-27T11:28:06.676049+0000","last_became_peered":"2023-08-27T11:28:06.676049+0000","last_unstale":"2023-08-27T11:28:08.931902+0000","last_undegraded":"2023-08-27T11:28:08.931902+0000","last_fullsized":"2023-08-27T11:28:08.931902+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:42:00.713332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664623+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:08.664623+0000","last_peered":"2023-08-27T11:28:08.664623+0000","last_clean":"2023-08-27T11:28:08.664623+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:08.664623+0000","last_undegraded":"2023-08-27T11:28:08.664623+0000","last_fullsized":"2023-08-27T11:28:08.664623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:30.750930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925314+0000","last_change":"2023-08-27T11:28:07.676539+0000","last_active":"2023-08-27T11:28:08.925314+0000","last_peered":"2023-08-27T11:28:08.925314+0000","last_clean":"2023-08-27T11:28:08.925314+0000","last_became_active":"2023-08-27T11:28:07.676277+0000","last_became_peered":"2023-08-27T11:28:07.676277+0000","last_unstale":"2023-08-27T11:28:08.925314+0000","last_undegraded":"2023-08-27T11:28:08.925314+0000","last_fullsized":"2023-08-27T11:28:08.925314+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:35:09.323698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867941+0000","last_change":"2023-08-27T11:28:06.680009+0000","last_active":"2023-08-27T11:28:09.867941+0000","last_peered":"2023-08-27T11:28:09.867941+0000","last_clean":"2023-08-27T11:28:09.867941+0000","last_became_active":"2023-08-27T11:28:06.679754+0000","last_became_peered":"2023-08-27T11:28:06.679754+0000","last_unstale":"2023-08-27T11:28:09.867941+0000","last_undegraded":"2023-08-27T11:28:09.867941+0000","last_fullsized":"2023-08-27T11:28:09.867941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:51:02.824686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662485+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:08.662485+0000","last_peered":"2023-08-27T11:28:08.662485+0000","last_clean":"2023-08-27T11:28:08.662485+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:08.662485+0000","last_undegraded":"2023-08-27T11:28:08.662485+0000","last_fullsized":"2023-08-27T11:28:08.662485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:48.287729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868209+0000","last_change":"2023-08-27T11:28:07.674884+0000","last_active":"2023-08-27T11:28:09.868209+0000","last_peered":"2023-08-27T11:28:09.868209+0000","last_clean":"2023-08-27T11:28:09.868209+0000","last_became_active":"2023-08-27T11:28:07.674625+0000","last_became_peered":"2023-08-27T11:28:07.674625+0000","last_unstale":"2023-08-27T11:28:09.868209+0000","last_undegraded":"2023-08-27T11:28:09.868209+0000","last_fullsized":"2023-08-27T11:28:09.868209+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:24:34.281107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"99'1","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925500+0000","last_change":"2023-08-27T11:28:06.680561+0000","last_active":"2023-08-27T11:28:08.925500+0000","last_peered":"2023-08-27T11:28:08.925500+0000","last_clean":"2023-08-27T11:28:08.925500+0000","last_became_active":"2023-08-27T11:28:06.680307+0000","last_became_peered":"2023-08-27T11:28:06.680307+0000","last_unstale":"2023-08-27T11:28:08.925500+0000","last_undegraded":"2023-08-27T11:28:08.925500+0000","last_fullsized":"2023-08-27T11:28:08.925500+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:28:38.737568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664404+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:08.664404+0000","last_peered":"2023-08-27T11:28:08.664404+0000","last_clean":"2023-08-27T11:28:08.664404+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:08.664404+0000","last_undegraded":"2023-08-27T11:28:08.664404+0000","last_fullsized":"2023-08-27T11:28:08.664404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:31:30.892704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"100'2897","reported_seq":3267,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:12.491412+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:12.491412+0000","last_peered":"2023-08-27T11:28:12.491412+0000","last_clean":"2023-08-27T11:28:12.491412+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:12.491412+0000","last_undegraded":"2023-08-27T11:28:12.491412+0000","last_fullsized":"2023-08-27T11:28:12.491412+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":2897,"log_dups_size":0,"ondisk_log_size":2897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:20:45.957307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":595,"num_read_kb":1134,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868166+0000","last_change":"2023-08-27T11:28:07.678018+0000","last_active":"2023-08-27T11:28:09.868166+0000","last_peered":"2023-08-27T11:28:09.868166+0000","last_clean":"2023-08-27T11:28:09.868166+0000","last_became_active":"2023-08-27T11:28:07.677528+0000","last_became_peered":"2023-08-27T11:28:07.677528+0000","last_unstale":"2023-08-27T11:28:09.868166+0000","last_undegraded":"2023-08-27T11:28:09.868166+0000","last_fullsized":"2023-08-27T11:28:09.868166+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:35:35.000343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.931924+0000","last_change":"2023-08-27T11:28:06.678931+0000","last_active":"2023-08-27T11:28:08.931924+0000","last_peered":"2023-08-27T11:28:08.931924+0000","last_clean":"2023-08-27T11:28:08.931924+0000","last_became_active":"2023-08-27T11:28:06.678661+0000","last_became_peered":"2023-08-27T11:28:06.678661+0000","last_unstale":"2023-08-27T11:28:08.931924+0000","last_undegraded":"2023-08-27T11:28:08.931924+0000","last_fullsized":"2023-08-27T11:28:08.931924+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:18.410935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925380+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:08.925380+0000","last_peered":"2023-08-27T11:28:08.925380+0000","last_clean":"2023-08-27T11:28:08.925380+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:08.925380+0000","last_undegraded":"2023-08-27T11:28:08.925380+0000","last_fullsized":"2023-08-27T11:28:08.925380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:01.769251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662622+0000","last_change":"2023-08-27T11:28:07.677007+0000","last_active":"2023-08-27T11:28:08.662622+0000","last_peered":"2023-08-27T11:28:08.662622+0000","last_clean":"2023-08-27T11:28:08.662622+0000","last_became_active":"2023-08-27T11:28:07.676740+0000","last_became_peered":"2023-08-27T11:28:07.676740+0000","last_unstale":"2023-08-27T11:28:08.662622+0000","last_undegraded":"2023-08-27T11:28:08.662622+0000","last_fullsized":"2023-08-27T11:28:08.662622+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:35:34.026496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"99'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867786+0000","last_change":"2023-08-27T11:28:06.672441+0000","last_active":"2023-08-27T11:28:09.867786+0000","last_peered":"2023-08-27T11:28:09.867786+0000","last_clean":"2023-08-27T11:28:09.867786+0000","last_became_active":"2023-08-27T11:28:06.672183+0000","last_became_peered":"2023-08-27T11:28:06.672183+0000","last_unstale":"2023-08-27T11:28:09.867786+0000","last_undegraded":"2023-08-27T11:28:09.867786+0000","last_fullsized":"2023-08-27T11:28:09.867786+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:53:51.459344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.925575+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:08.925575+0000","last_peered":"2023-08-27T11:28:08.925575+0000","last_clean":"2023-08-27T11:28:08.925575+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:08.925575+0000","last_undegraded":"2023-08-27T11:28:08.925575+0000","last_fullsized":"2023-08-27T11:28:08.925575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:38:37.335560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.664264+0000","last_change":"2023-08-27T11:28:07.676628+0000","last_active":"2023-08-27T11:28:08.664264+0000","last_peered":"2023-08-27T11:28:08.664264+0000","last_clean":"2023-08-27T11:28:08.664264+0000","last_became_active":"2023-08-27T11:28:07.676306+0000","last_became_peered":"2023-08-27T11:28:07.676306+0000","last_unstale":"2023-08-27T11:28:08.664264+0000","last_undegraded":"2023-08-27T11:28:08.664264+0000","last_fullsized":"2023-08-27T11:28:08.664264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:37:58.662891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867444+0000","last_change":"2023-08-27T11:28:06.666649+0000","last_active":"2023-08-27T11:28:09.867444+0000","last_peered":"2023-08-27T11:28:09.867444+0000","last_clean":"2023-08-27T11:28:09.867444+0000","last_became_active":"2023-08-27T11:28:06.666348+0000","last_became_peered":"2023-08-27T11:28:06.666348+0000","last_unstale":"2023-08-27T11:28:09.867444+0000","last_undegraded":"2023-08-27T11:28:09.867444+0000","last_fullsized":"2023-08-27T11:28:09.867444+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:33:18.521265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.868713+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:09.868713+0000","last_peered":"2023-08-27T11:28:09.868713+0000","last_clean":"2023-08-27T11:28:09.868713+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:09.868713+0000","last_undegraded":"2023-08-27T11:28:09.868713+0000","last_fullsized":"2023-08-27T11:28:09.868713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:20:19.563680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662331+0000","last_change":"2023-08-27T11:28:07.671383+0000","last_active":"2023-08-27T11:28:08.662331+0000","last_peered":"2023-08-27T11:28:08.662331+0000","last_clean":"2023-08-27T11:28:08.662331+0000","last_became_active":"2023-08-27T11:28:07.669562+0000","last_became_peered":"2023-08-27T11:28:07.669562+0000","last_unstale":"2023-08-27T11:28:08.662331+0000","last_undegraded":"2023-08-27T11:28:08.662331+0000","last_fullsized":"2023-08-27T11:28:08.662331+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:06.651961+0000","last_clean_scrub_stamp":"2023-08-27T11:28:06.651961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:50:34.528742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:08.662281+0000","last_change":"2023-08-27T11:28:06.683725+0000","last_active":"2023-08-27T11:28:08.662281+0000","last_peered":"2023-08-27T11:28:08.662281+0000","last_clean":"2023-08-27T11:28:08.662281+0000","last_became_active":"2023-08-27T11:28:06.683424+0000","last_became_peered":"2023-08-27T11:28:06.683424+0000","last_unstale":"2023-08-27T11:28:08.662281+0000","last_undegraded":"2023-08-27T11:28:08.662281+0000","last_fullsized":"2023-08-27T11:28:08.662281+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:05.648057+0000","last_clean_scrub_stamp":"2023-08-27T11:28:05.648057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:11:03.672577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":162,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-08-27T11:28:09.867790+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:09.867790+0000","last_peered":"2023-08-27T11:28:09.867790+0000","last_clean":"2023-08-27T11:28:09.867790+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:09.867790+0000","last_undegraded":"2023-08-27T11:28:09.867790+0000","last_fullsized":"2023-08-27T11:28:09.867790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:06:40.407425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":595,"num_read_kb":1134,"num_write":253,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2897,"ondisk_log_size":2897,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575042,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28796,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343044,"statfs":{"total":96636764160,"available":96607277056,"internally_reserved":0,"allocated":1822720,"data_stored":1232038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.38500000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":1.601,"15min":1.601},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.59999999999999998,"15min":0.58299999999999996},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.38200000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.32500000000000001},"max":{"1min":0.71199999999999997,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.314},"max":{"1min":0.77000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46700000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.32400000000000001,"15min":0.252},"max":{"1min":0.80100000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.29099999999999998},"max":{"1min":0.749,"5min":1.778,"15min":1.778},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.375,"5min":0.35199999999999998,"15min":0.35099999999999998},"max":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63200000000000001,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.60699999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.23499999999999999},"max":{"1min":0.65300000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.68799999999999994,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70299999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65300000000000002,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69299999999999995}]},{"osd":6,"last update":"Sun Aug 27 11:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.45200000000000001,"5min":0.36899999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.85199999999999998,"15min":0.90100000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.623}]}]},{"osd":6,"up_from":13,"seq":55834575042,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1236992,"data_stored":642294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.40799999999999997,"5min":0.28000000000000003,"15min":0.26400000000000001},"max":{"1min":0.76700000000000002,"5min":0.95299999999999996,"15min":1.3400000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.628},"min":{"1min":0.47399999999999998,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":1.3540000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":3.3780000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.41199999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.80700000000000005,"5min":0.98099999999999998,"15min":3.3490000000000002},"last":0.46600000000000003}]},{"osd":2,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.61699999999999999},"min":{"1min":0.40600000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.73399999999999999,"5min":0.81899999999999995,"15min":9.3770000000000007},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":1.325},"last":0.42999999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.60499999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.71699999999999997,"5min":1.1910000000000001,"15min":8.3390000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.86299999999999999,"15min":1.395},"last":0.54600000000000004}]},{"osd":4,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47799999999999998,"15min":0.46400000000000002},"min":{"1min":0.312,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"min":{"1min":0.35199999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.624,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57799999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.33100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.72499999999999998,"5min":1.157,"15min":1.157},"last":0.65500000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.81299999999999994,"15min":0.88900000000000001},"last":0.624},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.70399999999999996,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.59599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40708,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331132,"statfs":{"total":96636764160,"available":96595079168,"internally_reserved":0,"allocated":1236992,"data_stored":641794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38200000000000001,"15min":0.38700000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.20399999999999999},"max":{"1min":0.52100000000000002,"5min":0.60199999999999998,"15min":0.65600000000000003},"last":0.254},{"interface":"front","average":{"1min":0.435,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.28899999999999998,"15min":0.254},"max":{"1min":0.628,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.36599999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42699999999999999,"15min":0.42399999999999999},"min":{"1min":0.27500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48099999999999998},"min":{"1min":0.34100000000000003,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58999999999999997,"5min":0.75800000000000001,"15min":0.78600000000000003},"last":0.40300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47699999999999998,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":1.169,"15min":1.169},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44600000000000001,"15min":0.44},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.23400000000000001},"max":{"1min":0.55900000000000005,"5min":1.139,"15min":1.139},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.30599999999999999},"max":{"1min":0.82599999999999996,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.64900000000000002,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.81499999999999995,"5min":0.89300000000000002,"15min":2.9380000000000002},"last":0.57399999999999995}]},{"osd":5,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.39100000000000001},"max":{"1min":0.83299999999999996,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.58899999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42099999999999999},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67400000000000004,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88700000000000001,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.63400000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.42499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89400000000000002,"15min":1.139},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79600000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575042,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28796,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343044,"statfs":{"total":96636764160,"available":96607277056,"internally_reserved":0,"allocated":1822720,"data_stored":1232002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.40300000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53100000000000003,"5min":0.78400000000000003,"15min":0.90200000000000002},"last":0.22},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.27400000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67400000000000004,"5min":0.68700000000000006,"15min":1.2889999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":1.3340000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.437,"15min":0.441},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":3.7469999999999999,"15min":8.1349999999999998},"last":0.35099999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.42799999999999999,"15min":0.46300000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23899999999999999},"max":{"1min":0.56999999999999995,"5min":0.84999999999999998,"15min":11.026999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.54700000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.628,"5min":0.73299999999999998,"15min":11.926},"last":0.443}]},{"osd":4,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.67600000000000005},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":1.0069999999999999,"15min":10.964},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.872,"15min":1.3640000000000001},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.89300000000000002,"5min":3.6400000000000001,"15min":8.0779999999999994},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66300000000000003,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.96799999999999997,"15min":10.922000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.38700000000000001,"15min":0.35699999999999998},"max":{"1min":0.85499999999999998,"5min":0.98699999999999999,"15min":1.3919999999999999},"last":0.625},{"interface":"front","average":{"1min":0.63,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":3.6859999999999999,"15min":8.0380000000000003},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Aug 27 11:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70099999999999996,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":3.7160000000000002,"15min":8.1069999999999993},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.036,"15min":10.996},"last":0.70599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1228800,"data_stored":642182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.224,"5min":0.191,"15min":0.191},"max":{"1min":0.48999999999999999,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42999999999999999,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.217,"15min":0.20799999999999999},"max":{"1min":0.54500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40400000000000003}]},{"osd":1,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.753},"last":0.433},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.44800000000000001,"15min":0.44900000000000001},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52500000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.378},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45300000000000001},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":1.163},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.621}]},{"osd":5,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":1.083,"15min":1.2470000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.84899999999999998,"5min":1.0109999999999999,"15min":1.2330000000000001},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.66600000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":1.2050000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88500000000000001,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.88900000000000001,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.57899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1236992,"data_stored":643212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40699999999999997,"15min":0.42499999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.22},"max":{"1min":0.60799999999999998,"5min":1.3959999999999999,"15min":11.885999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.224},"max":{"1min":0.60699999999999998,"5min":0.68100000000000005,"15min":0.71399999999999997},"last":0.31900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.64100000000000001,"15min":6.0830000000000002},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46200000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.245},"max":{"1min":0.57399999999999995,"5min":0.89800000000000002,"15min":12.050000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.68000000000000005,"5min":0.71199999999999997,"15min":1.25},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":0.84499999999999997,"15min":11.939},"last":0.34100000000000003}]},{"osd":4,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65800000000000003,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.376},"max":{"1min":0.83099999999999996,"5min":1.0329999999999999,"15min":11.007},"last":0.439},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82699999999999996,"5min":1.113,"15min":1.2929999999999999},"last":0.40500000000000003}]},{"osd":5,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93300000000000005,"5min":3.0219999999999998,"15min":11.981},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":1.218,"15min":11.09},"last":0.497}]},{"osd":6,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69099999999999995,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.38200000000000001},"max":{"1min":0.90800000000000003,"5min":1.2010000000000001,"15min":12.789999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.75600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96799999999999997,"5min":1.0269999999999999,"15min":12.016},"last":0.55000000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68999999999999995,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":0.96699999999999997,"15min":11.054},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.53500000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84099999999999997,"5min":1.0640000000000001,"15min":1.361},"last":0.50900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575042,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1224704,"data_stored":641600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.59099999999999997},"min":{"1min":0.373,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79300000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.255},"max":{"1min":0.79300000000000004,"5min":0.86499999999999999,"15min":0.873},"last":0.374}]},{"osd":1,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.624,"15min":0.60899999999999999},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.312},"max":{"1min":0.82199999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.442}]},{"osd":3,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.58399999999999996,"5min":0.34999999999999998,"15min":0.30099999999999999},"max":{"1min":0.751,"5min":0.879,"15min":0.879},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.88800000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.312,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.755,"5min":0.79100000000000004,"15min":0.85699999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.26200000000000001},"max":{"1min":0.70599999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.39100000000000001}]},{"osd":6,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.27800000000000002},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":1.0860000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.33400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.40600000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73799999999999999,"5min":0.874,"15min":0.878},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.85599999999999998,"15min":1.1339999999999999},"last":0.52900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575041,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36560,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335280,"statfs":{"total":96636764160,"available":96599326720,"internally_reserved":0,"allocated":1241088,"data_stored":643234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86599999999999999,"15min":0.93300000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.26200000000000001},"max":{"1min":0.78100000000000003,"5min":0.85199999999999998,"15min":0.88100000000000001},"last":0.37}]},{"osd":1,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.33400000000000002,"15min":0.248},"max":{"1min":0.71199999999999997,"5min":0.85299999999999998,"15min":0.96999999999999997},"last":0.439},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51500000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.87,"15min":0.87},"last":0.505},{"interface":"front","average":{"1min":0.629,"5min":0.624,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.28399999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":1.0109999999999999},"last":0.41599999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.32000000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":1.073,"15min":1.073},"last":0.38600000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":1.157,"15min":1.157},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.34200000000000003,"5min":0.317,"15min":0.26000000000000001},"max":{"1min":0.72299999999999998,"5min":0.77700000000000002,"15min":0.86599999999999999},"last":0.496}]},{"osd":6,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.495,"15min":0.497},"min":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.46899999999999997}]},{"osd":7,"last update":"Sun Aug 27 11:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.27800000000000002},"max":{"1min":0.77900000000000003,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.40400000000000003,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:16.893 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:28:16.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:28:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.057+0000 7f75611ac700 1 Processor -- start 2023-08-27T11:28:17.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.057+0000 7f75611ac700 1 -- start start 2023-08-27T11:28:17.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.058+0000 7f75611ac700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f755c06ed90 0x7f755c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.058+0000 7f75611ac700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755c0cfae0 con 0x7f755c0cd300 2023-08-27T11:28:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.058+0000 7f75611ac700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755c15e900 con 0x7f755c15a0f0 2023-08-27T11:28:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.058+0000 7f75611ac700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755c0cae60 con 0x7f755c06ed90 2023-08-27T11:28:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.058+0000 7f755a59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f755c0cd300 0x7f755c0cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45956/0 (socket says 172.21.15.45:45956) 2023-08-27T11:28:17.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.058+0000 7f755a59c700 1 -- 172.21.15.45:0/2597784989 learned_addr learned my addr 172.21.15.45:0/2597784989 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972014415 0 0) 0x7f755c0cfae0 con 0x7f755c0cd300 2023-08-27T11:28:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75440035b0 con 0x7f755c0cd300 2023-08-27T11:28:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 943047230 0 0) 0x7f75440035b0 con 0x7f755c0cd300 2023-08-27T11:28:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 >> v2:172.21.15.135:3300/0 conn(0x7f755c06ed90 msgr2=0x7f755c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:17.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 --2- 172.21.15.45:0/2597784989 >> v2:172.21.15.135:3300/0 conn(0x7f755c06ed90 0x7f755c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 >> v1:172.21.15.135:6789/0 conn(0x7f755c15a0f0 legacy=0x7f755c15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c0cb140 con 0x7f755c0cd300 2023-08-27T11:28:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f7550001670 con 0x7f755c0cd300 2023-08-27T11:28:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 >> v1:172.21.15.45:6789/0 conn(0x7f755c0cd300 legacy=0x7f755c0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544003a00 0x7f7544003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7544004850 0x7f7544004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7544005330 0x7f754400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0cae60 con 0x7f7544004850 2023-08-27T11:28:17.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754400c230 con 0x7f7544003a00 2023-08-27T11:28:17.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754400c3f0 con 0x7f7544005330 2023-08-27T11:28:17.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f755a59c700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7544004850 0x7f7544004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f755ad9d700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544003a00 0x7f7544003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f755a59c700 1 -- 172.21.15.45:0/2597784989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7544005330 msgr2=0x7f754400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:17.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f755a59c700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7544005330 0x7f754400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f755a59c700 1 -- 172.21.15.45:0/2597784989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544003a00 msgr2=0x7f7544003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:17.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f755a59c700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544003a00 0x7f7544003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f755a59c700 1 -- 172.21.15.45:0/2597784989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75440035b0 con 0x7f7544004850 2023-08-27T11:28:17.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f755ad9d700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544003a00 0x7f7544003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:17.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.059+0000 7f755b59e700 1 -- 172.21.15.45:0/2597784989 reap_dead start 2023-08-27T11:28:17.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f755a59c700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7544004850 0x7f7544004cd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7550001eb0 tx=0x7f7544005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf1310469cf30cb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755000ce00 con 0x7f7544004850 2023-08-27T11:28:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7550005960 con 0x7f7544004850 2023-08-27T11:28:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f7559d9b700 1 -- 172.21.15.45:0/2597784989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550005d10 con 0x7f7544004850 2023-08-27T11:28:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f75611ac700 1 -- 172.21.15.45:0/2597784989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7544004850 msgr2=0x7f7544004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f75611ac700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7544004850 0x7f7544004cd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7550001eb0 tx=0x7f7544005880 comp rx=0 tx=0).stop 2023-08-27T11:28:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f75611ac700 1 -- 172.21.15.45:0/2597784989 shutdown_connections 2023-08-27T11:28:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f75611ac700 1 --2- 172.21.15.45:0/2597784989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7544004850 0x7f7544004cd0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f75611ac700 1 -- 172.21.15.45:0/2597784989 >> 172.21.15.45:0/2597784989 conn(0x7f755c0cc7e0 msgr2=0x7f755c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f75611ac700 1 -- 172.21.15.45:0/2597784989 shutdown_connections 2023-08-27T11:28:17.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.060+0000 7f75611ac700 1 -- 172.21.15.45:0/2597784989 wait complete. 2023-08-27T11:28:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.062+0000 7f75611ac700 1 Processor -- start 2023-08-27T11:28:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.062+0000 7f75611ac700 1 -- start start 2023-08-27T11:28:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.063+0000 7f75611ac700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544004850 0x7f755c155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.063+0000 7f75611ac700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f755c0c3890 0x7f755c0c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.063+0000 7f75611ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f755c0b0ad0 0x7f755c0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.063+0000 7f75611ac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0b1490 con 0x7f755c0b0ad0 2023-08-27T11:28:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.063+0000 7f75611ac700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0c5c60 con 0x7f7544004850 2023-08-27T11:28:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.063+0000 7f75611ac700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0c5360 con 0x7f755c0c3890 2023-08-27T11:28:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755ad9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544004850 0x7f755c155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755ad9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544004850 0x7f755c155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:48046/0 (socket says 172.21.15.45:48046) 2023-08-27T11:28:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755ad9d700 1 -- 172.21.15.45:0/1206721048 learned_addr learned my addr 172.21.15.45:0/1206721048 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755a59c700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f755c0c3890 0x7f755c0c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755b59e700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f755c0b0ad0 0x7f755c0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755a59c700 1 -- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544004850 msgr2=0x7f755c155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755a59c700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544004850 0x7f755c155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755a59c700 1 -- 172.21.15.45:0/1206721048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f755c0b0ad0 msgr2=0x7f755c0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755a59c700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f755c0b0ad0 0x7f755c0b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755a59c700 1 -- 172.21.15.45:0/1206721048 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c0c4a10 con 0x7f755c0c3890 2023-08-27T11:28:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.064+0000 7f755ad9d700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544004850 0x7f755c155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:17.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.065+0000 7f755a59c700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f755c0c3890 0x7f755c0c5720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7550014000 tx=0x7f7550001a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:17.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.065+0000 7f75437fe700 1 -- 172.21.15.45:0/1206721048 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550004a80 con 0x7f755c0c3890 2023-08-27T11:28:17.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.065+0000 7f75437fe700 1 -- 172.21.15.45:0/1206721048 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7550004c00 con 0x7f755c0c3890 2023-08-27T11:28:17.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.065+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c0c3e00 con 0x7f755c0c3890 2023-08-27T11:28:17.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.065+0000 7f75437fe700 1 -- 172.21.15.45:0/1206721048 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75500070d0 con 0x7f755c0c3890 2023-08-27T11:28:17.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.065+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c0c43c0 con 0x7f755c0c3890 2023-08-27T11:28:17.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.066+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c0c4ba0 con 0x7f755c0c3890 2023-08-27T11:28:17.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.067+0000 7f75437fe700 1 -- 172.21.15.45:0/1206721048 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f7550007230 con 0x7f755c0c3890 2023-08-27T11:28:17.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.067+0000 7f75437fe700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f754806a170 0x7f754806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.067+0000 7f75437fe700 1 -- 172.21.15.45:0/1206721048 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f7550089dd0 con 0x7f755c0c3890 2023-08-27T11:28:17.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.067+0000 7f755ad9d700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f754806a170 0x7f754806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.068+0000 7f755ad9d700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f754806a170 0x7f754806c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f754c001a90 tx=0x7f754c009040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:17.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.072+0000 7f75437fe700 1 -- 172.21.15.45:0/1206721048 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f75500553f0 con 0x7f755c0c3890 2023-08-27T11:28:17.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.354+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f755c0726d0 con 0x7f755c0c3890 2023-08-27T11:28:17.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.359+0000 7f75437fe700 1 -- 172.21.15.45:0/1206721048 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f75500170b0 con 0x7f755c0c3890 2023-08-27T11:28:17.360 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-08-27T11:28:17.360 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ== 2023-08-27T11:28:17.360 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2023-08-27T11:28:17.361 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2023-08-27T11:28:17.361 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:28:17.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f754806a170 msgr2=0x7f754806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f754806a170 0x7f754806c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f754c001a90 tx=0x7f754c009040 comp rx=0 tx=0).stop 2023-08-27T11:28:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f755c0c3890 msgr2=0x7f755c0c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f755c0c3890 0x7f755c0c5720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7550014000 tx=0x7f7550001a00 comp rx=0 tx=0).stop 2023-08-27T11:28:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 shutdown_connections 2023-08-27T11:28:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f754806a170 0x7f754806c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f755c0b0ad0 0x7f755c0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7544004850 0x7f755c155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 --2- 172.21.15.45:0/1206721048 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f755c0c3890 0x7f755c0c5720 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 >> 172.21.15.45:0/1206721048 conn(0x7f755c0cc7e0 msgr2=0x7f755c0ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:17.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 shutdown_connections 2023-08-27T11:28:17.365 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.361+0000 7f75611ac700 1 -- 172.21.15.45:0/1206721048 wait complete. 2023-08-27T11:28:17.365 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-08-27T11:28:17.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:17.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.498+0000 7f4b21ef9700 1 Processor -- start 2023-08-27T11:28:17.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.498+0000 7f4b21ef9700 1 -- start start 2023-08-27T11:28:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.499+0000 7f4b21ef9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b1c0ceba0 0x7f4b1c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.499+0000 7f4b21ef9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0cbe70 0x7f4b1c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.499+0000 7f4b21ef9700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1c06d690 con 0x7f4b1c15a240 2023-08-27T11:28:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.499+0000 7f4b21ef9700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1c0cc440 con 0x7f4b1c0ceba0 2023-08-27T11:28:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.499+0000 7f4b21ef9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c06d040 con 0x7f4b1c0cbe70 2023-08-27T11:28:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.499+0000 7f4b1affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b1c0ceba0 0x7f4b1c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.499+0000 7f4b1affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b1c0ceba0 0x7f4b1c0cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59200/0 (socket says 172.21.15.45:59200) 2023-08-27T11:28:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.499+0000 7f4b1affd700 1 -- 172.21.15.45:0/3240278740 learned_addr learned my addr 172.21.15.45:0/3240278740 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.499+0000 7f4b1bfff700 1 --2- 172.21.15.45:0/3240278740 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0cbe70 0x7f4b1c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.500+0000 7f4b1affd700 1 -- 172.21.15.45:0/3240278740 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0cbe70 msgr2=0x7f4b1c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.500+0000 7f4b1affd700 1 --2- 172.21.15.45:0/3240278740 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0cbe70 0x7f4b1c06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.500+0000 7f4b1affd700 1 -- 172.21.15.45:0/3240278740 >> v1:172.21.15.135:6789/0 conn(0x7f4b1c15a240 legacy=0x7f4b1c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.500+0000 7f4b1affd700 1 -- 172.21.15.45:0/3240278740 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c06d320 con 0x7f4b1c0ceba0 2023-08-27T11:28:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.500+0000 7f4b1affd700 1 --2- 172.21.15.45:0/3240278740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c0ceba0 0x7f4b1c0cae80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4b100098f0 tx=0x7f4b10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a21787eae87ac6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.500+0000 7f4b1a7fc700 1 -- 172.21.15.45:0/3240278740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1000b8a0 con 0x7f4b1c0ceba0 2023-08-27T11:28:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.500+0000 7f4b1a7fc700 1 -- 172.21.15.45:0/3240278740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1000ba00 con 0x7f4b1c0ceba0 2023-08-27T11:28:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.500+0000 7f4b1a7fc700 1 -- 172.21.15.45:0/3240278740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1000bd90 con 0x7f4b1c0ceba0 2023-08-27T11:28:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.501+0000 7f4b21ef9700 1 -- 172.21.15.45:0/3240278740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c0ceba0 msgr2=0x7f4b1c0cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.501+0000 7f4b21ef9700 1 --2- 172.21.15.45:0/3240278740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c0ceba0 0x7f4b1c0cae80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4b100098f0 tx=0x7f4b10009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.501+0000 7f4b21ef9700 1 -- 172.21.15.45:0/3240278740 shutdown_connections 2023-08-27T11:28:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.501+0000 7f4b21ef9700 1 --2- 172.21.15.45:0/3240278740 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0cbe70 0x7f4b1c06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.501+0000 7f4b21ef9700 1 --2- 172.21.15.45:0/3240278740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c0ceba0 0x7f4b1c0cae80 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.501+0000 7f4b21ef9700 1 -- 172.21.15.45:0/3240278740 >> 172.21.15.45:0/3240278740 conn(0x7f4b1c0cfe50 msgr2=0x7f4b1c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.501+0000 7f4b21ef9700 1 -- 172.21.15.45:0/3240278740 shutdown_connections 2023-08-27T11:28:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.501+0000 7f4b21ef9700 1 -- 172.21.15.45:0/3240278740 wait complete. 2023-08-27T11:28:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.502+0000 7f4b21ef9700 1 Processor -- start 2023-08-27T11:28:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.502+0000 7f4b21ef9700 1 -- start start 2023-08-27T11:28:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b21ef9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b1c0cbe70 0x7f4b1c1f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b21ef9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0ceba0 0x7f4b1c0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b21ef9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c15a240 0x7f4b1c1e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b21ef9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c1f3230 con 0x7f4b1c15a240 2023-08-27T11:28:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b21ef9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c1576d0 con 0x7f4b1c0cbe70 2023-08-27T11:28:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b21ef9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c1579b0 con 0x7f4b1c0ceba0 2023-08-27T11:28:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c15a240 0x7f4b1c1e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c15a240 0x7f4b1c1e6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59208/0 (socket says 172.21.15.45:59208) 2023-08-27T11:28:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1bfff700 1 -- 172.21.15.45:0/1778558103 learned_addr learned my addr 172.21.15.45:0/1778558103 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1b7fe700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b1c0cbe70 0x7f4b1c1f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1affd700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0ceba0 0x7f4b1c0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1bfff700 1 -- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0ceba0 msgr2=0x7f4b1c0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1bfff700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0ceba0 0x7f4b1c0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1bfff700 1 -- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b1c0cbe70 msgr2=0x7f4b1c1f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1bfff700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b1c0cbe70 0x7f4b1c1f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1bfff700 1 -- 172.21.15.45:0/1778558103 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10009550 con 0x7f4b1c15a240 2023-08-27T11:28:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.503+0000 7f4b1bfff700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c15a240 0x7f4b1c1e6cb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4b14007e40 tx=0x7f4b1400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.504+0000 7f4b03fff700 1 -- 172.21.15.45:0/1778558103 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400ed30 con 0x7f4b1c15a240 2023-08-27T11:28:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.504+0000 7f4b03fff700 1 -- 172.21.15.45:0/1778558103 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b14014420 con 0x7f4b1c15a240 2023-08-27T11:28:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.504+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c157c90 con 0x7f4b1c15a240 2023-08-27T11:28:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.504+0000 7f4b03fff700 1 -- 172.21.15.45:0/1778558103 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b140147d0 con 0x7f4b1c15a240 2023-08-27T11:28:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.504+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c158250 con 0x7f4b1c15a240 2023-08-27T11:28:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.505+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1c072810 con 0x7f4b1c15a240 2023-08-27T11:28:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.505+0000 7f4b03fff700 1 -- 172.21.15.45:0/1778558103 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f4b14014930 con 0x7f4b1c15a240 2023-08-27T11:28:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.505+0000 7f4b03fff700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f4b0806a170 0x7f4b0806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.506+0000 7f4b03fff700 1 -- 172.21.15.45:0/1778558103 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f4b14088e50 con 0x7f4b1c15a240 2023-08-27T11:28:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.506+0000 7f4b1b7fe700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f4b0806a170 0x7f4b0806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.506+0000 7f4b1b7fe700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f4b0806a170 0x7f4b0806c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c007fd0 tx=0x7f4b0c009040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.510+0000 7f4b03fff700 1 -- 172.21.15.45:0/1778558103 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4b14054320 con 0x7f4b1c15a240 2023-08-27T11:28:17.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.755+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b1c14fa50 con 0x7f4b1c15a240 2023-08-27T11:28:17.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.756+0000 7f4b03fff700 1 -- 172.21.15.45:0/1778558103 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f4b14057940 con 0x7f4b1c15a240 2023-08-27T11:28:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:17.758 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3604468746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3604468746},{"type":"v1","addr":"172.21.15.135:6836","nonce":3604468746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1319491874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1319491874},{"type":"v1","addr":"172.21.15.135:6837","nonce":1319491874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6937,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2036943131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2036943131},{"type":"v1","addr":"172.21.15.45:6835","nonce":2036943131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:07.092667+0000","modified":"2023-08-27T11:28:10.679683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7446":{"gid":7446,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/3257398445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3257398445},{"type":"v1","addr":"172.21.15.45:6833","nonce":3257398445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:28:17.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f4b0806a170 msgr2=0x7f4b0806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f4b0806a170 0x7f4b0806c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c007fd0 tx=0x7f4b0c009040 comp rx=0 tx=0).stop 2023-08-27T11:28:17.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c15a240 msgr2=0x7f4b1c1e6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:17.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c15a240 0x7f4b1c1e6cb0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4b14007e40 tx=0x7f4b1400e3c0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 shutdown_connections 2023-08-27T11:28:17.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f4b0806a170 0x7f4b0806c630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b1c15a240 0x7f4b1c1e6cb0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b1c0cbe70 0x7f4b1c1f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 --2- 172.21.15.45:0/1778558103 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f4b1c0ceba0 0x7f4b1c0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:17.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 >> 172.21.15.45:0/1778558103 conn(0x7f4b1c0cfe50 msgr2=0x7f4b1c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:17.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 shutdown_connections 2023-08-27T11:28:17.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:17.758+0000 7f4b21ef9700 1 -- 172.21.15.45:0/1778558103 wait complete. 2023-08-27T11:28:17.762 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 41 2023-08-27T11:28:18.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:28:07.092667+0000', 'modified': '2023-08-27T11:28:10.679683+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7446}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7446': {'gid': 7446, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6833/3257398445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 3257398445}, {'type': 'v1', 'addr': '172.21.15.45:6833', 'nonce': 3257398445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:28:18.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:28:18.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-27T11:28:18.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.303+0000 7fd420c61700 1 Processor -- start 2023-08-27T11:28:18.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.303+0000 7fd420c61700 1 -- start start 2023-08-27T11:28:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.304+0000 7fd420c61700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd41c15d3f0 0x7fd41c160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.304+0000 7fd420c61700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd41c0af420 0x7fd41c06ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.304+0000 7fd420c61700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41c071de0 con 0x7fd41c0aed60 2023-08-27T11:28:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.304+0000 7fd420c61700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c0c6ca0 con 0x7fd41c15d3f0 2023-08-27T11:28:18.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.304+0000 7fd420c61700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c15d7d0 con 0x7fd41c0af420 2023-08-27T11:28:18.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.304+0000 7fd41a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd41c15d3f0 0x7fd41c160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd419d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd41c0aed60 0x7fd41c161390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45988/0 (socket says 172.21.15.45:45988) 2023-08-27T11:28:18.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd419d9b700 1 -- 172.21.15.45:0/288527906 learned_addr learned my addr 172.21.15.45:0/288527906 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:18.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd41c15d3f0 0x7fd41c160e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59212/0 (socket says 172.21.15.45:59212) 2023-08-27T11:28:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41a59c700 1 -- 172.21.15.45:0/288527906 >> v2:172.21.15.135:3300/0 conn(0x7fd41c0af420 msgr2=0x7fd41c06ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41a59c700 1 --2- 172.21.15.45:0/288527906 >> v2:172.21.15.135:3300/0 conn(0x7fd41c0af420 0x7fd41c06ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41a59c700 1 -- 172.21.15.45:0/288527906 >> v1:172.21.15.45:6789/0 conn(0x7fd41c0aed60 legacy=0x7fd41c161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41a59c700 1 -- 172.21.15.45:0/288527906 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c0c5310 con 0x7fd41c15d3f0 2023-08-27T11:28:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41959a700 1 -- 172.21.15.45:0/288527906 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177139102 0 0) 0x7fd41c071de0 con 0x7fd41c0aed60 2023-08-27T11:28:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41a59c700 1 --2- 172.21.15.45:0/288527906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c15d3f0 0x7fd41c160e50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd41c06b320 tx=0x7fd40c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944fbe47ad6dfad7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41959a700 1 -- 172.21.15.45:0/288527906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c011830 con 0x7fd41c15d3f0 2023-08-27T11:28:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41959a700 1 -- 172.21.15.45:0/288527906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c011990 con 0x7fd41c15d3f0 2023-08-27T11:28:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.305+0000 7fd41959a700 1 -- 172.21.15.45:0/288527906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c011d20 con 0x7fd41c15d3f0 2023-08-27T11:28:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.306+0000 7fd420c61700 1 -- 172.21.15.45:0/288527906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c15d3f0 msgr2=0x7fd41c160e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.306+0000 7fd420c61700 1 --2- 172.21.15.45:0/288527906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c15d3f0 0x7fd41c160e50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd41c06b320 tx=0x7fd40c009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.306+0000 7fd420c61700 1 -- 172.21.15.45:0/288527906 shutdown_connections 2023-08-27T11:28:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.306+0000 7fd420c61700 1 --2- 172.21.15.45:0/288527906 >> v2:172.21.15.135:3300/0 conn(0x7fd41c0af420 0x7fd41c06ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.306+0000 7fd420c61700 1 --2- 172.21.15.45:0/288527906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c15d3f0 0x7fd41c160e50 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.306+0000 7fd420c61700 1 -- 172.21.15.45:0/288527906 >> 172.21.15.45:0/288527906 conn(0x7fd41c0cfdd0 msgr2=0x7fd41c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.306+0000 7fd420c61700 1 -- 172.21.15.45:0/288527906 shutdown_connections 2023-08-27T11:28:18.312 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.306+0000 7fd420c61700 1 -- 172.21.15.45:0/288527906 wait complete. 2023-08-27T11:28:18.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.307+0000 7fd420c61700 1 Processor -- start 2023-08-27T11:28:18.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.307+0000 7fd420c61700 1 -- start start 2023-08-27T11:28:18.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd420c61700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd41c0aed60 0x7fd41c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.316 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd420c61700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c0af420 0x7fd41c0adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd420c61700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd41c15d3f0 0x7fd41c1603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd420c61700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0ae1b0 con 0x7fd41c0af420 2023-08-27T11:28:18.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd420c61700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0acbf0 con 0x7fd41c0aed60 2023-08-27T11:28:18.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd420c61700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0aced0 con 0x7fd41c15d3f0 2023-08-27T11:28:18.317 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd419d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c0af420 0x7fd41c0adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd419d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c0af420 0x7fd41c0adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59220/0 (socket says 172.21.15.45:59220) 2023-08-27T11:28:18.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd419d9b700 1 -- 172.21.15.45:0/4001128854 learned_addr learned my addr 172.21.15.45:0/4001128854 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:18.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd41a59c700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd41c0aed60 0x7fd41c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd41c15d3f0 0x7fd41c1603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.318 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd419d9b700 1 -- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd41c15d3f0 msgr2=0x7fd41c1603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd419d9b700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd41c15d3f0 0x7fd41c1603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd419d9b700 1 -- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd41c0aed60 msgr2=0x7fd41c0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd419d9b700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd41c0aed60 0x7fd41c0cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd419d9b700 1 -- 172.21.15.45:0/4001128854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0094a0 con 0x7fd41c0af420 2023-08-27T11:28:18.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.308+0000 7fd41a59c700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd41c0aed60 0x7fd41c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:18.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.309+0000 7fd419d9b700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c0af420 0x7fd41c0adc70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd4100011e0 tx=0x7fd41000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.309+0000 7fd40affd700 1 -- 172.21.15.45:0/4001128854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410004c80 con 0x7fd41c0af420 2023-08-27T11:28:18.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.309+0000 7fd40affd700 1 -- 172.21.15.45:0/4001128854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd410004e00 con 0x7fd41c0af420 2023-08-27T11:28:18.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.309+0000 7fd420c61700 1 -- 172.21.15.45:0/4001128854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c0ad1b0 con 0x7fd41c0af420 2023-08-27T11:28:18.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.309+0000 7fd40affd700 1 -- 172.21.15.45:0/4001128854 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410007310 con 0x7fd41c0af420 2023-08-27T11:28:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.309+0000 7fd420c61700 1 -- 172.21.15.45:0/4001128854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c0ad770 con 0x7fd41c0af420 2023-08-27T11:28:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.310+0000 7fd40affd700 1 -- 172.21.15.45:0/4001128854 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 37) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fd410007470 con 0x7fd41c0af420 2023-08-27T11:28:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.311+0000 7fd40affd700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd3fc06a170 0x7fd3fc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.311+0000 7fd40affd700 1 -- 172.21.15.45:0/4001128854 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fd410088400 con 0x7fd41c0af420 2023-08-27T11:28:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.311+0000 7fd41a59c700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd3fc06a170 0x7fd3fc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.311+0000 7fd41a59c700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd3fc06a170 0x7fd3fc06c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0090b0 tx=0x7fd40c01c040 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.313+0000 7fd42ce23b80 1 Processor -- start 2023-08-27T11:28:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.313+0000 7fd42ce23b80 1 -- start start 2023-08-27T11:28:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.313+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.313+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86439b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.313+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc8643a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.313+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8636b730 con 0x55bc8644e960 2023-08-27T11:28:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.313+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc86369050 con 0x55bc8643cf40 2023-08-27T11:28:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.313+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc86364940 con 0x55bc864103d0 2023-08-27T11:28:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd419d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86439b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd419d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86439b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59228/0 (socket says 172.21.15.45:59228) 2023-08-27T11:28:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd419d9b700 1 -- 172.21.15.45:0/4267370769 learned_addr learned my addr 172.21.15.45:0/4267370769 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc8643a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd41a59c700 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd419d9b700 1 -- 172.21.15.45:0/4267370769 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 msgr2=0x55bc8643a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd419d9b700 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc8643a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd419d9b700 1 -- 172.21.15.45:0/4267370769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 msgr2=0x55bc8643d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd419d9b700 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd419d9b700 1 -- 172.21.15.45:0/4267370769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc861e45e0 con 0x55bc8644e960 2023-08-27T11:28:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd41a59c700 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd419d9b700 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86439b70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd410053bb0 tx=0x7fd4100570e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5a3d3d7f6cdd12 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd3fbfff700 1 -- 172.21.15.45:0/4267370769 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41005b470 con 0x55bc8644e960 2023-08-27T11:28:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd3fbfff700 1 -- 172.21.15.45:0/4267370769 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41005f3c0 con 0x55bc8644e960 2023-08-27T11:28:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.314+0000 7fd3fbfff700 1 -- 172.21.15.45:0/4267370769 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41005b0b0 con 0x55bc8644e960 2023-08-27T11:28:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.315+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4267370769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 msgr2=0x55bc86439b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.315+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86439b70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd410053bb0 tx=0x7fd4100570e0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.315+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4267370769 shutdown_connections 2023-08-27T11:28:18.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.315+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc8643a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.315+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.315+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4267370769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86439b70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.315+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4267370769 >> 172.21.15.45:0/4267370769 conn(0x55bc86370c30 msgr2=0x55bc86457d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.315+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4267370769 shutdown_connections 2023-08-27T11:28:18.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.315+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4267370769 wait complete. 2023-08-27T11:28:18.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.316+0000 7fd42ce23b80 1 -- start start 2023-08-27T11:28:18.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.316+0000 7fd42ce23b80 1 Processor -- start 2023-08-27T11:28:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc864103d0 0x55bc864668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc8643cf40 0x55bc86423100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86423640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc86302b60 con 0x55bc8644e960 2023-08-27T11:28:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc86302860 con 0x55bc864103d0 2023-08-27T11:28:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc861e68d0 con 0x55bc8643cf40 2023-08-27T11:28:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd42ce23b80 10 client.0 _ll_register_callbacks cb 0x55bc86300630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:28:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86423640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86423640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59236/0 (socket says 172.21.15.45:59236) 2023-08-27T11:28:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41ad9d700 1 -- 172.21.15.45:0/3321458107 learned_addr learned my addr 172.21.15.45:0/3321458107 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41a59c700 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc864103d0 0x55bc864668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd419d9b700 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc8643cf40 0x55bc86423100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41ad9d700 1 -- 172.21.15.45:0/3321458107 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc8643cf40 msgr2=0x55bc86423100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc8643cf40 0x55bc86423100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41ad9d700 1 -- 172.21.15.45:0/3321458107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc864103d0 msgr2=0x55bc864668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc864103d0 0x55bc864668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41ad9d700 1 -- 172.21.15.45:0/3321458107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c0ad770 con 0x55bc8644e960 2023-08-27T11:28:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86423640 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd41400be80 tx=0x7fd41400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5548993dcd686400 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd3f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:28:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3321458107 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414013070 con 0x55bc8644e960 2023-08-27T11:28:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3321458107 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fd414006e40 con 0x55bc8644e960 2023-08-27T11:28:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3321458107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc864054f0 con 0x55bc8644e960 2023-08-27T11:28:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3321458107 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd414009bc0 con 0x55bc8644e960 2023-08-27T11:28:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.317+0000 7fd42ce23b80 10 client.7623 fetch_fsmap learned FSMap version 41 2023-08-27T11:28:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3321458107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55bc8637c750 con 0x55bc8644e960 2023-08-27T11:28:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3321458107 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc864054f0 con 0x55bc8644e960 2023-08-27T11:28:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3321458107 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd414007670 con 0x55bc8644e960 2023-08-27T11:28:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd42ce23b80 10 client.7623 fetch_fsmap finished waiting for FSMap version 41 2023-08-27T11:28:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd42ce23b80 10 client.7623 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-27T11:28:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd42ce23b80 20 client.7623 populate_metadata read hostname 'smithi045' 2023-08-27T11:28:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] conn(0x55bc86440590 0x55bc86440970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd42ce23b80 4 client.7623 mds_command: new command op to 7446 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:28:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3321458107 --> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc8635d920 con 0x55bc86440590 2023-08-27T11:28:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd41a59c700 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] conn(0x55bc86440590 0x55bc86440970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd41a59c700 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] conn(0x55bc86440590 0x55bc86440970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23205293b19a0077 server_cookie=d86b71bb0a44ff89 in_seq=0 out_seq=0 2023-08-27T11:28:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.318+0000 7fd3f97fa700 10 client.7623 ms_handle_connect on v2:172.21.15.45:6832/3257398445 2023-08-27T11:28:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd3f97fa700 1 -- 172.21.15.45:0/3321458107 <== mds.0 v2:172.21.15.45:6832/3257398445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc8635d920 con 0x55bc86440590 2023-08-27T11:28:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd3f97fa700 10 client.7623 handle_command_reply: tid=0 2023-08-27T11:28:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 client.7623 shutdown 2023-08-27T11:28:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3321458107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 msgr2=0x55bc86423640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86423640 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd41400be80 tx=0x7fd41400bf60 comp rx=0 tx=0).stop 2023-08-27T11:28:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3321458107 shutdown_connections 2023-08-27T11:28:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] conn(0x55bc86440590 0x55bc86440970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc86423640 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc864103d0 0x55bc864668c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3321458107 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc8643cf40 0x55bc86423100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd3f97fa700 0 client.7623 ms_handle_reset on v2:172.21.15.45:6832/3257398445 2023-08-27T11:28:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3321458107 >> 172.21.15.45:0/3321458107 conn(0x55bc86370c30 msgr2=0x55bc864369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3321458107 shutdown_connections 2023-08-27T11:28:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3321458107 wait complete. 2023-08-27T11:28:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.319+0000 7fd42ce23b80 20 client.7623 trim_cache size 0 max 16384 2023-08-27T11:28:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.337+0000 7fd42ce23b80 1 Processor -- start 2023-08-27T11:28:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.337+0000 7fd42ce23b80 1 -- start start 2023-08-27T11:28:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc86458170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc8643d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc864108b0 con 0x55bc8644e960 2023-08-27T11:28:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8640c3e0 con 0x55bc8643cf40 2023-08-27T11:28:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc86405320 con 0x55bc864103d0 2023-08-27T11:28:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc8643d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc8643d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59244/0 (socket says 172.21.15.45:59244) 2023-08-27T11:28:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41ad9d700 1 -- 172.21.15.45:0/3882227163 learned_addr learned my addr 172.21.15.45:0/3882227163 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd419d9b700 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41a59c700 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc86458170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41ad9d700 1 -- 172.21.15.45:0/3882227163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 msgr2=0x55bc86458170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc86458170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41ad9d700 1 -- 172.21.15.45:0/3882227163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 msgr2=0x55bc8643d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41ad9d700 1 -- 172.21.15.45:0/3882227163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc86190d90 con 0x55bc8644e960 2023-08-27T11:28:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd419d9b700 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc8643d860 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd414009530 tx=0x7fd414013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f1de997cd43337 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd408ff9700 1 -- 172.21.15.45:0/3882227163 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414006e60 con 0x55bc8644e960 2023-08-27T11:28:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd408ff9700 1 -- 172.21.15.45:0/3882227163 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414009820 con 0x55bc8644e960 2023-08-27T11:28:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.338+0000 7fd408ff9700 1 -- 172.21.15.45:0/3882227163 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414007bd0 con 0x55bc8644e960 2023-08-27T11:28:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3882227163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 msgr2=0x55bc8643d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc8643d860 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd414009530 tx=0x7fd414013000 comp rx=0 tx=0).stop 2023-08-27T11:28:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3882227163 shutdown_connections 2023-08-27T11:28:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc8643d860 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc86458170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/3882227163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc8643d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3882227163 >> 172.21.15.45:0/3882227163 conn(0x55bc86370c30 msgr2=0x55bc86413d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3882227163 shutdown_connections 2023-08-27T11:28:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 -- 172.21.15.45:0/3882227163 wait complete. 2023-08-27T11:28:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.339+0000 7fd42ce23b80 1 -- start start 2023-08-27T11:28:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.340+0000 7fd42ce23b80 1 Processor -- start 2023-08-27T11:28:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.340+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc86439870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.340+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc86439db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.340+0000 7fd42ce23b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc864c2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.340+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc86422e40 con 0x55bc8644e960 2023-08-27T11:28:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.340+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc86436a30 con 0x55bc8643cf40 2023-08-27T11:28:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.340+0000 7fd42ce23b80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc86411a90 con 0x55bc864103d0 2023-08-27T11:28:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd42ce23b80 10 client.0 _ll_register_callbacks cb 0x55bc86300630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:28:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc864c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc864c2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59250/0 (socket says 172.21.15.45:59250) 2023-08-27T11:28:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41ad9d700 1 -- 172.21.15.45:0/4292486841 learned_addr learned my addr 172.21.15.45:0/4292486841 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd419d9b700 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc86439db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41a59c700 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc86439870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41ad9d700 1 -- 172.21.15.45:0/4292486841 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 msgr2=0x55bc86439870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc86439870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41ad9d700 1 -- 172.21.15.45:0/4292486841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 msgr2=0x55bc86439db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc86439db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41ad9d700 1 -- 172.21.15.45:0/4292486841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc8637c750 con 0x55bc8644e960 2023-08-27T11:28:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd41ad9d700 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc864c2340 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd41402a040 tx=0x7fd414004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4637f808f056976 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd3f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:28:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd3f97fa700 1 -- 172.21.15.45:0/4292486841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414013040 con 0x55bc8644e960 2023-08-27T11:28:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd3f97fa700 1 -- 172.21.15.45:0/4292486841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fd414007d90 con 0x55bc8644e960 2023-08-27T11:28:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4292486841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd41005f3c0 con 0x55bc8644e960 2023-08-27T11:28:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd3f97fa700 1 -- 172.21.15.45:0/4292486841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd414032450 con 0x55bc8644e960 2023-08-27T11:28:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd42ce23b80 10 client.7629 fetch_fsmap learned FSMap version 41 2023-08-27T11:28:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.341+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4292486841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55bc86179c10 con 0x55bc8644e960 2023-08-27T11:28:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd3f97fa700 1 -- 172.21.15.45:0/4292486841 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41005f3c0 con 0x55bc8644e960 2023-08-27T11:28:18.358 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd3f97fa700 1 -- 172.21.15.45:0/4292486841 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd41403b3f0 con 0x55bc8644e960 2023-08-27T11:28:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd42ce23b80 10 client.7629 fetch_fsmap finished waiting for FSMap version 41 2023-08-27T11:28:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd42ce23b80 10 client.7629 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-27T11:28:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd42ce23b80 20 client.7629 populate_metadata read hostname 'smithi045' 2023-08-27T11:28:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] conn(0x55bc864c2880 0x55bc8644d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.359 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd42ce23b80 4 client.7629 mds_command: new command op to 7446 tid=0[{"prefix": "status"}] 2023-08-27T11:28:18.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4292486841 --> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc8631e000 con 0x55bc864c2880 2023-08-27T11:28:18.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd41a59c700 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] conn(0x55bc864c2880 0x55bc8644d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd41a59c700 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] conn(0x55bc864c2880 0x55bc8644d6e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a839de2240c0d8 server_cookie=17478eec58d2af65 in_seq=0 out_seq=0 2023-08-27T11:28:18.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd3f97fa700 10 client.7629 ms_handle_connect on v2:172.21.15.45:6832/3257398445 2023-08-27T11:28:18.360 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd3f97fa700 1 -- 172.21.15.45:0/4292486841 <== mds.0 v2:172.21.15.45:6832/3257398445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55bc8631e000 con 0x55bc864c2880 2023-08-27T11:28:18.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd3f97fa700 10 client.7629 handle_command_reply: tid=0 2023-08-27T11:28:18.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.342+0000 7fd42ce23b80 1 client.7629 shutdown 2023-08-27T11:28:18.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4292486841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 msgr2=0x55bc864c2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc864c2340 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd41402a040 tx=0x7fd414004be0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.361 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4292486841 shutdown_connections 2023-08-27T11:28:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.45:6832/3257398445,v1:172.21.15.45:6833/3257398445] conn(0x55bc864c2880 0x55bc8644d6e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc8644e960 0x55bc864c2340 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55bc864103d0 0x55bc86439870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 --2- 172.21.15.45:0/4292486841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8643cf40 0x55bc86439db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd3f97fa700 0 client.7629 ms_handle_reset on v2:172.21.15.45:6832/3257398445 2023-08-27T11:28:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4292486841 >> 172.21.15.45:0/4292486841 conn(0x55bc86370c30 msgr2=0x55bc86413b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4292486841 shutdown_connections 2023-08-27T11:28:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 1 -- 172.21.15.45:0/4292486841 wait complete. 2023-08-27T11:28:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd42ce23b80 20 client.7629 trim_cache size 0 max 16384 2023-08-27T11:28:18.363 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:28:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "8ea1ae34-1697-4b79-b2ce-3f741cd3c828", 2023-08-27T11:28:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-08-27T11:28:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7446, 2023-08-27T11:28:18.364 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-08-27T11:28:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-08-27T11:28:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-08-27T11:28:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.668466806, 2023-08-27T11:28:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 41, 2023-08-27T11:28:18.365 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 101, 2023-08-27T11:28:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 100, 2023-08-27T11:28:18.366 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 16.498520437 2023-08-27T11:28:18.366 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:28:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 -- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd3fc06a170 msgr2=0x7fd3fc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd3fc06a170 0x7fd3fc06c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0090b0 tx=0x7fd40c01c040 comp rx=0 tx=0).stop 2023-08-27T11:28:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 -- 172.21.15.45:0/4001128854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c0af420 msgr2=0x7fd41c0adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c0af420 0x7fd41c0adc70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd4100011e0 tx=0x7fd41000cae0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 -- 172.21.15.45:0/4001128854 shutdown_connections 2023-08-27T11:28:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd3fc06a170 0x7fd3fc06c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd41c15d3f0 0x7fd41c1603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd41c0aed60 0x7fd41c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 --2- 172.21.15.45:0/4001128854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41c0af420 0x7fd41c0adc70 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 -- 172.21.15.45:0/4001128854 >> 172.21.15.45:0/4001128854 conn(0x7fd41c0cfdd0 msgr2=0x7fd41c15c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 -- 172.21.15.45:0/4001128854 shutdown_connections 2023-08-27T11:28:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.343+0000 7fd420c61700 1 -- 172.21.15.45:0/4001128854 wait complete. 2023-08-27T11:28:18.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:28:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.494+0000 7fd8b8ec4700 1 Processor -- start 2023-08-27T11:28:18.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.494+0000 7fd8b8ec4700 1 -- start start 2023-08-27T11:28:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b8ec4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8b415a0f0 0x7fd8b40cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b8ec4700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd8b406ed90 0x7fd8b415e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b8ec4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b40cfae0 con 0x7fd8b40cd300 2023-08-27T11:28:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b8ec4700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b415e900 con 0x7fd8b406ed90 2023-08-27T11:28:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b8ec4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40cae60 con 0x7fd8b415a0f0 2023-08-27T11:28:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b1d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd8b40cd300 0x7fd8b415daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46032/0 (socket says 172.21.15.45:46032) 2023-08-27T11:28:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b1d9b700 1 -- 172.21.15.45:0/3252518944 learned_addr learned my addr 172.21.15.45:0/3252518944 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615713126 0 0) 0x7fd8b40cfae0 con 0x7fd8b40cd300 2023-08-27T11:28:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b2d9d700 1 --2- 172.21.15.45:0/3252518944 >> v2:172.21.15.135:3300/0 conn(0x7fd8b406ed90 0x7fd8b415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b259c700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8b415a0f0 0x7fd8b40cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.495+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd89c0035b0 con 0x7fd8b40cd300 2023-08-27T11:28:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1662349595 0 0) 0x7fd89c0035b0 con 0x7fd8b40cd300 2023-08-27T11:28:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8b415a0f0 msgr2=0x7fd8b40cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8b415a0f0 0x7fd8b40cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 >> v2:172.21.15.135:3300/0 conn(0x7fd8b406ed90 msgr2=0x7fd8b415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 --2- 172.21.15.45:0/3252518944 >> v2:172.21.15.135:3300/0 conn(0x7fd8b406ed90 0x7fd8b415e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b40cb140 con 0x7fd8b40cd300 2023-08-27T11:28:18.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fd8a4001670 con 0x7fd8b40cd300 2023-08-27T11:28:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 >> v1:172.21.15.45:6789/0 conn(0x7fd8b40cd300 legacy=0x7fd8b415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89c003a00 0x7fd89c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c004850 0x7fd89c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8b415a0f0 0x7fd89c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b415e900 con 0x7fd89c003a00 2023-08-27T11:28:18.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40cae60 con 0x7fd89c004850 2023-08-27T11:28:18.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c00ba10 con 0x7fd8b415a0f0 2023-08-27T11:28:18.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b259c700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89c003a00 0x7fd89c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b2d9d700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8b415a0f0 0x7fd89c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b1d9b700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c004850 0x7fd89c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b259c700 1 -- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8b415a0f0 msgr2=0x7fd89c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b259c700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd8b415a0f0 0x7fd89c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b259c700 1 -- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c004850 msgr2=0x7fd89c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b259c700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c004850 0x7fd89c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b259c700 1 -- 172.21.15.45:0/3252518944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c0035b0 con 0x7fd89c003a00 2023-08-27T11:28:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b259c700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89c003a00 0x7fd89c003e20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80096f0 tx=0x7fd89c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797a3535268f86d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800e6c0 con 0x7fd89c003a00 2023-08-27T11:28:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a800e820 con 0x7fd89c003a00 2023-08-27T11:28:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.496+0000 7fd8b159a700 1 -- 172.21.15.45:0/3252518944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800ebb0 con 0x7fd89c003a00 2023-08-27T11:28:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.497+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/3252518944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89c003a00 msgr2=0x7fd89c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.497+0000 7fd8b8ec4700 1 --2- 172.21.15.45:0/3252518944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89c003a00 0x7fd89c003e20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80096f0 tx=0x7fd89c005880 comp rx=0 tx=0).stop 2023-08-27T11:28:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.497+0000 7fd8b2d9d700 1 -- 172.21.15.45:0/3252518944 reap_dead start 2023-08-27T11:28:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.497+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/3252518944 shutdown_connections 2023-08-27T11:28:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.497+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/3252518944 >> 172.21.15.45:0/3252518944 conn(0x7fd8b40cc7e0 msgr2=0x7fd8b40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.497+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/3252518944 shutdown_connections 2023-08-27T11:28:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.497+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/3252518944 wait complete. 2023-08-27T11:28:18.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.498+0000 7fd8b8ec4700 1 Processor -- start 2023-08-27T11:28:18.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.498+0000 7fd8b8ec4700 1 -- start start 2023-08-27T11:28:18.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b8ec4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd89c003a00 0x7fd8b40c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b8ec4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8b40c9cd0 0x7fd8b40c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b8ec4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8b40c6e30 0x7fd8b40c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b8ec4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40c50a0 con 0x7fd8b40c6e30 2023-08-27T11:28:18.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b8ec4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40c5380 con 0x7fd8b40c9cd0 2023-08-27T11:28:18.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b8ec4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40c5660 con 0x7fd89c003a00 2023-08-27T11:28:18.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8b40c6e30 0x7fd8b40c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b2d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8b40c6e30 0x7fd8b40c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59270/0 (socket says 172.21.15.45:59270) 2023-08-27T11:28:18.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b2d9d700 1 -- 172.21.15.45:0/1297397843 learned_addr learned my addr 172.21.15.45:0/1297397843 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:18.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b1d9b700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8b40c9cd0 0x7fd8b40c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.499+0000 7fd8b259c700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd89c003a00 0x7fd8b40c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8b2d9d700 1 -- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd89c003a00 msgr2=0x7fd8b40c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8b2d9d700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd89c003a00 0x7fd8b40c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8b2d9d700 1 -- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8b40c9cd0 msgr2=0x7fd8b40c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8b2d9d700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8b40c9cd0 0x7fd8b40c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8b2d9d700 1 -- 172.21.15.45:0/1297397843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b40c5940 con 0x7fd8b40c6e30 2023-08-27T11:28:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8b2d9d700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8b40c6e30 0x7fd8b40c7b00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4070e00 tx=0x7fd8ac011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8a2ffd700 1 -- 172.21.15.45:0/1297397843 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00e040 con 0x7fd8b40c6e30 2023-08-27T11:28:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8a2ffd700 1 -- 172.21.15.45:0/1297397843 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac00f040 con 0x7fd8b40c6e30 2023-08-27T11:28:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b40c5c20 con 0x7fd8b40c6e30 2023-08-27T11:28:18.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8a2ffd700 1 -- 172.21.15.45:0/1297397843 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00f940 con 0x7fd8b40c6e30 2023-08-27T11:28:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.500+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b40c4c50 con 0x7fd8b40c6e30 2023-08-27T11:28:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.501+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b40c4e70 con 0x7fd8b40c6e30 2023-08-27T11:28:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.502+0000 7fd8a2ffd700 1 -- 172.21.15.45:0/1297397843 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 38) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd8ac00faa0 con 0x7fd8b40c6e30 2023-08-27T11:28:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.502+0000 7fd8a2ffd700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd89406a200 0x7fd89406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:18.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.502+0000 7fd8a2ffd700 1 -- 172.21.15.45:0/1297397843 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fd8ac088d50 con 0x7fd8b40c6e30 2023-08-27T11:28:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.502+0000 7fd8b259c700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd89406a200 0x7fd89406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.503+0000 7fd8b259c700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd89406a200 0x7fd89406c6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8a801c040 tx=0x7fd8a8015000 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.507+0000 7fd8a2ffd700 1 -- 172.21.15.45:0/1297397843 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd8ac08c030 con 0x7fd8b40c6e30 2023-08-27T11:28:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.752+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd8b40676b0 con 0x7fd8b40c6e30 2023-08-27T11:28:18.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.753+0000 7fd8a2ffd700 1 -- 172.21.15.45:0/1297397843 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd8b40676b0 con 0x7fd8b40c6e30 2023-08-27T11:28:18.754 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:28:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd89406a200 msgr2=0x7fd89406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd89406a200 0x7fd89406c6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd8a801c040 tx=0x7fd8a8015000 comp rx=0 tx=0).stop 2023-08-27T11:28:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8b40c6e30 msgr2=0x7fd8b40c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8b40c6e30 0x7fd8b40c7b00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4070e00 tx=0x7fd8ac011f60 comp rx=0 tx=0).stop 2023-08-27T11:28:18.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 shutdown_connections 2023-08-27T11:28:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd89406a200 0x7fd89406c6c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8b40c6e30 0x7fd8b40c7b00 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd89c003a00 0x7fd8b40c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 --2- 172.21.15.45:0/1297397843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8b40c9cd0 0x7fd8b40c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:18.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 >> 172.21.15.45:0/1297397843 conn(0x7fd8b40cc7e0 msgr2=0x7fd8b40cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 shutdown_connections 2023-08-27T11:28:18.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:18.755+0000 7fd8b8ec4700 1 -- 172.21.15.45:0/1297397843 wait complete. 2023-08-27T11:28:18.768 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:28:18.769 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:28:18.769 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-08-27T11:28:18.770 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:28:18.770 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:18.770 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:28:18.771 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:28:18.771 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:28:18.771 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:28:18.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:18.772 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-08-27T11:28:18.790 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:28:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:28:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:28:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:28:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:18.791 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:18.791 INFO:teuthology.orchestra.run.smithi045.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:28:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-27T11:28:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41442sec preferred_lft 41442sec 2023-08-27T11:28:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-08-27T11:28:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:18.793 INFO:teuthology.orchestra.run.smithi045.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:18.793 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:28:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:18.793 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:28:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:28:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:28:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a41d:a1ff:fed9:3bf2/64 scope link 2023-08-27T11:28:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:18.795 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether a6:e4:29:97:ce:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:28:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a4e4:29ff:fe97:ced1/64 scope link 2023-08-27T11:28:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:18.795 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:28:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:28:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c84:f3ff:fe1a:a84/64 scope link 2023-08-27T11:28:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:28:18.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:18.797 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:28:18.856 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-08-27T11:28:18.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:18.859 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-08-27T11:28:18.917 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-08-27T11:28:18.918 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:28:18.919 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-27T11:28:18.919 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-27T11:28:18.919 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:28:18.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:18.920 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:28:18.920 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:28:18.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-27T11:28:18.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-27T11:28:18.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-27T11:28:18.922 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:28:19.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:19.025 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:28:19.042 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:28:19.042 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:28:19.042 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:28:19.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:19.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-08-27T11:28:19.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:19.110 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:19.166 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:28:19.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:28:19.167 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:19.223 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:28:19.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:19.224 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:19.310 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:19.310 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:28:19.310 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:28:19.319 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:19.356 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:28:19.356 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:28:19.356 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:28:19.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:28:19.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:28:19.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790" 2023-08-27T11:28:19.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2023-08-27T11:28:19.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8ea1ae34-1697-4b79-b2ce-3f741cd3c828" will pass to kernel 2023-08-27T11:28:19.377 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:28:19.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:19.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:19.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:19.407 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:28:19.490 INFO:teuthology.orchestra.run.smithi045.stdout:client7638 2023-08-27T11:28:19.490 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:28:19.491 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:19.547 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:28:19.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.713+0000 7ff7a7323700 1 Processor -- start 2023-08-27T11:28:19.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.713+0000 7ff7a7323700 1 -- start start 2023-08-27T11:28:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a7323700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7a015d340 0x7ff7a0160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a7323700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a00b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a7323700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a006b160 con 0x7ff7a00cd880 2023-08-27T11:28:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a7323700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a0072010 con 0x7ff7a015d340 2023-08-27T11:28:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a7323700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00af590 con 0x7ff7a006a990 2023-08-27T11:28:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a50bf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7a015d340 0x7ff7a0160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a50bf700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7a015d340 0x7ff7a0160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59278/0 (socket says 172.21.15.45:59278) 2023-08-27T11:28:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a50bf700 1 -- 172.21.15.45:0/3615030658 learned_addr learned my addr 172.21.15.45:0/3615030658 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a58c0700 1 --2- 172.21.15.45:0/3615030658 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a00b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a50bf700 1 -- 172.21.15.45:0/3615030658 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 msgr2=0x7ff7a00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a50bf700 1 --2- 172.21.15.45:0/3615030658 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a00b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a50bf700 1 -- 172.21.15.45:0/3615030658 >> v1:172.21.15.135:6789/0 conn(0x7ff7a00cd880 legacy=0x7ff7a006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:19.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a50bf700 1 -- 172.21.15.45:0/3615030658 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a00af2b0 con 0x7ff7a015d340 2023-08-27T11:28:19.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a58c0700 1 --2- 172.21.15.45:0/3615030658 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a00b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:19.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff7a50bf700 1 --2- 172.21.15.45:0/3615030658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a015d340 0x7ff7a0160d00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0098f0 tx=0x7ff79c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473c7d7a0bb302ef server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:19.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff797fff700 1 -- 172.21.15.45:0/3615030658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c004230 con 0x7ff7a015d340 2023-08-27T11:28:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff797fff700 1 -- 172.21.15.45:0/3615030658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c004390 con 0x7ff7a015d340 2023-08-27T11:28:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.714+0000 7ff797fff700 1 -- 172.21.15.45:0/3615030658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0036a0 con 0x7ff7a015d340 2023-08-27T11:28:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.715+0000 7ff7a7323700 1 -- 172.21.15.45:0/3615030658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a015d340 msgr2=0x7ff7a0160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.715+0000 7ff7a7323700 1 --2- 172.21.15.45:0/3615030658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a015d340 0x7ff7a0160d00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0098f0 tx=0x7ff79c009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.715+0000 7ff7a7323700 1 -- 172.21.15.45:0/3615030658 shutdown_connections 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.715+0000 7ff7a7323700 1 --2- 172.21.15.45:0/3615030658 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a00b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.715+0000 7ff7a7323700 1 --2- 172.21.15.45:0/3615030658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a015d340 0x7ff7a0160d00 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.715+0000 7ff7a7323700 1 -- 172.21.15.45:0/3615030658 >> 172.21.15.45:0/3615030658 conn(0x7ff7a00cc7c0 msgr2=0x7ff7a00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.715+0000 7ff7a7323700 1 -- 172.21.15.45:0/3615030658 shutdown_connections 2023-08-27T11:28:19.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.715+0000 7ff7a7323700 1 -- 172.21.15.45:0/3615030658 wait complete. 2023-08-27T11:28:19.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.717+0000 7ff7a7323700 1 Processor -- start 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.717+0000 7ff7a7323700 1 -- start start 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a7323700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a01f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a7323700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cd880 0x7ff7a014d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a7323700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7a015d340 0x7ff7a006ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a7323700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00b54c0 con 0x7ff7a00cd880 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a7323700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a014acf0 con 0x7ff7a015d340 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a7323700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a014df10 con 0x7ff7a006a990 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a48be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cd880 0x7ff7a014d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a48be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cd880 0x7ff7a014d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59286/0 (socket says 172.21.15.45:59286) 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a48be700 1 -- 172.21.15.45:0/1131637455 learned_addr learned my addr 172.21.15.45:0/1131637455 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a50bf700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a01f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a58c0700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7a015d340 0x7ff7a006ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a48be700 1 -- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 msgr2=0x7ff7a01f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a48be700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a01f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a48be700 1 -- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7a015d340 msgr2=0x7ff7a006ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a48be700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7a015d340 0x7ff7a006ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a48be700 1 -- 172.21.15.45:0/1131637455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c009550 con 0x7ff7a00cd880 2023-08-27T11:28:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a50bf700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a01f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a58c0700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7a015d340 0x7ff7a006ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a48be700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cd880 0x7ff7a014d9d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff79000b4a0 tx=0x7ff79000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff795ffb700 1 -- 172.21.15.45:0/1131637455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790011830 con 0x7ff7a00cd880 2023-08-27T11:28:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff795ffb700 1 -- 172.21.15.45:0/1131637455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790011990 con 0x7ff7a00cd880 2023-08-27T11:28:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a00b4840 con 0x7ff7a00cd880 2023-08-27T11:28:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.718+0000 7ff795ffb700 1 -- 172.21.15.45:0/1131637455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790011d40 con 0x7ff7a00cd880 2023-08-27T11:28:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.719+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a00b4e00 con 0x7ff7a00cd880 2023-08-27T11:28:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.719+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a00b5600 con 0x7ff7a00cd880 2023-08-27T11:28:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.720+0000 7ff795ffb700 1 -- 172.21.15.45:0/1131637455 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 38) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff79001a400 con 0x7ff7a00cd880 2023-08-27T11:28:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.720+0000 7ff795ffb700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff78406a200 0x7ff78406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.720+0000 7ff795ffb700 1 -- 172.21.15.45:0/1131637455 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7ff790088530 con 0x7ff7a00cd880 2023-08-27T11:28:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.720+0000 7ff7a50bf700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff78406a200 0x7ff78406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.721+0000 7ff7a50bf700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff78406a200 0x7ff78406c6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00b17f0 tx=0x7ff79c008030 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.725+0000 7ff795ffb700 1 -- 172.21.15.45:0/1131637455 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff7a00b5600 con 0x7ff7a00cd880 2023-08-27T11:28:19.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.955+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 --> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7a0071c30 con 0x7ff78406a200 2023-08-27T11:28:19.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.956+0000 7ff795ffb700 1 -- 172.21.15.45:0/1131637455 <== mgr.6709 v2:172.21.15.135:6832/3812189695 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff79c020070 con 0x7ff78406a200 2023-08-27T11:28:19.958 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:28:19.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:19.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephfs" 2023-08-27T11:28:19.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:19.958 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:28:19.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff78406a200 msgr2=0x7ff78406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff78406a200 0x7ff78406c6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00b17f0 tx=0x7ff79c008030 comp rx=0 tx=0).stop 2023-08-27T11:28:19.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cd880 msgr2=0x7ff7a014d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:19.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cd880 0x7ff7a014d9d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff79000b4a0 tx=0x7ff79000b860 comp rx=0 tx=0).stop 2023-08-27T11:28:19.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 shutdown_connections 2023-08-27T11:28:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7ff78406a200 0x7ff78406c6c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7a015d340 0x7ff7a006ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff7a006a990 0x7ff7a01f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 --2- 172.21.15.45:0/1131637455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7a00cd880 0x7ff7a014d9d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 >> 172.21.15.45:0/1131637455 conn(0x7ff7a00cc7c0 msgr2=0x7ff7a00afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 shutdown_connections 2023-08-27T11:28:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:19.959+0000 7ff7a7323700 1 -- 172.21.15.45:0/1131637455 wait complete. 2023-08-27T11:28:19.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-27T11:28:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.098+0000 7fd43fa54700 1 Processor -- start 2023-08-27T11:28:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.098+0000 7fd43fa54700 1 -- start start 2023-08-27T11:28:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd43fa54700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43815a000 0x7fd438071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd43fa54700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43815ea60 con 0x7fd438071e10 2023-08-27T11:28:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd43fa54700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43815dec0 con 0x7fd4380c5740 2023-08-27T11:28:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd43fa54700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43815b3f0 con 0x7fd43815a000 2023-08-27T11:28:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd43cfef700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd438071e10 0x7fd43815e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46070/0 (socket says 172.21.15.45:46070) 2023-08-27T11:28:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd43cfef700 1 -- 172.21.15.45:0/3431685979 learned_addr learned my addr 172.21.15.45:0/3431685979 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd43d7f0700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43815a000 0x7fd438071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963698765 0 0) 0x7fd434001110 con 0x7fd438071e10 2023-08-27T11:28:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4240035b0 con 0x7fd438071e10 2023-08-27T11:28:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 778934057 0 0) 0x7fd4240035b0 con 0x7fd438071e10 2023-08-27T11:28:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43815a000 msgr2=0x7fd438071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.099+0000 7fd42ffff700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43815a000 0x7fd438071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 >> v1:172.21.15.135:6789/0 conn(0x7fd4380c5740 legacy=0x7fd43806ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380c64a0 con 0x7fd438071e10 2023-08-27T11:28:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fd434001800 con 0x7fd438071e10 2023-08-27T11:28:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 >> v1:172.21.15.45:6789/0 conn(0x7fd438071e10 legacy=0x7fd43815e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd424003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43815a000 0x7fd424004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd424004d90 0x7fd424005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43815b3f0 con 0x7fd424003a30 2023-08-27T11:28:20.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42400ba10 con 0x7fd424004d90 2023-08-27T11:28:20.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42400bbd0 con 0x7fd43815a000 2023-08-27T11:28:20.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43d7f0700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd424003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43cfef700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43815a000 0x7fd424004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43dff1700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd424004d90 0x7fd424005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43d7f0700 1 -- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43815a000 msgr2=0x7fd424004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43d7f0700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43815a000 0x7fd424004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43d7f0700 1 -- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd424004d90 msgr2=0x7fd424005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43d7f0700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd424004d90 0x7fd424005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43d7f0700 1 -- 172.21.15.45:0/3431685979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4240035b0 con 0x7fd424003a30 2023-08-27T11:28:20.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43dff1700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd424004d90 0x7fd424005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:20.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd43d7f0700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd424003e10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd428009b60 tx=0x7fd424005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dd16847671a82c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42800e6c0 con 0x7fd424003a30 2023-08-27T11:28:20.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42800e820 con 0x7fd424003a30 2023-08-27T11:28:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.100+0000 7fd42ffff700 1 -- 172.21.15.45:0/3431685979 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42800eb90 con 0x7fd424003a30 2023-08-27T11:28:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.101+0000 7fd43fa54700 1 -- 172.21.15.45:0/3431685979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 msgr2=0x7fd424003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.101+0000 7fd43fa54700 1 --2- 172.21.15.45:0/3431685979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd424003e10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd428009b60 tx=0x7fd424005880 comp rx=0 tx=0).stop 2023-08-27T11:28:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.101+0000 7fd43dff1700 1 -- 172.21.15.45:0/3431685979 reap_dead start 2023-08-27T11:28:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.101+0000 7fd43fa54700 1 -- 172.21.15.45:0/3431685979 shutdown_connections 2023-08-27T11:28:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.101+0000 7fd43fa54700 1 -- 172.21.15.45:0/3431685979 >> 172.21.15.45:0/3431685979 conn(0x7fd4380cfdd0 msgr2=0x7fd4380d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.101+0000 7fd43fa54700 1 -- 172.21.15.45:0/3431685979 shutdown_connections 2023-08-27T11:28:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.101+0000 7fd43fa54700 1 -- 172.21.15.45:0/3431685979 wait complete. 2023-08-27T11:28:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.103+0000 7fd43fa54700 1 Processor -- start 2023-08-27T11:28:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.103+0000 7fd43fa54700 1 -- start start 2023-08-27T11:28:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43fa54700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd43815c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43fa54700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd43815cbd0 0x7fd4380ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43fa54700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43814aba0 0x7fd4380cef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43fa54700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4381453b0 con 0x7fd424003a30 2023-08-27T11:28:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43fa54700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd438156de0 con 0x7fd43815cbd0 2023-08-27T11:28:20.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43fa54700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd438156260 con 0x7fd43814aba0 2023-08-27T11:28:20.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43d7f0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd43815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43d7f0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd43815c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59304/0 (socket says 172.21.15.45:59304) 2023-08-27T11:28:20.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43d7f0700 1 -- 172.21.15.45:0/1588109097 learned_addr learned my addr 172.21.15.45:0/1588109097 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:20.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43d7f0700 1 -- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43814aba0 msgr2=0x7fd4380cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43d7f0700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43814aba0 0x7fd4380cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43d7f0700 1 -- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd43815cbd0 msgr2=0x7fd4380ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:20.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43d7f0700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd43815cbd0 0x7fd4380ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:20.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43d7f0700 1 -- 172.21.15.45:0/1588109097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd438156540 con 0x7fd424003a30 2023-08-27T11:28:20.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43d7f0700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd43815c690 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd4280090b0 tx=0x7fd4280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd42dffb700 1 -- 172.21.15.45:0/1588109097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428021070 con 0x7fd424003a30 2023-08-27T11:28:20.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd42dffb700 1 -- 172.21.15.45:0/1588109097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd428009ed0 con 0x7fd424003a30 2023-08-27T11:28:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd438156820 con 0x7fd424003a30 2023-08-27T11:28:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.104+0000 7fd42dffb700 1 -- 172.21.15.45:0/1588109097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42800e6c0 con 0x7fd424003a30 2023-08-27T11:28:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.105+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd438155ea0 con 0x7fd424003a30 2023-08-27T11:28:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.106+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd438148c60 con 0x7fd424003a30 2023-08-27T11:28:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.106+0000 7fd42dffb700 1 -- 172.21.15.45:0/1588109097 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 38) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd428025070 con 0x7fd424003a30 2023-08-27T11:28:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.106+0000 7fd42dffb700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd41c06a200 0x7fd41c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.106+0000 7fd42dffb700 1 -- 172.21.15.45:0/1588109097 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fd428089990 con 0x7fd424003a30 2023-08-27T11:28:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.107+0000 7fd43cfef700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd41c06a200 0x7fd41c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.107+0000 7fd43cfef700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd41c06a200 0x7fd41c06c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd434001390 tx=0x7fd434001110 comp rx=0 tx=0).ready entity=mgr.6709 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.111+0000 7fd42dffb700 1 -- 172.21.15.45:0/1588109097 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd428054d40 con 0x7fd424003a30 2023-08-27T11:28:20.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:20.372+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd4380727c0 con 0x7fd424003a30 2023-08-27T11:28:21.376 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.374+0000 7fd42dffb700 1 -- 172.21.15.45:0/1588109097 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 39) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fd428009700 con 0x7fd424003a30 2023-08-27T11:28:21.377 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.376+0000 7fd42dffb700 1 -- 172.21.15.45:0/1588109097 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v39) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd4280599a0 con 0x7fd424003a30 2023-08-27T11:28:21.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.378+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd41c06a200 msgr2=0x7fd41c06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.378+0000 7fd43fa54700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd41c06a200 0x7fd41c06c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd434001390 tx=0x7fd434001110 comp rx=0 tx=0).stop 2023-08-27T11:28:21.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.378+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 msgr2=0x7fd43815c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.378+0000 7fd43fa54700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd43815c690 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd4280090b0 tx=0x7fd4280047c0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.379+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 shutdown_connections 2023-08-27T11:28:21.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.379+0000 7fd43fa54700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7fd41c06a200 0x7fd41c06c6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.379+0000 7fd43fa54700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd424003a30 0x7fd43815c690 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.379+0000 7fd43fa54700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd43814aba0 0x7fd4380cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.379+0000 7fd43fa54700 1 --2- 172.21.15.45:0/1588109097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd43815cbd0 0x7fd4380ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.379+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 >> 172.21.15.45:0/1588109097 conn(0x7fd4380cfdd0 msgr2=0x7fd4380b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:21.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.379+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 shutdown_connections 2023-08-27T11:28:21.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.379+0000 7fd43fa54700 1 -- 172.21.15.45:0/1588109097 wait complete. 2023-08-27T11:28:21.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-27T11:28:21.513 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:21.512+0000 7fa2af533200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:28:21.526 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:21.526+0000 7f11b25cf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:28:21.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.532+0000 7f1f6707d700 1 Processor -- start 2023-08-27T11:28:21.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.532+0000 7f1f6707d700 1 -- start start 2023-08-27T11:28:21.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.533+0000 7f1f6707d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 0x7f1f600cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.533+0000 7f1f6707d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1f600cd300 0x7f1f6015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.533+0000 7f1f6707d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1f6015e030 0x7f1f600cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.533+0000 7f1f6707d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600cfae0 con 0x7f1f6015e030 2023-08-27T11:28:21.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.533+0000 7f1f6707d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6015e900 con 0x7f1f600cd300 2023-08-27T11:28:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.533+0000 7f1f6707d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600cae60 con 0x7f1f6015a0f0 2023-08-27T11:28:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.533+0000 7f1f6561a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1f6015e030 0x7f1f600cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.533+0000 7f1f6561a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1f6015e030 0x7f1f600cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59342/0 (socket says 172.21.15.45:59342) 2023-08-27T11:28:21.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.533+0000 7f1f6561a700 1 -- 172.21.15.45:0/1771172821 learned_addr learned my addr 172.21.15.45:0/1771172821 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:21.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f64e19700 1 --2- 172.21.15.45:0/1771172821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 0x7f1f600cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f5ffff700 1 --2- 172.21.15.45:0/1771172821 >> v2:172.21.15.135:3300/0 conn(0x7f1f600cd300 0x7f1f6015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f6561a700 1 -- 172.21.15.45:0/1771172821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 msgr2=0x7f1f600cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f6561a700 1 --2- 172.21.15.45:0/1771172821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 0x7f1f600cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f6561a700 1 -- 172.21.15.45:0/1771172821 >> v2:172.21.15.135:3300/0 conn(0x7f1f600cd300 msgr2=0x7f1f6015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f6561a700 1 --2- 172.21.15.45:0/1771172821 >> v2:172.21.15.135:3300/0 conn(0x7f1f600cd300 0x7f1f6015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f6561a700 1 -- 172.21.15.45:0/1771172821 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600cb140 con 0x7f1f6015e030 2023-08-27T11:28:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f6561a700 1 --2- 172.21.15.45:0/1771172821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f6015e030 0x7f1f600cd930 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f1f6006c400 tx=0x7f1f5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5439d454705ffaeb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f5f7fe700 1 -- 172.21.15.45:0/1771172821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800e070 con 0x7f1f6015e030 2023-08-27T11:28:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f5f7fe700 1 -- 172.21.15.45:0/1771172821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5800b910 con 0x7f1f6015e030 2023-08-27T11:28:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.534+0000 7f1f5f7fe700 1 -- 172.21.15.45:0/1771172821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800bcc0 con 0x7f1f6015e030 2023-08-27T11:28:21.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.535+0000 7f1f6707d700 1 -- 172.21.15.45:0/1771172821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f6015e030 msgr2=0x7f1f600cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.535+0000 7f1f6707d700 1 --2- 172.21.15.45:0/1771172821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f6015e030 0x7f1f600cd930 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f1f6006c400 tx=0x7f1f5800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.535+0000 7f1f6707d700 1 -- 172.21.15.45:0/1771172821 shutdown_connections 2023-08-27T11:28:21.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.535+0000 7f1f6707d700 1 --2- 172.21.15.45:0/1771172821 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 0x7f1f600cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.535+0000 7f1f6707d700 1 --2- 172.21.15.45:0/1771172821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f6015e030 0x7f1f600cd930 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.535+0000 7f1f6707d700 1 --2- 172.21.15.45:0/1771172821 >> v2:172.21.15.135:3300/0 conn(0x7f1f600cd300 0x7f1f6015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.535+0000 7f1f6707d700 1 -- 172.21.15.45:0/1771172821 >> 172.21.15.45:0/1771172821 conn(0x7f1f600cc7e0 msgr2=0x7f1f600ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.535+0000 7f1f6707d700 1 -- 172.21.15.45:0/1771172821 shutdown_connections 2023-08-27T11:28:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.535+0000 7f1f6707d700 1 -- 172.21.15.45:0/1771172821 wait complete. 2023-08-27T11:28:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.536+0000 7f1f6707d700 1 Processor -- start 2023-08-27T11:28:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.536+0000 7f1f6707d700 1 -- start start 2023-08-27T11:28:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f6707d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cd300 0x7f1f6014e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f6707d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 0x7f1f60151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f6707d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6015e030 0x7f1f6014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.549 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f6707d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6014ef50 con 0x7f1f600cd300 2023-08-27T11:28:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f6707d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6014f230 con 0x7f1f6015e030 2023-08-27T11:28:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f6707d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6014f510 con 0x7f1f6015a0f0 2023-08-27T11:28:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f64e19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cd300 0x7f1f6014e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f64e19700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cd300 0x7f1f6014e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59344/0 (socket says 172.21.15.45:59344) 2023-08-27T11:28:21.550 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f64e19700 1 -- 172.21.15.45:0/3934576469 learned_addr learned my addr 172.21.15.45:0/3934576469 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f5ffff700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 0x7f1f60151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f6561a700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6015e030 0x7f1f6014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f64e19700 1 -- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 msgr2=0x7f1f60151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f64e19700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 0x7f1f60151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f64e19700 1 -- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6015e030 msgr2=0x7f1f6014dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f64e19700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6015e030 0x7f1f6014dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.537+0000 7f1f64e19700 1 -- 172.21.15.45:0/3934576469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f58009b30 con 0x7f1f600cd300 2023-08-27T11:28:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.538+0000 7f1f64e19700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cd300 0x7f1f6014e850 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400e870 tx=0x7f1f5400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.538+0000 7f1f5d7fa700 1 -- 172.21.15.45:0/3934576469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54009590 con 0x7f1f600cd300 2023-08-27T11:28:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.538+0000 7f1f5d7fa700 1 -- 172.21.15.45:0/3934576469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f54013680 con 0x7f1f600cd300 2023-08-27T11:28:21.552 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.538+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6014f7f0 con 0x7f1f600cd300 2023-08-27T11:28:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.538+0000 7f1f5d7fa700 1 -- 172.21.15.45:0/3934576469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f540139f0 con 0x7f1f600cd300 2023-08-27T11:28:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.538+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6014e4e0 con 0x7f1f600cd300 2023-08-27T11:28:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.539+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6014ff70 con 0x7f1f600cd300 2023-08-27T11:28:21.553 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.540+0000 7f1f5d7fa700 1 -- 172.21.15.45:0/3934576469 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 39) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f1f54013b50 con 0x7f1f600cd300 2023-08-27T11:28:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.540+0000 7f1f5d7fa700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f1f4406a250 0x7f1f4406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.540+0000 7f1f5ffff700 1 -- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f1f4406a250 msgr2=0x7f1f4406c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3812189695 2023-08-27T11:28:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.540+0000 7f1f5d7fa700 1 -- 172.21.15.45:0/3934576469 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f1f54088e60 con 0x7f1f600cd300 2023-08-27T11:28:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.540+0000 7f1f5ffff700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f1f4406a250 0x7f1f4406c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.545+0000 7f1f5d7fa700 1 -- 172.21.15.45:0/3934576469 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1f540b5990 con 0x7f1f600cd300 2023-08-27T11:28:21.580 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:21.578+0000 7fa2af533200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:28:21.599 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:21.598+0000 7f11b25cf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:28:21.650 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:21.649+0000 7fa2af533200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:28:21.672 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:21.672+0000 7f11b25cf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:28:21.727 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:21.726+0000 7fa2af533200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:28:21.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.741+0000 7f1f5ffff700 1 -- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f1f4406a250 msgr2=0x7f1f4406c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3812189695 2023-08-27T11:28:21.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.741+0000 7f1f5ffff700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f1f4406a250 0x7f1f4406c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.747+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1f600726d0 con 0x7f1f600cd300 2023-08-27T11:28:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.750+0000 7f1f5d7fa700 1 -- 172.21.15.45:0/3934576469 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1f540170d0 con 0x7f1f600cd300 2023-08-27T11:28:21.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.752+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f1f4406a250 msgr2=0x7f1f4406c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:21.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.752+0000 7f1f6707d700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f1f4406a250 0x7f1f4406c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.752+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cd300 msgr2=0x7f1f6014e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.752+0000 7f1f6707d700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cd300 0x7f1f6014e850 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400e870 tx=0x7f1f5400eb80 comp rx=0 tx=0).stop 2023-08-27T11:28:21.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.753+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 shutdown_connections 2023-08-27T11:28:21.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.753+0000 7f1f6707d700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f1f4406a250 0x7f1f4406c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.753+0000 7f1f6707d700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f600cd300 0x7f1f6014e850 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.753+0000 7f1f6707d700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6015e030 0x7f1f6014dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.753+0000 7f1f6707d700 1 --2- 172.21.15.45:0/3934576469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f1f6015a0f0 0x7f1f60151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.753+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 >> 172.21.15.45:0/3934576469 conn(0x7f1f600cc7e0 msgr2=0x7f1f600ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:21.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.753+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 shutdown_connections 2023-08-27T11:28:21.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.753+0000 7f1f6707d700 1 -- 172.21.15.45:0/3934576469 wait complete. 2023-08-27T11:28:21.757 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:21.754+0000 7f11b25cf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:28:21.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-27T11:28:21.869 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:21.867+0000 7fa2af533200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:28:21.896 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.895+0000 7f3dfb238700 1 Processor -- start 2023-08-27T11:28:21.897 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.895+0000 7f3dfb238700 1 -- start start 2023-08-27T11:28:21.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.896+0000 7f3dfb238700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3df415d3a0 0x7f3df40d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.896+0000 7f3dfb238700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df415d780 con 0x7f3df406c4b0 2023-08-27T11:28:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.896+0000 7f3dfb238700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df40cfe30 con 0x7f3df40d02c0 2023-08-27T11:28:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.896+0000 7f3dfb238700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df40cf220 con 0x7f3df415d3a0 2023-08-27T11:28:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.896+0000 7f3df8fd4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3df415d3a0 0x7f3df40d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.896+0000 7f3df8fd4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3df415d3a0 0x7f3df40d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59398/0 (socket says 172.21.15.45:59398) 2023-08-27T11:28:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.896+0000 7f3df8fd4700 1 -- 172.21.15.45:0/325949548 learned_addr learned my addr 172.21.15.45:0/325949548 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.896+0000 7f3df97d5700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3df406c4b0 0x7f3df40cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46178/0 (socket says 172.21.15.45:46178) 2023-08-27T11:28:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.897+0000 7f3df8fd4700 1 -- 172.21.15.45:0/325949548 >> v1:172.21.15.135:6789/0 conn(0x7f3df40d02c0 legacy=0x7f3df406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.897+0000 7f3df8fd4700 1 -- 172.21.15.45:0/325949548 >> v1:172.21.15.45:6789/0 conn(0x7f3df406c4b0 legacy=0x7f3df40cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.897+0000 7f3df8fd4700 1 -- 172.21.15.45:0/325949548 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40cf500 con 0x7f3df415d3a0 2023-08-27T11:28:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.897+0000 7f3df8fd4700 1 --2- 172.21.15.45:0/325949548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 0x7f3df40d1550 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3de80098f0 tx=0x7f3de8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f5d930c95e39 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.897+0000 7f3df37fe700 1 -- 172.21.15.45:0/325949548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de800b8a0 con 0x7f3df415d3a0 2023-08-27T11:28:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.897+0000 7f3df37fe700 1 -- 172.21.15.45:0/325949548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de800ba00 con 0x7f3df415d3a0 2023-08-27T11:28:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.897+0000 7f3df37fe700 1 -- 172.21.15.45:0/325949548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de800bd90 con 0x7f3df415d3a0 2023-08-27T11:28:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.898+0000 7f3dfb238700 1 -- 172.21.15.45:0/325949548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 msgr2=0x7f3df40d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.898+0000 7f3dfb238700 1 --2- 172.21.15.45:0/325949548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 0x7f3df40d1550 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3de80098f0 tx=0x7f3de8009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.898+0000 7f3dfb238700 1 -- 172.21.15.45:0/325949548 shutdown_connections 2023-08-27T11:28:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.898+0000 7f3dfb238700 1 --2- 172.21.15.45:0/325949548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 0x7f3df40d1550 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.898+0000 7f3dfb238700 1 -- 172.21.15.45:0/325949548 >> 172.21.15.45:0/325949548 conn(0x7f3df409a1a0 msgr2=0x7f3df40d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.898+0000 7f3dfb238700 1 -- 172.21.15.45:0/325949548 shutdown_connections 2023-08-27T11:28:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.898+0000 7f3dfb238700 1 -- 172.21.15.45:0/325949548 wait complete. 2023-08-27T11:28:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.899+0000 7f3dfb238700 1 Processor -- start 2023-08-27T11:28:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.899+0000 7f3dfb238700 1 -- start start 2023-08-27T11:28:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3dfb238700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3df406c4b0 0x7f3df4157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3dfb238700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3df40d02c0 0x7f3df40b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3dfb238700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 0x7f3df40b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3dfb238700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40b3630 con 0x7f3df415d3a0 2023-08-27T11:28:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3dfb238700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41e8a40 con 0x7f3df40d02c0 2023-08-27T11:28:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3dfb238700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41e8110 con 0x7f3df406c4b0 2023-08-27T11:28:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df97d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 0x7f3df40b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df97d5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 0x7f3df40b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59408/0 (socket says 172.21.15.45:59408) 2023-08-27T11:28:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df97d5700 1 -- 172.21.15.45:0/4180352215 learned_addr learned my addr 172.21.15.45:0/4180352215 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df3fff700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3df40d02c0 0x7f3df40b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df8fd4700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3df406c4b0 0x7f3df4157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df97d5700 1 -- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3df406c4b0 msgr2=0x7f3df4157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df97d5700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3df406c4b0 0x7f3df4157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df97d5700 1 -- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3df40d02c0 msgr2=0x7f3df40b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df97d5700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3df40d02c0 0x7f3df40b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df97d5700 1 -- 172.21.15.45:0/4180352215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8009550 con 0x7f3df415d3a0 2023-08-27T11:28:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.900+0000 7f3df97d5700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 0x7f3df40b3a60 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00cbd0 tx=0x7f3dec00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.901+0000 7f3df17fa700 1 -- 172.21.15.45:0/4180352215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec00aaa0 con 0x7f3df415d3a0 2023-08-27T11:28:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.901+0000 7f3df17fa700 1 -- 172.21.15.45:0/4180352215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec00ac00 con 0x7f3df415d3a0 2023-08-27T11:28:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.901+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df41e83f0 con 0x7f3df415d3a0 2023-08-27T11:28:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.901+0000 7f3df17fa700 1 -- 172.21.15.45:0/4180352215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec007070 con 0x7f3df415d3a0 2023-08-27T11:28:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.901+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df40b3fa0 con 0x7f3df415d3a0 2023-08-27T11:28:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.902+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df41e8bb0 con 0x7f3df415d3a0 2023-08-27T11:28:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.902+0000 7f3df17fa700 1 -- 172.21.15.45:0/4180352215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 39) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f3dec0071d0 con 0x7f3df415d3a0 2023-08-27T11:28:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.903+0000 7f3df17fa700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f3dd806a250 0x7f3dd806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.903+0000 7f3df8fd4700 1 -- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f3dd806a250 msgr2=0x7f3dd806c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3812189695 2023-08-27T11:28:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.903+0000 7f3df17fa700 1 -- 172.21.15.45:0/4180352215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f3dec0882b0 con 0x7f3df415d3a0 2023-08-27T11:28:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.903+0000 7f3df8fd4700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f3dd806a250 0x7f3dd806c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:21.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:21.907+0000 7f3df17fa700 1 -- 172.21.15.45:0/4180352215 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3dec053830 con 0x7f3df415d3a0 2023-08-27T11:28:21.913 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:21.908+0000 7f11b25cf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:28:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.104+0000 7f3df8fd4700 1 -- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f3dd806a250 msgr2=0x7f3dd806c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3812189695 2023-08-27T11:28:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.104+0000 7f3df8fd4700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f3dd806a250 0x7f3dd806c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.109+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f3df415d780 con 0x7f3df415d3a0 2023-08-27T11:28:22.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.113+0000 7f3df17fa700 1 -- 172.21.15.45:0/4180352215 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3df415d780 con 0x7f3df415d3a0 2023-08-27T11:28:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f3dd806a250 msgr2=0x7f3dd806c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f3dd806a250 0x7f3dd806c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 msgr2=0x7f3df40b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:22.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 0x7f3df40b3a60 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00cbd0 tx=0x7f3dec00cee0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 shutdown_connections 2023-08-27T11:28:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f3dd806a250 0x7f3dd806c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3df415d3a0 0x7f3df40b3a60 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3df406c4b0 0x7f3df4157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 --2- 172.21.15.45:0/4180352215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3df40d02c0 0x7f3df40b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 >> 172.21.15.45:0/4180352215 conn(0x7f3df409a1a0 msgr2=0x7f3df415b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 shutdown_connections 2023-08-27T11:28:22.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.115+0000 7f3dfb238700 1 -- 172.21.15.45:0/4180352215 wait complete. 2023-08-27T11:28:22.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-08-27T11:28:22.193 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:22.192+0000 7fa2af533200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:28:22.255 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:22.255+0000 7f11b25cf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:28:22.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.258+0000 7f2420ca8700 1 Processor -- start 2023-08-27T11:28:22.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.258+0000 7f2420ca8700 1 -- start start 2023-08-27T11:28:22.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f2420ca8700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f241c15b160 0x7f241c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:22.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f2420ca8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f241c0c33f0 0x7f241c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f2420ca8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c06f840 con 0x7f241c06a7c0 2023-08-27T11:28:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f2420ca8700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c06ac50 con 0x7f241c15b160 2023-08-27T11:28:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f2420ca8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0ca4e0 con 0x7f241c0c33f0 2023-08-27T11:28:22.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f241a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f241c15b160 0x7f241c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:22.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f241a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f241c15b160 0x7f241c06d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59458/0 (socket says 172.21.15.45:59458) 2023-08-27T11:28:22.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f241a59c700 1 -- 172.21.15.45:0/3869817525 learned_addr learned my addr 172.21.15.45:0/3869817525 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:22.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f2419d9b700 1 --2- 172.21.15.45:0/3869817525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f241c0c33f0 0x7f241c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:22.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f241a59c700 1 -- 172.21.15.45:0/3869817525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f241c0c33f0 msgr2=0x7f241c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:22.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f241a59c700 1 --2- 172.21.15.45:0/3869817525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f241c0c33f0 0x7f241c06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f241a59c700 1 -- 172.21.15.45:0/3869817525 >> v1:172.21.15.135:6789/0 conn(0x7f241c06a7c0 legacy=0x7f241c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:22.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.259+0000 7f241a59c700 1 -- 172.21.15.45:0/3869817525 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c06f280 con 0x7f241c15b160 2023-08-27T11:28:22.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.260+0000 7f241a59c700 1 --2- 172.21.15.45:0/3869817525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f241c15b160 0x7f241c06d900 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f240c0098f0 tx=0x7f240c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8603095a4ade9bd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:22.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.260+0000 7f241959a700 1 -- 172.21.15.45:0/3869817525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c011830 con 0x7f241c15b160 2023-08-27T11:28:22.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.260+0000 7f241959a700 1 -- 172.21.15.45:0/3869817525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c011990 con 0x7f241c15b160 2023-08-27T11:28:22.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.260+0000 7f241959a700 1 -- 172.21.15.45:0/3869817525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c011d20 con 0x7f241c15b160 2023-08-27T11:28:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.260+0000 7f2420ca8700 1 -- 172.21.15.45:0/3869817525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f241c15b160 msgr2=0x7f241c06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.260+0000 7f2420ca8700 1 --2- 172.21.15.45:0/3869817525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f241c15b160 0x7f241c06d900 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f240c0098f0 tx=0x7f240c009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.261+0000 7f2420ca8700 1 -- 172.21.15.45:0/3869817525 shutdown_connections 2023-08-27T11:28:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.261+0000 7f2420ca8700 1 --2- 172.21.15.45:0/3869817525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f241c15b160 0x7f241c06d900 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.261+0000 7f2420ca8700 1 --2- 172.21.15.45:0/3869817525 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f241c0c33f0 0x7f241c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.261+0000 7f2420ca8700 1 -- 172.21.15.45:0/3869817525 >> 172.21.15.45:0/3869817525 conn(0x7f241c0cc7e0 msgr2=0x7f241c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:22.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.261+0000 7f2420ca8700 1 -- 172.21.15.45:0/3869817525 shutdown_connections 2023-08-27T11:28:22.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.261+0000 7f2420ca8700 1 -- 172.21.15.45:0/3869817525 wait complete. 2023-08-27T11:28:22.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.262+0000 7f2420ca8700 1 Processor -- start 2023-08-27T11:28:22.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.262+0000 7f2420ca8700 1 -- start start 2023-08-27T11:28:22.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f2420ca8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f241c06a7c0 0x7f241c1f0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:22.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f2420ca8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c0c33f0 0x7f241c0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:22.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f2420ca8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f241c15b160 0x7f241c14e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f2420ca8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0c9d10 con 0x7f241c15b160 2023-08-27T11:28:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f2420ca8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c06e160 con 0x7f241c0c33f0 2023-08-27T11:28:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f2420ca8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c1f0d90 con 0x7f241c06a7c0 2023-08-27T11:28:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f241c15b160 0x7f241c14e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f241c15b160 0x7f241c14e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59468/0 (socket says 172.21.15.45:59468) 2023-08-27T11:28:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241ad9d700 1 -- 172.21.15.45:0/1126084161 learned_addr learned my addr 172.21.15.45:0/1126084161 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241a59c700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f241c06a7c0 0x7f241c1f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f2419d9b700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c0c33f0 0x7f241c0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241ad9d700 1 -- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f241c06a7c0 msgr2=0x7f241c1f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241ad9d700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f241c06a7c0 0x7f241c1f0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241ad9d700 1 -- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c0c33f0 msgr2=0x7f241c0c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241ad9d700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f241c0c33f0 0x7f241c0c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241ad9d700 1 -- 172.21.15.45:0/1126084161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c009550 con 0x7f241c15b160 2023-08-27T11:28:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.263+0000 7f241ad9d700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f241c15b160 0x7f241c14e1c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f241400cc50 tx=0x7f241400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.264+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414003c00 con 0x7f241c15b160 2023-08-27T11:28:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.264+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2414003d60 con 0x7f241c15b160 2023-08-27T11:28:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.264+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400aab0 con 0x7f241c15b160 2023-08-27T11:28:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.264+0000 7f2420ca8700 1 -- 172.21.15.45:0/1126084161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c1f1070 con 0x7f241c15b160 2023-08-27T11:28:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.264+0000 7f2420ca8700 1 -- 172.21.15.45:0/1126084161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c1f1630 con 0x7f241c15b160 2023-08-27T11:28:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.265+0000 7f2420ca8700 1 -- 172.21.15.45:0/1126084161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c0726d0 con 0x7f241c15b160 2023-08-27T11:28:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.265+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 39) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f241400ac10 con 0x7f241c15b160 2023-08-27T11:28:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.266+0000 7f240affd700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 0x7f23fc06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.266+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f24140892c0 con 0x7f241c15b160 2023-08-27T11:28:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.266+0000 7f241a59c700 1 -- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 msgr2=0x7f23fc06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3812189695 2023-08-27T11:28:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.266+0000 7f241a59c700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 0x7f23fc06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.270+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f24140547e0 con 0x7f241c15b160 2023-08-27T11:28:22.309 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:22.308+0000 7fa2af533200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:28:22.380 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:22.380+0000 7f11b25cf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:28:22.382 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:22.381+0000 7fa2af533200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:28:22.457 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:22.456+0000 7f11b25cf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:28:22.462 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:22.461+0000 7fa2af533200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:28:22.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.467+0000 7f241a59c700 1 -- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 msgr2=0x7f23fc06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3812189695 2023-08-27T11:28:22.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.467+0000 7f241a59c700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 0x7f23fc06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.494+0000 7f2420ca8700 1 -- 172.21.15.45:0/1126084161 --> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f241c0ca780 con 0x7f23fc06a250 2023-08-27T11:28:22.541 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:22.541+0000 7f11b25cf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:28:22.831 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:22.830+0000 7fa2af533200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:28:22.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.868+0000 7f241a59c700 1 -- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 msgr2=0x7f23fc06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3812189695 2023-08-27T11:28:22.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:22.868+0000 7f241a59c700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 0x7f23fc06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:28:22.930 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:22.930+0000 7f11b25cf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:28:23.399 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:23.398+0000 7fa2af533200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:28:23.468 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:23.467+0000 7fa2af533200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:28:23.540 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:23.539+0000 7f11b25cf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:28:23.613 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:23.612+0000 7f11b25cf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:28:23.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:23.669+0000 7f241a59c700 1 -- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 msgr2=0x7f23fc06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3812189695 2023-08-27T11:28:23.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:23.669+0000 7f241a59c700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 0x7f23fc06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:28:24.096 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:24.095+0000 7fa2af533200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:28:24.173 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:24.172+0000 7fa2af533200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:28:24.277 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:24.276+0000 7f11b25cf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:28:24.316 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:24.315+0000 7fa2af533200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:28:24.359 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:24.358+0000 7f11b25cf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:28:24.509 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:24.509+0000 7f11b25cf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:28:24.542 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:24.541+0000 7fa2af533200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:28:24.615 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:24.614+0000 7fa2af533200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:28:24.749 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:24.748+0000 7f11b25cf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:28:24.827 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:24.826+0000 7f11b25cf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:28:24.879 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:24.878+0000 7fa2af533200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:28:25.098 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:25.097+0000 7fa2af533200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:28:25.104 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:25.103+0000 7f11b25cf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:28:25.244 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:25.243+0000 7fa2af533200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:28:25.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:25.270+0000 7f241a59c700 1 -- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 msgr2=0x7f23fc06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3812189695 2023-08-27T11:28:25.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:25.270+0000 7f241a59c700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 0x7f23fc06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-27T11:28:25.336 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:25.335+0000 7f11b25cf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:28:25.358 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:25.357+0000 7fa2af533200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:28:25.492 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:25.491+0000 7f11b25cf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:28:25.514 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:25.512+0000 7fa2af533200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:28:25.611 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:25.611+0000 7f11b25cf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:28:25.774 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:25.773+0000 7f11b25cf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:28:25.775 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:25.774+0000 7fa2af533200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:28:25.937 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:25.935+0000 7fa2af533200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:28:26.047 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:26.047+0000 7f11b25cf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:28:26.216 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:26.216+0000 7f11b25cf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:28:26.460 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.459+0000 7fa2af533200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:28:26.481 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:26.481+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 40) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f2414054bb0 con 0x7f241c15b160 2023-08-27T11:28:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:26.481+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 msgr2=0x7f23fc06c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:26.482 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:26.481+0000 7f240affd700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3812189695,v1:172.21.15.135:6833/3812189695] conn(0x7f23fc06a250 0x7f23fc06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:26.594 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa222591700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.595 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa222591700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.595 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa222591700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.595 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa222591700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.596 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa222591700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.596 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa225597700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.596 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa225597700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.597 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa225597700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.597 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa225597700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.597 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:26.592+0000 7fa225597700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:26.786 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:26.786+0000 7f11b25cf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:28:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.484+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 41) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f241401a360 con 0x7f241c15b160 2023-08-27T11:28:27.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.485+0000 7f240affd700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f23fc0353a0 0x7f23fc074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.486 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.485+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 --> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f23fc035780 con 0x7f23fc0353a0 2023-08-27T11:28:27.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.487+0000 7f241a59c700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f23fc0353a0 0x7f23fc074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.488+0000 7f241a59c700 1 --2- 172.21.15.45:0/1126084161 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f23fc0353a0 0x7f23fc074da0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f240c014040 tx=0x7f240c008030 comp rx=0 tx=0).ready entity=mgr.7084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.497 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:27.496+0000 7fa22c5e5700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-08-27T11:28:27.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.497+0000 7f240affd700 1 -- 172.21.15.45:0/1126084161 <== mgr.7084 v2:172.21.15.135:6832/3112377367 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f23fc035780 con 0x7f23fc0353a0 2023-08-27T11:28:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-08-27T11:28:27.503 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-08-27T11:28:27.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-27T11:28:27.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.640+0000 7f991e4a3700 1 Processor -- start 2023-08-27T11:28:27.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.640+0000 7f991e4a3700 1 -- start start 2023-08-27T11:28:27.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.641+0000 7f991e4a3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9918160d00 0x7f991806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.641+0000 7f991e4a3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f991806a7c0 0x7f99180b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.641+0000 7f991e4a3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991806b160 con 0x7f991815d340 2023-08-27T11:28:27.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.641+0000 7f991e4a3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9918072010 con 0x7f991806a7c0 2023-08-27T11:28:27.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.641+0000 7f991e4a3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991806ac50 con 0x7f9918160d00 2023-08-27T11:28:27.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.641+0000 7f991ca40700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f991806a7c0 0x7f99180b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.641+0000 7f991ca40700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f991806a7c0 0x7f99180b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59552/0 (socket says 172.21.15.45:59552) 2023-08-27T11:28:27.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.641+0000 7f991ca40700 1 -- 172.21.15.45:0/3316862240 learned_addr learned my addr 172.21.15.45:0/3316862240 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:27.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f99177fe700 1 --2- 172.21.15.45:0/3316862240 >> v2:172.21.15.135:3300/0 conn(0x7f9918160d00 0x7f991806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f991ca40700 1 -- 172.21.15.45:0/3316862240 >> v2:172.21.15.135:3300/0 conn(0x7f9918160d00 msgr2=0x7f991806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:27.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f991ca40700 1 --2- 172.21.15.45:0/3316862240 >> v2:172.21.15.135:3300/0 conn(0x7f9918160d00 0x7f991806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f991ca40700 1 -- 172.21.15.45:0/3316862240 >> v1:172.21.15.135:6789/0 conn(0x7f991815d340 legacy=0x7f99180cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:27.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f991ca40700 1 -- 172.21.15.45:0/3316862240 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180cdf90 con 0x7f991806a7c0 2023-08-27T11:28:27.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f99177fe700 1 --2- 172.21.15.45:0/3316862240 >> v2:172.21.15.135:3300/0 conn(0x7f9918160d00 0x7f991806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:27.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f991ca40700 1 --2- 172.21.15.45:0/3316862240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 0x7f99180b18a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f99180cd7f0 tx=0x7f991000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc6008aa385b589 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f9916ffd700 1 -- 172.21.15.45:0/3316862240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991000e070 con 0x7f991806a7c0 2023-08-27T11:28:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f9916ffd700 1 -- 172.21.15.45:0/3316862240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9910003a20 con 0x7f991806a7c0 2023-08-27T11:28:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f9916ffd700 1 -- 172.21.15.45:0/3316862240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910003dd0 con 0x7f991806a7c0 2023-08-27T11:28:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f991e4a3700 1 -- 172.21.15.45:0/3316862240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 msgr2=0x7f99180b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.642+0000 7f991e4a3700 1 --2- 172.21.15.45:0/3316862240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 0x7f99180b18a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f99180cd7f0 tx=0x7f991000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.643+0000 7f991e4a3700 1 -- 172.21.15.45:0/3316862240 shutdown_connections 2023-08-27T11:28:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.643+0000 7f991e4a3700 1 --2- 172.21.15.45:0/3316862240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 0x7f99180b18a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.643+0000 7f991e4a3700 1 --2- 172.21.15.45:0/3316862240 >> v2:172.21.15.135:3300/0 conn(0x7f9918160d00 0x7f991806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.643+0000 7f991e4a3700 1 -- 172.21.15.45:0/3316862240 >> 172.21.15.45:0/3316862240 conn(0x7f99180cc7c0 msgr2=0x7f99180ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.643+0000 7f991e4a3700 1 -- 172.21.15.45:0/3316862240 shutdown_connections 2023-08-27T11:28:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.643+0000 7f991e4a3700 1 -- 172.21.15.45:0/3316862240 wait complete. 2023-08-27T11:28:27.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.644+0000 7f991e4a3700 1 Processor -- start 2023-08-27T11:28:27.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.644+0000 7f991e4a3700 1 -- start start 2023-08-27T11:28:27.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f991e4a3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 0x7f99181f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f991e4a3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815d340 0x7f99180c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f991e4a3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9918160d00 0x7f99180c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f991e4a3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991815a380 con 0x7f991806a7c0 2023-08-27T11:28:27.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f991e4a3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99181f6bd0 con 0x7f991815d340 2023-08-27T11:28:27.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f991e4a3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99181f6eb0 con 0x7f9918160d00 2023-08-27T11:28:27.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9917fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 0x7f99181f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9917fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 0x7f99181f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59554/0 (socket says 172.21.15.45:59554) 2023-08-27T11:28:27.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9917fff700 1 -- 172.21.15.45:0/2620104175 learned_addr learned my addr 172.21.15.45:0/2620104175 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:27.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f99177fe700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815d340 0x7f99180c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f991ca40700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9918160d00 0x7f99180c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9917fff700 1 -- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9918160d00 msgr2=0x7f99180c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9917fff700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9918160d00 0x7f99180c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9917fff700 1 -- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815d340 msgr2=0x7f99180c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:27.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9917fff700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815d340 0x7f99180c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:27.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9917fff700 1 -- 172.21.15.45:0/2620104175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9910009b30 con 0x7f991806a7c0 2023-08-27T11:28:27.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9917fff700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 0x7f99181f62f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f990c00b4a0 tx=0x7f990c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.645+0000 7f9914ff9700 1 -- 172.21.15.45:0/2620104175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c011830 con 0x7f991806a7c0 2023-08-27T11:28:27.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.646+0000 7f9914ff9700 1 -- 172.21.15.45:0/2620104175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c011990 con 0x7f991806a7c0 2023-08-27T11:28:27.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.646+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99181f7190 con 0x7f991806a7c0 2023-08-27T11:28:27.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.646+0000 7f9914ff9700 1 -- 172.21.15.45:0/2620104175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c011d40 con 0x7f991806a7c0 2023-08-27T11:28:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.646+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99181f6830 con 0x7f991806a7c0 2023-08-27T11:28:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.647+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9918072690 con 0x7f991806a7c0 2023-08-27T11:28:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.647+0000 7f9914ff9700 1 -- 172.21.15.45:0/2620104175 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 41) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f990c01a400 con 0x7f991806a7c0 2023-08-27T11:28:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.647+0000 7f9914ff9700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f9904069f80 0x7f990406c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.648+0000 7f9914ff9700 1 -- 172.21.15.45:0/2620104175 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f990c088260 con 0x7f991806a7c0 2023-08-27T11:28:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.648+0000 7f99177fe700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f9904069f80 0x7f990406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.648+0000 7f99177fe700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f9904069f80 0x7f990406c440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99180cc0d0 tx=0x7f9908008040 comp rx=0 tx=0).ready entity=mgr.7084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.652+0000 7f9914ff9700 1 -- 172.21.15.45:0/2620104175 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f990c0539a0 con 0x7f991806a7c0 2023-08-27T11:28:27.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:27.917+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f99180734a0 con 0x7f991806a7c0 2023-08-27T11:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.536+0000 7f9914ff9700 1 -- 172.21.15.45:0/2620104175 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f990c053d70 con 0x7f991806a7c0 2023-08-27T11:28:28.538 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.537+0000 7f9914ff9700 1 -- 172.21.15.45:0/2620104175 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v42) v1 ==== 93+0+0 (secure 0 0 0) 0x7f990c057430 con 0x7f991806a7c0 2023-08-27T11:28:28.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f9904069f80 msgr2=0x7f990406c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f9904069f80 0x7f990406c440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99180cc0d0 tx=0x7f9908008040 comp rx=0 tx=0).stop 2023-08-27T11:28:28.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 msgr2=0x7f99181f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 0x7f99181f62f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f990c00b4a0 tx=0x7f990c00b7b0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 shutdown_connections 2023-08-27T11:28:28.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f9904069f80 0x7f990406c440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f991806a7c0 0x7f99181f62f0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9918160d00 0x7f99180c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 --2- 172.21.15.45:0/2620104175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f991815d340 0x7f99180c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 >> 172.21.15.45:0/2620104175 conn(0x7f99180cc7c0 msgr2=0x7f991806d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:28.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 shutdown_connections 2023-08-27T11:28:28.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.540+0000 7f991e4a3700 1 -- 172.21.15.45:0/2620104175 wait complete. 2023-08-27T11:28:28.559 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:28:28.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:28.608 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:28.650 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:28:28.650 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:28:28.651 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:28.668 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:28:28.669 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:28.686 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:28.685+0000 7fee539ef200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:28:28.690 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:28.689+0000 7fc8dce40200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:28:28.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:28.751 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:28.750+0000 7fee539ef200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:28:28.764 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:28.764+0000 7fc8dce40200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:28:28.823 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:28.822+0000 7fee539ef200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:28:28.842 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:28.841+0000 7fc8dce40200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:28:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.875+0000 7fb9a7f1c700 1 Processor -- start 2023-08-27T11:28:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.875+0000 7fb9a7f1c700 1 -- start start 2023-08-27T11:28:28.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a7f1c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb9a00ceba0 0x7fb9a00cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a7f1c700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb9a00cbe70 0x7fb9a006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a7f1c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a006d690 con 0x7fb9a015a240 2023-08-27T11:28:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a7f1c700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a00cc440 con 0x7fb9a00ceba0 2023-08-27T11:28:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a7f1c700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a006d040 con 0x7fb9a00cbe70 2023-08-27T11:28:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a54b7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb9a00ceba0 0x7fb9a00cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a54b7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb9a00ceba0 0x7fb9a00cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59616/0 (socket says 172.21.15.45:59616) 2023-08-27T11:28:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a54b7700 1 -- 172.21.15.45:0/4005031934 learned_addr learned my addr 172.21.15.45:0/4005031934 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a5cb8700 1 --1- 172.21.15.45:0/4005031934 >> v1:172.21.15.45:6789/0 conn(0x7fb9a015a240 0x7fb9a00ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46390/0 (socket says 172.21.15.45:46390) 2023-08-27T11:28:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a64b9700 1 --2- 172.21.15.45:0/4005031934 >> v2:172.21.15.135:3300/0 conn(0x7fb9a00cbe70 0x7fb9a006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a54b7700 1 -- 172.21.15.45:0/4005031934 >> v2:172.21.15.135:3300/0 conn(0x7fb9a00cbe70 msgr2=0x7fb9a006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a54b7700 1 --2- 172.21.15.45:0/4005031934 >> v2:172.21.15.135:3300/0 conn(0x7fb9a00cbe70 0x7fb9a006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a54b7700 1 -- 172.21.15.45:0/4005031934 >> v1:172.21.15.45:6789/0 conn(0x7fb9a015a240 legacy=0x7fb9a00ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a54b7700 1 -- 172.21.15.45:0/4005031934 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a006d320 con 0x7fb9a00ceba0 2023-08-27T11:28:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.876+0000 7fb9a54b7700 1 --2- 172.21.15.45:0/4005031934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00ceba0 0x7fb9a00cae80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00af860 tx=0x7fb99000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfe2b7507b6c35e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a4cb6700 1 -- 172.21.15.45:0/4005031934 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99000dc70 con 0x7fb9a00ceba0 2023-08-27T11:28:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a4cb6700 1 -- 172.21.15.45:0/4005031934 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99000ddd0 con 0x7fb9a00ceba0 2023-08-27T11:28:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a4cb6700 1 -- 172.21.15.45:0/4005031934 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb990010690 con 0x7fb9a00ceba0 2023-08-27T11:28:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/4005031934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00ceba0 msgr2=0x7fb9a00cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a7f1c700 1 --2- 172.21.15.45:0/4005031934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00ceba0 0x7fb9a00cae80 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00af860 tx=0x7fb99000be70 comp rx=0 tx=0).stop 2023-08-27T11:28:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/4005031934 shutdown_connections 2023-08-27T11:28:28.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a7f1c700 1 --2- 172.21.15.45:0/4005031934 >> v2:172.21.15.135:3300/0 conn(0x7fb9a00cbe70 0x7fb9a006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a7f1c700 1 --2- 172.21.15.45:0/4005031934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00ceba0 0x7fb9a00cae80 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/4005031934 >> 172.21.15.45:0/4005031934 conn(0x7fb9a00cfe50 msgr2=0x7fb9a00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:28.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/4005031934 shutdown_connections 2023-08-27T11:28:28.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.877+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/4005031934 wait complete. 2023-08-27T11:28:28.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.878+0000 7fb9a7f1c700 1 Processor -- start 2023-08-27T11:28:28.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a7f1c700 1 -- start start 2023-08-27T11:28:28.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a7f1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00cbe70 0x7fb9a00c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a7f1c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9a00ceba0 0x7fb9a01f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a7f1c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9a015a240 0x7fb9a01583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a7f1c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00c3980 con 0x7fb9a00cbe70 2023-08-27T11:28:28.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a7f1c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00c8180 con 0x7fb9a015a240 2023-08-27T11:28:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a7f1c700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00c8460 con 0x7fb9a00ceba0 2023-08-27T11:28:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a5cb8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00cbe70 0x7fb9a00c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a5cb8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00cbe70 0x7fb9a00c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59620/0 (socket says 172.21.15.45:59620) 2023-08-27T11:28:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a5cb8700 1 -- 172.21.15.45:0/2067080469 learned_addr learned my addr 172.21.15.45:0/2067080469 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a64b9700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9a015a240 0x7fb9a01583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.879+0000 7fb9a54b7700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9a00ceba0 0x7fb9a01f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9a5cb8700 1 -- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9a00ceba0 msgr2=0x7fb9a01f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9a5cb8700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9a00ceba0 0x7fb9a01f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9a5cb8700 1 -- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9a015a240 msgr2=0x7fb9a01583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9a5cb8700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9a015a240 0x7fb9a01583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9a5cb8700 1 -- 172.21.15.45:0/2067080469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99000b7f0 con 0x7fb9a00cbe70 2023-08-27T11:28:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9a5cb8700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00cbe70 0x7fb9a00c7930 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00c500 tx=0x7fb99c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9967fc700 1 -- 172.21.15.45:0/2067080469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c005cb0 con 0x7fb9a00cbe70 2023-08-27T11:28:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9967fc700 1 -- 172.21.15.45:0/2067080469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c005e10 con 0x7fb9a00cbe70 2023-08-27T11:28:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a00c8740 con 0x7fb9a00cbe70 2023-08-27T11:28:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9967fc700 1 -- 172.21.15.45:0/2067080469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0072d0 con 0x7fb9a00cbe70 2023-08-27T11:28:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.880+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a00c8d00 con 0x7fb9a00cbe70 2023-08-27T11:28:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.881+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a015b670 con 0x7fb9a00cbe70 2023-08-27T11:28:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.882+0000 7fb9967fc700 1 -- 172.21.15.45:0/2067080469 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fb99c007430 con 0x7fb9a00cbe70 2023-08-27T11:28:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.882+0000 7fb9967fc700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb98406a200 0x7fb98406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.882+0000 7fb9967fc700 1 -- 172.21.15.45:0/2067080469 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb99c087c10 con 0x7fb9a00cbe70 2023-08-27T11:28:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.882+0000 7fb9a54b7700 1 -- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb98406a200 msgr2=0x7fb98406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.882+0000 7fb9a54b7700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb98406a200 0x7fb98406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:28.887+0000 7fb9967fc700 1 -- 172.21.15.45:0/2067080469 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb9a015b670 con 0x7fb9a00cbe70 2023-08-27T11:28:28.901 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:28.900+0000 7fee539ef200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:28:28.929 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:28.928+0000 7fc8dce40200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:28:29.041 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:29.040+0000 7fee539ef200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:28:29.079 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:29.078+0000 7fc8dce40200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:28:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.083+0000 7fb9a54b7700 1 -- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb98406a200 msgr2=0x7fb98406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:29.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.083+0000 7fb9a54b7700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb98406a200 0x7fb98406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.091+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9a015b670 con 0x7fb9a00cbe70 2023-08-27T11:28:29.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.092+0000 7fb9967fc700 1 -- 172.21.15.45:0/2067080469 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+19199 (secure 0 0 0) 0x7fb9a015b670 con 0x7fb9a00cbe70 2023-08-27T11:28:29.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:29.094 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":102,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:26.474836+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:05.410178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:06.051204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:6833/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:6832/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3142979284":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/569758288":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/129865033":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/1640490812":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/12937390":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/2560756484":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3438903793":"2023-08-27T12:28:16.346002+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb98406a200 msgr2=0x7fb98406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb98406a200 0x7fb98406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00cbe70 msgr2=0x7fb9a00c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00cbe70 0x7fb9a00c7930 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00c500 tx=0x7fb99c00c810 comp rx=0 tx=0).stop 2023-08-27T11:28:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 shutdown_connections 2023-08-27T11:28:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb98406a200 0x7fb98406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9a00cbe70 0x7fb9a00c7930 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9a015a240 0x7fb9a01583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 --2- 172.21.15.45:0/2067080469 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb9a00ceba0 0x7fb9a01f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 >> 172.21.15.45:0/2067080469 conn(0x7fb9a00cfe50 msgr2=0x7fb9a00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 shutdown_connections 2023-08-27T11:28:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.095+0000 7fb9a7f1c700 1 -- 172.21.15.45:0/2067080469 wait complete. 2023-08-27T11:28:29.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:28:05.410178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:28:06.051204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:29.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:29.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.277+0000 7f876b3ee700 1 Processor -- start 2023-08-27T11:28:29.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.277+0000 7f876b3ee700 1 -- start start 2023-08-27T11:28:29.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876b3ee700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f87640af200 0x7f87640cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876b3ee700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f87640caa00 0x7f87640acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876b3ee700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640ce340 con 0x7f87640ca620 2023-08-27T11:28:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876b3ee700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640ceec0 con 0x7f87640caa00 2023-08-27T11:28:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876b3ee700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876415d7c0 con 0x7f87640af200 2023-08-27T11:28:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876998b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f87640caa00 0x7f87640acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876998b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f87640caa00 0x7f87640acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59658/0 (socket says 172.21.15.45:59658) 2023-08-27T11:28:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f8768989700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f87640af200 0x7f87640cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876998b700 1 -- 172.21.15.45:0/1916326125 learned_addr learned my addr 172.21.15.45:0/1916326125 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:29.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876998b700 1 -- 172.21.15.45:0/1916326125 >> v2:172.21.15.135:3300/0 conn(0x7f87640af200 msgr2=0x7f87640cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:29.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876998b700 1 --2- 172.21.15.45:0/1916326125 >> v2:172.21.15.135:3300/0 conn(0x7f87640af200 0x7f87640cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876998b700 1 -- 172.21.15.45:0/1916326125 >> v1:172.21.15.135:6789/0 conn(0x7f87640ca620 legacy=0x7f87640ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.278+0000 7f876998b700 1 -- 172.21.15.45:0/1916326125 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87640cc570 con 0x7f87640caa00 2023-08-27T11:28:29.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f876998b700 1 --2- 172.21.15.45:0/1916326125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640caa00 0x7f87640acfc0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f876406f860 tx=0x7f875c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4930c2181b6bb49c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:29.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f875bfff700 1 -- 172.21.15.45:0/1916326125 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00e070 con 0x7f87640caa00 2023-08-27T11:28:29.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f875bfff700 1 -- 172.21.15.45:0/1916326125 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00b910 con 0x7f87640caa00 2023-08-27T11:28:29.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f875bfff700 1 -- 172.21.15.45:0/1916326125 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00bcc0 con 0x7f87640caa00 2023-08-27T11:28:29.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f876b3ee700 1 -- 172.21.15.45:0/1916326125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640caa00 msgr2=0x7f87640acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f876b3ee700 1 --2- 172.21.15.45:0/1916326125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640caa00 0x7f87640acfc0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f876406f860 tx=0x7f875c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:29.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f876b3ee700 1 -- 172.21.15.45:0/1916326125 shutdown_connections 2023-08-27T11:28:29.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f876b3ee700 1 --2- 172.21.15.45:0/1916326125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640caa00 0x7f87640acfc0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f876b3ee700 1 --2- 172.21.15.45:0/1916326125 >> v2:172.21.15.135:3300/0 conn(0x7f87640af200 0x7f87640cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.279+0000 7f876b3ee700 1 -- 172.21.15.45:0/1916326125 >> 172.21.15.45:0/1916326125 conn(0x7f87640cc7e0 msgr2=0x7f87640ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:29.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.280+0000 7f876b3ee700 1 -- 172.21.15.45:0/1916326125 shutdown_connections 2023-08-27T11:28:29.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.280+0000 7f876b3ee700 1 -- 172.21.15.45:0/1916326125 wait complete. 2023-08-27T11:28:29.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.282+0000 7f876b3ee700 1 Processor -- start 2023-08-27T11:28:29.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.282+0000 7f876b3ee700 1 -- start start 2023-08-27T11:28:29.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876b3ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640af200 0x7f876406d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876b3ee700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87640ca620 0x7f8764159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876b3ee700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87640caa00 0x7f8764159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876b3ee700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876406e250 con 0x7f87640af200 2023-08-27T11:28:29.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876b3ee700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876406e530 con 0x7f87640ca620 2023-08-27T11:28:29.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876b3ee700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c3eb0 con 0x7f87640caa00 2023-08-27T11:28:29.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876918a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640af200 0x7f876406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876918a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640af200 0x7f876406d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59660/0 (socket says 172.21.15.45:59660) 2023-08-27T11:28:29.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876918a700 1 -- 172.21.15.45:0/1144415541 learned_addr learned my addr 172.21.15.45:0/1144415541 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:29.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f8768989700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87640ca620 0x7f8764159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876998b700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87640caa00 0x7f8764159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876918a700 1 -- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87640caa00 msgr2=0x7f8764159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876918a700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87640caa00 0x7f8764159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876918a700 1 -- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87640ca620 msgr2=0x7f8764159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876918a700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87640ca620 0x7f8764159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876918a700 1 -- 172.21.15.45:0/1144415541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c009b30 con 0x7f87640af200 2023-08-27T11:28:29.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.283+0000 7f876918a700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640af200 0x7f876406d950 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f876000c4e0 tx=0x7f876000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:29.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.284+0000 7f8759ffb700 1 -- 172.21.15.45:0/1144415541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87600052d0 con 0x7f87640af200 2023-08-27T11:28:29.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.284+0000 7f8759ffb700 1 -- 172.21.15.45:0/1144415541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8760005430 con 0x7f87640af200 2023-08-27T11:28:29.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.284+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87640c4190 con 0x7f87640af200 2023-08-27T11:28:29.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.284+0000 7f8759ffb700 1 -- 172.21.15.45:0/1144415541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87600038c0 con 0x7f87640af200 2023-08-27T11:28:29.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.284+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876406de90 con 0x7f87640af200 2023-08-27T11:28:29.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.285+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87640c4380 con 0x7f87640af200 2023-08-27T11:28:29.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.285+0000 7f8759ffb700 1 -- 172.21.15.45:0/1144415541 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f8760003a20 con 0x7f87640af200 2023-08-27T11:28:29.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.286+0000 7f8759ffb700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f874806a200 0x7f874806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.286+0000 7f8768989700 1 -- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f874806a200 msgr2=0x7f874806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:29.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.286+0000 7f8759ffb700 1 -- 172.21.15.45:0/1144415541 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8760087d70 con 0x7f87640af200 2023-08-27T11:28:29.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.286+0000 7f8768989700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f874806a200 0x7f874806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:29.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.290+0000 7f8759ffb700 1 -- 172.21.15.45:0/1144415541 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8760053060 con 0x7f87640af200 2023-08-27T11:28:29.365 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:29.364+0000 7fee539ef200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:28:29.430 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:29.429+0000 7fc8dce40200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:28:29.480 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:29.479+0000 7fee539ef200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:28:29.487 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.487+0000 7f8768989700 1 -- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f874806a200 msgr2=0x7f874806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:29.488 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.487+0000 7f8768989700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f874806a200 0x7f874806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.494+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87640cbce0 con 0x7f87640af200 2023-08-27T11:28:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.495+0000 7f8759ffb700 1 -- 172.21.15.45:0/1144415541 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8760017020 con 0x7f87640af200 2023-08-27T11:28:29.496 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:28:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f874806a200 msgr2=0x7f874806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:29.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f874806a200 0x7f874806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640af200 msgr2=0x7f876406d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640af200 0x7f876406d950 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f876000c4e0 tx=0x7f876000c7f0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 shutdown_connections 2023-08-27T11:28:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f874806a200 0x7f874806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87640af200 0x7f876406d950 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f87640caa00 0x7f8764159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 --2- 172.21.15.45:0/1144415541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87640ca620 0x7f8764159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 >> 172.21.15.45:0/1144415541 conn(0x7f87640cc7e0 msgr2=0x7f87640ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:29.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 shutdown_connections 2023-08-27T11:28:29.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.497+0000 7f876b3ee700 1 -- 172.21.15.45:0/1144415541 wait complete. 2023-08-27T11:28:29.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:29.554 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:29.554+0000 7fc8dce40200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:28:29.556 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:29.555+0000 7fee539ef200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:28:29.634 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:29.633+0000 7fc8dce40200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:28:29.636 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:29.635+0000 7fee539ef200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:28:29.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.641+0000 7fa2e86fb700 1 Processor -- start 2023-08-27T11:28:29.642 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.641+0000 7fa2e86fb700 1 -- start start 2023-08-27T11:28:29.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.642+0000 7fa2e86fb700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa2e015d3a0 0x7fa2e00d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.642+0000 7fa2e86fb700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa2e00d02c0 0x7fa2e006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.642+0000 7fa2e86fb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 0x7fa2e006c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.642+0000 7fa2e86fb700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e015d780 con 0x7fa2e00d02c0 2023-08-27T11:28:29.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.642+0000 7fa2e86fb700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e00cfe30 con 0x7fa2e015d3a0 2023-08-27T11:28:29.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.642+0000 7fa2e86fb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00cf220 con 0x7fa2e00b0320 2023-08-27T11:28:29.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.642+0000 7fa2e5c96700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa2e00d02c0 0x7fa2e006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5c96700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa2e00d02c0 0x7fa2e006bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59672/0 (socket says 172.21.15.45:59672) 2023-08-27T11:28:29.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5c96700 1 -- 172.21.15.45:0/524944700 learned_addr learned my addr 172.21.15.45:0/524944700 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:29.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e6497700 1 --2- 172.21.15.45:0/524944700 >> v2:172.21.15.135:3300/0 conn(0x7fa2e015d3a0 0x7fa2e00d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e6c98700 1 --2- 172.21.15.45:0/524944700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 0x7fa2e006c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5c96700 1 -- 172.21.15.45:0/524944700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 msgr2=0x7fa2e006c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:29.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5c96700 1 --2- 172.21.15.45:0/524944700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 0x7fa2e006c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5c96700 1 -- 172.21.15.45:0/524944700 >> v2:172.21.15.135:3300/0 conn(0x7fa2e015d3a0 msgr2=0x7fa2e00d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:29.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5c96700 1 --2- 172.21.15.45:0/524944700 >> v2:172.21.15.135:3300/0 conn(0x7fa2e015d3a0 0x7fa2e00d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5c96700 1 -- 172.21.15.45:0/524944700 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00cf500 con 0x7fa2e00d02c0 2023-08-27T11:28:29.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5c96700 1 --2- 172.21.15.45:0/524944700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e00d02c0 0x7fa2e006bda0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00701d0 tx=0x7fa2d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45687b7b876531fe server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:29.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5495700 1 -- 172.21.15.45:0/524944700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0011830 con 0x7fa2e00d02c0 2023-08-27T11:28:29.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5495700 1 -- 172.21.15.45:0/524944700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d0011990 con 0x7fa2e00d02c0 2023-08-27T11:28:29.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.643+0000 7fa2e5495700 1 -- 172.21.15.45:0/524944700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0011d20 con 0x7fa2e00d02c0 2023-08-27T11:28:29.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.644+0000 7fa2e86fb700 1 -- 172.21.15.45:0/524944700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e00d02c0 msgr2=0x7fa2e006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.644+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/524944700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e00d02c0 0x7fa2e006bda0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00701d0 tx=0x7fa2d0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.644+0000 7fa2e86fb700 1 -- 172.21.15.45:0/524944700 shutdown_connections 2023-08-27T11:28:29.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.644+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/524944700 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 0x7fa2e006c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.644+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/524944700 >> v2:172.21.15.135:3300/0 conn(0x7fa2e015d3a0 0x7fa2e00d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.644+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/524944700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e00d02c0 0x7fa2e006bda0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.644+0000 7fa2e86fb700 1 -- 172.21.15.45:0/524944700 >> 172.21.15.45:0/524944700 conn(0x7fa2e009a1a0 msgr2=0x7fa2e00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.644+0000 7fa2e86fb700 1 -- 172.21.15.45:0/524944700 shutdown_connections 2023-08-27T11:28:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.644+0000 7fa2e86fb700 1 -- 172.21.15.45:0/524944700 wait complete. 2023-08-27T11:28:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.646+0000 7fa2e86fb700 1 Processor -- start 2023-08-27T11:28:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.646+0000 7fa2e86fb700 1 -- start start 2023-08-27T11:28:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.647+0000 7fa2e86fb700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 0x7fa2e00b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e86fb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2e00d02c0 0x7fa2e0150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e86fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e015d3a0 0x7fa2e00b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e86fb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b1180 con 0x7fa2e015d3a0 2023-08-27T11:28:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e86fb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b3180 con 0x7fa2e00d02c0 2023-08-27T11:28:29.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e86fb700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b3460 con 0x7fa2e00b0320 2023-08-27T11:28:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6c98700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e015d3a0 0x7fa2e00b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6c98700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e015d3a0 0x7fa2e00b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59680/0 (socket says 172.21.15.45:59680) 2023-08-27T11:28:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6c98700 1 -- 172.21.15.45:0/3531414736 learned_addr learned my addr 172.21.15.45:0/3531414736 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6497700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 0x7fa2e00b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e5c96700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2e00d02c0 0x7fa2e0150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6c98700 1 -- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 msgr2=0x7fa2e00b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6c98700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 0x7fa2e00b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6c98700 1 -- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2e00d02c0 msgr2=0x7fa2e0150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6c98700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2e00d02c0 0x7fa2e0150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6c98700 1 -- 172.21.15.45:0/3531414736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0009550 con 0x7fa2e015d3a0 2023-08-27T11:28:29.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e6c98700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e015d3a0 0x7fa2e00b13a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800efc0 tx=0x7fa2d800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:29.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2d6ffd700 1 -- 172.21.15.45:0/3531414736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800cad0 con 0x7fa2e015d3a0 2023-08-27T11:28:29.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2d6ffd700 1 -- 172.21.15.45:0/3531414736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d800f040 con 0x7fa2e015d3a0 2023-08-27T11:28:29.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e00b3740 con 0x7fa2e015d3a0 2023-08-27T11:28:29.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2d6ffd700 1 -- 172.21.15.45:0/3531414736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800ce70 con 0x7fa2e015d3a0 2023-08-27T11:28:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.648+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e00b6900 con 0x7fa2e015d3a0 2023-08-27T11:28:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.649+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e00b6af0 con 0x7fa2e015d3a0 2023-08-27T11:28:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.650+0000 7fa2d6ffd700 1 -- 172.21.15.45:0/3531414736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fa2d8014480 con 0x7fa2e015d3a0 2023-08-27T11:28:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.650+0000 7fa2d6ffd700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fa2c406a200 0x7fa2c406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.650+0000 7fa2e6497700 1 -- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fa2c406a200 msgr2=0x7fa2c406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.650+0000 7fa2d6ffd700 1 -- 172.21.15.45:0/3531414736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa2d8089330 con 0x7fa2e015d3a0 2023-08-27T11:28:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.650+0000 7fa2e6497700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fa2c406a200 0x7fa2c406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.655+0000 7fa2d6ffd700 1 -- 172.21.15.45:0/3531414736 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa2d8057f20 con 0x7fa2e015d3a0 2023-08-27T11:28:29.723 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:29.722+0000 7fc8dce40200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:28:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.851+0000 7fa2e6497700 1 -- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fa2c406a200 msgr2=0x7fa2c406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:29.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.851+0000 7fa2e6497700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fa2c406a200 0x7fa2c406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.855+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa2e00cf7b0 con 0x7fa2e015d3a0 2023-08-27T11:28:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.855+0000 7fa2d6ffd700 1 -- 172.21.15.45:0/3531414736 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa2d8057ab0 con 0x7fa2e015d3a0 2023-08-27T11:28:29.857 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:28:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fa2c406a200 msgr2=0x7fa2c406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fa2c406a200 0x7fa2c406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e015d3a0 msgr2=0x7fa2e00b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e015d3a0 0x7fa2e00b13a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800efc0 tx=0x7fa2d800c0c0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 shutdown_connections 2023-08-27T11:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fa2c406a200 0x7fa2c406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2e015d3a0 0x7fa2e00b13a0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2e00b0320 0x7fa2e00b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 --2- 172.21.15.45:0/3531414736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa2e00d02c0 0x7fa2e0150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 >> 172.21.15.45:0/3531414736 conn(0x7fa2e009a1a0 msgr2=0x7fa2e00af990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 shutdown_connections 2023-08-27T11:28:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:29.858+0000 7fa2e86fb700 1 -- 172.21.15.45:0/3531414736 wait complete. 2023-08-27T11:28:29.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:28:30.009 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:30.008+0000 7fee539ef200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:28:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.040+0000 7f10a8c9a700 1 Processor -- start 2023-08-27T11:28:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.040+0000 7f10a8c9a700 1 -- start start 2023-08-27T11:28:30.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a8c9a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f10a40b5220 0x7f10a40cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a8c9a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40cdf40 con 0x7f10a4160890 2023-08-27T11:28:30.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a8c9a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40d0fd0 con 0x7f10a40b62c0 2023-08-27T11:28:30.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a8c9a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40ce190 con 0x7f10a40b5220 2023-08-27T11:28:30.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f10a4160890 0x7f10a40b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46476/0 (socket says 172.21.15.45:46476) 2023-08-27T11:28:30.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a259c700 1 -- 172.21.15.45:0/4165319275 learned_addr learned my addr 172.21.15.45:0/4165319275 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:30.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a1d9b700 1 --2- 172.21.15.45:0/4165319275 >> v2:172.21.15.135:3300/0 conn(0x7f10a40b5220 0x7f10a40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a159a700 1 -- 172.21.15.45:0/4165319275 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936603858 0 0) 0x7f10a40cdf40 con 0x7f10a4160890 2023-08-27T11:28:30.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a159a700 1 -- 172.21.15.45:0/4165319275 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108c0035b0 con 0x7f10a4160890 2023-08-27T11:28:30.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a159a700 1 -- 172.21.15.45:0/4165319275 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384255290 0 0) 0x7f10a40d0fd0 con 0x7f10a40b62c0 2023-08-27T11:28:30.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.041+0000 7f10a159a700 1 -- 172.21.15.45:0/4165319275 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a40cdf40 con 0x7f10a40b62c0 2023-08-27T11:28:30.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.042+0000 7f10a159a700 1 -- 172.21.15.45:0/4165319275 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2637227670 0 0) 0x7f108c0035b0 con 0x7f10a4160890 2023-08-27T11:28:30.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.042+0000 7f10a1d9b700 1 -- 172.21.15.45:0/4165319275 >> v1:172.21.15.135:6789/0 conn(0x7f10a40b62c0 legacy=0x7f10a40d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.042+0000 7f10a1d9b700 1 -- 172.21.15.45:0/4165319275 >> v1:172.21.15.45:6789/0 conn(0x7f10a4160890 legacy=0x7f10a40b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.042+0000 7f10a1d9b700 1 -- 172.21.15.45:0/4165319275 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40ce470 con 0x7f10a40b5220 2023-08-27T11:28:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.042+0000 7f10a1d9b700 1 --2- 172.21.15.45:0/4165319275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a40b5220 0x7f10a40cd8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1098009840 tx=0x7f1098009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7913fb2c3977a2b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.042+0000 7f10a159a700 1 -- 172.21.15.45:0/4165319275 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094001670 con 0x7f10a40b5220 2023-08-27T11:28:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.042+0000 7f10a159a700 1 -- 172.21.15.45:0/4165319275 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098004030 con 0x7f10a40b5220 2023-08-27T11:28:30.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.042+0000 7f10a159a700 1 -- 172.21.15.45:0/4165319275 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10980043c0 con 0x7f10a40b5220 2023-08-27T11:28:30.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.043+0000 7f10a8c9a700 1 -- 172.21.15.45:0/4165319275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a40b5220 msgr2=0x7f10a40cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.043+0000 7f10a8c9a700 1 --2- 172.21.15.45:0/4165319275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a40b5220 0x7f10a40cd8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1098009840 tx=0x7f1098009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.043+0000 7f10a8c9a700 1 -- 172.21.15.45:0/4165319275 shutdown_connections 2023-08-27T11:28:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.043+0000 7f10a8c9a700 1 --2- 172.21.15.45:0/4165319275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a40b5220 0x7f10a40cd8f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.043+0000 7f10a8c9a700 1 -- 172.21.15.45:0/4165319275 >> 172.21.15.45:0/4165319275 conn(0x7f10a40cfb10 msgr2=0x7f10a40cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.043+0000 7f10a8c9a700 1 -- 172.21.15.45:0/4165319275 shutdown_connections 2023-08-27T11:28:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.043+0000 7f10a8c9a700 1 -- 172.21.15.45:0/4165319275 wait complete. 2023-08-27T11:28:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.044+0000 7f10a8c9a700 1 Processor -- start 2023-08-27T11:28:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.044+0000 7f10a8c9a700 1 -- start start 2023-08-27T11:28:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a8c9a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10a40b5220 0x7f10a40ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a8c9a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10a40b62c0 0x7f10a40a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a8c9a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a4160890 0x7f10a40a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a8c9a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40aa490 con 0x7f10a40b62c0 2023-08-27T11:28:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a8c9a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40aa6d0 con 0x7f10a4160890 2023-08-27T11:28:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a8c9a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40aa9b0 con 0x7f10a40b5220 2023-08-27T11:28:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10a40b62c0 0x7f10a40a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10a40b62c0 0x7f10a40a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59706/0 (socket says 172.21.15.45:59706) 2023-08-27T11:28:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a1d9b700 1 -- 172.21.15.45:0/3360256248 learned_addr learned my addr 172.21.15.45:0/3360256248 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a259c700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10a40b5220 0x7f10a40ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a2d9d700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a4160890 0x7f10a40a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a1d9b700 1 -- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10a40b5220 msgr2=0x7f10a40ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a1d9b700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10a40b5220 0x7f10a40ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a1d9b700 1 -- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a4160890 msgr2=0x7f10a40a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a1d9b700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a4160890 0x7f10a40a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.045+0000 7f10a1d9b700 1 -- 172.21.15.45:0/3360256248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980094a0 con 0x7f10a40b62c0 2023-08-27T11:28:30.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.046+0000 7f10a1d9b700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10a40b62c0 0x7f10a40a9a10 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f109800a010 tx=0x7f1098004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:30.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.046+0000 7f1092ffd700 1 -- 172.21.15.45:0/3360256248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098003e20 con 0x7f10a40b62c0 2023-08-27T11:28:30.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.046+0000 7f1092ffd700 1 -- 172.21.15.45:0/3360256248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109801f070 con 0x7f10a40b62c0 2023-08-27T11:28:30.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.046+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a40aac90 con 0x7f10a40b62c0 2023-08-27T11:28:30.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.046+0000 7f1092ffd700 1 -- 172.21.15.45:0/3360256248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098011870 con 0x7f10a40b62c0 2023-08-27T11:28:30.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.046+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a40ab250 con 0x7f10a40b62c0 2023-08-27T11:28:30.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.047+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a406c120 con 0x7f10a40b62c0 2023-08-27T11:28:30.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.048+0000 7f1092ffd700 1 -- 172.21.15.45:0/3360256248 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f10980119d0 con 0x7f10a40b62c0 2023-08-27T11:28:30.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.048+0000 7f1092ffd700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f108406a200 0x7f108406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.048+0000 7f1092ffd700 1 -- 172.21.15.45:0/3360256248 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1098021070 con 0x7f10a40b62c0 2023-08-27T11:28:30.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.048+0000 7f10a259c700 1 -- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f108406a200 msgr2=0x7f108406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:30.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.048+0000 7f10a259c700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f108406a200 0x7f108406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:30.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.052+0000 7f1092ffd700 1 -- 172.21.15.45:0/3360256248 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1098055540 con 0x7f10a40b62c0 2023-08-27T11:28:30.119 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:30.119+0000 7fc8dce40200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:28:30.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.249+0000 7f10a259c700 1 -- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f108406a200 msgr2=0x7f108406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:30.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.249+0000 7f10a259c700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f108406a200 0x7f108406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:30.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.256+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10a40734a0 con 0x7f10a40b62c0 2023-08-27T11:28:30.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.256+0000 7f1092ffd700 1 -- 172.21.15.45:0/3360256248 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1098058b60 con 0x7f10a40b62c0 2023-08-27T11:28:30.257 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:28:30.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f108406a200 msgr2=0x7f108406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:30.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f108406a200 0x7f108406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10a40b62c0 msgr2=0x7f10a40a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10a40b62c0 0x7f10a40a9a10 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f109800a010 tx=0x7f1098004750 comp rx=0 tx=0).stop 2023-08-27T11:28:30.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 shutdown_connections 2023-08-27T11:28:30.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f108406a200 0x7f108406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a4160890 0x7f10a40a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f10a40b5220 0x7f10a40ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 --2- 172.21.15.45:0/3360256248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10a40b62c0 0x7f10a40a9a10 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.258+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 >> 172.21.15.45:0/3360256248 conn(0x7f10a40cfb10 msgr2=0x7f10a415fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:30.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.259+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 shutdown_connections 2023-08-27T11:28:30.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.259+0000 7f10a8c9a700 1 -- 172.21.15.45:0/3360256248 wait complete. 2023-08-27T11:28:30.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:28:30.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.400+0000 7f94848c7700 1 Processor -- start 2023-08-27T11:28:30.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.400+0000 7f94848c7700 1 -- start start 2023-08-27T11:28:30.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.401+0000 7f94848c7700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f948015d3a0 0x7f94800d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.401+0000 7f94848c7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94800d02c0 0x7f948006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.401+0000 7f94848c7700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948015d780 con 0x7f94800b0320 2023-08-27T11:28:30.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.401+0000 7f94848c7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94800cfe30 con 0x7f94800d02c0 2023-08-27T11:28:30.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.401+0000 7f94848c7700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94800cf220 con 0x7f948015d3a0 2023-08-27T11:28:30.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947dd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94800d02c0 0x7f948006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947dd9b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94800d02c0 0x7f948006bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59714/0 (socket says 172.21.15.45:59714) 2023-08-27T11:28:30.405 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947dd9b700 1 -- 172.21.15.45:0/2419617239 learned_addr learned my addr 172.21.15.45:0/2419617239 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:30.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947e59c700 1 --2- 172.21.15.45:0/2419617239 >> v2:172.21.15.135:3300/0 conn(0x7f948015d3a0 0x7f94800d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947dd9b700 1 -- 172.21.15.45:0/2419617239 >> v2:172.21.15.135:3300/0 conn(0x7f948015d3a0 msgr2=0x7f94800d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:30.406 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947dd9b700 1 --2- 172.21.15.45:0/2419617239 >> v2:172.21.15.135:3300/0 conn(0x7f948015d3a0 0x7f94800d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947dd9b700 1 -- 172.21.15.45:0/2419617239 >> v1:172.21.15.135:6789/0 conn(0x7f94800b0320 legacy=0x7f94800cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:30.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947dd9b700 1 -- 172.21.15.45:0/2419617239 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800cf500 con 0x7f94800d02c0 2023-08-27T11:28:30.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947dd9b700 1 --2- 172.21.15.45:0/2419617239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 0x7f948006bda0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f94800701d0 tx=0x7f9468009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97548d5a03061e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:30.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947d59a700 1 -- 172.21.15.45:0/2419617239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468004230 con 0x7f94800d02c0 2023-08-27T11:28:30.407 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947d59a700 1 -- 172.21.15.45:0/2419617239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468004390 con 0x7f94800d02c0 2023-08-27T11:28:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.402+0000 7f947d59a700 1 -- 172.21.15.45:0/2419617239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680036a0 con 0x7f94800d02c0 2023-08-27T11:28:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.403+0000 7f94848c7700 1 -- 172.21.15.45:0/2419617239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 msgr2=0x7f948006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.403+0000 7f94848c7700 1 --2- 172.21.15.45:0/2419617239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 0x7f948006bda0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f94800701d0 tx=0x7f9468009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.403+0000 7f94848c7700 1 -- 172.21.15.45:0/2419617239 shutdown_connections 2023-08-27T11:28:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.403+0000 7f94848c7700 1 --2- 172.21.15.45:0/2419617239 >> v2:172.21.15.135:3300/0 conn(0x7f948015d3a0 0x7f94800d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.403+0000 7f94848c7700 1 --2- 172.21.15.45:0/2419617239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 0x7f948006bda0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.403+0000 7f94848c7700 1 -- 172.21.15.45:0/2419617239 >> 172.21.15.45:0/2419617239 conn(0x7f948009a1a0 msgr2=0x7f94800d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.403+0000 7f94848c7700 1 -- 172.21.15.45:0/2419617239 shutdown_connections 2023-08-27T11:28:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.403+0000 7f94848c7700 1 -- 172.21.15.45:0/2419617239 wait complete. 2023-08-27T11:28:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.404+0000 7f94848c7700 1 Processor -- start 2023-08-27T11:28:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.404+0000 7f94848c7700 1 -- start start 2023-08-27T11:28:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f94848c7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94800b0320 0x7f94801574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f94848c7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 0x7f94801579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f94848c7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948015d3a0 0x7f94801f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f94848c7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94801f3880 con 0x7f94800d02c0 2023-08-27T11:28:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f94848c7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94801f2a10 con 0x7f948015d3a0 2023-08-27T11:28:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f94848c7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94801f2c70 con 0x7f94800b0320 2023-08-27T11:28:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f947dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 0x7f94801579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f947dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 0x7f94801579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59722/0 (socket says 172.21.15.45:59722) 2023-08-27T11:28:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f947dd9b700 1 -- 172.21.15.45:0/1900044591 learned_addr learned my addr 172.21.15.45:0/1900044591 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f947e59c700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94800b0320 0x7f94801574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.405+0000 7f947ed9d700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948015d3a0 0x7f94801f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f947dd9b700 1 -- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94800b0320 msgr2=0x7f94801574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f947dd9b700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94800b0320 0x7f94801574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f947dd9b700 1 -- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948015d3a0 msgr2=0x7f94801f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f947dd9b700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948015d3a0 0x7f94801f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f947dd9b700 1 -- 172.21.15.45:0/1900044591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468009550 con 0x7f94800d02c0 2023-08-27T11:28:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f947dd9b700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 0x7f94801579e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9468009130 tx=0x7f9468004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f9476ffd700 1 -- 172.21.15.45:0/1900044591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468003b90 con 0x7f94800d02c0 2023-08-27T11:28:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f9476ffd700 1 -- 172.21.15.45:0/1900044591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946801f070 con 0x7f94800d02c0 2023-08-27T11:28:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f9476ffd700 1 -- 172.21.15.45:0/1900044591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468011630 con 0x7f94800d02c0 2023-08-27T11:28:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94801f2f50 con 0x7f94800d02c0 2023-08-27T11:28:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.406+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94801f3510 con 0x7f94800d02c0 2023-08-27T11:28:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.407+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94801f36a0 con 0x7f94800d02c0 2023-08-27T11:28:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.408+0000 7f9476ffd700 1 -- 172.21.15.45:0/1900044591 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f9468021030 con 0x7f94800d02c0 2023-08-27T11:28:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.408+0000 7f9476ffd700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f946006a200 0x7f946006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.408+0000 7f947e59c700 1 -- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f946006a200 msgr2=0x7f946006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.408+0000 7f9476ffd700 1 -- 172.21.15.45:0/1900044591 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9468089c60 con 0x7f94800d02c0 2023-08-27T11:28:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.408+0000 7f947e59c700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f946006a200 0x7f946006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.412+0000 7f9476ffd700 1 -- 172.21.15.45:0/1900044591 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9468054f60 con 0x7f94800d02c0 2023-08-27T11:28:30.579 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:30.578+0000 7fee539ef200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:28:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.609+0000 7f947e59c700 1 -- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f946006a200 msgr2=0x7f946006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.609+0000 7f947e59c700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f946006a200 0x7f946006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:30.649 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:30.648+0000 7fee539ef200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:28:30.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.664+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94801f39c0 con 0x7f94800d02c0 2023-08-27T11:28:30.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.664+0000 7f9476ffd700 1 -- 172.21.15.45:0/1900044591 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9468058580 con 0x7f94800d02c0 2023-08-27T11:28:30.666 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:28:30.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f946006a200 msgr2=0x7f946006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:30.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f946006a200 0x7f946006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 msgr2=0x7f94801579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 0x7f94801579e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9468009130 tx=0x7f9468004750 comp rx=0 tx=0).stop 2023-08-27T11:28:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 shutdown_connections 2023-08-27T11:28:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f946006a200 0x7f946006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948015d3a0 0x7f94801f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f94800b0320 0x7f94801574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 --2- 172.21.15.45:0/1900044591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800d02c0 0x7f94801579e0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 >> 172.21.15.45:0/1900044591 conn(0x7f948009a1a0 msgr2=0x7f948006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 shutdown_connections 2023-08-27T11:28:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.667+0000 7f94848c7700 1 -- 172.21.15.45:0/1900044591 wait complete. 2023-08-27T11:28:30.682 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:28:30.742 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:30.741+0000 7fc8dce40200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:28:30.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.812+0000 7fc599ff7700 1 Processor -- start 2023-08-27T11:28:30.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.812+0000 7fc599ff7700 1 -- start start 2023-08-27T11:28:30.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc599ff7700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc5940af200 0x7fc5940cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc599ff7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5940ce340 con 0x7fc5940ca620 2023-08-27T11:28:30.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc599ff7700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5940ceec0 con 0x7fc5940caa00 2023-08-27T11:28:30.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc599ff7700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59415d7c0 con 0x7fc5940af200 2023-08-27T11:28:30.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc5937fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc5940ca620 0x7fc5940ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46506/0 (socket says 172.21.15.45:46506) 2023-08-27T11:28:30.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc5937fe700 1 -- 172.21.15.45:0/1172715243 learned_addr learned my addr 172.21.15.45:0/1172715243 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc592ffd700 1 --2- 172.21.15.45:0/1172715243 >> v2:172.21.15.135:3300/0 conn(0x7fc5940af200 0x7fc5940cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc5927fc700 1 -- 172.21.15.45:0/1172715243 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90098058 0 0) 0x7fc5940ce340 con 0x7fc5940ca620 2023-08-27T11:28:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc5927fc700 1 -- 172.21.15.45:0/1172715243 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc57c0035b0 con 0x7fc5940ca620 2023-08-27T11:28:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc5927fc700 1 -- 172.21.15.45:0/1172715243 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967622711 0 0) 0x7fc5940ceec0 con 0x7fc5940caa00 2023-08-27T11:28:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc5927fc700 1 -- 172.21.15.45:0/1172715243 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5940ce340 con 0x7fc5940caa00 2023-08-27T11:28:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc592ffd700 1 -- 172.21.15.45:0/1172715243 >> v1:172.21.15.135:6789/0 conn(0x7fc5940caa00 legacy=0x7fc5940acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc592ffd700 1 -- 172.21.15.45:0/1172715243 >> v1:172.21.15.45:6789/0 conn(0x7fc5940ca620 legacy=0x7fc5940ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.813+0000 7fc592ffd700 1 -- 172.21.15.45:0/1172715243 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5940cc570 con 0x7fc5940af200 2023-08-27T11:28:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc592ffd700 1 --2- 172.21.15.45:0/1172715243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5940af200 0x7fc5940cf350 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc588009840 tx=0x7fc588009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=277347cbe510d0bc server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc5927fc700 1 -- 172.21.15.45:0/1172715243 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584001670 con 0x7fc5940af200 2023-08-27T11:28:30.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc5927fc700 1 -- 172.21.15.45:0/1172715243 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc588004030 con 0x7fc5940af200 2023-08-27T11:28:30.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc5927fc700 1 -- 172.21.15.45:0/1172715243 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880043c0 con 0x7fc5940af200 2023-08-27T11:28:30.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc599ff7700 1 -- 172.21.15.45:0/1172715243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5940af200 msgr2=0x7fc5940cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc599ff7700 1 --2- 172.21.15.45:0/1172715243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5940af200 0x7fc5940cf350 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc588009840 tx=0x7fc588009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:30.819 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc599ff7700 1 -- 172.21.15.45:0/1172715243 shutdown_connections 2023-08-27T11:28:30.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc599ff7700 1 --2- 172.21.15.45:0/1172715243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5940af200 0x7fc5940cf350 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc599ff7700 1 -- 172.21.15.45:0/1172715243 >> 172.21.15.45:0/1172715243 conn(0x7fc5940cc7e0 msgr2=0x7fc5940ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:30.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc599ff7700 1 -- 172.21.15.45:0/1172715243 shutdown_connections 2023-08-27T11:28:30.820 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.814+0000 7fc599ff7700 1 -- 172.21.15.45:0/1172715243 wait complete. 2023-08-27T11:28:30.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.816+0000 7fc599ff7700 1 Processor -- start 2023-08-27T11:28:30.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.816+0000 7fc599ff7700 1 -- start start 2023-08-27T11:28:30.821 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc8dce40200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:28:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc599ff7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5940af200 0x7fc5940b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc599ff7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5940ca620 0x7fc594156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc599ff7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5940caa00 0x7fc5940c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc599ff7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940c5b50 con 0x7fc5940ca620 2023-08-27T11:28:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc599ff7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940c5410 con 0x7fc5940af200 2023-08-27T11:28:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc599ff7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc594156dd0 con 0x7fc5940caa00 2023-08-27T11:28:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc592ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5940ca620 0x7fc594156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc592ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5940ca620 0x7fc594156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59738/0 (socket says 172.21.15.45:59738) 2023-08-27T11:28:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc592ffd700 1 -- 172.21.15.45:0/1494566702 learned_addr learned my addr 172.21.15.45:0/1494566702 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc5937fe700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5940af200 0x7fc5940b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.817+0000 7fc593fff700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5940caa00 0x7fc5940c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc592ffd700 1 -- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5940caa00 msgr2=0x7fc5940c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc592ffd700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5940caa00 0x7fc5940c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc592ffd700 1 -- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5940af200 msgr2=0x7fc5940b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc592ffd700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5940af200 0x7fc5940b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc592ffd700 1 -- 172.21.15.45:0/1494566702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5880094a0 con 0x7fc5940ca620 2023-08-27T11:28:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc593fff700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5940caa00 0x7fc5940c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc592ffd700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5940ca620 0x7fc594156890 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc588004000 tx=0x7fc588004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc57bfff700 1 -- 172.21.15.45:0/1494566702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588003d30 con 0x7fc5940ca620 2023-08-27T11:28:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc57bfff700 1 -- 172.21.15.45:0/1494566702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc588003e90 con 0x7fc5940ca620 2023-08-27T11:28:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5940c4e50 con 0x7fc5940ca620 2023-08-27T11:28:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc57bfff700 1 -- 172.21.15.45:0/1494566702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880107b0 con 0x7fc5940ca620 2023-08-27T11:28:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.818+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5940c57e0 con 0x7fc5940ca620 2023-08-27T11:28:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.819+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5940c59a0 con 0x7fc5940ca620 2023-08-27T11:28:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.819+0000 7fc57bfff700 1 -- 172.21.15.45:0/1494566702 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fc588010910 con 0x7fc5940ca620 2023-08-27T11:28:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.820+0000 7fc57bfff700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fc58006a200 0x7fc58006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.820+0000 7fc57bfff700 1 -- 172.21.15.45:0/1494566702 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc588089ba0 con 0x7fc5940ca620 2023-08-27T11:28:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.820+0000 7fc5937fe700 1 -- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fc58006a200 msgr2=0x7fc58006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.820+0000 7fc5937fe700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fc58006a200 0x7fc58006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:30.824+0000 7fc57bfff700 1 -- 172.21.15.45:0/1494566702 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc588054e10 con 0x7fc5940ca620 2023-08-27T11:28:31.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.021+0000 7fc5937fe700 1 -- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fc58006a200 msgr2=0x7fc58006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:31.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.021+0000 7fc5937fe700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fc58006a200 0x7fc58006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:31.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.041+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc5940c55d0 con 0x7fc5940ca620 2023-08-27T11:28:31.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.042+0000 7fc57bfff700 1 -- 172.21.15.45:0/1494566702 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 9 entries v102) v1 ==== 80+0+533 (secure 0 0 0) 0x7fc588058430 con 0x7fc5940ca620 2023-08-27T11:28:31.043 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6833/3812189695 2023-08-28T11:28:26.474800+0000 2023-08-27T11:28:31.043 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6832/3812189695 2023-08-28T11:28:26.474800+0000 2023-08-27T11:28:31.043 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/3142979284 2023-08-28T11:28:26.474800+0000 2023-08-27T11:28:31.044 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/569758288 2023-08-28T11:28:26.474800+0000 2023-08-27T11:28:31.044 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/129865033 2023-08-28T11:28:26.474800+0000 2023-08-27T11:28:31.044 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/1640490812 2023-08-28T11:28:26.474800+0000 2023-08-27T11:28:31.044 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/12937390 2023-08-28T11:28:26.474800+0000 2023-08-27T11:28:31.044 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/2560756484 2023-08-28T11:28:26.474800+0000 2023-08-27T11:28:31.045 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/3438903793 2023-08-27T12:28:16.346002+0000 2023-08-27T11:28:31.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fc58006a200 msgr2=0x7fc58006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:31.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fc58006a200 0x7fc58006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5940ca620 msgr2=0x7fc594156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5940ca620 0x7fc594156890 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc588004000 tx=0x7fc588004750 comp rx=0 tx=0).stop 2023-08-27T11:28:31.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 shutdown_connections 2023-08-27T11:28:31.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fc58006a200 0x7fc58006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc5940caa00 0x7fc5940c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5940af200 0x7fc5940b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 --2- 172.21.15.45:0/1494566702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5940ca620 0x7fc594156890 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 >> 172.21.15.45:0/1494566702 conn(0x7fc5940cc7e0 msgr2=0x7fc59406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 shutdown_connections 2023-08-27T11:28:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.044+0000 7fc599ff7700 1 -- 172.21.15.45:0/1494566702 wait complete. 2023-08-27T11:28:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:listed 9 entries 2023-08-27T11:28:31.062 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:28:31.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:31.063 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:28:31.102 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:28:31.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:28:31.103 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:28:31.159 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:28:31.160 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:31.177 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:31.178 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:28:31.178 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:31.217 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:31.217 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:28:31.218 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:31.235 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:31.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:31.284 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:31.282+0000 7fee539ef200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:28:31.361 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:31.360+0000 7fee539ef200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:28:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.386+0000 7f692fea2700 1 Processor -- start 2023-08-27T11:28:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.386+0000 7f692fea2700 1 -- start start 2023-08-27T11:28:31.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692fea2700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f692815d2c0 0x7f69280d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692fea2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69280b8c20 0x7f69280cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692fea2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69280ce2e0 0x7f692806c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692fea2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69280ce770 con 0x7f69280ce2e0 2023-08-27T11:28:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692fea2700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692815d6a0 con 0x7f692815d2c0 2023-08-27T11:28:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692fea2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69280d2c90 con 0x7f69280b8c20 2023-08-27T11:28:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692e43f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69280ce2e0 0x7f692806c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692e43f700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69280ce2e0 0x7f692806c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59750/0 (socket says 172.21.15.45:59750) 2023-08-27T11:28:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692e43f700 1 -- 172.21.15.45:0/4048817835 learned_addr learned my addr 172.21.15.45:0/4048817835 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692d43d700 1 --2- 172.21.15.45:0/4048817835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69280b8c20 0x7f69280cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.387+0000 7f692dc3e700 1 --2- 172.21.15.45:0/4048817835 >> v2:172.21.15.135:3300/0 conn(0x7f692815d2c0 0x7f69280d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692e43f700 1 -- 172.21.15.45:0/4048817835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69280b8c20 msgr2=0x7f69280cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692e43f700 1 --2- 172.21.15.45:0/4048817835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69280b8c20 0x7f69280cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692e43f700 1 -- 172.21.15.45:0/4048817835 >> v2:172.21.15.135:3300/0 conn(0x7f692815d2c0 msgr2=0x7f69280d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:31.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692e43f700 1 --2- 172.21.15.45:0/4048817835 >> v2:172.21.15.135:3300/0 conn(0x7f692815d2c0 0x7f69280d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692e43f700 1 -- 172.21.15.45:0/4048817835 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69280d1bf0 con 0x7f69280ce2e0 2023-08-27T11:28:31.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692dc3e700 1 --2- 172.21.15.45:0/4048817835 >> v2:172.21.15.135:3300/0 conn(0x7f692815d2c0 0x7f69280d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:31.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692e43f700 1 --2- 172.21.15.45:0/4048817835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280ce2e0 0x7f692806c260 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f69280d4040 tx=0x7f692000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e098d40d20b3ab46 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:31.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692cc3c700 1 -- 172.21.15.45:0/4048817835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692000e070 con 0x7f69280ce2e0 2023-08-27T11:28:31.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692cc3c700 1 -- 172.21.15.45:0/4048817835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692000b910 con 0x7f69280ce2e0 2023-08-27T11:28:31.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.388+0000 7f692cc3c700 1 -- 172.21.15.45:0/4048817835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692000bcc0 con 0x7f69280ce2e0 2023-08-27T11:28:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.389+0000 7f692fea2700 1 -- 172.21.15.45:0/4048817835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280ce2e0 msgr2=0x7f692806c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.389+0000 7f692fea2700 1 --2- 172.21.15.45:0/4048817835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280ce2e0 0x7f692806c260 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f69280d4040 tx=0x7f692000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.389+0000 7f692fea2700 1 -- 172.21.15.45:0/4048817835 shutdown_connections 2023-08-27T11:28:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.389+0000 7f692fea2700 1 --2- 172.21.15.45:0/4048817835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280ce2e0 0x7f692806c260 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.389+0000 7f692fea2700 1 --2- 172.21.15.45:0/4048817835 >> v2:172.21.15.135:3300/0 conn(0x7f692815d2c0 0x7f69280d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.389+0000 7f692fea2700 1 --2- 172.21.15.45:0/4048817835 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69280b8c20 0x7f69280cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.389+0000 7f692fea2700 1 -- 172.21.15.45:0/4048817835 >> 172.21.15.45:0/4048817835 conn(0x7f69280d2fd0 msgr2=0x7f69280d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.389+0000 7f692fea2700 1 -- 172.21.15.45:0/4048817835 shutdown_connections 2023-08-27T11:28:31.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.389+0000 7f692fea2700 1 -- 172.21.15.45:0/4048817835 wait complete. 2023-08-27T11:28:31.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.390+0000 7f692fea2700 1 Processor -- start 2023-08-27T11:28:31.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.390+0000 7f692fea2700 1 -- start start 2023-08-27T11:28:31.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692fea2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b8c20 0x7f692815e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.398 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692fea2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69280ce2e0 0x7f6928153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692fea2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f692815d2c0 0x7f6928154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692fea2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692815d880 con 0x7f69280b8c20 2023-08-27T11:28:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692fea2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692815db60 con 0x7f69280ce2e0 2023-08-27T11:28:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692fea2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692815de40 con 0x7f692815d2c0 2023-08-27T11:28:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692dc3e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b8c20 0x7f692815e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692dc3e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b8c20 0x7f692815e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59752/0 (socket says 172.21.15.45:59752) 2023-08-27T11:28:31.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692dc3e700 1 -- 172.21.15.45:0/3654481566 learned_addr learned my addr 172.21.15.45:0/3654481566 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:31.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692e43f700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f692815d2c0 0x7f6928154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692d43d700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69280ce2e0 0x7f6928153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.400 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692e43f700 1 -- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69280ce2e0 msgr2=0x7f6928153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692e43f700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69280ce2e0 0x7f6928153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692e43f700 1 -- 172.21.15.45:0/3654481566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b8c20 msgr2=0x7f692815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692e43f700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b8c20 0x7f692815e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692e43f700 1 -- 172.21.15.45:0/3654481566 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920009b30 con 0x7f692815d2c0 2023-08-27T11:28:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.391+0000 7f692dc3e700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b8c20 0x7f692815e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.392+0000 7f692e43f700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f692815d2c0 0x7f6928154360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6920015000 tx=0x7f6920004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.392+0000 7f691e7fc700 1 -- 172.21.15.45:0/3654481566 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69200099b0 con 0x7f692815d2c0 2023-08-27T11:28:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.392+0000 7f691e7fc700 1 -- 172.21.15.45:0/3654481566 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69200092e0 con 0x7f692815d2c0 2023-08-27T11:28:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.392+0000 7f691e7fc700 1 -- 172.21.15.45:0/3654481566 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920004470 con 0x7f692815d2c0 2023-08-27T11:28:31.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.392+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692815e120 con 0x7f692815d2c0 2023-08-27T11:28:31.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.392+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69280c8a20 con 0x7f692815d2c0 2023-08-27T11:28:31.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.393+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69280d2470 con 0x7f692815d2c0 2023-08-27T11:28:31.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.394+0000 7f691e7fc700 1 -- 172.21.15.45:0/3654481566 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f69200045d0 con 0x7f692815d2c0 2023-08-27T11:28:31.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.394+0000 7f691e7fc700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f690c06a200 0x7f690c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.403 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.394+0000 7f691e7fc700 1 -- 172.21.15.45:0/3654481566 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6920089de0 con 0x7f692815d2c0 2023-08-27T11:28:31.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.394+0000 7f692dc3e700 1 -- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f690c06a200 msgr2=0x7f690c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:31.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.394+0000 7f692dc3e700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f690c06a200 0x7f690c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:31.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.399+0000 7f691e7fc700 1 -- 172.21.15.45:0/3654481566 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f69280d2470 con 0x7f692815d2c0 2023-08-27T11:28:31.507 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:31.505+0000 7fee539ef200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:28:31.508 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:31.507+0000 7fc8dce40200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:28:31.592 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:31.591+0000 7fc8dce40200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:28:31.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.595+0000 7f692dc3e700 1 -- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f690c06a200 msgr2=0x7f690c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:31.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.595+0000 7f692dc3e700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f690c06a200 0x7f690c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.644+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f692815dca0 con 0x7f692815d2c0 2023-08-27T11:28:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.645+0000 7f691e7fc700 1 -- 172.21.15.45:0/3654481566 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f692815dca0 con 0x7f692815d2c0 2023-08-27T11:28:31.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:31.647 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3604468746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3604468746},{"type":"v1","addr":"172.21.15.135:6836","nonce":3604468746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1319491874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1319491874},{"type":"v1","addr":"172.21.15.135:6837","nonce":1319491874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6937,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2036943131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2036943131},{"type":"v1","addr":"172.21.15.45:6835","nonce":2036943131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:07.092667+0000","modified":"2023-08-27T11:28:10.679683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7446":{"gid":7446,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/3257398445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3257398445},{"type":"v1","addr":"172.21.15.45:6833","nonce":3257398445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:28:31.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f690c06a200 msgr2=0x7f690c06c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:31.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f690c06a200 0x7f690c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f692815d2c0 msgr2=0x7f6928154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f692815d2c0 0x7f6928154360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6920015000 tx=0x7f6920004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 shutdown_connections 2023-08-27T11:28:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f690c06a200 0x7f690c06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b8c20 0x7f692815e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f692815d2c0 0x7f6928154360 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 --2- 172.21.15.45:0/3654481566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69280ce2e0 0x7f6928153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 >> 172.21.15.45:0/3654481566 conn(0x7f69280d2fd0 msgr2=0x7f6928158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 shutdown_connections 2023-08-27T11:28:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.648+0000 7f692fea2700 1 -- 172.21.15.45:0/3654481566 wait complete. 2023-08-27T11:28:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 41 2023-08-27T11:28:31.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:31.738 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:31.737+0000 7fee539ef200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:28:31.750 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:31.749+0000 7fc8dce40200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:28:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.790+0000 7f3246ffe700 1 Processor -- start 2023-08-27T11:28:31.791 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.790+0000 7f3246ffe700 1 -- start start 2023-08-27T11:28:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.790+0000 7f3246ffe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f324015b120 0x7f32400cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.790+0000 7f3246ffe700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32400ada50 con 0x7f32400cdb80 2023-08-27T11:28:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f3246ffe700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32400ccd60 con 0x7f32400d1a20 2023-08-27T11:28:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f3246ffe700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32400b0860 con 0x7f324015b120 2023-08-27T11:28:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f3244d9a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f324015b120 0x7f32400cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f324559b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f32400cdb80 0x7f32400b1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46564/0 (socket says 172.21.15.45:46564) 2023-08-27T11:28:31.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f324559b700 1 -- 172.21.15.45:0/699523254 learned_addr learned my addr 172.21.15.45:0/699523254 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:31.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f3244d9a700 1 -- 172.21.15.45:0/699523254 >> v1:172.21.15.135:6789/0 conn(0x7f32400d1a20 legacy=0x7f32400cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f3244d9a700 1 -- 172.21.15.45:0/699523254 >> v1:172.21.15.45:6789/0 conn(0x7f32400cdb80 legacy=0x7f32400b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f3244d9a700 1 -- 172.21.15.45:0/699523254 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32400b0b40 con 0x7f324015b120 2023-08-27T11:28:31.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f323f7fe700 1 -- 172.21.15.45:0/699523254 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279744047 0 0) 0x7f32400ada50 con 0x7f32400cdb80 2023-08-27T11:28:31.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f3244d9a700 1 --2- 172.21.15.45:0/699523254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324015b120 0x7f32400cc430 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3234009840 tx=0x7f3234009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ba7dace03af5b2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:31.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f323f7fe700 1 -- 172.21.15.45:0/699523254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234011830 con 0x7f324015b120 2023-08-27T11:28:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f323f7fe700 1 -- 172.21.15.45:0/699523254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3234011990 con 0x7f324015b120 2023-08-27T11:28:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.791+0000 7f323f7fe700 1 -- 172.21.15.45:0/699523254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234011d20 con 0x7f324015b120 2023-08-27T11:28:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.792+0000 7f3246ffe700 1 -- 172.21.15.45:0/699523254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324015b120 msgr2=0x7f32400cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.792+0000 7f3246ffe700 1 --2- 172.21.15.45:0/699523254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324015b120 0x7f32400cc430 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3234009840 tx=0x7f3234009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:31.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.792+0000 7f3246ffe700 1 -- 172.21.15.45:0/699523254 shutdown_connections 2023-08-27T11:28:31.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.792+0000 7f3246ffe700 1 --2- 172.21.15.45:0/699523254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324015b120 0x7f32400cc430 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.792+0000 7f3246ffe700 1 -- 172.21.15.45:0/699523254 >> 172.21.15.45:0/699523254 conn(0x7f324009a160 msgr2=0x7f32400d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:31.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.792+0000 7f3246ffe700 1 -- 172.21.15.45:0/699523254 shutdown_connections 2023-08-27T11:28:31.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.792+0000 7f3246ffe700 1 -- 172.21.15.45:0/699523254 wait complete. 2023-08-27T11:28:31.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.793+0000 7f3246ffe700 1 Processor -- start 2023-08-27T11:28:31.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.793+0000 7f3246ffe700 1 -- start start 2023-08-27T11:28:31.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f3246ffe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32400cdb80 0x7f324006d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f3246ffe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32400d1a20 0x7f324006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f3246ffe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324015b120 0x7f32400b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f3246ffe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324006c890 con 0x7f32400d1a20 2023-08-27T11:28:31.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f3246ffe700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324006c2e0 con 0x7f32400cdb80 2023-08-27T11:28:31.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f3246ffe700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324006c5b0 con 0x7f324015b120 2023-08-27T11:28:31.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f323ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32400d1a20 0x7f324006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f323ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32400d1a20 0x7f324006bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59792/0 (socket says 172.21.15.45:59792) 2023-08-27T11:28:31.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f323ffff700 1 -- 172.21.15.45:0/4173766453 learned_addr learned my addr 172.21.15.45:0/4173766453 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f324559b700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324015b120 0x7f32400b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.794+0000 7f3244d9a700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32400cdb80 0x7f324006d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f323ffff700 1 -- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324015b120 msgr2=0x7f32400b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f323ffff700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324015b120 0x7f32400b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f323ffff700 1 -- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32400cdb80 msgr2=0x7f324006d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:31.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f323ffff700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32400cdb80 0x7f324006d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:31.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f323ffff700 1 -- 172.21.15.45:0/4173766453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32340094a0 con 0x7f32400d1a20 2023-08-27T11:28:31.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f323ffff700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32400d1a20 0x7f324006bda0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c4e0 tx=0x7f322c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:31.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f323d7fa700 1 -- 172.21.15.45:0/4173766453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c0052d0 con 0x7f32400d1a20 2023-08-27T11:28:31.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f323d7fa700 1 -- 172.21.15.45:0/4173766453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c005430 con 0x7f32400d1a20 2023-08-27T11:28:31.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324015c5f0 con 0x7f32400d1a20 2023-08-27T11:28:31.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f323d7fa700 1 -- 172.21.15.45:0/4173766453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c0038c0 con 0x7f32400d1a20 2023-08-27T11:28:31.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.795+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324015cbb0 con 0x7f32400d1a20 2023-08-27T11:28:31.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.796+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32400cada0 con 0x7f32400d1a20 2023-08-27T11:28:31.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.797+0000 7f323d7fa700 1 -- 172.21.15.45:0/4173766453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f322c003a20 con 0x7f32400d1a20 2023-08-27T11:28:31.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.798+0000 7f323d7fa700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f322406a200 0x7f322406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:31.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.798+0000 7f323d7fa700 1 -- 172.21.15.45:0/4173766453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f322c087ce0 con 0x7f32400d1a20 2023-08-27T11:28:31.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.798+0000 7f3244d9a700 1 -- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f322406a200 msgr2=0x7f322406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:31.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.798+0000 7f3244d9a700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f322406a200 0x7f322406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.802+0000 7f323d7fa700 1 -- 172.21.15.45:0/4173766453 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f322c053060 con 0x7f32400d1a20 2023-08-27T11:28:31.812 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:31.812+0000 7fee539ef200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:28:31.994 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:31.993+0000 7fc8dce40200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:28:31.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.999+0000 7f3244d9a700 1 -- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f322406a200 msgr2=0x7f322406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:32.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:31.999+0000 7f3244d9a700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f322406a200 0x7f322406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:32.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.004+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f324009edd0 con 0x7f32400d1a20 2023-08-27T11:28:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.005+0000 7f323d7fa700 1 -- 172.21.15.45:0/4173766453 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+19199 (secure 0 0 0) 0x7f322c056680 con 0x7f32400d1a20 2023-08-27T11:28:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:32.007 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":102,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:26.474836+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:05.410178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:06.051204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:6833/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:6832/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3142979284":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/569758288":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/129865033":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/1640490812":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/12937390":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/2560756484":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3438903793":"2023-08-27T12:28:16.346002+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:32.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f322406a200 msgr2=0x7f322406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:32.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f322406a200 0x7f322406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32400d1a20 msgr2=0x7f324006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32400d1a20 0x7f324006bda0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c4e0 tx=0x7f322c00c7f0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 shutdown_connections 2023-08-27T11:28:32.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f322406a200 0x7f322406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324015b120 0x7f32400b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32400cdb80 0x7f324006d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 --2- 172.21.15.45:0/4173766453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32400d1a20 0x7f324006bda0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 >> 172.21.15.45:0/4173766453 conn(0x7f324009a160 msgr2=0x7f32400cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 shutdown_connections 2023-08-27T11:28:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.008+0000 7f3246ffe700 1 -- 172.21.15.45:0/4173766453 wait complete. 2023-08-27T11:28:32.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:28:05.410178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:28:06.051204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:32.024 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:32.079 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:32.078+0000 7fc8dce40200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:28:32.079 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:32.077+0000 7fee539ef200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:28:32.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.154+0000 7fb3bb0e0700 1 Processor -- start 2023-08-27T11:28:32.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.154+0000 7fb3bb0e0700 1 -- start start 2023-08-27T11:28:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.155+0000 7fb3bb0e0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b4164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.155+0000 7fb3bb0e0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b40721e0 con 0x7fb3b40b2ef0 2023-08-27T11:28:32.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.155+0000 7fb3bb0e0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b4071e10 con 0x7fb3b4159a20 2023-08-27T11:28:32.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.155+0000 7fb3bb0e0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b40cc710 con 0x7fb3b4072670 2023-08-27T11:28:32.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.155+0000 7fb3b3fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb3b40b2ef0 0x7fb3b40cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46576/0 (socket says 172.21.15.45:46576) 2023-08-27T11:28:32.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.155+0000 7fb3b3fff700 1 -- 172.21.15.45:0/2165301892 learned_addr learned my addr 172.21.15.45:0/2165301892 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:32.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.155+0000 7fb3b967d700 1 --2- 172.21.15.45:0/2165301892 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b4164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.155+0000 7fb3b37fe700 1 -- 172.21.15.45:0/2165301892 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366716146 0 0) 0x7fb3b40721e0 con 0x7fb3b40b2ef0 2023-08-27T11:28:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.155+0000 7fb3b37fe700 1 -- 172.21.15.45:0/2165301892 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3a40035b0 con 0x7fb3b40b2ef0 2023-08-27T11:28:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.156+0000 7fb3b37fe700 1 -- 172.21.15.45:0/2165301892 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305411171 0 0) 0x7fb3b4071e10 con 0x7fb3b4159a20 2023-08-27T11:28:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.156+0000 7fb3b37fe700 1 -- 172.21.15.45:0/2165301892 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b40721e0 con 0x7fb3b4159a20 2023-08-27T11:28:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.156+0000 7fb3b967d700 1 -- 172.21.15.45:0/2165301892 >> v1:172.21.15.135:6789/0 conn(0x7fb3b4159a20 legacy=0x7fb3b40cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.156+0000 7fb3b967d700 1 -- 172.21.15.45:0/2165301892 >> v1:172.21.15.45:6789/0 conn(0x7fb3b40b2ef0 legacy=0x7fb3b40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.156+0000 7fb3b967d700 1 -- 172.21.15.45:0/2165301892 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b40c62f0 con 0x7fb3b4072670 2023-08-27T11:28:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.156+0000 7fb3b967d700 1 --2- 172.21.15.45:0/2165301892 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b4164400 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00aed0 tx=0x7fb3ac00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c94dd0db03de5e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.156+0000 7fb3b37fe700 1 -- 172.21.15.45:0/2165301892 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00f070 con 0x7fb3b4072670 2023-08-27T11:28:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.156+0000 7fb3b37fe700 1 -- 172.21.15.45:0/2165301892 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ac00a2e0 con 0x7fb3b4072670 2023-08-27T11:28:32.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.156+0000 7fb3b37fe700 1 -- 172.21.15.45:0/2165301892 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00cc10 con 0x7fb3b4072670 2023-08-27T11:28:32.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.157+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/2165301892 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 msgr2=0x7fb3b4164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.157+0000 7fb3bb0e0700 1 --2- 172.21.15.45:0/2165301892 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b4164400 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00aed0 tx=0x7fb3ac00c330 comp rx=0 tx=0).stop 2023-08-27T11:28:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.157+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/2165301892 shutdown_connections 2023-08-27T11:28:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.157+0000 7fb3bb0e0700 1 --2- 172.21.15.45:0/2165301892 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b4164400 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.157+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/2165301892 >> 172.21.15.45:0/2165301892 conn(0x7fb3b40b23d0 msgr2=0x7fb3b40b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.157+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/2165301892 shutdown_connections 2023-08-27T11:28:32.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.157+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/2165301892 wait complete. 2023-08-27T11:28:32.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.158+0000 7fb3bb0e0700 1 Processor -- start 2023-08-27T11:28:32.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.158+0000 7fb3bb0e0700 1 -- start start 2023-08-27T11:28:32.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3bb0e0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b406da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.165 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3bb0e0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3b40b2ef0 0x7fb3b4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3bb0e0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3b4159a20 0x7fb3b41f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3bb0e0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41f6540 con 0x7fb3b40b2ef0 2023-08-27T11:28:32.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3bb0e0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b406df70 con 0x7fb3b4159a20 2023-08-27T11:28:32.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3bb0e0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41f34c0 con 0x7fb3b4072670 2023-08-27T11:28:32.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3b3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3b40b2ef0 0x7fb3b4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3b8e7c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3b967d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3b4159a20 0x7fb3b41f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3b3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3b40b2ef0 0x7fb3b4157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59808/0 (socket says 172.21.15.45:59808) 2023-08-27T11:28:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3b8e7c700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b406da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:57716/0 (socket says 172.21.15.45:57716) 2023-08-27T11:28:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.159+0000 7fb3b3fff700 1 -- 172.21.15.45:0/1514924893 learned_addr learned my addr 172.21.15.45:0/1514924893 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b8e7c700 1 -- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3b4159a20 msgr2=0x7fb3b41f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b8e7c700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3b4159a20 0x7fb3b41f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b8e7c700 1 -- 172.21.15.45:0/1514924893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3b40b2ef0 msgr2=0x7fb3b4157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b8e7c700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3b40b2ef0 0x7fb3b4157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b8e7c700 1 -- 172.21.15.45:0/1514924893 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac00ab80 con 0x7fb3b4072670 2023-08-27T11:28:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b3fff700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3b40b2ef0 0x7fb3b4157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b967d700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3b4159a20 0x7fb3b41f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b8e7c700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b406da30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c3c0 tx=0x7fb3a800c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b17fa700 1 -- 172.21.15.45:0/1514924893 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80052d0 con 0x7fb3b4072670 2023-08-27T11:28:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b17fa700 1 -- 172.21.15.45:0/1514924893 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a8005430 con 0x7fb3b4072670 2023-08-27T11:28:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3b17fa700 1 -- 172.21.15.45:0/1514924893 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8003880 con 0x7fb3b4072670 2023-08-27T11:28:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b41f37a0 con 0x7fb3b4072670 2023-08-27T11:28:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.160+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b41f3d60 con 0x7fb3b4072670 2023-08-27T11:28:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.161+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b41f3f20 con 0x7fb3b4072670 2023-08-27T11:28:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.162+0000 7fb3b17fa700 1 -- 172.21.15.45:0/1514924893 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fb3a80039e0 con 0x7fb3b4072670 2023-08-27T11:28:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.162+0000 7fb3b17fa700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb39806a200 0x7fb39806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.162+0000 7fb3b3fff700 1 -- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb39806a200 msgr2=0x7fb39806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.162+0000 7fb3b17fa700 1 -- 172.21.15.45:0/1514924893 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb3a8087bd0 con 0x7fb3b4072670 2023-08-27T11:28:32.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.162+0000 7fb3b3fff700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb39806a200 0x7fb39806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:32.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.166+0000 7fb3b17fa700 1 -- 172.21.15.45:0/1514924893 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb3a8052f70 con 0x7fb3b4072670 2023-08-27T11:28:32.301 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:32.299+0000 7fee539ef200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:28:32.359 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:32.358+0000 7fc8dce40200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:28:32.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.363+0000 7fb3b3fff700 1 -- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb39806a200 msgr2=0x7fb39806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:32.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.363+0000 7fb3b3fff700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb39806a200 0x7fb39806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.367+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3b409d5b0 con 0x7fb3b4072670 2023-08-27T11:28:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.367+0000 7fb3b17fa700 1 -- 172.21.15.45:0/1514924893 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3a8017020 con 0x7fb3b4072670 2023-08-27T11:28:32.368 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:28:32.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb39806a200 msgr2=0x7fb39806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:32.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb39806a200 0x7fb39806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 msgr2=0x7fb3b406da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b406da30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800c3c0 tx=0x7fb3a800c6d0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 shutdown_connections 2023-08-27T11:28:32.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7fb39806a200 0x7fb39806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3b4159a20 0x7fb3b41f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb3b4072670 0x7fb3b406da30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 --2- 172.21.15.45:0/1514924893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3b40b2ef0 0x7fb3b4157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.369+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 >> 172.21.15.45:0/1514924893 conn(0x7fb3b40b23d0 msgr2=0x7fb3b4153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.370+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 shutdown_connections 2023-08-27T11:28:32.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.370+0000 7fb3bb0e0700 1 -- 172.21.15.45:0/1514924893 wait complete. 2023-08-27T11:28:32.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:32.451 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:32.448+0000 7fee539ef200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:28:32.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.512+0000 7f0f88f9d700 1 Processor -- start 2023-08-27T11:28:32.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.512+0000 7f0f88f9d700 1 -- start start 2023-08-27T11:28:32.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.513+0000 7f0f88f9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f8415ae00 0x7f0f840ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.513+0000 7f0f88f9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f8406d010 0x7f0f840cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.513+0000 7f0f88f9d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f840b33a0 con 0x7f0f840d0880 2023-08-27T11:28:32.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.513+0000 7f0f88f9d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f840b0610 con 0x7f0f8415ae00 2023-08-27T11:28:32.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.513+0000 7f0f88f9d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f840b2790 con 0x7f0f8406d010 2023-08-27T11:28:32.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.513+0000 7f0f837fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f8415ae00 0x7f0f840ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.513+0000 7f0f82ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0f840d0880 0x7f0f8406c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46594/0 (socket says 172.21.15.45:46594) 2023-08-27T11:28:32.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.513+0000 7f0f82ffd700 1 -- 172.21.15.45:0/914603888 learned_addr learned my addr 172.21.15.45:0/914603888 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:32.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.513+0000 7f0f83fff700 1 --2- 172.21.15.45:0/914603888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f8406d010 0x7f0f840cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f837fe700 1 -- 172.21.15.45:0/914603888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f8406d010 msgr2=0x7f0f840cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:32.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f837fe700 1 --2- 172.21.15.45:0/914603888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f8406d010 0x7f0f840cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f837fe700 1 -- 172.21.15.45:0/914603888 >> v1:172.21.15.45:6789/0 conn(0x7f0f840d0880 legacy=0x7f0f8406c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f837fe700 1 -- 172.21.15.45:0/914603888 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f840b2a70 con 0x7f0f8415ae00 2023-08-27T11:28:32.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f837fe700 1 --2- 172.21.15.45:0/914603888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8415ae00 0x7f0f840ce850 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009840 tx=0x7f0f74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbe9ec4b85ae1c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f827fc700 1 -- 172.21.15.45:0/914603888 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74011830 con 0x7f0f8415ae00 2023-08-27T11:28:32.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f827fc700 1 -- 172.21.15.45:0/914603888 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f74011990 con 0x7f0f8415ae00 2023-08-27T11:28:32.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f827fc700 1 -- 172.21.15.45:0/914603888 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74011d20 con 0x7f0f8415ae00 2023-08-27T11:28:32.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f88f9d700 1 -- 172.21.15.45:0/914603888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8415ae00 msgr2=0x7f0f840ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f88f9d700 1 --2- 172.21.15.45:0/914603888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8415ae00 0x7f0f840ce850 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009840 tx=0x7f0f74009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f88f9d700 1 -- 172.21.15.45:0/914603888 shutdown_connections 2023-08-27T11:28:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f88f9d700 1 --2- 172.21.15.45:0/914603888 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f8406d010 0x7f0f840cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f88f9d700 1 --2- 172.21.15.45:0/914603888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8415ae00 0x7f0f840ce850 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.514+0000 7f0f88f9d700 1 -- 172.21.15.45:0/914603888 >> 172.21.15.45:0/914603888 conn(0x7f0f8406ddd0 msgr2=0x7f0f8406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.515+0000 7f0f88f9d700 1 -- 172.21.15.45:0/914603888 shutdown_connections 2023-08-27T11:28:32.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.515+0000 7f0f88f9d700 1 -- 172.21.15.45:0/914603888 wait complete. 2023-08-27T11:28:32.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.516+0000 7f0f88f9d700 1 Processor -- start 2023-08-27T11:28:32.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.516+0000 7f0f88f9d700 1 -- start start 2023-08-27T11:28:32.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f88f9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8406d010 0x7f0f841486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f88f9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f840d0880 0x7f0f840ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f88f9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f8415ae00 0x7f0f840b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f88f9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f84148bf0 con 0x7f0f8406d010 2023-08-27T11:28:32.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f88f9d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f840b5d60 con 0x7f0f8415ae00 2023-08-27T11:28:32.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f88f9d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f840b6040 con 0x7f0f840d0880 2023-08-27T11:28:32.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8406d010 0x7f0f841486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8406d010 0x7f0f841486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59822/0 (socket says 172.21.15.45:59822) 2023-08-27T11:28:32.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f837fe700 1 -- 172.21.15.45:0/2743298064 learned_addr learned my addr 172.21.15.45:0/2743298064 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:32.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f82ffd700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f840d0880 0x7f0f840ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f83fff700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f8415ae00 0x7f0f840b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f837fe700 1 -- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f840d0880 msgr2=0x7f0f840ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f837fe700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f840d0880 0x7f0f840ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f837fe700 1 -- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f8415ae00 msgr2=0x7f0f840b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f837fe700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f8415ae00 0x7f0f840b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f837fe700 1 -- 172.21.15.45:0/2743298064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f740094a0 con 0x7f0f8406d010 2023-08-27T11:28:32.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f83fff700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f8415ae00 0x7f0f840b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:32.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f837fe700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8406d010 0x7f0f841486b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0f74000c00 tx=0x7f0f74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f6bfff700 1 -- 172.21.15.45:0/2743298064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f740036a0 con 0x7f0f8406d010 2023-08-27T11:28:32.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f6bfff700 1 -- 172.21.15.45:0/2743298064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7401e070 con 0x7f0f8406d010 2023-08-27T11:28:32.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f6bfff700 1 -- 172.21.15.45:0/2743298064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74003a60 con 0x7f0f8406d010 2023-08-27T11:28:32.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f840b6320 con 0x7f0f8406d010 2023-08-27T11:28:32.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.517+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f840b68e0 con 0x7f0f8406d010 2023-08-27T11:28:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.518+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8406e650 con 0x7f0f8406d010 2023-08-27T11:28:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.519+0000 7f0f6bfff700 1 -- 172.21.15.45:0/2743298064 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f0f74003bc0 con 0x7f0f8406d010 2023-08-27T11:28:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.519+0000 7f0f6bfff700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f0f7006a200 0x7f0f7006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.519+0000 7f0f6bfff700 1 -- 172.21.15.45:0/2743298064 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0f74089920 con 0x7f0f8406d010 2023-08-27T11:28:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.519+0000 7f0f82ffd700 1 -- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f0f7006a200 msgr2=0x7f0f7006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.519+0000 7f0f82ffd700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f0f7006a200 0x7f0f7006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.524+0000 7f0f6bfff700 1 -- 172.21.15.45:0/2743298064 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0f74054ca0 con 0x7f0f8406d010 2023-08-27T11:28:32.565 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:32.563+0000 7fee539ef200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:28:32.602 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:32.601+0000 7fc8dce40200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:28:32.720 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:32.718+0000 7fee539ef200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:28:32.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.720+0000 7f0f82ffd700 1 -- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f0f7006a200 msgr2=0x7f0f7006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:32.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.720+0000 7f0f82ffd700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f0f7006a200 0x7f0f7006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.723+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0f840d0610 con 0x7f0f8406d010 2023-08-27T11:28:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.724+0000 7f0f6bfff700 1 -- 172.21.15.45:0/2743298064 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v102) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0f740582c0 con 0x7f0f8406d010 2023-08-27T11:28:32.725 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:28:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f0f7006a200 msgr2=0x7f0f7006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f0f7006a200 0x7f0f7006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8406d010 msgr2=0x7f0f841486b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8406d010 0x7f0f841486b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0f74000c00 tx=0x7f0f74004750 comp rx=0 tx=0).stop 2023-08-27T11:28:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 shutdown_connections 2023-08-27T11:28:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f0f7006a200 0x7f0f7006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f8406d010 0x7f0f841486b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f8415ae00 0x7f0f840b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 --2- 172.21.15.45:0/2743298064 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0f840d0880 0x7f0f840ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 >> 172.21.15.45:0/2743298064 conn(0x7f0f8406ddd0 msgr2=0x7f0f840b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 shutdown_connections 2023-08-27T11:28:32.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.726+0000 7f0f88f9d700 1 -- 172.21.15.45:0/2743298064 wait complete. 2023-08-27T11:28:32.740 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:28:32.763 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:32.762+0000 7fc8dce40200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:28:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.869+0000 7f348df06700 1 Processor -- start 2023-08-27T11:28:32.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.869+0000 7f348df06700 1 -- start start 2023-08-27T11:28:32.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f348df06700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f34880d02c0 0x7f348806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f348df06700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880b0320 0x7f348806c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f348df06700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348815d780 con 0x7f348815d3a0 2023-08-27T11:28:32.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f348df06700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34880cfe30 con 0x7f34880d02c0 2023-08-27T11:28:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f348df06700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34880cf220 con 0x7f34880b0320 2023-08-27T11:28:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f3486ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f34880d02c0 0x7f348806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f3486ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f34880d02c0 0x7f348806bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59834/0 (socket says 172.21.15.45:59834) 2023-08-27T11:28:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f34877fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f348815d3a0 0x7f34880d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46608/0 (socket says 172.21.15.45:46608) 2023-08-27T11:28:32.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f3486ffd700 1 -- 172.21.15.45:0/2228824452 learned_addr learned my addr 172.21.15.45:0/2228824452 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.870+0000 7f3487fff700 1 --2- 172.21.15.45:0/2228824452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880b0320 0x7f348806c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.871+0000 7f3486ffd700 1 -- 172.21.15.45:0/2228824452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880b0320 msgr2=0x7f348806c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.871+0000 7f3486ffd700 1 --2- 172.21.15.45:0/2228824452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880b0320 0x7f348806c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.871+0000 7f3486ffd700 1 -- 172.21.15.45:0/2228824452 >> v1:172.21.15.45:6789/0 conn(0x7f348815d3a0 legacy=0x7f34880d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.871+0000 7f3486ffd700 1 -- 172.21.15.45:0/2228824452 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34880cf500 con 0x7f34880d02c0 2023-08-27T11:28:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.871+0000 7f34867fc700 1 -- 172.21.15.45:0/2228824452 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106910314 0 0) 0x7f3478001490 con 0x7f348815d3a0 2023-08-27T11:28:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.871+0000 7f3486ffd700 1 --2- 172.21.15.45:0/2228824452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880d02c0 0x7f348806bda0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f347c0098f0 tx=0x7f347c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81941ba63ef84071 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.871+0000 7f34867fc700 1 -- 172.21.15.45:0/2228824452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c011830 con 0x7f34880d02c0 2023-08-27T11:28:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.871+0000 7f34867fc700 1 -- 172.21.15.45:0/2228824452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c011990 con 0x7f34880d02c0 2023-08-27T11:28:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.871+0000 7f34867fc700 1 -- 172.21.15.45:0/2228824452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c011d20 con 0x7f34880d02c0 2023-08-27T11:28:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.872+0000 7f348df06700 1 -- 172.21.15.45:0/2228824452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880d02c0 msgr2=0x7f348806bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.872+0000 7f348df06700 1 --2- 172.21.15.45:0/2228824452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880d02c0 0x7f348806bda0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f347c0098f0 tx=0x7f347c009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.872+0000 7f348df06700 1 -- 172.21.15.45:0/2228824452 shutdown_connections 2023-08-27T11:28:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.872+0000 7f348df06700 1 --2- 172.21.15.45:0/2228824452 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880b0320 0x7f348806c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.872+0000 7f348df06700 1 --2- 172.21.15.45:0/2228824452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880d02c0 0x7f348806bda0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.872+0000 7f348df06700 1 -- 172.21.15.45:0/2228824452 >> 172.21.15.45:0/2228824452 conn(0x7f348809a1a0 msgr2=0x7f34880d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.872+0000 7f348df06700 1 -- 172.21.15.45:0/2228824452 shutdown_connections 2023-08-27T11:28:32.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.872+0000 7f348df06700 1 -- 172.21.15.45:0/2228824452 wait complete. 2023-08-27T11:28:32.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.873+0000 7f348df06700 1 Processor -- start 2023-08-27T11:28:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.873+0000 7f348df06700 1 -- start start 2023-08-27T11:28:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f348df06700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34880b0320 0x7f3488154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f348df06700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880d02c0 0x7f34880b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f348df06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348815d3a0 0x7f34880b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f348df06700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34881548c0 con 0x7f348815d3a0 2023-08-27T11:28:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f348df06700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34880c5d30 con 0x7f34880b0320 2023-08-27T11:28:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f348df06700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34880c6010 con 0x7f34880d02c0 2023-08-27T11:28:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3487fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348815d3a0 0x7f34880b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3486ffd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880d02c0 0x7f34880b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3487fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348815d3a0 0x7f34880b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59842/0 (socket says 172.21.15.45:59842) 2023-08-27T11:28:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3486ffd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880d02c0 0x7f34880b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3301/0 says I am v2:172.21.15.45:57750/0 (socket says 172.21.15.45:57750) 2023-08-27T11:28:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3487fff700 1 -- 172.21.15.45:0/1745192329 learned_addr learned my addr 172.21.15.45:0/1745192329 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f34877fe700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34880b0320 0x7f3488154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3486ffd700 1 -- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34880b0320 msgr2=0x7f3488154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3486ffd700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34880b0320 0x7f3488154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3486ffd700 1 -- 172.21.15.45:0/1745192329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348815d3a0 msgr2=0x7f34880b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3486ffd700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348815d3a0 0x7f34880b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.874+0000 7f3486ffd700 1 -- 172.21.15.45:0/1745192329 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c009550 con 0x7f34880d02c0 2023-08-27T11:28:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.875+0000 7f3486ffd700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880d02c0 0x7f34880b23f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f347c016000 tx=0x7f347c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.875+0000 7f346ffff700 1 -- 172.21.15.45:0/1745192329 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c004300 con 0x7f34880d02c0 2023-08-27T11:28:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.875+0000 7f346ffff700 1 -- 172.21.15.45:0/1745192329 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c01f070 con 0x7f34880d02c0 2023-08-27T11:28:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.875+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34880c80b0 con 0x7f34880d02c0 2023-08-27T11:28:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.875+0000 7f346ffff700 1 -- 172.21.15.45:0/1745192329 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0036a0 con 0x7f34880d02c0 2023-08-27T11:28:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.875+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34880b2930 con 0x7f34880d02c0 2023-08-27T11:28:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.876+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34880c85d0 con 0x7f34880d02c0 2023-08-27T11:28:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.876+0000 7f346ffff700 1 -- 172.21.15.45:0/1745192329 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f347c021030 con 0x7f34880d02c0 2023-08-27T11:28:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.877+0000 7f346ffff700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f347406a200 0x7f347406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.877+0000 7f346ffff700 1 -- 172.21.15.45:0/1745192329 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f347c089cc0 con 0x7f34880d02c0 2023-08-27T11:28:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.877+0000 7f34877fe700 1 -- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f347406a200 msgr2=0x7f347406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.877+0000 7f34877fe700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f347406a200 0x7f347406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:32.881+0000 7f346ffff700 1 -- 172.21.15.45:0/1745192329 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f34880c85d0 con 0x7f34880d02c0 2023-08-27T11:28:32.889 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:32.889+0000 7fc8dce40200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:28:32.983 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:32.981+0000 7fee539ef200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:28:33.055 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:33.054+0000 7fc8dce40200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.078+0000 7f34877fe700 1 -- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f347406a200 msgr2=0x7f347406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.078+0000 7f34877fe700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f347406a200 0x7f347406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.080+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34880c85d0 con 0x7f34880d02c0 2023-08-27T11:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.081+0000 7f346ffff700 1 -- 172.21.15.45:0/1745192329 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34880c85d0 con 0x7f34880d02c0 2023-08-27T11:28:33.082 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f347406a200 msgr2=0x7f347406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f347406a200 0x7f347406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880d02c0 msgr2=0x7f34880b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880d02c0 0x7f34880b23f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f347c016000 tx=0x7f347c004750 comp rx=0 tx=0).stop 2023-08-27T11:28:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 shutdown_connections 2023-08-27T11:28:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f347406a200 0x7f347406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f348815d3a0 0x7f34880b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34880b0320 0x7f3488154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 --2- 172.21.15.45:0/1745192329 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f34880d02c0 0x7f34880b23f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 >> 172.21.15.45:0/1745192329 conn(0x7f348809a1a0 msgr2=0x7f348815f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 shutdown_connections 2023-08-27T11:28:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.083+0000 7f348df06700 1 -- 172.21.15.45:0/1745192329 wait complete. 2023-08-27T11:28:33.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:28:33.144 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.142+0000 7fee539ef200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:28:33.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.270+0000 7f2ae9fd7700 1 Processor -- start 2023-08-27T11:28:33.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.270+0000 7f2ae9fd7700 1 -- start start 2023-08-27T11:28:33.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae9fd7700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2ae415d7d0 0x7f2ae4070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae9fd7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ae4161130 0x7f2ae40cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae9fd7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae40d0bf0 con 0x7f2ae40cd770 2023-08-27T11:28:33.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae9fd7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae4162040 con 0x7f2ae4161130 2023-08-27T11:28:33.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae9fd7700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae41617a0 con 0x7f2ae415d7d0 2023-08-27T11:28:33.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae3fff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ae4161130 0x7f2ae40cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae37fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2ae40cd770 0x7f2ae40cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46624/0 (socket says 172.21.15.45:46624) 2023-08-27T11:28:33.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae37fe700 1 -- 172.21.15.45:0/3011883193 learned_addr learned my addr 172.21.15.45:0/3011883193 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:33.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae2ffd700 1 --2- 172.21.15.45:0/3011883193 >> v2:172.21.15.135:3300/0 conn(0x7f2ae415d7d0 0x7f2ae4070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae3fff700 1 -- 172.21.15.45:0/3011883193 >> v2:172.21.15.135:3300/0 conn(0x7f2ae415d7d0 msgr2=0x7f2ae4070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae3fff700 1 --2- 172.21.15.45:0/3011883193 >> v2:172.21.15.135:3300/0 conn(0x7f2ae415d7d0 0x7f2ae4070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae3fff700 1 -- 172.21.15.45:0/3011883193 >> v1:172.21.15.45:6789/0 conn(0x7f2ae40cd770 legacy=0x7f2ae40cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae3fff700 1 -- 172.21.15.45:0/3011883193 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae4161a80 con 0x7f2ae4161130 2023-08-27T11:28:33.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae3fff700 1 --2- 172.21.15.45:0/3011883193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae4161130 0x7f2ae40cb4a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40afb70 tx=0x7f2adc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b977048383c4ee2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.271+0000 7f2ae27fc700 1 -- 172.21.15.45:0/3011883193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00e070 con 0x7f2ae4161130 2023-08-27T11:28:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae27fc700 1 -- 172.21.15.45:0/3011883193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc00bb20 con 0x7f2ae4161130 2023-08-27T11:28:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae27fc700 1 -- 172.21.15.45:0/3011883193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc012430 con 0x7f2ae4161130 2023-08-27T11:28:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/3011883193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae4161130 msgr2=0x7f2ae40cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae9fd7700 1 --2- 172.21.15.45:0/3011883193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae4161130 0x7f2ae40cb4a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40afb70 tx=0x7f2adc00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:33.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/3011883193 shutdown_connections 2023-08-27T11:28:33.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae9fd7700 1 --2- 172.21.15.45:0/3011883193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae4161130 0x7f2ae40cb4a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae9fd7700 1 --2- 172.21.15.45:0/3011883193 >> v2:172.21.15.135:3300/0 conn(0x7f2ae415d7d0 0x7f2ae4070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/3011883193 >> 172.21.15.45:0/3011883193 conn(0x7f2ae406f960 msgr2=0x7f2ae406fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/3011883193 shutdown_connections 2023-08-27T11:28:33.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.272+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/3011883193 wait complete. 2023-08-27T11:28:33.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.273+0000 7f2ae9fd7700 1 Processor -- start 2023-08-27T11:28:33.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.274+0000 7f2ae9fd7700 1 -- start start 2023-08-27T11:28:33.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.274+0000 7f2ae9fd7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae40cd770 0x7f2ae40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.274+0000 7f2ae9fd7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae415d7d0 0x7f2ae415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.274+0000 7f2ae9fd7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2ae4161130 0x7f2ae406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.274+0000 7f2ae9fd7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae415a1a0 con 0x7f2ae415d7d0 2023-08-27T11:28:33.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.274+0000 7f2ae9fd7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae415c830 con 0x7f2ae40cd770 2023-08-27T11:28:33.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.274+0000 7f2ae9fd7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40b22f0 con 0x7f2ae4161130 2023-08-27T11:28:33.283 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.274+0000 7f2ae2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae415d7d0 0x7f2ae415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.274+0000 7f2ae2ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae415d7d0 0x7f2ae415c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59856/0 (socket says 172.21.15.45:59856) 2023-08-27T11:28:33.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae2ffd700 1 -- 172.21.15.45:0/606264007 learned_addr learned my addr 172.21.15.45:0/606264007 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:33.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae37fe700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae40cd770 0x7f2ae40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae3fff700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2ae4161130 0x7f2ae406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.284 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae2ffd700 1 -- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2ae4161130 msgr2=0x7f2ae406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae2ffd700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2ae4161130 0x7f2ae406c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae2ffd700 1 -- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae40cd770 msgr2=0x7f2ae40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae2ffd700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae40cd770 0x7f2ae40c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae2ffd700 1 -- 172.21.15.45:0/606264007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc009a70 con 0x7f2ae415d7d0 2023-08-27T11:28:33.285 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae2ffd700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae415d7d0 0x7f2ae415c2f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800e9b0 tx=0x7f2ad800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2acbfff700 1 -- 172.21.15.45:0/606264007 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad80096b0 con 0x7f2ae415d7d0 2023-08-27T11:28:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2acbfff700 1 -- 172.21.15.45:0/606264007 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad800ee50 con 0x7f2ae415d7d0 2023-08-27T11:28:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae40b25d0 con 0x7f2ae415d7d0 2023-08-27T11:28:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2acbfff700 1 -- 172.21.15.45:0/606264007 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad800c9f0 con 0x7f2ae415d7d0 2023-08-27T11:28:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.275+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae40b2b90 con 0x7f2ae415d7d0 2023-08-27T11:28:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.276+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae404e620 con 0x7f2ae415d7d0 2023-08-27T11:28:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.277+0000 7f2acbfff700 1 -- 172.21.15.45:0/606264007 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f2ad800cb50 con 0x7f2ae415d7d0 2023-08-27T11:28:33.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.277+0000 7f2acbfff700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f2ad006a200 0x7f2ad006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.277+0000 7f2ae37fe700 1 -- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f2ad006a200 msgr2=0x7f2ad006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.277+0000 7f2acbfff700 1 -- 172.21.15.45:0/606264007 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2ad8088920 con 0x7f2ae415d7d0 2023-08-27T11:28:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.277+0000 7f2ae37fe700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f2ad006a200 0x7f2ad006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:33.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.286+0000 7f2acbfff700 1 -- 172.21.15.45:0/606264007 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2ad80576a0 con 0x7f2ae415d7d0 2023-08-27T11:28:33.350 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:33.349+0000 7fc8dce40200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:28:33.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.478+0000 7f2ae37fe700 1 -- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f2ad006a200 msgr2=0x7f2ad006c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:33.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.478+0000 7f2ae37fe700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f2ad006a200 0x7f2ad006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:28:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.496+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ae4159f70 con 0x7f2ae415d7d0 2023-08-27T11:28:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.496+0000 7f2acbfff700 1 -- 172.21.15.45:0/606264007 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ad8057230 con 0x7f2ae415d7d0 2023-08-27T11:28:33.498 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:28:33.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f2ad006a200 msgr2=0x7f2ad006c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:33.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f2ad006a200 0x7f2ad006c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae415d7d0 msgr2=0x7f2ae415c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae415d7d0 0x7f2ae415c2f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800e9b0 tx=0x7f2ad800ecc0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 shutdown_connections 2023-08-27T11:28:33.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f2ad006a200 0x7f2ad006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2ae4161130 0x7f2ae406c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae40cd770 0x7f2ae40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 --2- 172.21.15.45:0/606264007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae415d7d0 0x7f2ae415c2f0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 >> 172.21.15.45:0/606264007 conn(0x7f2ae406f960 msgr2=0x7f2ae41580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 shutdown_connections 2023-08-27T11:28:33.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.499+0000 7f2ae9fd7700 1 -- 172.21.15.45:0/606264007 wait complete. 2023-08-27T11:28:33.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:33.521 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:33.521+0000 7fc8dce40200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:28:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.640+0000 7f611d1fe700 1 Processor -- start 2023-08-27T11:28:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.640+0000 7f611d1fe700 1 -- start start 2023-08-27T11:28:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611d1fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f611815d140 0x7f61180ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611d1fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f611806bda0 0x7f611806c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611d1fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f61180cc790 0x7f61180ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611d1fe700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61180cff50 con 0x7f61180cc790 2023-08-27T11:28:33.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611d1fe700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61180cf0a0 con 0x7f611815d140 2023-08-27T11:28:33.644 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611d1fe700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611806c760 con 0x7f611806bda0 2023-08-27T11:28:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611759e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f61180cc790 0x7f61180ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611759e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f61180cc790 0x7f61180ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59868/0 (socket says 172.21.15.45:59868) 2023-08-27T11:28:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611759e700 1 -- 172.21.15.45:0/1834117637 learned_addr learned my addr 172.21.15.45:0/1834117637 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611659c700 1 --2- 172.21.15.45:0/1834117637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f611806bda0 0x7f611806c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f6116d9d700 1 --2- 172.21.15.45:0/1834117637 >> v2:172.21.15.135:3300/0 conn(0x7f611815d140 0x7f61180ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611759e700 1 -- 172.21.15.45:0/1834117637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f611806bda0 msgr2=0x7f611806c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:33.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611759e700 1 --2- 172.21.15.45:0/1834117637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f611806bda0 0x7f611806c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611759e700 1 -- 172.21.15.45:0/1834117637 >> v2:172.21.15.135:3300/0 conn(0x7f611815d140 msgr2=0x7f61180ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:33.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611759e700 1 --2- 172.21.15.45:0/1834117637 >> v2:172.21.15.135:3300/0 conn(0x7f611815d140 0x7f61180ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611759e700 1 -- 172.21.15.45:0/1834117637 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61180cf340 con 0x7f61180cc790 2023-08-27T11:28:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.641+0000 7f611759e700 1 --2- 172.21.15.45:0/1834117637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61180cc790 0x7f61180ccc20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f611808b450 tx=0x7f611000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f440407ffb97f1ad server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f6115d9b700 1 -- 172.21.15.45:0/1834117637 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611000e070 con 0x7f61180cc790 2023-08-27T11:28:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f6115d9b700 1 -- 172.21.15.45:0/1834117637 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611000b910 con 0x7f61180cc790 2023-08-27T11:28:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f6115d9b700 1 -- 172.21.15.45:0/1834117637 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611000bcc0 con 0x7f61180cc790 2023-08-27T11:28:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f611d1fe700 1 -- 172.21.15.45:0/1834117637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61180cc790 msgr2=0x7f61180ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f611d1fe700 1 --2- 172.21.15.45:0/1834117637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61180cc790 0x7f61180ccc20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f611808b450 tx=0x7f611000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f611d1fe700 1 -- 172.21.15.45:0/1834117637 shutdown_connections 2023-08-27T11:28:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f611d1fe700 1 --2- 172.21.15.45:0/1834117637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61180cc790 0x7f61180ccc20 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f611d1fe700 1 --2- 172.21.15.45:0/1834117637 >> v2:172.21.15.135:3300/0 conn(0x7f611815d140 0x7f61180ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f611d1fe700 1 --2- 172.21.15.45:0/1834117637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f611806bda0 0x7f611806c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f611d1fe700 1 -- 172.21.15.45:0/1834117637 >> 172.21.15.45:0/1834117637 conn(0x7f611809a160 msgr2=0x7f61180d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f611d1fe700 1 -- 172.21.15.45:0/1834117637 shutdown_connections 2023-08-27T11:28:33.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.642+0000 7f611d1fe700 1 -- 172.21.15.45:0/1834117637 wait complete. 2023-08-27T11:28:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.644+0000 7f611d1fe700 1 Processor -- start 2023-08-27T11:28:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.644+0000 7f611d1fe700 1 -- start start 2023-08-27T11:28:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.644+0000 7f611d1fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f611806bda0 0x7f61180ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.644+0000 7f611d1fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61180cc790 0x7f611815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611d1fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f611815d140 0x7f611814e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611d1fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61180ae920 con 0x7f611815d140 2023-08-27T11:28:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611d1fe700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611815ff10 con 0x7f611806bda0 2023-08-27T11:28:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611d1fe700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611815ef80 con 0x7f61180cc790 2023-08-27T11:28:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f611815d140 0x7f611814e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f611815d140 0x7f611814e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59872/0 (socket says 172.21.15.45:59872) 2023-08-27T11:28:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611759e700 1 -- 172.21.15.45:0/1583381471 learned_addr learned my addr 172.21.15.45:0/1583381471 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f6116d9d700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f611806bda0 0x7f61180ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611659c700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61180cc790 0x7f611815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611759e700 1 -- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61180cc790 msgr2=0x7f611815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611759e700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61180cc790 0x7f611815f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611759e700 1 -- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f611806bda0 msgr2=0x7f61180ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611759e700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f611806bda0 0x7f61180ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611759e700 1 -- 172.21.15.45:0/1583381471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6110009b30 con 0x7f611815d140 2023-08-27T11:28:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611759e700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f611815d140 0x7f611814e1c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f611000b680 tx=0x7f6110004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f60ff7fe700 1 -- 172.21.15.45:0/1583381471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61100099b0 con 0x7f611815d140 2023-08-27T11:28:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f60ff7fe700 1 -- 172.21.15.45:0/1583381471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611000ce80 con 0x7f611815d140 2023-08-27T11:28:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611d1fe700 1 -- 172.21.15.45:0/1583381471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611815f540 con 0x7f611815d140 2023-08-27T11:28:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f60ff7fe700 1 -- 172.21.15.45:0/1583381471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110004490 con 0x7f611815d140 2023-08-27T11:28:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.645+0000 7f611d1fe700 1 -- 172.21.15.45:0/1583381471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61181e6ff0 con 0x7f611815d140 2023-08-27T11:28:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.646+0000 7f611d1fe700 1 -- 172.21.15.45:0/1583381471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611815f6d0 con 0x7f611815d140 2023-08-27T11:28:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.647+0000 7f60ff7fe700 1 -- 172.21.15.45:0/1583381471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 42) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f61100045f0 con 0x7f611815d140 2023-08-27T11:28:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.648+0000 7f60ff7fe700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f610406a200 0x7f610406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:33.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.648+0000 7f60ff7fe700 1 -- 172.21.15.45:0/1583381471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6110089ad0 con 0x7f611815d140 2023-08-27T11:28:33.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.648+0000 7f6116d9d700 1 -- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f610406a200 msgr2=0x7f610406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3112377367 2023-08-27T11:28:33.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.648+0000 7f6116d9d700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f610406a200 0x7f610406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:28:33.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.652+0000 7f60ff7fe700 1 -- 172.21.15.45:0/1583381471 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6110054d50 con 0x7f611815d140 2023-08-27T11:28:33.673 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.671+0000 7fee539ef200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:28:33.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.696+0000 7f60ff7fe700 1 -- 172.21.15.45:0/1583381471 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 43) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f6110055120 con 0x7f611815d140 2023-08-27T11:28:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.697+0000 7f60ff7fe700 1 -- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f610406a200 msgr2=0x7f610406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:28:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.697+0000 7f60ff7fe700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f610406a200 0x7f610406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.755 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedcd25a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.755 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedcd25a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.756 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedcd25a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.756 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedcd25a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.756 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedcd25a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.756 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedd02a0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.756 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedd02a0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.757 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedd02a0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.757 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedd02a0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.757 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:33.753+0000 7fedd02a0700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:28:33.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.908+0000 7f611d1fe700 1 -- 172.21.15.45:0/1583381471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f611806c490 con 0x7f611815d140 2023-08-27T11:28:33.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.909+0000 7f60ff7fe700 1 -- 172.21.15.45:0/1583381471 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f611006f0f0 con 0x7f611815d140 2023-08-27T11:28:33.910 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:33.910 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3604468746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3604468746},{"type":"v1","addr":"172.21.15.135:6836","nonce":3604468746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1319491874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1319491874},{"type":"v1","addr":"172.21.15.135:6837","nonce":1319491874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6937,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2036943131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2036943131},{"type":"v1","addr":"172.21.15.45:6835","nonce":2036943131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:07.092667+0000","modified":"2023-08-27T11:28:10.679683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7446":{"gid":7446,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/3257398445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3257398445},{"type":"v1","addr":"172.21.15.45:6833","nonce":3257398445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:28:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 -- 172.21.15.45:0/1583381471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f611815d140 msgr2=0x7f611814e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f611815d140 0x7f611814e1c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f611000b680 tx=0x7f6110004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 -- 172.21.15.45:0/1583381471 shutdown_connections 2023-08-27T11:28:33.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:6832/3112377367,v1:172.21.15.135:6833/3112377367] conn(0x7f610406a200 0x7f610406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f611815d140 0x7f611814e1c0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f611806bda0 0x7f61180ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 --2- 172.21.15.45:0/1583381471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f61180cc790 0x7f611815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:33.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 -- 172.21.15.45:0/1583381471 >> 172.21.15.45:0/1583381471 conn(0x7f611809a160 msgr2=0x7f611806b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 -- 172.21.15.45:0/1583381471 shutdown_connections 2023-08-27T11:28:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:33.911+0000 7f611d1fe700 1 -- 172.21.15.45:0/1583381471 wait complete. 2023-08-27T11:28:33.914 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 41 2023-08-27T11:28:33.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:34.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.089+0000 7f0128c68700 1 Processor -- start 2023-08-27T11:28:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.089+0000 7f0128c68700 1 -- start start 2023-08-27T11:28:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.090+0000 7f0128c68700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f012415b120 0x7f01240cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.090+0000 7f0128c68700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01240ada50 con 0x7f01240cdb80 2023-08-27T11:28:34.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.090+0000 7f0128c68700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01240ccd60 con 0x7f01240d1a20 2023-08-27T11:28:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.090+0000 7f0128c68700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01240b0860 con 0x7f012415b120 2023-08-27T11:28:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f0122d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f01240cdb80 0x7f01240b1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46656/0 (socket says 172.21.15.45:46656) 2023-08-27T11:28:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f0122d9d700 1 -- 172.21.15.45:0/1755599568 learned_addr learned my addr 172.21.15.45:0/1755599568 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f012259c700 1 --2- 172.21.15.45:0/1755599568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f012415b120 0x7f01240cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f012159a700 1 -- 172.21.15.45:0/1755599568 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927598018 0 0) 0x7f01240ada50 con 0x7f01240cdb80 2023-08-27T11:28:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f012159a700 1 -- 172.21.15.45:0/1755599568 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f010c0035b0 con 0x7f01240cdb80 2023-08-27T11:28:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f012159a700 1 -- 172.21.15.45:0/1755599568 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842537646 0 0) 0x7f01240ccd60 con 0x7f01240d1a20 2023-08-27T11:28:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f012159a700 1 -- 172.21.15.45:0/1755599568 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01240ada50 con 0x7f01240d1a20 2023-08-27T11:28:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f012259c700 1 -- 172.21.15.45:0/1755599568 >> v1:172.21.15.135:6789/0 conn(0x7f01240d1a20 legacy=0x7f01240cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f012259c700 1 -- 172.21.15.45:0/1755599568 >> v1:172.21.15.45:6789/0 conn(0x7f01240cdb80 legacy=0x7f01240b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f012259c700 1 -- 172.21.15.45:0/1755599568 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01240b0b40 con 0x7f012415b120 2023-08-27T11:28:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.091+0000 7f012259c700 1 --2- 172.21.15.45:0/1755599568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f012415b120 0x7f01240cc430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0114009870 tx=0x7f0114009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121aab20e147aec3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f012159a700 1 -- 172.21.15.45:0/1755599568 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114011830 con 0x7f012415b120 2023-08-27T11:28:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f012159a700 1 -- 172.21.15.45:0/1755599568 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0114011990 con 0x7f012415b120 2023-08-27T11:28:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f012159a700 1 -- 172.21.15.45:0/1755599568 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114011d20 con 0x7f012415b120 2023-08-27T11:28:34.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f0128c68700 1 -- 172.21.15.45:0/1755599568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f012415b120 msgr2=0x7f01240cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f0128c68700 1 --2- 172.21.15.45:0/1755599568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f012415b120 0x7f01240cc430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0114009870 tx=0x7f0114009b80 comp rx=0 tx=0).stop 2023-08-27T11:28:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f0128c68700 1 -- 172.21.15.45:0/1755599568 shutdown_connections 2023-08-27T11:28:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f0128c68700 1 --2- 172.21.15.45:0/1755599568 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f012415b120 0x7f01240cc430 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f0128c68700 1 -- 172.21.15.45:0/1755599568 >> 172.21.15.45:0/1755599568 conn(0x7f012409a160 msgr2=0x7f01240d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f0128c68700 1 -- 172.21.15.45:0/1755599568 shutdown_connections 2023-08-27T11:28:34.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.092+0000 7f0128c68700 1 -- 172.21.15.45:0/1755599568 wait complete. 2023-08-27T11:28:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.094+0000 7f0128c68700 1 Processor -- start 2023-08-27T11:28:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.094+0000 7f0128c68700 1 -- start start 2023-08-27T11:28:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.094+0000 7f0128c68700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01240cdb80 0x7f0124158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.094+0000 7f0128c68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01240d1a20 0x7f012409c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.094+0000 7f0128c68700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f012415b120 0x7f012409d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.094+0000 7f0128c68700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012409b590 con 0x7f01240d1a20 2023-08-27T11:28:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.094+0000 7f0128c68700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012409b870 con 0x7f012415b120 2023-08-27T11:28:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.094+0000 7f0128c68700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012409bb50 con 0x7f01240cdb80 2023-08-27T11:28:34.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0121d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01240d1a20 0x7f012409c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0121d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01240d1a20 0x7f012409c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59884/0 (socket says 172.21.15.45:59884) 2023-08-27T11:28:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0121d9b700 1 -- 172.21.15.45:0/4262579189 learned_addr learned my addr 172.21.15.45:0/4262579189 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0122d9d700 1 --2- 172.21.15.45:0/4262579189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f012415b120 0x7f012409d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f012259c700 1 --2- 172.21.15.45:0/4262579189 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01240cdb80 0x7f0124158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0121d9b700 1 -- 172.21.15.45:0/4262579189 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01240cdb80 msgr2=0x7f0124158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0121d9b700 1 --2- 172.21.15.45:0/4262579189 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01240cdb80 0x7f0124158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0121d9b700 1 -- 172.21.15.45:0/4262579189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f012415b120 msgr2=0x7f012409d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0121d9b700 1 --2- 172.21.15.45:0/4262579189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f012415b120 0x7f012409d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0121d9b700 1 -- 172.21.15.45:0/4262579189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01140094a0 con 0x7f01240d1a20 2023-08-27T11:28:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0121d9b700 1 --2- 172.21.15.45:0/4262579189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01240d1a20 0x7f012409c630 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f011800c520 tx=0x7f011800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0112ffd700 1 -- 172.21.15.45:0/4262579189 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180052d0 con 0x7f01240d1a20 2023-08-27T11:28:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0112ffd700 1 -- 172.21.15.45:0/4262579189 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0118005430 con 0x7f01240d1a20 2023-08-27T11:28:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0128c68700 1 -- 172.21.15.45:0/4262579189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012409be30 con 0x7f01240d1a20 2023-08-27T11:28:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0112ffd700 1 -- 172.21.15.45:0/4262579189 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180038c0 con 0x7f01240d1a20 2023-08-27T11:28:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.095+0000 7f0128c68700 1 -- 172.21.15.45:0/4262579189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012409cb70 con 0x7f01240d1a20 2023-08-27T11:28:34.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.096+0000 7f0128c68700 1 -- 172.21.15.45:0/4262579189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012409c350 con 0x7f01240d1a20 2023-08-27T11:28:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.097+0000 7f0112ffd700 1 -- 172.21.15.45:0/4262579189 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 43) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f0118003a20 con 0x7f01240d1a20 2023-08-27T11:28:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.097+0000 7f0112ffd700 1 -- 172.21.15.45:0/4262579189 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f0118087850 con 0x7f01240d1a20 2023-08-27T11:28:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.102+0000 7f0112ffd700 1 -- 172.21.15.45:0/4262579189 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0118052f40 con 0x7f01240d1a20 2023-08-27T11:28:34.106 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:28:34.104+0000 7fc8dce40200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:28:34.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.302+0000 7f0128c68700 1 -- 172.21.15.45:0/4262579189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f012409cd00 con 0x7f01240d1a20 2023-08-27T11:28:34.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.304+0000 7f0112ffd700 1 -- 172.21.15.45:0/4262579189 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+19706 (secure 0 0 0) 0x7f0118056560 con 0x7f01240d1a20 2023-08-27T11:28:34.305 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:34.305 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":103,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:33.686782+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:05.410178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:06.051204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/626141668":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/1023575426":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/233288315":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/3946297931":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6833/3112377367":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6832/3112377367":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6833/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/4034194156":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6832/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3142979284":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/569758288":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/129865033":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/823048739":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/1640490812":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/12937390":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/2560756484":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3438903793":"2023-08-27T12:28:16.346002+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.306+0000 7f0128c68700 1 -- 172.21.15.45:0/4262579189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01240d1a20 msgr2=0x7f012409c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.306+0000 7f0128c68700 1 --2- 172.21.15.45:0/4262579189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01240d1a20 0x7f012409c630 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f011800c520 tx=0x7f011800c830 comp rx=0 tx=0).stop 2023-08-27T11:28:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.306+0000 7f0128c68700 1 -- 172.21.15.45:0/4262579189 shutdown_connections 2023-08-27T11:28:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.306+0000 7f0128c68700 1 --2- 172.21.15.45:0/4262579189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f012415b120 0x7f012409d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.306+0000 7f0128c68700 1 --2- 172.21.15.45:0/4262579189 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f01240cdb80 0x7f0124158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.306+0000 7f0128c68700 1 --2- 172.21.15.45:0/4262579189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01240d1a20 0x7f012409c630 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.306+0000 7f0128c68700 1 -- 172.21.15.45:0/4262579189 >> 172.21.15.45:0/4262579189 conn(0x7f012409a160 msgr2=0x7f012415a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.306+0000 7f0128c68700 1 -- 172.21.15.45:0/4262579189 shutdown_connections 2023-08-27T11:28:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.306+0000 7f0128c68700 1 -- 172.21.15.45:0/4262579189 wait complete. 2023-08-27T11:28:34.319 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:28:34.319 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:34.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.443+0000 7f8eb5776700 1 Processor -- start 2023-08-27T11:28:34.445 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.443+0000 7f8eb5776700 1 -- start start 2023-08-27T11:28:34.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eb5776700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8eb006a8b0 0x7f8eb00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eb5776700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8eb00b23c0 0x7f8eb006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eb5776700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb00ce0d0 con 0x7f8eb00b1e70 2023-08-27T11:28:34.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eb5776700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb00b9040 con 0x7f8eb00b23c0 2023-08-27T11:28:34.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eb5776700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb006b1d0 con 0x7f8eb006a8b0 2023-08-27T11:28:34.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eae7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8eb00b23c0 0x7f8eb006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eaf7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8eb00b1e70 0x7f8eb00d0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46670/0 (socket says 172.21.15.45:46670) 2023-08-27T11:28:34.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eae7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8eb00b23c0 0x7f8eb006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59890/0 (socket says 172.21.15.45:59890) 2023-08-27T11:28:34.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eaf7fe700 1 -- 172.21.15.45:0/1986004343 learned_addr learned my addr 172.21.15.45:0/1986004343 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:34.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.444+0000 7f8eaeffd700 1 --2- 172.21.15.45:0/1986004343 >> v2:172.21.15.135:3300/0 conn(0x7f8eb006a8b0 0x7f8eb00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eae7fc700 1 -- 172.21.15.45:0/1986004343 >> v2:172.21.15.135:3300/0 conn(0x7f8eb006a8b0 msgr2=0x7f8eb00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:34.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eae7fc700 1 --2- 172.21.15.45:0/1986004343 >> v2:172.21.15.135:3300/0 conn(0x7f8eb006a8b0 0x7f8eb00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eae7fc700 1 -- 172.21.15.45:0/1986004343 >> v1:172.21.15.45:6789/0 conn(0x7f8eb00b1e70 legacy=0x7f8eb00d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eae7fc700 1 -- 172.21.15.45:0/1986004343 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb00d2a10 con 0x7f8eb00b23c0 2023-08-27T11:28:34.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eadffb700 1 -- 172.21.15.45:0/1986004343 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996017245 0 0) 0x7f8eb00ce0d0 con 0x7f8eb00b1e70 2023-08-27T11:28:34.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eae7fc700 1 --2- 172.21.15.45:0/1986004343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b23c0 0x7f8eb006ac90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8eb008c130 tx=0x7f8ea0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7167a814d3224de server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eadffb700 1 -- 172.21.15.45:0/1986004343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0011830 con 0x7f8eb00b23c0 2023-08-27T11:28:34.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eadffb700 1 -- 172.21.15.45:0/1986004343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea0011990 con 0x7f8eb00b23c0 2023-08-27T11:28:34.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eadffb700 1 -- 172.21.15.45:0/1986004343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0011d20 con 0x7f8eb00b23c0 2023-08-27T11:28:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eb5776700 1 -- 172.21.15.45:0/1986004343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b23c0 msgr2=0x7f8eb006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.445+0000 7f8eb5776700 1 --2- 172.21.15.45:0/1986004343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b23c0 0x7f8eb006ac90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8eb008c130 tx=0x7f8ea0009b20 comp rx=0 tx=0).stop 2023-08-27T11:28:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.446+0000 7f8eb5776700 1 -- 172.21.15.45:0/1986004343 shutdown_connections 2023-08-27T11:28:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.446+0000 7f8eb5776700 1 --2- 172.21.15.45:0/1986004343 >> v2:172.21.15.135:3300/0 conn(0x7f8eb006a8b0 0x7f8eb00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.446+0000 7f8eb5776700 1 --2- 172.21.15.45:0/1986004343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b23c0 0x7f8eb006ac90 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.446+0000 7f8eb5776700 1 -- 172.21.15.45:0/1986004343 >> 172.21.15.45:0/1986004343 conn(0x7f8eb00d3070 msgr2=0x7f8eb00d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.446+0000 7f8eb5776700 1 -- 172.21.15.45:0/1986004343 shutdown_connections 2023-08-27T11:28:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.446+0000 7f8eb5776700 1 -- 172.21.15.45:0/1986004343 wait complete. 2023-08-27T11:28:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.447+0000 7f8eb5776700 1 Processor -- start 2023-08-27T11:28:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.447+0000 7f8eb5776700 1 -- start start 2023-08-27T11:28:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.448+0000 7f8eb5776700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb006a8b0 0x7f8eb00ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.448+0000 7f8eb5776700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b1e70 0x7f8eb00b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.448+0000 7f8eb5776700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b23c0 0x7f8eb00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.448+0000 7f8eb5776700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00b51b0 con 0x7f8eb00b1e70 2023-08-27T11:28:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.448+0000 7f8eb5776700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00b45a0 con 0x7f8eb006a8b0 2023-08-27T11:28:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.448+0000 7f8eb5776700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00b4880 con 0x7f8eb00b23c0 2023-08-27T11:28:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.448+0000 7f8eae7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b1e70 0x7f8eb00b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.448+0000 7f8eae7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b1e70 0x7f8eb00b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59898/0 (socket says 172.21.15.45:59898) 2023-08-27T11:28:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.448+0000 7f8eae7fc700 1 -- 172.21.15.45:0/4117149343 learned_addr learned my addr 172.21.15.45:0/4117149343 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:34.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eaf7fe700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b23c0 0x7f8eb00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eaeffd700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb006a8b0 0x7f8eb00ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eae7fc700 1 -- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b23c0 msgr2=0x7f8eb00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eae7fc700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b23c0 0x7f8eb00cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eae7fc700 1 -- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb006a8b0 msgr2=0x7f8eb00ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eae7fc700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb006a8b0 0x7f8eb00ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eae7fc700 1 -- 172.21.15.45:0/4117149343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea00094a0 con 0x7f8eb00b1e70 2023-08-27T11:28:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eae7fc700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b1e70 0x7f8eb00b3b20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009fd0 tx=0x7f8ea0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8ea77fe700 1 -- 172.21.15.45:0/4117149343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0004510 con 0x7f8eb00b1e70 2023-08-27T11:28:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8ea77fe700 1 -- 172.21.15.45:0/4117149343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea00036a0 con 0x7f8eb00b1e70 2023-08-27T11:28:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb00b4b60 con 0x7f8eb00b1e70 2023-08-27T11:28:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8ea77fe700 1 -- 172.21.15.45:0/4117149343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0003a10 con 0x7f8eb00b1e70 2023-08-27T11:28:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.449+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb00caee0 con 0x7f8eb00b1e70 2023-08-27T11:28:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.450+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb00b5360 con 0x7f8eb00b1e70 2023-08-27T11:28:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.451+0000 7f8ea77fe700 1 -- 172.21.15.45:0/4117149343 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 43) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f8ea0003b70 con 0x7f8eb00b1e70 2023-08-27T11:28:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.451+0000 7f8ea77fe700 1 -- 172.21.15.45:0/4117149343 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f8ea0089820 con 0x7f8eb00b1e70 2023-08-27T11:28:34.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.455+0000 7f8ea77fe700 1 -- 172.21.15.45:0/4117149343 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8eb00b5360 con 0x7f8eb00b1e70 2023-08-27T11:28:34.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.700+0000 7f8ea77fe700 1 -- 172.21.15.45:0/4117149343 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 44) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f8ea0054a60 con 0x7f8eb00b1e70 2023-08-27T11:28:34.702 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.701+0000 7f8ea77fe700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8e90070550 0x7f8e90072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.703+0000 7f8eaeffd700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8e90070550 0x7f8e90072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.704+0000 7f8eaeffd700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8e90070550 0x7f8e90072940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006b20 tx=0x7f8e98008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.707+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8eb00b5360 con 0x7f8eb00b1e70 2023-08-27T11:28:34.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.707+0000 7f8ea77fe700 1 -- 172.21.15.45:0/4117149343 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f8eb00b5360 con 0x7f8eb00b1e70 2023-08-27T11:28:34.708 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:34.709 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3604468746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3604468746},{"type":"v1","addr":"172.21.15.135:6836","nonce":3604468746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1319491874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1319491874},{"type":"v1","addr":"172.21.15.135:6837","nonce":1319491874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6937,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2036943131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2036943131},{"type":"v1","addr":"172.21.15.45:6835","nonce":2036943131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:07.092667+0000","modified":"2023-08-27T11:28:10.679683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7446":{"gid":7446,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/3257398445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3257398445},{"type":"v1","addr":"172.21.15.45:6833","nonce":3257398445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:28:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8e90070550 msgr2=0x7f8e90072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8e90070550 0x7f8e90072940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006b20 tx=0x7f8e98008040 comp rx=0 tx=0).stop 2023-08-27T11:28:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b1e70 msgr2=0x7f8eb00b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b1e70 0x7f8eb00b3b20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009fd0 tx=0x7f8ea0004750 comp rx=0 tx=0).stop 2023-08-27T11:28:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 shutdown_connections 2023-08-27T11:28:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8e90070550 0x7f8e90072940 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8eb00b23c0 0x7f8eb00cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8eb006a8b0 0x7f8eb00ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 --2- 172.21.15.45:0/4117149343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00b1e70 0x7f8eb00b3b20 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 >> 172.21.15.45:0/4117149343 conn(0x7f8eb00d3070 msgr2=0x7f8eb00b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 shutdown_connections 2023-08-27T11:28:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.710+0000 7f8eb5776700 1 -- 172.21.15.45:0/4117149343 wait complete. 2023-08-27T11:28:34.714 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 41 2023-08-27T11:28:34.723 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.847+0000 7f28fe198700 1 Processor -- start 2023-08-27T11:28:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.847+0000 7f28fe198700 1 -- start start 2023-08-27T11:28:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28fe198700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28f80cc650 0x7f28f80ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28fe198700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 0x7f28f80adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28fe198700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f80ccb80 con 0x7f28f815e990 2023-08-27T11:28:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28fe198700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80cce60 con 0x7f28f80cc650 2023-08-27T11:28:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28fe198700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80cd600 con 0x7f28f80adad0 2023-08-27T11:28:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28f80cc650 0x7f28f80ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f6ffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28f80cc650 0x7f28f80ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59920/0 (socket says 172.21.15.45:59920) 2023-08-27T11:28:34.852 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f6ffd700 1 -- 172.21.15.45:0/1332622181 learned_addr learned my addr 172.21.15.45:0/1332622181 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f7fff700 1 --2- 172.21.15.45:0/1332622181 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 0x7f28f80adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f6ffd700 1 -- 172.21.15.45:0/1332622181 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 msgr2=0x7f28f80adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f6ffd700 1 --2- 172.21.15.45:0/1332622181 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 0x7f28f80adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f6ffd700 1 -- 172.21.15.45:0/1332622181 >> v1:172.21.15.45:6789/0 conn(0x7f28f815e990 legacy=0x7f28f80ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f6ffd700 1 -- 172.21.15.45:0/1332622181 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f80cd8e0 con 0x7f28f80cc650 2023-08-27T11:28:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f67fc700 1 -- 172.21.15.45:0/1332622181 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482249137 0 0) 0x7f28f80ccb80 con 0x7f28f815e990 2023-08-27T11:28:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f7fff700 1 --2- 172.21.15.45:0/1332622181 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 0x7f28f80adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f6ffd700 1 --2- 172.21.15.45:0/1332622181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f80cc650 0x7f28f80ced40 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0098f0 tx=0x7f28ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ce9fd87386a930 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.848+0000 7f28f67fc700 1 -- 172.21.15.45:0/1332622181 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00b8a0 con 0x7f28f80cc650 2023-08-27T11:28:34.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28f67fc700 1 -- 172.21.15.45:0/1332622181 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ec00ba00 con 0x7f28f80cc650 2023-08-27T11:28:34.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28f67fc700 1 -- 172.21.15.45:0/1332622181 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00bd90 con 0x7f28f80cc650 2023-08-27T11:28:34.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28fe198700 1 -- 172.21.15.45:0/1332622181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f80cc650 msgr2=0x7f28f80ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28fe198700 1 --2- 172.21.15.45:0/1332622181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f80cc650 0x7f28f80ced40 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0098f0 tx=0x7f28ec009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28fe198700 1 -- 172.21.15.45:0/1332622181 shutdown_connections 2023-08-27T11:28:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28fe198700 1 --2- 172.21.15.45:0/1332622181 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 0x7f28f80adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28fe198700 1 --2- 172.21.15.45:0/1332622181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f80cc650 0x7f28f80ced40 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28fe198700 1 -- 172.21.15.45:0/1332622181 >> 172.21.15.45:0/1332622181 conn(0x7f28f80d0f60 msgr2=0x7f28f80d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28fe198700 1 -- 172.21.15.45:0/1332622181 shutdown_connections 2023-08-27T11:28:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.849+0000 7f28fe198700 1 -- 172.21.15.45:0/1332622181 wait complete. 2023-08-27T11:28:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.850+0000 7f28fe198700 1 Processor -- start 2023-08-27T11:28:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.850+0000 7f28fe198700 1 -- start start 2023-08-27T11:28:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28fe198700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 0x7f28f81535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28fe198700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f80cc650 0x7f28f80b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28fe198700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f815e990 0x7f28f806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28fe198700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f806c960 con 0x7f28f815e990 2023-08-27T11:28:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28fe198700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8153af0 con 0x7f28f80cc650 2023-08-27T11:28:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28fe198700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80b6340 con 0x7f28f80adad0 2023-08-27T11:28:34.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28f7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f815e990 0x7f28f806cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28f7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f815e990 0x7f28f806cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59928/0 (socket says 172.21.15.45:59928) 2023-08-27T11:28:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28f7fff700 1 -- 172.21.15.45:0/4139365454 learned_addr learned my addr 172.21.15.45:0/4139365454 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28f77fe700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 0x7f28f81535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.851+0000 7f28f6ffd700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f80cc650 0x7f28f80b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28f7fff700 1 -- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 msgr2=0x7f28f81535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28f7fff700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 0x7f28f81535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28f7fff700 1 -- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f80cc650 msgr2=0x7f28f80b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28f7fff700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f80cc650 0x7f28f80b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:34.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28f7fff700 1 -- 172.21.15.45:0/4139365454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec009550 con 0x7f28f815e990 2023-08-27T11:28:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28f6ffd700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f80cc650 0x7f28f80b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28f7fff700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f815e990 0x7f28f806cdf0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f28f0007e20 tx=0x7f28f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28dffff700 1 -- 172.21.15.45:0/4139365454 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000ddd0 con 0x7f28f815e990 2023-08-27T11:28:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28dffff700 1 -- 172.21.15.45:0/4139365454 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f000f040 con 0x7f28f815e990 2023-08-27T11:28:34.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28dffff700 1 -- 172.21.15.45:0/4139365454 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0003c60 con 0x7f28f815e990 2023-08-27T11:28:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f80b6610 con 0x7f28f815e990 2023-08-27T11:28:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.852+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f806c5f0 con 0x7f28f815e990 2023-08-27T11:28:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.853+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f806cb10 con 0x7f28f815e990 2023-08-27T11:28:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.853+0000 7f28dffff700 1 -- 172.21.15.45:0/4139365454 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 44) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f28f0004030 con 0x7f28f815e990 2023-08-27T11:28:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.854+0000 7f28dffff700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f28e4069f30 0x7f28e406c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.854+0000 7f28dffff700 1 -- 172.21.15.45:0/4139365454 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f28f0088690 con 0x7f28f815e990 2023-08-27T11:28:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.854+0000 7f28f77fe700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f28e4069f30 0x7f28e406c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.854+0000 7f28f77fe700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f28e4069f30 0x7f28e406c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f28e8007fd0 tx=0x7f28e8009040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:34.858+0000 7f28dffff700 1 -- 172.21.15.45:0/4139365454 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f28f806cb10 con 0x7f28f815e990 2023-08-27T11:28:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.104+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28f806cb10 con 0x7f28f815e990 2023-08-27T11:28:35.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.105+0000 7f28dffff700 1 -- 172.21.15.45:0/4139365454 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f28f806cb10 con 0x7f28f815e990 2023-08-27T11:28:35.106 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:35.106 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3604468746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3604468746},{"type":"v1","addr":"172.21.15.135:6836","nonce":3604468746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1319491874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1319491874},{"type":"v1","addr":"172.21.15.135:6837","nonce":1319491874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6937,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2036943131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2036943131},{"type":"v1","addr":"172.21.15.45:6835","nonce":2036943131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:07.092667+0000","modified":"2023-08-27T11:28:10.679683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7446":{"gid":7446,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6833/3257398445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3257398445},{"type":"v1","addr":"172.21.15.45:6833","nonce":3257398445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-27T11:28:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f28e4069f30 msgr2=0x7f28e406c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f28e4069f30 0x7f28e406c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f28e8007fd0 tx=0x7f28e8009040 comp rx=0 tx=0).stop 2023-08-27T11:28:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f815e990 msgr2=0x7f28f806cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f815e990 0x7f28f806cdf0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f28f0007e20 tx=0x7f28f000d3b0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 shutdown_connections 2023-08-27T11:28:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f28e4069f30 0x7f28e406c3f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28f815e990 0x7f28f806cdf0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f28f80adad0 0x7f28f81535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 --2- 172.21.15.45:0/4139365454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28f80cc650 0x7f28f80b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 >> 172.21.15.45:0/4139365454 conn(0x7f28f80d0f60 msgr2=0x7f28f806f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 shutdown_connections 2023-08-27T11:28:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.107+0000 7f28fe198700 1 -- 172.21.15.45:0/4139365454 wait complete. 2023-08-27T11:28:35.110 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 41 2023-08-27T11:28:35.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:35.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.244+0000 7fa56af11700 1 Processor -- start 2023-08-27T11:28:35.245 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.244+0000 7fa56af11700 1 -- start start 2023-08-27T11:28:35.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.245+0000 7fa56af11700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa56406b130 0x7fa56406b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.245+0000 7fa56af11700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5640b0af0 con 0x7fa5640aea90 2023-08-27T11:28:35.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.245+0000 7fa56af11700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56406a510 con 0x7fa5640c9c00 2023-08-27T11:28:35.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.245+0000 7fa56af11700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56406bb20 con 0x7fa56406b130 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.245+0000 7fa568cad700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa5640aea90 0x7fa5640cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46710/0 (socket says 172.21.15.45:46710) 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.245+0000 7fa568cad700 1 -- 172.21.15.45:0/909023070 learned_addr learned my addr 172.21.15.45:0/909023070 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.245+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159017401 0 0) 0x7fa5640b0af0 con 0x7fa5640aea90 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.245+0000 7fa563fff700 1 --2- 172.21.15.45:0/909023070 >> v2:172.21.15.135:3300/0 conn(0x7fa56406b130 0x7fa56406b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.245+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5540035b0 con 0x7fa5640aea90 2023-08-27T11:28:35.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 800752484 0 0) 0x7fa5540035b0 con 0x7fa5640aea90 2023-08-27T11:28:35.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 >> v2:172.21.15.135:3300/0 conn(0x7fa56406b130 msgr2=0x7fa56406b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:35.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 --2- 172.21.15.45:0/909023070 >> v2:172.21.15.135:3300/0 conn(0x7fa56406b130 0x7fa56406b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 >> v1:172.21.15.135:6789/0 conn(0x7fa5640c9c00 legacy=0x7fa5640ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56406a7b0 con 0x7fa5640aea90 2023-08-27T11:28:35.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7fa558001670 con 0x7fa5640aea90 2023-08-27T11:28:35.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 >> v1:172.21.15.45:6789/0 conn(0x7fa5640aea90 legacy=0x7fa5640cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa554003a00 0x7fa554003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa554004850 0x7fa554004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa554005210 0x7fa55400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56406bb20 con 0x7fa554003a00 2023-08-27T11:28:35.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55400c0b0 con 0x7fa554004850 2023-08-27T11:28:35.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55400c270 con 0x7fa554005210 2023-08-27T11:28:35.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa568cad700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa554003a00 0x7fa554003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa563fff700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa554004850 0x7fa554004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5694ae700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa554005210 0x7fa55400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa568cad700 1 -- 172.21.15.45:0/909023070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa554005210 msgr2=0x7fa55400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa568cad700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa554005210 0x7fa55400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa568cad700 1 -- 172.21.15.45:0/909023070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa554004850 msgr2=0x7fa554004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa568cad700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa554004850 0x7fa554004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa568cad700 1 -- 172.21.15.45:0/909023070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5540035b0 con 0x7fa554003a00 2023-08-27T11:28:35.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5694ae700 1 -- 172.21.15.45:0/909023070 reap_dead start 2023-08-27T11:28:35.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa568cad700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa554003a00 0x7fa554003e20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c240 tx=0x7fa554005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e27b5a3a9907d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558007c10 con 0x7fa554003a00 2023-08-27T11:28:35.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55800ce00 con 0x7fa554003a00 2023-08-27T11:28:35.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.246+0000 7fa5637fe700 1 -- 172.21.15.45:0/909023070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558005ba0 con 0x7fa554003a00 2023-08-27T11:28:35.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.247+0000 7fa56af11700 1 -- 172.21.15.45:0/909023070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa554003a00 msgr2=0x7fa554003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.247+0000 7fa56af11700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa554003a00 0x7fa554003e20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c240 tx=0x7fa554005880 comp rx=0 tx=0).stop 2023-08-27T11:28:35.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.247+0000 7fa56af11700 1 -- 172.21.15.45:0/909023070 shutdown_connections 2023-08-27T11:28:35.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.247+0000 7fa56af11700 1 --2- 172.21.15.45:0/909023070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa554003a00 0x7fa554003e20 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.247+0000 7fa56af11700 1 -- 172.21.15.45:0/909023070 >> 172.21.15.45:0/909023070 conn(0x7fa5640b2470 msgr2=0x7fa5640b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.247+0000 7fa56af11700 1 -- 172.21.15.45:0/909023070 shutdown_connections 2023-08-27T11:28:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.247+0000 7fa56af11700 1 -- 172.21.15.45:0/909023070 wait complete. 2023-08-27T11:28:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.248+0000 7fa56af11700 1 Processor -- start 2023-08-27T11:28:35.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.248+0000 7fa56af11700 1 -- start start 2023-08-27T11:28:35.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa56af11700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa554003a00 0x7fa5640c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa56af11700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564098530 0x7fa5640cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa56af11700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5640cc2e0 0x7fa5640c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa56af11700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c4c20 con 0x7fa564098530 2023-08-27T11:28:35.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa56af11700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c9730 con 0x7fa554003a00 2023-08-27T11:28:35.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa56af11700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c6bc0 con 0x7fa5640cc2e0 2023-08-27T11:28:35.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa563fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564098530 0x7fa5640cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa563fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564098530 0x7fa5640cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59948/0 (socket says 172.21.15.45:59948) 2023-08-27T11:28:35.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa563fff700 1 -- 172.21.15.45:0/3771669258 learned_addr learned my addr 172.21.15.45:0/3771669258 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:35.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa568cad700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa554003a00 0x7fa5640c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.249+0000 7fa5694ae700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5640cc2e0 0x7fa5640c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa563fff700 1 -- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5640cc2e0 msgr2=0x7fa5640c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa563fff700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5640cc2e0 0x7fa5640c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa563fff700 1 -- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa554003a00 msgr2=0x7fa5640c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa563fff700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa554003a00 0x7fa5640c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa563fff700 1 -- 172.21.15.45:0/3771669258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640c6ea0 con 0x7fa564098530 2023-08-27T11:28:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa563fff700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564098530 0x7fa5640cbda0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa5500115d0 tx=0x7fa5500118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa5617fa700 1 -- 172.21.15.45:0/3771669258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550015870 con 0x7fa564098530 2023-08-27T11:28:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa5617fa700 1 -- 172.21.15.45:0/3771669258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5500159d0 con 0x7fa564098530 2023-08-27T11:28:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5640c48b0 con 0x7fa564098530 2023-08-27T11:28:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa5617fa700 1 -- 172.21.15.45:0/3771669258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550015d40 con 0x7fa564098530 2023-08-27T11:28:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.250+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5640989b0 con 0x7fa564098530 2023-08-27T11:28:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.251+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56409d5b0 con 0x7fa564098530 2023-08-27T11:28:35.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.251+0000 7fa5617fa700 1 -- 172.21.15.45:0/3771669258 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 44) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fa5500143e0 con 0x7fa564098530 2023-08-27T11:28:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.252+0000 7fa5617fa700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa548069f30 0x7fa54806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.252+0000 7fa5617fa700 1 -- 172.21.15.45:0/3771669258 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7157+0+0 (secure 0 0 0) 0x7fa5500887a0 con 0x7fa564098530 2023-08-27T11:28:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.252+0000 7fa568cad700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa548069f30 0x7fa54806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.253+0000 7fa568cad700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa548069f30 0x7fa54806c3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa558000c00 tx=0x7fa558007000 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.256+0000 7fa5617fa700 1 -- 172.21.15.45:0/3771669258 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa550053d80 con 0x7fa564098530 2023-08-27T11:28:35.459 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.458+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5640734a0 con 0x7fa564098530 2023-08-27T11:28:35.460 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.459+0000 7fa5617fa700 1 -- 172.21.15.45:0/3771669258 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+19706 (secure 0 0 0) 0x7fa5500573a0 con 0x7fa564098530 2023-08-27T11:28:35.460 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:35.461 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":103,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:33.686782+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:05.410178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:06.051204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/626141668":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/1023575426":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/233288315":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/3946297931":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6833/3112377367":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6832/3112377367":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6833/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/4034194156":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6832/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3142979284":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/569758288":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/129865033":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/823048739":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/1640490812":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/12937390":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/2560756484":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3438903793":"2023-08-27T12:28:16.346002+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:35.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.461+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa548069f30 msgr2=0x7fa54806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.461+0000 7fa56af11700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa548069f30 0x7fa54806c3f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa558000c00 tx=0x7fa558007000 comp rx=0 tx=0).stop 2023-08-27T11:28:35.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.461+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564098530 msgr2=0x7fa5640cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.461+0000 7fa56af11700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564098530 0x7fa5640cbda0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa5500115d0 tx=0x7fa5500118e0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.462+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 shutdown_connections 2023-08-27T11:28:35.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.462+0000 7fa56af11700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa548069f30 0x7fa54806c3f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.462+0000 7fa56af11700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa5640cc2e0 0x7fa5640c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.462+0000 7fa56af11700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa554003a00 0x7fa5640c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.462+0000 7fa56af11700 1 --2- 172.21.15.45:0/3771669258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564098530 0x7fa5640cbda0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.462+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 >> 172.21.15.45:0/3771669258 conn(0x7fa5640b2470 msgr2=0x7fa5640cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.462+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 shutdown_connections 2023-08-27T11:28:35.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.462+0000 7fa56af11700 1 -- 172.21.15.45:0/3771669258 wait complete. 2023-08-27T11:28:35.475 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:28:35.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.600+0000 7f69c7eb3700 1 Processor -- start 2023-08-27T11:28:35.601 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.600+0000 7f69c7eb3700 1 -- start start 2023-08-27T11:28:35.603 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c7eb3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69c015d600 0x7f69c00cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c7eb3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f69c00aed20 0x7f69c00ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c7eb3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c00af270 con 0x7f69c00ca3b0 2023-08-27T11:28:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c7eb3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c00af550 con 0x7f69c015d600 2023-08-27T11:28:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c7eb3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c006ac90 con 0x7f69c00aed20 2023-08-27T11:28:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c544e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69c015d600 0x7f69c00cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c544e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f69c015d600 0x7f69c00cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59956/0 (socket says 172.21.15.45:59956) 2023-08-27T11:28:35.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c544e700 1 -- 172.21.15.45:0/433032762 learned_addr learned my addr 172.21.15.45:0/433032762 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:35.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c5c4f700 1 --1- 172.21.15.45:0/433032762 >> v1:172.21.15.45:6789/0 conn(0x7f69c00ca3b0 0x7f69c00ca790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46730/0 (socket says 172.21.15.45:46730) 2023-08-27T11:28:35.605 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c6450700 1 --2- 172.21.15.45:0/433032762 >> v2:172.21.15.135:3300/0 conn(0x7f69c00aed20 0x7f69c00ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c544e700 1 -- 172.21.15.45:0/433032762 >> v2:172.21.15.135:3300/0 conn(0x7f69c00aed20 msgr2=0x7f69c00ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c544e700 1 --2- 172.21.15.45:0/433032762 >> v2:172.21.15.135:3300/0 conn(0x7f69c00aed20 0x7f69c00ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c544e700 1 -- 172.21.15.45:0/433032762 >> v1:172.21.15.45:6789/0 conn(0x7f69c00ca3b0 legacy=0x7f69c00ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c544e700 1 -- 172.21.15.45:0/433032762 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c006af70 con 0x7f69c015d600 2023-08-27T11:28:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c6450700 1 --2- 172.21.15.45:0/433032762 >> v2:172.21.15.135:3300/0 conn(0x7f69c00aed20 0x7f69c00ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.601+0000 7f69c544e700 1 --2- 172.21.15.45:0/433032762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c015d600 0x7f69c00cdd10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f69c006fa00 tx=0x7f69b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792bef15020c9f9b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c4c4d700 1 -- 172.21.15.45:0/433032762 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b000b870 con 0x7f69c015d600 2023-08-27T11:28:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c4c4d700 1 -- 172.21.15.45:0/433032762 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b000b9d0 con 0x7f69c015d600 2023-08-27T11:28:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c4c4d700 1 -- 172.21.15.45:0/433032762 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b000bd60 con 0x7f69c015d600 2023-08-27T11:28:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c7eb3700 1 -- 172.21.15.45:0/433032762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c015d600 msgr2=0x7f69c00cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c7eb3700 1 --2- 172.21.15.45:0/433032762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c015d600 0x7f69c00cdd10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f69c006fa00 tx=0x7f69b0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c7eb3700 1 -- 172.21.15.45:0/433032762 shutdown_connections 2023-08-27T11:28:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c7eb3700 1 --2- 172.21.15.45:0/433032762 >> v2:172.21.15.135:3300/0 conn(0x7f69c00aed20 0x7f69c00ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c7eb3700 1 --2- 172.21.15.45:0/433032762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c015d600 0x7f69c00cdd10 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c7eb3700 1 -- 172.21.15.45:0/433032762 >> 172.21.15.45:0/433032762 conn(0x7f69c00cc7e0 msgr2=0x7f69c00ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c7eb3700 1 -- 172.21.15.45:0/433032762 shutdown_connections 2023-08-27T11:28:35.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.602+0000 7f69c7eb3700 1 -- 172.21.15.45:0/433032762 wait complete. 2023-08-27T11:28:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.603+0000 7f69c7eb3700 1 Processor -- start 2023-08-27T11:28:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.603+0000 7f69c7eb3700 1 -- start start 2023-08-27T11:28:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c7eb3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69c00aed20 0x7f69c00c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c7eb3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c00ca3b0 0x7f69c00c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c7eb3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69c015a690 0x7f69c00c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c7eb3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c0159950 con 0x7f69c00ca3b0 2023-08-27T11:28:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c7eb3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00c66b0 con 0x7f69c00aed20 2023-08-27T11:28:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c7eb3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c0159bb0 con 0x7f69c015a690 2023-08-27T11:28:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c544e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c00ca3b0 0x7f69c00c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c544e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c00ca3b0 0x7f69c00c40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59962/0 (socket says 172.21.15.45:59962) 2023-08-27T11:28:35.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c544e700 1 -- 172.21.15.45:0/2449859004 learned_addr learned my addr 172.21.15.45:0/2449859004 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:35.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c5c4f700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69c00aed20 0x7f69c00c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.604+0000 7f69c6450700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69c015a690 0x7f69c00c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69c544e700 1 -- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69c015a690 msgr2=0x7f69c00c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69c544e700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69c015a690 0x7f69c00c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69c544e700 1 -- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69c00aed20 msgr2=0x7f69c00c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69c544e700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69c00aed20 0x7f69c00c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69c544e700 1 -- 172.21.15.45:0/2449859004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b0009550 con 0x7f69c00ca3b0 2023-08-27T11:28:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69c6450700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69c015a690 0x7f69c00c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69c544e700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c00ca3b0 0x7f69c00c40d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f69b0014040 tx=0x7f69b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69b67fc700 1 -- 172.21.15.45:0/2449859004 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0004520 con 0x7f69c00ca3b0 2023-08-27T11:28:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69b67fc700 1 -- 172.21.15.45:0/2449859004 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b001e070 con 0x7f69c00ca3b0 2023-08-27T11:28:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c0159e90 con 0x7f69c00ca3b0 2023-08-27T11:28:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69b67fc700 1 -- 172.21.15.45:0/2449859004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b00038c0 con 0x7f69c00ca3b0 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.605+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c0158740 con 0x7f69c00ca3b0 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.606+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c00726d0 con 0x7f69c00ca3b0 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.607+0000 7f69b67fc700 1 -- 172.21.15.45:0/2449859004 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 44) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f69b0003a20 con 0x7f69c00ca3b0 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.607+0000 7f69b67fc700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f69a4069f30 0x7f69a406c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.607+0000 7f69b67fc700 1 -- 172.21.15.45:0/2449859004 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f69b0020070 con 0x7f69c00ca3b0 2023-08-27T11:28:35.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.607+0000 7f69c5c4f700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f69a4069f30 0x7f69a406c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:35.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.608+0000 7f69c5c4f700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f69a4069f30 0x7f69a406c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f69bc000f60 tx=0x7f69bc007ed0 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:35.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.611+0000 7f69b67fc700 1 -- 172.21.15.45:0/2449859004 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f69b0055210 con 0x7f69c00ca3b0 2023-08-27T11:28:35.738 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:35.737+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.754+0000 7f69b67fc700 1 -- 172.21.15.45:0/2449859004 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f69b00555e0 con 0x7f69c00ca3b0 2023-08-27T11:28:35.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.863+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f69c006c1e0 con 0x7f69c00ca3b0 2023-08-27T11:28:35.865 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-08-27T11:28:35.864+0000 7f87a1c48700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:28:35.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.868+0000 7f69b67fc700 1 -- 172.21.15.45:0/2449859004 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f69b0058ca0 con 0x7f69c00ca3b0 2023-08-27T11:28:35.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f69a4069f30 msgr2=0x7f69a406c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f69a4069f30 0x7f69a406c3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f69bc000f60 tx=0x7f69bc007ed0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c00ca3b0 msgr2=0x7f69c00c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:35.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c00ca3b0 0x7f69c00c40d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f69b0014040 tx=0x7f69b0004750 comp rx=0 tx=0).stop 2023-08-27T11:28:35.872 INFO:tasks.ceph.mds.a.smithi045.stderr: -3933> 2023-08-27T11:28:12.647+0000 7f2bb7a3e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:28:35.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 shutdown_connections 2023-08-27T11:28:35.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f69a4069f30 0x7f69a406c3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f69c015a690 0x7f69c00c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69c00aed20 0x7f69c00c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 --2- 172.21.15.45:0/2449859004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69c00ca3b0 0x7f69c00c40d0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 >> 172.21.15.45:0/2449859004 conn(0x7f69c00cc7e0 msgr2=0x7f69c006e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 shutdown_connections 2023-08-27T11:28:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:35.870+0000 7f69c7eb3700 1 -- 172.21.15.45:0/2449859004 wait complete. 2023-08-27T11:28:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:28:35.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:28:35.902 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-08-27T11:28:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.016+0000 7fb9a21ea700 1 Processor -- start 2023-08-27T11:28:36.017 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.016+0000 7fb9a21ea700 1 -- start start 2023-08-27T11:28:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.016+0000 7fb9a21ea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c157be0 0x7fb99c0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.016+0000 7fb9a21ea700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb99c0b17c0 0x7fb99c0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.016+0000 7fb9a21ea700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb99c06ced0 0x7fb99c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.016+0000 7fb9a21ea700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c157fc0 con 0x7fb99c0b17c0 2023-08-27T11:28:36.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.016+0000 7fb9a21ea700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c06d860 con 0x7fb99c06ced0 2023-08-27T11:28:36.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.016+0000 7fb9a21ea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c15b460 con 0x7fb99c157be0 2023-08-27T11:28:36.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb99c0b17c0 0x7fb99c0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99b7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c157be0 0x7fb99c0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99bfff700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb99c06ced0 0x7fb99c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb99c0b17c0 0x7fb99c0b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59982/0 (socket says 172.21.15.45:59982) 2023-08-27T11:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99affd700 1 -- 172.21.15.45:0/474522369 learned_addr learned my addr 172.21.15.45:0/474522369 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99affd700 1 -- 172.21.15.45:0/474522369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c157be0 msgr2=0x7fb99c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99affd700 1 --2- 172.21.15.45:0/474522369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c157be0 0x7fb99c0716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99affd700 1 -- 172.21.15.45:0/474522369 >> v2:172.21.15.135:3300/0 conn(0x7fb99c06ced0 msgr2=0x7fb99c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:36.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99affd700 1 --2- 172.21.15.45:0/474522369 >> v2:172.21.15.135:3300/0 conn(0x7fb99c06ced0 0x7fb99c06d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99affd700 1 -- 172.21.15.45:0/474522369 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c0b2280 con 0x7fb99c0b17c0 2023-08-27T11:28:36.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99bfff700 1 --2- 172.21.15.45:0/474522369 >> v2:172.21.15.135:3300/0 conn(0x7fb99c06ced0 0x7fb99c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:36.022 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99affd700 1 --2- 172.21.15.45:0/474522369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 0x7fb99c0b1c40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0cd3f0 tx=0x7fb984009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d368ea7bcd0076 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:36.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99a7fc700 1 -- 172.21.15.45:0/474522369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984011830 con 0x7fb99c0b17c0 2023-08-27T11:28:36.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99a7fc700 1 -- 172.21.15.45:0/474522369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb984011990 con 0x7fb99c0b17c0 2023-08-27T11:28:36.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.017+0000 7fb99a7fc700 1 -- 172.21.15.45:0/474522369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984011d20 con 0x7fb99c0b17c0 2023-08-27T11:28:36.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.018+0000 7fb9a21ea700 1 -- 172.21.15.45:0/474522369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 msgr2=0x7fb99c0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.018+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/474522369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 0x7fb99c0b1c40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0cd3f0 tx=0x7fb984009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.018+0000 7fb9a21ea700 1 -- 172.21.15.45:0/474522369 shutdown_connections 2023-08-27T11:28:36.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.018+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/474522369 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c157be0 0x7fb99c0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.018+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/474522369 >> v2:172.21.15.135:3300/0 conn(0x7fb99c06ced0 0x7fb99c06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.018+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/474522369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 0x7fb99c0b1c40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.018+0000 7fb9a21ea700 1 -- 172.21.15.45:0/474522369 >> 172.21.15.45:0/474522369 conn(0x7fb99c0cc7c0 msgr2=0x7fb99c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:36.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.018+0000 7fb9a21ea700 1 -- 172.21.15.45:0/474522369 shutdown_connections 2023-08-27T11:28:36.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.018+0000 7fb9a21ea700 1 -- 172.21.15.45:0/474522369 wait complete. 2023-08-27T11:28:36.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.019+0000 7fb9a21ea700 1 Processor -- start 2023-08-27T11:28:36.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb9a21ea700 1 -- start start 2023-08-27T11:28:36.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb9a21ea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c06ced0 0x7fb99c154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb9a21ea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 0x7fb99c0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb9a21ea700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb99c157be0 0x7fb99c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb9a21ea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c154af0 con 0x7fb99c0b17c0 2023-08-27T11:28:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb9a21ea700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0c72d0 con 0x7fb99c157be0 2023-08-27T11:28:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb9a21ea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0c75b0 con 0x7fb99c06ced0 2023-08-27T11:28:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb99affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 0x7fb99c0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb99affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 0x7fb99c0c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59988/0 (socket says 172.21.15.45:59988) 2023-08-27T11:28:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb99affd700 1 -- 172.21.15.45:0/1635072853 learned_addr learned my addr 172.21.15.45:0/1635072853 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb99b7fe700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c06ced0 0x7fb99c154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.020+0000 7fb99bfff700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb99c157be0 0x7fb99c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb99affd700 1 -- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c06ced0 msgr2=0x7fb99c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb99affd700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c06ced0 0x7fb99c154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb99affd700 1 -- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb99c157be0 msgr2=0x7fb99c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb99affd700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb99c157be0 0x7fb99c1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb99affd700 1 -- 172.21.15.45:0/1635072853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb994009a70 con 0x7fb99c0b17c0 2023-08-27T11:28:36.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb99affd700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 0x7fb99c0c8370 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fb984009fd0 tx=0x7fb984004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:36.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb993fff700 1 -- 172.21.15.45:0/1635072853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9840044d0 con 0x7fb99c0b17c0 2023-08-27T11:28:36.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb993fff700 1 -- 172.21.15.45:0/1635072853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98401f070 con 0x7fb99c0b17c0 2023-08-27T11:28:36.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb984009550 con 0x7fb99c0b17c0 2023-08-27T11:28:36.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb993fff700 1 -- 172.21.15.45:0/1635072853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9840038e0 con 0x7fb99c0b17c0 2023-08-27T11:28:36.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.021+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c0c7b70 con 0x7fb99c0b17c0 2023-08-27T11:28:36.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.022+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99c072690 con 0x7fb99c0b17c0 2023-08-27T11:28:36.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.023+0000 7fb993fff700 1 -- 172.21.15.45:0/1635072853 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fb984003a40 con 0x7fb99c0b17c0 2023-08-27T11:28:36.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.024+0000 7fb993fff700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb97c06a170 0x7fb97c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:36.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.024+0000 7fb993fff700 1 -- 172.21.15.45:0/1635072853 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7243+0+0 (secure 0 0 0) 0x7fb984021070 con 0x7fb99c0b17c0 2023-08-27T11:28:36.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.024+0000 7fb99b7fe700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb97c06a170 0x7fb97c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:36.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.024+0000 7fb99b7fe700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb97c06a170 0x7fb97c06c630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0b2150 tx=0x7fb98c009210 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:36.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.027+0000 7fb993fff700 1 -- 172.21.15.45:0/1635072853 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb984055870 con 0x7fb99c0b17c0 2023-08-27T11:28:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.289+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb99c0734a0 con 0x7fb99c0b17c0 2023-08-27T11:28:36.513 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:36.511+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:36.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.868+0000 7fb993fff700 1 -- 172.21.15.45:0/1635072853 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb9840150a0 con 0x7fb99c0b17c0 2023-08-27T11:28:36.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb97c06a170 msgr2=0x7fb97c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb97c06a170 0x7fb97c06c630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0b2150 tx=0x7fb98c009210 comp rx=0 tx=0).stop 2023-08-27T11:28:36.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 msgr2=0x7fb99c0c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:36.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 0x7fb99c0c8370 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fb984009fd0 tx=0x7fb984004750 comp rx=0 tx=0).stop 2023-08-27T11:28:36.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 shutdown_connections 2023-08-27T11:28:36.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb97c06a170 0x7fb97c06c630 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb99c157be0 0x7fb99c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb99c06ced0 0x7fb99c154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 --2- 172.21.15.45:0/1635072853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb99c0b17c0 0x7fb99c0c8370 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:36.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 >> 172.21.15.45:0/1635072853 conn(0x7fb99c0cc7c0 msgr2=0x7fb99c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:36.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 shutdown_connections 2023-08-27T11:28:36.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:36.870+0000 7fb9a21ea700 1 -- 172.21.15.45:0/1635072853 wait complete. 2023-08-27T11:28:36.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:28:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.054+0000 7fa8b79d3700 1 Processor -- start 2023-08-27T11:28:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.054+0000 7fa8b79d3700 1 -- start start 2023-08-27T11:28:37.057 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b79d3700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa8b00af200 0x7fa8b00cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b79d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b00acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b79d3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b00ce340 con 0x7fa8b00ca620 2023-08-27T11:28:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b79d3700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b00ceec0 con 0x7fa8b00af200 2023-08-27T11:28:37.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b79d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b015d7c0 con 0x7fa8b00caa00 2023-08-27T11:28:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b4f6e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa8b00af200 0x7fa8b00cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b5f70700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b00acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b576f700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa8b00ca620 0x7fa8b00ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:42436/0 (socket says 172.21.15.45:42436) 2023-08-27T11:28:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b576f700 1 -- 172.21.15.45:0/185574965 learned_addr learned my addr 172.21.15.45:0/185574965 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8a7fff700 1 -- 172.21.15.45:0/185574965 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434271027 0 0) 0x7fa8b00ce340 con 0x7fa8b00ca620 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8a7fff700 1 -- 172.21.15.45:0/185574965 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa89c0035b0 con 0x7fa8b00ca620 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b4f6e700 1 -- 172.21.15.45:0/185574965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 msgr2=0x7fa8b00acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b4f6e700 1 --2- 172.21.15.45:0/185574965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b00acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b4f6e700 1 -- 172.21.15.45:0/185574965 >> v1:172.21.15.135:6789/0 conn(0x7fa8b00ca620 legacy=0x7fa8b00ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b4f6e700 1 -- 172.21.15.45:0/185574965 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b00cc570 con 0x7fa8b00af200 2023-08-27T11:28:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.055+0000 7fa8b5f70700 1 --2- 172.21.15.45:0/185574965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b00acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8b4f6e700 1 --2- 172.21.15.45:0/185574965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8b00af200 0x7fa8b00cf350 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009840 tx=0x7fa8ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f2a09e9c53a5fbb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:37.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8a7fff700 1 -- 172.21.15.45:0/185574965 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac011830 con 0x7fa8b00af200 2023-08-27T11:28:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8a7fff700 1 -- 172.21.15.45:0/185574965 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac011990 con 0x7fa8b00af200 2023-08-27T11:28:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8a7fff700 1 -- 172.21.15.45:0/185574965 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac011d00 con 0x7fa8b00af200 2023-08-27T11:28:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8b79d3700 1 -- 172.21.15.45:0/185574965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8b00af200 msgr2=0x7fa8b00cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8b79d3700 1 --2- 172.21.15.45:0/185574965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8b00af200 0x7fa8b00cf350 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009840 tx=0x7fa8ac009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8b79d3700 1 -- 172.21.15.45:0/185574965 shutdown_connections 2023-08-27T11:28:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8b79d3700 1 --2- 172.21.15.45:0/185574965 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b00acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8b79d3700 1 --2- 172.21.15.45:0/185574965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8b00af200 0x7fa8b00cf350 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8b79d3700 1 -- 172.21.15.45:0/185574965 >> 172.21.15.45:0/185574965 conn(0x7fa8b00cc7e0 msgr2=0x7fa8b00ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:37.063 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8b79d3700 1 -- 172.21.15.45:0/185574965 shutdown_connections 2023-08-27T11:28:37.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.056+0000 7fa8b79d3700 1 -- 172.21.15.45:0/185574965 wait complete. 2023-08-27T11:28:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.058+0000 7fa8b79d3700 1 Processor -- start 2023-08-27T11:28:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.058+0000 7fa8b79d3700 1 -- start start 2023-08-27T11:28:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.058+0000 7fa8b79d3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8b00af200 0x7fa8b01f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.058+0000 7fa8b79d3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b00ca620 0x7fa8b0148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.058+0000 7fa8b79d3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b006da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.058+0000 7fa8b79d3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0148c40 con 0x7fa8b00ca620 2023-08-27T11:28:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.058+0000 7fa8b79d3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0148f20 con 0x7fa8b00af200 2023-08-27T11:28:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.058+0000 7fa8b79d3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0149200 con 0x7fa8b00caa00 2023-08-27T11:28:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b4f6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b00ca620 0x7fa8b0148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b5f70700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b4f6e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b00ca620 0x7fa8b0148180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60004/0 (socket says 172.21.15.45:60004) 2023-08-27T11:28:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b4f6e700 1 -- 172.21.15.45:0/232438528 learned_addr learned my addr 172.21.15.45:0/232438528 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b576f700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8b00af200 0x7fa8b01f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b4f6e700 1 -- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 msgr2=0x7fa8b006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b4f6e700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b006da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b4f6e700 1 -- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8b00af200 msgr2=0x7fa8b01f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b4f6e700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8b00af200 0x7fa8b01f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b4f6e700 1 -- 172.21.15.45:0/232438528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a8009a70 con 0x7fa8b00ca620 2023-08-27T11:28:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b5f70700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b006da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b4f6e700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b00ca620 0x7fa8b0148180 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0090b0 tx=0x7fa8ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8a5ffb700 1 -- 172.21.15.45:0/232438528 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac004550 con 0x7fa8b00ca620 2023-08-27T11:28:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8a5ffb700 1 -- 172.21.15.45:0/232438528 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac01f070 con 0x7fa8b00ca620 2023-08-27T11:28:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac0094a0 con 0x7fa8b00ca620 2023-08-27T11:28:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.059+0000 7fa8a5ffb700 1 -- 172.21.15.45:0/232438528 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0038c0 con 0x7fa8b00ca620 2023-08-27T11:28:37.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.060+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b01497c0 con 0x7fa8b00ca620 2023-08-27T11:28:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.061+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b00cbce0 con 0x7fa8b00ca620 2023-08-27T11:28:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.061+0000 7fa8a5ffb700 1 -- 172.21.15.45:0/232438528 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fa8ac003a20 con 0x7fa8b00ca620 2023-08-27T11:28:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.061+0000 7fa8a5ffb700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa89406a170 0x7fa89406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.061+0000 7fa8a5ffb700 1 -- 172.21.15.45:0/232438528 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7243+0+0 (secure 0 0 0) 0x7fa8ac021070 con 0x7fa8b00ca620 2023-08-27T11:28:37.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.062+0000 7fa8b576f700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa89406a170 0x7fa89406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.062+0000 7fa8b576f700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa89406a170 0x7fa89406c630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0007fd0 tx=0x7fa8a0009040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:37.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.066+0000 7fa8a5ffb700 1 -- 172.21.15.45:0/232438528 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa8ac0bc990 con 0x7fa8b00ca620 2023-08-27T11:28:37.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:28:37.075 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:28:37.277 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:37.275+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:37.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.276+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa8b00734a0 con 0x7fa8b00ca620 2023-08-27T11:28:37.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.877+0000 7fa8a5ffb700 1 -- 172.21.15.45:0/232438528 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa8ac02b0d0 con 0x7fa8b00ca620 2023-08-27T11:28:37.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa89406a170 msgr2=0x7fa89406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa89406a170 0x7fa89406c630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0007fd0 tx=0x7fa8a0009040 comp rx=0 tx=0).stop 2023-08-27T11:28:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b00ca620 msgr2=0x7fa8b0148180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b00ca620 0x7fa8b0148180 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0090b0 tx=0x7fa8ac004750 comp rx=0 tx=0).stop 2023-08-27T11:28:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 shutdown_connections 2023-08-27T11:28:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fa89406a170 0x7fa89406c630 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa8b00caa00 0x7fa8b006da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8b00af200 0x7fa8b01f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 --2- 172.21.15.45:0/232438528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b00ca620 0x7fa8b0148180 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.880+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 >> 172.21.15.45:0/232438528 conn(0x7fa8b00cc7e0 msgr2=0x7fa8b00cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.881+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 shutdown_connections 2023-08-27T11:28:37.884 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:37.881+0000 7fa8b79d3700 1 -- 172.21.15.45:0/232438528 wait complete. 2023-08-27T11:28:37.884 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:28:37.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:28:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.023+0000 7f0527bd6700 1 Processor -- start 2023-08-27T11:28:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.023+0000 7f0527bd6700 1 -- start start 2023-08-27T11:28:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0527bd6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f052015a0f0 0x7f05200cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0527bd6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05200cd300 0x7f052015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0527bd6700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f052015e030 0x7f05200cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0527bd6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05200cfae0 con 0x7f052015a0f0 2023-08-27T11:28:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0527bd6700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052015e900 con 0x7f052015e030 2023-08-27T11:28:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0527bd6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05200cae60 con 0x7f05200cd300 2023-08-27T11:28:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0525972700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f052015a0f0 0x7f05200cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0525972700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f052015a0f0 0x7f05200cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60012/0 (socket says 172.21.15.45:60012) 2023-08-27T11:28:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0525972700 1 -- 172.21.15.45:0/3946879101 learned_addr learned my addr 172.21.15.45:0/3946879101 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0525171700 1 --2- 172.21.15.45:0/3946879101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05200cd300 0x7f052015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.024+0000 7f0526173700 1 --2- 172.21.15.45:0/3946879101 >> v2:172.21.15.135:3300/0 conn(0x7f052015e030 0x7f05200cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0525972700 1 -- 172.21.15.45:0/3946879101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05200cd300 msgr2=0x7f052015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0525972700 1 --2- 172.21.15.45:0/3946879101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05200cd300 0x7f052015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0525972700 1 -- 172.21.15.45:0/3946879101 >> v2:172.21.15.135:3300/0 conn(0x7f052015e030 msgr2=0x7f05200cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0525972700 1 --2- 172.21.15.45:0/3946879101 >> v2:172.21.15.135:3300/0 conn(0x7f052015e030 0x7f05200cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0525972700 1 -- 172.21.15.45:0/3946879101 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200cb140 con 0x7f052015a0f0 2023-08-27T11:28:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0525972700 1 --2- 172.21.15.45:0/3946879101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052015a0f0 0x7f05200cbc20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f0520072ab0 tx=0x7f051c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14728d7687b7bf3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0524970700 1 -- 172.21.15.45:0/3946879101 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00e070 con 0x7f052015a0f0 2023-08-27T11:28:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0524970700 1 -- 172.21.15.45:0/3946879101 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c00b9c0 con 0x7f052015a0f0 2023-08-27T11:28:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0524970700 1 -- 172.21.15.45:0/3946879101 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00bd70 con 0x7f052015a0f0 2023-08-27T11:28:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0527bd6700 1 -- 172.21.15.45:0/3946879101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052015a0f0 msgr2=0x7f05200cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0527bd6700 1 --2- 172.21.15.45:0/3946879101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052015a0f0 0x7f05200cbc20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f0520072ab0 tx=0x7f051c00b330 comp rx=0 tx=0).stop 2023-08-27T11:28:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0527bd6700 1 -- 172.21.15.45:0/3946879101 shutdown_connections 2023-08-27T11:28:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0527bd6700 1 --2- 172.21.15.45:0/3946879101 >> v2:172.21.15.135:3300/0 conn(0x7f052015e030 0x7f05200cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.025+0000 7f0527bd6700 1 --2- 172.21.15.45:0/3946879101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052015a0f0 0x7f05200cbc20 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.026+0000 7f0527bd6700 1 --2- 172.21.15.45:0/3946879101 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f05200cd300 0x7f052015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.026+0000 7f0527bd6700 1 -- 172.21.15.45:0/3946879101 >> 172.21.15.45:0/3946879101 conn(0x7f05200cc7e0 msgr2=0x7f05200ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.026+0000 7f0527bd6700 1 -- 172.21.15.45:0/3946879101 shutdown_connections 2023-08-27T11:28:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.026+0000 7f0527bd6700 1 -- 172.21.15.45:0/3946879101 wait complete. 2023-08-27T11:28:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.027+0000 7f0527bd6700 1 Processor -- start 2023-08-27T11:28:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.027+0000 7f0527bd6700 1 -- start start 2023-08-27T11:28:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.027+0000 7f0527bd6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05200cd300 0x7f05200c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0527bd6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052015a0f0 0x7f0520155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0527bd6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f052015e030 0x7f0520151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0527bd6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05200c5760 con 0x7f05200cd300 2023-08-27T11:28:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0527bd6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05200c4b50 con 0x7f052015a0f0 2023-08-27T11:28:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0527bd6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05200c4e30 con 0x7f052015e030 2023-08-27T11:28:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525972700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05200cd300 0x7f05200c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525972700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05200cd300 0x7f05200c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60018/0 (socket says 172.21.15.45:60018) 2023-08-27T11:28:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525972700 1 -- 172.21.15.45:0/2813687540 learned_addr learned my addr 172.21.15.45:0/2813687540 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525171700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052015a0f0 0x7f0520155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0526173700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f052015e030 0x7f0520151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525972700 1 -- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f052015e030 msgr2=0x7f0520151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525972700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f052015e030 0x7f0520151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525972700 1 -- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052015a0f0 msgr2=0x7f0520155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525972700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052015a0f0 0x7f0520155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525972700 1 -- 172.21.15.45:0/2813687540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c009b30 con 0x7f05200cd300 2023-08-27T11:28:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0526173700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f052015e030 0x7f0520151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0525972700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05200cd300 0x7f05200c5bf0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f051c00b370 tx=0x7f051c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f05167fc700 1 -- 172.21.15.45:0/2813687540 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c004250 con 0x7f05200cd300 2023-08-27T11:28:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f05167fc700 1 -- 172.21.15.45:0/2813687540 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c0043b0 con 0x7f05200cd300 2023-08-27T11:28:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05200c5110 con 0x7f05200cd300 2023-08-27T11:28:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f05167fc700 1 -- 172.21.15.45:0/2813687540 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c004760 con 0x7f05200cd300 2023-08-27T11:28:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.028+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05200c6130 con 0x7f05200cd300 2023-08-27T11:28:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.029+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05200c62c0 con 0x7f05200cd300 2023-08-27T11:28:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.030+0000 7f05167fc700 1 -- 172.21.15.45:0/2813687540 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f051c01b430 con 0x7f05200cd300 2023-08-27T11:28:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.030+0000 7f05167fc700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f050406a170 0x7f050406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.030+0000 7f05167fc700 1 -- 172.21.15.45:0/2813687540 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6805+0+0 (secure 0 0 0) 0x7f051c089be0 con 0x7f05200cd300 2023-08-27T11:28:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.030+0000 7f0525171700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f050406a170 0x7f050406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.031+0000 7f0525171700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f050406a170 0x7f050406c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f05201514c0 tx=0x7f0510008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.035+0000 7f05167fc700 1 -- 172.21.15.45:0/2813687540 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f051c054f90 con 0x7f05200cd300 2023-08-27T11:28:38.049 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:38.048+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:38.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:38.238+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f05200726d0 con 0x7f05200cd300 2023-08-27T11:28:38.812 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:38.810+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:39.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.053+0000 7f05167fc700 1 -- 172.21.15.45:0/2813687540 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7f051c0585b0 con 0x7f05200cd300 2023-08-27T11:28:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f050406a170 msgr2=0x7f050406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f050406a170 0x7f050406c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f05201514c0 tx=0x7f0510008040 comp rx=0 tx=0).stop 2023-08-27T11:28:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05200cd300 msgr2=0x7f05200c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.058 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05200cd300 0x7f05200c5bf0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f051c00b370 tx=0x7f051c004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 shutdown_connections 2023-08-27T11:28:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f050406a170 0x7f050406c630 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05200cd300 0x7f05200c5bf0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f052015e030 0x7f0520151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 --2- 172.21.15.45:0/2813687540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052015a0f0 0x7f0520155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 >> 172.21.15.45:0/2813687540 conn(0x7f05200cc7e0 msgr2=0x7f052006f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 shutdown_connections 2023-08-27T11:28:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.057+0000 7f0527bd6700 1 -- 172.21.15.45:0/2813687540 wait complete. 2023-08-27T11:28:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-08-27T11:28:39.071 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-27T11:28:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.202+0000 7fc031d8e700 1 Processor -- start 2023-08-27T11:28:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.202+0000 7fc031d8e700 1 -- start start 2023-08-27T11:28:39.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc031d8e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c15d0a0 0x7fc02c06ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc031d8e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc02c0d0a50 0x7fc02c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc031d8e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c161870 con 0x7fc02c15d480 2023-08-27T11:28:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc031d8e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c06d3f0 con 0x7fc02c0d0a50 2023-08-27T11:28:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc031d8e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c161da0 con 0x7fc02c15d0a0 2023-08-27T11:28:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc02c0d0a50 0x7fc02c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02affd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc02c0d0a50 0x7fc02c0cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60026/0 (socket says 172.21.15.45:60026) 2023-08-27T11:28:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02affd700 1 -- 172.21.15.45:0/2391124113 learned_addr learned my addr 172.21.15.45:0/2391124113 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02b7fe700 1 --2- 172.21.15.45:0/2391124113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c15d0a0 0x7fc02c06ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02affd700 1 -- 172.21.15.45:0/2391124113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c15d0a0 msgr2=0x7fc02c06ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:39.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02affd700 1 --2- 172.21.15.45:0/2391124113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c15d0a0 0x7fc02c06ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02affd700 1 -- 172.21.15.45:0/2391124113 >> v1:172.21.15.135:6789/0 conn(0x7fc02c15d480 legacy=0x7fc02c0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02affd700 1 -- 172.21.15.45:0/2391124113 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c161ac0 con 0x7fc02c0d0a50 2023-08-27T11:28:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02b7fe700 1 --2- 172.21.15.45:0/2391124113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c15d0a0 0x7fc02c06ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02affd700 1 --2- 172.21.15.45:0/2391124113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c0d0a50 0x7fc02c0cd880 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc0200098f0 tx=0x7fc020009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2ee603d944426c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02a7fc700 1 -- 172.21.15.45:0/2391124113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02000b8a0 con 0x7fc02c0d0a50 2023-08-27T11:28:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.203+0000 7fc02a7fc700 1 -- 172.21.15.45:0/2391124113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02000ba00 con 0x7fc02c0d0a50 2023-08-27T11:28:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.204+0000 7fc02a7fc700 1 -- 172.21.15.45:0/2391124113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02000bd90 con 0x7fc02c0d0a50 2023-08-27T11:28:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.204+0000 7fc031d8e700 1 -- 172.21.15.45:0/2391124113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c0d0a50 msgr2=0x7fc02c0cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.204+0000 7fc031d8e700 1 --2- 172.21.15.45:0/2391124113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c0d0a50 0x7fc02c0cd880 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc0200098f0 tx=0x7fc020009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.204+0000 7fc031d8e700 1 -- 172.21.15.45:0/2391124113 shutdown_connections 2023-08-27T11:28:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.204+0000 7fc031d8e700 1 --2- 172.21.15.45:0/2391124113 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c15d0a0 0x7fc02c06ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.204+0000 7fc031d8e700 1 --2- 172.21.15.45:0/2391124113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c0d0a50 0x7fc02c0cd880 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.204+0000 7fc031d8e700 1 -- 172.21.15.45:0/2391124113 >> 172.21.15.45:0/2391124113 conn(0x7fc02c06f960 msgr2=0x7fc02c06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.204+0000 7fc031d8e700 1 -- 172.21.15.45:0/2391124113 shutdown_connections 2023-08-27T11:28:39.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.204+0000 7fc031d8e700 1 -- 172.21.15.45:0/2391124113 wait complete. 2023-08-27T11:28:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.205+0000 7fc031d8e700 1 Processor -- start 2023-08-27T11:28:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc031d8e700 1 -- start start 2023-08-27T11:28:39.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc031d8e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c0d0a50 0x7fc02c0af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc031d8e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc02c15d0a0 0x7fc02c0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc031d8e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c15d480 0x7fc02c0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc031d8e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0c4e10 con 0x7fc02c15d480 2023-08-27T11:28:39.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc031d8e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0c50f0 con 0x7fc02c15d0a0 2023-08-27T11:28:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc031d8e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0c53d0 con 0x7fc02c0d0a50 2023-08-27T11:28:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc02bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c15d480 0x7fc02c0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc02bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c15d480 0x7fc02c0b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60036/0 (socket says 172.21.15.45:60036) 2023-08-27T11:28:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc02bfff700 1 -- 172.21.15.45:0/3221267350 learned_addr learned my addr 172.21.15.45:0/3221267350 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc02b7fe700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c0d0a50 0x7fc02c0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.206+0000 7fc02affd700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc02c15d0a0 0x7fc02c0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc02bfff700 1 -- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c0d0a50 msgr2=0x7fc02c0af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc02bfff700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c0d0a50 0x7fc02c0af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc02bfff700 1 -- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc02c15d0a0 msgr2=0x7fc02c0c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc02bfff700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc02c15d0a0 0x7fc02c0c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc02bfff700 1 -- 172.21.15.45:0/3221267350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020009550 con 0x7fc02c15d480 2023-08-27T11:28:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc02bfff700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c15d480 0x7fc02c0b1b10 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc02400cc50 tx=0x7fc02400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc013fff700 1 -- 172.21.15.45:0/3221267350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024003c00 con 0x7fc02c15d480 2023-08-27T11:28:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc013fff700 1 -- 172.21.15.45:0/3221267350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024003d60 con 0x7fc02c15d480 2023-08-27T11:28:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c0c56b0 con 0x7fc02c15d480 2023-08-27T11:28:39.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc013fff700 1 -- 172.21.15.45:0/3221267350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400aab0 con 0x7fc02c15d480 2023-08-27T11:28:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.207+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c0c63f0 con 0x7fc02c15d480 2023-08-27T11:28:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.208+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c0c5bd0 con 0x7fc02c15d480 2023-08-27T11:28:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.208+0000 7fc013fff700 1 -- 172.21.15.45:0/3221267350 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fc02400ac10 con 0x7fc02c15d480 2023-08-27T11:28:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.209+0000 7fc013fff700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fc01806a170 0x7fc01806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.209+0000 7fc013fff700 1 -- 172.21.15.45:0/3221267350 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fc024088670 con 0x7fc02c15d480 2023-08-27T11:28:39.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.209+0000 7fc02b7fe700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fc01806a170 0x7fc01806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.210+0000 7fc02b7fe700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fc01806a170 0x7fc01806c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc01c006f10 tx=0x7fc01c008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.213+0000 7fc013fff700 1 -- 172.21.15.45:0/3221267350 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc02c0c5bd0 con 0x7fc02c15d480 2023-08-27T11:28:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.420+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc02c0c5bd0 con 0x7fc02c15d480 2023-08-27T11:28:39.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.423+0000 7fc013fff700 1 -- 172.21.15.45:0/3221267350 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc02c0c5bd0 con 0x7fc02c15d480 2023-08-27T11:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.425+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fc01806a170 msgr2=0x7fc01806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fc01806a170 0x7fc01806c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc01c006f10 tx=0x7fc01c008040 comp rx=0 tx=0).stop 2023-08-27T11:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c15d480 msgr2=0x7fc02c0b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c15d480 0x7fc02c0b1b10 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc02400cc50 tx=0x7fc02400cf60 comp rx=0 tx=0).stop 2023-08-27T11:28:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 shutdown_connections 2023-08-27T11:28:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fc01806a170 0x7fc01806c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc02c15d480 0x7fc02c0b1b10 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc02c0d0a50 0x7fc02c0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 --2- 172.21.15.45:0/3221267350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc02c15d0a0 0x7fc02c0c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 >> 172.21.15.45:0/3221267350 conn(0x7fc02c06f960 msgr2=0x7fc02c06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 shutdown_connections 2023-08-27T11:28:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.426+0000 7fc031d8e700 1 -- 172.21.15.45:0/3221267350 wait complete. 2023-08-27T11:28:39.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-27T11:28:39.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.567+0000 7f9482d86700 1 Processor -- start 2023-08-27T11:28:39.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.567+0000 7f9482d86700 1 -- start start 2023-08-27T11:28:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9482d86700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f947c15d2c0 0x7f947c0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9482d86700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c0ce4b0 0x7f947c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9482d86700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947c15d6a0 con 0x7f947c0b8c20 2023-08-27T11:28:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9482d86700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947c0d2c90 con 0x7f947c15d2c0 2023-08-27T11:28:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9482d86700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0d1bf0 con 0x7f947c0ce4b0 2023-08-27T11:28:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9480b22700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f947c15d2c0 0x7f947c0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9480b22700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f947c15d2c0 0x7f947c0d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60040/0 (socket says 172.21.15.45:60040) 2023-08-27T11:28:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9480b22700 1 -- 172.21.15.45:0/3002799483 learned_addr learned my addr 172.21.15.45:0/3002799483 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9481323700 1 --2- 172.21.15.45:0/3002799483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c0ce4b0 0x7f947c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9480b22700 1 -- 172.21.15.45:0/3002799483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c0ce4b0 msgr2=0x7f947c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9480b22700 1 --2- 172.21.15.45:0/3002799483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c0ce4b0 0x7f947c0b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9480b22700 1 -- 172.21.15.45:0/3002799483 >> v1:172.21.15.135:6789/0 conn(0x7f947c0b8c20 legacy=0x7f947c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9480b22700 1 -- 172.21.15.45:0/3002799483 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0d1ed0 con 0x7f947c15d2c0 2023-08-27T11:28:39.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f9480b22700 1 --2- 172.21.15.45:0/3002799483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c15d2c0 0x7f947c0d13a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9470009920 tx=0x7f9470009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f62a9b08f5e578 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f947b7fe700 1 -- 172.21.15.45:0/3002799483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470011830 con 0x7f947c15d2c0 2023-08-27T11:28:39.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f947b7fe700 1 -- 172.21.15.45:0/3002799483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9470011990 con 0x7f947c15d2c0 2023-08-27T11:28:39.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.568+0000 7f947b7fe700 1 -- 172.21.15.45:0/3002799483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470011d20 con 0x7f947c15d2c0 2023-08-27T11:28:39.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.569+0000 7f9482d86700 1 -- 172.21.15.45:0/3002799483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c15d2c0 msgr2=0x7f947c0d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.569+0000 7f9482d86700 1 --2- 172.21.15.45:0/3002799483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c15d2c0 0x7f947c0d13a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9470009920 tx=0x7f9470009c30 comp rx=0 tx=0).stop 2023-08-27T11:28:39.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.569+0000 7f9482d86700 1 -- 172.21.15.45:0/3002799483 shutdown_connections 2023-08-27T11:28:39.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.569+0000 7f9482d86700 1 --2- 172.21.15.45:0/3002799483 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c0ce4b0 0x7f947c0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.569+0000 7f9482d86700 1 --2- 172.21.15.45:0/3002799483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c15d2c0 0x7f947c0d13a0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.569+0000 7f9482d86700 1 -- 172.21.15.45:0/3002799483 >> 172.21.15.45:0/3002799483 conn(0x7f947c0d2fd0 msgr2=0x7f947c0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.569+0000 7f9482d86700 1 -- 172.21.15.45:0/3002799483 shutdown_connections 2023-08-27T11:28:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.569+0000 7f9482d86700 1 -- 172.21.15.45:0/3002799483 wait complete. 2023-08-27T11:28:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.570+0000 7f9482d86700 1 Processor -- start 2023-08-27T11:28:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.570+0000 7f9482d86700 1 -- start start 2023-08-27T11:28:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f9482d86700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f947c0b8c20 0x7f947c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f9482d86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c0ce4b0 0x7f947c0b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f9482d86700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c15d2c0 0x7f947c1f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f9482d86700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0b59e0 con 0x7f947c0ce4b0 2023-08-27T11:28:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f9482d86700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0b4d00 con 0x7f947c0b8c20 2023-08-27T11:28:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f9482d86700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0cc8d0 con 0x7f947c15d2c0 2023-08-27T11:28:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f947bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c0ce4b0 0x7f947c0b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f947bfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c0ce4b0 0x7f947c0b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60048/0 (socket says 172.21.15.45:60048) 2023-08-27T11:28:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f947bfff700 1 -- 172.21.15.45:0/1657922522 learned_addr learned my addr 172.21.15.45:0/1657922522 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f9480b22700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f947c0b8c20 0x7f947c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.571+0000 7f9481323700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c15d2c0 0x7f947c1f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f947bfff700 1 -- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c15d2c0 msgr2=0x7f947c1f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f947bfff700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c15d2c0 0x7f947c1f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f947bfff700 1 -- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f947c0b8c20 msgr2=0x7f947c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f947bfff700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f947c0b8c20 0x7f947c0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f947bfff700 1 -- 172.21.15.45:0/1657922522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470009550 con 0x7f947c0ce4b0 2023-08-27T11:28:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f9480b22700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f947c0b8c20 0x7f947c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f947bfff700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c0ce4b0 0x7f947c0b54a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f946800c4e0 tx=0x7f946800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f94797fa700 1 -- 172.21.15.45:0/1657922522 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680052d0 con 0x7f947c0ce4b0 2023-08-27T11:28:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f94797fa700 1 -- 172.21.15.45:0/1657922522 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468005430 con 0x7f947c0ce4b0 2023-08-27T11:28:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c0ccbb0 con 0x7f947c0ce4b0 2023-08-27T11:28:39.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f94797fa700 1 -- 172.21.15.45:0/1657922522 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680038c0 con 0x7f947c0ce4b0 2023-08-27T11:28:39.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.572+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c0b4f80 con 0x7f947c0ce4b0 2023-08-27T11:28:39.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.573+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c04e620 con 0x7f947c0ce4b0 2023-08-27T11:28:39.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.574+0000 7f94797fa700 1 -- 172.21.15.45:0/1657922522 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f9468003a20 con 0x7f947c0ce4b0 2023-08-27T11:28:39.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.574+0000 7f94797fa700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f946006a170 0x7f946006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.574+0000 7f94797fa700 1 -- 172.21.15.45:0/1657922522 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f9468087990 con 0x7f947c0ce4b0 2023-08-27T11:28:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.575+0000 7f9480b22700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f946006a170 0x7f946006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.575+0000 7f9480b22700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f946006a170 0x7f946006c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9470014040 tx=0x7f9470008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.579+0000 7f94797fa700 1 -- 172.21.15.45:0/1657922522 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9468052fd0 con 0x7f947c0ce4b0 2023-08-27T11:28:39.605 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:39.600+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.801+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f947c15d6a0 con 0x7f947c0ce4b0 2023-08-27T11:28:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.805+0000 7f94797fa700 1 -- 172.21.15.45:0/1657922522 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v32) v1 ==== 120+0+0 (secure 0 0 0) 0x7f947c15d6a0 con 0x7f947c0ce4b0 2023-08-27T11:28:39.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f946006a170 msgr2=0x7f946006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f946006a170 0x7f946006c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9470014040 tx=0x7f9470008030 comp rx=0 tx=0).stop 2023-08-27T11:28:39.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c0ce4b0 msgr2=0x7f947c0b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c0ce4b0 0x7f947c0b54a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f946800c4e0 tx=0x7f946800c8a0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 shutdown_connections 2023-08-27T11:28:39.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f946006a170 0x7f946006c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f947c15d2c0 0x7f947c1f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f947c0b8c20 0x7f947c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 --2- 172.21.15.45:0/1657922522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f947c0ce4b0 0x7f947c0b54a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 >> 172.21.15.45:0/1657922522 conn(0x7f947c0d2fd0 msgr2=0x7f947c15b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:39.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 shutdown_connections 2023-08-27T11:28:39.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.808+0000 7f9482d86700 1 -- 172.21.15.45:0/1657922522 wait complete. 2023-08-27T11:28:39.821 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-27T11:28:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.948+0000 7ff56f54b700 1 Processor -- start 2023-08-27T11:28:39.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.949+0000 7ff56f54b700 1 -- start start 2023-08-27T11:28:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.949+0000 7ff56f54b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff56815d340 0x7ff568160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.949+0000 7ff56f54b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 0x7ff56806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56f54b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56806b160 con 0x7ff5680cdd00 2023-08-27T11:28:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56f54b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff568072010 con 0x7ff56815d340 2023-08-27T11:28:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56f54b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680af590 con 0x7ff5680cd880 2023-08-27T11:28:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56dae8700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff5680cdd00 0x7ff5680b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46830/0 (socket says 172.21.15.45:46830) 2023-08-27T11:28:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56dae8700 1 -- 172.21.15.45:0/1689144625 learned_addr learned my addr 172.21.15.45:0/1689144625 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56d2e7700 1 --2- 172.21.15.45:0/1689144625 >> v2:172.21.15.45:3300/0 conn(0x7ff56815d340 0x7ff568160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56cae6700 1 --2- 172.21.15.45:0/1689144625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 0x7ff56806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56d2e7700 1 -- 172.21.15.45:0/1689144625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 msgr2=0x7ff56806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56d2e7700 1 --2- 172.21.15.45:0/1689144625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 0x7ff56806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56d2e7700 1 -- 172.21.15.45:0/1689144625 >> v1:172.21.15.45:6789/0 conn(0x7ff5680cdd00 legacy=0x7ff5680b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff55ffff700 1 -- 172.21.15.45:0/1689144625 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857810221 0 0) 0x7ff56806b160 con 0x7ff5680cdd00 2023-08-27T11:28:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56d2e7700 1 -- 172.21.15.45:0/1689144625 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680af2b0 con 0x7ff56815d340 2023-08-27T11:28:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff56d2e7700 1 --2- 172.21.15.45:0/1689144625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56815d340 0x7ff568160d00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff564009840 tx=0x7ff564009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a6251b9f062d64 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff55ffff700 1 -- 172.21.15.45:0/1689144625 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564011830 con 0x7ff56815d340 2023-08-27T11:28:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff55ffff700 1 -- 172.21.15.45:0/1689144625 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff564011990 con 0x7ff56815d340 2023-08-27T11:28:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.950+0000 7ff55ffff700 1 -- 172.21.15.45:0/1689144625 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564011d20 con 0x7ff56815d340 2023-08-27T11:28:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.951+0000 7ff56f54b700 1 -- 172.21.15.45:0/1689144625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56815d340 msgr2=0x7ff568160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.951+0000 7ff56f54b700 1 --2- 172.21.15.45:0/1689144625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56815d340 0x7ff568160d00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff564009840 tx=0x7ff564009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.951+0000 7ff56f54b700 1 -- 172.21.15.45:0/1689144625 shutdown_connections 2023-08-27T11:28:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.951+0000 7ff56f54b700 1 --2- 172.21.15.45:0/1689144625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff56815d340 0x7ff568160d00 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.951+0000 7ff56f54b700 1 --2- 172.21.15.45:0/1689144625 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 0x7ff56806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.951+0000 7ff56f54b700 1 -- 172.21.15.45:0/1689144625 >> 172.21.15.45:0/1689144625 conn(0x7ff5680cc7c0 msgr2=0x7ff5680ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.951+0000 7ff56f54b700 1 -- 172.21.15.45:0/1689144625 shutdown_connections 2023-08-27T11:28:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.951+0000 7ff56f54b700 1 -- 172.21.15.45:0/1689144625 wait complete. 2023-08-27T11:28:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.952+0000 7ff56f54b700 1 Processor -- start 2023-08-27T11:28:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.952+0000 7ff56f54b700 1 -- start start 2023-08-27T11:28:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56f54b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 0x7ff5680c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56f54b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5680cdd00 0x7ff568159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56f54b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56815d340 0x7ff5680cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56f54b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680c37c0 con 0x7ff5680cdd00 2023-08-27T11:28:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56f54b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680cbf50 con 0x7ff56815d340 2023-08-27T11:28:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56f54b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680c3a10 con 0x7ff5680cd880 2023-08-27T11:28:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56cae6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5680cdd00 0x7ff568159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56cae6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5680cdd00 0x7ff568159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60062/0 (socket says 172.21.15.45:60062) 2023-08-27T11:28:39.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56cae6700 1 -- 172.21.15.45:0/2353476588 learned_addr learned my addr 172.21.15.45:0/2353476588 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:39.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56d2e7700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 0x7ff5680c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.953+0000 7ff56dae8700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56815d340 0x7ff5680cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff56cae6700 1 -- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 msgr2=0x7ff5680c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff56cae6700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 0x7ff5680c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff56cae6700 1 -- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56815d340 msgr2=0x7ff5680cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:39.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff56cae6700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56815d340 0x7ff5680cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:39.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff56cae6700 1 -- 172.21.15.45:0/2353476588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640094a0 con 0x7ff5680cdd00 2023-08-27T11:28:39.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff56cae6700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5680cdd00 0x7ff568159fe0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff55800e870 tx=0x7ff55800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff55dffb700 1 -- 172.21.15.45:0/2353476588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558009590 con 0x7ff5680cdd00 2023-08-27T11:28:39.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff55dffb700 1 -- 172.21.15.45:0/2353476588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff558013680 con 0x7ff5680cdd00 2023-08-27T11:28:39.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5680c3cf0 con 0x7ff5680cdd00 2023-08-27T11:28:39.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff55dffb700 1 -- 172.21.15.45:0/2353476588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5580139f0 con 0x7ff5680cdd00 2023-08-27T11:28:39.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.954+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff568151780 con 0x7ff5680cdd00 2023-08-27T11:28:39.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.955+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff568072690 con 0x7ff5680cdd00 2023-08-27T11:28:39.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.956+0000 7ff55dffb700 1 -- 172.21.15.45:0/2353476588 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7ff558013b50 con 0x7ff5680cdd00 2023-08-27T11:28:39.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.956+0000 7ff55dffb700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff54c06a170 0x7ff54c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:39.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.956+0000 7ff55dffb700 1 -- 172.21.15.45:0/2353476588 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7ff5580884e0 con 0x7ff5680cdd00 2023-08-27T11:28:39.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.956+0000 7ff56d2e7700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff54c06a170 0x7ff54c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:39.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.957+0000 7ff56d2e7700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff54c06a170 0x7ff54c06c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff564000c00 tx=0x7ff56401c040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:39.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:39.961+0000 7ff55dffb700 1 -- 172.21.15.45:0/2353476588 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff5580575b0 con 0x7ff5680cdd00 2023-08-27T11:28:40.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.169+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff5680734a0 con 0x7ff5680cdd00 2023-08-27T11:28:40.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.173+0000 7ff55dffb700 1 -- 172.21.15.45:0/2353476588 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v33) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff558057140 con 0x7ff5680cdd00 2023-08-27T11:28:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff54c06a170 msgr2=0x7ff54c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff54c06a170 0x7ff54c06c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff564000c00 tx=0x7ff56401c040 comp rx=0 tx=0).stop 2023-08-27T11:28:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5680cdd00 msgr2=0x7ff568159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5680cdd00 0x7ff568159fe0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff55800e870 tx=0x7ff55800eb80 comp rx=0 tx=0).stop 2023-08-27T11:28:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 shutdown_connections 2023-08-27T11:28:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff54c06a170 0x7ff54c06c630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff56815d340 0x7ff5680cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.177 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff5680cd880 0x7ff5680c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 --2- 172.21.15.45:0/2353476588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5680cdd00 0x7ff568159fe0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 >> 172.21.15.45:0/2353476588 conn(0x7ff5680cc7c0 msgr2=0x7ff56815bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:40.178 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 shutdown_connections 2023-08-27T11:28:40.179 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.175+0000 7ff56f54b700 1 -- 172.21.15.45:0/2353476588 wait complete. 2023-08-27T11:28:40.189 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-08-27T11:28:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.318+0000 7f7050b67700 1 Processor -- start 2023-08-27T11:28:40.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.318+0000 7f7050b67700 1 -- start start 2023-08-27T11:28:40.320 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f7050b67700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c070e40 0x7f704c071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f7050b67700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f704c0cb380 0x7f704c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f7050b67700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c0caef0 con 0x7f704c15dbc0 2023-08-27T11:28:40.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f7050b67700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704c0cac50 con 0x7f704c0cb380 2023-08-27T11:28:40.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f7050b67700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0ce000 con 0x7f704c070e40 2023-08-27T11:28:40.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f704ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f704c0cb380 0x7f704c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f704ad9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f704c0cb380 0x7f704c0cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60072/0 (socket says 172.21.15.45:60072) 2023-08-27T11:28:40.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f704ad9d700 1 -- 172.21.15.45:0/1903978857 learned_addr learned my addr 172.21.15.45:0/1903978857 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:40.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f704a59c700 1 --2- 172.21.15.45:0/1903978857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c070e40 0x7f704c071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f7049d9b700 1 --1- 172.21.15.45:0/1903978857 >> v1:172.21.15.135:6789/0 conn(0x7f704c15dbc0 0x7f704c15a210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:42510/0 (socket says 172.21.15.45:42510) 2023-08-27T11:28:40.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f704ad9d700 1 -- 172.21.15.45:0/1903978857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c070e40 msgr2=0x7f704c071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:40.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f704ad9d700 1 --2- 172.21.15.45:0/1903978857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c070e40 0x7f704c071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f704ad9d700 1 -- 172.21.15.45:0/1903978857 >> v1:172.21.15.135:6789/0 conn(0x7f704c15dbc0 legacy=0x7f704c15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.319+0000 7f704ad9d700 1 -- 172.21.15.45:0/1903978857 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c15b5c0 con 0x7f704c0cb380 2023-08-27T11:28:40.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f704ad9d700 1 --2- 172.21.15.45:0/1903978857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c0cb380 0x7f704c0cb7f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7044009e50 tx=0x7f704400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478f3c7ffb5e4d00 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:40.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f704959a700 1 -- 172.21.15.45:0/1903978857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704400e070 con 0x7f704c0cb380 2023-08-27T11:28:40.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f704959a700 1 -- 172.21.15.45:0/1903978857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704400b910 con 0x7f704c0cb380 2023-08-27T11:28:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f704959a700 1 -- 172.21.15.45:0/1903978857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704400bcc0 con 0x7f704c0cb380 2023-08-27T11:28:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f7050b67700 1 -- 172.21.15.45:0/1903978857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c0cb380 msgr2=0x7f704c0cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f7050b67700 1 --2- 172.21.15.45:0/1903978857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c0cb380 0x7f704c0cb7f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7044009e50 tx=0x7f704400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f7050b67700 1 -- 172.21.15.45:0/1903978857 shutdown_connections 2023-08-27T11:28:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f7050b67700 1 --2- 172.21.15.45:0/1903978857 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c070e40 0x7f704c071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f7050b67700 1 --2- 172.21.15.45:0/1903978857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c0cb380 0x7f704c0cb7f0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f7050b67700 1 -- 172.21.15.45:0/1903978857 >> 172.21.15.45:0/1903978857 conn(0x7f704c0cfd60 msgr2=0x7f704c0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.320+0000 7f7050b67700 1 -- 172.21.15.45:0/1903978857 shutdown_connections 2023-08-27T11:28:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.321+0000 7f7050b67700 1 -- 172.21.15.45:0/1903978857 wait complete. 2023-08-27T11:28:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.322+0000 7f7050b67700 1 Processor -- start 2023-08-27T11:28:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.322+0000 7f7050b67700 1 -- start start 2023-08-27T11:28:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f7050b67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c070e40 0x7f704c0c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f7050b67700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f704c0cb380 0x7f704c1487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f7050b67700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c15dbc0 0x7f704c14d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f7050b67700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c14cc70 con 0x7f704c070e40 2023-08-27T11:28:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f7050b67700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0c9d80 con 0x7f704c0cb380 2023-08-27T11:28:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f7050b67700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0ca060 con 0x7f704c15dbc0 2023-08-27T11:28:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c070e40 0x7f704c0c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c070e40 0x7f704c0c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60074/0 (socket says 172.21.15.45:60074) 2023-08-27T11:28:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704a59c700 1 -- 172.21.15.45:0/921700264 learned_addr learned my addr 172.21.15.45:0/921700264 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704ad9d700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c15dbc0 0x7f704c14d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f7049d9b700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f704c0cb380 0x7f704c1487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704a59c700 1 -- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c15dbc0 msgr2=0x7f704c14d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704a59c700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c15dbc0 0x7f704c14d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704a59c700 1 -- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f704c0cb380 msgr2=0x7f704c1487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704a59c700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f704c0cb380 0x7f704c1487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704a59c700 1 -- 172.21.15.45:0/921700264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044009b30 con 0x7f704c070e40 2023-08-27T11:28:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f704a59c700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c070e40 0x7f704c0c9470 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e870 tx=0x7f703c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f703affd700 1 -- 172.21.15.45:0/921700264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c009590 con 0x7f704c070e40 2023-08-27T11:28:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f703affd700 1 -- 172.21.15.45:0/921700264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c013680 con 0x7f704c070e40 2023-08-27T11:28:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c14c900 con 0x7f704c070e40 2023-08-27T11:28:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f703affd700 1 -- 172.21.15.45:0/921700264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c0139f0 con 0x7f704c070e40 2023-08-27T11:28:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.323+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c0c99b0 con 0x7f704c070e40 2023-08-27T11:28:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.324+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c14cf50 con 0x7f704c070e40 2023-08-27T11:28:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.325+0000 7f703affd700 1 -- 172.21.15.45:0/921700264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f703c013b50 con 0x7f704c070e40 2023-08-27T11:28:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.325+0000 7f703affd700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f702c06a170 0x7f702c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.325+0000 7f703affd700 1 -- 172.21.15.45:0/921700264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f703c088450 con 0x7f704c070e40 2023-08-27T11:28:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.326+0000 7f7049d9b700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f702c06a170 0x7f702c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:40.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.326+0000 7f7049d9b700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f702c06a170 0x7f702c06c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f704c1f45a0 tx=0x7f7040007ed0 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:40.334 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.330+0000 7f703affd700 1 -- 172.21.15.45:0/921700264 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f703c057520 con 0x7f704c070e40 2023-08-27T11:28:40.406 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:40.405+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:40.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:40.623+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f704c072180 con 0x7f704c070e40 2023-08-27T11:28:41.056 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.055+0000 7f703affd700 1 -- 172.21.15.45:0/921700264 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v634) v1 ==== 166+0+0 (secure 0 0 0) 0x7f703c0570b0 con 0x7f704c070e40 2023-08-27T11:28:41.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f702c06a170 msgr2=0x7f702c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f702c06a170 0x7f702c06c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f704c1f45a0 tx=0x7f7040007ed0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c070e40 msgr2=0x7f704c0c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c070e40 0x7f704c0c9470 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e870 tx=0x7f703c00eb80 comp rx=0 tx=0).stop 2023-08-27T11:28:41.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 shutdown_connections 2023-08-27T11:28:41.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f702c06a170 0x7f702c06c630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704c070e40 0x7f704c0c9470 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f704c15dbc0 0x7f704c14d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 --2- 172.21.15.45:0/921700264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f704c0cb380 0x7f704c1487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.060 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.057+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 >> 172.21.15.45:0/921700264 conn(0x7f704c0cfd60 msgr2=0x7f704c1654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:41.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.058+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 shutdown_connections 2023-08-27T11:28:41.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.058+0000 7f7050b67700 1 -- 172.21.15.45:0/921700264 wait complete. 2023-08-27T11:28:41.070 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-08-27T11:28:41.071 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-08-27T11:28:41.071 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-27T11:28:41.072 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-08-27T11:28:41.189 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:41.187+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.198+0000 7fea51793700 1 Processor -- start 2023-08-27T11:28:41.200 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.198+0000 7fea51793700 1 -- start start 2023-08-27T11:28:41.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.199+0000 7fea51793700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fea4c15d2a0 0x7fea4c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.199+0000 7fea51793700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea4c0b1330 0x7fea4c0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.199+0000 7fea51793700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c160ea0 con 0x7fea4c0cd970 2023-08-27T11:28:41.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.199+0000 7fea51793700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c161180 con 0x7fea4c0b1330 2023-08-27T11:28:41.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.199+0000 7fea51793700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c161460 con 0x7fea4c15d2a0 2023-08-27T11:28:41.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.199+0000 7fea4b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea4c0b1330 0x7fea4c0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.199+0000 7fea4a7fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fea4c0cd970 0x7fea4c0b0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46858/0 (socket says 172.21.15.45:46858) 2023-08-27T11:28:41.203 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.199+0000 7fea4a7fc700 1 -- 172.21.15.45:0/3242067529 learned_addr learned my addr 172.21.15.45:0/3242067529 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea4b7fe700 1 -- 172.21.15.45:0/3242067529 >> v2:172.21.15.135:3300/0 conn(0x7fea4c15d2a0 msgr2=0x7fea4c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea4affd700 1 --2- 172.21.15.45:0/3242067529 >> v2:172.21.15.135:3300/0 conn(0x7fea4c15d2a0 0x7fea4c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea4b7fe700 1 --2- 172.21.15.45:0/3242067529 >> v2:172.21.15.135:3300/0 conn(0x7fea4c15d2a0 0x7fea4c0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea49ffb700 1 -- 172.21.15.45:0/3242067529 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424804413 0 0) 0x7fea4c160ea0 con 0x7fea4c0cd970 2023-08-27T11:28:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea4b7fe700 1 -- 172.21.15.45:0/3242067529 >> v1:172.21.15.45:6789/0 conn(0x7fea4c0cd970 legacy=0x7fea4c0b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea4b7fe700 1 -- 172.21.15.45:0/3242067529 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c161740 con 0x7fea4c0b1330 2023-08-27T11:28:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea4affd700 1 --2- 172.21.15.45:0/3242067529 >> v2:172.21.15.135:3300/0 conn(0x7fea4c15d2a0 0x7fea4c0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea4b7fe700 1 --2- 172.21.15.45:0/3242067529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c0b1330 0x7fea4c0ca3f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0cfd00 tx=0x7fea4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c756ae47fa4095c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea49ffb700 1 -- 172.21.15.45:0/3242067529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4400e070 con 0x7fea4c0b1330 2023-08-27T11:28:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea49ffb700 1 -- 172.21.15.45:0/3242067529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4400bbd0 con 0x7fea4c0b1330 2023-08-27T11:28:41.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.200+0000 7fea49ffb700 1 -- 172.21.15.45:0/3242067529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44012430 con 0x7fea4c0b1330 2023-08-27T11:28:41.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.201+0000 7fea51793700 1 -- 172.21.15.45:0/3242067529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c0b1330 msgr2=0x7fea4c0ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.201+0000 7fea51793700 1 --2- 172.21.15.45:0/3242067529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c0b1330 0x7fea4c0ca3f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0cfd00 tx=0x7fea4400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:41.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.201+0000 7fea51793700 1 -- 172.21.15.45:0/3242067529 shutdown_connections 2023-08-27T11:28:41.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.201+0000 7fea51793700 1 --2- 172.21.15.45:0/3242067529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c0b1330 0x7fea4c0ca3f0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.201+0000 7fea51793700 1 --2- 172.21.15.45:0/3242067529 >> v2:172.21.15.135:3300/0 conn(0x7fea4c15d2a0 0x7fea4c0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.201+0000 7fea51793700 1 -- 172.21.15.45:0/3242067529 >> 172.21.15.45:0/3242067529 conn(0x7fea4c0b1970 msgr2=0x7fea4c0b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.201+0000 7fea51793700 1 -- 172.21.15.45:0/3242067529 shutdown_connections 2023-08-27T11:28:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.201+0000 7fea51793700 1 -- 172.21.15.45:0/3242067529 wait complete. 2023-08-27T11:28:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.202+0000 7fea51793700 1 Processor -- start 2023-08-27T11:28:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.202+0000 7fea51793700 1 -- start start 2023-08-27T11:28:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea51793700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea4c0b1330 0x7fea4c1f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea51793700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea4c0cd970 0x7fea4c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea51793700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c15d2a0 0x7fea4c06d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea51793700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0c34d0 con 0x7fea4c15d2a0 2023-08-27T11:28:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea51793700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c154560 con 0x7fea4c0b1330 2023-08-27T11:28:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea51793700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c154790 con 0x7fea4c0cd970 2023-08-27T11:28:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c15d2a0 0x7fea4c06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4a7fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea4c0cd970 0x7fea4c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4affd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea4c0b1330 0x7fea4c1f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c15d2a0 0x7fea4c06d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60090/0 (socket says 172.21.15.45:60090) 2023-08-27T11:28:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4b7fe700 1 -- 172.21.15.45:0/1633883665 learned_addr learned my addr 172.21.15.45:0/1633883665 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:41.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4b7fe700 1 -- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea4c0cd970 msgr2=0x7fea4c154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4b7fe700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea4c0cd970 0x7fea4c154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4b7fe700 1 -- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea4c0b1330 msgr2=0x7fea4c1f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:41.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4b7fe700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea4c0b1330 0x7fea4c1f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:41.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4b7fe700 1 -- 172.21.15.45:0/1633883665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44009a70 con 0x7fea4c15d2a0 2023-08-27T11:28:41.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.203+0000 7fea4affd700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea4c0b1330 0x7fea4c1f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:41.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.204+0000 7fea4b7fe700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c15d2a0 0x7fea4c06d890 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fea4400b370 tx=0x7fea44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:41.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.204+0000 7fea337fe700 1 -- 172.21.15.45:0/1633883665 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44012430 con 0x7fea4c15d2a0 2023-08-27T11:28:41.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.204+0000 7fea337fe700 1 -- 172.21.15.45:0/1633883665 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44020070 con 0x7fea4c15d2a0 2023-08-27T11:28:41.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.204+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c154a70 con 0x7fea4c15d2a0 2023-08-27T11:28:41.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.204+0000 7fea337fe700 1 -- 172.21.15.45:0/1633883665 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44004280 con 0x7fea4c15d2a0 2023-08-27T11:28:41.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.204+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c1553b0 con 0x7fea4c15d2a0 2023-08-27T11:28:41.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.205+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c154c60 con 0x7fea4c15d2a0 2023-08-27T11:28:41.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.205+0000 7fea337fe700 1 -- 172.21.15.45:0/1633883665 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fea440043e0 con 0x7fea4c15d2a0 2023-08-27T11:28:41.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.206+0000 7fea337fe700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fea3806a170 0x7fea3806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:41.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.206+0000 7fea337fe700 1 -- 172.21.15.45:0/1633883665 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fea44022070 con 0x7fea4c15d2a0 2023-08-27T11:28:41.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.206+0000 7fea4affd700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fea3806a170 0x7fea3806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:41.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.207+0000 7fea4affd700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fea3806a170 0x7fea3806c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fea40006fd0 tx=0x7fea40008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:41.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.210+0000 7fea337fe700 1 -- 172.21.15.45:0/1633883665 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fea44055d90 con 0x7fea4c15d2a0 2023-08-27T11:28:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:41.494+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fea4c154e60 con 0x7fea4c15d2a0 2023-08-27T11:28:41.964 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:41.962+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:42.059 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.058+0000 7fea337fe700 1 -- 172.21.15.45:0/1633883665 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v635) v1 ==== 186+0+0 (secure 0 0 0) 0x7fea44012e60 con 0x7fea4c15d2a0 2023-08-27T11:28:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.060+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fea3806a170 msgr2=0x7fea3806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.060+0000 7fea51793700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fea3806a170 0x7fea3806c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fea40006fd0 tx=0x7fea40008040 comp rx=0 tx=0).stop 2023-08-27T11:28:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.060+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c15d2a0 msgr2=0x7fea4c06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.060+0000 7fea51793700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c15d2a0 0x7fea4c06d890 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fea4400b370 tx=0x7fea44004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.060+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 shutdown_connections 2023-08-27T11:28:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.060+0000 7fea51793700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fea3806a170 0x7fea3806c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.060+0000 7fea51793700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c15d2a0 0x7fea4c06d890 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.060+0000 7fea51793700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea4c0b1330 0x7fea4c1f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.060+0000 7fea51793700 1 --2- 172.21.15.45:0/1633883665 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea4c0cd970 0x7fea4c154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.061+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 >> 172.21.15.45:0/1633883665 conn(0x7fea4c0b1970 msgr2=0x7fea4c06a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.061+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 shutdown_connections 2023-08-27T11:28:42.065 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.061+0000 7fea51793700 1 -- 172.21.15.45:0/1633883665 wait complete. 2023-08-27T11:28:42.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.201+0000 7f7a1342e700 1 Processor -- start 2023-08-27T11:28:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.201+0000 7f7a1342e700 1 -- start start 2023-08-27T11:28:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a1342e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a0c06a280 0x7f7a0c0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a1342e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a740 0x7f7a0c0aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a1342e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c0b0d50 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a1342e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c073ef0 con 0x7f7a0c06a280 2023-08-27T11:28:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a1342e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0cc6c0 con 0x7f7a0c06a740 2023-08-27T11:28:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a109c9700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a0c06a280 0x7f7a0c0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a111ca700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7a0c06b2e0 0x7f7a0c0b0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46870/0 (socket says 172.21.15.45:46870) 2023-08-27T11:28:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a111ca700 1 -- 172.21.15.45:0/3520369881 learned_addr learned my addr 172.21.15.45:0/3520369881 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a119cb700 1 --2- 172.21.15.45:0/3520369881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a740 0x7f7a0c0aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a109c9700 1 -- 172.21.15.45:0/3520369881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a740 msgr2=0x7f7a0c0aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a109c9700 1 --2- 172.21.15.45:0/3520369881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a740 0x7f7a0c0aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a109c9700 1 -- 172.21.15.45:0/3520369881 >> v1:172.21.15.45:6789/0 conn(0x7f7a0c06b2e0 legacy=0x7f7a0c0b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a03fff700 1 -- 172.21.15.45:0/3520369881 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56553399 0 0) 0x7f7a0c0b0d50 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a109c9700 1 -- 172.21.15.45:0/3520369881 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c073930 con 0x7f7a0c06a280 2023-08-27T11:28:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a119cb700 1 --2- 172.21.15.45:0/3520369881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a740 0x7f7a0c0aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a109c9700 1 --2- 172.21.15.45:0/3520369881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06a280 0x7f7a0c0ae610 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00bae0 tx=0x7f79fc00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efa32c21c4ca590 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a03fff700 1 -- 172.21.15.45:0/3520369881 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc00dd30 con 0x7f7a0c06a280 2023-08-27T11:28:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a03fff700 1 -- 172.21.15.45:0/3520369881 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc00de90 con 0x7f7a0c06a280 2023-08-27T11:28:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.202+0000 7f7a03fff700 1 -- 172.21.15.45:0/3520369881 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc010690 con 0x7f7a0c06a280 2023-08-27T11:28:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.203+0000 7f7a1342e700 1 -- 172.21.15.45:0/3520369881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06a280 msgr2=0x7f7a0c0ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.203+0000 7f7a1342e700 1 --2- 172.21.15.45:0/3520369881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06a280 0x7f7a0c0ae610 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00bae0 tx=0x7f79fc00bdf0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.203+0000 7f7a1342e700 1 -- 172.21.15.45:0/3520369881 shutdown_connections 2023-08-27T11:28:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.203+0000 7f7a1342e700 1 --2- 172.21.15.45:0/3520369881 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a740 0x7f7a0c0aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.203+0000 7f7a1342e700 1 --2- 172.21.15.45:0/3520369881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06a280 0x7f7a0c0ae610 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.203+0000 7f7a1342e700 1 -- 172.21.15.45:0/3520369881 >> 172.21.15.45:0/3520369881 conn(0x7f7a0c0b2460 msgr2=0x7f7a0c0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.203+0000 7f7a1342e700 1 -- 172.21.15.45:0/3520369881 shutdown_connections 2023-08-27T11:28:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.203+0000 7f7a1342e700 1 -- 172.21.15.45:0/3520369881 wait complete. 2023-08-27T11:28:42.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.204+0000 7f7a1342e700 1 Processor -- start 2023-08-27T11:28:42.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.204+0000 7f7a1342e700 1 -- start start 2023-08-27T11:28:42.212 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a1342e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a280 0x7f7a0c151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a1342e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a0c06a740 0x7f7a0c1f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a1342e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06b2e0 0x7f7a0c0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a1342e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c08b9f0 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a1342e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c151ec0 con 0x7f7a0c06a740 2023-08-27T11:28:42.213 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a1342e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c099650 con 0x7f7a0c06a280 2023-08-27T11:28:42.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a119cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06b2e0 0x7f7a0c0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a111ca700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a280 0x7f7a0c151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a119cb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06b2e0 0x7f7a0c0ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60104/0 (socket says 172.21.15.45:60104) 2023-08-27T11:28:42.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a109c9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a0c06a740 0x7f7a0c1f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.205+0000 7f7a119cb700 1 -- 172.21.15.45:0/1337114723 learned_addr learned my addr 172.21.15.45:0/1337114723 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:42.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a119cb700 1 -- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a280 msgr2=0x7f7a0c151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a119cb700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a280 0x7f7a0c151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a119cb700 1 -- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a0c06a740 msgr2=0x7f7a0c1f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a119cb700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a0c06a740 0x7f7a0c1f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.215 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a119cb700 1 -- 172.21.15.45:0/1337114723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc00b740 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a119cb700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06b2e0 0x7f7a0c0ca960 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400cef0 tx=0x7f7a0400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a01ffb700 1 -- 172.21.15.45:0/1337114723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0400ad00 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a01ffb700 1 -- 172.21.15.45:0/1337114723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0400ae60 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c099930 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.216 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a01ffb700 1 -- 172.21.15.45:0/1337114723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04015680 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.206+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c099ef0 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.207+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c14ef20 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.208+0000 7f7a01ffb700 1 -- 172.21.15.45:0/1337114723 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f7a0400d430 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.208+0000 7f7a01ffb700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f79f006a170 0x7f79f006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.217 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.208+0000 7f7a01ffb700 1 -- 172.21.15.45:0/1337114723 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f7a04088b50 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.208+0000 7f7a111ca700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f79f006a170 0x7f79f006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.209+0000 7f7a111ca700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f79f006a170 0x7f79f006c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7a08007fd0 tx=0x7f7a08009040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.213+0000 7f7a01ffb700 1 -- 172.21.15.45:0/1337114723 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7a0c14ef20 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.420+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a0c14ef20 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.421+0000 7f7a01ffb700 1 -- 172.21.15.45:0/1337114723 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16133 (secure 0 0 0) 0x7f7a0c14ef20 con 0x7f7a0c06b2e0 2023-08-27T11:28:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:42.422 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:39.050400+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6833/3257398445":"2023-08-28T11:28:35.864806+0000","172.21.15.45:6832/3257398445":"2023-08-28T11:28:35.864806+0000","172.21.15.135:0/626141668":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/1023575426":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/233288315":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/3946297931":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6833/3112377367":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6832/3112377367":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6833/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/4034194156":"2023-08-28T11:28:33.686723+0000","172.21.15.135:6832/3812189695":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3142979284":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/569758288":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/129865033":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/823048739":"2023-08-28T11:28:33.686723+0000","172.21.15.135:0/1640490812":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/12937390":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/2560756484":"2023-08-28T11:28:26.474800+0000","172.21.15.135:0/3438903793":"2023-08-27T12:28:16.346002+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f79f006a170 msgr2=0x7f79f006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f79f006a170 0x7f79f006c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7a08007fd0 tx=0x7f7a08009040 comp rx=0 tx=0).stop 2023-08-27T11:28:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06b2e0 msgr2=0x7f7a0c0ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06b2e0 0x7f7a0c0ca960 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400cef0 tx=0x7f7a0400cf20 comp rx=0 tx=0).stop 2023-08-27T11:28:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 shutdown_connections 2023-08-27T11:28:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f79f006a170 0x7f79f006c630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c06b2e0 0x7f7a0c0ca960 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f7a0c06a280 0x7f7a0c151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 --2- 172.21.15.45:0/1337114723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a0c06a740 0x7f7a0c1f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 >> 172.21.15.45:0/1337114723 conn(0x7f7a0c0b2460 msgr2=0x7f7a0c15c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 shutdown_connections 2023-08-27T11:28:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.423+0000 7f7a1342e700 1 -- 172.21.15.45:0/1337114723 wait complete. 2023-08-27T11:28:42.437 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:42.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.567+0000 7f2766a37700 1 Processor -- start 2023-08-27T11:28:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.567+0000 7f2766a37700 1 -- start start 2023-08-27T11:28:42.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f2766a37700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015af30 0x7f27600cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f2766a37700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27600cf960 0x7f27600cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f2766a37700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f276015b310 0x7f27600cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f2766a37700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276006f9c0 con 0x7f27600cf960 2023-08-27T11:28:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f2766a37700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27600cb8b0 con 0x7f276015b310 2023-08-27T11:28:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f2766a37700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27600cecd0 con 0x7f276015af30 2023-08-27T11:28:42.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27600cf960 0x7f27600cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27600cf960 0x7f27600cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60110/0 (socket says 172.21.15.45:60110) 2023-08-27T11:28:42.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275f7fe700 1 -- 172.21.15.45:0/3332806123 learned_addr learned my addr 172.21.15.45:0/3332806123 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:42.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f2764fd4700 1 --2- 172.21.15.45:0/3332806123 >> v2:172.21.15.135:3300/0 conn(0x7f276015b310 0x7f27600cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275ffff700 1 --2- 172.21.15.45:0/3332806123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015af30 0x7f27600cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275f7fe700 1 -- 172.21.15.45:0/3332806123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015af30 msgr2=0x7f27600cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:42.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275f7fe700 1 --2- 172.21.15.45:0/3332806123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015af30 0x7f27600cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275f7fe700 1 -- 172.21.15.45:0/3332806123 >> v2:172.21.15.135:3300/0 conn(0x7f276015b310 msgr2=0x7f27600cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:42.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275f7fe700 1 --2- 172.21.15.45:0/3332806123 >> v2:172.21.15.135:3300/0 conn(0x7f276015b310 0x7f27600cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275f7fe700 1 -- 172.21.15.45:0/3332806123 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600cbae0 con 0x7f27600cf960 2023-08-27T11:28:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275f7fe700 1 --2- 172.21.15.45:0/3332806123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 0x7f27600cd770 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f276015fd50 tx=0x7f2750009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cf0db48fde090a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.568+0000 7f275effd700 1 -- 172.21.15.45:0/3332806123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275000b870 con 0x7f27600cf960 2023-08-27T11:28:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f275effd700 1 -- 172.21.15.45:0/3332806123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275000b9d0 con 0x7f27600cf960 2023-08-27T11:28:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f275effd700 1 -- 172.21.15.45:0/3332806123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275000bd60 con 0x7f27600cf960 2023-08-27T11:28:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f2766a37700 1 -- 172.21.15.45:0/3332806123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 msgr2=0x7f27600cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f2766a37700 1 --2- 172.21.15.45:0/3332806123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 0x7f27600cd770 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f276015fd50 tx=0x7f2750009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f2766a37700 1 -- 172.21.15.45:0/3332806123 shutdown_connections 2023-08-27T11:28:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f2766a37700 1 --2- 172.21.15.45:0/3332806123 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015af30 0x7f27600cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f2766a37700 1 --2- 172.21.15.45:0/3332806123 >> v2:172.21.15.135:3300/0 conn(0x7f276015b310 0x7f27600cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f2766a37700 1 --2- 172.21.15.45:0/3332806123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 0x7f27600cd770 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f2766a37700 1 -- 172.21.15.45:0/3332806123 >> 172.21.15.45:0/3332806123 conn(0x7f276009a1a0 msgr2=0x7f27600d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f2766a37700 1 -- 172.21.15.45:0/3332806123 shutdown_connections 2023-08-27T11:28:42.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.569+0000 7f2766a37700 1 -- 172.21.15.45:0/3332806123 wait complete. 2023-08-27T11:28:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.570+0000 7f2766a37700 1 Processor -- start 2023-08-27T11:28:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.571+0000 7f2766a37700 1 -- start start 2023-08-27T11:28:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.571+0000 7f2766a37700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 0x7f27601f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.571+0000 7f2766a37700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f276015af30 0x7f276006c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.571+0000 7f2766a37700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015b310 0x7f27600ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.571+0000 7f2766a37700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601f36d0 con 0x7f27600cf960 2023-08-27T11:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.571+0000 7f2766a37700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27600c84c0 con 0x7f276015af30 2023-08-27T11:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.571+0000 7f2766a37700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27600c87a0 con 0x7f276015b310 2023-08-27T11:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.571+0000 7f275ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 0x7f27601f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 0x7f27601f3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60114/0 (socket says 172.21.15.45:60114) 2023-08-27T11:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275ffff700 1 -- 172.21.15.45:0/1727693617 learned_addr learned my addr 172.21.15.45:0/1727693617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275f7fe700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f276015af30 0x7f276006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f2764fd4700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015b310 0x7f27600ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275ffff700 1 -- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015b310 msgr2=0x7f27600ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275ffff700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015b310 0x7f27600ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275ffff700 1 -- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f276015af30 msgr2=0x7f276006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275ffff700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f276015af30 0x7f276006c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275ffff700 1 -- 172.21.15.45:0/1727693617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2750009550 con 0x7f27600cf960 2023-08-27T11:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275ffff700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 0x7f27601f3190 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f275400e810 tx=0x7f275400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275cff9700 1 -- 172.21.15.45:0/1727693617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275400c070 con 0x7f27600cf960 2023-08-27T11:28:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275cff9700 1 -- 172.21.15.45:0/1727693617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2754013610 con 0x7f27600cf960 2023-08-27T11:28:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f275cff9700 1 -- 172.21.15.45:0/1727693617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27540139c0 con 0x7f27600cf960 2023-08-27T11:28:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27600c8a80 con 0x7f27600cf960 2023-08-27T11:28:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.572+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27600c9040 con 0x7f27600cf960 2023-08-27T11:28:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.573+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276009ee10 con 0x7f27600cf960 2023-08-27T11:28:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.574+0000 7f275cff9700 1 -- 172.21.15.45:0/1727693617 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f2754013b20 con 0x7f27600cf960 2023-08-27T11:28:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.574+0000 7f275cff9700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f274c06a170 0x7f274c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.574+0000 7f275cff9700 1 -- 172.21.15.45:0/1727693617 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f2754088660 con 0x7f27600cf960 2023-08-27T11:28:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.575+0000 7f275f7fe700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f274c06a170 0x7f274c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.575+0000 7f275f7fe700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f274c06a170 0x7f274c06c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f276015fd50 tx=0x7f2750008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.579+0000 7f275cff9700 1 -- 172.21.15.45:0/1727693617 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2754053a10 con 0x7f27600cf960 2023-08-27T11:28:42.758 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:42.757+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:42.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.783+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27600734a0 con 0x7f27600cf960 2023-08-27T11:28:42.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.783+0000 7f275cff9700 1 -- 172.21.15.45:0/1727693617 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2754057030 con 0x7f27600cf960 2023-08-27T11:28:42.785 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:28:42.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.785+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f274c06a170 msgr2=0x7f274c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.785+0000 7f2766a37700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f274c06a170 0x7f274c06c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f276015fd50 tx=0x7f2750008030 comp rx=0 tx=0).stop 2023-08-27T11:28:42.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.785+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 msgr2=0x7f27601f3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.785+0000 7f2766a37700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 0x7f27601f3190 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f275400e810 tx=0x7f275400eb20 comp rx=0 tx=0).stop 2023-08-27T11:28:42.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.786+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 shutdown_connections 2023-08-27T11:28:42.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.786+0000 7f2766a37700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f274c06a170 0x7f274c06c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.786+0000 7f2766a37700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27600cf960 0x7f27601f3190 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.786+0000 7f2766a37700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f276015b310 0x7f27600ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.786+0000 7f2766a37700 1 --2- 172.21.15.45:0/1727693617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f276015af30 0x7f276006c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.786+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 >> 172.21.15.45:0/1727693617 conn(0x7f276009a1a0 msgr2=0x7f27600b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.786+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 shutdown_connections 2023-08-27T11:28:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.786+0000 7f2766a37700 1 -- 172.21.15.45:0/1727693617 wait complete. 2023-08-27T11:28:42.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.931+0000 7f5ade596700 1 Processor -- start 2023-08-27T11:28:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.931+0000 7f5ade596700 1 -- start start 2023-08-27T11:28:42.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5ade596700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 0x7f5ad80716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5ade596700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5ad80b17c0 0x7f5ad80b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5ade596700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ad806ced0 0x7f5ad806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5ade596700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad8157fc0 con 0x7f5ad806ced0 2023-08-27T11:28:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5ade596700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad806d860 con 0x7f5ad80b17c0 2023-08-27T11:28:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5ade596700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad815b460 con 0x7f5ad8157be0 2023-08-27T11:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5adcb33700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ad806ced0 0x7f5ad806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5adcb33700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ad806ced0 0x7f5ad806d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60126/0 (socket says 172.21.15.45:60126) 2023-08-27T11:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5adcb33700 1 -- 172.21.15.45:0/499760210 learned_addr learned my addr 172.21.15.45:0/499760210 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5ad7fff700 1 --2- 172.21.15.45:0/499760210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 0x7f5ad80716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5ad77fe700 1 --2- 172.21.15.45:0/499760210 >> v2:172.21.15.135:3300/0 conn(0x7f5ad80b17c0 0x7f5ad80b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5adcb33700 1 -- 172.21.15.45:0/499760210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 msgr2=0x7f5ad80716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5adcb33700 1 --2- 172.21.15.45:0/499760210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 0x7f5ad80716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5adcb33700 1 -- 172.21.15.45:0/499760210 >> v2:172.21.15.135:3300/0 conn(0x7f5ad80b17c0 msgr2=0x7f5ad80b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5adcb33700 1 --2- 172.21.15.45:0/499760210 >> v2:172.21.15.135:3300/0 conn(0x7f5ad80b17c0 0x7f5ad80b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.932+0000 7f5adcb33700 1 -- 172.21.15.45:0/499760210 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad80b2280 con 0x7f5ad806ced0 2023-08-27T11:28:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5adcb33700 1 --2- 172.21.15.45:0/499760210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 0x7f5ad806d320 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5ad806c3b0 tx=0x7f5ad000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d628ce02c945ca9 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ad6ffd700 1 -- 172.21.15.45:0/499760210 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000e070 con 0x7f5ad806ced0 2023-08-27T11:28:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ad6ffd700 1 -- 172.21.15.45:0/499760210 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad000b910 con 0x7f5ad806ced0 2023-08-27T11:28:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ad6ffd700 1 -- 172.21.15.45:0/499760210 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000bcc0 con 0x7f5ad806ced0 2023-08-27T11:28:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ade596700 1 -- 172.21.15.45:0/499760210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 msgr2=0x7f5ad806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ade596700 1 --2- 172.21.15.45:0/499760210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 0x7f5ad806d320 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5ad806c3b0 tx=0x7f5ad000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ade596700 1 -- 172.21.15.45:0/499760210 shutdown_connections 2023-08-27T11:28:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ade596700 1 --2- 172.21.15.45:0/499760210 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 0x7f5ad80716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ade596700 1 --2- 172.21.15.45:0/499760210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 0x7f5ad806d320 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ade596700 1 --2- 172.21.15.45:0/499760210 >> v2:172.21.15.135:3300/0 conn(0x7f5ad80b17c0 0x7f5ad80b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ade596700 1 -- 172.21.15.45:0/499760210 >> 172.21.15.45:0/499760210 conn(0x7f5ad80cc7c0 msgr2=0x7f5ad80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ade596700 1 -- 172.21.15.45:0/499760210 shutdown_connections 2023-08-27T11:28:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.933+0000 7f5ade596700 1 -- 172.21.15.45:0/499760210 wait complete. 2023-08-27T11:28:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.935+0000 7f5ade596700 1 Processor -- start 2023-08-27T11:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.935+0000 7f5ade596700 1 -- start start 2023-08-27T11:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ade596700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 0x7f5ad814f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ade596700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ad80b17c0 0x7f5ad815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ade596700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 0x7f5ad815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ade596700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad81534d0 con 0x7f5ad806ced0 2023-08-27T11:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ade596700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad815cb20 con 0x7f5ad80b17c0 2023-08-27T11:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ade596700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad815ce00 con 0x7f5ad8157be0 2023-08-27T11:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ad7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 0x7f5ad814f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ad7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 0x7f5ad814f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60128/0 (socket says 172.21.15.45:60128) 2023-08-27T11:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ad7fff700 1 -- 172.21.15.45:0/4117710374 learned_addr learned my addr 172.21.15.45:0/4117710374 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.936+0000 7f5ad77fe700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ad80b17c0 0x7f5ad815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5adcb33700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 0x7f5ad815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ad7fff700 1 -- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 msgr2=0x7f5ad815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ad7fff700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 0x7f5ad815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ad7fff700 1 -- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ad80b17c0 msgr2=0x7f5ad815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ad7fff700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ad80b17c0 0x7f5ad815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ad7fff700 1 -- 172.21.15.45:0/4117710374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad0009b30 con 0x7f5ad806ced0 2023-08-27T11:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ad7fff700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 0x7f5ad814f460 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00e870 tx=0x7f5acc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ad4ff9700 1 -- 172.21.15.45:0/4117710374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc009590 con 0x7f5ad806ced0 2023-08-27T11:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ad4ff9700 1 -- 172.21.15.45:0/4117710374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc00ee70 con 0x7f5ad806ced0 2023-08-27T11:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad8153720 con 0x7f5ad806ced0 2023-08-27T11:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ad4ff9700 1 -- 172.21.15.45:0/4117710374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00ca50 con 0x7f5ad806ced0 2023-08-27T11:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.937+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad80b3250 con 0x7f5ad806ced0 2023-08-27T11:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.938+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad8153d70 con 0x7f5ad806ced0 2023-08-27T11:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.939+0000 7f5ad4ff9700 1 -- 172.21.15.45:0/4117710374 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f5acc00cbb0 con 0x7f5ad806ced0 2023-08-27T11:28:42.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.939+0000 7f5ad4ff9700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5ac406a170 0x7f5ac406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.940+0000 7f5ad4ff9700 1 -- 172.21.15.45:0/4117710374 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f5acc088e30 con 0x7f5ad806ced0 2023-08-27T11:28:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.940+0000 7f5ad77fe700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5ac406a170 0x7f5ac406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.940+0000 7f5ad77fe700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5ac406a170 0x7f5ac406c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80c69c0 tx=0x7f5ac8008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:42.943+0000 7f5ad4ff9700 1 -- 172.21.15.45:0/4117710374 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5acc057ef0 con 0x7f5ad806ced0 2023-08-27T11:28:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.143+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5ad815cfc0 con 0x7f5ad806ced0 2023-08-27T11:28:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.143+0000 7f5ad4ff9700 1 -- 172.21.15.45:0/4117710374 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v106) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5acc0170a0 con 0x7f5ad806ced0 2023-08-27T11:28:43.145 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5ac406a170 msgr2=0x7f5ac406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5ac406a170 0x7f5ac406c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80c69c0 tx=0x7f5ac8008040 comp rx=0 tx=0).stop 2023-08-27T11:28:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 msgr2=0x7f5ad814f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 0x7f5ad814f460 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00e870 tx=0x7f5acc00ec30 comp rx=0 tx=0).stop 2023-08-27T11:28:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 shutdown_connections 2023-08-27T11:28:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5ac406a170 0x7f5ac406c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ad806ced0 0x7f5ad814f460 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5ad8157be0 0x7f5ad815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 --2- 172.21.15.45:0/4117710374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ad80b17c0 0x7f5ad815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 >> 172.21.15.45:0/4117710374 conn(0x7f5ad80cc7c0 msgr2=0x7f5ad8156580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:43.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 shutdown_connections 2023-08-27T11:28:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.145+0000 7f5ade596700 1 -- 172.21.15.45:0/4117710374 wait complete. 2023-08-27T11:28:43.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-27T11:28:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.289+0000 7f52a6cea700 1 Processor -- start 2023-08-27T11:28:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.289+0000 7f52a6cea700 1 -- start start 2023-08-27T11:28:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f52a6cea700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f52a00cde80 0x7f52a0165ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f52a6cea700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a00cb8b0 con 0x7f52a015b310 2023-08-27T11:28:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f52a6cea700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a00cecd0 con 0x7f52a015af30 2023-08-27T11:28:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f52a6cea700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a006f9c0 con 0x7f52a00cde80 2023-08-27T11:28:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f529ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f52a015b310 0x7f52a00cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46914/0 (socket says 172.21.15.45:46914) 2023-08-27T11:28:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f529ffff700 1 -- 172.21.15.45:0/3380660308 learned_addr learned my addr 172.21.15.45:0/3380660308 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:43.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f52a5287700 1 --2- 172.21.15.45:0/3380660308 >> v2:172.21.15.135:3300/0 conn(0x7f52a00cde80 0x7f52a0165ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f529f7fe700 1 -- 172.21.15.45:0/3380660308 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817894358 0 0) 0x7f52a00cb8b0 con 0x7f52a015b310 2023-08-27T11:28:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f529f7fe700 1 -- 172.21.15.45:0/3380660308 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52900035b0 con 0x7f52a015b310 2023-08-27T11:28:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f52a5287700 1 -- 172.21.15.45:0/3380660308 >> v1:172.21.15.135:6789/0 conn(0x7f52a015af30 legacy=0x7f52a00cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f529f7fe700 1 -- 172.21.15.45:0/3380660308 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452501617 0 0) 0x7f52a00cecd0 con 0x7f52a015af30 2023-08-27T11:28:43.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f52a5287700 1 -- 172.21.15.45:0/3380660308 >> v1:172.21.15.45:6789/0 conn(0x7f52a015b310 legacy=0x7f52a00cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.290+0000 7f52a5287700 1 -- 172.21.15.45:0/3380660308 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a00cbae0 con 0x7f52a00cde80 2023-08-27T11:28:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f52a5287700 1 --2- 172.21.15.45:0/3380660308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a00cde80 0x7f52a0165ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5298009d90 tx=0x7f529800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52adee0ce718d22a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f529f7fe700 1 -- 172.21.15.45:0/3380660308 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529800e070 con 0x7f52a00cde80 2023-08-27T11:28:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f529f7fe700 1 -- 172.21.15.45:0/3380660308 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529800bb20 con 0x7f52a00cde80 2023-08-27T11:28:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f529f7fe700 1 -- 172.21.15.45:0/3380660308 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529800bed0 con 0x7f52a00cde80 2023-08-27T11:28:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f52a6cea700 1 -- 172.21.15.45:0/3380660308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a00cde80 msgr2=0x7f52a0165ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f52a6cea700 1 --2- 172.21.15.45:0/3380660308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a00cde80 0x7f52a0165ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5298009d90 tx=0x7f529800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f52a6cea700 1 -- 172.21.15.45:0/3380660308 shutdown_connections 2023-08-27T11:28:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f52a6cea700 1 --2- 172.21.15.45:0/3380660308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a00cde80 0x7f52a0165ff0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f52a6cea700 1 -- 172.21.15.45:0/3380660308 >> 172.21.15.45:0/3380660308 conn(0x7f52a009a1a0 msgr2=0x7f52a00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f52a6cea700 1 -- 172.21.15.45:0/3380660308 shutdown_connections 2023-08-27T11:28:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.291+0000 7f52a6cea700 1 -- 172.21.15.45:0/3380660308 wait complete. 2023-08-27T11:28:43.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.292+0000 7f52a6cea700 1 Processor -- start 2023-08-27T11:28:43.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.293+0000 7f52a6cea700 1 -- start start 2023-08-27T11:28:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.293+0000 7f52a6cea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00cde80 0x7f52a01490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.293+0000 7f52a6cea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52a015af30 0x7f52a0156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.293+0000 7f52a6cea700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a015b310 0x7f52a00b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.293+0000 7f52a6cea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a006d400 con 0x7f52a00cde80 2023-08-27T11:28:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.293+0000 7f52a6cea700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a0149600 con 0x7f52a015b310 2023-08-27T11:28:43.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.293+0000 7f52a6cea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a006d120 con 0x7f52a015af30 2023-08-27T11:28:43.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a4a86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00cde80 0x7f52a01490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a4a86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00cde80 0x7f52a01490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60142/0 (socket says 172.21.15.45:60142) 2023-08-27T11:28:43.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a4a86700 1 -- 172.21.15.45:0/4276679981 learned_addr learned my addr 172.21.15.45:0/4276679981 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:43.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f529ffff700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52a015af30 0x7f52a0156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a5287700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a015b310 0x7f52a00b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a4a86700 1 -- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52a015af30 msgr2=0x7f52a0156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a4a86700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52a015af30 0x7f52a0156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a4a86700 1 -- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a015b310 msgr2=0x7f52a00b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a4a86700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a015b310 0x7f52a00b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a4a86700 1 -- 172.21.15.45:0/4276679981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5298009a70 con 0x7f52a00cde80 2023-08-27T11:28:43.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a5287700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a015b310 0x7f52a00b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:43.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a4a86700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00cde80 0x7f52a01490c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f52940011e0 tx=0x7f529400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f529d7fa700 1 -- 172.21.15.45:0/4276679981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940054e0 con 0x7f52a00cde80 2023-08-27T11:28:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f529d7fa700 1 -- 172.21.15.45:0/4276679981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5294003680 con 0x7f52a00cde80 2023-08-27T11:28:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a00b5660 con 0x7f52a00cde80 2023-08-27T11:28:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f529d7fa700 1 -- 172.21.15.45:0/4276679981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294003a30 con 0x7f52a00cde80 2023-08-27T11:28:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.294+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a00b5c20 con 0x7f52a00cde80 2023-08-27T11:28:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.295+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a00b54c0 con 0x7f52a00cde80 2023-08-27T11:28:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.296+0000 7f529d7fa700 1 -- 172.21.15.45:0/4276679981 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f5294003b90 con 0x7f52a00cde80 2023-08-27T11:28:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.297+0000 7f529d7fa700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f528406a170 0x7f528406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.297+0000 7f529d7fa700 1 -- 172.21.15.45:0/4276679981 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f5294087d90 con 0x7f52a00cde80 2023-08-27T11:28:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.297+0000 7f529ffff700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f528406a170 0x7f528406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.297+0000 7f529ffff700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f528406a170 0x7f528406c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f52a006f5a0 tx=0x7f528c009040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.301+0000 7f529d7fa700 1 -- 172.21.15.45:0/4276679981 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f52940533c0 con 0x7f52a00cde80 2023-08-27T11:28:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.512+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f52a00b54c0 con 0x7f52a00cde80 2023-08-27T11:28:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.516+0000 7f529d7fa700 1 -- 172.21.15.45:0/4276679981 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f52a00b54c0 con 0x7f52a00cde80 2023-08-27T11:28:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f528406a170 msgr2=0x7f528406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f528406a170 0x7f528406c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f52a006f5a0 tx=0x7f528c009040 comp rx=0 tx=0).stop 2023-08-27T11:28:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00cde80 msgr2=0x7f52a01490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00cde80 0x7f52a01490c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f52940011e0 tx=0x7f529400ca30 comp rx=0 tx=0).stop 2023-08-27T11:28:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 shutdown_connections 2023-08-27T11:28:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f528406a170 0x7f528406c630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00cde80 0x7f52a01490c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52a015b310 0x7f52a00b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 --2- 172.21.15.45:0/4276679981 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52a015af30 0x7f52a0156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 >> 172.21.15.45:0/4276679981 conn(0x7f52a009a1a0 msgr2=0x7f52a015a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:43.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 shutdown_connections 2023-08-27T11:28:43.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.518+0000 7f52a6cea700 1 -- 172.21.15.45:0/4276679981 wait complete. 2023-08-27T11:28:43.531 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:28:43.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:43.571 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:28:43.575 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:43.574+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:43.614 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:28:43.615 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:28:43.615 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:43.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:43.633 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:28:43.634 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:43.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:43.672 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:28:43.673 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:43.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:28:43.691 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:28:43.691 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:28:43.729 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:28:43.730 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-27T11:28:43.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.858+0000 7fb0c4b79700 1 Processor -- start 2023-08-27T11:28:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.858+0000 7fb0c4b79700 1 -- start start 2023-08-27T11:28:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0c4b79700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb0c015d3d0 0x7fb0c01619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0c4b79700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c006e3a0 0x7fb0c006e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0c4b79700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c00cfb30 con 0x7fb0c00af040 2023-08-27T11:28:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0c4b79700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c00cdf90 con 0x7fb0c015d3d0 2023-08-27T11:28:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0c4b79700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c006ed70 con 0x7fb0c006e3a0 2023-08-27T11:28:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0bdd9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb0c00af040 0x7fb0c00cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46928/0 (socket says 172.21.15.45:46928) 2023-08-27T11:28:43.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0bdd9b700 1 -- 172.21.15.45:0/802691120 learned_addr learned my addr 172.21.15.45:0/802691120 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0be59c700 1 --2- 172.21.15.45:0/802691120 >> v2:172.21.15.135:3300/0 conn(0x7fb0c015d3d0 0x7fb0c01619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0bed9d700 1 --2- 172.21.15.45:0/802691120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c006e3a0 0x7fb0c006e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0bd59a700 1 -- 172.21.15.45:0/802691120 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027650277 0 0) 0x7fb0c00cfb30 con 0x7fb0c00af040 2023-08-27T11:28:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0bd59a700 1 -- 172.21.15.45:0/802691120 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a80035b0 con 0x7fb0c00af040 2023-08-27T11:28:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0be59c700 1 -- 172.21.15.45:0/802691120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c006e3a0 msgr2=0x7fb0c006e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0be59c700 1 --2- 172.21.15.45:0/802691120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c006e3a0 0x7fb0c006e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0be59c700 1 -- 172.21.15.45:0/802691120 >> v1:172.21.15.45:6789/0 conn(0x7fb0c00af040 legacy=0x7fb0c00cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0be59c700 1 -- 172.21.15.45:0/802691120 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c0161550 con 0x7fb0c015d3d0 2023-08-27T11:28:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.859+0000 7fb0bed9d700 1 --2- 172.21.15.45:0/802691120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c006e3a0 0x7fb0c006e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0be59c700 1 --2- 172.21.15.45:0/802691120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0c015d3d0 0x7fb0c01619e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009840 tx=0x7fb0b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cff862bbb855e283 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0bd59a700 1 -- 172.21.15.45:0/802691120 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0011830 con 0x7fb0c015d3d0 2023-08-27T11:28:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0bd59a700 1 -- 172.21.15.45:0/802691120 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b0011990 con 0x7fb0c015d3d0 2023-08-27T11:28:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0bd59a700 1 -- 172.21.15.45:0/802691120 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0011d20 con 0x7fb0c015d3d0 2023-08-27T11:28:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0c4b79700 1 -- 172.21.15.45:0/802691120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0c015d3d0 msgr2=0x7fb0c01619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0c4b79700 1 --2- 172.21.15.45:0/802691120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0c015d3d0 0x7fb0c01619e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009840 tx=0x7fb0b0009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:43.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0c4b79700 1 -- 172.21.15.45:0/802691120 shutdown_connections 2023-08-27T11:28:43.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0c4b79700 1 --2- 172.21.15.45:0/802691120 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c006e3a0 0x7fb0c006e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0c4b79700 1 --2- 172.21.15.45:0/802691120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0c015d3d0 0x7fb0c01619e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0c4b79700 1 -- 172.21.15.45:0/802691120 >> 172.21.15.45:0/802691120 conn(0x7fb0c00cfd60 msgr2=0x7fb0c00d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:43.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0c4b79700 1 -- 172.21.15.45:0/802691120 shutdown_connections 2023-08-27T11:28:43.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.860+0000 7fb0c4b79700 1 -- 172.21.15.45:0/802691120 wait complete. 2023-08-27T11:28:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.862+0000 7fb0c4b79700 1 Processor -- start 2023-08-27T11:28:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.862+0000 7fb0c4b79700 1 -- start start 2023-08-27T11:28:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0c4b79700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0c006e3a0 0x7fb0c014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0c4b79700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0c00af040 0x7fb0c00c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0c4b79700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c015d3d0 0x7fb0c00c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0c4b79700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00b2a20 con 0x7fb0c00af040 2023-08-27T11:28:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0c4b79700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00c6690 con 0x7fb0c006e3a0 2023-08-27T11:28:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0c4b79700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00b2cc0 con 0x7fb0c015d3d0 2023-08-27T11:28:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bdd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0c00af040 0x7fb0c00c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bdd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0c00af040 0x7fb0c00c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60158/0 (socket says 172.21.15.45:60158) 2023-08-27T11:28:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bdd9b700 1 -- 172.21.15.45:0/1133491040 learned_addr learned my addr 172.21.15.45:0/1133491040 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0be59c700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0c006e3a0 0x7fb0c014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bed9d700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c015d3d0 0x7fb0c00c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bdd9b700 1 -- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c015d3d0 msgr2=0x7fb0c00c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bdd9b700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c015d3d0 0x7fb0c00c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bdd9b700 1 -- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0c006e3a0 msgr2=0x7fb0c014e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bdd9b700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0c006e3a0 0x7fb0c014e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bdd9b700 1 -- 172.21.15.45:0/1133491040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8009a70 con 0x7fb0c00af040 2023-08-27T11:28:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0bdd9b700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0c00af040 0x7fb0c00c9000 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c700 tx=0x7fb0b400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0aeffd700 1 -- 172.21.15.45:0/1133491040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4005d70 con 0x7fb0c00af040 2023-08-27T11:28:43.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.863+0000 7fb0aeffd700 1 -- 172.21.15.45:0/1133491040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b40070d0 con 0x7fb0c00af040 2023-08-27T11:28:43.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.864+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b00094a0 con 0x7fb0c00af040 2023-08-27T11:28:43.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.864+0000 7fb0aeffd700 1 -- 172.21.15.45:0/1133491040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4007460 con 0x7fb0c00af040 2023-08-27T11:28:43.874 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.864+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c00c6970 con 0x7fb0c00af040 2023-08-27T11:28:43.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.864+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c00b31e0 con 0x7fb0c00af040 2023-08-27T11:28:43.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.865+0000 7fb0aeffd700 1 -- 172.21.15.45:0/1133491040 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fb0b40075c0 con 0x7fb0c00af040 2023-08-27T11:28:43.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.865+0000 7fb0aeffd700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb0a006a170 0x7fb0a006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:43.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.865+0000 7fb0aeffd700 1 -- 172.21.15.45:0/1133491040 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fb0b4087c30 con 0x7fb0c00af040 2023-08-27T11:28:43.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.865+0000 7fb0be59c700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb0a006a170 0x7fb0a006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:43.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.866+0000 7fb0be59c700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb0a006a170 0x7fb0a006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0000c00 tx=0x7fb0b001c040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:43.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:43.870+0000 7fb0aeffd700 1 -- 172.21.15.45:0/1133491040 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb0c00b31e0 con 0x7fb0c00af040 2023-08-27T11:28:44.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.127+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0c00b31e0 con 0x7fb0c00af040 2023-08-27T11:28:44.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.127+0000 7fb0aeffd700 1 -- 172.21.15.45:0/1133491040 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fb0c00b31e0 con 0x7fb0c00af040 2023-08-27T11:28:44.129 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:44.129 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3604468746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3604468746},{"type":"v1","addr":"172.21.15.135:6836","nonce":3604468746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1319491874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":1319491874},{"type":"v1","addr":"172.21.15.135:6837","nonce":1319491874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6937,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2036943131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2036943131},{"type":"v1","addr":"172.21.15.45:6835","nonce":2036943131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7908,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/864412092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":864412092},{"type":"v1","addr":"172.21.15.45:6833","nonce":864412092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[]} 2023-08-27T11:28:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb0a006a170 msgr2=0x7fb0a006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb0a006a170 0x7fb0a006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0000c00 tx=0x7fb0b001c040 comp rx=0 tx=0).stop 2023-08-27T11:28:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0c00af040 msgr2=0x7fb0c00c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.131 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0c00af040 0x7fb0c00c9000 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c700 tx=0x7fb0b400ca10 comp rx=0 tx=0).stop 2023-08-27T11:28:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 shutdown_connections 2023-08-27T11:28:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb0a006a170 0x7fb0a006c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb0c015d3d0 0x7fb0c00c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0c006e3a0 0x7fb0c014e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 --2- 172.21.15.45:0/1133491040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0c00af040 0x7fb0c00c9000 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 >> 172.21.15.45:0/1133491040 conn(0x7fb0c00cfd60 msgr2=0x7fb0c006a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 shutdown_connections 2023-08-27T11:28:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.130+0000 7fb0c4b79700 1 -- 172.21.15.45:0/1133491040 wait complete. 2023-08-27T11:28:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 43 2023-08-27T11:28:44.143 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-27T11:28:44.144 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-27T11:28:44.144 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-27T11:28:44.144 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:44.145 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-27T11:28:44.145 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-27T11:28:44.145 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-27T11:28:44.145 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:44.146 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-27T11:28:44.146 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-27T11:28:44.146 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-27T11:28:44.147 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:44.147 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-27T11:28:44.147 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-27T11:28:44.147 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-27T11:28:44.148 INFO:teuthology.orchestra.run:waiting for 300 2023-08-27T11:28:44.152 INFO:tasks.ceph.mds.a:Stopped 2023-08-27T11:28:44.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-27T11:28:44.154 INFO:tasks.ceph.mds.c:Stopped 2023-08-27T11:28:44.155 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-27T11:28:44.156 INFO:tasks.ceph.mds.b:Stopped 2023-08-27T11:28:44.156 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-27T11:28:44.157 INFO:tasks.ceph.mds.d:Stopped 2023-08-27T11:28:44.157 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-27T11:28:44.196 INFO:tasks.ceph.mds.a:Started 2023-08-27T11:28:44.197 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:44.198 INFO:tasks.ceph.mds.c:Started 2023-08-27T11:28:44.198 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:44.199 INFO:tasks.ceph.mds.b:Started 2023-08-27T11:28:44.200 INFO:tasks.ceph.mds.d:Started 2023-08-27T11:28:44.200 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:44.200 DEBUG:teuthology.parallel:result is None 2023-08-27T11:28:44.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-27T11:28:44.278 INFO:tasks.ceph.mds.b.smithi135.stdout:starting mds.b at 2023-08-27T11:28:44.279 INFO:tasks.ceph.mds.d.smithi135.stdout:starting mds.d at 2023-08-27T11:28:44.281 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2023-08-27T11:28:44.282 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-08-27T11:28:44.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.335+0000 7f5d00d38700 1 Processor -- start 2023-08-27T11:28:44.336 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.335+0000 7f5d00d38700 1 -- start start 2023-08-27T11:28:44.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5d00d38700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5cfc15a0f0 0x7f5cfc0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5d00d38700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc06ed90 0x7f5cfc15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.338 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5d00d38700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cfc0cfae0 con 0x7f5cfc0cd300 2023-08-27T11:28:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5d00d38700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc15e900 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5d00d38700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0cae60 con 0x7f5cfc06ed90 2023-08-27T11:28:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cfa59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5cfc15a0f0 0x7f5cfc0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cfa59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5cfc15a0f0 0x7f5cfc0cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60188/0 (socket says 172.21.15.45:60188) 2023-08-27T11:28:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cf9d9b700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f5cfc0cd300 0x7f5cfc15daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:42628/0 (socket says 172.21.15.45:42628) 2023-08-27T11:28:44.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cfa59c700 1 -- 172.21.15.45:0/457237289 learned_addr learned my addr 172.21.15.45:0/457237289 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:44.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cfad9d700 1 --2- 172.21.15.45:0/457237289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc06ed90 0x7f5cfc15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cfa59c700 1 -- 172.21.15.45:0/457237289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc06ed90 msgr2=0x7f5cfc15e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:44.340 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cfa59c700 1 --2- 172.21.15.45:0/457237289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc06ed90 0x7f5cfc15e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cfa59c700 1 -- 172.21.15.45:0/457237289 >> v1:172.21.15.135:6789/0 conn(0x7f5cfc0cd300 legacy=0x7f5cfc15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cfa59c700 1 -- 172.21.15.45:0/457237289 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0cb140 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cf959a700 1 -- 172.21.15.45:0/457237289 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329807158 0 0) 0x7f5cfc0cfae0 con 0x7f5cfc0cd300 2023-08-27T11:28:44.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.336+0000 7f5cfa59c700 1 --2- 172.21.15.45:0/457237289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 0x7f5cfc0cbc20 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098f0 tx=0x7f5cec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0abb0476d0d213 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:44.341 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5cf959a700 1 -- 172.21.15.45:0/457237289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00f8a0 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5cf959a700 1 -- 172.21.15.45:0/457237289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec00fa00 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5cf959a700 1 -- 172.21.15.45:0/457237289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00fd90 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5d00d38700 1 -- 172.21.15.45:0/457237289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 msgr2=0x7f5cfc0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.342 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5d00d38700 1 --2- 172.21.15.45:0/457237289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 0x7f5cfc0cbc20 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098f0 tx=0x7f5cec009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:44.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5d00d38700 1 -- 172.21.15.45:0/457237289 shutdown_connections 2023-08-27T11:28:44.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5d00d38700 1 --2- 172.21.15.45:0/457237289 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc06ed90 0x7f5cfc15e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5d00d38700 1 --2- 172.21.15.45:0/457237289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 0x7f5cfc0cbc20 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5d00d38700 1 -- 172.21.15.45:0/457237289 >> 172.21.15.45:0/457237289 conn(0x7f5cfc0cc7e0 msgr2=0x7f5cfc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:44.343 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5d00d38700 1 -- 172.21.15.45:0/457237289 shutdown_connections 2023-08-27T11:28:44.344 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.337+0000 7f5d00d38700 1 -- 172.21.15.45:0/457237289 wait complete. 2023-08-27T11:28:44.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.339+0000 7f5d00d38700 1 Processor -- start 2023-08-27T11:28:44.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.339+0000 7f5d00d38700 1 -- start start 2023-08-27T11:28:44.346 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.339+0000 7f5d00d38700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cfc06ed90 0x7f5cfc0c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.339+0000 7f5d00d38700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc0cd300 0x7f5cfc1f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.339+0000 7f5d00d38700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 0x7f5cfc1ee000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5d00d38700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc1f2d10 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5d00d38700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0c3ec0 con 0x7f5cfc06ed90 2023-08-27T11:28:44.347 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5d00d38700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0c41a0 con 0x7f5cfc0cd300 2023-08-27T11:28:44.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 0x7f5cfc1ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 0x7f5cfc1ee000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60198/0 (socket says 172.21.15.45:60198) 2023-08-27T11:28:44.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfad9d700 1 -- 172.21.15.45:0/1113202494 learned_addr learned my addr 172.21.15.45:0/1113202494 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:44.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfa59c700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cfc06ed90 0x7f5cfc0c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cf9d9b700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc0cd300 0x7f5cfc1f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.348 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfad9d700 1 -- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc0cd300 msgr2=0x7f5cfc1f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfad9d700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc0cd300 0x7f5cfc1f27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfad9d700 1 -- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cfc06ed90 msgr2=0x7f5cfc0c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:44.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfad9d700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cfc06ed90 0x7f5cfc0c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:44.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfad9d700 1 -- 172.21.15.45:0/1113202494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec009550 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.349 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5cfad9d700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 0x7f5cfc1ee000 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400efc0 tx=0x7f5cf400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:44.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5ceaffd700 1 -- 172.21.15.45:0/1113202494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400cd40 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5ceaffd700 1 -- 172.21.15.45:0/1113202494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf400f040 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5ceaffd700 1 -- 172.21.15.45:0/1113202494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40146c0 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc151920 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.340+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc0c4480 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.341+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc151e40 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.342+0000 7f5ceaffd700 1 -- 172.21.15.45:0/1113202494 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f5cf4014820 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.342+0000 7f5ceaffd700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5cdc06a170 0x7f5cdc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:44.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.342+0000 7f5ceaffd700 1 -- 172.21.15.45:0/1113202494 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f5cf40890d0 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.342+0000 7f5cfa59c700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5cdc06a170 0x7f5cdc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:44.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.343+0000 7f5cfa59c700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5cdc06a170 0x7f5cdc06c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000c00 tx=0x7f5cec008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:44.352 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.347+0000 7f5ceaffd700 1 -- 172.21.15.45:0/1113202494 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5cf4058090 con 0x7f5cfc15a0f0 2023-08-27T11:28:44.411 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:44.409+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:44.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:44.572+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5cfc0726d0 con 0x7f5cfc15a0f0 2023-08-27T11:28:45.204 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:45.203+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:45.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.528+0000 7f5ceaffd700 1 -- 172.21.15.45:0/1113202494 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5cf4018080 con 0x7f5cfc15a0f0 2023-08-27T11:28:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5cdc06a170 msgr2=0x7f5cdc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5cdc06a170 0x7f5cdc06c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000c00 tx=0x7f5cec008030 comp rx=0 tx=0).stop 2023-08-27T11:28:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 msgr2=0x7f5cfc1ee000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 0x7f5cfc1ee000 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400efc0 tx=0x7f5cf400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 shutdown_connections 2023-08-27T11:28:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5cdc06a170 0x7f5cdc06c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5cfc15a0f0 0x7f5cfc1ee000 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5cfc06ed90 0x7f5cfc0c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 --2- 172.21.15.45:0/1113202494 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5cfc0cd300 0x7f5cfc1f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 >> 172.21.15.45:0/1113202494 conn(0x7f5cfc0cc7e0 msgr2=0x7f5cfc0ca390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 shutdown_connections 2023-08-27T11:28:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.530+0000 7f5d00d38700 1 -- 172.21.15.45:0/1113202494 wait complete. 2023-08-27T11:28:45.535 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2023-08-27T11:28:45.544 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-27T11:28:45.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.670+0000 7f076ca01700 1 Processor -- start 2023-08-27T11:28:45.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.670+0000 7f076ca01700 1 -- start start 2023-08-27T11:28:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.670+0000 7f076ca01700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f07680b4880 0x7f0768070400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.670+0000 7f076ca01700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815f4c0 0x7f076815e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.670+0000 7f076ca01700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f076815ee40 0x7f07680c9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076ca01700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076815f2d0 con 0x7f076815ee40 2023-08-27T11:28:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076ca01700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07680b4c60 con 0x7f07680b4880 2023-08-27T11:28:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076ca01700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07680cb720 con 0x7f076815f4c0 2023-08-27T11:28:45.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076759e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f076815ee40 0x7f07680c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076759e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f076815ee40 0x7f07680c9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60210/0 (socket says 172.21.15.45:60210) 2023-08-27T11:28:45.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076759e700 1 -- 172.21.15.45:0/2130566047 learned_addr learned my addr 172.21.15.45:0/2130566047 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:45.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076659c700 1 --2- 172.21.15.45:0/2130566047 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815f4c0 0x7f076815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f0766d9d700 1 --2- 172.21.15.45:0/2130566047 >> v2:172.21.15.135:3300/0 conn(0x7f07680b4880 0x7f0768070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076759e700 1 -- 172.21.15.45:0/2130566047 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815f4c0 msgr2=0x7f076815e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:45.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076759e700 1 --2- 172.21.15.45:0/2130566047 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815f4c0 0x7f076815e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076759e700 1 -- 172.21.15.45:0/2130566047 >> v2:172.21.15.135:3300/0 conn(0x7f07680b4880 msgr2=0x7f0768070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:45.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076759e700 1 --2- 172.21.15.45:0/2130566047 >> v2:172.21.15.135:3300/0 conn(0x7f07680b4880 0x7f0768070400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076759e700 1 -- 172.21.15.45:0/2130566047 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07680cba00 con 0x7f076815ee40 2023-08-27T11:28:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f0766d9d700 1 --2- 172.21.15.45:0/2130566047 >> v2:172.21.15.135:3300/0 conn(0x7f07680b4880 0x7f0768070400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f076759e700 1 --2- 172.21.15.45:0/2130566047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815ee40 0x7f07680c9940 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0768066d80 tx=0x7f076000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d63873f6d0ffde server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f0765d9b700 1 -- 172.21.15.45:0/2130566047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000e070 con 0x7f076815ee40 2023-08-27T11:28:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f0765d9b700 1 -- 172.21.15.45:0/2130566047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076000b910 con 0x7f076815ee40 2023-08-27T11:28:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.671+0000 7f0765d9b700 1 -- 172.21.15.45:0/2130566047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000bcc0 con 0x7f076815ee40 2023-08-27T11:28:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.672+0000 7f076ca01700 1 -- 172.21.15.45:0/2130566047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815ee40 msgr2=0x7f07680c9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.672+0000 7f076ca01700 1 --2- 172.21.15.45:0/2130566047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815ee40 0x7f07680c9940 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0768066d80 tx=0x7f076000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.672+0000 7f076ca01700 1 -- 172.21.15.45:0/2130566047 shutdown_connections 2023-08-27T11:28:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.672+0000 7f076ca01700 1 --2- 172.21.15.45:0/2130566047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815ee40 0x7f07680c9940 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.672+0000 7f076ca01700 1 --2- 172.21.15.45:0/2130566047 >> v2:172.21.15.135:3300/0 conn(0x7f07680b4880 0x7f0768070400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.672+0000 7f076ca01700 1 --2- 172.21.15.45:0/2130566047 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815f4c0 0x7f076815e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.672+0000 7f076ca01700 1 -- 172.21.15.45:0/2130566047 >> 172.21.15.45:0/2130566047 conn(0x7f07680cbd70 msgr2=0x7f07680cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.672+0000 7f076ca01700 1 -- 172.21.15.45:0/2130566047 shutdown_connections 2023-08-27T11:28:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.672+0000 7f076ca01700 1 -- 172.21.15.45:0/2130566047 wait complete. 2023-08-27T11:28:45.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.674+0000 7f076ca01700 1 Processor -- start 2023-08-27T11:28:45.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.674+0000 7f076ca01700 1 -- start start 2023-08-27T11:28:45.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076ca01700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07680b4880 0x7f076815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076ca01700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815ee40 0x7f07681f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076ca01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815f4c0 0x7f07680c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076ca01700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076815c2e0 con 0x7f076815f4c0 2023-08-27T11:28:45.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076ca01700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076815b6d0 con 0x7f07680b4880 2023-08-27T11:28:45.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076ca01700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076815b9b0 con 0x7f076815ee40 2023-08-27T11:28:45.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815f4c0 0x7f07680c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815f4c0 0x7f07680c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60216/0 (socket says 172.21.15.45:60216) 2023-08-27T11:28:45.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076759e700 1 -- 172.21.15.45:0/2436142699 learned_addr learned my addr 172.21.15.45:0/2436142699 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:45.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076659c700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815ee40 0x7f07681f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f0766d9d700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07680b4880 0x7f076815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076759e700 1 -- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815ee40 msgr2=0x7f07681f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076759e700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815ee40 0x7f07681f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076759e700 1 -- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07680b4880 msgr2=0x7f076815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076759e700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07680b4880 0x7f076815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.675+0000 7f076759e700 1 -- 172.21.15.45:0/2436142699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0760009b30 con 0x7f076815f4c0 2023-08-27T11:28:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.676+0000 7f076759e700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815f4c0 0x7f07680c65e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f07680cf170 tx=0x7f0760004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.676+0000 7f074f7fe700 1 -- 172.21.15.45:0/2436142699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07600099b0 con 0x7f076815f4c0 2023-08-27T11:28:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.676+0000 7f074f7fe700 1 -- 172.21.15.45:0/2436142699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07600092e0 con 0x7f076815f4c0 2023-08-27T11:28:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.676+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076815bc90 con 0x7f076815f4c0 2023-08-27T11:28:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.676+0000 7f074f7fe700 1 -- 172.21.15.45:0/2436142699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760004470 con 0x7f076815f4c0 2023-08-27T11:28:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.676+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07680c6b20 con 0x7f076815f4c0 2023-08-27T11:28:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.677+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076815c450 con 0x7f076815f4c0 2023-08-27T11:28:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.677+0000 7f074f7fe700 1 -- 172.21.15.45:0/2436142699 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f07600045d0 con 0x7f076815f4c0 2023-08-27T11:28:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.678+0000 7f074f7fe700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f075406a170 0x7f075406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.678+0000 7f074f7fe700 1 -- 172.21.15.45:0/2436142699 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f0760089870 con 0x7f076815f4c0 2023-08-27T11:28:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.678+0000 7f0766d9d700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f075406a170 0x7f075406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.678+0000 7f0766d9d700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f075406a170 0x7f075406c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0758006f10 tx=0x7f0758008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:45.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.682+0000 7f074f7fe700 1 -- 172.21.15.45:0/2436142699 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0760058ae0 con 0x7f076815f4c0 2023-08-27T11:28:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.965+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f076806d760 con 0x7f076815f4c0 2023-08-27T11:28:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.965+0000 7f074f7fe700 1 -- 172.21.15.45:0/2436142699 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0760058670 con 0x7f076815f4c0 2023-08-27T11:28:45.966 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:45.967 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:28:45.967 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2023-08-27T11:28:45.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.967 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2023-08-27T11:28:45.967 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tknzGlBhAABVGsUvHGyOdTgH9miKO0Cg==", 2023-08-27T11:28:45.968 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.968 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:45.968 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:45.968 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:45.969 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.969 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2023-08-27T11:28:45.970 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkEWbPExAAgt/9BpPD1vK9QoG8l3HeFw==", 2023-08-27T11:28:45.970 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.970 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:45.970 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:45.970 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:45.971 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.971 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2023-08-27T11:28:45.972 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk6qRCDhAAGaj+NY32IDjdRcuSNbASpw==", 2023-08-27T11:28:45.972 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.972 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:45.972 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:45.972 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:45.973 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.974 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2023-08-27T11:28:45.974 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkE9NxGxAAnWum05rSbYQSLZ9IjjqDpQ==", 2023-08-27T11:28:45.974 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.974 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:45.974 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:45.975 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:45.975 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.975 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.976 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2023-08-27T11:28:45.976 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBML+tkOzvBBBAA196QdZiLzcsm5hLBGDl0Qw==", 2023-08-27T11:28:45.976 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.976 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.976 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.977 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.977 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2023-08-27T11:28:45.978 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBPL+tkIXaxBBAAfHwZ2SsmOENhnNxZsG230Q==", 2023-08-27T11:28:45.978 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.978 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.978 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.978 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.979 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2023-08-27T11:28:45.979 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBSL+tkpzphBBAA37UzwlmuLdSWDIrkBu/8Rw==", 2023-08-27T11:28:45.980 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.980 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.980 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.981 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVL+tkt63hAxAAJReGReJHqyggSbdFEhm3+A==", 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.982 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBbL+tkTOFXBxAAZ1I44e50IrNzgj7mjKyuGA==", 2023-08-27T11:28:45.983 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.984 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.984 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.984 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.985 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2023-08-27T11:28:45.985 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBeL+tkdezFBhAANof2C80WiBZ6hWm3oB3kfg==", 2023-08-27T11:28:45.985 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.986 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.986 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.986 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.986 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBhL+tk7auXBhAAJPwKwTVIUzao5D3t4iXYcw==", 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.987 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2023-08-27T11:28:45.988 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBkL+tkY+ztBRAAvM2oxELLN8WV0i0spqfY6Q==", 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ==", 2023-08-27T11:28:45.990 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.992 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2023-08-27T11:28:45.992 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkMCDoJhAAevee0PlTshlWJcPzX8fpYg==", 2023-08-27T11:28:45.992 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.992 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:45.992 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:28:45.993 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:28:45.993 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:28:45.993 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.994 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2023-08-27T11:28:45.994 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tk1xh+IxAAswoUpd+D70hUwfeKI2gc0w==", 2023-08-27T11:28:45.994 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.994 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:45.994 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:28:45.994 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:28:45.995 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:28:45.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.995 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2023-08-27T11:28:45.996 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkGWuaLBAAYM2t+TMEs59u635jL9f9kg==", 2023-08-27T11:28:45.996 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.996 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2023-08-27T11:28:45.996 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2023-08-27T11:28:45.996 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2023-08-27T11:28:45.996 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2023-08-27T11:28:45.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.997 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2023-08-27T11:28:45.997 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkkw7pGRAAVim28nKfumQg/LcBfJpivA==", 2023-08-27T11:28:45.998 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:45.998 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:28:45.998 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2023-08-27T11:28:45.998 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2023-08-27T11:28:45.998 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:45.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:45.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:45.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:45.999 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2023-08-27T11:28:45.999 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBKL+tkNxfXAhAA/K5r8Mx/C7nRitNuzXXvKg==", 2023-08-27T11:28:46.000 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:46.000 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:28:46.000 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:28:46.000 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:46.000 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:46.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:46.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:46.001 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2023-08-27T11:28:46.001 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBJL+tkW6B8NRAAF2hsNVRi3Ji4M8anuYwpDg==", 2023-08-27T11:28:46.001 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2023-08-27T11:28:46.001 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2023-08-27T11:28:46.002 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2023-08-27T11:28:46.002 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2023-08-27T11:28:46.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:46.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:46.002 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-08-27T11:28:46.003 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:28:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.967+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f075406a170 msgr2=0x7f075406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.967+0000 7f076ca01700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f075406a170 0x7f075406c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0758006f10 tx=0x7f0758008040 comp rx=0 tx=0).stop 2023-08-27T11:28:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.967+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815f4c0 msgr2=0x7f07680c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.967+0000 7f076ca01700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815f4c0 0x7f07680c65e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f07680cf170 tx=0x7f0760004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.967+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 shutdown_connections 2023-08-27T11:28:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.968+0000 7f076ca01700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f075406a170 0x7f075406c630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.968+0000 7f076ca01700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f076815f4c0 0x7f07680c65e0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.968+0000 7f076ca01700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07680b4880 0x7f076815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.968+0000 7f076ca01700 1 --2- 172.21.15.45:0/2436142699 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f076815ee40 0x7f07681f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.968+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 >> 172.21.15.45:0/2436142699 conn(0x7f07680cbd70 msgr2=0x7f07680b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.968+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 shutdown_connections 2023-08-27T11:28:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:45.968+0000 7f076ca01700 1 -- 172.21.15.45:0/2436142699 wait complete. 2023-08-27T11:28:46.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:46.008 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:46.006+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:46.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.134+0000 7fd267350700 1 Processor -- start 2023-08-27T11:28:46.135 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.134+0000 7fd267350700 1 -- start start 2023-08-27T11:28:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.135+0000 7fd267350700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd26006cf40 0x7fd26006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.135+0000 7fd267350700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd260071e80 0x7fd2600caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.135+0000 7fd267350700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26006f8b0 con 0x7fd26015b270 2023-08-27T11:28:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.135+0000 7fd267350700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2600ca650 con 0x7fd26006cf40 2023-08-27T11:28:46.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.135+0000 7fd267350700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26006f010 con 0x7fd260071e80 2023-08-27T11:28:46.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.135+0000 7fd2650ec700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd26006cf40 0x7fd26006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.135+0000 7fd2650ec700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd26006cf40 0x7fd26006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60220/0 (socket says 172.21.15.45:60220) 2023-08-27T11:28:46.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.135+0000 7fd2650ec700 1 -- 172.21.15.45:0/3422923581 learned_addr learned my addr 172.21.15.45:0/3422923581 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:46.139 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.136+0000 7fd2658ed700 1 --2- 172.21.15.45:0/3422923581 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd260071e80 0x7fd2600caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.136+0000 7fd2650ec700 1 -- 172.21.15.45:0/3422923581 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd260071e80 msgr2=0x7fd2600caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:46.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.136+0000 7fd2650ec700 1 --2- 172.21.15.45:0/3422923581 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd260071e80 0x7fd2600caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.136+0000 7fd2650ec700 1 -- 172.21.15.45:0/3422923581 >> v1:172.21.15.135:6789/0 conn(0x7fd26015b270 legacy=0x7fd26006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:46.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.136+0000 7fd2650ec700 1 -- 172.21.15.45:0/3422923581 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26006f2f0 con 0x7fd26006cf40 2023-08-27T11:28:46.140 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.136+0000 7fd2650ec700 1 --2- 172.21.15.45:0/3422923581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26006cf40 0x7fd26006ac90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0098f0 tx=0x7fd25c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d79adb709119e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.136+0000 7fd253fff700 1 -- 172.21.15.45:0/3422923581 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c004230 con 0x7fd26006cf40 2023-08-27T11:28:46.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.136+0000 7fd253fff700 1 -- 172.21.15.45:0/3422923581 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c004390 con 0x7fd26006cf40 2023-08-27T11:28:46.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.136+0000 7fd253fff700 1 -- 172.21.15.45:0/3422923581 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0036a0 con 0x7fd26006cf40 2023-08-27T11:28:46.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.137+0000 7fd267350700 1 -- 172.21.15.45:0/3422923581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26006cf40 msgr2=0x7fd26006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.137+0000 7fd267350700 1 --2- 172.21.15.45:0/3422923581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26006cf40 0x7fd26006ac90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0098f0 tx=0x7fd25c009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:46.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.137+0000 7fd267350700 1 -- 172.21.15.45:0/3422923581 shutdown_connections 2023-08-27T11:28:46.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.137+0000 7fd267350700 1 --2- 172.21.15.45:0/3422923581 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd260071e80 0x7fd2600caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.137+0000 7fd267350700 1 --2- 172.21.15.45:0/3422923581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26006cf40 0x7fd26006ac90 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.137+0000 7fd267350700 1 -- 172.21.15.45:0/3422923581 >> 172.21.15.45:0/3422923581 conn(0x7fd2600cc7e0 msgr2=0x7fd2600ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.137+0000 7fd267350700 1 -- 172.21.15.45:0/3422923581 shutdown_connections 2023-08-27T11:28:46.142 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.137+0000 7fd267350700 1 -- 172.21.15.45:0/3422923581 wait complete. 2023-08-27T11:28:46.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.138+0000 7fd267350700 1 Processor -- start 2023-08-27T11:28:46.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.138+0000 7fd267350700 1 -- start start 2023-08-27T11:28:46.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd267350700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd26006cf40 0x7fd2600c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd267350700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd260071e80 0x7fd26014ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd267350700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26015b270 0x7fd2601f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd267350700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600c5d70 con 0x7fd26015b270 2023-08-27T11:28:46.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd267350700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600b2b60 con 0x7fd260071e80 2023-08-27T11:28:46.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd267350700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26014f2c0 con 0x7fd26006cf40 2023-08-27T11:28:46.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2658ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26015b270 0x7fd2601f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2658ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26015b270 0x7fd2601f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60230/0 (socket says 172.21.15.45:60230) 2023-08-27T11:28:46.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2658ed700 1 -- 172.21.15.45:0/4158391808 learned_addr learned my addr 172.21.15.45:0/4158391808 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:46.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2648eb700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd260071e80 0x7fd26014ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2650ec700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd26006cf40 0x7fd2600c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2658ed700 1 -- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd26006cf40 msgr2=0x7fd2600c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2658ed700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd26006cf40 0x7fd2600c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2658ed700 1 -- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd260071e80 msgr2=0x7fd26014ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2658ed700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd260071e80 0x7fd26014ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2658ed700 1 -- 172.21.15.45:0/4158391808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c009550 con 0x7fd26015b270 2023-08-27T11:28:46.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd2658ed700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26015b270 0x7fd2601f3c00 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd258007e40 tx=0x7fd25800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.139+0000 7fd251ffb700 1 -- 172.21.15.45:0/4158391808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25800ed30 con 0x7fd26015b270 2023-08-27T11:28:46.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.140+0000 7fd251ffb700 1 -- 172.21.15.45:0/4158391808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd258014420 con 0x7fd26015b270 2023-08-27T11:28:46.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.140+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2600b2e00 con 0x7fd26015b270 2023-08-27T11:28:46.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.140+0000 7fd251ffb700 1 -- 172.21.15.45:0/4158391808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2580147d0 con 0x7fd26015b270 2023-08-27T11:28:46.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.140+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2600b33c0 con 0x7fd26015b270 2023-08-27T11:28:46.152 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.141+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26006dca0 con 0x7fd26015b270 2023-08-27T11:28:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.141+0000 7fd251ffb700 1 -- 172.21.15.45:0/4158391808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fd258014930 con 0x7fd26015b270 2023-08-27T11:28:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.141+0000 7fd251ffb700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd24406a170 0x7fd24406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.142+0000 7fd251ffb700 1 -- 172.21.15.45:0/4158391808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fd258088af0 con 0x7fd26015b270 2023-08-27T11:28:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.142+0000 7fd2650ec700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd24406a170 0x7fd24406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.142+0000 7fd2650ec700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd24406a170 0x7fd24406c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009130 tx=0x7fd25c008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.146+0000 7fd251ffb700 1 -- 172.21.15.45:0/4158391808 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd2580542e0 con 0x7fd26015b270 2023-08-27T11:28:46.350 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.349+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2600726d0 con 0x7fd26015b270 2023-08-27T11:28:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.350+0000 7fd251ffb700 1 -- 172.21.15.45:0/4158391808 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+14928 (secure 0 0 0) 0x7fd258057900 con 0x7fd26015b270 2023-08-27T11:28:46.351 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:46.351 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":111,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:45.525418+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:46.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd24406a170 msgr2=0x7fd24406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd24406a170 0x7fd24406c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009130 tx=0x7fd25c008030 comp rx=0 tx=0).stop 2023-08-27T11:28:46.353 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26015b270 msgr2=0x7fd2601f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26015b270 0x7fd2601f3c00 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd258007e40 tx=0x7fd25800e3c0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 shutdown_connections 2023-08-27T11:28:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd24406a170 0x7fd24406c630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd26015b270 0x7fd2601f3c00 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd26006cf40 0x7fd2600c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 --2- 172.21.15.45:0/4158391808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd260071e80 0x7fd26014ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 >> 172.21.15.45:0/4158391808 conn(0x7fd2600cc7e0 msgr2=0x7fd2600c9470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 shutdown_connections 2023-08-27T11:28:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.352+0000 7fd267350700 1 -- 172.21.15.45:0/4158391808 wait complete. 2023-08-27T11:28:46.366 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:28:46.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:28:46.496 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.495+0000 7fd6bc983700 1 Processor -- start 2023-08-27T11:28:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.495+0000 7fd6bc983700 1 -- start start 2023-08-27T11:28:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6bc983700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6b80c5e50 0x7fd6b8071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6bc983700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b815ea60 con 0x7fd6b8071e10 2023-08-27T11:28:46.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6bc983700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b815dec0 con 0x7fd6b815a000 2023-08-27T11:28:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6bc983700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b815b3f0 con 0x7fd6b80c5e50 2023-08-27T11:28:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b5d9b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd6b8071e10 0x7fd6b80c5740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47012/0 (socket says 172.21.15.45:47012) 2023-08-27T11:28:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b5d9b700 1 -- 172.21.15.45:0/2126943979 learned_addr learned my addr 172.21.15.45:0/2126943979 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b6d9d700 1 --2- 172.21.15.45:0/2126943979 >> v2:172.21.15.45:3300/0 conn(0x7fd6b80c5e50 0x7fd6b8071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.500 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b559a700 1 -- 172.21.15.45:0/2126943979 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960395043 0 0) 0x7fd6a8001110 con 0x7fd6b8071e10 2023-08-27T11:28:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b559a700 1 -- 172.21.15.45:0/2126943979 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a40035b0 con 0x7fd6b8071e10 2023-08-27T11:28:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b559a700 1 -- 172.21.15.45:0/2126943979 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909867283 0 0) 0x7fd6b815dec0 con 0x7fd6b815a000 2023-08-27T11:28:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b559a700 1 -- 172.21.15.45:0/2126943979 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a8001110 con 0x7fd6b815a000 2023-08-27T11:28:46.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b6d9d700 1 -- 172.21.15.45:0/2126943979 >> v1:172.21.15.135:6789/0 conn(0x7fd6b815a000 legacy=0x7fd6b815e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b6d9d700 1 -- 172.21.15.45:0/2126943979 >> v1:172.21.15.45:6789/0 conn(0x7fd6b8071e10 legacy=0x7fd6b80c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b6d9d700 1 -- 172.21.15.45:0/2126943979 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b80c64a0 con 0x7fd6b80c5e50 2023-08-27T11:28:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.497+0000 7fd6b6d9d700 1 --2- 172.21.15.45:0/2126943979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 0x7fd6b8071820 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009d90 tx=0x7fd6b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee689a4bf13544a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6b559a700 1 -- 172.21.15.45:0/2126943979 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000e070 con 0x7fd6b80c5e50 2023-08-27T11:28:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6b559a700 1 -- 172.21.15.45:0/2126943979 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b000bb20 con 0x7fd6b80c5e50 2023-08-27T11:28:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6b559a700 1 -- 172.21.15.45:0/2126943979 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0012430 con 0x7fd6b80c5e50 2023-08-27T11:28:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6bc983700 1 -- 172.21.15.45:0/2126943979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 msgr2=0x7fd6b8071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6bc983700 1 --2- 172.21.15.45:0/2126943979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 0x7fd6b8071820 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009d90 tx=0x7fd6b000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6bc983700 1 -- 172.21.15.45:0/2126943979 shutdown_connections 2023-08-27T11:28:46.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6bc983700 1 --2- 172.21.15.45:0/2126943979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 0x7fd6b8071820 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6bc983700 1 -- 172.21.15.45:0/2126943979 >> 172.21.15.45:0/2126943979 conn(0x7fd6b80cfdd0 msgr2=0x7fd6b80d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6bc983700 1 -- 172.21.15.45:0/2126943979 shutdown_connections 2023-08-27T11:28:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.498+0000 7fd6bc983700 1 -- 172.21.15.45:0/2126943979 wait complete. 2023-08-27T11:28:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6bc983700 1 Processor -- start 2023-08-27T11:28:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6bc983700 1 -- start start 2023-08-27T11:28:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6bc983700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b8071e10 0x7fd6b814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6bc983700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 0x7fd6b80b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6bc983700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6b815a000 0x7fd6b80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6bc983700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b80b3eb0 con 0x7fd6b80c5e50 2023-08-27T11:28:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6bc983700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b8150de0 con 0x7fd6b8071e10 2023-08-27T11:28:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6bc983700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b814f1f0 con 0x7fd6b815a000 2023-08-27T11:28:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6b5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 0x7fd6b80b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.500+0000 7fd6b5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 0x7fd6b80b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60242/0 (socket says 172.21.15.45:60242) 2023-08-27T11:28:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b5d9b700 1 -- 172.21.15.45:0/2167775345 learned_addr learned my addr 172.21.15.45:0/2167775345 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b6d9d700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6b815a000 0x7fd6b80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b659c700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b8071e10 0x7fd6b814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b5d9b700 1 -- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6b815a000 msgr2=0x7fd6b80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b5d9b700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6b815a000 0x7fd6b80ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b5d9b700 1 -- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b8071e10 msgr2=0x7fd6b814bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b5d9b700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b8071e10 0x7fd6b814bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b5d9b700 1 -- 172.21.15.45:0/2167775345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b0009a70 con 0x7fd6b80c5e50 2023-08-27T11:28:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b659c700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b8071e10 0x7fd6b814bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b6d9d700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6b815a000 0x7fd6b80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6b5d9b700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 0x7fd6b80b3970 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c730 tx=0x7fd6a800caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6aeffd700 1 -- 172.21.15.45:0/2167775345 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8005d70 con 0x7fd6b80c5e50 2023-08-27T11:28:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6aeffd700 1 -- 172.21.15.45:0/2167775345 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a8007070 con 0x7fd6b80c5e50 2023-08-27T11:28:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6aeffd700 1 -- 172.21.15.45:0/2167775345 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8007420 con 0x7fd6b80c5e50 2023-08-27T11:28:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b80b4960 con 0x7fd6b80c5e50 2023-08-27T11:28:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.501+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b80b43a0 con 0x7fd6b80c5e50 2023-08-27T11:28:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.502+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b80727c0 con 0x7fd6b80c5e50 2023-08-27T11:28:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.503+0000 7fd6aeffd700 1 -- 172.21.15.45:0/2167775345 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fd6a8007580 con 0x7fd6b80c5e50 2023-08-27T11:28:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.503+0000 7fd6aeffd700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd69806a170 0x7fd69806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.503+0000 7fd6aeffd700 1 -- 172.21.15.45:0/2167775345 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fd6a8087760 con 0x7fd6b80c5e50 2023-08-27T11:28:46.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.503+0000 7fd6b659c700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd69806a170 0x7fd69806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.504+0000 7fd6b659c700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd69806a170 0x7fd69806c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00011e0 tx=0x7fd6a0009040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.508+0000 7fd6aeffd700 1 -- 172.21.15.45:0/2167775345 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd6a80530d0 con 0x7fd6b80c5e50 2023-08-27T11:28:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.709+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6b815ea60 con 0x7fd6b80c5e50 2023-08-27T11:28:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.710+0000 7fd6aeffd700 1 -- 172.21.15.45:0/2167775345 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6b815ea60 con 0x7fd6b80c5e50 2023-08-27T11:28:46.711 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:28:46.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd69806a170 msgr2=0x7fd69806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd69806a170 0x7fd69806c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00011e0 tx=0x7fd6a0009040 comp rx=0 tx=0).stop 2023-08-27T11:28:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 msgr2=0x7fd6b80b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 0x7fd6b80b3970 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c730 tx=0x7fd6a800caf0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 shutdown_connections 2023-08-27T11:28:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd69806a170 0x7fd69806c630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd6b815a000 0x7fd6b80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6b8071e10 0x7fd6b814bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 --2- 172.21.15.45:0/2167775345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b80c5e50 0x7fd6b80b3970 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 >> 172.21.15.45:0/2167775345 conn(0x7fd6b80cfdd0 msgr2=0x7fd6b815e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 shutdown_connections 2023-08-27T11:28:46.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.712+0000 7fd6bc983700 1 -- 172.21.15.45:0/2167775345 wait complete. 2023-08-27T11:28:46.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:28:46.823 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:46.821+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:46.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.855+0000 7f0640070700 1 Processor -- start 2023-08-27T11:28:46.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.855+0000 7f0640070700 1 -- start start 2023-08-27T11:28:46.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.856+0000 7f0640070700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f063815d3f0 0x7f0638160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.856+0000 7f0640070700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06380aed60 0x7f0638161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.856+0000 7f0640070700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06380af420 0x7f063806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.856+0000 7f0640070700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06380c6ca0 con 0x7f06380aed60 2023-08-27T11:28:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.856+0000 7f0640070700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063815d7d0 con 0x7f063815d3f0 2023-08-27T11:28:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.856+0000 7f0640070700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0638071de0 con 0x7f06380af420 2023-08-27T11:28:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.856+0000 7f063d60b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06380aed60 0x7f0638161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.856+0000 7f063d60b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06380aed60 0x7f0638161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60250/0 (socket says 172.21.15.45:60250) 2023-08-27T11:28:46.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.856+0000 7f063d60b700 1 -- 172.21.15.45:0/1235477883 learned_addr learned my addr 172.21.15.45:0/1235477883 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:46.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063e60d700 1 --2- 172.21.15.45:0/1235477883 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06380af420 0x7f063806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063de0c700 1 --2- 172.21.15.45:0/1235477883 >> v2:172.21.15.135:3300/0 conn(0x7f063815d3f0 0x7f0638160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063d60b700 1 -- 172.21.15.45:0/1235477883 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06380af420 msgr2=0x7f063806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:46.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063d60b700 1 --2- 172.21.15.45:0/1235477883 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06380af420 0x7f063806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.860 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063d60b700 1 -- 172.21.15.45:0/1235477883 >> v2:172.21.15.135:3300/0 conn(0x7f063815d3f0 msgr2=0x7f0638160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:46.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063d60b700 1 --2- 172.21.15.45:0/1235477883 >> v2:172.21.15.135:3300/0 conn(0x7f063815d3f0 0x7f0638160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063d60b700 1 -- 172.21.15.45:0/1235477883 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380c5310 con 0x7f06380aed60 2023-08-27T11:28:46.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063d60b700 1 --2- 172.21.15.45:0/1235477883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380aed60 0x7f0638161390 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f06380cd8b0 tx=0x7f062c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a282a2d7d32cc6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063ce0a700 1 -- 172.21.15.45:0/1235477883 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c011830 con 0x7f06380aed60 2023-08-27T11:28:46.861 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063ce0a700 1 -- 172.21.15.45:0/1235477883 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c011990 con 0x7f06380aed60 2023-08-27T11:28:46.862 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.857+0000 7f063ce0a700 1 -- 172.21.15.45:0/1235477883 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c011d20 con 0x7f06380aed60 2023-08-27T11:28:46.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.858+0000 7f0640070700 1 -- 172.21.15.45:0/1235477883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380aed60 msgr2=0x7f0638161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.858+0000 7f0640070700 1 --2- 172.21.15.45:0/1235477883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380aed60 0x7f0638161390 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f06380cd8b0 tx=0x7f062c009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:46.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.858+0000 7f0640070700 1 -- 172.21.15.45:0/1235477883 shutdown_connections 2023-08-27T11:28:46.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.858+0000 7f0640070700 1 --2- 172.21.15.45:0/1235477883 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f06380af420 0x7f063806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.865 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.858+0000 7f0640070700 1 --2- 172.21.15.45:0/1235477883 >> v2:172.21.15.135:3300/0 conn(0x7f063815d3f0 0x7f0638160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.858+0000 7f0640070700 1 --2- 172.21.15.45:0/1235477883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380aed60 0x7f0638161390 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.858+0000 7f0640070700 1 -- 172.21.15.45:0/1235477883 >> 172.21.15.45:0/1235477883 conn(0x7f06380cfdd0 msgr2=0x7f06380d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:46.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.858+0000 7f0640070700 1 -- 172.21.15.45:0/1235477883 shutdown_connections 2023-08-27T11:28:46.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.858+0000 7f0640070700 1 -- 172.21.15.45:0/1235477883 wait complete. 2023-08-27T11:28:46.866 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.859+0000 7f0640070700 1 Processor -- start 2023-08-27T11:28:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.859+0000 7f0640070700 1 -- start start 2023-08-27T11:28:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f0640070700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06380aed60 0x7f0638153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f0640070700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380af420 0x7f0638152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f0640070700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f063815d3f0 0x7f06380b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f0640070700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638153c50 con 0x7f06380af420 2023-08-27T11:28:46.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f0640070700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638154250 con 0x7f06380aed60 2023-08-27T11:28:46.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f0640070700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0638154530 con 0x7f063815d3f0 2023-08-27T11:28:46.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063d60b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380af420 0x7f0638152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063d60b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380af420 0x7f0638152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60256/0 (socket says 172.21.15.45:60256) 2023-08-27T11:28:46.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063d60b700 1 -- 172.21.15.45:0/165308987 learned_addr learned my addr 172.21.15.45:0/165308987 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:46.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063de0c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06380aed60 0x7f0638153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063e60d700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f063815d3f0 0x7f06380b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063d60b700 1 -- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f063815d3f0 msgr2=0x7f06380b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063d60b700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f063815d3f0 0x7f06380b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.869 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063d60b700 1 -- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06380aed60 msgr2=0x7f0638153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:46.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063d60b700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06380aed60 0x7f0638153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:46.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063d60b700 1 -- 172.21.15.45:0/165308987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c009550 con 0x7f06380af420 2023-08-27T11:28:46.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063de0c700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06380aed60 0x7f0638153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:46.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.860+0000 7f063d60b700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380af420 0x7f0638152a20 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f062c0098f0 tx=0x7f062c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.870 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.861+0000 7f06367fc700 1 -- 172.21.15.45:0/165308987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c004240 con 0x7f06380af420 2023-08-27T11:28:46.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.861+0000 7f06367fc700 1 -- 172.21.15.45:0/165308987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c01f070 con 0x7f06380af420 2023-08-27T11:28:46.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.861+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0638154810 con 0x7f06380af420 2023-08-27T11:28:46.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.861+0000 7f06367fc700 1 -- 172.21.15.45:0/165308987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0045c0 con 0x7f06380af420 2023-08-27T11:28:46.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.861+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0638154dd0 con 0x7f06380af420 2023-08-27T11:28:46.871 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.862+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06380727c0 con 0x7f06380af420 2023-08-27T11:28:46.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.862+0000 7f06367fc700 1 -- 172.21.15.45:0/165308987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f062c021030 con 0x7f06380af420 2023-08-27T11:28:46.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.863+0000 7f06367fc700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f062406a170 0x7f062406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:46.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.863+0000 7f06367fc700 1 -- 172.21.15.45:0/165308987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f062c089440 con 0x7f06380af420 2023-08-27T11:28:46.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.863+0000 7f063de0c700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f062406a170 0x7f062406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:46.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.863+0000 7f063de0c700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f062406a170 0x7f062406c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0628009fd0 tx=0x7f062800b040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:46.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:46.867+0000 7f06367fc700 1 -- 172.21.15.45:0/165308987 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f062c054db0 con 0x7f06380af420 2023-08-27T11:28:47.070 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.069+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f06380b5ca0 con 0x7f06380af420 2023-08-27T11:28:47.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.070+0000 7f06367fc700 1 -- 172.21.15.45:0/165308987 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7f062c0583d0 con 0x7f06380af420 2023-08-27T11:28:47.071 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:28:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f062406a170 msgr2=0x7f062406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.073 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f062406a170 0x7f062406c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0628009fd0 tx=0x7f062800b040 comp rx=0 tx=0).stop 2023-08-27T11:28:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380af420 msgr2=0x7f0638152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380af420 0x7f0638152a20 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f062c0098f0 tx=0x7f062c004750 comp rx=0 tx=0).stop 2023-08-27T11:28:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 shutdown_connections 2023-08-27T11:28:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f062406a170 0x7f062406c630 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.074 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f063815d3f0 0x7f06380b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06380aed60 0x7f0638153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 --2- 172.21.15.45:0/165308987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380af420 0x7f0638152a20 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 >> 172.21.15.45:0/165308987 conn(0x7f06380cfdd0 msgr2=0x7f063806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:47.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 shutdown_connections 2023-08-27T11:28:47.075 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.072+0000 7f0640070700 1 -- 172.21.15.45:0/165308987 wait complete. 2023-08-27T11:28:47.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-27T11:28:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.213+0000 7f5d9a3a8700 1 Processor -- start 2023-08-27T11:28:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.213+0000 7f5d9a3a8700 1 -- start start 2023-08-27T11:28:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d9a3a8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d940cf980 0x7f5d940d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d9a3a8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d940ceae0 con 0x7f5d940b45d0 2023-08-27T11:28:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d9a3a8700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d940cedc0 con 0x7f5d940cfd60 2023-08-27T11:28:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d9a3a8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940cf0a0 con 0x7f5d940cf980 2023-08-27T11:28:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d937fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5d940b45d0 0x7f5d940d1460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47042/0 (socket says 172.21.15.45:47042) 2023-08-27T11:28:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d937fe700 1 -- 172.21.15.45:0/2124559637 learned_addr learned my addr 172.21.15.45:0/2124559637 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d93fff700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d940cf980 0x7f5d940d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838244235 0 0) 0x7f5d940ceae0 con 0x7f5d940b45d0 2023-08-27T11:28:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d800035b0 con 0x7f5d940b45d0 2023-08-27T11:28:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2133832305 0 0) 0x7f5d800035b0 con 0x7f5d940b45d0 2023-08-27T11:28:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d940cf980 msgr2=0x7f5d940d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d940cf980 0x7f5d940d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 >> v1:172.21.15.135:6789/0 conn(0x7f5d940cfd60 legacy=0x7f5d940b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940b22c0 con 0x7f5d940b45d0 2023-08-27T11:28:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f5d7c001670 con 0x7f5d940b45d0 2023-08-27T11:28:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 >> v1:172.21.15.45:6789/0 conn(0x7f5d940b45d0 legacy=0x7f5d940d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d80003a30 0x7f5d80003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.221 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d800043e0 0x7f5d800048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d940cf980 0x7f5d80005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940cf0a0 con 0x7f5d800043e0 2023-08-27T11:28:47.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8000ba10 con 0x7f5d80003a30 2023-08-27T11:28:47.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.214+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8000bbd0 con 0x7f5d940cf980 2023-08-27T11:28:47.222 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d937fe700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d800043e0 0x7f5d800048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d93fff700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d80003a30 0x7f5d80003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d98945700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d940cf980 0x7f5d80005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d937fe700 1 -- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d940cf980 msgr2=0x7f5d80005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:47.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d937fe700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d940cf980 0x7f5d80005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.223 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d937fe700 1 -- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d80003a30 msgr2=0x7f5d80003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:47.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d937fe700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d80003a30 0x7f5d80003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d937fe700 1 -- 172.21.15.45:0/2124559637 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d800035b0 con 0x7f5d800043e0 2023-08-27T11:28:47.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d937fe700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d800043e0 0x7f5d800048b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f5d940b49f0 tx=0x7f5d80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6694b79fc0f56412 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00cd40 con 0x7f5d800043e0 2023-08-27T11:28:47.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c005960 con 0x7f5d800043e0 2023-08-27T11:28:47.224 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.215+0000 7f5d92ffd700 1 -- 172.21.15.45:0/2124559637 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c005d10 con 0x7f5d800043e0 2023-08-27T11:28:47.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.216+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/2124559637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d800043e0 msgr2=0x7f5d800048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.216+0000 7f5d9a3a8700 1 --2- 172.21.15.45:0/2124559637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d800043e0 0x7f5d800048b0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f5d940b49f0 tx=0x7f5d80005880 comp rx=0 tx=0).stop 2023-08-27T11:28:47.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.216+0000 7f5d98945700 1 -- 172.21.15.45:0/2124559637 reap_dead start 2023-08-27T11:28:47.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.216+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/2124559637 shutdown_connections 2023-08-27T11:28:47.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.216+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/2124559637 >> 172.21.15.45:0/2124559637 conn(0x7f5d940d3020 msgr2=0x7f5d940d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:47.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.216+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/2124559637 shutdown_connections 2023-08-27T11:28:47.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.216+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/2124559637 wait complete. 2023-08-27T11:28:47.226 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.217+0000 7f5d9a3a8700 1 Processor -- start 2023-08-27T11:28:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.218+0000 7f5d9a3a8700 1 -- start start 2023-08-27T11:28:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.218+0000 7f5d9a3a8700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d800043e0 0x7f5d940c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d9a3a8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94156110 0x7f5d940c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d9a3a8700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d940c79a0 0x7f5d940b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d9a3a8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940c99a0 con 0x7f5d94156110 2023-08-27T11:28:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d9a3a8700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941e7090 con 0x7f5d940c79a0 2023-08-27T11:28:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d9a3a8700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941e7370 con 0x7f5d800043e0 2023-08-27T11:28:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94156110 0x7f5d940c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d93fff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d800043e0 0x7f5d940c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d937fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94156110 0x7f5d940c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60276/0 (socket says 172.21.15.45:60276) 2023-08-27T11:28:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d937fe700 1 -- 172.21.15.45:0/698549876 learned_addr learned my addr 172.21.15.45:0/698549876 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d98945700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d940c79a0 0x7f5d940b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d937fe700 1 -- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d800043e0 msgr2=0x7f5d940c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d937fe700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d800043e0 0x7f5d940c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d937fe700 1 -- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d940c79a0 msgr2=0x7f5d940b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d937fe700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d940c79a0 0x7f5d940b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d937fe700 1 -- 172.21.15.45:0/698549876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d941e7650 con 0x7f5d94156110 2023-08-27T11:28:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d937fe700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94156110 0x7f5d940c7460 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c001800 tx=0x7f5d7c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d90ff9700 1 -- 172.21.15.45:0/698549876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c001a70 con 0x7f5d94156110 2023-08-27T11:28:47.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d90ff9700 1 -- 172.21.15.45:0/698549876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c005ce0 con 0x7f5d94156110 2023-08-27T11:28:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d90ff9700 1 -- 172.21.15.45:0/698549876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c007310 con 0x7f5d94156110 2023-08-27T11:28:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d941e7930 con 0x7f5d94156110 2023-08-27T11:28:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.219+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d94156570 con 0x7f5d94156110 2023-08-27T11:28:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.220+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d94156760 con 0x7f5d94156110 2023-08-27T11:28:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.221+0000 7f5d90ff9700 1 -- 172.21.15.45:0/698549876 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f5d7c019030 con 0x7f5d94156110 2023-08-27T11:28:47.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.221+0000 7f5d90ff9700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5d7406a170 0x7f5d7406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.221+0000 7f5d90ff9700 1 -- 172.21.15.45:0/698549876 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f5d7c0895e0 con 0x7f5d94156110 2023-08-27T11:28:47.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.221+0000 7f5d93fff700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5d7406a170 0x7f5d7406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.222+0000 7f5d93fff700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5d7406a170 0x7f5d7406c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d940ca1e0 tx=0x7f5d84009210 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.226+0000 7f5d90ff9700 1 -- 172.21.15.45:0/698549876 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5d7c058a80 con 0x7f5d94156110 2023-08-27T11:28:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.427+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5d940a1030 con 0x7f5d94156110 2023-08-27T11:28:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.428+0000 7f5d90ff9700 1 -- 172.21.15.45:0/698549876 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v111) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5d7c007da0 con 0x7f5d94156110 2023-08-27T11:28:47.429 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:47.430 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:28:47.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:47.430 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2023-08-27T11:28:47.430 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2023-08-27T11:28:47.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2023-08-27T11:28:47.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:28:47.431 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 2, 2023-08-27T11:28:47.431 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": "rbd" 2023-08-27T11:28:47.431 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:28:47.431 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:28:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5d7406a170 msgr2=0x7f5d7406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5d7406a170 0x7f5d7406c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d940ca1e0 tx=0x7f5d84009210 comp rx=0 tx=0).stop 2023-08-27T11:28:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94156110 msgr2=0x7f5d940c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94156110 0x7f5d940c7460 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c001800 tx=0x7f5d7c001a00 comp rx=0 tx=0).stop 2023-08-27T11:28:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 shutdown_connections 2023-08-27T11:28:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f5d7406a170 0x7f5d7406c630 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d940c79a0 0x7f5d940b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f5d800043e0 0x7f5d940c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 --2- 172.21.15.45:0/698549876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d94156110 0x7f5d940c7460 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 >> 172.21.15.45:0/698549876 conn(0x7f5d940d3020 msgr2=0x7f5d940b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 shutdown_connections 2023-08-27T11:28:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.430+0000 7f5d9a3a8700 1 -- 172.21.15.45:0/698549876 wait complete. 2023-08-27T11:28:47.443 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-27T11:28:47.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-27T11:28:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.575+0000 7fbb9d7c9700 1 Processor -- start 2023-08-27T11:28:47.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.575+0000 7fbb9d7c9700 1 -- start start 2023-08-27T11:28:47.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.576+0000 7fbb9d7c9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 0x7fbb980cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.576+0000 7fbb9d7c9700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbb980ce2e0 0x7fbb980b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.576+0000 7fbb9d7c9700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9815d6a0 con 0x7fbb9815d2c0 2023-08-27T11:28:47.579 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.576+0000 7fbb9d7c9700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb980ce770 con 0x7fbb980ce2e0 2023-08-27T11:28:47.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.576+0000 7fbb9d7c9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980d2c90 con 0x7fbb980b8c20 2023-08-27T11:28:47.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.576+0000 7fbb967fc700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 0x7fbb980cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.576+0000 7fbb977fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbb980ce2e0 0x7fbb980b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.576+0000 7fbb96ffd700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fbb9815d2c0 0x7fbb980d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:42722/0 (socket says 172.21.15.45:42722) 2023-08-27T11:28:47.580 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.576+0000 7fbb96ffd700 1 -- 172.21.15.45:0/1491809104 learned_addr learned my addr 172.21.15.45:0/1491809104 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:47.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb977fe700 1 -- 172.21.15.45:0/1491809104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 msgr2=0x7fbb980cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:47.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb977fe700 1 --2- 172.21.15.45:0/1491809104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 0x7fbb980cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb977fe700 1 -- 172.21.15.45:0/1491809104 >> v1:172.21.15.135:6789/0 conn(0x7fbb9815d2c0 legacy=0x7fbb980d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb977fe700 1 -- 172.21.15.45:0/1491809104 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980d1bf0 con 0x7fbb980ce2e0 2023-08-27T11:28:47.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb95ffb700 1 -- 172.21.15.45:0/1491809104 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398944629 0 0) 0x7fbb8c001110 con 0x7fbb9815d2c0 2023-08-27T11:28:47.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb967fc700 1 --2- 172.21.15.45:0/1491809104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 0x7fbb980cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:47.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb977fe700 1 --2- 172.21.15.45:0/1491809104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb980ce2e0 0x7fbb980b6100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009d90 tx=0x7fbb9000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec348b38aa71ec41 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb95ffb700 1 -- 172.21.15.45:0/1491809104 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000e070 con 0x7fbb980ce2e0 2023-08-27T11:28:47.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb95ffb700 1 -- 172.21.15.45:0/1491809104 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9000bb20 con 0x7fbb980ce2e0 2023-08-27T11:28:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb95ffb700 1 -- 172.21.15.45:0/1491809104 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90012430 con 0x7fbb980ce2e0 2023-08-27T11:28:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/1491809104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb980ce2e0 msgr2=0x7fbb980b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.577+0000 7fbb9d7c9700 1 --2- 172.21.15.45:0/1491809104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb980ce2e0 0x7fbb980b6100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009d90 tx=0x7fbb9000b330 comp rx=0 tx=0).stop 2023-08-27T11:28:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.578+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/1491809104 shutdown_connections 2023-08-27T11:28:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.578+0000 7fbb9d7c9700 1 --2- 172.21.15.45:0/1491809104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb980ce2e0 0x7fbb980b6100 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.578+0000 7fbb9d7c9700 1 --2- 172.21.15.45:0/1491809104 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 0x7fbb980cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.578+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/1491809104 >> 172.21.15.45:0/1491809104 conn(0x7fbb980d2fd0 msgr2=0x7fbb980d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:47.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.578+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/1491809104 shutdown_connections 2023-08-27T11:28:47.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.578+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/1491809104 wait complete. 2023-08-27T11:28:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.579+0000 7fbb9d7c9700 1 Processor -- start 2023-08-27T11:28:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.579+0000 7fbb9d7c9700 1 -- start start 2023-08-27T11:28:47.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb9d7c9700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 0x7fbb980c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb9d7c9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb980ce2e0 0x7fbb980c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb9d7c9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9815d2c0 0x7fbb981ea150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb9d7c9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980c9990 con 0x7fbb980ce2e0 2023-08-27T11:28:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb9d7c9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980ca890 con 0x7fbb9815d2c0 2023-08-27T11:28:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb9d7c9700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980c8cc0 con 0x7fbb980b8c20 2023-08-27T11:28:47.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb967fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb980ce2e0 0x7fbb980c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb967fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb980ce2e0 0x7fbb980c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60290/0 (socket says 172.21.15.45:60290) 2023-08-27T11:28:47.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb967fc700 1 -- 172.21.15.45:0/2368522683 learned_addr learned my addr 172.21.15.45:0/2368522683 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:47.589 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb96ffd700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 0x7fbb980c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.580+0000 7fbb977fe700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9815d2c0 0x7fbb981ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb967fc700 1 -- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 msgr2=0x7fbb980c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb967fc700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 0x7fbb980c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.590 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb967fc700 1 -- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9815d2c0 msgr2=0x7fbb981ea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb967fc700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9815d2c0 0x7fbb981ea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb967fc700 1 -- 172.21.15.45:0/2368522683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb880094a0 con 0x7fbb980ce2e0 2023-08-27T11:28:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb967fc700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb980ce2e0 0x7fbb980c9450 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800ec10 tx=0x7fbb88009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb7f7fe700 1 -- 172.21.15.45:0/2368522683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800c8e0 con 0x7fbb980ce2e0 2023-08-27T11:28:47.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb7f7fe700 1 -- 172.21.15.45:0/2368522683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8800ca40 con 0x7fbb980ce2e0 2023-08-27T11:28:47.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb7f7fe700 1 -- 172.21.15.45:0/2368522683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800cdd0 con 0x7fbb980ce2e0 2023-08-27T11:28:47.594 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb90009a70 con 0x7fbb980ce2e0 2023-08-27T11:28:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.581+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb980ca520 con 0x7fbb980ce2e0 2023-08-27T11:28:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.582+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb980ca0a0 con 0x7fbb980ce2e0 2023-08-27T11:28:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.582+0000 7fbb7f7fe700 1 -- 172.21.15.45:0/2368522683 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fbb88013470 con 0x7fbb980ce2e0 2023-08-27T11:28:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.583+0000 7fbb7f7fe700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fbb8406a1a0 0x7fbb8406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.583+0000 7fbb7f7fe700 1 -- 172.21.15.45:0/2368522683 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fbb88088790 con 0x7fbb980ce2e0 2023-08-27T11:28:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.583+0000 7fbb96ffd700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fbb8406a1a0 0x7fbb8406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.583+0000 7fbb96ffd700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fbb8406a1a0 0x7fbb8406c660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c000f60 tx=0x7fbb8c007a40 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:47.598 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.588+0000 7fbb7f7fe700 1 -- 172.21.15.45:0/2368522683 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbb980ca0a0 con 0x7fbb980ce2e0 2023-08-27T11:28:47.641 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:47.640+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:47.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:47.801+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbb980ca0a0 con 0x7fbb980ce2e0 2023-08-27T11:28:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.070+0000 7fbb7f7fe700 1 -- 172.21.15.45:0/2368522683 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v112) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbb980ca0a0 con 0x7fbb980ce2e0 2023-08-27T11:28:48.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.075+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fbb8406a1a0 msgr2=0x7fbb8406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.075+0000 7fbb9d7c9700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fbb8406a1a0 0x7fbb8406c660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c000f60 tx=0x7fbb8c007a40 comp rx=0 tx=0).stop 2023-08-27T11:28:48.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.075+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb980ce2e0 msgr2=0x7fbb980c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.077 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.075+0000 7fbb9d7c9700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb980ce2e0 0x7fbb980c9450 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800ec10 tx=0x7fbb88009e20 comp rx=0 tx=0).stop 2023-08-27T11:28:48.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.076+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 shutdown_connections 2023-08-27T11:28:48.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.076+0000 7fbb9d7c9700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fbb8406a1a0 0x7fbb8406c660 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.078 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.076+0000 7fbb9d7c9700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9815d2c0 0x7fbb981ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.076+0000 7fbb9d7c9700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fbb980b8c20 0x7fbb980c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.076+0000 7fbb9d7c9700 1 --2- 172.21.15.45:0/2368522683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb980ce2e0 0x7fbb980c9450 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.076+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 >> 172.21.15.45:0/2368522683 conn(0x7fbb980d2fd0 msgr2=0x7fbb9806c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.077+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 shutdown_connections 2023-08-27T11:28:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.077+0000 7fbb9d7c9700 1 -- 172.21.15.45:0/2368522683 wait complete. 2023-08-27T11:28:48.080 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2023-08-27T11:28:48.091 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-27T11:28:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.226+0000 7fb2809ed700 1 Processor -- start 2023-08-27T11:28:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.227+0000 7fb2809ed700 1 -- start start 2023-08-27T11:28:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.227+0000 7fb2809ed700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb27c15d340 0x7fb27c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.227+0000 7fb2809ed700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb27c0cd880 0x7fb27c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.227+0000 7fb2809ed700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cdd00 0x7fb27c06a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.227+0000 7fb2809ed700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27c072010 con 0x7fb27c15d340 2023-08-27T11:28:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb2809ed700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27c06b160 con 0x7fb27c0cd880 2023-08-27T11:28:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb2809ed700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c0af590 con 0x7fb27c0cdd00 2023-08-27T11:28:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb27c15d340 0x7fb27c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27a59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb27c15d340 0x7fb27c160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60296/0 (socket says 172.21.15.45:60296) 2023-08-27T11:28:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27a59c700 1 -- 172.21.15.45:0/4181393588 learned_addr learned my addr 172.21.15.45:0/4181393588 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27ad9d700 1 --2- 172.21.15.45:0/4181393588 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cdd00 0x7fb27c06a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb279d9b700 1 --2- 172.21.15.45:0/4181393588 >> v2:172.21.15.135:3300/0 conn(0x7fb27c0cd880 0x7fb27c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27a59c700 1 -- 172.21.15.45:0/4181393588 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cdd00 msgr2=0x7fb27c06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27a59c700 1 --2- 172.21.15.45:0/4181393588 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cdd00 0x7fb27c06a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27a59c700 1 -- 172.21.15.45:0/4181393588 >> v2:172.21.15.135:3300/0 conn(0x7fb27c0cd880 msgr2=0x7fb27c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27a59c700 1 --2- 172.21.15.45:0/4181393588 >> v2:172.21.15.135:3300/0 conn(0x7fb27c0cd880 0x7fb27c06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27a59c700 1 -- 172.21.15.45:0/4181393588 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c0af2b0 con 0x7fb27c15d340 2023-08-27T11:28:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27a59c700 1 --2- 172.21.15.45:0/4181393588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 0x7fb27c160d00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0cd7f0 tx=0x7fb26c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd21b84a0d10864b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27959a700 1 -- 172.21.15.45:0/4181393588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c011830 con 0x7fb27c15d340 2023-08-27T11:28:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27959a700 1 -- 172.21.15.45:0/4181393588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c011990 con 0x7fb27c15d340 2023-08-27T11:28:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.228+0000 7fb27959a700 1 -- 172.21.15.45:0/4181393588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c011d20 con 0x7fb27c15d340 2023-08-27T11:28:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.229+0000 7fb2809ed700 1 -- 172.21.15.45:0/4181393588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 msgr2=0x7fb27c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.229+0000 7fb2809ed700 1 --2- 172.21.15.45:0/4181393588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 0x7fb27c160d00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0cd7f0 tx=0x7fb26c009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.229+0000 7fb2809ed700 1 -- 172.21.15.45:0/4181393588 shutdown_connections 2023-08-27T11:28:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.229+0000 7fb2809ed700 1 --2- 172.21.15.45:0/4181393588 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cdd00 0x7fb27c06a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.229+0000 7fb2809ed700 1 --2- 172.21.15.45:0/4181393588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 0x7fb27c160d00 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.229+0000 7fb2809ed700 1 --2- 172.21.15.45:0/4181393588 >> v2:172.21.15.135:3300/0 conn(0x7fb27c0cd880 0x7fb27c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.229+0000 7fb2809ed700 1 -- 172.21.15.45:0/4181393588 >> 172.21.15.45:0/4181393588 conn(0x7fb27c0cc7c0 msgr2=0x7fb27c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.229+0000 7fb2809ed700 1 -- 172.21.15.45:0/4181393588 shutdown_connections 2023-08-27T11:28:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.229+0000 7fb2809ed700 1 -- 172.21.15.45:0/4181393588 wait complete. 2023-08-27T11:28:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.230+0000 7fb2809ed700 1 Processor -- start 2023-08-27T11:28:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.230+0000 7fb2809ed700 1 -- start start 2023-08-27T11:28:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb2809ed700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cd880 0x7fb27c0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb2809ed700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb27c0cdd00 0x7fb27c1512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb2809ed700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 0x7fb27c0ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb2809ed700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c151830 con 0x7fb27c15d340 2023-08-27T11:28:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb2809ed700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c15fe00 con 0x7fb27c0cdd00 2023-08-27T11:28:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb2809ed700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c1f3950 con 0x7fb27c0cd880 2023-08-27T11:28:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb27ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 0x7fb27c0ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb27ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 0x7fb27c0ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60306/0 (socket says 172.21.15.45:60306) 2023-08-27T11:28:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb27ad9d700 1 -- 172.21.15.45:0/2566889067 learned_addr learned my addr 172.21.15.45:0/2566889067 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.231+0000 7fb279d9b700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb27c0cdd00 0x7fb27c1512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb27a59c700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cd880 0x7fb27c0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb27ad9d700 1 -- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cd880 msgr2=0x7fb27c0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb27ad9d700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cd880 0x7fb27c0ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb27ad9d700 1 -- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb27c0cdd00 msgr2=0x7fb27c1512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb27ad9d700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb27c0cdd00 0x7fb27c1512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb27ad9d700 1 -- 172.21.15.45:0/2566889067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c009550 con 0x7fb27c15d340 2023-08-27T11:28:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb279d9b700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb27c0cdd00 0x7fb27c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb27ad9d700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 0x7fb27c0ad900 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb27400efc0 tx=0x7fb27400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb272ffd700 1 -- 172.21.15.45:0/2566889067 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274009b20 con 0x7fb27c15d340 2023-08-27T11:28:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb272ffd700 1 -- 172.21.15.45:0/2566889067 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27400f040 con 0x7fb27c15d340 2023-08-27T11:28:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb272ffd700 1 -- 172.21.15.45:0/2566889067 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274009e50 con 0x7fb27c15d340 2023-08-27T11:28:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27c1f3c30 con 0x7fb27c15d340 2023-08-27T11:28:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.232+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27c1f41f0 con 0x7fb27c15d340 2023-08-27T11:28:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.233+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27c072690 con 0x7fb27c15d340 2023-08-27T11:28:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.234+0000 7fb272ffd700 1 -- 172.21.15.45:0/2566889067 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fb274014480 con 0x7fb27c15d340 2023-08-27T11:28:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.234+0000 7fb272ffd700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb25c06a170 0x7fb25c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.234+0000 7fb272ffd700 1 -- 172.21.15.45:0/2566889067 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5944+0+0 (secure 0 0 0) 0x7fb274089160 con 0x7fb27c15d340 2023-08-27T11:28:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.234+0000 7fb27a59c700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb25c06a170 0x7fb25c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.235+0000 7fb27a59c700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb25c06a170 0x7fb25c06c630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb26c014040 tx=0x7fb26c008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.239+0000 7fb272ffd700 1 -- 172.21.15.45:0/2566889067 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb274058250 con 0x7fb27c15d340 2023-08-27T11:28:48.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:48.448+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb27c0734a0 con 0x7fb27c15d340 2023-08-27T11:28:48.463 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:48.462+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.075+0000 7fb272ffd700 1 -- 172.21.15.45:0/2566889067 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v113) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb274057de0 con 0x7fb27c15d340 2023-08-27T11:28:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.081+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb25c06a170 msgr2=0x7fb25c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.081+0000 7fb2809ed700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb25c06a170 0x7fb25c06c630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb26c014040 tx=0x7fb26c008030 comp rx=0 tx=0).stop 2023-08-27T11:28:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.081+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 msgr2=0x7fb27c0ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.083 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.081+0000 7fb2809ed700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 0x7fb27c0ad900 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb27400efc0 tx=0x7fb27400c3a0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.086+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 shutdown_connections 2023-08-27T11:28:49.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.086+0000 7fb2809ed700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fb25c06a170 0x7fb25c06c630 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.086+0000 7fb2809ed700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27c15d340 0x7fb27c0ad900 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.086+0000 7fb2809ed700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fb27c0cd880 0x7fb27c0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.086+0000 7fb2809ed700 1 --2- 172.21.15.45:0/2566889067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb27c0cdd00 0x7fb27c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.086+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 >> 172.21.15.45:0/2566889067 conn(0x7fb27c0cc7c0 msgr2=0x7fb27c06e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.087+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 shutdown_connections 2023-08-27T11:28:49.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.089+0000 7fb2809ed700 1 -- 172.21.15.45:0/2566889067 wait complete. 2023-08-27T11:28:49.091 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2023-08-27T11:28:49.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-27T11:28:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.236+0000 7f6bb55c3700 1 Processor -- start 2023-08-27T11:28:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.236+0000 7f6bb55c3700 1 -- start start 2023-08-27T11:28:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6bb55c3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb015b270 0x7f6bb006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6bb55c3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6bb0071e80 0x7f6bb00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6bb55c3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb006f8b0 con 0x7f6bb006cf40 2023-08-27T11:28:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6bb55c3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb00ca650 con 0x7f6bb0071e80 2023-08-27T11:28:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6bb55c3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb006f010 con 0x7f6bb015b270 2023-08-27T11:28:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6baf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6bb0071e80 0x7f6bb00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6baeffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6bb006cf40 0x7f6bb006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47086/0 (socket says 172.21.15.45:47086) 2023-08-27T11:28:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6baeffd700 1 -- 172.21.15.45:0/2072079395 learned_addr learned my addr 172.21.15.45:0/2072079395 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6baf7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6bb0071e80 0x7f6bb00caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60314/0 (socket says 172.21.15.45:60314) 2023-08-27T11:28:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6bae7fc700 1 --2- 172.21.15.45:0/2072079395 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb015b270 0x7f6bb006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6baf7fe700 1 -- 172.21.15.45:0/2072079395 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb015b270 msgr2=0x7f6bb006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6baf7fe700 1 --2- 172.21.15.45:0/2072079395 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb015b270 0x7f6bb006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6baf7fe700 1 -- 172.21.15.45:0/2072079395 >> v1:172.21.15.45:6789/0 conn(0x7f6bb006cf40 legacy=0x7f6bb006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6baf7fe700 1 -- 172.21.15.45:0/2072079395 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb006f2f0 con 0x7f6bb0071e80 2023-08-27T11:28:49.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6badffb700 1 -- 172.21.15.45:0/2072079395 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551883037 0 0) 0x7f6ba0001110 con 0x7f6bb006cf40 2023-08-27T11:28:49.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6baf7fe700 1 --2- 172.21.15.45:0/2072079395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0071e80 0x7f6bb00caa70 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009d90 tx=0x7f6ba800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c65d638124a924 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:49.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6badffb700 1 -- 172.21.15.45:0/2072079395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800e070 con 0x7f6bb0071e80 2023-08-27T11:28:49.242 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6badffb700 1 -- 172.21.15.45:0/2072079395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba800bb20 con 0x7f6bb0071e80 2023-08-27T11:28:49.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.237+0000 7f6badffb700 1 -- 172.21.15.45:0/2072079395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8012430 con 0x7f6bb0071e80 2023-08-27T11:28:49.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.238+0000 7f6bb55c3700 1 -- 172.21.15.45:0/2072079395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0071e80 msgr2=0x7f6bb00caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.238+0000 7f6bb55c3700 1 --2- 172.21.15.45:0/2072079395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0071e80 0x7f6bb00caa70 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009d90 tx=0x7f6ba800b330 comp rx=0 tx=0).stop 2023-08-27T11:28:49.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.238+0000 7f6bb55c3700 1 -- 172.21.15.45:0/2072079395 shutdown_connections 2023-08-27T11:28:49.243 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.238+0000 7f6bb55c3700 1 --2- 172.21.15.45:0/2072079395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0071e80 0x7f6bb00caa70 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.238+0000 7f6bb55c3700 1 --2- 172.21.15.45:0/2072079395 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb015b270 0x7f6bb006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.238+0000 7f6bb55c3700 1 -- 172.21.15.45:0/2072079395 >> 172.21.15.45:0/2072079395 conn(0x7f6bb00cc7e0 msgr2=0x7f6bb00ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.238+0000 7f6bb55c3700 1 -- 172.21.15.45:0/2072079395 shutdown_connections 2023-08-27T11:28:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.238+0000 7f6bb55c3700 1 -- 172.21.15.45:0/2072079395 wait complete. 2023-08-27T11:28:49.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.239+0000 7f6bb55c3700 1 Processor -- start 2023-08-27T11:28:49.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.239+0000 7f6bb55c3700 1 -- start start 2023-08-27T11:28:49.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.240+0000 7f6bb55c3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bb006cf40 0x7f6bb01558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.240+0000 7f6bb55c3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb0071e80 0x7f6bb0153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.240+0000 7f6bb55c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb015b270 0x7f6bb01f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.240+0000 7f6bb55c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01554c0 con 0x7f6bb015b270 2023-08-27T11:28:49.247 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.240+0000 7f6bb55c3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb0154840 con 0x7f6bb006cf40 2023-08-27T11:28:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.240+0000 7f6bb55c3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb0154b20 con 0x7f6bb0071e80 2023-08-27T11:28:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.240+0000 7f6baf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb015b270 0x7f6bb01f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.240+0000 7f6baf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb015b270 0x7f6bb01f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60320/0 (socket says 172.21.15.45:60320) 2023-08-27T11:28:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.240+0000 7f6baf7fe700 1 -- 172.21.15.45:0/1301049866 learned_addr learned my addr 172.21.15.45:0/1301049866 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6bae7fc700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb0071e80 0x7f6bb0153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6baeffd700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bb006cf40 0x7f6bb01558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6baf7fe700 1 -- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb0071e80 msgr2=0x7f6bb0153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6baf7fe700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb0071e80 0x7f6bb0153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6baf7fe700 1 -- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bb006cf40 msgr2=0x7f6bb01558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6baf7fe700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bb006cf40 0x7f6bb01558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6baf7fe700 1 -- 172.21.15.45:0/1301049866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba8009a70 con 0x7f6bb015b270 2023-08-27T11:28:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6baeffd700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bb006cf40 0x7f6bb01558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6baf7fe700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb015b270 0x7f6bb01f4a80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8014040 tx=0x7f6ba8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6b977fe700 1 -- 172.21.15.45:0/1301049866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8012430 con 0x7f6bb015b270 2023-08-27T11:28:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6b977fe700 1 -- 172.21.15.45:0/1301049866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba8020070 con 0x7f6bb015b270 2023-08-27T11:28:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6b977fe700 1 -- 172.21.15.45:0/1301049866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8004280 con 0x7f6bb015b270 2023-08-27T11:28:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb0154e00 con 0x7f6bb015b270 2023-08-27T11:28:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.241+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb0154530 con 0x7f6bb015b270 2023-08-27T11:28:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.242+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb0155600 con 0x7f6bb015b270 2023-08-27T11:28:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.243+0000 7f6b977fe700 1 -- 172.21.15.45:0/1301049866 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f6ba80043e0 con 0x7f6bb015b270 2023-08-27T11:28:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.243+0000 7f6b977fe700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f6b9c06a170 0x7f6b9c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.243+0000 7f6b977fe700 1 -- 172.21.15.45:0/1301049866 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f6ba8022070 con 0x7f6bb015b270 2023-08-27T11:28:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.243+0000 7f6baeffd700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f6b9c06a170 0x7f6b9c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.244+0000 7f6baeffd700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f6b9c06a170 0x7f6b9c06c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0000f60 tx=0x7f6ba0007ae0 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.248+0000 7f6b977fe700 1 -- 172.21.15.45:0/1301049866 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6bb0155600 con 0x7f6bb015b270 2023-08-27T11:28:49.303 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:49.302+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:49.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:49.510+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6bb0155600 con 0x7f6bb015b270 2023-08-27T11:28:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.079+0000 7f6b977fe700 1 -- 172.21.15.45:0/1301049866 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v45) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6bb0155600 con 0x7f6bb015b270 2023-08-27T11:28:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.086+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f6b9c06a170 msgr2=0x7f6b9c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.086+0000 7f6bb55c3700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f6b9c06a170 0x7f6b9c06c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0000f60 tx=0x7f6ba0007ae0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.086+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb015b270 msgr2=0x7f6bb01f4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.086+0000 7f6bb55c3700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb015b270 0x7f6bb01f4a80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8014040 tx=0x7f6ba8004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.087+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 shutdown_connections 2023-08-27T11:28:50.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.087+0000 7f6bb55c3700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f6b9c06a170 0x7f6b9c06c630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.087+0000 7f6bb55c3700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb015b270 0x7f6bb01f4a80 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.087+0000 7f6bb55c3700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6bb006cf40 0x7f6bb01558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.087+0000 7f6bb55c3700 1 --2- 172.21.15.45:0/1301049866 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6bb0071e80 0x7f6bb0153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.090 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.087+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 >> 172.21.15.45:0/1301049866 conn(0x7f6bb00cc7e0 msgr2=0x7f6bb006a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.093 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.092+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 shutdown_connections 2023-08-27T11:28:50.095 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.094+0000 7f6bb55c3700 1 -- 172.21.15.45:0/1301049866 wait complete. 2023-08-27T11:28:50.102 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 14 and data pool 15 2023-08-27T11:28:50.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:50.126 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:50.125+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.245+0000 7fe90332a700 1 Processor -- start 2023-08-27T11:28:50.246 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.245+0000 7fe90332a700 1 -- start start 2023-08-27T11:28:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe90332a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8fc15d3d0 0x7fe8fc1619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.248 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe90332a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 0x7fe8fc0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe90332a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8fc161550 con 0x7fe8fc06e3a0 2023-08-27T11:28:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe90332a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8fc0cdf90 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe90332a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc161f20 con 0x7fe8fc0af040 2023-08-27T11:28:50.249 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe9010c6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8fc15d3d0 0x7fe8fc1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.250 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe9018c7700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe8fc06e3a0 0x7fe8fc070e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47118/0 (socket says 172.21.15.45:47118) 2023-08-27T11:28:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe9018c7700 1 -- 172.21.15.45:0/102934517 learned_addr learned my addr 172.21.15.45:0/102934517 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe9008c5700 1 --2- 172.21.15.45:0/102934517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 0x7fe8fc0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe9010c6700 1 -- 172.21.15.45:0/102934517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 msgr2=0x7fe8fc0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:50.252 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe9010c6700 1 --2- 172.21.15.45:0/102934517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 0x7fe8fc0cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe9010c6700 1 -- 172.21.15.45:0/102934517 >> v1:172.21.15.45:6789/0 conn(0x7fe8fc06e3a0 legacy=0x7fe8fc070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe8f3fff700 1 -- 172.21.15.45:0/102934517 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679167940 0 0) 0x7fe8fc161550 con 0x7fe8fc06e3a0 2023-08-27T11:28:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.246+0000 7fe9010c6700 1 -- 172.21.15.45:0/102934517 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc0cd9d0 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe9010c6700 1 --2- 172.21.15.45:0/102934517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 0x7fe8fc1619e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009840 tx=0x7fe8f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b6af6bce967349 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe8f3fff700 1 -- 172.21.15.45:0/102934517 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8011830 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe8f3fff700 1 -- 172.21.15.45:0/102934517 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f8011990 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe8f3fff700 1 -- 172.21.15.45:0/102934517 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8011d20 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe90332a700 1 -- 172.21.15.45:0/102934517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 msgr2=0x7fe8fc1619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe90332a700 1 --2- 172.21.15.45:0/102934517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 0x7fe8fc1619e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009840 tx=0x7fe8f8009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:50.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe90332a700 1 -- 172.21.15.45:0/102934517 shutdown_connections 2023-08-27T11:28:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe90332a700 1 --2- 172.21.15.45:0/102934517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 0x7fe8fc1619e0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe90332a700 1 --2- 172.21.15.45:0/102934517 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 0x7fe8fc0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe90332a700 1 -- 172.21.15.45:0/102934517 >> 172.21.15.45:0/102934517 conn(0x7fe8fc0cfd60 msgr2=0x7fe8fc0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe90332a700 1 -- 172.21.15.45:0/102934517 shutdown_connections 2023-08-27T11:28:50.255 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.247+0000 7fe90332a700 1 -- 172.21.15.45:0/102934517 wait complete. 2023-08-27T11:28:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe90332a700 1 Processor -- start 2023-08-27T11:28:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe90332a700 1 -- start start 2023-08-27T11:28:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe90332a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8fc06e3a0 0x7fe8fc1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe90332a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 0x7fe8fc0c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe90332a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 0x7fe8fc15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe90332a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc0c4c80 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe90332a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc0c5520 con 0x7fe8fc06e3a0 2023-08-27T11:28:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe90332a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc0c5800 con 0x7fe8fc0af040 2023-08-27T11:28:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe9018c7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 0x7fe8fc15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe9018c7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 0x7fe8fc15d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60348/0 (socket says 172.21.15.45:60348) 2023-08-27T11:28:50.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.249+0000 7fe9018c7700 1 -- 172.21.15.45:0/224945730 learned_addr learned my addr 172.21.15.45:0/224945730 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe9008c5700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 0x7fe8fc0c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe9010c6700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8fc06e3a0 0x7fe8fc1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe9018c7700 1 -- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 msgr2=0x7fe8fc0c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe9018c7700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 0x7fe8fc0c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe9018c7700 1 -- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8fc06e3a0 msgr2=0x7fe8fc1593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe9018c7700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8fc06e3a0 0x7fe8fc1593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe9018c7700 1 -- 172.21.15.45:0/224945730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f80094a0 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe9010c6700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8fc06e3a0 0x7fe8fc1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:50.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe9018c7700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 0x7fe8fc15d7b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400cb60 tx=0x7fe8f400ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe8f1ffb700 1 -- 172.21.15.45:0/224945730 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400aaa0 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe8f1ffb700 1 -- 172.21.15.45:0/224945730 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f400ac00 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8fc0c5ae0 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe8f1ffb700 1 -- 172.21.15.45:0/224945730 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40070d0 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.250+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8fc0c5180 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.251+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8fc0c5cd0 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.252+0000 7fe8f1ffb700 1 -- 172.21.15.45:0/224945730 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fe8f4007230 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.252+0000 7fe8f1ffb700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fe8e006a170 0x7fe8e006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.252+0000 7fe8f1ffb700 1 -- 172.21.15.45:0/224945730 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe8f4088360 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.252+0000 7fe9010c6700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fe8e006a170 0x7fe8e006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.253+0000 7fe9010c6700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fe8e006a170 0x7fe8e006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8000c00 tx=0x7fe8f8008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.257+0000 7fe8f1ffb700 1 -- 172.21.15.45:0/224945730 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe8f4053980 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.462+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8fc072780 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.464+0000 7fe8f1ffb700 1 -- 172.21.15.45:0/224945730 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18601 (secure 0 0 0) 0x7fe8f4056fa0 con 0x7fe8fc15d3d0 2023-08-27T11:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":114,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:49.511581+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.834012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.480260+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:50.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fe8e006a170 msgr2=0x7fe8e006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fe8e006a170 0x7fe8e006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8000c00 tx=0x7fe8f8008030 comp rx=0 tx=0).stop 2023-08-27T11:28:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 msgr2=0x7fe8fc15d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 0x7fe8fc15d7b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400cb60 tx=0x7fe8f400ce70 comp rx=0 tx=0).stop 2023-08-27T11:28:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 shutdown_connections 2023-08-27T11:28:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fe8e006a170 0x7fe8e006c630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8fc15d3d0 0x7fe8fc15d7b0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8fc06e3a0 0x7fe8fc1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 --2- 172.21.15.45:0/224945730 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe8fc0af040 0x7fe8fc0c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 >> 172.21.15.45:0/224945730 conn(0x7fe8fc0cfd60 msgr2=0x7fe8fc0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 shutdown_connections 2023-08-27T11:28:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.466+0000 7fe90332a700 1 -- 172.21.15.45:0/224945730 wait complete. 2023-08-27T11:28:50.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:50.607 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.607+0000 7f9e02969700 1 Processor -- start 2023-08-27T11:28:50.608 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.607+0000 7f9e02969700 1 -- start start 2023-08-27T11:28:50.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.607+0000 7f9e02969700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9dfc0af200 0x7f9dfc0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.607+0000 7f9e02969700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9dfc0caa00 0x7f9dfc0acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.610 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.607+0000 7f9e02969700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc0ce340 con 0x7f9dfc0ca620 2023-08-27T11:28:50.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.607+0000 7f9e02969700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc0ceec0 con 0x7f9dfc0caa00 2023-08-27T11:28:50.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.607+0000 7f9e02969700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc15d7c0 con 0x7f9dfc0af200 2023-08-27T11:28:50.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9dfbfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9dfc0ca620 0x7f9dfc0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47128/0 (socket says 172.21.15.45:47128) 2023-08-27T11:28:50.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9dfbfff700 1 -- 172.21.15.45:0/2710155709 learned_addr learned my addr 172.21.15.45:0/2710155709 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:50.611 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9e00f06700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9dfc0caa00 0x7f9dfc0acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9dfb7fe700 1 --2- 172.21.15.45:0/2710155709 >> v2:172.21.15.135:3300/0 conn(0x7f9dfc0af200 0x7f9dfc0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9e00f06700 1 -- 172.21.15.45:0/2710155709 >> v2:172.21.15.135:3300/0 conn(0x7f9dfc0af200 msgr2=0x7f9dfc0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9e00f06700 1 --2- 172.21.15.45:0/2710155709 >> v2:172.21.15.135:3300/0 conn(0x7f9dfc0af200 0x7f9dfc0cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9e00f06700 1 -- 172.21.15.45:0/2710155709 >> v1:172.21.15.45:6789/0 conn(0x7f9dfc0ca620 legacy=0x7f9dfc0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9e00f06700 1 -- 172.21.15.45:0/2710155709 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0cc570 con 0x7f9dfc0caa00 2023-08-27T11:28:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9dfaffd700 1 -- 172.21.15.45:0/2710155709 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94322684 0 0) 0x7f9dfc0ce340 con 0x7f9dfc0ca620 2023-08-27T11:28:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9e00f06700 1 --2- 172.21.15.45:0/2710155709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 0x7f9dfc0acfc0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc06f860 tx=0x7f9df400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e14ebb2c6dd9cd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9dfaffd700 1 -- 172.21.15.45:0/2710155709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400e070 con 0x7f9dfc0caa00 2023-08-27T11:28:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9dfaffd700 1 -- 172.21.15.45:0/2710155709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df400bb20 con 0x7f9dfc0caa00 2023-08-27T11:28:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.608+0000 7f9dfaffd700 1 -- 172.21.15.45:0/2710155709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4012430 con 0x7f9dfc0caa00 2023-08-27T11:28:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.609+0000 7f9e02969700 1 -- 172.21.15.45:0/2710155709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 msgr2=0x7f9dfc0acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.609+0000 7f9e02969700 1 --2- 172.21.15.45:0/2710155709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 0x7f9dfc0acfc0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc06f860 tx=0x7f9df400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.609+0000 7f9e02969700 1 -- 172.21.15.45:0/2710155709 shutdown_connections 2023-08-27T11:28:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.609+0000 7f9e02969700 1 --2- 172.21.15.45:0/2710155709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 0x7f9dfc0acfc0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.609+0000 7f9e02969700 1 --2- 172.21.15.45:0/2710155709 >> v2:172.21.15.135:3300/0 conn(0x7f9dfc0af200 0x7f9dfc0cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.609+0000 7f9e02969700 1 -- 172.21.15.45:0/2710155709 >> 172.21.15.45:0/2710155709 conn(0x7f9dfc0cc7e0 msgr2=0x7f9dfc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.609+0000 7f9e02969700 1 -- 172.21.15.45:0/2710155709 shutdown_connections 2023-08-27T11:28:50.615 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.609+0000 7f9e02969700 1 -- 172.21.15.45:0/2710155709 wait complete. 2023-08-27T11:28:50.617 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.610+0000 7f9e02969700 1 Processor -- start 2023-08-27T11:28:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.610+0000 7f9e02969700 1 -- start start 2023-08-27T11:28:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9e02969700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dfc0af200 0x7f9dfc0c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9e02969700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dfc0ca620 0x7f9dfc1f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9e02969700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 0x7f9dfc158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9e02969700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc158790 con 0x7f9dfc0caa00 2023-08-27T11:28:50.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9e02969700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0c9cc0 con 0x7f9dfc0af200 2023-08-27T11:28:50.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9e02969700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc157b10 con 0x7f9dfc0ca620 2023-08-27T11:28:50.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9e00f06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 0x7f9dfc158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9e00f06700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 0x7f9dfc158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60362/0 (socket says 172.21.15.45:60362) 2023-08-27T11:28:50.619 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9e00f06700 1 -- 172.21.15.45:0/2403419044 learned_addr learned my addr 172.21.15.45:0/2403419044 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9dfbfff700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dfc0af200 0x7f9dfc0c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.611+0000 7f9dfb7fe700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dfc0ca620 0x7f9dfc1f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9e00f06700 1 -- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dfc0ca620 msgr2=0x7f9dfc1f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.620 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9e00f06700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dfc0ca620 0x7f9dfc1f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9e00f06700 1 -- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dfc0af200 msgr2=0x7f9dfc0c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9e00f06700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dfc0af200 0x7f9dfc0c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.621 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9e00f06700 1 -- 172.21.15.45:0/2403419044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4009a70 con 0x7f9dfc0caa00 2023-08-27T11:28:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9e00f06700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 0x7f9dfc158bb0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9df4014040 tx=0x7f9df4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.622 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9df8ff9700 1 -- 172.21.15.45:0/2403419044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4012430 con 0x7f9dfc0caa00 2023-08-27T11:28:50.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9df8ff9700 1 -- 172.21.15.45:0/2403419044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4020070 con 0x7f9dfc0caa00 2023-08-27T11:28:50.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc157df0 con 0x7f9dfc0caa00 2023-08-27T11:28:50.623 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9df8ff9700 1 -- 172.21.15.45:0/2403419044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4004280 con 0x7f9dfc0caa00 2023-08-27T11:28:50.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.612+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc1583b0 con 0x7f9dfc0caa00 2023-08-27T11:28:50.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.613+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc0ce850 con 0x7f9dfc0caa00 2023-08-27T11:28:50.624 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.614+0000 7f9df8ff9700 1 -- 172.21.15.45:0/2403419044 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f9df40043e0 con 0x7f9dfc0caa00 2023-08-27T11:28:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.614+0000 7f9df8ff9700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9de806a170 0x7f9de806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.614+0000 7f9df8ff9700 1 -- 172.21.15.45:0/2403419044 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9df4022070 con 0x7f9dfc0caa00 2023-08-27T11:28:50.625 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.615+0000 7f9dfbfff700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9de806a170 0x7f9de806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.615+0000 7f9dfbfff700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9de806a170 0x7f9de806c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9dec007fd0 tx=0x7f9dec007de0 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.626 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.619+0000 7f9df8ff9700 1 -- 172.21.15.45:0/2403419044 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9dfc0ce850 con 0x7f9dfc0caa00 2023-08-27T11:28:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.824+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dfc0ce850 con 0x7f9dfc0caa00 2023-08-27T11:28:50.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.825+0000 7f9df8ff9700 1 -- 172.21.15.45:0/2403419044 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18601 (secure 0 0 0) 0x7f9dfc0ce850 con 0x7f9dfc0caa00 2023-08-27T11:28:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:50.826 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":114,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:49.511581+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.834012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.480260+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:50.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9de806a170 msgr2=0x7f9de806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9de806a170 0x7f9de806c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9dec007fd0 tx=0x7f9dec007de0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 msgr2=0x7f9dfc158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 0x7f9dfc158bb0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f9df4014040 tx=0x7f9df4004ad0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 shutdown_connections 2023-08-27T11:28:50.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9de806a170 0x7f9de806c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dfc0caa00 0x7f9dfc158bb0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dfc0af200 0x7f9dfc0c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 --2- 172.21.15.45:0/2403419044 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9dfc0ca620 0x7f9dfc1f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 >> 172.21.15.45:0/2403419044 conn(0x7f9dfc0cc7e0 msgr2=0x7f9dfc0cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 shutdown_connections 2023-08-27T11:28:50.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.827+0000 7f9e02969700 1 -- 172.21.15.45:0/2403419044 wait complete. 2023-08-27T11:28:50.841 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-27T11:28:50.966 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:50.964+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:50.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.972+0000 7fd9b4af6700 1 Processor -- start 2023-08-27T11:28:50.973 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.972+0000 7fd9b4af6700 1 -- start start 2023-08-27T11:28:50.974 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9b4af6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd9b015a2c0 0x7fd9b0073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9b4af6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b00b2f60 0x7fd9b006b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9b4af6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b0073d40 con 0x7fd9b006b890 2023-08-27T11:28:50.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9b4af6700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b00cf360 con 0x7fd9b015a2c0 2023-08-27T11:28:50.975 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9b4af6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b015a6a0 con 0x7fd9b00b2f60 2023-08-27T11:28:50.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9ae59c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd9b015a2c0 0x7fd9b0073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9aed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd9b006b890 0x7fd9b006bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47146/0 (socket says 172.21.15.45:47146) 2023-08-27T11:28:50.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9aed9d700 1 -- 172.21.15.45:0/3506851085 learned_addr learned my addr 172.21.15.45:0/3506851085 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:50.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9add9b700 1 --2- 172.21.15.45:0/3506851085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b00b2f60 0x7fd9b006b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9ae59c700 1 -- 172.21.15.45:0/3506851085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b00b2f60 msgr2=0x7fd9b006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:50.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9ae59c700 1 --2- 172.21.15.45:0/3506851085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b00b2f60 0x7fd9b006b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9ae59c700 1 -- 172.21.15.45:0/3506851085 >> v1:172.21.15.45:6789/0 conn(0x7fd9b006b890 legacy=0x7fd9b006bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9ad59a700 1 -- 172.21.15.45:0/3506851085 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849728782 0 0) 0x7fd9b0073d40 con 0x7fd9b006b890 2023-08-27T11:28:50.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.973+0000 7fd9ae59c700 1 -- 172.21.15.45:0/3506851085 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00ceda0 con 0x7fd9b015a2c0 2023-08-27T11:28:50.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9ae59c700 1 --2- 172.21.15.45:0/3506851085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b015a2c0 0x7fd9b0073800 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009820 tx=0x7fd9a0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae35dea36c82c5a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9ad59a700 1 -- 172.21.15.45:0/3506851085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0005e30 con 0x7fd9b015a2c0 2023-08-27T11:28:50.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9ad59a700 1 -- 172.21.15.45:0/3506851085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a0006030 con 0x7fd9b015a2c0 2023-08-27T11:28:50.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9ad59a700 1 -- 172.21.15.45:0/3506851085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00063e0 con 0x7fd9b015a2c0 2023-08-27T11:28:50.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9b4af6700 1 -- 172.21.15.45:0/3506851085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b015a2c0 msgr2=0x7fd9b0073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9b4af6700 1 --2- 172.21.15.45:0/3506851085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b015a2c0 0x7fd9b0073800 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009820 tx=0x7fd9a0009b30 comp rx=0 tx=0).stop 2023-08-27T11:28:50.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9b4af6700 1 -- 172.21.15.45:0/3506851085 shutdown_connections 2023-08-27T11:28:50.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9b4af6700 1 --2- 172.21.15.45:0/3506851085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b015a2c0 0x7fd9b0073800 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9b4af6700 1 --2- 172.21.15.45:0/3506851085 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b00b2f60 0x7fd9b006b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9b4af6700 1 -- 172.21.15.45:0/3506851085 >> 172.21.15.45:0/3506851085 conn(0x7fd9b00b2440 msgr2=0x7fd9b00b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:50.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9b4af6700 1 -- 172.21.15.45:0/3506851085 shutdown_connections 2023-08-27T11:28:50.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.974+0000 7fd9b4af6700 1 -- 172.21.15.45:0/3506851085 wait complete. 2023-08-27T11:28:50.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.976+0000 7fd9b4af6700 1 Processor -- start 2023-08-27T11:28:50.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.976+0000 7fd9b4af6700 1 -- start start 2023-08-27T11:28:50.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9b4af6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b006b890 0x7fd9b0158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9b4af6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b00b2f60 0x7fd9b015a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9b4af6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9b015a2c0 0x7fd9b015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9b4af6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0158da0 con 0x7fd9b00b2f60 2023-08-27T11:28:50.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9b4af6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b015b360 con 0x7fd9b015a2c0 2023-08-27T11:28:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9b4af6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b015b640 con 0x7fd9b006b890 2023-08-27T11:28:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9add9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b00b2f60 0x7fd9b015a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9add9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b00b2f60 0x7fd9b015a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60374/0 (socket says 172.21.15.45:60374) 2023-08-27T11:28:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9add9b700 1 -- 172.21.15.45:0/1368692307 learned_addr learned my addr 172.21.15.45:0/1368692307 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:50.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.977+0000 7fd9ae59c700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b006b890 0x7fd9b0158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9aed9d700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9b015a2c0 0x7fd9b015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9add9b700 1 -- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b006b890 msgr2=0x7fd9b0158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9add9b700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b006b890 0x7fd9b0158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9add9b700 1 -- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9b015a2c0 msgr2=0x7fd9b015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9add9b700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9b015a2c0 0x7fd9b015ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:50.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9add9b700 1 -- 172.21.15.45:0/1368692307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0009430 con 0x7fd9b00b2f60 2023-08-27T11:28:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9aed9d700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9b015a2c0 0x7fd9b015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9add9b700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b00b2f60 0x7fd9b015a7f0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400e810 tx=0x7fd9a400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd99effd700 1 -- 172.21.15.45:0/1368692307 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4013610 con 0x7fd9b00b2f60 2023-08-27T11:28:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd99effd700 1 -- 172.21.15.45:0/1368692307 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a4013770 con 0x7fd9b00b2f60 2023-08-27T11:28:50.987 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd99effd700 1 -- 172.21.15.45:0/1368692307 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4013b20 con 0x7fd9b00b2f60 2023-08-27T11:28:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b015b920 con 0x7fd9b00b2f60 2023-08-27T11:28:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.978+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b00c4220 con 0x7fd9b00b2f60 2023-08-27T11:28:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.979+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b00c4080 con 0x7fd9b00b2f60 2023-08-27T11:28:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.979+0000 7fd99effd700 1 -- 172.21.15.45:0/1368692307 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fd9a4013c80 con 0x7fd9b00b2f60 2023-08-27T11:28:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.980+0000 7fd99effd700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd99006a170 0x7fd99006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.980+0000 7fd99effd700 1 -- 172.21.15.45:0/1368692307 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fd9a4088520 con 0x7fd9b00b2f60 2023-08-27T11:28:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.980+0000 7fd9ae59c700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd99006a170 0x7fd99006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.980+0000 7fd9ae59c700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd99006a170 0x7fd99006c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0013040 tx=0x7fd9a0008070 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:50.984+0000 7fd99effd700 1 -- 172.21.15.45:0/1368692307 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd9b00c4080 con 0x7fd9b00b2f60 2023-08-27T11:28:51.253 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:51.253+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd9b00c4080 con 0x7fd9b00b2f60 2023-08-27T11:28:51.796 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:51.795+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:52.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.108+0000 7fd99effd700 1 -- 172.21.15.45:0/1368692307 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd9b00c4080 con 0x7fd9b00b2f60 2023-08-27T11:28:52.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd99006a170 msgr2=0x7fd99006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd99006a170 0x7fd99006c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0013040 tx=0x7fd9a0008070 comp rx=0 tx=0).stop 2023-08-27T11:28:52.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b00b2f60 msgr2=0x7fd9b015a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b00b2f60 0x7fd9b015a7f0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400e810 tx=0x7fd9a400ebd0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 shutdown_connections 2023-08-27T11:28:52.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fd99006a170 0x7fd99006c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9b015a2c0 0x7fd9b015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd9b006b890 0x7fd9b0158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 --2- 172.21.15.45:0/1368692307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b00b2f60 0x7fd9b015a7f0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 >> 172.21.15.45:0/1368692307 conn(0x7fd9b00b2440 msgr2=0x7fd9b0071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:52.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 shutdown_connections 2023-08-27T11:28:52.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.110+0000 7fd9b4af6700 1 -- 172.21.15.45:0/1368692307 wait complete. 2023-08-27T11:28:52.123 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-27T11:28:52.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-27T11:28:52.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.253+0000 7f1792e26700 1 Processor -- start 2023-08-27T11:28:52.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.253+0000 7f1792e26700 1 -- start start 2023-08-27T11:28:52.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.254+0000 7f1792e26700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f178c15d3a0 0x7f178c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.254+0000 7f1792e26700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f178c06c4b0 0x7f178c0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.254+0000 7f1792e26700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c15d780 con 0x7f178c0d02c0 2023-08-27T11:28:52.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.254+0000 7f1792e26700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0cfe30 con 0x7f178c06c4b0 2023-08-27T11:28:52.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.254+0000 7f1792e26700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0cf220 con 0x7f178c15d3a0 2023-08-27T11:28:52.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.254+0000 7f17913c3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f178c06c4b0 0x7f178c0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.254+0000 7f17913c3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f178c06c4b0 0x7f178c0cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60384/0 (socket says 172.21.15.45:60384) 2023-08-27T11:28:52.258 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.254+0000 7f17913c3700 1 -- 172.21.15.45:0/4187420135 learned_addr learned my addr 172.21.15.45:0/4187420135 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:52.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.254+0000 7f1790bc2700 1 --2- 172.21.15.45:0/4187420135 >> v2:172.21.15.135:3300/0 conn(0x7f178c15d3a0 0x7f178c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f17913c3700 1 -- 172.21.15.45:0/4187420135 >> v2:172.21.15.135:3300/0 conn(0x7f178c15d3a0 msgr2=0x7f178c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:52.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f17913c3700 1 --2- 172.21.15.45:0/4187420135 >> v2:172.21.15.135:3300/0 conn(0x7f178c15d3a0 0x7f178c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f17913c3700 1 -- 172.21.15.45:0/4187420135 >> v1:172.21.15.135:6789/0 conn(0x7f178c0d02c0 legacy=0x7f178c06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f17913c3700 1 -- 172.21.15.45:0/4187420135 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0cf500 con 0x7f178c06c4b0 2023-08-27T11:28:52.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f17913c3700 1 --2- 172.21.15.45:0/4187420135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 0x7f178c0cb490 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f178c0701d0 tx=0x7f178400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703fd3e39be265a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:52.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f178b7fe700 1 -- 172.21.15.45:0/4187420135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400e070 con 0x7f178c06c4b0 2023-08-27T11:28:52.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f178b7fe700 1 -- 172.21.15.45:0/4187420135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178400b910 con 0x7f178c06c4b0 2023-08-27T11:28:52.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f178b7fe700 1 -- 172.21.15.45:0/4187420135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400bcc0 con 0x7f178c06c4b0 2023-08-27T11:28:52.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f1792e26700 1 -- 172.21.15.45:0/4187420135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 msgr2=0x7f178c0cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.255+0000 7f1792e26700 1 --2- 172.21.15.45:0/4187420135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 0x7f178c0cb490 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f178c0701d0 tx=0x7f178400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.256+0000 7f1792e26700 1 -- 172.21.15.45:0/4187420135 shutdown_connections 2023-08-27T11:28:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.256+0000 7f1792e26700 1 --2- 172.21.15.45:0/4187420135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 0x7f178c0cb490 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.256+0000 7f1792e26700 1 --2- 172.21.15.45:0/4187420135 >> v2:172.21.15.135:3300/0 conn(0x7f178c15d3a0 0x7f178c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.256+0000 7f1792e26700 1 -- 172.21.15.45:0/4187420135 >> 172.21.15.45:0/4187420135 conn(0x7f178c09a1a0 msgr2=0x7f178c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.256+0000 7f1792e26700 1 -- 172.21.15.45:0/4187420135 shutdown_connections 2023-08-27T11:28:52.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.256+0000 7f1792e26700 1 -- 172.21.15.45:0/4187420135 wait complete. 2023-08-27T11:28:52.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.257+0000 7f1792e26700 1 Processor -- start 2023-08-27T11:28:52.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.257+0000 7f1792e26700 1 -- start start 2023-08-27T11:28:52.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.258+0000 7f1792e26700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 0x7f178c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.258+0000 7f1792e26700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178c0d02c0 0x7f178c156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.258+0000 7f1792e26700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178c15d3a0 0x7f178c156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.258+0000 7f1792e26700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0b0cf0 con 0x7f178c06c4b0 2023-08-27T11:28:52.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.258+0000 7f1792e26700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0c62d0 con 0x7f178c0d02c0 2023-08-27T11:28:52.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.258+0000 7f1792e26700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0b03c0 con 0x7f178c15d3a0 2023-08-27T11:28:52.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.258+0000 7f1790bc2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 0x7f178c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.258+0000 7f1790bc2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 0x7f178c0b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60386/0 (socket says 172.21.15.45:60386) 2023-08-27T11:28:52.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.258+0000 7f1790bc2700 1 -- 172.21.15.45:0/2588535284 learned_addr learned my addr 172.21.15.45:0/2588535284 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:52.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f178bfff700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178c0d02c0 0x7f178c156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f1790bc2700 1 -- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178c15d3a0 msgr2=0x7f178c156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f17913c3700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178c15d3a0 0x7f178c156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f1790bc2700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178c15d3a0 0x7f178c156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f1790bc2700 1 -- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178c0d02c0 msgr2=0x7f178c156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f1790bc2700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178c0d02c0 0x7f178c156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f1790bc2700 1 -- 172.21.15.45:0/2588535284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784009b30 con 0x7f178c06c4b0 2023-08-27T11:28:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f178bfff700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178c0d02c0 0x7f178c156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f1790bc2700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 0x7f178c0b4e10 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f178000e870 tx=0x7f178000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f17897fa700 1 -- 172.21.15.45:0/2588535284 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780009590 con 0x7f178c06c4b0 2023-08-27T11:28:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f17897fa700 1 -- 172.21.15.45:0/2588535284 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178000ee70 con 0x7f178c06c4b0 2023-08-27T11:28:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f17897fa700 1 -- 172.21.15.45:0/2588535284 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178000ca50 con 0x7f178c06c4b0 2023-08-27T11:28:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c0b06a0 con 0x7f178c06c4b0 2023-08-27T11:28:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.259+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c0c72b0 con 0x7f178c06c4b0 2023-08-27T11:28:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.260+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c0b0fd0 con 0x7f178c06c4b0 2023-08-27T11:28:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.261+0000 7f17897fa700 1 -- 172.21.15.45:0/2588535284 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f178000cbb0 con 0x7f178c06c4b0 2023-08-27T11:28:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.261+0000 7f17897fa700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f177006a170 0x7f177006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.261+0000 7f17897fa700 1 -- 172.21.15.45:0/2588535284 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1780088660 con 0x7f178c06c4b0 2023-08-27T11:28:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.261+0000 7f178bfff700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f177006a170 0x7f177006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.262+0000 7f178bfff700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f177006a170 0x7f177006c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f178c1f67d0 tx=0x7f1778009040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.266+0000 7f17897fa700 1 -- 172.21.15.45:0/2588535284 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1780057680 con 0x7f178c06c4b0 2023-08-27T11:28:52.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:52.517+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f178c160180 con 0x7f178c06c4b0 2023-08-27T11:28:52.624 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:52.622+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:53.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.108+0000 7f17897fa700 1 -- 172.21.15.45:0/2588535284 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7f178000ce60 con 0x7f178c06c4b0 2023-08-27T11:28:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f177006a170 msgr2=0x7f177006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f177006a170 0x7f177006c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f178c1f67d0 tx=0x7f1778009040 comp rx=0 tx=0).stop 2023-08-27T11:28:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 msgr2=0x7f178c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 0x7f178c0b4e10 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f178000e870 tx=0x7f178000ec30 comp rx=0 tx=0).stop 2023-08-27T11:28:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 shutdown_connections 2023-08-27T11:28:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f177006a170 0x7f177006c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f178c06c4b0 0x7f178c0b4e10 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f178c15d3a0 0x7f178c156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 --2- 172.21.15.45:0/2588535284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f178c0d02c0 0x7f178c156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.110+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 >> 172.21.15.45:0/2588535284 conn(0x7f178c09a1a0 msgr2=0x7f178c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.111+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 shutdown_connections 2023-08-27T11:28:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.111+0000 7f1792e26700 1 -- 172.21.15.45:0/2588535284 wait complete. 2023-08-27T11:28:53.124 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-27T11:28:53.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:53.256 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.256+0000 7fabb1f53700 1 Processor -- start 2023-08-27T11:28:53.257 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.256+0000 7fabb1f53700 1 -- start start 2023-08-27T11:28:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.256+0000 7fabb1f53700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac158cb0 0x7fabac0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.256+0000 7fabb1f53700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fabac06ac60 0x7fabac06ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.256+0000 7fabb1f53700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fabac06d3a0 0x7fabac15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.256+0000 7fabb1f53700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac06b230 con 0x7fabac06ac60 2023-08-27T11:28:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.256+0000 7fabb1f53700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac15b780 con 0x7fabac06d3a0 2023-08-27T11:28:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.256+0000 7fabb1f53700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac15ba20 con 0x7fabac158cb0 2023-08-27T11:28:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fabac06ac60 0x7fabac06ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabab7fe700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac158cb0 0x7fabac0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaaffd700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fabac06ac60 0x7fabac06ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60394/0 (socket says 172.21.15.45:60394) 2023-08-27T11:28:53.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaaffd700 1 -- 172.21.15.45:0/3898024733 learned_addr learned my addr 172.21.15.45:0/3898024733 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:53.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fababfff700 1 --2- 172.21.15.45:0/3898024733 >> v2:172.21.15.135:3300/0 conn(0x7fabac06d3a0 0x7fabac15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaaffd700 1 -- 172.21.15.45:0/3898024733 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac158cb0 msgr2=0x7fabac0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaaffd700 1 --2- 172.21.15.45:0/3898024733 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac158cb0 0x7fabac0ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaaffd700 1 -- 172.21.15.45:0/3898024733 >> v2:172.21.15.135:3300/0 conn(0x7fabac06d3a0 msgr2=0x7fabac15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaaffd700 1 --2- 172.21.15.45:0/3898024733 >> v2:172.21.15.135:3300/0 conn(0x7fabac06d3a0 0x7fabac15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaaffd700 1 -- 172.21.15.45:0/3898024733 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0cc5c0 con 0x7fabac06ac60 2023-08-27T11:28:53.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fababfff700 1 --2- 172.21.15.45:0/3898024733 >> v2:172.21.15.135:3300/0 conn(0x7fabac06d3a0 0x7fabac15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:53.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaaffd700 1 --2- 172.21.15.45:0/3898024733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 0x7fabac06ce60 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fabac0c6db0 tx=0x7faba0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ffed46486870bb server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaa7fc700 1 -- 172.21.15.45:0/3898024733 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0011830 con 0x7fabac06ac60 2023-08-27T11:28:53.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaa7fc700 1 -- 172.21.15.45:0/3898024733 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba0011990 con 0x7fabac06ac60 2023-08-27T11:28:53.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.257+0000 7fabaa7fc700 1 -- 172.21.15.45:0/3898024733 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0011d20 con 0x7fabac06ac60 2023-08-27T11:28:53.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.258+0000 7fabb1f53700 1 -- 172.21.15.45:0/3898024733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 msgr2=0x7fabac06ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.258+0000 7fabb1f53700 1 --2- 172.21.15.45:0/3898024733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 0x7fabac06ce60 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fabac0c6db0 tx=0x7faba0009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.258+0000 7fabb1f53700 1 -- 172.21.15.45:0/3898024733 shutdown_connections 2023-08-27T11:28:53.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.258+0000 7fabb1f53700 1 --2- 172.21.15.45:0/3898024733 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac158cb0 0x7fabac0ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.258+0000 7fabb1f53700 1 --2- 172.21.15.45:0/3898024733 >> v2:172.21.15.135:3300/0 conn(0x7fabac06d3a0 0x7fabac15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.258+0000 7fabb1f53700 1 --2- 172.21.15.45:0/3898024733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 0x7fabac06ce60 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.258+0000 7fabb1f53700 1 -- 172.21.15.45:0/3898024733 >> 172.21.15.45:0/3898024733 conn(0x7fabac0cfd60 msgr2=0x7fabac0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.258+0000 7fabb1f53700 1 -- 172.21.15.45:0/3898024733 shutdown_connections 2023-08-27T11:28:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.258+0000 7fabb1f53700 1 -- 172.21.15.45:0/3898024733 wait complete. 2023-08-27T11:28:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.259+0000 7fabb1f53700 1 Processor -- start 2023-08-27T11:28:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.259+0000 7fabb1f53700 1 -- start start 2023-08-27T11:28:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabb1f53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 0x7fabac0c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabb1f53700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac06d3a0 0x7fabac1552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabb1f53700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabac158cb0 0x7fabac15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabb1f53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0c8700 con 0x7fabac06ac60 2023-08-27T11:28:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabb1f53700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0b2980 con 0x7fabac158cb0 2023-08-27T11:28:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabb1f53700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac155800 con 0x7fabac06d3a0 2023-08-27T11:28:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabab7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 0x7fabac0c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabaaffd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac06d3a0 0x7fabac1552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabab7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 0x7fabac0c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60398/0 (socket says 172.21.15.45:60398) 2023-08-27T11:28:53.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fabab7fe700 1 -- 172.21.15.45:0/1603354564 learned_addr learned my addr 172.21.15.45:0/1603354564 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:53.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.260+0000 7fababfff700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabac158cb0 0x7fabac15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fabab7fe700 1 -- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac06d3a0 msgr2=0x7fabac1552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fabab7fe700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac06d3a0 0x7fabac1552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fabab7fe700 1 -- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabac158cb0 msgr2=0x7fabac15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fabab7fe700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabac158cb0 0x7fabac15ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fabab7fe700 1 -- 172.21.15.45:0/1603354564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba4009a70 con 0x7fabac06ac60 2023-08-27T11:28:53.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fabaaffd700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac06d3a0 0x7fabac1552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:28:53.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fabab7fe700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 0x7fabac0c7840 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e870 tx=0x7fab9c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fab93fff700 1 -- 172.21.15.45:0/1603354564 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c009590 con 0x7fabac06ac60 2023-08-27T11:28:53.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fab93fff700 1 -- 172.21.15.45:0/1603354564 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c013680 con 0x7fabac06ac60 2023-08-27T11:28:53.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba0009550 con 0x7fabac06ac60 2023-08-27T11:28:53.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fab93fff700 1 -- 172.21.15.45:0/1603354564 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0139f0 con 0x7fabac06ac60 2023-08-27T11:28:53.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.261+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac0c9530 con 0x7fabac06ac60 2023-08-27T11:28:53.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.262+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac0b2c60 con 0x7fabac06ac60 2023-08-27T11:28:53.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.263+0000 7fab93fff700 1 -- 172.21.15.45:0/1603354564 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fab9c013b50 con 0x7fabac06ac60 2023-08-27T11:28:53.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.263+0000 7fab93fff700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fab9806a170 0x7fab9806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.263+0000 7fab93fff700 1 -- 172.21.15.45:0/1603354564 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fab9c088820 con 0x7fabac06ac60 2023-08-27T11:28:53.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.263+0000 7fabaaffd700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fab9806a170 0x7fab9806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.264+0000 7fabaaffd700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fab9806a170 0x7fab9806c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faba0009c10 tx=0x7faba0008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.268+0000 7fab93fff700 1 -- 172.21.15.45:0/1603354564 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fab9c053e40 con 0x7fabac06ac60 2023-08-27T11:28:53.451 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:53.450+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:53.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.516+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabac072780 con 0x7fabac06ac60 2023-08-27T11:28:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.517+0000 7fab93fff700 1 -- 172.21.15.45:0/1603354564 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4874 (secure 0 0 0) 0x7fab9c057460 con 0x7fabac06ac60 2023-08-27T11:28:53.518 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:53.518 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7402,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2571849064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2571849064},{"type":"v1","addr":"172.21.15.135:6836","nonce":2571849064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/509114843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":509114843},{"type":"v1","addr":"172.21.15.135:6837","nonce":509114843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/528899713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":528899713},{"type":"v1","addr":"172.21.15.45:6833","nonce":528899713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.511894+0000","modified":"2023-08-27T11:28:53.106978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8019":{"gid":8019,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/343785163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":343785163},{"type":"v1","addr":"172.21.15.45:6835","nonce":343785163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:28:53.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fab9806a170 msgr2=0x7fab9806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fab9806a170 0x7fab9806c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faba0009c10 tx=0x7faba0008030 comp rx=0 tx=0).stop 2023-08-27T11:28:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 msgr2=0x7fabac0c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 0x7fabac0c7840 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e870 tx=0x7fab9c00eb80 comp rx=0 tx=0).stop 2023-08-27T11:28:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 shutdown_connections 2023-08-27T11:28:53.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7fab9806a170 0x7fab9806c630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabac06ac60 0x7fabac0c7840 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fabac158cb0 0x7fabac15ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 --2- 172.21.15.45:0/1603354564 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fabac06d3a0 0x7fabac1552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 >> 172.21.15.45:0/1603354564 conn(0x7fabac0cfd60 msgr2=0x7fabac06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.519+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 shutdown_connections 2023-08-27T11:28:53.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.520+0000 7fabb1f53700 1 -- 172.21.15.45:0/1603354564 wait complete. 2023-08-27T11:28:53.523 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 49 2023-08-27T11:28:53.533 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:28:53.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.665+0000 7f491382b700 1 Processor -- start 2023-08-27T11:28:53.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.666+0000 7f491382b700 1 -- start start 2023-08-27T11:28:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.666+0000 7f491382b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f490c0ca620 0x7f490c0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.666+0000 7f491382b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f490c0af200 0x7f490c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.666+0000 7f491382b700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c0ce340 con 0x7f490c0caa00 2023-08-27T11:28:53.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.666+0000 7f491382b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c0ceec0 con 0x7f490c0af200 2023-08-27T11:28:53.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.666+0000 7f491382b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c15d7c0 con 0x7f490c0ca620 2023-08-27T11:28:53.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.666+0000 7f4910dc6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f490c0af200 0x7f490c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4910dc6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f490c0af200 0x7f490c0ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60408/0 (socket says 172.21.15.45:60408) 2023-08-27T11:28:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4910dc6700 1 -- 172.21.15.45:0/65669384 learned_addr learned my addr 172.21.15.45:0/65669384 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f49115c7700 1 --2- 172.21.15.45:0/65669384 >> v2:172.21.15.135:3300/0 conn(0x7f490c0ca620 0x7f490c0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4910dc6700 1 -- 172.21.15.45:0/65669384 >> v2:172.21.15.135:3300/0 conn(0x7f490c0ca620 msgr2=0x7f490c0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4910dc6700 1 --2- 172.21.15.45:0/65669384 >> v2:172.21.15.135:3300/0 conn(0x7f490c0ca620 0x7f490c0cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4910dc6700 1 -- 172.21.15.45:0/65669384 >> v1:172.21.15.135:6789/0 conn(0x7f490c0caa00 legacy=0x7f490c0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4910dc6700 1 -- 172.21.15.45:0/65669384 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0cc570 con 0x7f490c0af200 2023-08-27T11:28:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4910dc6700 1 --2- 172.21.15.45:0/65669384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0af200 0x7f490c0ac8b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f490c069790 tx=0x7f48fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4b0333c654f3de server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4903fff700 1 -- 172.21.15.45:0/65669384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc004230 con 0x7f490c0af200 2023-08-27T11:28:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4903fff700 1 -- 172.21.15.45:0/65669384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc004390 con 0x7f490c0af200 2023-08-27T11:28:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.667+0000 7f4903fff700 1 -- 172.21.15.45:0/65669384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0036a0 con 0x7f490c0af200 2023-08-27T11:28:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.668+0000 7f491382b700 1 -- 172.21.15.45:0/65669384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0af200 msgr2=0x7f490c0ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.668+0000 7f491382b700 1 --2- 172.21.15.45:0/65669384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0af200 0x7f490c0ac8b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f490c069790 tx=0x7f48fc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.668+0000 7f491382b700 1 -- 172.21.15.45:0/65669384 shutdown_connections 2023-08-27T11:28:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.668+0000 7f491382b700 1 --2- 172.21.15.45:0/65669384 >> v2:172.21.15.135:3300/0 conn(0x7f490c0ca620 0x7f490c0cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.668+0000 7f491382b700 1 --2- 172.21.15.45:0/65669384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0af200 0x7f490c0ac8b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.668+0000 7f491382b700 1 -- 172.21.15.45:0/65669384 >> 172.21.15.45:0/65669384 conn(0x7f490c0cc7e0 msgr2=0x7f490c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.668+0000 7f491382b700 1 -- 172.21.15.45:0/65669384 shutdown_connections 2023-08-27T11:28:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.668+0000 7f491382b700 1 -- 172.21.15.45:0/65669384 wait complete. 2023-08-27T11:28:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.669+0000 7f491382b700 1 Processor -- start 2023-08-27T11:28:53.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.669+0000 7f491382b700 1 -- start start 2023-08-27T11:28:53.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f491382b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f490c0af200 0x7f490c0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f491382b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0ca620 0x7f490c14b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f491382b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f490c0caa00 0x7f490c0b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f491382b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c14b970 con 0x7f490c0ca620 2023-08-27T11:28:53.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f491382b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0b58f0 con 0x7f490c0caa00 2023-08-27T11:28:53.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f491382b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0c49d0 con 0x7f490c0af200 2023-08-27T11:28:53.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4910dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0ca620 0x7f490c14b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4910dc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0ca620 0x7f490c14b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60414/0 (socket says 172.21.15.45:60414) 2023-08-27T11:28:53.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4910dc6700 1 -- 172.21.15.45:0/2743716925 learned_addr learned my addr 172.21.15.45:0/2743716925 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:53.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4911dc8700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f490c0caa00 0x7f490c0b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f49115c7700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f490c0af200 0x7f490c0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4910dc6700 1 -- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f490c0af200 msgr2=0x7f490c0b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4910dc6700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f490c0af200 0x7f490c0b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4910dc6700 1 -- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f490c0caa00 msgr2=0x7f490c0b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4910dc6700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f490c0caa00 0x7f490c0b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4910dc6700 1 -- 172.21.15.45:0/2743716925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc009550 con 0x7f490c0ca620 2023-08-27T11:28:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.670+0000 7f4910dc6700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0ca620 0x7f490c14b430 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009130 tx=0x7f48fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.671+0000 7f4901ffb700 1 -- 172.21.15.45:0/2743716925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc003c90 con 0x7f490c0ca620 2023-08-27T11:28:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.671+0000 7f4901ffb700 1 -- 172.21.15.45:0/2743716925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc01f070 con 0x7f490c0ca620 2023-08-27T11:28:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.671+0000 7f4901ffb700 1 -- 172.21.15.45:0/2743716925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011630 con 0x7f490c0ca620 2023-08-27T11:28:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.671+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c0c4cb0 con 0x7f490c0ca620 2023-08-27T11:28:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.671+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c0c5270 con 0x7f490c0ca620 2023-08-27T11:28:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.672+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c0aecb0 con 0x7f490c0ca620 2023-08-27T11:28:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.672+0000 7f4901ffb700 1 -- 172.21.15.45:0/2743716925 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f48fc021030 con 0x7f490c0ca620 2023-08-27T11:28:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.673+0000 7f4901ffb700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f48f006a170 0x7f48f006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.673+0000 7f4901ffb700 1 -- 172.21.15.45:0/2743716925 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f48fc089c60 con 0x7f490c0ca620 2023-08-27T11:28:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.673+0000 7f49115c7700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f48f006a170 0x7f48f006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.673+0000 7f49115c7700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f48f006a170 0x7f48f006c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f490800ab60 tx=0x7f4908008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.677+0000 7f4901ffb700 1 -- 172.21.15.45:0/2743716925 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f48fc055280 con 0x7f490c0ca620 2023-08-27T11:28:53.887 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.886+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f490c0cbce0 con 0x7f490c0ca620 2023-08-27T11:28:53.888 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.887+0000 7f4901ffb700 1 -- 172.21.15.45:0/2743716925 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+18588 (secure 0 0 0) 0x7f48fc0588a0 con 0x7f490c0ca620 2023-08-27T11:28:53.889 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:53.889 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:28:51.075516+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.834012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.480260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:28:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f48f006a170 msgr2=0x7f48f006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f48f006a170 0x7f48f006c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f490800ab60 tx=0x7f4908008040 comp rx=0 tx=0).stop 2023-08-27T11:28:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0ca620 msgr2=0x7f490c14b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0ca620 0x7f490c14b430 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009130 tx=0x7f48fc004750 comp rx=0 tx=0).stop 2023-08-27T11:28:53.892 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 shutdown_connections 2023-08-27T11:28:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f48f006a170 0x7f48f006c630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f490c0caa00 0x7f490c0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f490c0af200 0x7f490c0b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 --2- 172.21.15.45:0/2743716925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0ca620 0x7f490c14b430 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 >> 172.21.15.45:0/2743716925 conn(0x7f490c0cc7e0 msgr2=0x7f490c0ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 shutdown_connections 2023-08-27T11:28:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:53.890+0000 7f491382b700 1 -- 172.21.15.45:0/2743716925 wait complete. 2023-08-27T11:28:53.903 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-27T11:28:53.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:54.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.032+0000 7ff1ad154700 1 Processor -- start 2023-08-27T11:28:54.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.032+0000 7ff1ad154700 1 -- start start 2023-08-27T11:28:54.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1ad154700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1a80d1a20 0x7ff1a80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1ad154700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff1a80cd9b0 0x7ff1a80b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1ad154700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a80ada50 con 0x7ff1a815b120 2023-08-27T11:28:54.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1ad154700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a80ccb40 con 0x7ff1a80d1a20 2023-08-27T11:28:54.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1ad154700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a80cde20 con 0x7ff1a80cd9b0 2023-08-27T11:28:54.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a659c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1a80d1a20 0x7ff1a80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a6d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff1a815b120 0x7ff1a80cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47196/0 (socket says 172.21.15.45:47196) 2023-08-27T11:28:54.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a6d9d700 1 -- 172.21.15.45:0/1565572871 learned_addr learned my addr 172.21.15.45:0/1565572871 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:54.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a759e700 1 --2- 172.21.15.45:0/1565572871 >> v2:172.21.15.135:3300/0 conn(0x7ff1a80cd9b0 0x7ff1a80b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a659c700 1 -- 172.21.15.45:0/1565572871 >> v2:172.21.15.135:3300/0 conn(0x7ff1a80cd9b0 msgr2=0x7ff1a80b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:54.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a659c700 1 --2- 172.21.15.45:0/1565572871 >> v2:172.21.15.135:3300/0 conn(0x7ff1a80cd9b0 0x7ff1a80b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a659c700 1 -- 172.21.15.45:0/1565572871 >> v1:172.21.15.45:6789/0 conn(0x7ff1a815b120 legacy=0x7ff1a80cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a659c700 1 -- 172.21.15.45:0/1565572871 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a80ccd60 con 0x7ff1a80d1a20 2023-08-27T11:28:54.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1565572871 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172887667 0 0) 0x7ff1a80ada50 con 0x7ff1a815b120 2023-08-27T11:28:54.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.033+0000 7ff1a659c700 1 --2- 172.21.15.45:0/1565572871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a80d1a20 0x7ff1a80cd470 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff1a809a870 tx=0x7ff198009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965b6eb340164029 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1565572871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198011830 con 0x7ff1a80d1a20 2023-08-27T11:28:54.045 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1565572871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198011990 con 0x7ff1a80d1a20 2023-08-27T11:28:54.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1a5d9b700 1 -- 172.21.15.45:0/1565572871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198011d20 con 0x7ff1a80d1a20 2023-08-27T11:28:54.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1ad154700 1 -- 172.21.15.45:0/1565572871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a80d1a20 msgr2=0x7ff1a80cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1ad154700 1 --2- 172.21.15.45:0/1565572871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a80d1a20 0x7ff1a80cd470 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff1a809a870 tx=0x7ff198009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:54.046 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1ad154700 1 -- 172.21.15.45:0/1565572871 shutdown_connections 2023-08-27T11:28:54.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1ad154700 1 --2- 172.21.15.45:0/1565572871 >> v2:172.21.15.135:3300/0 conn(0x7ff1a80cd9b0 0x7ff1a80b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1ad154700 1 --2- 172.21.15.45:0/1565572871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a80d1a20 0x7ff1a80cd470 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1ad154700 1 -- 172.21.15.45:0/1565572871 >> 172.21.15.45:0/1565572871 conn(0x7ff1a809a160 msgr2=0x7ff1a80d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1ad154700 1 -- 172.21.15.45:0/1565572871 shutdown_connections 2023-08-27T11:28:54.047 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.034+0000 7ff1ad154700 1 -- 172.21.15.45:0/1565572871 wait complete. 2023-08-27T11:28:54.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.036+0000 7ff1ad154700 1 Processor -- start 2023-08-27T11:28:54.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.036+0000 7ff1ad154700 1 -- start start 2023-08-27T11:28:54.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1ad154700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a80cd9b0 0x7ff1a815c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1ad154700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a80d1a20 0x7ff1a8157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.048 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1ad154700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a815b120 0x7ff1a8150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1ad154700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a815d960 con 0x7ff1a815b120 2023-08-27T11:28:54.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1ad154700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a815cd50 con 0x7ff1a80cd9b0 2023-08-27T11:28:54.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1ad154700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a815d030 con 0x7ff1a80d1a20 2023-08-27T11:28:54.049 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1a759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a815b120 0x7ff1a8150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1a759e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a815b120 0x7ff1a8150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60430/0 (socket says 172.21.15.45:60430) 2023-08-27T11:28:54.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1a759e700 1 -- 172.21.15.45:0/71476166 learned_addr learned my addr 172.21.15.45:0/71476166 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:54.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1a6d9d700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a80cd9b0 0x7ff1a815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.037+0000 7ff1a659c700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a80d1a20 0x7ff1a8157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.050 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff1a759e700 1 -- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a80d1a20 msgr2=0x7ff1a8157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff1a759e700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a80d1a20 0x7ff1a8157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff1a759e700 1 -- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a80cd9b0 msgr2=0x7ff1a815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff1a759e700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a80cd9b0 0x7ff1a815c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.051 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff1a759e700 1 -- 172.21.15.45:0/71476166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980094a0 con 0x7ff1a815b120 2023-08-27T11:28:54.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff1a759e700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a815b120 0x7ff1a8150b10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000efc0 tx=0x7ff1a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff18f7fe700 1 -- 172.21.15.45:0/71476166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0009ea0 con 0x7ff1a815b120 2023-08-27T11:28:54.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff18f7fe700 1 -- 172.21.15.45:0/71476166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a000f040 con 0x7ff1a815b120 2023-08-27T11:28:54.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff18f7fe700 1 -- 172.21.15.45:0/71476166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000ced0 con 0x7ff1a815b120 2023-08-27T11:28:54.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a815d310 con 0x7ff1a815b120 2023-08-27T11:28:54.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.038+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a8155d50 con 0x7ff1a815b120 2023-08-27T11:28:54.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.039+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a809edd0 con 0x7ff1a815b120 2023-08-27T11:28:54.053 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.040+0000 7ff18f7fe700 1 -- 172.21.15.45:0/71476166 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7ff1a0014480 con 0x7ff1a815b120 2023-08-27T11:28:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.040+0000 7ff18f7fe700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff19406a170 0x7ff19406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.040+0000 7ff18f7fe700 1 -- 172.21.15.45:0/71476166 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff1a0089030 con 0x7ff1a815b120 2023-08-27T11:28:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.040+0000 7ff1a6d9d700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff19406a170 0x7ff19406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.041+0000 7ff1a6d9d700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff19406a170 0x7ff19406c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff19c000f60 tx=0x7ff19c007e70 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.055 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.045+0000 7ff18f7fe700 1 -- 172.21.15.45:0/71476166 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff1a00544b0 con 0x7ff1a815b120 2023-08-27T11:28:54.244 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.243+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 --> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1a80b0b40 con 0x7ff19406a170 2023-08-27T11:28:54.254 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.253+0000 7ff18f7fe700 1 -- 172.21.15.45:0/71476166 <== mgr.7156 v2:172.21.15.135:6832/3759624346 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306609 (secure 0 0 0) 0x7ff19c001290 con 0x7ff19406a170 2023-08-27T11:28:54.257 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:54.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.256+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff19406a170 msgr2=0x7ff19406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.256+0000 7ff1ad154700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff19406a170 0x7ff19406c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff19c000f60 tx=0x7ff19c007e70 comp rx=0 tx=0).stop 2023-08-27T11:28:54.260 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.256+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a815b120 msgr2=0x7ff1a8150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.256+0000 7ff1ad154700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a815b120 0x7ff1a8150b10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000efc0 tx=0x7ff1a000c3a0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.257+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 shutdown_connections 2023-08-27T11:28:54.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.257+0000 7ff1ad154700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff19406a170 0x7ff19406c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.257+0000 7ff1ad154700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a815b120 0x7ff1a8150b10 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.261 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.257+0000 7ff1ad154700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a80cd9b0 0x7ff1a815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.257+0000 7ff1ad154700 1 --2- 172.21.15.45:0/71476166 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1a80d1a20 0x7ff1a8157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.257+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 >> 172.21.15.45:0/71476166 conn(0x7ff1a809a160 msgr2=0x7ff1a80af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.257+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 shutdown_connections 2023-08-27T11:28:54.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.257+0000 7ff1ad154700 1 -- 172.21.15.45:0/71476166 wait complete. 2023-08-27T11:28:54.263 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:54.270 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:28:53.707854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":761,"num_read_kb":1467,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5183,"ondisk_log_size":5183,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278112,"kb_used_data":12016,"kb_used_omap":13,"kb_used_meta":265970,"kb_avail":754696608,"statfs":{"total":773094113280,"available":772809326592,"internally_reserved":0,"allocated":12304384,"data_stored":7020288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":272353368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.637638"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085340+0000","last_change":"2023-08-27T11:28:49.085340+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085340+0000","last_undegraded":"2023-08-27T11:28:49.085340+0000","last_fullsized":"2023-08-27T11:28:49.085340+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.092601+0000","last_change":"2023-08-27T11:28:49.092601+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.092601+0000","last_undegraded":"2023-08-27T11:28:49.092601+0000","last_fullsized":"2023-08-27T11:28:49.092601+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079160+0000","last_change":"2023-08-27T11:28:49.079160+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079160+0000","last_undegraded":"2023-08-27T11:28:49.079160+0000","last_fullsized":"2023-08-27T11:28:49.079160+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080168+0000","last_change":"2023-08-27T11:28:49.080168+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080168+0000","last_undegraded":"2023-08-27T11:28:49.080168+0000","last_fullsized":"2023-08-27T11:28:49.080168+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085503+0000","last_change":"2023-08-27T11:28:49.085503+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085503+0000","last_undegraded":"2023-08-27T11:28:49.085503+0000","last_fullsized":"2023-08-27T11:28:49.085503+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.091634+0000","last_change":"2023-08-27T11:28:49.091634+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.091634+0000","last_undegraded":"2023-08-27T11:28:49.091634+0000","last_fullsized":"2023-08-27T11:28:49.091634+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089205+0000","last_change":"2023-08-27T11:28:49.089205+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089205+0000","last_undegraded":"2023-08-27T11:28:49.089205+0000","last_fullsized":"2023-08-27T11:28:49.089205+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.091503+0000","last_change":"2023-08-27T11:28:49.091503+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.091503+0000","last_undegraded":"2023-08-27T11:28:49.091503+0000","last_fullsized":"2023-08-27T11:28:49.091503+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083331+0000","last_change":"2023-08-27T11:28:49.083331+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083331+0000","last_undegraded":"2023-08-27T11:28:49.083331+0000","last_fullsized":"2023-08-27T11:28:49.083331+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084616+0000","last_change":"2023-08-27T11:28:49.084616+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084616+0000","last_undegraded":"2023-08-27T11:28:49.084616+0000","last_fullsized":"2023-08-27T11:28:49.084616+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090012+0000","last_change":"2023-08-27T11:28:49.090012+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090012+0000","last_undegraded":"2023-08-27T11:28:49.090012+0000","last_fullsized":"2023-08-27T11:28:49.090012+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083870+0000","last_change":"2023-08-27T11:28:49.083870+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083870+0000","last_undegraded":"2023-08-27T11:28:49.083870+0000","last_fullsized":"2023-08-27T11:28:49.083870+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079983+0000","last_change":"2023-08-27T11:28:49.079983+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079983+0000","last_undegraded":"2023-08-27T11:28:49.079983+0000","last_fullsized":"2023-08-27T11:28:49.079983+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080749+0000","last_change":"2023-08-27T11:28:49.080749+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080749+0000","last_undegraded":"2023-08-27T11:28:49.080749+0000","last_fullsized":"2023-08-27T11:28:49.080749+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080853+0000","last_change":"2023-08-27T11:28:49.080853+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080853+0000","last_undegraded":"2023-08-27T11:28:49.080853+0000","last_fullsized":"2023-08-27T11:28:49.080853+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090451+0000","last_change":"2023-08-27T11:28:49.090451+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090451+0000","last_undegraded":"2023-08-27T11:28:49.090451+0000","last_fullsized":"2023-08-27T11:28:49.090451+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090009+0000","last_change":"2023-08-27T11:28:49.090009+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090009+0000","last_undegraded":"2023-08-27T11:28:49.090009+0000","last_fullsized":"2023-08-27T11:28:49.090009+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080580+0000","last_change":"2023-08-27T11:28:49.080580+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080580+0000","last_undegraded":"2023-08-27T11:28:49.080580+0000","last_fullsized":"2023-08-27T11:28:49.080580+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079857+0000","last_change":"2023-08-27T11:28:49.079857+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079857+0000","last_undegraded":"2023-08-27T11:28:49.079857+0000","last_fullsized":"2023-08-27T11:28:49.079857+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083451+0000","last_change":"2023-08-27T11:28:49.083451+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083451+0000","last_undegraded":"2023-08-27T11:28:49.083451+0000","last_fullsized":"2023-08-27T11:28:49.083451+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080787+0000","last_change":"2023-08-27T11:28:49.080787+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080787+0000","last_undegraded":"2023-08-27T11:28:49.080787+0000","last_fullsized":"2023-08-27T11:28:49.080787+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.091564+0000","last_change":"2023-08-27T11:28:49.091564+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.091564+0000","last_undegraded":"2023-08-27T11:28:49.091564+0000","last_fullsized":"2023-08-27T11:28:49.091564+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084393+0000","last_change":"2023-08-27T11:28:49.084393+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084393+0000","last_undegraded":"2023-08-27T11:28:49.084393+0000","last_fullsized":"2023-08-27T11:28:49.084393+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090030+0000","last_change":"2023-08-27T11:28:49.090030+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090030+0000","last_undegraded":"2023-08-27T11:28:49.090030+0000","last_fullsized":"2023-08-27T11:28:49.090030+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.086685+0000","last_change":"2023-08-27T11:28:49.086685+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.086685+0000","last_undegraded":"2023-08-27T11:28:49.086685+0000","last_fullsized":"2023-08-27T11:28:49.086685+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084953+0000","last_change":"2023-08-27T11:28:49.084953+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084953+0000","last_undegraded":"2023-08-27T11:28:49.084953+0000","last_fullsized":"2023-08-27T11:28:49.084953+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089211+0000","last_change":"2023-08-27T11:28:49.089211+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089211+0000","last_undegraded":"2023-08-27T11:28:49.089211+0000","last_fullsized":"2023-08-27T11:28:49.089211+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085408+0000","last_change":"2023-08-27T11:28:49.085408+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085408+0000","last_undegraded":"2023-08-27T11:28:49.085408+0000","last_fullsized":"2023-08-27T11:28:49.085408+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084163+0000","last_change":"2023-08-27T11:28:49.084163+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084163+0000","last_undegraded":"2023-08-27T11:28:49.084163+0000","last_fullsized":"2023-08-27T11:28:49.084163+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080300+0000","last_change":"2023-08-27T11:28:49.080300+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080300+0000","last_undegraded":"2023-08-27T11:28:49.080300+0000","last_fullsized":"2023-08-27T11:28:49.080300+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080473+0000","last_change":"2023-08-27T11:28:49.080473+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080473+0000","last_undegraded":"2023-08-27T11:28:49.080473+0000","last_fullsized":"2023-08-27T11:28:49.080473+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.081947+0000","last_change":"2023-08-27T11:28:49.081947+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.081947+0000","last_undegraded":"2023-08-27T11:28:49.081947+0000","last_fullsized":"2023-08-27T11:28:49.081947+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080179+0000","last_change":"2023-08-27T11:28:49.080179+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080179+0000","last_undegraded":"2023-08-27T11:28:49.080179+0000","last_fullsized":"2023-08-27T11:28:49.080179+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103625+0000","last_change":"2023-08-27T11:28:49.093411+0000","last_active":"2023-08-27T11:28:49.103625+0000","last_peered":"2023-08-27T11:28:49.103625+0000","last_clean":"2023-08-27T11:28:49.103625+0000","last_became_active":"2023-08-27T11:28:49.092946+0000","last_became_peered":"2023-08-27T11:28:49.092946+0000","last_unstale":"2023-08-27T11:28:49.103625+0000","last_undegraded":"2023-08-27T11:28:49.103625+0000","last_fullsized":"2023-08-27T11:28:49.103625+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:54.413756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079935+0000","last_change":"2023-08-27T11:28:49.079935+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079935+0000","last_undegraded":"2023-08-27T11:28:49.079935+0000","last_fullsized":"2023-08-27T11:28:49.079935+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097092+0000","last_change":"2023-08-27T11:28:49.095509+0000","last_active":"2023-08-27T11:28:49.097092+0000","last_peered":"2023-08-27T11:28:49.097092+0000","last_clean":"2023-08-27T11:28:49.097092+0000","last_became_active":"2023-08-27T11:28:49.095273+0000","last_became_peered":"2023-08-27T11:28:49.095273+0000","last_unstale":"2023-08-27T11:28:49.097092+0000","last_undegraded":"2023-08-27T11:28:49.097092+0000","last_fullsized":"2023-08-27T11:28:49.097092+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:27:36.936742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084149+0000","last_change":"2023-08-27T11:28:49.084149+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084149+0000","last_undegraded":"2023-08-27T11:28:49.084149+0000","last_fullsized":"2023-08-27T11:28:49.084149+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103577+0000","last_change":"2023-08-27T11:28:49.093022+0000","last_active":"2023-08-27T11:28:49.103577+0000","last_peered":"2023-08-27T11:28:49.103577+0000","last_clean":"2023-08-27T11:28:49.103577+0000","last_became_active":"2023-08-27T11:28:49.092689+0000","last_became_peered":"2023-08-27T11:28:49.092689+0000","last_unstale":"2023-08-27T11:28:49.103577+0000","last_undegraded":"2023-08-27T11:28:49.103577+0000","last_fullsized":"2023-08-27T11:28:49.103577+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:36:04.006868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082739+0000","last_change":"2023-08-27T11:28:49.082739+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082739+0000","last_undegraded":"2023-08-27T11:28:49.082739+0000","last_fullsized":"2023-08-27T11:28:49.082739+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103796+0000","last_change":"2023-08-27T11:28:49.098336+0000","last_active":"2023-08-27T11:28:49.103796+0000","last_peered":"2023-08-27T11:28:49.103796+0000","last_clean":"2023-08-27T11:28:49.103796+0000","last_became_active":"2023-08-27T11:28:49.098055+0000","last_became_peered":"2023-08-27T11:28:49.098055+0000","last_unstale":"2023-08-27T11:28:49.103796+0000","last_undegraded":"2023-08-27T11:28:49.103796+0000","last_fullsized":"2023-08-27T11:28:49.103796+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:39:49.220050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082760+0000","last_change":"2023-08-27T11:28:49.082760+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082760+0000","last_undegraded":"2023-08-27T11:28:49.082760+0000","last_fullsized":"2023-08-27T11:28:49.082760+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092434+0000","last_change":"2023-08-27T11:28:49.088759+0000","last_active":"2023-08-27T11:28:49.092434+0000","last_peered":"2023-08-27T11:28:49.092434+0000","last_clean":"2023-08-27T11:28:49.092434+0000","last_became_active":"2023-08-27T11:28:49.088495+0000","last_became_peered":"2023-08-27T11:28:49.088495+0000","last_unstale":"2023-08-27T11:28:49.092434+0000","last_undegraded":"2023-08-27T11:28:49.092434+0000","last_fullsized":"2023-08-27T11:28:49.092434+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:06.208356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089263+0000","last_change":"2023-08-27T11:28:49.089263+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089263+0000","last_undegraded":"2023-08-27T11:28:49.089263+0000","last_fullsized":"2023-08-27T11:28:49.089263+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103618+0000","last_change":"2023-08-27T11:28:49.093290+0000","last_active":"2023-08-27T11:28:49.103618+0000","last_peered":"2023-08-27T11:28:49.103618+0000","last_clean":"2023-08-27T11:28:49.103618+0000","last_became_active":"2023-08-27T11:28:49.092886+0000","last_became_peered":"2023-08-27T11:28:49.092886+0000","last_unstale":"2023-08-27T11:28:49.103618+0000","last_undegraded":"2023-08-27T11:28:49.103618+0000","last_fullsized":"2023-08-27T11:28:49.103618+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:19:27.269549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080199+0000","last_change":"2023-08-27T11:28:49.080199+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080199+0000","last_undegraded":"2023-08-27T11:28:49.080199+0000","last_fullsized":"2023-08-27T11:28:49.080199+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.094961+0000","last_change":"2023-08-27T11:28:49.093546+0000","last_active":"2023-08-27T11:28:49.094961+0000","last_peered":"2023-08-27T11:28:49.094961+0000","last_clean":"2023-08-27T11:28:49.094961+0000","last_became_active":"2023-08-27T11:28:49.093291+0000","last_became_peered":"2023-08-27T11:28:49.093291+0000","last_unstale":"2023-08-27T11:28:49.094961+0000","last_undegraded":"2023-08-27T11:28:49.094961+0000","last_fullsized":"2023-08-27T11:28:49.094961+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:47.251433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083545+0000","last_change":"2023-08-27T11:28:49.083545+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083545+0000","last_undegraded":"2023-08-27T11:28:49.083545+0000","last_fullsized":"2023-08-27T11:28:49.083545+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.090413+0000","last_change":"2023-08-27T11:28:49.087457+0000","last_active":"2023-08-27T11:28:49.090413+0000","last_peered":"2023-08-27T11:28:49.090413+0000","last_clean":"2023-08-27T11:28:49.090413+0000","last_became_active":"2023-08-27T11:28:49.087210+0000","last_became_peered":"2023-08-27T11:28:49.087210+0000","last_unstale":"2023-08-27T11:28:49.090413+0000","last_undegraded":"2023-08-27T11:28:49.090413+0000","last_fullsized":"2023-08-27T11:28:49.090413+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:50:00.915296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084549+0000","last_change":"2023-08-27T11:28:49.084549+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084549+0000","last_undegraded":"2023-08-27T11:28:49.084549+0000","last_fullsized":"2023-08-27T11:28:49.084549+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.093549+0000","last_change":"2023-08-27T11:28:49.089014+0000","last_active":"2023-08-27T11:28:49.093549+0000","last_peered":"2023-08-27T11:28:49.093549+0000","last_clean":"2023-08-27T11:28:49.093549+0000","last_became_active":"2023-08-27T11:28:49.088750+0000","last_became_peered":"2023-08-27T11:28:49.088750+0000","last_unstale":"2023-08-27T11:28:49.093549+0000","last_undegraded":"2023-08-27T11:28:49.093549+0000","last_fullsized":"2023-08-27T11:28:49.093549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:12.555970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082603+0000","last_change":"2023-08-27T11:28:49.082603+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082603+0000","last_undegraded":"2023-08-27T11:28:49.082603+0000","last_fullsized":"2023-08-27T11:28:49.082603+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097693+0000","last_change":"2023-08-27T11:28:49.093294+0000","last_active":"2023-08-27T11:28:49.097693+0000","last_peered":"2023-08-27T11:28:49.097693+0000","last_clean":"2023-08-27T11:28:49.097693+0000","last_became_active":"2023-08-27T11:28:49.092967+0000","last_became_peered":"2023-08-27T11:28:49.092967+0000","last_unstale":"2023-08-27T11:28:49.097693+0000","last_undegraded":"2023-08-27T11:28:49.097693+0000","last_fullsized":"2023-08-27T11:28:49.097693+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:30.559346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085066+0000","last_change":"2023-08-27T11:28:49.085066+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085066+0000","last_undegraded":"2023-08-27T11:28:49.085066+0000","last_fullsized":"2023-08-27T11:28:49.085066+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097072+0000","last_change":"2023-08-27T11:28:49.095274+0000","last_active":"2023-08-27T11:28:49.097072+0000","last_peered":"2023-08-27T11:28:49.097072+0000","last_clean":"2023-08-27T11:28:49.097072+0000","last_became_active":"2023-08-27T11:28:49.095022+0000","last_became_peered":"2023-08-27T11:28:49.095022+0000","last_unstale":"2023-08-27T11:28:49.097072+0000","last_undegraded":"2023-08-27T11:28:49.097072+0000","last_fullsized":"2023-08-27T11:28:49.097072+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:20.172740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090393+0000","last_change":"2023-08-27T11:28:49.090393+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090393+0000","last_undegraded":"2023-08-27T11:28:49.090393+0000","last_fullsized":"2023-08-27T11:28:49.090393+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.091984+0000","last_change":"2023-08-27T11:28:49.087250+0000","last_active":"2023-08-27T11:28:49.091984+0000","last_peered":"2023-08-27T11:28:49.091984+0000","last_clean":"2023-08-27T11:28:49.091984+0000","last_became_active":"2023-08-27T11:28:49.087002+0000","last_became_peered":"2023-08-27T11:28:49.087002+0000","last_unstale":"2023-08-27T11:28:49.091984+0000","last_undegraded":"2023-08-27T11:28:49.091984+0000","last_fullsized":"2023-08-27T11:28:49.091984+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:52.621160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079943+0000","last_change":"2023-08-27T11:28:49.079943+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079943+0000","last_undegraded":"2023-08-27T11:28:49.079943+0000","last_fullsized":"2023-08-27T11:28:49.079943+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.091510+0000","last_change":"2023-08-27T11:28:49.086306+0000","last_active":"2023-08-27T11:28:49.091510+0000","last_peered":"2023-08-27T11:28:49.091510+0000","last_clean":"2023-08-27T11:28:49.091510+0000","last_became_active":"2023-08-27T11:28:49.086033+0000","last_became_peered":"2023-08-27T11:28:49.086033+0000","last_unstale":"2023-08-27T11:28:49.091510+0000","last_undegraded":"2023-08-27T11:28:49.091510+0000","last_fullsized":"2023-08-27T11:28:49.091510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:13:39.364173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080758+0000","last_change":"2023-08-27T11:28:49.080758+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080758+0000","last_undegraded":"2023-08-27T11:28:49.080758+0000","last_fullsized":"2023-08-27T11:28:49.080758+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.091231+0000","last_change":"2023-08-27T11:28:49.088909+0000","last_active":"2023-08-27T11:28:49.091231+0000","last_peered":"2023-08-27T11:28:49.091231+0000","last_clean":"2023-08-27T11:28:49.091231+0000","last_became_active":"2023-08-27T11:28:49.088497+0000","last_became_peered":"2023-08-27T11:28:49.088497+0000","last_unstale":"2023-08-27T11:28:49.091231+0000","last_undegraded":"2023-08-27T11:28:49.091231+0000","last_fullsized":"2023-08-27T11:28:49.091231+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:46.497129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082660+0000","last_change":"2023-08-27T11:28:49.082660+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082660+0000","last_undegraded":"2023-08-27T11:28:49.082660+0000","last_fullsized":"2023-08-27T11:28:49.082660+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092067+0000","last_change":"2023-08-27T11:28:49.087910+0000","last_active":"2023-08-27T11:28:49.092067+0000","last_peered":"2023-08-27T11:28:49.092067+0000","last_clean":"2023-08-27T11:28:49.092067+0000","last_became_active":"2023-08-27T11:28:49.087656+0000","last_became_peered":"2023-08-27T11:28:49.087656+0000","last_unstale":"2023-08-27T11:28:49.092067+0000","last_undegraded":"2023-08-27T11:28:49.092067+0000","last_fullsized":"2023-08-27T11:28:49.092067+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:25.771051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079616+0000","last_change":"2023-08-27T11:28:49.079616+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079616+0000","last_undegraded":"2023-08-27T11:28:49.079616+0000","last_fullsized":"2023-08-27T11:28:49.079616+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.094978+0000","last_change":"2023-08-27T11:28:49.092647+0000","last_active":"2023-08-27T11:28:49.094978+0000","last_peered":"2023-08-27T11:28:49.094978+0000","last_clean":"2023-08-27T11:28:49.094978+0000","last_became_active":"2023-08-27T11:28:49.092425+0000","last_became_peered":"2023-08-27T11:28:49.092425+0000","last_unstale":"2023-08-27T11:28:49.094978+0000","last_undegraded":"2023-08-27T11:28:49.094978+0000","last_fullsized":"2023-08-27T11:28:49.094978+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:45.451239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084132+0000","last_change":"2023-08-27T11:28:49.084132+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084132+0000","last_undegraded":"2023-08-27T11:28:49.084132+0000","last_fullsized":"2023-08-27T11:28:49.084132+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092980+0000","last_change":"2023-08-27T11:28:49.090840+0000","last_active":"2023-08-27T11:28:49.092980+0000","last_peered":"2023-08-27T11:28:49.092980+0000","last_clean":"2023-08-27T11:28:49.092980+0000","last_became_active":"2023-08-27T11:28:49.090655+0000","last_became_peered":"2023-08-27T11:28:49.090655+0000","last_unstale":"2023-08-27T11:28:49.092980+0000","last_undegraded":"2023-08-27T11:28:49.092980+0000","last_fullsized":"2023-08-27T11:28:49.092980+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:03.433230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084933+0000","last_change":"2023-08-27T11:28:49.084933+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084933+0000","last_undegraded":"2023-08-27T11:28:49.084933+0000","last_fullsized":"2023-08-27T11:28:49.084933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092990+0000","last_change":"2023-08-27T11:28:49.088886+0000","last_active":"2023-08-27T11:28:49.092990+0000","last_peered":"2023-08-27T11:28:49.092990+0000","last_clean":"2023-08-27T11:28:49.092990+0000","last_became_active":"2023-08-27T11:28:49.088618+0000","last_became_peered":"2023-08-27T11:28:49.088618+0000","last_unstale":"2023-08-27T11:28:49.092990+0000","last_undegraded":"2023-08-27T11:28:49.092990+0000","last_fullsized":"2023-08-27T11:28:49.092990+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:45:46.741280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.081075+0000","last_change":"2023-08-27T11:28:49.081075+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.081075+0000","last_undegraded":"2023-08-27T11:28:49.081075+0000","last_fullsized":"2023-08-27T11:28:49.081075+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103821+0000","last_change":"2023-08-27T11:28:49.098450+0000","last_active":"2023-08-27T11:28:49.103821+0000","last_peered":"2023-08-27T11:28:49.103821+0000","last_clean":"2023-08-27T11:28:49.103821+0000","last_became_active":"2023-08-27T11:28:49.098071+0000","last_became_peered":"2023-08-27T11:28:49.098071+0000","last_unstale":"2023-08-27T11:28:49.103821+0000","last_undegraded":"2023-08-27T11:28:49.103821+0000","last_fullsized":"2023-08-27T11:28:49.103821+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:05.557628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084983+0000","last_change":"2023-08-27T11:28:49.084983+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084983+0000","last_undegraded":"2023-08-27T11:28:49.084983+0000","last_fullsized":"2023-08-27T11:28:49.084983+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097260+0000","last_change":"2023-08-27T11:28:49.094459+0000","last_active":"2023-08-27T11:28:49.097260+0000","last_peered":"2023-08-27T11:28:49.097260+0000","last_clean":"2023-08-27T11:28:49.097260+0000","last_became_active":"2023-08-27T11:28:49.094215+0000","last_became_peered":"2023-08-27T11:28:49.094215+0000","last_unstale":"2023-08-27T11:28:49.097260+0000","last_undegraded":"2023-08-27T11:28:49.097260+0000","last_fullsized":"2023-08-27T11:28:49.097260+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:27:04.328482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.077745+0000","last_change":"2023-08-27T11:28:49.077745+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.077745+0000","last_undegraded":"2023-08-27T11:28:49.077745+0000","last_fullsized":"2023-08-27T11:28:49.077745+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.102914+0000","last_change":"2023-08-27T11:28:49.091063+0000","last_active":"2023-08-27T11:28:49.102914+0000","last_peered":"2023-08-27T11:28:49.102914+0000","last_clean":"2023-08-27T11:28:49.102914+0000","last_became_active":"2023-08-27T11:28:49.090005+0000","last_became_peered":"2023-08-27T11:28:49.090005+0000","last_unstale":"2023-08-27T11:28:49.102914+0000","last_undegraded":"2023-08-27T11:28:49.102914+0000","last_fullsized":"2023-08-27T11:28:49.102914+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:37.416680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082576+0000","last_change":"2023-08-27T11:28:49.082576+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082576+0000","last_undegraded":"2023-08-27T11:28:49.082576+0000","last_fullsized":"2023-08-27T11:28:49.082576+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092298+0000","last_change":"2023-08-27T11:28:49.089703+0000","last_active":"2023-08-27T11:28:49.092298+0000","last_peered":"2023-08-27T11:28:49.092298+0000","last_clean":"2023-08-27T11:28:49.092298+0000","last_became_active":"2023-08-27T11:28:49.089514+0000","last_became_peered":"2023-08-27T11:28:49.089514+0000","last_unstale":"2023-08-27T11:28:49.092298+0000","last_undegraded":"2023-08-27T11:28:49.092298+0000","last_fullsized":"2023-08-27T11:28:49.092298+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:01:44.852062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.077759+0000","last_change":"2023-08-27T11:28:49.077759+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.077759+0000","last_undegraded":"2023-08-27T11:28:49.077759+0000","last_fullsized":"2023-08-27T11:28:49.077759+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.089958+0000","last_change":"2023-08-27T11:28:49.087175+0000","last_active":"2023-08-27T11:28:49.089958+0000","last_peered":"2023-08-27T11:28:49.089958+0000","last_clean":"2023-08-27T11:28:49.089958+0000","last_became_active":"2023-08-27T11:28:49.086854+0000","last_became_peered":"2023-08-27T11:28:49.086854+0000","last_unstale":"2023-08-27T11:28:49.089958+0000","last_undegraded":"2023-08-27T11:28:49.089958+0000","last_fullsized":"2023-08-27T11:28:49.089958+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:19.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085540+0000","last_change":"2023-08-27T11:28:49.085540+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085540+0000","last_undegraded":"2023-08-27T11:28:49.085540+0000","last_fullsized":"2023-08-27T11:28:49.085540+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097152+0000","last_change":"2023-08-27T11:28:49.096229+0000","last_active":"2023-08-27T11:28:49.097152+0000","last_peered":"2023-08-27T11:28:49.097152+0000","last_clean":"2023-08-27T11:28:49.097152+0000","last_became_active":"2023-08-27T11:28:49.095951+0000","last_became_peered":"2023-08-27T11:28:49.095951+0000","last_unstale":"2023-08-27T11:28:49.097152+0000","last_undegraded":"2023-08-27T11:28:49.097152+0000","last_fullsized":"2023-08-27T11:28:49.097152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:37:05.774883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.086269+0000","last_change":"2023-08-27T11:28:49.086269+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.086269+0000","last_undegraded":"2023-08-27T11:28:49.086269+0000","last_fullsized":"2023-08-27T11:28:49.086269+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103699+0000","last_change":"2023-08-27T11:28:49.095920+0000","last_active":"2023-08-27T11:28:49.103699+0000","last_peered":"2023-08-27T11:28:49.103699+0000","last_clean":"2023-08-27T11:28:49.103699+0000","last_became_active":"2023-08-27T11:28:49.095652+0000","last_became_peered":"2023-08-27T11:28:49.095652+0000","last_unstale":"2023-08-27T11:28:49.103699+0000","last_undegraded":"2023-08-27T11:28:49.103699+0000","last_fullsized":"2023-08-27T11:28:49.103699+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:59.514247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":193,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.078437+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:49.078437+0000","last_peered":"2023-08-27T11:28:49.078437+0000","last_clean":"2023-08-27T11:28:49.078437+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:49.078437+0000","last_undegraded":"2023-08-27T11:28:49.078437+0000","last_fullsized":"2023-08-27T11:28:49.078437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:15:19.298171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.086169+0000","last_change":"2023-08-27T11:28:49.086169+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.086169+0000","last_undegraded":"2023-08-27T11:28:49.086169+0000","last_fullsized":"2023-08-27T11:28:49.086169+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.094004+0000","last_change":"2023-08-27T11:28:49.092776+0000","last_active":"2023-08-27T11:28:49.094004+0000","last_peered":"2023-08-27T11:28:49.094004+0000","last_clean":"2023-08-27T11:28:49.094004+0000","last_became_active":"2023-08-27T11:28:49.092503+0000","last_became_peered":"2023-08-27T11:28:49.092503+0000","last_unstale":"2023-08-27T11:28:49.094004+0000","last_undegraded":"2023-08-27T11:28:49.094004+0000","last_fullsized":"2023-08-27T11:28:49.094004+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:41.137920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":195,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.084685+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:49.084685+0000","last_peered":"2023-08-27T11:28:49.084685+0000","last_clean":"2023-08-27T11:28:49.084685+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:49.084685+0000","last_undegraded":"2023-08-27T11:28:49.084685+0000","last_fullsized":"2023-08-27T11:28:49.084685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:17:55.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084016+0000","last_change":"2023-08-27T11:28:49.084016+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084016+0000","last_undegraded":"2023-08-27T11:28:49.084016+0000","last_fullsized":"2023-08-27T11:28:49.084016+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092250+0000","last_change":"2023-08-27T11:28:49.088492+0000","last_active":"2023-08-27T11:28:49.092250+0000","last_peered":"2023-08-27T11:28:49.092250+0000","last_clean":"2023-08-27T11:28:49.092250+0000","last_became_active":"2023-08-27T11:28:49.088156+0000","last_became_peered":"2023-08-27T11:28:49.088156+0000","last_unstale":"2023-08-27T11:28:49.092250+0000","last_undegraded":"2023-08-27T11:28:49.092250+0000","last_fullsized":"2023-08-27T11:28:49.092250+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:11:26.876964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":183,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.075434+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:49.075434+0000","last_peered":"2023-08-27T11:28:49.075434+0000","last_clean":"2023-08-27T11:28:49.075434+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:49.075434+0000","last_undegraded":"2023-08-27T11:28:49.075434+0000","last_fullsized":"2023-08-27T11:28:49.075434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:20:55.742674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079769+0000","last_change":"2023-08-27T11:28:49.079769+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079769+0000","last_undegraded":"2023-08-27T11:28:49.079769+0000","last_fullsized":"2023-08-27T11:28:49.079769+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.098655+0000","last_change":"2023-08-27T11:28:49.098322+0000","last_active":"2023-08-27T11:28:49.098655+0000","last_peered":"2023-08-27T11:28:49.098655+0000","last_clean":"2023-08-27T11:28:49.098655+0000","last_became_active":"2023-08-27T11:28:49.098035+0000","last_became_peered":"2023-08-27T11:28:49.098035+0000","last_unstale":"2023-08-27T11:28:49.098655+0000","last_undegraded":"2023-08-27T11:28:49.098655+0000","last_fullsized":"2023-08-27T11:28:49.098655+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:21:27.504700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":195,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.083440+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:49.083440+0000","last_peered":"2023-08-27T11:28:49.083440+0000","last_clean":"2023-08-27T11:28:49.083440+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:49.083440+0000","last_undegraded":"2023-08-27T11:28:49.083440+0000","last_fullsized":"2023-08-27T11:28:49.083440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:57:44.263129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"113'5183","reported_seq":5642,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.700641+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:49.700641+0000","last_peered":"2023-08-27T11:28:49.700641+0000","last_clean":"2023-08-27T11:28:49.700641+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:49.700641+0000","last_undegraded":"2023-08-27T11:28:49.700641+0000","last_fullsized":"2023-08-27T11:28:49.700641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":5183,"log_dups_size":0,"ondisk_log_size":5183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:49.306768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":761,"num_read_kb":1467,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089647+0000","last_change":"2023-08-27T11:28:49.089647+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089647+0000","last_undegraded":"2023-08-27T11:28:49.089647+0000","last_fullsized":"2023-08-27T11:28:49.089647+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103818+0000","last_change":"2023-08-27T11:28:49.098393+0000","last_active":"2023-08-27T11:28:49.103818+0000","last_peered":"2023-08-27T11:28:49.103818+0000","last_clean":"2023-08-27T11:28:49.103818+0000","last_became_active":"2023-08-27T11:28:49.098104+0000","last_became_peered":"2023-08-27T11:28:49.098104+0000","last_unstale":"2023-08-27T11:28:49.103818+0000","last_undegraded":"2023-08-27T11:28:49.103818+0000","last_fullsized":"2023-08-27T11:28:49.103818+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:14.479332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":195,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092502+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:49.092502+0000","last_peered":"2023-08-27T11:28:49.092502+0000","last_clean":"2023-08-27T11:28:49.092502+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:49.092502+0000","last_undegraded":"2023-08-27T11:28:49.092502+0000","last_fullsized":"2023-08-27T11:28:49.092502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:06:07.246346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.078553+0000","last_change":"2023-08-27T11:28:49.078553+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.078553+0000","last_undegraded":"2023-08-27T11:28:49.078553+0000","last_fullsized":"2023-08-27T11:28:49.078553+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092559+0000","last_change":"2023-08-27T11:28:49.090592+0000","last_active":"2023-08-27T11:28:49.092559+0000","last_peered":"2023-08-27T11:28:49.092559+0000","last_clean":"2023-08-27T11:28:49.092559+0000","last_became_active":"2023-08-27T11:28:49.090309+0000","last_became_peered":"2023-08-27T11:28:49.090309+0000","last_unstale":"2023-08-27T11:28:49.092559+0000","last_undegraded":"2023-08-27T11:28:49.092559+0000","last_fullsized":"2023-08-27T11:28:49.092559+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:29:21.154979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":194,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092856+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:49.092856+0000","last_peered":"2023-08-27T11:28:49.092856+0000","last_clean":"2023-08-27T11:28:49.092856+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:49.092856+0000","last_undegraded":"2023-08-27T11:28:49.092856+0000","last_fullsized":"2023-08-27T11:28:49.092856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:29:19.522512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080372+0000","last_change":"2023-08-27T11:28:49.080372+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080372+0000","last_undegraded":"2023-08-27T11:28:49.080372+0000","last_fullsized":"2023-08-27T11:28:49.080372+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092543+0000","last_change":"2023-08-27T11:28:49.089867+0000","last_active":"2023-08-27T11:28:49.092543+0000","last_peered":"2023-08-27T11:28:49.092543+0000","last_clean":"2023-08-27T11:28:49.092543+0000","last_became_active":"2023-08-27T11:28:49.089621+0000","last_became_peered":"2023-08-27T11:28:49.089621+0000","last_unstale":"2023-08-27T11:28:49.092543+0000","last_undegraded":"2023-08-27T11:28:49.092543+0000","last_fullsized":"2023-08-27T11:28:49.092543+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:03.921396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.078190+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:49.078190+0000","last_peered":"2023-08-27T11:28:49.078190+0000","last_clean":"2023-08-27T11:28:49.078190+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:49.078190+0000","last_undegraded":"2023-08-27T11:28:49.078190+0000","last_fullsized":"2023-08-27T11:28:49.078190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:30.750930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079866+0000","last_change":"2023-08-27T11:28:49.079866+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079866+0000","last_undegraded":"2023-08-27T11:28:49.079866+0000","last_fullsized":"2023-08-27T11:28:49.079866+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.102888+0000","last_change":"2023-08-27T11:28:49.089632+0000","last_active":"2023-08-27T11:28:49.102888+0000","last_peered":"2023-08-27T11:28:49.102888+0000","last_clean":"2023-08-27T11:28:49.102888+0000","last_became_active":"2023-08-27T11:28:49.089329+0000","last_became_peered":"2023-08-27T11:28:49.089329+0000","last_unstale":"2023-08-27T11:28:49.102888+0000","last_undegraded":"2023-08-27T11:28:49.102888+0000","last_fullsized":"2023-08-27T11:28:49.102888+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:50:24.495137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.088916+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:49.088916+0000","last_peered":"2023-08-27T11:28:49.088916+0000","last_clean":"2023-08-27T11:28:49.088916+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:49.088916+0000","last_undegraded":"2023-08-27T11:28:49.088916+0000","last_fullsized":"2023-08-27T11:28:49.088916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:47:44.825080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":761,"num_read_kb":1467,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5183,"ondisk_log_size":5183,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1986560,"data_stored":1322060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.53900000000000003,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66000000000000003,"5min":1.601,"15min":1.601},"last":0.247},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.72799999999999998,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.38100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.53500000000000003,"15min":0.56899999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.55100000000000005,"15min":0.58299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42899999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":0.60199999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59499999999999997,"15min":0.629},"min":{"1min":0.32000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":1.778,"15min":1.778},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58199999999999996,"15min":0.62},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.75900000000000001,"5min":0.78100000000000003,"15min":0.82999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.61099999999999999,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82299999999999995,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.72899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.35699999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.47199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88500000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.5}]},{"osd":6,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.442}]}]},{"osd":6,"up_from":13,"seq":55834575048,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56799999999999995,"15min":0.60199999999999998},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74299999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.1679999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":10.992000000000001,"5min":10.992000000000001,"15min":10.992000000000001},"last":0.45100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.624,"5min":0.78800000000000003,"15min":0.82599999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.64100000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52600000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.67600000000000005,"15min":0.624},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":11.019,"5min":11.019,"15min":11.019},"last":0.376},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53900000000000003,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59199999999999997,"5min":0.77200000000000002,"15min":0.83399999999999996},"last":0.42899999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54600000000000004,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60999999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.161,"5min":0.71099999999999997,"15min":0.64400000000000002},"min":{"1min":0.36899999999999999,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":10.956,"5min":10.956,"15min":10.956},"last":0.51700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63800000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.622,"15min":0.53500000000000003},"min":{"1min":0.33400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.069000000000001,"5min":11.069000000000001,"15min":11.069000000000001},"last":0.497}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.31900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56599999999999995,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.65500000000000003,"5min":1.157,"15min":1.157},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":0.64700000000000002,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":11.045,"5min":11.045,"15min":11.045},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.36899999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.50700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1380352,"data_stored":724700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.371,"15min":0.38300000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.20399999999999999},"max":{"1min":0.5,"5min":0.53500000000000003,"15min":0.65600000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.27800000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.41099999999999998,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.32900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.63300000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45900000000000002,"15min":0.48199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67000000000000004,"5min":1.169,"15min":1.169},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.439,"15min":0.439},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.23400000000000001},"max":{"1min":0.53000000000000003,"5min":1.139,"15min":1.139},"last":0.48499999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.621,"15min":0.623},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.88600000000000001,"15min":2.9380000000000002},"last":0.55300000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.625,"15min":0.63800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64300000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39100000000000001},"max":{"1min":0.77100000000000002,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.53600000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.64800000000000002,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69499999999999995,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.59999999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.752,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575048,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1986560,"data_stored":1322060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61099999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55400000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35899999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40699999999999997,"15min":0.41399999999999998},"min":{"1min":0.22900000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.495,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.23599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.46700000000000003,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.29599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42099999999999999,"15min":0.42599999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63900000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.81100000000000005},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60899999999999999,"15min":0.63800000000000001},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.86599999999999999,"15min":1.202},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.91200000000000003},"last":0.59499999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.96199999999999997,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.63500000000000001,"15min":0.66700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.248},"last":0.67100000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63800000000000001,"15min":0.65400000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.98699999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0189999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76300000000000001,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.62},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.65500000000000003,"15min":0.69199999999999995},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86699999999999999,"5min":0.90700000000000003,"15min":1.2809999999999999},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575049,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36599999999999999,"15min":0.372},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.44600000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.35199999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47099999999999997,"5min":0.60899999999999999,"15min":0.753},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.55500000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.308}]},{"osd":3,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52900000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.432,"15min":0.44900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.498,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.373}]},{"osd":4,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.60599999999999998,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":1.163},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.436}]},{"osd":5,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.36499999999999999},"max":{"1min":1.034,"5min":1.083,"15min":1.2470000000000001},"last":0.502},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40699999999999997},"max":{"1min":0.879,"5min":0.96799999999999997,"15min":1.2330000000000001},"last":0.46999999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81599999999999995,"5min":0.99299999999999999,"15min":1.1319999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64600000000000002,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.45100000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.65400000000000003,"15min":0.67800000000000005},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.68899999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.99299999999999999,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1380352,"data_stored":724700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.39100000000000001,"15min":0.38800000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.22},"max":{"1min":0.52100000000000002,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.38900000000000001,"15min":0.40899999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.42299999999999999,"5min":0.61299999999999999,"15min":0.71399999999999997},"last":0.22900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.38500000000000001,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.71999999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.439,"15min":0.46000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.245},"max":{"1min":0.53800000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.252}]},{"osd":2,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44700000000000001,"15min":0.46400000000000002},"min":{"1min":0.33200000000000002,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.49399999999999999,"5min":0.71199999999999997,"15min":0.93100000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67600000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.29999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72899999999999998,"5min":0.99399999999999999,"15min":1.0329999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78800000000000003,"5min":0.91000000000000003,"15min":1.113},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.37,"15min":0.37},"max":{"1min":0.872,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":1.0620000000000001,"15min":1.218},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65300000000000002,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":0.97199999999999998,"15min":1.2010000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.68799999999999994,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":1.0269999999999999,"15min":1.0389999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65000000000000002,"15min":0.67900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.93799999999999994,"15min":1.006},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70899999999999996,"5min":1.0389999999999999,"15min":1.0640000000000001},"last":0.48999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575048,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55900000000000005,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.255},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.873},"last":0.376}]},{"osd":1,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.76200000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.85299999999999998},"last":0.28999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42999999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77000000000000002,"5min":0.879,"15min":0.879},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.629,"15min":0.627},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.88},"last":0.45400000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.755,"15min":0.85699999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.26200000000000001},"max":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.82799999999999996},"last":0.39500000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.73299999999999998,"15min":1.0860000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.747,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.46700000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.874},"last":0.499},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.1339999999999999},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575047,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.58399999999999996},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57599999999999996,"15min":0.59999999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58399999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.62}]},{"osd":2,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32000000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.46899999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.23100000000000001},"max":{"1min":0.60699999999999998,"5min":1.157,"15min":1.157},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.61299999999999999,"5min":0.76700000000000002,"15min":0.86599999999999999},"last":0.30199999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70699999999999996,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.34300000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.27800000000000002},"max":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.503}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:54.277 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:54.309 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:54.308+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.435+0000 7f171422a700 1 Processor -- start 2023-08-27T11:28:54.436 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.435+0000 7f171422a700 1 -- start start 2023-08-27T11:28:54.438 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.436+0000 7f171422a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f170c15b120 0x7f170c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.436+0000 7f171422a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0d1a20 0x7f170c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.436+0000 7f171422a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c0ada50 con 0x7f170c0cd9b0 2023-08-27T11:28:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.436+0000 7f171422a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c0cde40 con 0x7f170c15b120 2023-08-27T11:28:54.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.436+0000 7f171422a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0ccd60 con 0x7f170c0d1a20 2023-08-27T11:28:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.436+0000 7f1711fc6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f170c15b120 0x7f170c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.436+0000 7f17127c7700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f170c0cd9b0 0x7f170c0cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47214/0 (socket says 172.21.15.45:47214) 2023-08-27T11:28:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.436+0000 7f17127c7700 1 -- 172.21.15.45:0/1408880046 learned_addr learned my addr 172.21.15.45:0/1408880046 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.436+0000 7f17117c5700 1 --2- 172.21.15.45:0/1408880046 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0d1a20 0x7f170c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.440 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f1711fc6700 1 -- 172.21.15.45:0/1408880046 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0d1a20 msgr2=0x7f170c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f1711fc6700 1 --2- 172.21.15.45:0/1408880046 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0d1a20 0x7f170c0cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f1711fc6700 1 -- 172.21.15.45:0/1408880046 >> v1:172.21.15.45:6789/0 conn(0x7f170c0cd9b0 legacy=0x7f170c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f1711fc6700 1 -- 172.21.15.45:0/1408880046 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c0b0860 con 0x7f170c15b120 2023-08-27T11:28:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f1711fc6700 1 --2- 172.21.15.45:0/1408880046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c15b120 0x7f170c0cc430 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009840 tx=0x7f16fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739614e4b614f801 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.441 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f1710fc4700 1 -- 172.21.15.45:0/1408880046 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc011830 con 0x7f170c15b120 2023-08-27T11:28:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f1710fc4700 1 -- 172.21.15.45:0/1408880046 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc011990 con 0x7f170c15b120 2023-08-27T11:28:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f1710fc4700 1 -- 172.21.15.45:0/1408880046 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc011d20 con 0x7f170c15b120 2023-08-27T11:28:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f171422a700 1 -- 172.21.15.45:0/1408880046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c15b120 msgr2=0x7f170c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.442 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f171422a700 1 --2- 172.21.15.45:0/1408880046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c15b120 0x7f170c0cc430 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009840 tx=0x7f16fc009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f171422a700 1 -- 172.21.15.45:0/1408880046 shutdown_connections 2023-08-27T11:28:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f171422a700 1 --2- 172.21.15.45:0/1408880046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c15b120 0x7f170c0cc430 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f171422a700 1 --2- 172.21.15.45:0/1408880046 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c0d1a20 0x7f170c0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.437+0000 7f171422a700 1 -- 172.21.15.45:0/1408880046 >> 172.21.15.45:0/1408880046 conn(0x7f170c09a160 msgr2=0x7f170c0d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.443 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.438+0000 7f171422a700 1 -- 172.21.15.45:0/1408880046 shutdown_connections 2023-08-27T11:28:54.444 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.438+0000 7f171422a700 1 -- 172.21.15.45:0/1408880046 wait complete. 2023-08-27T11:28:54.446 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.439+0000 7f171422a700 1 Processor -- start 2023-08-27T11:28:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.439+0000 7f171422a700 1 -- start start 2023-08-27T11:28:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.440+0000 7f171422a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c0cd9b0 0x7f170c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.440+0000 7f171422a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d1a20 0x7f170c15de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.440+0000 7f171422a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c15b120 0x7f170c158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.440+0000 7f171422a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c1f05d0 con 0x7f170c0cd9b0 2023-08-27T11:28:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.440+0000 7f171422a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c15e350 con 0x7f170c0d1a20 2023-08-27T11:28:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.440+0000 7f171422a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c15d350 con 0x7f170c15b120 2023-08-27T11:28:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.440+0000 7f1711fc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c0cd9b0 0x7f170c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.440+0000 7f1711fc6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c0cd9b0 0x7f170c15cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60440/0 (socket says 172.21.15.45:60440) 2023-08-27T11:28:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.440+0000 7f1711fc6700 1 -- 172.21.15.45:0/2725401667 learned_addr learned my addr 172.21.15.45:0/2725401667 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f17117c5700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d1a20 0x7f170c15de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f17127c7700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c15b120 0x7f170c158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f1711fc6700 1 -- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c15b120 msgr2=0x7f170c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f1711fc6700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c15b120 0x7f170c158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f1711fc6700 1 -- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d1a20 msgr2=0x7f170c15de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f1711fc6700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d1a20 0x7f170c15de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f1711fc6700 1 -- 172.21.15.45:0/2725401667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc0094a0 con 0x7f170c0cd9b0 2023-08-27T11:28:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f17117c5700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d1a20 0x7f170c15de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f17127c7700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c15b120 0x7f170c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f1711fc6700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c0cd9b0 0x7f170c15cd00 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009fd0 tx=0x7f16fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f17027fc700 1 -- 172.21.15.45:0/2725401667 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0038a0 con 0x7f170c0cd9b0 2023-08-27T11:28:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f17027fc700 1 -- 172.21.15.45:0/2725401667 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc01e070 con 0x7f170c0cd9b0 2023-08-27T11:28:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c15b5a0 con 0x7f170c0cd9b0 2023-08-27T11:28:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f17027fc700 1 -- 172.21.15.45:0/2725401667 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc003c40 con 0x7f170c0cd9b0 2023-08-27T11:28:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.441+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c15bb60 con 0x7f170c0cd9b0 2023-08-27T11:28:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.442+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c09edd0 con 0x7f170c0cd9b0 2023-08-27T11:28:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.443+0000 7f17027fc700 1 -- 172.21.15.45:0/2725401667 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f16fc02d450 con 0x7f170c0cd9b0 2023-08-27T11:28:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.443+0000 7f17027fc700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f16f006a170 0x7f16f006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.443+0000 7f17027fc700 1 -- 172.21.15.45:0/2725401667 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f16fc0897d0 con 0x7f170c0cd9b0 2023-08-27T11:28:54.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.443+0000 7f17117c5700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f16f006a170 0x7f16f006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.444+0000 7f17117c5700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f16f006a170 0x7f16f006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f170c06f480 tx=0x7f1708008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.448+0000 7f17027fc700 1 -- 172.21.15.45:0/2725401667 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f16fc054d60 con 0x7f170c0cd9b0 2023-08-27T11:28:54.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.651+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 --> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f170c15d520 con 0x7f16f006a170 2023-08-27T11:28:54.660 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.659+0000 7f17027fc700 1 -- 172.21.15.45:0/2725401667 <== mgr.7156 v2:172.21.15.135:6832/3759624346 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306609 (secure 0 0 0) 0x7f170c15d520 con 0x7f16f006a170 2023-08-27T11:28:54.664 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:54.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f16f006a170 msgr2=0x7f16f006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f16f006a170 0x7f16f006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f170c06f480 tx=0x7f1708008040 comp rx=0 tx=0).stop 2023-08-27T11:28:54.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c0cd9b0 msgr2=0x7f170c15cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c0cd9b0 0x7f170c15cd00 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009fd0 tx=0x7f16fc004750 comp rx=0 tx=0).stop 2023-08-27T11:28:54.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 shutdown_connections 2023-08-27T11:28:54.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f16f006a170 0x7f16f006c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f170c0cd9b0 0x7f170c15cd00 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f170c15b120 0x7f170c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.668 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 --2- 172.21.15.45:0/2725401667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f170c0d1a20 0x7f170c15de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 >> 172.21.15.45:0/2725401667 conn(0x7f170c09a160 msgr2=0x7f170c0aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 shutdown_connections 2023-08-27T11:28:54.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.663+0000 7f171422a700 1 -- 172.21.15.45:0/2725401667 wait complete. 2023-08-27T11:28:54.669 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:54.676 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:28:53.707854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":761,"num_read_kb":1467,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5183,"ondisk_log_size":5183,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278112,"kb_used_data":12016,"kb_used_omap":13,"kb_used_meta":265970,"kb_avail":754696608,"statfs":{"total":773094113280,"available":772809326592,"internally_reserved":0,"allocated":12304384,"data_stored":7020288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":272353368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.637638"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085340+0000","last_change":"2023-08-27T11:28:49.085340+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085340+0000","last_undegraded":"2023-08-27T11:28:49.085340+0000","last_fullsized":"2023-08-27T11:28:49.085340+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.092601+0000","last_change":"2023-08-27T11:28:49.092601+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.092601+0000","last_undegraded":"2023-08-27T11:28:49.092601+0000","last_fullsized":"2023-08-27T11:28:49.092601+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079160+0000","last_change":"2023-08-27T11:28:49.079160+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079160+0000","last_undegraded":"2023-08-27T11:28:49.079160+0000","last_fullsized":"2023-08-27T11:28:49.079160+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080168+0000","last_change":"2023-08-27T11:28:49.080168+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080168+0000","last_undegraded":"2023-08-27T11:28:49.080168+0000","last_fullsized":"2023-08-27T11:28:49.080168+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085503+0000","last_change":"2023-08-27T11:28:49.085503+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085503+0000","last_undegraded":"2023-08-27T11:28:49.085503+0000","last_fullsized":"2023-08-27T11:28:49.085503+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.091634+0000","last_change":"2023-08-27T11:28:49.091634+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.091634+0000","last_undegraded":"2023-08-27T11:28:49.091634+0000","last_fullsized":"2023-08-27T11:28:49.091634+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089205+0000","last_change":"2023-08-27T11:28:49.089205+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089205+0000","last_undegraded":"2023-08-27T11:28:49.089205+0000","last_fullsized":"2023-08-27T11:28:49.089205+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.091503+0000","last_change":"2023-08-27T11:28:49.091503+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.091503+0000","last_undegraded":"2023-08-27T11:28:49.091503+0000","last_fullsized":"2023-08-27T11:28:49.091503+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083331+0000","last_change":"2023-08-27T11:28:49.083331+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083331+0000","last_undegraded":"2023-08-27T11:28:49.083331+0000","last_fullsized":"2023-08-27T11:28:49.083331+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084616+0000","last_change":"2023-08-27T11:28:49.084616+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084616+0000","last_undegraded":"2023-08-27T11:28:49.084616+0000","last_fullsized":"2023-08-27T11:28:49.084616+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090012+0000","last_change":"2023-08-27T11:28:49.090012+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090012+0000","last_undegraded":"2023-08-27T11:28:49.090012+0000","last_fullsized":"2023-08-27T11:28:49.090012+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083870+0000","last_change":"2023-08-27T11:28:49.083870+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083870+0000","last_undegraded":"2023-08-27T11:28:49.083870+0000","last_fullsized":"2023-08-27T11:28:49.083870+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079983+0000","last_change":"2023-08-27T11:28:49.079983+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079983+0000","last_undegraded":"2023-08-27T11:28:49.079983+0000","last_fullsized":"2023-08-27T11:28:49.079983+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080749+0000","last_change":"2023-08-27T11:28:49.080749+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080749+0000","last_undegraded":"2023-08-27T11:28:49.080749+0000","last_fullsized":"2023-08-27T11:28:49.080749+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080853+0000","last_change":"2023-08-27T11:28:49.080853+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080853+0000","last_undegraded":"2023-08-27T11:28:49.080853+0000","last_fullsized":"2023-08-27T11:28:49.080853+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090451+0000","last_change":"2023-08-27T11:28:49.090451+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090451+0000","last_undegraded":"2023-08-27T11:28:49.090451+0000","last_fullsized":"2023-08-27T11:28:49.090451+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090009+0000","last_change":"2023-08-27T11:28:49.090009+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090009+0000","last_undegraded":"2023-08-27T11:28:49.090009+0000","last_fullsized":"2023-08-27T11:28:49.090009+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080580+0000","last_change":"2023-08-27T11:28:49.080580+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080580+0000","last_undegraded":"2023-08-27T11:28:49.080580+0000","last_fullsized":"2023-08-27T11:28:49.080580+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079857+0000","last_change":"2023-08-27T11:28:49.079857+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079857+0000","last_undegraded":"2023-08-27T11:28:49.079857+0000","last_fullsized":"2023-08-27T11:28:49.079857+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083451+0000","last_change":"2023-08-27T11:28:49.083451+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083451+0000","last_undegraded":"2023-08-27T11:28:49.083451+0000","last_fullsized":"2023-08-27T11:28:49.083451+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080787+0000","last_change":"2023-08-27T11:28:49.080787+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080787+0000","last_undegraded":"2023-08-27T11:28:49.080787+0000","last_fullsized":"2023-08-27T11:28:49.080787+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.091564+0000","last_change":"2023-08-27T11:28:49.091564+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.091564+0000","last_undegraded":"2023-08-27T11:28:49.091564+0000","last_fullsized":"2023-08-27T11:28:49.091564+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084393+0000","last_change":"2023-08-27T11:28:49.084393+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084393+0000","last_undegraded":"2023-08-27T11:28:49.084393+0000","last_fullsized":"2023-08-27T11:28:49.084393+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090030+0000","last_change":"2023-08-27T11:28:49.090030+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090030+0000","last_undegraded":"2023-08-27T11:28:49.090030+0000","last_fullsized":"2023-08-27T11:28:49.090030+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.086685+0000","last_change":"2023-08-27T11:28:49.086685+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.086685+0000","last_undegraded":"2023-08-27T11:28:49.086685+0000","last_fullsized":"2023-08-27T11:28:49.086685+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084953+0000","last_change":"2023-08-27T11:28:49.084953+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084953+0000","last_undegraded":"2023-08-27T11:28:49.084953+0000","last_fullsized":"2023-08-27T11:28:49.084953+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089211+0000","last_change":"2023-08-27T11:28:49.089211+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089211+0000","last_undegraded":"2023-08-27T11:28:49.089211+0000","last_fullsized":"2023-08-27T11:28:49.089211+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085408+0000","last_change":"2023-08-27T11:28:49.085408+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085408+0000","last_undegraded":"2023-08-27T11:28:49.085408+0000","last_fullsized":"2023-08-27T11:28:49.085408+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084163+0000","last_change":"2023-08-27T11:28:49.084163+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084163+0000","last_undegraded":"2023-08-27T11:28:49.084163+0000","last_fullsized":"2023-08-27T11:28:49.084163+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080300+0000","last_change":"2023-08-27T11:28:49.080300+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080300+0000","last_undegraded":"2023-08-27T11:28:49.080300+0000","last_fullsized":"2023-08-27T11:28:49.080300+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080473+0000","last_change":"2023-08-27T11:28:49.080473+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080473+0000","last_undegraded":"2023-08-27T11:28:49.080473+0000","last_fullsized":"2023-08-27T11:28:49.080473+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.081947+0000","last_change":"2023-08-27T11:28:49.081947+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.081947+0000","last_undegraded":"2023-08-27T11:28:49.081947+0000","last_fullsized":"2023-08-27T11:28:49.081947+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080179+0000","last_change":"2023-08-27T11:28:49.080179+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080179+0000","last_undegraded":"2023-08-27T11:28:49.080179+0000","last_fullsized":"2023-08-27T11:28:49.080179+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103625+0000","last_change":"2023-08-27T11:28:49.093411+0000","last_active":"2023-08-27T11:28:49.103625+0000","last_peered":"2023-08-27T11:28:49.103625+0000","last_clean":"2023-08-27T11:28:49.103625+0000","last_became_active":"2023-08-27T11:28:49.092946+0000","last_became_peered":"2023-08-27T11:28:49.092946+0000","last_unstale":"2023-08-27T11:28:49.103625+0000","last_undegraded":"2023-08-27T11:28:49.103625+0000","last_fullsized":"2023-08-27T11:28:49.103625+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:54.413756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079935+0000","last_change":"2023-08-27T11:28:49.079935+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079935+0000","last_undegraded":"2023-08-27T11:28:49.079935+0000","last_fullsized":"2023-08-27T11:28:49.079935+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097092+0000","last_change":"2023-08-27T11:28:49.095509+0000","last_active":"2023-08-27T11:28:49.097092+0000","last_peered":"2023-08-27T11:28:49.097092+0000","last_clean":"2023-08-27T11:28:49.097092+0000","last_became_active":"2023-08-27T11:28:49.095273+0000","last_became_peered":"2023-08-27T11:28:49.095273+0000","last_unstale":"2023-08-27T11:28:49.097092+0000","last_undegraded":"2023-08-27T11:28:49.097092+0000","last_fullsized":"2023-08-27T11:28:49.097092+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:27:36.936742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084149+0000","last_change":"2023-08-27T11:28:49.084149+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084149+0000","last_undegraded":"2023-08-27T11:28:49.084149+0000","last_fullsized":"2023-08-27T11:28:49.084149+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103577+0000","last_change":"2023-08-27T11:28:49.093022+0000","last_active":"2023-08-27T11:28:49.103577+0000","last_peered":"2023-08-27T11:28:49.103577+0000","last_clean":"2023-08-27T11:28:49.103577+0000","last_became_active":"2023-08-27T11:28:49.092689+0000","last_became_peered":"2023-08-27T11:28:49.092689+0000","last_unstale":"2023-08-27T11:28:49.103577+0000","last_undegraded":"2023-08-27T11:28:49.103577+0000","last_fullsized":"2023-08-27T11:28:49.103577+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:36:04.006868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082739+0000","last_change":"2023-08-27T11:28:49.082739+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082739+0000","last_undegraded":"2023-08-27T11:28:49.082739+0000","last_fullsized":"2023-08-27T11:28:49.082739+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103796+0000","last_change":"2023-08-27T11:28:49.098336+0000","last_active":"2023-08-27T11:28:49.103796+0000","last_peered":"2023-08-27T11:28:49.103796+0000","last_clean":"2023-08-27T11:28:49.103796+0000","last_became_active":"2023-08-27T11:28:49.098055+0000","last_became_peered":"2023-08-27T11:28:49.098055+0000","last_unstale":"2023-08-27T11:28:49.103796+0000","last_undegraded":"2023-08-27T11:28:49.103796+0000","last_fullsized":"2023-08-27T11:28:49.103796+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:39:49.220050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082760+0000","last_change":"2023-08-27T11:28:49.082760+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082760+0000","last_undegraded":"2023-08-27T11:28:49.082760+0000","last_fullsized":"2023-08-27T11:28:49.082760+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092434+0000","last_change":"2023-08-27T11:28:49.088759+0000","last_active":"2023-08-27T11:28:49.092434+0000","last_peered":"2023-08-27T11:28:49.092434+0000","last_clean":"2023-08-27T11:28:49.092434+0000","last_became_active":"2023-08-27T11:28:49.088495+0000","last_became_peered":"2023-08-27T11:28:49.088495+0000","last_unstale":"2023-08-27T11:28:49.092434+0000","last_undegraded":"2023-08-27T11:28:49.092434+0000","last_fullsized":"2023-08-27T11:28:49.092434+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:06.208356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089263+0000","last_change":"2023-08-27T11:28:49.089263+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089263+0000","last_undegraded":"2023-08-27T11:28:49.089263+0000","last_fullsized":"2023-08-27T11:28:49.089263+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103618+0000","last_change":"2023-08-27T11:28:49.093290+0000","last_active":"2023-08-27T11:28:49.103618+0000","last_peered":"2023-08-27T11:28:49.103618+0000","last_clean":"2023-08-27T11:28:49.103618+0000","last_became_active":"2023-08-27T11:28:49.092886+0000","last_became_peered":"2023-08-27T11:28:49.092886+0000","last_unstale":"2023-08-27T11:28:49.103618+0000","last_undegraded":"2023-08-27T11:28:49.103618+0000","last_fullsized":"2023-08-27T11:28:49.103618+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:19:27.269549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080199+0000","last_change":"2023-08-27T11:28:49.080199+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080199+0000","last_undegraded":"2023-08-27T11:28:49.080199+0000","last_fullsized":"2023-08-27T11:28:49.080199+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.094961+0000","last_change":"2023-08-27T11:28:49.093546+0000","last_active":"2023-08-27T11:28:49.094961+0000","last_peered":"2023-08-27T11:28:49.094961+0000","last_clean":"2023-08-27T11:28:49.094961+0000","last_became_active":"2023-08-27T11:28:49.093291+0000","last_became_peered":"2023-08-27T11:28:49.093291+0000","last_unstale":"2023-08-27T11:28:49.094961+0000","last_undegraded":"2023-08-27T11:28:49.094961+0000","last_fullsized":"2023-08-27T11:28:49.094961+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:47.251433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083545+0000","last_change":"2023-08-27T11:28:49.083545+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083545+0000","last_undegraded":"2023-08-27T11:28:49.083545+0000","last_fullsized":"2023-08-27T11:28:49.083545+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.090413+0000","last_change":"2023-08-27T11:28:49.087457+0000","last_active":"2023-08-27T11:28:49.090413+0000","last_peered":"2023-08-27T11:28:49.090413+0000","last_clean":"2023-08-27T11:28:49.090413+0000","last_became_active":"2023-08-27T11:28:49.087210+0000","last_became_peered":"2023-08-27T11:28:49.087210+0000","last_unstale":"2023-08-27T11:28:49.090413+0000","last_undegraded":"2023-08-27T11:28:49.090413+0000","last_fullsized":"2023-08-27T11:28:49.090413+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:50:00.915296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084549+0000","last_change":"2023-08-27T11:28:49.084549+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084549+0000","last_undegraded":"2023-08-27T11:28:49.084549+0000","last_fullsized":"2023-08-27T11:28:49.084549+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.093549+0000","last_change":"2023-08-27T11:28:49.089014+0000","last_active":"2023-08-27T11:28:49.093549+0000","last_peered":"2023-08-27T11:28:49.093549+0000","last_clean":"2023-08-27T11:28:49.093549+0000","last_became_active":"2023-08-27T11:28:49.088750+0000","last_became_peered":"2023-08-27T11:28:49.088750+0000","last_unstale":"2023-08-27T11:28:49.093549+0000","last_undegraded":"2023-08-27T11:28:49.093549+0000","last_fullsized":"2023-08-27T11:28:49.093549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:12.555970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082603+0000","last_change":"2023-08-27T11:28:49.082603+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082603+0000","last_undegraded":"2023-08-27T11:28:49.082603+0000","last_fullsized":"2023-08-27T11:28:49.082603+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097693+0000","last_change":"2023-08-27T11:28:49.093294+0000","last_active":"2023-08-27T11:28:49.097693+0000","last_peered":"2023-08-27T11:28:49.097693+0000","last_clean":"2023-08-27T11:28:49.097693+0000","last_became_active":"2023-08-27T11:28:49.092967+0000","last_became_peered":"2023-08-27T11:28:49.092967+0000","last_unstale":"2023-08-27T11:28:49.097693+0000","last_undegraded":"2023-08-27T11:28:49.097693+0000","last_fullsized":"2023-08-27T11:28:49.097693+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:30.559346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085066+0000","last_change":"2023-08-27T11:28:49.085066+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085066+0000","last_undegraded":"2023-08-27T11:28:49.085066+0000","last_fullsized":"2023-08-27T11:28:49.085066+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097072+0000","last_change":"2023-08-27T11:28:49.095274+0000","last_active":"2023-08-27T11:28:49.097072+0000","last_peered":"2023-08-27T11:28:49.097072+0000","last_clean":"2023-08-27T11:28:49.097072+0000","last_became_active":"2023-08-27T11:28:49.095022+0000","last_became_peered":"2023-08-27T11:28:49.095022+0000","last_unstale":"2023-08-27T11:28:49.097072+0000","last_undegraded":"2023-08-27T11:28:49.097072+0000","last_fullsized":"2023-08-27T11:28:49.097072+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:20.172740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090393+0000","last_change":"2023-08-27T11:28:49.090393+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090393+0000","last_undegraded":"2023-08-27T11:28:49.090393+0000","last_fullsized":"2023-08-27T11:28:49.090393+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.091984+0000","last_change":"2023-08-27T11:28:49.087250+0000","last_active":"2023-08-27T11:28:49.091984+0000","last_peered":"2023-08-27T11:28:49.091984+0000","last_clean":"2023-08-27T11:28:49.091984+0000","last_became_active":"2023-08-27T11:28:49.087002+0000","last_became_peered":"2023-08-27T11:28:49.087002+0000","last_unstale":"2023-08-27T11:28:49.091984+0000","last_undegraded":"2023-08-27T11:28:49.091984+0000","last_fullsized":"2023-08-27T11:28:49.091984+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:52.621160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079943+0000","last_change":"2023-08-27T11:28:49.079943+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079943+0000","last_undegraded":"2023-08-27T11:28:49.079943+0000","last_fullsized":"2023-08-27T11:28:49.079943+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.091510+0000","last_change":"2023-08-27T11:28:49.086306+0000","last_active":"2023-08-27T11:28:49.091510+0000","last_peered":"2023-08-27T11:28:49.091510+0000","last_clean":"2023-08-27T11:28:49.091510+0000","last_became_active":"2023-08-27T11:28:49.086033+0000","last_became_peered":"2023-08-27T11:28:49.086033+0000","last_unstale":"2023-08-27T11:28:49.091510+0000","last_undegraded":"2023-08-27T11:28:49.091510+0000","last_fullsized":"2023-08-27T11:28:49.091510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:13:39.364173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080758+0000","last_change":"2023-08-27T11:28:49.080758+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080758+0000","last_undegraded":"2023-08-27T11:28:49.080758+0000","last_fullsized":"2023-08-27T11:28:49.080758+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.091231+0000","last_change":"2023-08-27T11:28:49.088909+0000","last_active":"2023-08-27T11:28:49.091231+0000","last_peered":"2023-08-27T11:28:49.091231+0000","last_clean":"2023-08-27T11:28:49.091231+0000","last_became_active":"2023-08-27T11:28:49.088497+0000","last_became_peered":"2023-08-27T11:28:49.088497+0000","last_unstale":"2023-08-27T11:28:49.091231+0000","last_undegraded":"2023-08-27T11:28:49.091231+0000","last_fullsized":"2023-08-27T11:28:49.091231+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:46.497129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082660+0000","last_change":"2023-08-27T11:28:49.082660+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082660+0000","last_undegraded":"2023-08-27T11:28:49.082660+0000","last_fullsized":"2023-08-27T11:28:49.082660+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092067+0000","last_change":"2023-08-27T11:28:49.087910+0000","last_active":"2023-08-27T11:28:49.092067+0000","last_peered":"2023-08-27T11:28:49.092067+0000","last_clean":"2023-08-27T11:28:49.092067+0000","last_became_active":"2023-08-27T11:28:49.087656+0000","last_became_peered":"2023-08-27T11:28:49.087656+0000","last_unstale":"2023-08-27T11:28:49.092067+0000","last_undegraded":"2023-08-27T11:28:49.092067+0000","last_fullsized":"2023-08-27T11:28:49.092067+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:25.771051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079616+0000","last_change":"2023-08-27T11:28:49.079616+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079616+0000","last_undegraded":"2023-08-27T11:28:49.079616+0000","last_fullsized":"2023-08-27T11:28:49.079616+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.094978+0000","last_change":"2023-08-27T11:28:49.092647+0000","last_active":"2023-08-27T11:28:49.094978+0000","last_peered":"2023-08-27T11:28:49.094978+0000","last_clean":"2023-08-27T11:28:49.094978+0000","last_became_active":"2023-08-27T11:28:49.092425+0000","last_became_peered":"2023-08-27T11:28:49.092425+0000","last_unstale":"2023-08-27T11:28:49.094978+0000","last_undegraded":"2023-08-27T11:28:49.094978+0000","last_fullsized":"2023-08-27T11:28:49.094978+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:45.451239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084132+0000","last_change":"2023-08-27T11:28:49.084132+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084132+0000","last_undegraded":"2023-08-27T11:28:49.084132+0000","last_fullsized":"2023-08-27T11:28:49.084132+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092980+0000","last_change":"2023-08-27T11:28:49.090840+0000","last_active":"2023-08-27T11:28:49.092980+0000","last_peered":"2023-08-27T11:28:49.092980+0000","last_clean":"2023-08-27T11:28:49.092980+0000","last_became_active":"2023-08-27T11:28:49.090655+0000","last_became_peered":"2023-08-27T11:28:49.090655+0000","last_unstale":"2023-08-27T11:28:49.092980+0000","last_undegraded":"2023-08-27T11:28:49.092980+0000","last_fullsized":"2023-08-27T11:28:49.092980+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:03.433230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084933+0000","last_change":"2023-08-27T11:28:49.084933+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084933+0000","last_undegraded":"2023-08-27T11:28:49.084933+0000","last_fullsized":"2023-08-27T11:28:49.084933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092990+0000","last_change":"2023-08-27T11:28:49.088886+0000","last_active":"2023-08-27T11:28:49.092990+0000","last_peered":"2023-08-27T11:28:49.092990+0000","last_clean":"2023-08-27T11:28:49.092990+0000","last_became_active":"2023-08-27T11:28:49.088618+0000","last_became_peered":"2023-08-27T11:28:49.088618+0000","last_unstale":"2023-08-27T11:28:49.092990+0000","last_undegraded":"2023-08-27T11:28:49.092990+0000","last_fullsized":"2023-08-27T11:28:49.092990+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:45:46.741280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.081075+0000","last_change":"2023-08-27T11:28:49.081075+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.081075+0000","last_undegraded":"2023-08-27T11:28:49.081075+0000","last_fullsized":"2023-08-27T11:28:49.081075+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103821+0000","last_change":"2023-08-27T11:28:49.098450+0000","last_active":"2023-08-27T11:28:49.103821+0000","last_peered":"2023-08-27T11:28:49.103821+0000","last_clean":"2023-08-27T11:28:49.103821+0000","last_became_active":"2023-08-27T11:28:49.098071+0000","last_became_peered":"2023-08-27T11:28:49.098071+0000","last_unstale":"2023-08-27T11:28:49.103821+0000","last_undegraded":"2023-08-27T11:28:49.103821+0000","last_fullsized":"2023-08-27T11:28:49.103821+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:05.557628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084983+0000","last_change":"2023-08-27T11:28:49.084983+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084983+0000","last_undegraded":"2023-08-27T11:28:49.084983+0000","last_fullsized":"2023-08-27T11:28:49.084983+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097260+0000","last_change":"2023-08-27T11:28:49.094459+0000","last_active":"2023-08-27T11:28:49.097260+0000","last_peered":"2023-08-27T11:28:49.097260+0000","last_clean":"2023-08-27T11:28:49.097260+0000","last_became_active":"2023-08-27T11:28:49.094215+0000","last_became_peered":"2023-08-27T11:28:49.094215+0000","last_unstale":"2023-08-27T11:28:49.097260+0000","last_undegraded":"2023-08-27T11:28:49.097260+0000","last_fullsized":"2023-08-27T11:28:49.097260+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:27:04.328482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.077745+0000","last_change":"2023-08-27T11:28:49.077745+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.077745+0000","last_undegraded":"2023-08-27T11:28:49.077745+0000","last_fullsized":"2023-08-27T11:28:49.077745+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.102914+0000","last_change":"2023-08-27T11:28:49.091063+0000","last_active":"2023-08-27T11:28:49.102914+0000","last_peered":"2023-08-27T11:28:49.102914+0000","last_clean":"2023-08-27T11:28:49.102914+0000","last_became_active":"2023-08-27T11:28:49.090005+0000","last_became_peered":"2023-08-27T11:28:49.090005+0000","last_unstale":"2023-08-27T11:28:49.102914+0000","last_undegraded":"2023-08-27T11:28:49.102914+0000","last_fullsized":"2023-08-27T11:28:49.102914+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:37.416680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082576+0000","last_change":"2023-08-27T11:28:49.082576+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082576+0000","last_undegraded":"2023-08-27T11:28:49.082576+0000","last_fullsized":"2023-08-27T11:28:49.082576+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092298+0000","last_change":"2023-08-27T11:28:49.089703+0000","last_active":"2023-08-27T11:28:49.092298+0000","last_peered":"2023-08-27T11:28:49.092298+0000","last_clean":"2023-08-27T11:28:49.092298+0000","last_became_active":"2023-08-27T11:28:49.089514+0000","last_became_peered":"2023-08-27T11:28:49.089514+0000","last_unstale":"2023-08-27T11:28:49.092298+0000","last_undegraded":"2023-08-27T11:28:49.092298+0000","last_fullsized":"2023-08-27T11:28:49.092298+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:01:44.852062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.077759+0000","last_change":"2023-08-27T11:28:49.077759+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.077759+0000","last_undegraded":"2023-08-27T11:28:49.077759+0000","last_fullsized":"2023-08-27T11:28:49.077759+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.089958+0000","last_change":"2023-08-27T11:28:49.087175+0000","last_active":"2023-08-27T11:28:49.089958+0000","last_peered":"2023-08-27T11:28:49.089958+0000","last_clean":"2023-08-27T11:28:49.089958+0000","last_became_active":"2023-08-27T11:28:49.086854+0000","last_became_peered":"2023-08-27T11:28:49.086854+0000","last_unstale":"2023-08-27T11:28:49.089958+0000","last_undegraded":"2023-08-27T11:28:49.089958+0000","last_fullsized":"2023-08-27T11:28:49.089958+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:19.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085540+0000","last_change":"2023-08-27T11:28:49.085540+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085540+0000","last_undegraded":"2023-08-27T11:28:49.085540+0000","last_fullsized":"2023-08-27T11:28:49.085540+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097152+0000","last_change":"2023-08-27T11:28:49.096229+0000","last_active":"2023-08-27T11:28:49.097152+0000","last_peered":"2023-08-27T11:28:49.097152+0000","last_clean":"2023-08-27T11:28:49.097152+0000","last_became_active":"2023-08-27T11:28:49.095951+0000","last_became_peered":"2023-08-27T11:28:49.095951+0000","last_unstale":"2023-08-27T11:28:49.097152+0000","last_undegraded":"2023-08-27T11:28:49.097152+0000","last_fullsized":"2023-08-27T11:28:49.097152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:37:05.774883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.086269+0000","last_change":"2023-08-27T11:28:49.086269+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.086269+0000","last_undegraded":"2023-08-27T11:28:49.086269+0000","last_fullsized":"2023-08-27T11:28:49.086269+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103699+0000","last_change":"2023-08-27T11:28:49.095920+0000","last_active":"2023-08-27T11:28:49.103699+0000","last_peered":"2023-08-27T11:28:49.103699+0000","last_clean":"2023-08-27T11:28:49.103699+0000","last_became_active":"2023-08-27T11:28:49.095652+0000","last_became_peered":"2023-08-27T11:28:49.095652+0000","last_unstale":"2023-08-27T11:28:49.103699+0000","last_undegraded":"2023-08-27T11:28:49.103699+0000","last_fullsized":"2023-08-27T11:28:49.103699+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:59.514247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":193,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.078437+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:49.078437+0000","last_peered":"2023-08-27T11:28:49.078437+0000","last_clean":"2023-08-27T11:28:49.078437+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:49.078437+0000","last_undegraded":"2023-08-27T11:28:49.078437+0000","last_fullsized":"2023-08-27T11:28:49.078437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:15:19.298171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.086169+0000","last_change":"2023-08-27T11:28:49.086169+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.086169+0000","last_undegraded":"2023-08-27T11:28:49.086169+0000","last_fullsized":"2023-08-27T11:28:49.086169+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.094004+0000","last_change":"2023-08-27T11:28:49.092776+0000","last_active":"2023-08-27T11:28:49.094004+0000","last_peered":"2023-08-27T11:28:49.094004+0000","last_clean":"2023-08-27T11:28:49.094004+0000","last_became_active":"2023-08-27T11:28:49.092503+0000","last_became_peered":"2023-08-27T11:28:49.092503+0000","last_unstale":"2023-08-27T11:28:49.094004+0000","last_undegraded":"2023-08-27T11:28:49.094004+0000","last_fullsized":"2023-08-27T11:28:49.094004+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:41.137920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":195,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.084685+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:49.084685+0000","last_peered":"2023-08-27T11:28:49.084685+0000","last_clean":"2023-08-27T11:28:49.084685+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:49.084685+0000","last_undegraded":"2023-08-27T11:28:49.084685+0000","last_fullsized":"2023-08-27T11:28:49.084685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:17:55.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084016+0000","last_change":"2023-08-27T11:28:49.084016+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084016+0000","last_undegraded":"2023-08-27T11:28:49.084016+0000","last_fullsized":"2023-08-27T11:28:49.084016+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092250+0000","last_change":"2023-08-27T11:28:49.088492+0000","last_active":"2023-08-27T11:28:49.092250+0000","last_peered":"2023-08-27T11:28:49.092250+0000","last_clean":"2023-08-27T11:28:49.092250+0000","last_became_active":"2023-08-27T11:28:49.088156+0000","last_became_peered":"2023-08-27T11:28:49.088156+0000","last_unstale":"2023-08-27T11:28:49.092250+0000","last_undegraded":"2023-08-27T11:28:49.092250+0000","last_fullsized":"2023-08-27T11:28:49.092250+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:11:26.876964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":183,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.075434+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:49.075434+0000","last_peered":"2023-08-27T11:28:49.075434+0000","last_clean":"2023-08-27T11:28:49.075434+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:49.075434+0000","last_undegraded":"2023-08-27T11:28:49.075434+0000","last_fullsized":"2023-08-27T11:28:49.075434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:20:55.742674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079769+0000","last_change":"2023-08-27T11:28:49.079769+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079769+0000","last_undegraded":"2023-08-27T11:28:49.079769+0000","last_fullsized":"2023-08-27T11:28:49.079769+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.098655+0000","last_change":"2023-08-27T11:28:49.098322+0000","last_active":"2023-08-27T11:28:49.098655+0000","last_peered":"2023-08-27T11:28:49.098655+0000","last_clean":"2023-08-27T11:28:49.098655+0000","last_became_active":"2023-08-27T11:28:49.098035+0000","last_became_peered":"2023-08-27T11:28:49.098035+0000","last_unstale":"2023-08-27T11:28:49.098655+0000","last_undegraded":"2023-08-27T11:28:49.098655+0000","last_fullsized":"2023-08-27T11:28:49.098655+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:21:27.504700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":195,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.083440+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:49.083440+0000","last_peered":"2023-08-27T11:28:49.083440+0000","last_clean":"2023-08-27T11:28:49.083440+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:49.083440+0000","last_undegraded":"2023-08-27T11:28:49.083440+0000","last_fullsized":"2023-08-27T11:28:49.083440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:57:44.263129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"113'5183","reported_seq":5642,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.700641+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:49.700641+0000","last_peered":"2023-08-27T11:28:49.700641+0000","last_clean":"2023-08-27T11:28:49.700641+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:49.700641+0000","last_undegraded":"2023-08-27T11:28:49.700641+0000","last_fullsized":"2023-08-27T11:28:49.700641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":5183,"log_dups_size":0,"ondisk_log_size":5183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:49.306768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":761,"num_read_kb":1467,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089647+0000","last_change":"2023-08-27T11:28:49.089647+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089647+0000","last_undegraded":"2023-08-27T11:28:49.089647+0000","last_fullsized":"2023-08-27T11:28:49.089647+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103818+0000","last_change":"2023-08-27T11:28:49.098393+0000","last_active":"2023-08-27T11:28:49.103818+0000","last_peered":"2023-08-27T11:28:49.103818+0000","last_clean":"2023-08-27T11:28:49.103818+0000","last_became_active":"2023-08-27T11:28:49.098104+0000","last_became_peered":"2023-08-27T11:28:49.098104+0000","last_unstale":"2023-08-27T11:28:49.103818+0000","last_undegraded":"2023-08-27T11:28:49.103818+0000","last_fullsized":"2023-08-27T11:28:49.103818+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:14.479332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":195,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092502+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:49.092502+0000","last_peered":"2023-08-27T11:28:49.092502+0000","last_clean":"2023-08-27T11:28:49.092502+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:49.092502+0000","last_undegraded":"2023-08-27T11:28:49.092502+0000","last_fullsized":"2023-08-27T11:28:49.092502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:06:07.246346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.078553+0000","last_change":"2023-08-27T11:28:49.078553+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.078553+0000","last_undegraded":"2023-08-27T11:28:49.078553+0000","last_fullsized":"2023-08-27T11:28:49.078553+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092559+0000","last_change":"2023-08-27T11:28:49.090592+0000","last_active":"2023-08-27T11:28:49.092559+0000","last_peered":"2023-08-27T11:28:49.092559+0000","last_clean":"2023-08-27T11:28:49.092559+0000","last_became_active":"2023-08-27T11:28:49.090309+0000","last_became_peered":"2023-08-27T11:28:49.090309+0000","last_unstale":"2023-08-27T11:28:49.092559+0000","last_undegraded":"2023-08-27T11:28:49.092559+0000","last_fullsized":"2023-08-27T11:28:49.092559+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:29:21.154979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":194,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092856+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:49.092856+0000","last_peered":"2023-08-27T11:28:49.092856+0000","last_clean":"2023-08-27T11:28:49.092856+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:49.092856+0000","last_undegraded":"2023-08-27T11:28:49.092856+0000","last_fullsized":"2023-08-27T11:28:49.092856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:29:19.522512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080372+0000","last_change":"2023-08-27T11:28:49.080372+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080372+0000","last_undegraded":"2023-08-27T11:28:49.080372+0000","last_fullsized":"2023-08-27T11:28:49.080372+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092543+0000","last_change":"2023-08-27T11:28:49.089867+0000","last_active":"2023-08-27T11:28:49.092543+0000","last_peered":"2023-08-27T11:28:49.092543+0000","last_clean":"2023-08-27T11:28:49.092543+0000","last_became_active":"2023-08-27T11:28:49.089621+0000","last_became_peered":"2023-08-27T11:28:49.089621+0000","last_unstale":"2023-08-27T11:28:49.092543+0000","last_undegraded":"2023-08-27T11:28:49.092543+0000","last_fullsized":"2023-08-27T11:28:49.092543+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:03.921396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.078190+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:49.078190+0000","last_peered":"2023-08-27T11:28:49.078190+0000","last_clean":"2023-08-27T11:28:49.078190+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:49.078190+0000","last_undegraded":"2023-08-27T11:28:49.078190+0000","last_fullsized":"2023-08-27T11:28:49.078190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:30.750930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079866+0000","last_change":"2023-08-27T11:28:49.079866+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079866+0000","last_undegraded":"2023-08-27T11:28:49.079866+0000","last_fullsized":"2023-08-27T11:28:49.079866+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.102888+0000","last_change":"2023-08-27T11:28:49.089632+0000","last_active":"2023-08-27T11:28:49.102888+0000","last_peered":"2023-08-27T11:28:49.102888+0000","last_clean":"2023-08-27T11:28:49.102888+0000","last_became_active":"2023-08-27T11:28:49.089329+0000","last_became_peered":"2023-08-27T11:28:49.089329+0000","last_unstale":"2023-08-27T11:28:49.102888+0000","last_undegraded":"2023-08-27T11:28:49.102888+0000","last_fullsized":"2023-08-27T11:28:49.102888+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:50:24.495137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.088916+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:49.088916+0000","last_peered":"2023-08-27T11:28:49.088916+0000","last_clean":"2023-08-27T11:28:49.088916+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:49.088916+0000","last_undegraded":"2023-08-27T11:28:49.088916+0000","last_fullsized":"2023-08-27T11:28:49.088916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:47:44.825080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":761,"num_read_kb":1467,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5183,"ondisk_log_size":5183,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1986560,"data_stored":1322060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.53900000000000003,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66000000000000003,"5min":1.601,"15min":1.601},"last":0.247},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.72799999999999998,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.38100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.53500000000000003,"15min":0.56899999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.55100000000000005,"15min":0.58299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42899999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":0.60199999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59499999999999997,"15min":0.629},"min":{"1min":0.32000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":1.778,"15min":1.778},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58199999999999996,"15min":0.62},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.75900000000000001,"5min":0.78100000000000003,"15min":0.82999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.61099999999999999,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82299999999999995,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.72899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.35699999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.47199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88500000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.5}]},{"osd":6,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.442}]}]},{"osd":6,"up_from":13,"seq":55834575048,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56799999999999995,"15min":0.60199999999999998},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74299999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.1679999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":10.992000000000001,"5min":10.992000000000001,"15min":10.992000000000001},"last":0.45100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.624,"5min":0.78800000000000003,"15min":0.82599999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.64100000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52600000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.67600000000000005,"15min":0.624},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":11.019,"5min":11.019,"15min":11.019},"last":0.376},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53900000000000003,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59199999999999997,"5min":0.77200000000000002,"15min":0.83399999999999996},"last":0.42899999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54600000000000004,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60999999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.161,"5min":0.71099999999999997,"15min":0.64400000000000002},"min":{"1min":0.36899999999999999,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":10.956,"5min":10.956,"15min":10.956},"last":0.51700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63800000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.622,"15min":0.53500000000000003},"min":{"1min":0.33400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.069000000000001,"5min":11.069000000000001,"15min":11.069000000000001},"last":0.497}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.31900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56599999999999995,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.65500000000000003,"5min":1.157,"15min":1.157},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":0.64700000000000002,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":11.045,"5min":11.045,"15min":11.045},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.36899999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.50700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1380352,"data_stored":724700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.371,"15min":0.38300000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.20399999999999999},"max":{"1min":0.5,"5min":0.53500000000000003,"15min":0.65600000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.27800000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.41099999999999998,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.32900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.63300000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45900000000000002,"15min":0.48199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67000000000000004,"5min":1.169,"15min":1.169},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.439,"15min":0.439},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.23400000000000001},"max":{"1min":0.53000000000000003,"5min":1.139,"15min":1.139},"last":0.48499999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.621,"15min":0.623},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.88600000000000001,"15min":2.9380000000000002},"last":0.55300000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.625,"15min":0.63800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64300000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39100000000000001},"max":{"1min":0.77100000000000002,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.53600000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.64800000000000002,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69499999999999995,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.59999999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.752,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575048,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1986560,"data_stored":1322060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61099999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55400000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35899999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40699999999999997,"15min":0.41399999999999998},"min":{"1min":0.22900000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.495,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.23599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.46700000000000003,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.29599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42099999999999999,"15min":0.42599999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63900000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.81100000000000005},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60899999999999999,"15min":0.63800000000000001},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.86599999999999999,"15min":1.202},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.91200000000000003},"last":0.59499999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.96199999999999997,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.63500000000000001,"15min":0.66700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.248},"last":0.67100000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63800000000000001,"15min":0.65400000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.98699999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0189999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76300000000000001,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.62},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.65500000000000003,"15min":0.69199999999999995},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86699999999999999,"5min":0.90700000000000003,"15min":1.2809999999999999},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575049,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36599999999999999,"15min":0.372},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.44600000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.35199999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47099999999999997,"5min":0.60899999999999999,"15min":0.753},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.55500000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.308}]},{"osd":3,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52900000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.432,"15min":0.44900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.498,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.373}]},{"osd":4,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.60599999999999998,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":1.163},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.436}]},{"osd":5,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.36499999999999999},"max":{"1min":1.034,"5min":1.083,"15min":1.2470000000000001},"last":0.502},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40699999999999997},"max":{"1min":0.879,"5min":0.96799999999999997,"15min":1.2330000000000001},"last":0.46999999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81599999999999995,"5min":0.99299999999999999,"15min":1.1319999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64600000000000002,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.45100000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.65400000000000003,"15min":0.67800000000000005},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.68899999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.99299999999999999,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1380352,"data_stored":724700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.39100000000000001,"15min":0.38800000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.22},"max":{"1min":0.52100000000000002,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.38900000000000001,"15min":0.40899999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.42299999999999999,"5min":0.61299999999999999,"15min":0.71399999999999997},"last":0.22900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.38500000000000001,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.71999999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.439,"15min":0.46000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.245},"max":{"1min":0.53800000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.252}]},{"osd":2,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44700000000000001,"15min":0.46400000000000002},"min":{"1min":0.33200000000000002,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.49399999999999999,"5min":0.71199999999999997,"15min":0.93100000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67600000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.29999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72899999999999998,"5min":0.99399999999999999,"15min":1.0329999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78800000000000003,"5min":0.91000000000000003,"15min":1.113},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.37,"15min":0.37},"max":{"1min":0.872,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":1.0620000000000001,"15min":1.218},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65300000000000002,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":0.97199999999999998,"15min":1.2010000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.68799999999999994,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":1.0269999999999999,"15min":1.0389999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65000000000000002,"15min":0.67900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.93799999999999994,"15min":1.006},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70899999999999996,"5min":1.0389999999999999,"15min":1.0640000000000001},"last":0.48999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575048,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55900000000000005,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.255},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.873},"last":0.376}]},{"osd":1,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.76200000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.85299999999999998},"last":0.28999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42999999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77000000000000002,"5min":0.879,"15min":0.879},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.629,"15min":0.627},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.88},"last":0.45400000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.755,"15min":0.85699999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.26200000000000001},"max":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.82799999999999996},"last":0.39500000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.73299999999999998,"15min":1.0860000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.747,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.46700000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.874},"last":0.499},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.1339999999999999},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575047,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.58399999999999996},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57599999999999996,"15min":0.59999999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58399999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.62}]},{"osd":2,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32000000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.46899999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.23100000000000001},"max":{"1min":0.60699999999999998,"5min":1.157,"15min":1.157},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.61299999999999999,"5min":0.76700000000000002,"15min":0.86599999999999999},"last":0.30199999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70699999999999996,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.34300000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.27800000000000002},"max":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.503}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:54.682 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:54.842 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.842+0000 7f9edb1df700 1 Processor -- start 2023-08-27T11:28:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.842+0000 7f9edb1df700 1 -- start start 2023-08-27T11:28:54.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.842+0000 7f9edb1df700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ed415d450 0x7f9ed40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.842+0000 7f9edb1df700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed40cbaf0 0x7f9ed40cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.845 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.842+0000 7f9edb1df700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9ed40b2140 0x7f9ed40d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.842+0000 7f9edb1df700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed40ce1c0 con 0x7f9ed415d450 2023-08-27T11:28:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.842+0000 7f9edb1df700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed40cf290 con 0x7f9ed40b2140 2023-08-27T11:28:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.842+0000 7f9edb1df700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40ce440 con 0x7f9ed40cbaf0 2023-08-27T11:28:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed8f7b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ed415d450 0x7f9ed40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed8f7b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ed415d450 0x7f9ed40cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60448/0 (socket says 172.21.15.45:60448) 2023-08-27T11:28:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed8f7b700 1 -- 172.21.15.45:0/3068366094 learned_addr learned my addr 172.21.15.45:0/3068366094 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed977c700 1 --2- 172.21.15.45:0/3068366094 >> v2:172.21.15.135:3300/0 conn(0x7f9ed40b2140 0x7f9ed40d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed3fff700 1 --2- 172.21.15.45:0/3068366094 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed40cbaf0 0x7f9ed40cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed8f7b700 1 -- 172.21.15.45:0/3068366094 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed40cbaf0 msgr2=0x7f9ed40cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed8f7b700 1 --2- 172.21.15.45:0/3068366094 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed40cbaf0 0x7f9ed40cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed8f7b700 1 -- 172.21.15.45:0/3068366094 >> v2:172.21.15.135:3300/0 conn(0x7f9ed40b2140 msgr2=0x7f9ed40d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed8f7b700 1 --2- 172.21.15.45:0/3068366094 >> v2:172.21.15.135:3300/0 conn(0x7f9ed40b2140 0x7f9ed40d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed8f7b700 1 -- 172.21.15.45:0/3068366094 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40ce720 con 0x7f9ed415d450 2023-08-27T11:28:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed8f7b700 1 --2- 172.21.15.45:0/3068366094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed415d450 0x7f9ed40cb5b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40b20f0 tx=0x7f9ec8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244e4fd5917dfcc1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed37fe700 1 -- 172.21.15.45:0/3068366094 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800b870 con 0x7f9ed415d450 2023-08-27T11:28:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed37fe700 1 -- 172.21.15.45:0/3068366094 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec800b9d0 con 0x7f9ed415d450 2023-08-27T11:28:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.843+0000 7f9ed37fe700 1 -- 172.21.15.45:0/3068366094 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800bd60 con 0x7f9ed415d450 2023-08-27T11:28:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.844+0000 7f9edb1df700 1 -- 172.21.15.45:0/3068366094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed415d450 msgr2=0x7f9ed40cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.844+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3068366094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed415d450 0x7f9ed40cb5b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40b20f0 tx=0x7f9ec8009bd0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.844+0000 7f9edb1df700 1 -- 172.21.15.45:0/3068366094 shutdown_connections 2023-08-27T11:28:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.844+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3068366094 >> v2:172.21.15.135:3300/0 conn(0x7f9ed40b2140 0x7f9ed40d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.844+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3068366094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed415d450 0x7f9ed40cb5b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.844+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3068366094 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed40cbaf0 0x7f9ed40cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.844+0000 7f9edb1df700 1 -- 172.21.15.45:0/3068366094 >> 172.21.15.45:0/3068366094 conn(0x7f9ed40d0f60 msgr2=0x7f9ed40d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.844+0000 7f9edb1df700 1 -- 172.21.15.45:0/3068366094 shutdown_connections 2023-08-27T11:28:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.844+0000 7f9edb1df700 1 -- 172.21.15.45:0/3068366094 wait complete. 2023-08-27T11:28:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.846+0000 7f9edb1df700 1 Processor -- start 2023-08-27T11:28:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.846+0000 7f9edb1df700 1 -- start start 2023-08-27T11:28:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9edb1df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed40b2140 0x7f9ed40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9edb1df700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed40cbaf0 0x7f9ed4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9edb1df700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed415d450 0x7f9ed415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9edb1df700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40b5df0 con 0x7f9ed40b2140 2023-08-27T11:28:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9edb1df700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40b51e0 con 0x7f9ed40cbaf0 2023-08-27T11:28:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9edb1df700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40b54c0 con 0x7f9ed415d450 2023-08-27T11:28:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed8f7b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed40b2140 0x7f9ed40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed8f7b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed40b2140 0x7f9ed40b6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60454/0 (socket says 172.21.15.45:60454) 2023-08-27T11:28:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed8f7b700 1 -- 172.21.15.45:0/3026037954 learned_addr learned my addr 172.21.15.45:0/3026037954 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed3fff700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed40cbaf0 0x7f9ed4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed977c700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed415d450 0x7f9ed415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed8f7b700 1 -- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed415d450 msgr2=0x7f9ed415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed8f7b700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed415d450 0x7f9ed415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed8f7b700 1 -- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed40cbaf0 msgr2=0x7f9ed4159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:54.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed8f7b700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed40cbaf0 0x7f9ed4159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:54.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed8f7b700 1 -- 172.21.15.45:0/3026037954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8009550 con 0x7f9ed40b2140 2023-08-27T11:28:54.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed8f7b700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed40b2140 0x7f9ed40b6280 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.856 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.847+0000 7f9ed17fa700 1 -- 172.21.15.45:0/3026037954 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8004310 con 0x7f9ed40b2140 2023-08-27T11:28:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.848+0000 7f9ed17fa700 1 -- 172.21.15.45:0/3026037954 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec801e070 con 0x7f9ed40b2140 2023-08-27T11:28:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.848+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed40b57a0 con 0x7f9ed40b2140 2023-08-27T11:28:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.848+0000 7f9ed17fa700 1 -- 172.21.15.45:0/3026037954 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80036a0 con 0x7f9ed40b2140 2023-08-27T11:28:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.848+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed40b67c0 con 0x7f9ed40b2140 2023-08-27T11:28:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.848+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed409ee50 con 0x7f9ed40b2140 2023-08-27T11:28:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.849+0000 7f9ed17fa700 1 -- 172.21.15.45:0/3026037954 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f9ec8020030 con 0x7f9ed40b2140 2023-08-27T11:28:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.850+0000 7f9ed17fa700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9eb806a170 0x7f9eb806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.850+0000 7f9ed17fa700 1 -- 172.21.15.45:0/3026037954 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9ec8089580 con 0x7f9ed40b2140 2023-08-27T11:28:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.850+0000 7f9ed3fff700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9eb806a170 0x7f9eb806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.850+0000 7f9ed3fff700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9eb806a170 0x7f9eb806c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40b50d0 tx=0x7f9ec0008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:54.854+0000 7f9ed17fa700 1 -- 172.21.15.45:0/3026037954 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9ec8054ba0 con 0x7f9ed40b2140 2023-08-27T11:28:55.052 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.051+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 --> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ed40cf810 con 0x7f9eb806a170 2023-08-27T11:28:55.061 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.060+0000 7f9ed17fa700 1 -- 172.21.15.45:0/3026037954 <== mgr.7156 v2:172.21.15.135:6832/3759624346 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306609 (secure 0 0 0) 0x7f9ec0009e40 con 0x7f9eb806a170 2023-08-27T11:28:55.065 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:55.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9eb806a170 msgr2=0x7f9eb806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.067 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9eb806a170 0x7f9eb806c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40b50d0 tx=0x7f9ec0008040 comp rx=0 tx=0).stop 2023-08-27T11:28:55.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed40b2140 msgr2=0x7f9ed40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed40b2140 0x7f9ed40b6280 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8004750 comp rx=0 tx=0).stop 2023-08-27T11:28:55.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 shutdown_connections 2023-08-27T11:28:55.068 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f9eb806a170 0x7f9eb806c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ed40b2140 0x7f9ed40b6280 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.069 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f9ed415d450 0x7f9ed415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.071 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 --2- 172.21.15.45:0/3026037954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ed40cbaf0 0x7f9ed4159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.063+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 >> 172.21.15.45:0/3026037954 conn(0x7f9ed40d0f60 msgr2=0x7f9ed40cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:55.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.064+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 shutdown_connections 2023-08-27T11:28:55.072 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.064+0000 7f9edb1df700 1 -- 172.21.15.45:0/3026037954 wait complete. 2023-08-27T11:28:55.072 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:55.073 INFO:tasks.ceph.mds.a.smithi045.stderr:2023-08-27T11:28:55.068+0000 7f4f3933f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:28:55.077 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-08-27T11:28:53.707854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":761,"num_read_kb":1467,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5183,"ondisk_log_size":5183,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278112,"kb_used_data":12016,"kb_used_omap":13,"kb_used_meta":265970,"kb_avail":754696608,"statfs":{"total":773094113280,"available":772809326592,"internally_reserved":0,"allocated":12304384,"data_stored":7020288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":272353368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.637638"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085340+0000","last_change":"2023-08-27T11:28:49.085340+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085340+0000","last_undegraded":"2023-08-27T11:28:49.085340+0000","last_fullsized":"2023-08-27T11:28:49.085340+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.092601+0000","last_change":"2023-08-27T11:28:49.092601+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.092601+0000","last_undegraded":"2023-08-27T11:28:49.092601+0000","last_fullsized":"2023-08-27T11:28:49.092601+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079160+0000","last_change":"2023-08-27T11:28:49.079160+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079160+0000","last_undegraded":"2023-08-27T11:28:49.079160+0000","last_fullsized":"2023-08-27T11:28:49.079160+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080168+0000","last_change":"2023-08-27T11:28:49.080168+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080168+0000","last_undegraded":"2023-08-27T11:28:49.080168+0000","last_fullsized":"2023-08-27T11:28:49.080168+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085503+0000","last_change":"2023-08-27T11:28:49.085503+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085503+0000","last_undegraded":"2023-08-27T11:28:49.085503+0000","last_fullsized":"2023-08-27T11:28:49.085503+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.091634+0000","last_change":"2023-08-27T11:28:49.091634+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.091634+0000","last_undegraded":"2023-08-27T11:28:49.091634+0000","last_fullsized":"2023-08-27T11:28:49.091634+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089205+0000","last_change":"2023-08-27T11:28:49.089205+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089205+0000","last_undegraded":"2023-08-27T11:28:49.089205+0000","last_fullsized":"2023-08-27T11:28:49.089205+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.091503+0000","last_change":"2023-08-27T11:28:49.091503+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.091503+0000","last_undegraded":"2023-08-27T11:28:49.091503+0000","last_fullsized":"2023-08-27T11:28:49.091503+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083331+0000","last_change":"2023-08-27T11:28:49.083331+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083331+0000","last_undegraded":"2023-08-27T11:28:49.083331+0000","last_fullsized":"2023-08-27T11:28:49.083331+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084616+0000","last_change":"2023-08-27T11:28:49.084616+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084616+0000","last_undegraded":"2023-08-27T11:28:49.084616+0000","last_fullsized":"2023-08-27T11:28:49.084616+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090012+0000","last_change":"2023-08-27T11:28:49.090012+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090012+0000","last_undegraded":"2023-08-27T11:28:49.090012+0000","last_fullsized":"2023-08-27T11:28:49.090012+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083870+0000","last_change":"2023-08-27T11:28:49.083870+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083870+0000","last_undegraded":"2023-08-27T11:28:49.083870+0000","last_fullsized":"2023-08-27T11:28:49.083870+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079983+0000","last_change":"2023-08-27T11:28:49.079983+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079983+0000","last_undegraded":"2023-08-27T11:28:49.079983+0000","last_fullsized":"2023-08-27T11:28:49.079983+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080749+0000","last_change":"2023-08-27T11:28:49.080749+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080749+0000","last_undegraded":"2023-08-27T11:28:49.080749+0000","last_fullsized":"2023-08-27T11:28:49.080749+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080853+0000","last_change":"2023-08-27T11:28:49.080853+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080853+0000","last_undegraded":"2023-08-27T11:28:49.080853+0000","last_fullsized":"2023-08-27T11:28:49.080853+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090451+0000","last_change":"2023-08-27T11:28:49.090451+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090451+0000","last_undegraded":"2023-08-27T11:28:49.090451+0000","last_fullsized":"2023-08-27T11:28:49.090451+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090009+0000","last_change":"2023-08-27T11:28:49.090009+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090009+0000","last_undegraded":"2023-08-27T11:28:49.090009+0000","last_fullsized":"2023-08-27T11:28:49.090009+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080580+0000","last_change":"2023-08-27T11:28:49.080580+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080580+0000","last_undegraded":"2023-08-27T11:28:49.080580+0000","last_fullsized":"2023-08-27T11:28:49.080580+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079857+0000","last_change":"2023-08-27T11:28:49.079857+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079857+0000","last_undegraded":"2023-08-27T11:28:49.079857+0000","last_fullsized":"2023-08-27T11:28:49.079857+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083451+0000","last_change":"2023-08-27T11:28:49.083451+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083451+0000","last_undegraded":"2023-08-27T11:28:49.083451+0000","last_fullsized":"2023-08-27T11:28:49.083451+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080787+0000","last_change":"2023-08-27T11:28:49.080787+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080787+0000","last_undegraded":"2023-08-27T11:28:49.080787+0000","last_fullsized":"2023-08-27T11:28:49.080787+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.091564+0000","last_change":"2023-08-27T11:28:49.091564+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.091564+0000","last_undegraded":"2023-08-27T11:28:49.091564+0000","last_fullsized":"2023-08-27T11:28:49.091564+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084393+0000","last_change":"2023-08-27T11:28:49.084393+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084393+0000","last_undegraded":"2023-08-27T11:28:49.084393+0000","last_fullsized":"2023-08-27T11:28:49.084393+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090030+0000","last_change":"2023-08-27T11:28:49.090030+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090030+0000","last_undegraded":"2023-08-27T11:28:49.090030+0000","last_fullsized":"2023-08-27T11:28:49.090030+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.086685+0000","last_change":"2023-08-27T11:28:49.086685+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.086685+0000","last_undegraded":"2023-08-27T11:28:49.086685+0000","last_fullsized":"2023-08-27T11:28:49.086685+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084953+0000","last_change":"2023-08-27T11:28:49.084953+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084953+0000","last_undegraded":"2023-08-27T11:28:49.084953+0000","last_fullsized":"2023-08-27T11:28:49.084953+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089211+0000","last_change":"2023-08-27T11:28:49.089211+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089211+0000","last_undegraded":"2023-08-27T11:28:49.089211+0000","last_fullsized":"2023-08-27T11:28:49.089211+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085408+0000","last_change":"2023-08-27T11:28:49.085408+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085408+0000","last_undegraded":"2023-08-27T11:28:49.085408+0000","last_fullsized":"2023-08-27T11:28:49.085408+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084163+0000","last_change":"2023-08-27T11:28:49.084163+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084163+0000","last_undegraded":"2023-08-27T11:28:49.084163+0000","last_fullsized":"2023-08-27T11:28:49.084163+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080300+0000","last_change":"2023-08-27T11:28:49.080300+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080300+0000","last_undegraded":"2023-08-27T11:28:49.080300+0000","last_fullsized":"2023-08-27T11:28:49.080300+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080473+0000","last_change":"2023-08-27T11:28:49.080473+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080473+0000","last_undegraded":"2023-08-27T11:28:49.080473+0000","last_fullsized":"2023-08-27T11:28:49.080473+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.081947+0000","last_change":"2023-08-27T11:28:49.081947+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.081947+0000","last_undegraded":"2023-08-27T11:28:49.081947+0000","last_fullsized":"2023-08-27T11:28:49.081947+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080179+0000","last_change":"2023-08-27T11:28:49.080179+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080179+0000","last_undegraded":"2023-08-27T11:28:49.080179+0000","last_fullsized":"2023-08-27T11:28:49.080179+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103625+0000","last_change":"2023-08-27T11:28:49.093411+0000","last_active":"2023-08-27T11:28:49.103625+0000","last_peered":"2023-08-27T11:28:49.103625+0000","last_clean":"2023-08-27T11:28:49.103625+0000","last_became_active":"2023-08-27T11:28:49.092946+0000","last_became_peered":"2023-08-27T11:28:49.092946+0000","last_unstale":"2023-08-27T11:28:49.103625+0000","last_undegraded":"2023-08-27T11:28:49.103625+0000","last_fullsized":"2023-08-27T11:28:49.103625+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:54.413756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079935+0000","last_change":"2023-08-27T11:28:49.079935+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079935+0000","last_undegraded":"2023-08-27T11:28:49.079935+0000","last_fullsized":"2023-08-27T11:28:49.079935+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097092+0000","last_change":"2023-08-27T11:28:49.095509+0000","last_active":"2023-08-27T11:28:49.097092+0000","last_peered":"2023-08-27T11:28:49.097092+0000","last_clean":"2023-08-27T11:28:49.097092+0000","last_became_active":"2023-08-27T11:28:49.095273+0000","last_became_peered":"2023-08-27T11:28:49.095273+0000","last_unstale":"2023-08-27T11:28:49.097092+0000","last_undegraded":"2023-08-27T11:28:49.097092+0000","last_fullsized":"2023-08-27T11:28:49.097092+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:27:36.936742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084149+0000","last_change":"2023-08-27T11:28:49.084149+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084149+0000","last_undegraded":"2023-08-27T11:28:49.084149+0000","last_fullsized":"2023-08-27T11:28:49.084149+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103577+0000","last_change":"2023-08-27T11:28:49.093022+0000","last_active":"2023-08-27T11:28:49.103577+0000","last_peered":"2023-08-27T11:28:49.103577+0000","last_clean":"2023-08-27T11:28:49.103577+0000","last_became_active":"2023-08-27T11:28:49.092689+0000","last_became_peered":"2023-08-27T11:28:49.092689+0000","last_unstale":"2023-08-27T11:28:49.103577+0000","last_undegraded":"2023-08-27T11:28:49.103577+0000","last_fullsized":"2023-08-27T11:28:49.103577+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:36:04.006868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082739+0000","last_change":"2023-08-27T11:28:49.082739+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082739+0000","last_undegraded":"2023-08-27T11:28:49.082739+0000","last_fullsized":"2023-08-27T11:28:49.082739+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103796+0000","last_change":"2023-08-27T11:28:49.098336+0000","last_active":"2023-08-27T11:28:49.103796+0000","last_peered":"2023-08-27T11:28:49.103796+0000","last_clean":"2023-08-27T11:28:49.103796+0000","last_became_active":"2023-08-27T11:28:49.098055+0000","last_became_peered":"2023-08-27T11:28:49.098055+0000","last_unstale":"2023-08-27T11:28:49.103796+0000","last_undegraded":"2023-08-27T11:28:49.103796+0000","last_fullsized":"2023-08-27T11:28:49.103796+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:39:49.220050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082760+0000","last_change":"2023-08-27T11:28:49.082760+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082760+0000","last_undegraded":"2023-08-27T11:28:49.082760+0000","last_fullsized":"2023-08-27T11:28:49.082760+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092434+0000","last_change":"2023-08-27T11:28:49.088759+0000","last_active":"2023-08-27T11:28:49.092434+0000","last_peered":"2023-08-27T11:28:49.092434+0000","last_clean":"2023-08-27T11:28:49.092434+0000","last_became_active":"2023-08-27T11:28:49.088495+0000","last_became_peered":"2023-08-27T11:28:49.088495+0000","last_unstale":"2023-08-27T11:28:49.092434+0000","last_undegraded":"2023-08-27T11:28:49.092434+0000","last_fullsized":"2023-08-27T11:28:49.092434+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:48:06.208356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089263+0000","last_change":"2023-08-27T11:28:49.089263+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089263+0000","last_undegraded":"2023-08-27T11:28:49.089263+0000","last_fullsized":"2023-08-27T11:28:49.089263+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103618+0000","last_change":"2023-08-27T11:28:49.093290+0000","last_active":"2023-08-27T11:28:49.103618+0000","last_peered":"2023-08-27T11:28:49.103618+0000","last_clean":"2023-08-27T11:28:49.103618+0000","last_became_active":"2023-08-27T11:28:49.092886+0000","last_became_peered":"2023-08-27T11:28:49.092886+0000","last_unstale":"2023-08-27T11:28:49.103618+0000","last_undegraded":"2023-08-27T11:28:49.103618+0000","last_fullsized":"2023-08-27T11:28:49.103618+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:19:27.269549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080199+0000","last_change":"2023-08-27T11:28:49.080199+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080199+0000","last_undegraded":"2023-08-27T11:28:49.080199+0000","last_fullsized":"2023-08-27T11:28:49.080199+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.094961+0000","last_change":"2023-08-27T11:28:49.093546+0000","last_active":"2023-08-27T11:28:49.094961+0000","last_peered":"2023-08-27T11:28:49.094961+0000","last_clean":"2023-08-27T11:28:49.094961+0000","last_became_active":"2023-08-27T11:28:49.093291+0000","last_became_peered":"2023-08-27T11:28:49.093291+0000","last_unstale":"2023-08-27T11:28:49.094961+0000","last_undegraded":"2023-08-27T11:28:49.094961+0000","last_fullsized":"2023-08-27T11:28:49.094961+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:21:47.251433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.083545+0000","last_change":"2023-08-27T11:28:49.083545+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.083545+0000","last_undegraded":"2023-08-27T11:28:49.083545+0000","last_fullsized":"2023-08-27T11:28:49.083545+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.090413+0000","last_change":"2023-08-27T11:28:49.087457+0000","last_active":"2023-08-27T11:28:49.090413+0000","last_peered":"2023-08-27T11:28:49.090413+0000","last_clean":"2023-08-27T11:28:49.090413+0000","last_became_active":"2023-08-27T11:28:49.087210+0000","last_became_peered":"2023-08-27T11:28:49.087210+0000","last_unstale":"2023-08-27T11:28:49.090413+0000","last_undegraded":"2023-08-27T11:28:49.090413+0000","last_fullsized":"2023-08-27T11:28:49.090413+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:50:00.915296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084549+0000","last_change":"2023-08-27T11:28:49.084549+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084549+0000","last_undegraded":"2023-08-27T11:28:49.084549+0000","last_fullsized":"2023-08-27T11:28:49.084549+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.093549+0000","last_change":"2023-08-27T11:28:49.089014+0000","last_active":"2023-08-27T11:28:49.093549+0000","last_peered":"2023-08-27T11:28:49.093549+0000","last_clean":"2023-08-27T11:28:49.093549+0000","last_became_active":"2023-08-27T11:28:49.088750+0000","last_became_peered":"2023-08-27T11:28:49.088750+0000","last_unstale":"2023-08-27T11:28:49.093549+0000","last_undegraded":"2023-08-27T11:28:49.093549+0000","last_fullsized":"2023-08-27T11:28:49.093549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:12.555970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082603+0000","last_change":"2023-08-27T11:28:49.082603+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082603+0000","last_undegraded":"2023-08-27T11:28:49.082603+0000","last_fullsized":"2023-08-27T11:28:49.082603+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097693+0000","last_change":"2023-08-27T11:28:49.093294+0000","last_active":"2023-08-27T11:28:49.097693+0000","last_peered":"2023-08-27T11:28:49.097693+0000","last_clean":"2023-08-27T11:28:49.097693+0000","last_became_active":"2023-08-27T11:28:49.092967+0000","last_became_peered":"2023-08-27T11:28:49.092967+0000","last_unstale":"2023-08-27T11:28:49.097693+0000","last_undegraded":"2023-08-27T11:28:49.097693+0000","last_fullsized":"2023-08-27T11:28:49.097693+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:34:30.559346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085066+0000","last_change":"2023-08-27T11:28:49.085066+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085066+0000","last_undegraded":"2023-08-27T11:28:49.085066+0000","last_fullsized":"2023-08-27T11:28:49.085066+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097072+0000","last_change":"2023-08-27T11:28:49.095274+0000","last_active":"2023-08-27T11:28:49.097072+0000","last_peered":"2023-08-27T11:28:49.097072+0000","last_clean":"2023-08-27T11:28:49.097072+0000","last_became_active":"2023-08-27T11:28:49.095022+0000","last_became_peered":"2023-08-27T11:28:49.095022+0000","last_unstale":"2023-08-27T11:28:49.097072+0000","last_undegraded":"2023-08-27T11:28:49.097072+0000","last_fullsized":"2023-08-27T11:28:49.097072+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:20.172740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.090393+0000","last_change":"2023-08-27T11:28:49.090393+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.090393+0000","last_undegraded":"2023-08-27T11:28:49.090393+0000","last_fullsized":"2023-08-27T11:28:49.090393+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.091984+0000","last_change":"2023-08-27T11:28:49.087250+0000","last_active":"2023-08-27T11:28:49.091984+0000","last_peered":"2023-08-27T11:28:49.091984+0000","last_clean":"2023-08-27T11:28:49.091984+0000","last_became_active":"2023-08-27T11:28:49.087002+0000","last_became_peered":"2023-08-27T11:28:49.087002+0000","last_unstale":"2023-08-27T11:28:49.091984+0000","last_undegraded":"2023-08-27T11:28:49.091984+0000","last_fullsized":"2023-08-27T11:28:49.091984+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:29:52.621160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079943+0000","last_change":"2023-08-27T11:28:49.079943+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079943+0000","last_undegraded":"2023-08-27T11:28:49.079943+0000","last_fullsized":"2023-08-27T11:28:49.079943+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.091510+0000","last_change":"2023-08-27T11:28:49.086306+0000","last_active":"2023-08-27T11:28:49.091510+0000","last_peered":"2023-08-27T11:28:49.091510+0000","last_clean":"2023-08-27T11:28:49.091510+0000","last_became_active":"2023-08-27T11:28:49.086033+0000","last_became_peered":"2023-08-27T11:28:49.086033+0000","last_unstale":"2023-08-27T11:28:49.091510+0000","last_undegraded":"2023-08-27T11:28:49.091510+0000","last_fullsized":"2023-08-27T11:28:49.091510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:13:39.364173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080758+0000","last_change":"2023-08-27T11:28:49.080758+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080758+0000","last_undegraded":"2023-08-27T11:28:49.080758+0000","last_fullsized":"2023-08-27T11:28:49.080758+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.091231+0000","last_change":"2023-08-27T11:28:49.088909+0000","last_active":"2023-08-27T11:28:49.091231+0000","last_peered":"2023-08-27T11:28:49.091231+0000","last_clean":"2023-08-27T11:28:49.091231+0000","last_became_active":"2023-08-27T11:28:49.088497+0000","last_became_peered":"2023-08-27T11:28:49.088497+0000","last_unstale":"2023-08-27T11:28:49.091231+0000","last_undegraded":"2023-08-27T11:28:49.091231+0000","last_fullsized":"2023-08-27T11:28:49.091231+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:39:46.497129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082660+0000","last_change":"2023-08-27T11:28:49.082660+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082660+0000","last_undegraded":"2023-08-27T11:28:49.082660+0000","last_fullsized":"2023-08-27T11:28:49.082660+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092067+0000","last_change":"2023-08-27T11:28:49.087910+0000","last_active":"2023-08-27T11:28:49.092067+0000","last_peered":"2023-08-27T11:28:49.092067+0000","last_clean":"2023-08-27T11:28:49.092067+0000","last_became_active":"2023-08-27T11:28:49.087656+0000","last_became_peered":"2023-08-27T11:28:49.087656+0000","last_unstale":"2023-08-27T11:28:49.092067+0000","last_undegraded":"2023-08-27T11:28:49.092067+0000","last_fullsized":"2023-08-27T11:28:49.092067+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:27:25.771051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079616+0000","last_change":"2023-08-27T11:28:49.079616+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079616+0000","last_undegraded":"2023-08-27T11:28:49.079616+0000","last_fullsized":"2023-08-27T11:28:49.079616+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.094978+0000","last_change":"2023-08-27T11:28:49.092647+0000","last_active":"2023-08-27T11:28:49.094978+0000","last_peered":"2023-08-27T11:28:49.094978+0000","last_clean":"2023-08-27T11:28:49.094978+0000","last_became_active":"2023-08-27T11:28:49.092425+0000","last_became_peered":"2023-08-27T11:28:49.092425+0000","last_unstale":"2023-08-27T11:28:49.094978+0000","last_undegraded":"2023-08-27T11:28:49.094978+0000","last_fullsized":"2023-08-27T11:28:49.094978+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:07:45.451239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084132+0000","last_change":"2023-08-27T11:28:49.084132+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084132+0000","last_undegraded":"2023-08-27T11:28:49.084132+0000","last_fullsized":"2023-08-27T11:28:49.084132+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092980+0000","last_change":"2023-08-27T11:28:49.090840+0000","last_active":"2023-08-27T11:28:49.092980+0000","last_peered":"2023-08-27T11:28:49.092980+0000","last_clean":"2023-08-27T11:28:49.092980+0000","last_became_active":"2023-08-27T11:28:49.090655+0000","last_became_peered":"2023-08-27T11:28:49.090655+0000","last_unstale":"2023-08-27T11:28:49.092980+0000","last_undegraded":"2023-08-27T11:28:49.092980+0000","last_fullsized":"2023-08-27T11:28:49.092980+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:03.433230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084933+0000","last_change":"2023-08-27T11:28:49.084933+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084933+0000","last_undegraded":"2023-08-27T11:28:49.084933+0000","last_fullsized":"2023-08-27T11:28:49.084933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092990+0000","last_change":"2023-08-27T11:28:49.088886+0000","last_active":"2023-08-27T11:28:49.092990+0000","last_peered":"2023-08-27T11:28:49.092990+0000","last_clean":"2023-08-27T11:28:49.092990+0000","last_became_active":"2023-08-27T11:28:49.088618+0000","last_became_peered":"2023-08-27T11:28:49.088618+0000","last_unstale":"2023-08-27T11:28:49.092990+0000","last_undegraded":"2023-08-27T11:28:49.092990+0000","last_fullsized":"2023-08-27T11:28:49.092990+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:45:46.741280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.081075+0000","last_change":"2023-08-27T11:28:49.081075+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.081075+0000","last_undegraded":"2023-08-27T11:28:49.081075+0000","last_fullsized":"2023-08-27T11:28:49.081075+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103821+0000","last_change":"2023-08-27T11:28:49.098450+0000","last_active":"2023-08-27T11:28:49.103821+0000","last_peered":"2023-08-27T11:28:49.103821+0000","last_clean":"2023-08-27T11:28:49.103821+0000","last_became_active":"2023-08-27T11:28:49.098071+0000","last_became_peered":"2023-08-27T11:28:49.098071+0000","last_unstale":"2023-08-27T11:28:49.103821+0000","last_undegraded":"2023-08-27T11:28:49.103821+0000","last_fullsized":"2023-08-27T11:28:49.103821+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:05.557628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084983+0000","last_change":"2023-08-27T11:28:49.084983+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084983+0000","last_undegraded":"2023-08-27T11:28:49.084983+0000","last_fullsized":"2023-08-27T11:28:49.084983+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097260+0000","last_change":"2023-08-27T11:28:49.094459+0000","last_active":"2023-08-27T11:28:49.097260+0000","last_peered":"2023-08-27T11:28:49.097260+0000","last_clean":"2023-08-27T11:28:49.097260+0000","last_became_active":"2023-08-27T11:28:49.094215+0000","last_became_peered":"2023-08-27T11:28:49.094215+0000","last_unstale":"2023-08-27T11:28:49.097260+0000","last_undegraded":"2023-08-27T11:28:49.097260+0000","last_fullsized":"2023-08-27T11:28:49.097260+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:27:04.328482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.077745+0000","last_change":"2023-08-27T11:28:49.077745+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.077745+0000","last_undegraded":"2023-08-27T11:28:49.077745+0000","last_fullsized":"2023-08-27T11:28:49.077745+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.102914+0000","last_change":"2023-08-27T11:28:49.091063+0000","last_active":"2023-08-27T11:28:49.102914+0000","last_peered":"2023-08-27T11:28:49.102914+0000","last_clean":"2023-08-27T11:28:49.102914+0000","last_became_active":"2023-08-27T11:28:49.090005+0000","last_became_peered":"2023-08-27T11:28:49.090005+0000","last_unstale":"2023-08-27T11:28:49.102914+0000","last_undegraded":"2023-08-27T11:28:49.102914+0000","last_fullsized":"2023-08-27T11:28:49.102914+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:37.416680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.082576+0000","last_change":"2023-08-27T11:28:49.082576+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.082576+0000","last_undegraded":"2023-08-27T11:28:49.082576+0000","last_fullsized":"2023-08-27T11:28:49.082576+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092298+0000","last_change":"2023-08-27T11:28:49.089703+0000","last_active":"2023-08-27T11:28:49.092298+0000","last_peered":"2023-08-27T11:28:49.092298+0000","last_clean":"2023-08-27T11:28:49.092298+0000","last_became_active":"2023-08-27T11:28:49.089514+0000","last_became_peered":"2023-08-27T11:28:49.089514+0000","last_unstale":"2023-08-27T11:28:49.092298+0000","last_undegraded":"2023-08-27T11:28:49.092298+0000","last_fullsized":"2023-08-27T11:28:49.092298+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:01:44.852062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.077759+0000","last_change":"2023-08-27T11:28:49.077759+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.077759+0000","last_undegraded":"2023-08-27T11:28:49.077759+0000","last_fullsized":"2023-08-27T11:28:49.077759+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.089958+0000","last_change":"2023-08-27T11:28:49.087175+0000","last_active":"2023-08-27T11:28:49.089958+0000","last_peered":"2023-08-27T11:28:49.089958+0000","last_clean":"2023-08-27T11:28:49.089958+0000","last_became_active":"2023-08-27T11:28:49.086854+0000","last_became_peered":"2023-08-27T11:28:49.086854+0000","last_unstale":"2023-08-27T11:28:49.089958+0000","last_undegraded":"2023-08-27T11:28:49.089958+0000","last_fullsized":"2023-08-27T11:28:49.089958+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:19.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.085540+0000","last_change":"2023-08-27T11:28:49.085540+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.085540+0000","last_undegraded":"2023-08-27T11:28:49.085540+0000","last_fullsized":"2023-08-27T11:28:49.085540+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.097152+0000","last_change":"2023-08-27T11:28:49.096229+0000","last_active":"2023-08-27T11:28:49.097152+0000","last_peered":"2023-08-27T11:28:49.097152+0000","last_clean":"2023-08-27T11:28:49.097152+0000","last_became_active":"2023-08-27T11:28:49.095951+0000","last_became_peered":"2023-08-27T11:28:49.095951+0000","last_unstale":"2023-08-27T11:28:49.097152+0000","last_undegraded":"2023-08-27T11:28:49.097152+0000","last_fullsized":"2023-08-27T11:28:49.097152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:37:05.774883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.086269+0000","last_change":"2023-08-27T11:28:49.086269+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.086269+0000","last_undegraded":"2023-08-27T11:28:49.086269+0000","last_fullsized":"2023-08-27T11:28:49.086269+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103699+0000","last_change":"2023-08-27T11:28:49.095920+0000","last_active":"2023-08-27T11:28:49.103699+0000","last_peered":"2023-08-27T11:28:49.103699+0000","last_clean":"2023-08-27T11:28:49.103699+0000","last_became_active":"2023-08-27T11:28:49.095652+0000","last_became_peered":"2023-08-27T11:28:49.095652+0000","last_unstale":"2023-08-27T11:28:49.103699+0000","last_undegraded":"2023-08-27T11:28:49.103699+0000","last_fullsized":"2023-08-27T11:28:49.103699+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:59.514247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":193,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.078437+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:49.078437+0000","last_peered":"2023-08-27T11:28:49.078437+0000","last_clean":"2023-08-27T11:28:49.078437+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:49.078437+0000","last_undegraded":"2023-08-27T11:28:49.078437+0000","last_fullsized":"2023-08-27T11:28:49.078437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:15:19.298171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.086169+0000","last_change":"2023-08-27T11:28:49.086169+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.086169+0000","last_undegraded":"2023-08-27T11:28:49.086169+0000","last_fullsized":"2023-08-27T11:28:49.086169+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.094004+0000","last_change":"2023-08-27T11:28:49.092776+0000","last_active":"2023-08-27T11:28:49.094004+0000","last_peered":"2023-08-27T11:28:49.094004+0000","last_clean":"2023-08-27T11:28:49.094004+0000","last_became_active":"2023-08-27T11:28:49.092503+0000","last_became_peered":"2023-08-27T11:28:49.092503+0000","last_unstale":"2023-08-27T11:28:49.094004+0000","last_undegraded":"2023-08-27T11:28:49.094004+0000","last_fullsized":"2023-08-27T11:28:49.094004+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:09:41.137920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":195,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.084685+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:49.084685+0000","last_peered":"2023-08-27T11:28:49.084685+0000","last_clean":"2023-08-27T11:28:49.084685+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:49.084685+0000","last_undegraded":"2023-08-27T11:28:49.084685+0000","last_fullsized":"2023-08-27T11:28:49.084685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:17:55.415756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.084016+0000","last_change":"2023-08-27T11:28:49.084016+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.084016+0000","last_undegraded":"2023-08-27T11:28:49.084016+0000","last_fullsized":"2023-08-27T11:28:49.084016+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092250+0000","last_change":"2023-08-27T11:28:49.088492+0000","last_active":"2023-08-27T11:28:49.092250+0000","last_peered":"2023-08-27T11:28:49.092250+0000","last_clean":"2023-08-27T11:28:49.092250+0000","last_became_active":"2023-08-27T11:28:49.088156+0000","last_became_peered":"2023-08-27T11:28:49.088156+0000","last_unstale":"2023-08-27T11:28:49.092250+0000","last_undegraded":"2023-08-27T11:28:49.092250+0000","last_fullsized":"2023-08-27T11:28:49.092250+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:11:26.876964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":183,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.075434+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:49.075434+0000","last_peered":"2023-08-27T11:28:49.075434+0000","last_clean":"2023-08-27T11:28:49.075434+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:49.075434+0000","last_undegraded":"2023-08-27T11:28:49.075434+0000","last_fullsized":"2023-08-27T11:28:49.075434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:20:55.742674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079769+0000","last_change":"2023-08-27T11:28:49.079769+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079769+0000","last_undegraded":"2023-08-27T11:28:49.079769+0000","last_fullsized":"2023-08-27T11:28:49.079769+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.098655+0000","last_change":"2023-08-27T11:28:49.098322+0000","last_active":"2023-08-27T11:28:49.098655+0000","last_peered":"2023-08-27T11:28:49.098655+0000","last_clean":"2023-08-27T11:28:49.098655+0000","last_became_active":"2023-08-27T11:28:49.098035+0000","last_became_peered":"2023-08-27T11:28:49.098035+0000","last_unstale":"2023-08-27T11:28:49.098655+0000","last_undegraded":"2023-08-27T11:28:49.098655+0000","last_fullsized":"2023-08-27T11:28:49.098655+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:21:27.504700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":195,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.083440+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:49.083440+0000","last_peered":"2023-08-27T11:28:49.083440+0000","last_clean":"2023-08-27T11:28:49.083440+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:49.083440+0000","last_undegraded":"2023-08-27T11:28:49.083440+0000","last_fullsized":"2023-08-27T11:28:49.083440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:57:44.263129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"113'5183","reported_seq":5642,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.700641+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:49.700641+0000","last_peered":"2023-08-27T11:28:49.700641+0000","last_clean":"2023-08-27T11:28:49.700641+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:49.700641+0000","last_undegraded":"2023-08-27T11:28:49.700641+0000","last_fullsized":"2023-08-27T11:28:49.700641+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":5183,"log_dups_size":0,"ondisk_log_size":5183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:36:49.306768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":761,"num_read_kb":1467,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.089647+0000","last_change":"2023-08-27T11:28:49.089647+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.089647+0000","last_undegraded":"2023-08-27T11:28:49.089647+0000","last_fullsized":"2023-08-27T11:28:49.089647+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.103818+0000","last_change":"2023-08-27T11:28:49.098393+0000","last_active":"2023-08-27T11:28:49.103818+0000","last_peered":"2023-08-27T11:28:49.103818+0000","last_clean":"2023-08-27T11:28:49.103818+0000","last_became_active":"2023-08-27T11:28:49.098104+0000","last_became_peered":"2023-08-27T11:28:49.098104+0000","last_unstale":"2023-08-27T11:28:49.103818+0000","last_undegraded":"2023-08-27T11:28:49.103818+0000","last_fullsized":"2023-08-27T11:28:49.103818+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:14.479332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":195,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092502+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:49.092502+0000","last_peered":"2023-08-27T11:28:49.092502+0000","last_clean":"2023-08-27T11:28:49.092502+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:49.092502+0000","last_undegraded":"2023-08-27T11:28:49.092502+0000","last_fullsized":"2023-08-27T11:28:49.092502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:06:07.246346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.078553+0000","last_change":"2023-08-27T11:28:49.078553+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.078553+0000","last_undegraded":"2023-08-27T11:28:49.078553+0000","last_fullsized":"2023-08-27T11:28:49.078553+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092559+0000","last_change":"2023-08-27T11:28:49.090592+0000","last_active":"2023-08-27T11:28:49.092559+0000","last_peered":"2023-08-27T11:28:49.092559+0000","last_clean":"2023-08-27T11:28:49.092559+0000","last_became_active":"2023-08-27T11:28:49.090309+0000","last_became_peered":"2023-08-27T11:28:49.090309+0000","last_unstale":"2023-08-27T11:28:49.092559+0000","last_undegraded":"2023-08-27T11:28:49.092559+0000","last_fullsized":"2023-08-27T11:28:49.092559+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:29:21.154979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":194,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092856+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:49.092856+0000","last_peered":"2023-08-27T11:28:49.092856+0000","last_clean":"2023-08-27T11:28:49.092856+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:49.092856+0000","last_undegraded":"2023-08-27T11:28:49.092856+0000","last_fullsized":"2023-08-27T11:28:49.092856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:29:19.522512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.080372+0000","last_change":"2023-08-27T11:28:49.080372+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.080372+0000","last_undegraded":"2023-08-27T11:28:49.080372+0000","last_fullsized":"2023-08-27T11:28:49.080372+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.092543+0000","last_change":"2023-08-27T11:28:49.089867+0000","last_active":"2023-08-27T11:28:49.092543+0000","last_peered":"2023-08-27T11:28:49.092543+0000","last_clean":"2023-08-27T11:28:49.092543+0000","last_became_active":"2023-08-27T11:28:49.089621+0000","last_became_peered":"2023-08-27T11:28:49.089621+0000","last_unstale":"2023-08-27T11:28:49.092543+0000","last_undegraded":"2023-08-27T11:28:49.092543+0000","last_fullsized":"2023-08-27T11:28:49.092543+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:36:03.921396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.078190+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:49.078190+0000","last_peered":"2023-08-27T11:28:49.078190+0000","last_clean":"2023-08-27T11:28:49.078190+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:49.078190+0000","last_undegraded":"2023-08-27T11:28:49.078190+0000","last_fullsized":"2023-08-27T11:28:49.078190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:45:30.750930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":113,"state":"creating+peering","last_fresh":"2023-08-27T11:28:49.079866+0000","last_change":"2023-08-27T11:28:49.079866+0000","last_active":"2023-08-27T11:28:49.069412+0000","last_peered":"2023-08-27T11:28:49.069412+0000","last_clean":"2023-08-27T11:28:49.069412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-27T11:28:49.079866+0000","last_undegraded":"2023-08-27T11:28:49.079866+0000","last_fullsized":"2023-08-27T11:28:49.079866+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.102888+0000","last_change":"2023-08-27T11:28:49.089632+0000","last_active":"2023-08-27T11:28:49.102888+0000","last_peered":"2023-08-27T11:28:49.102888+0000","last_clean":"2023-08-27T11:28:49.102888+0000","last_became_active":"2023-08-27T11:28:49.089329+0000","last_became_peered":"2023-08-27T11:28:49.089329+0000","last_unstale":"2023-08-27T11:28:49.102888+0000","last_undegraded":"2023-08-27T11:28:49.102888+0000","last_fullsized":"2023-08-27T11:28:49.102888+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:50:24.495137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-08-27T11:28:49.088916+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:49.088916+0000","last_peered":"2023-08-27T11:28:49.088916+0000","last_clean":"2023-08-27T11:28:49.088916+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:49.088916+0000","last_undegraded":"2023-08-27T11:28:49.088916+0000","last_fullsized":"2023-08-27T11:28:49.088916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:47:44.825080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":761,"num_read_kb":1467,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5183,"ondisk_log_size":5183,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1986560,"data_stored":1322060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.53900000000000003,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66000000000000003,"5min":1.601,"15min":1.601},"last":0.247},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.72799999999999998,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.38100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.53500000000000003,"15min":0.56899999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.55100000000000005,"15min":0.58299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42899999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":0.60199999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59499999999999997,"15min":0.629},"min":{"1min":0.32000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":1.778,"15min":1.778},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58199999999999996,"15min":0.62},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.75900000000000001,"5min":0.78100000000000003,"15min":0.82999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.61099999999999999,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82299999999999995,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.51500000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.72899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.35699999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.47199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88500000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.5}]},{"osd":6,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.442}]}]},{"osd":6,"up_from":13,"seq":55834575048,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56799999999999995,"15min":0.60199999999999998},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74299999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.1679999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":10.992000000000001,"5min":10.992000000000001,"15min":10.992000000000001},"last":0.45100000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.624,"5min":0.78800000000000003,"15min":0.82599999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.64100000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52600000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.67600000000000005,"15min":0.624},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":11.019,"5min":11.019,"15min":11.019},"last":0.376},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53900000000000003,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59199999999999997,"5min":0.77200000000000002,"15min":0.83399999999999996},"last":0.42899999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54600000000000004,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60999999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.161,"5min":0.71099999999999997,"15min":0.64400000000000002},"min":{"1min":0.36899999999999999,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":10.956,"5min":10.956,"15min":10.956},"last":0.51700000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63800000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.622,"15min":0.53500000000000003},"min":{"1min":0.33400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.069000000000001,"5min":11.069000000000001,"15min":11.069000000000001},"last":0.497}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.31900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56599999999999995,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.65500000000000003,"5min":1.157,"15min":1.157},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":0.64700000000000002,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":11.045,"5min":11.045,"15min":11.045},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.36899999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.50700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":1380352,"data_stored":724700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.371,"15min":0.38300000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.20399999999999999},"max":{"1min":0.5,"5min":0.53500000000000003,"15min":0.65600000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.27800000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.41099999999999998,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.32900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.63300000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45900000000000002,"15min":0.48199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67000000000000004,"5min":1.169,"15min":1.169},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.439,"15min":0.439},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.23400000000000001},"max":{"1min":0.53000000000000003,"5min":1.139,"15min":1.139},"last":0.48499999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.621,"15min":0.623},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.88600000000000001,"15min":2.9380000000000002},"last":0.55300000000000005}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.625,"15min":0.63800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64300000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39100000000000001},"max":{"1min":0.77100000000000002,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.53600000000000003}]},{"osd":6,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.64800000000000002,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69499999999999995,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.59999999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.752,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575048,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1986560,"data_stored":1322060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61099999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55400000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35899999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40699999999999997,"15min":0.41399999999999998},"min":{"1min":0.22900000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.495,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.23599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.46700000000000003,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.29599999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42099999999999999,"15min":0.42599999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63900000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.81100000000000005},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60899999999999999,"15min":0.63800000000000001},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.86599999999999999,"15min":1.202},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.91200000000000003},"last":0.59499999999999997}]},{"osd":5,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.96199999999999997,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.63500000000000001,"15min":0.66700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.248},"last":0.67100000000000004}]},{"osd":6,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63800000000000001,"15min":0.65400000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.98699999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0189999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76300000000000001,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.62},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.65500000000000003,"15min":0.69199999999999995},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86699999999999999,"5min":0.90700000000000003,"15min":1.2809999999999999},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575049,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36599999999999999,"15min":0.372},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.44600000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.35199999999999998}]},{"osd":1,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47099999999999997,"5min":0.60899999999999999,"15min":0.753},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.55500000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.308}]},{"osd":3,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52900000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.432,"15min":0.44900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.498,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.373}]},{"osd":4,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.60599999999999998,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":1.163},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.436}]},{"osd":5,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.36499999999999999},"max":{"1min":1.034,"5min":1.083,"15min":1.2470000000000001},"last":0.502},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40699999999999997},"max":{"1min":0.879,"5min":0.96799999999999997,"15min":1.2330000000000001},"last":0.46999999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81599999999999995,"5min":0.99299999999999999,"15min":1.1319999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64600000000000002,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.45100000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.65400000000000003,"15min":0.67800000000000005},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.68899999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.99299999999999999,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1380352,"data_stored":724700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.39100000000000001,"15min":0.38800000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.22},"max":{"1min":0.52100000000000002,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.38900000000000001,"15min":0.40899999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.42299999999999999,"5min":0.61299999999999999,"15min":0.71399999999999997},"last":0.22900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.38500000000000001,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.71999999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.439,"15min":0.46000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.245},"max":{"1min":0.53800000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.252}]},{"osd":2,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44700000000000001,"15min":0.46400000000000002},"min":{"1min":0.33200000000000002,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.49399999999999999,"5min":0.71199999999999997,"15min":0.93100000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67600000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.29999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72899999999999998,"5min":0.99399999999999999,"15min":1.0329999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78800000000000003,"5min":0.91000000000000003,"15min":1.113},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.37,"15min":0.37},"max":{"1min":0.872,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":1.0620000000000001,"15min":1.218},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65300000000000002,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":0.97199999999999998,"15min":1.2010000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.68799999999999994,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":1.0269999999999999,"15min":1.0389999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65000000000000002,"15min":0.67900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.93799999999999994,"15min":1.006},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70899999999999996,"5min":1.0389999999999999,"15min":1.0640000000000001},"last":0.48999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575048,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55900000000000005,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.255},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.873},"last":0.376}]},{"osd":1,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.76200000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.85299999999999998},"last":0.28999999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42999999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77000000000000002,"5min":0.879,"15min":0.879},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.629,"15min":0.627},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.88},"last":0.45400000000000001}]},{"osd":5,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.755,"15min":0.85699999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.26200000000000001},"max":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.82799999999999996},"last":0.39500000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.73299999999999998,"15min":1.0860000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.747,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.46700000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.874},"last":0.499},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.1339999999999999},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575047,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1392640,"data_stored":731692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.58399999999999996},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57599999999999996,"15min":0.59999999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58399999999999996}]},{"osd":1,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.62}]},{"osd":2,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"last":0.52300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32000000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.46899999999999997}]},{"osd":4,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.23100000000000001},"max":{"1min":0.60699999999999998,"5min":1.157,"15min":1.157},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.61299999999999999,"5min":0.76700000000000002,"15min":0.86599999999999999},"last":0.30199999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70699999999999996,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.34300000000000003}]},{"osd":7,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.27800000000000002},"max":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.503}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:55.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2023-08-27T11:28:55.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.085340+0000', 'last_change': '2023-08-27T11:28:49.085340+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.085340+0000', 'last_undegraded': '2023-08-27T11:28:49.085340+0000', 'last_fullsized': '2023-08-27T11:28:49.085340+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:55.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2023-08-27T11:28:55.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.092601+0000', 'last_change': '2023-08-27T11:28:49.092601+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.092601+0000', 'last_undegraded': '2023-08-27T11:28:49.092601+0000', 'last_fullsized': '2023-08-27T11:28:49.092601+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:55.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2023-08-27T11:28:55.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.079160+0000', 'last_change': '2023-08-27T11:28:49.079160+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.079160+0000', 'last_undegraded': '2023-08-27T11:28:49.079160+0000', 'last_fullsized': '2023-08-27T11:28:49.079160+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:55.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2023-08-27T11:28:55.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080168+0000', 'last_change': '2023-08-27T11:28:49.080168+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080168+0000', 'last_undegraded': '2023-08-27T11:28:49.080168+0000', 'last_fullsized': '2023-08-27T11:28:49.080168+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:55.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2023-08-27T11:28:55.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.085503+0000', 'last_change': '2023-08-27T11:28:49.085503+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.085503+0000', 'last_undegraded': '2023-08-27T11:28:49.085503+0000', 'last_fullsized': '2023-08-27T11:28:49.085503+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:55.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2023-08-27T11:28:55.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.091634+0000', 'last_change': '2023-08-27T11:28:49.091634+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.091634+0000', 'last_undegraded': '2023-08-27T11:28:49.091634+0000', 'last_fullsized': '2023-08-27T11:28:49.091634+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:55.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2023-08-27T11:28:55.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.089205+0000', 'last_change': '2023-08-27T11:28:49.089205+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.089205+0000', 'last_undegraded': '2023-08-27T11:28:49.089205+0000', 'last_fullsized': '2023-08-27T11:28:49.089205+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:55.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2023-08-27T11:28:55.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.091503+0000', 'last_change': '2023-08-27T11:28:49.091503+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.091503+0000', 'last_undegraded': '2023-08-27T11:28:49.091503+0000', 'last_fullsized': '2023-08-27T11:28:49.091503+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:55.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2023-08-27T11:28:55.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.083331+0000', 'last_change': '2023-08-27T11:28:49.083331+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.083331+0000', 'last_undegraded': '2023-08-27T11:28:49.083331+0000', 'last_fullsized': '2023-08-27T11:28:49.083331+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:55.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2023-08-27T11:28:55.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084616+0000', 'last_change': '2023-08-27T11:28:49.084616+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084616+0000', 'last_undegraded': '2023-08-27T11:28:49.084616+0000', 'last_fullsized': '2023-08-27T11:28:49.084616+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:55.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2023-08-27T11:28:55.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.090012+0000', 'last_change': '2023-08-27T11:28:49.090012+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.090012+0000', 'last_undegraded': '2023-08-27T11:28:49.090012+0000', 'last_fullsized': '2023-08-27T11:28:49.090012+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:55.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2023-08-27T11:28:55.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.083870+0000', 'last_change': '2023-08-27T11:28:49.083870+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.083870+0000', 'last_undegraded': '2023-08-27T11:28:49.083870+0000', 'last_fullsized': '2023-08-27T11:28:49.083870+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:55.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2023-08-27T11:28:55.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.079983+0000', 'last_change': '2023-08-27T11:28:49.079983+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.079983+0000', 'last_undegraded': '2023-08-27T11:28:49.079983+0000', 'last_fullsized': '2023-08-27T11:28:49.079983+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:55.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2023-08-27T11:28:55.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080749+0000', 'last_change': '2023-08-27T11:28:49.080749+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080749+0000', 'last_undegraded': '2023-08-27T11:28:49.080749+0000', 'last_fullsized': '2023-08-27T11:28:49.080749+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2023-08-27T11:28:55.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080853+0000', 'last_change': '2023-08-27T11:28:49.080853+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080853+0000', 'last_undegraded': '2023-08-27T11:28:49.080853+0000', 'last_fullsized': '2023-08-27T11:28:49.080853+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:55.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2023-08-27T11:28:55.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.090451+0000', 'last_change': '2023-08-27T11:28:49.090451+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.090451+0000', 'last_undegraded': '2023-08-27T11:28:49.090451+0000', 'last_fullsized': '2023-08-27T11:28:49.090451+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:55.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2023-08-27T11:28:55.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.090009+0000', 'last_change': '2023-08-27T11:28:49.090009+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.090009+0000', 'last_undegraded': '2023-08-27T11:28:49.090009+0000', 'last_fullsized': '2023-08-27T11:28:49.090009+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:55.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2023-08-27T11:28:55.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080580+0000', 'last_change': '2023-08-27T11:28:49.080580+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080580+0000', 'last_undegraded': '2023-08-27T11:28:49.080580+0000', 'last_fullsized': '2023-08-27T11:28:49.080580+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:55.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2023-08-27T11:28:55.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.079857+0000', 'last_change': '2023-08-27T11:28:49.079857+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.079857+0000', 'last_undegraded': '2023-08-27T11:28:49.079857+0000', 'last_fullsized': '2023-08-27T11:28:49.079857+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2023-08-27T11:28:55.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.083451+0000', 'last_change': '2023-08-27T11:28:49.083451+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.083451+0000', 'last_undegraded': '2023-08-27T11:28:49.083451+0000', 'last_fullsized': '2023-08-27T11:28:49.083451+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:55.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2023-08-27T11:28:55.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080787+0000', 'last_change': '2023-08-27T11:28:49.080787+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080787+0000', 'last_undegraded': '2023-08-27T11:28:49.080787+0000', 'last_fullsized': '2023-08-27T11:28:49.080787+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2023-08-27T11:28:55.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.091564+0000', 'last_change': '2023-08-27T11:28:49.091564+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.091564+0000', 'last_undegraded': '2023-08-27T11:28:49.091564+0000', 'last_fullsized': '2023-08-27T11:28:49.091564+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:55.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2023-08-27T11:28:55.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084393+0000', 'last_change': '2023-08-27T11:28:49.084393+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084393+0000', 'last_undegraded': '2023-08-27T11:28:49.084393+0000', 'last_fullsized': '2023-08-27T11:28:49.084393+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:55.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2023-08-27T11:28:55.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.090030+0000', 'last_change': '2023-08-27T11:28:49.090030+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.090030+0000', 'last_undegraded': '2023-08-27T11:28:49.090030+0000', 'last_fullsized': '2023-08-27T11:28:49.090030+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:55.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2023-08-27T11:28:55.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.086685+0000', 'last_change': '2023-08-27T11:28:49.086685+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.086685+0000', 'last_undegraded': '2023-08-27T11:28:49.086685+0000', 'last_fullsized': '2023-08-27T11:28:49.086685+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:55.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2023-08-27T11:28:55.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084953+0000', 'last_change': '2023-08-27T11:28:49.084953+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084953+0000', 'last_undegraded': '2023-08-27T11:28:49.084953+0000', 'last_fullsized': '2023-08-27T11:28:49.084953+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:55.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2023-08-27T11:28:55.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.089211+0000', 'last_change': '2023-08-27T11:28:49.089211+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.089211+0000', 'last_undegraded': '2023-08-27T11:28:49.089211+0000', 'last_fullsized': '2023-08-27T11:28:49.089211+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:55.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2023-08-27T11:28:55.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.085408+0000', 'last_change': '2023-08-27T11:28:49.085408+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.085408+0000', 'last_undegraded': '2023-08-27T11:28:49.085408+0000', 'last_fullsized': '2023-08-27T11:28:49.085408+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:55.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2023-08-27T11:28:55.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084163+0000', 'last_change': '2023-08-27T11:28:49.084163+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084163+0000', 'last_undegraded': '2023-08-27T11:28:49.084163+0000', 'last_fullsized': '2023-08-27T11:28:49.084163+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:55.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2023-08-27T11:28:55.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080300+0000', 'last_change': '2023-08-27T11:28:49.080300+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080300+0000', 'last_undegraded': '2023-08-27T11:28:49.080300+0000', 'last_fullsized': '2023-08-27T11:28:49.080300+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2023-08-27T11:28:55.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080473+0000', 'last_change': '2023-08-27T11:28:49.080473+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080473+0000', 'last_undegraded': '2023-08-27T11:28:49.080473+0000', 'last_fullsized': '2023-08-27T11:28:49.080473+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:55.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2023-08-27T11:28:55.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.081947+0000', 'last_change': '2023-08-27T11:28:49.081947+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.081947+0000', 'last_undegraded': '2023-08-27T11:28:49.081947+0000', 'last_fullsized': '2023-08-27T11:28:49.081947+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:55.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-08-27T11:28:55.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080179+0000', 'last_change': '2023-08-27T11:28:49.080179+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080179+0000', 'last_undegraded': '2023-08-27T11:28:49.080179+0000', 'last_fullsized': '2023-08-27T11:28:49.080179+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-08-27T11:28:55.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.079935+0000', 'last_change': '2023-08-27T11:28:49.079935+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.079935+0000', 'last_undegraded': '2023-08-27T11:28:49.079935+0000', 'last_fullsized': '2023-08-27T11:28:49.079935+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:55.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2023-08-27T11:28:55.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084149+0000', 'last_change': '2023-08-27T11:28:49.084149+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084149+0000', 'last_undegraded': '2023-08-27T11:28:49.084149+0000', 'last_fullsized': '2023-08-27T11:28:49.084149+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:55.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-08-27T11:28:55.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.082739+0000', 'last_change': '2023-08-27T11:28:49.082739+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.082739+0000', 'last_undegraded': '2023-08-27T11:28:49.082739+0000', 'last_fullsized': '2023-08-27T11:28:49.082739+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:55.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-08-27T11:28:55.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.082760+0000', 'last_change': '2023-08-27T11:28:49.082760+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.082760+0000', 'last_undegraded': '2023-08-27T11:28:49.082760+0000', 'last_fullsized': '2023-08-27T11:28:49.082760+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:28:55.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-08-27T11:28:55.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.089263+0000', 'last_change': '2023-08-27T11:28:49.089263+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.089263+0000', 'last_undegraded': '2023-08-27T11:28:49.089263+0000', 'last_fullsized': '2023-08-27T11:28:49.089263+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:55.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-08-27T11:28:55.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080199+0000', 'last_change': '2023-08-27T11:28:49.080199+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080199+0000', 'last_undegraded': '2023-08-27T11:28:49.080199+0000', 'last_fullsized': '2023-08-27T11:28:49.080199+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:55.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-08-27T11:28:55.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.083545+0000', 'last_change': '2023-08-27T11:28:49.083545+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.083545+0000', 'last_undegraded': '2023-08-27T11:28:49.083545+0000', 'last_fullsized': '2023-08-27T11:28:49.083545+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:55.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-08-27T11:28:55.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084549+0000', 'last_change': '2023-08-27T11:28:49.084549+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084549+0000', 'last_undegraded': '2023-08-27T11:28:49.084549+0000', 'last_fullsized': '2023-08-27T11:28:49.084549+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:55.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-08-27T11:28:55.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.082603+0000', 'last_change': '2023-08-27T11:28:49.082603+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.082603+0000', 'last_undegraded': '2023-08-27T11:28:49.082603+0000', 'last_fullsized': '2023-08-27T11:28:49.082603+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:55.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-08-27T11:28:55.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.085066+0000', 'last_change': '2023-08-27T11:28:49.085066+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.085066+0000', 'last_undegraded': '2023-08-27T11:28:49.085066+0000', 'last_fullsized': '2023-08-27T11:28:49.085066+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:55.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-08-27T11:28:55.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.090393+0000', 'last_change': '2023-08-27T11:28:49.090393+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.090393+0000', 'last_undegraded': '2023-08-27T11:28:49.090393+0000', 'last_fullsized': '2023-08-27T11:28:49.090393+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:55.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-08-27T11:28:55.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.079943+0000', 'last_change': '2023-08-27T11:28:49.079943+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.079943+0000', 'last_undegraded': '2023-08-27T11:28:49.079943+0000', 'last_fullsized': '2023-08-27T11:28:49.079943+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-27T11:28:55.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2023-08-27T11:28:55.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080758+0000', 'last_change': '2023-08-27T11:28:49.080758+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080758+0000', 'last_undegraded': '2023-08-27T11:28:49.080758+0000', 'last_fullsized': '2023-08-27T11:28:49.080758+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:55.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-08-27T11:28:55.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.082660+0000', 'last_change': '2023-08-27T11:28:49.082660+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.082660+0000', 'last_undegraded': '2023-08-27T11:28:49.082660+0000', 'last_fullsized': '2023-08-27T11:28:49.082660+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:55.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-08-27T11:28:55.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.079616+0000', 'last_change': '2023-08-27T11:28:49.079616+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.079616+0000', 'last_undegraded': '2023-08-27T11:28:49.079616+0000', 'last_fullsized': '2023-08-27T11:28:49.079616+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-08-27T11:28:55.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084132+0000', 'last_change': '2023-08-27T11:28:49.084132+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084132+0000', 'last_undegraded': '2023-08-27T11:28:49.084132+0000', 'last_fullsized': '2023-08-27T11:28:49.084132+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-27T11:28:55.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2023-08-27T11:28:55.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084933+0000', 'last_change': '2023-08-27T11:28:49.084933+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084933+0000', 'last_undegraded': '2023-08-27T11:28:49.084933+0000', 'last_fullsized': '2023-08-27T11:28:49.084933+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:55.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2023-08-27T11:28:55.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.081075+0000', 'last_change': '2023-08-27T11:28:49.081075+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.081075+0000', 'last_undegraded': '2023-08-27T11:28:49.081075+0000', 'last_fullsized': '2023-08-27T11:28:49.081075+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-27T11:28:55.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-08-27T11:28:55.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084983+0000', 'last_change': '2023-08-27T11:28:49.084983+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084983+0000', 'last_undegraded': '2023-08-27T11:28:49.084983+0000', 'last_fullsized': '2023-08-27T11:28:49.084983+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:55.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-08-27T11:28:55.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.077745+0000', 'last_change': '2023-08-27T11:28:49.077745+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.077745+0000', 'last_undegraded': '2023-08-27T11:28:49.077745+0000', 'last_fullsized': '2023-08-27T11:28:49.077745+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-08-27T11:28:55.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.082576+0000', 'last_change': '2023-08-27T11:28:49.082576+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.082576+0000', 'last_undegraded': '2023-08-27T11:28:49.082576+0000', 'last_fullsized': '2023-08-27T11:28:49.082576+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-27T11:28:55.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-08-27T11:28:55.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.077759+0000', 'last_change': '2023-08-27T11:28:49.077759+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.077759+0000', 'last_undegraded': '2023-08-27T11:28:49.077759+0000', 'last_fullsized': '2023-08-27T11:28:49.077759+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-08-27T11:28:55.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.085540+0000', 'last_change': '2023-08-27T11:28:49.085540+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.085540+0000', 'last_undegraded': '2023-08-27T11:28:49.085540+0000', 'last_fullsized': '2023-08-27T11:28:49.085540+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:55.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-08-27T11:28:55.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.086269+0000', 'last_change': '2023-08-27T11:28:49.086269+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.086269+0000', 'last_undegraded': '2023-08-27T11:28:49.086269+0000', 'last_fullsized': '2023-08-27T11:28:49.086269+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:55.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-08-27T11:28:55.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.086169+0000', 'last_change': '2023-08-27T11:28:49.086169+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.086169+0000', 'last_undegraded': '2023-08-27T11:28:49.086169+0000', 'last_fullsized': '2023-08-27T11:28:49.086169+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-27T11:28:55.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-08-27T11:28:55.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.084016+0000', 'last_change': '2023-08-27T11:28:49.084016+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.084016+0000', 'last_undegraded': '2023-08-27T11:28:49.084016+0000', 'last_fullsized': '2023-08-27T11:28:49.084016+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-27T11:28:55.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-08-27T11:28:55.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.079769+0000', 'last_change': '2023-08-27T11:28:49.079769+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.079769+0000', 'last_undegraded': '2023-08-27T11:28:49.079769+0000', 'last_fullsized': '2023-08-27T11:28:49.079769+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-08-27T11:28:55.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.089647+0000', 'last_change': '2023-08-27T11:28:49.089647+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.089647+0000', 'last_undegraded': '2023-08-27T11:28:49.089647+0000', 'last_fullsized': '2023-08-27T11:28:49.089647+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-27T11:28:55.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-08-27T11:28:55.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.078553+0000', 'last_change': '2023-08-27T11:28:49.078553+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.078553+0000', 'last_undegraded': '2023-08-27T11:28:49.078553+0000', 'last_fullsized': '2023-08-27T11:28:49.078553+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-08-27T11:28:55.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.080372+0000', 'last_change': '2023-08-27T11:28:49.080372+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.080372+0000', 'last_undegraded': '2023-08-27T11:28:49.080372+0000', 'last_fullsized': '2023-08-27T11:28:49.080372+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-08-27T11:28:55.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 113, 'state': 'creating+peering', 'last_fresh': '2023-08-27T11:28:49.079866+0000', 'last_change': '2023-08-27T11:28:49.079866+0000', 'last_active': '2023-08-27T11:28:49.069412+0000', 'last_peered': '2023-08-27T11:28:49.069412+0000', 'last_clean': '2023-08-27T11:28:49.069412+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-27T11:28:49.079866+0000', 'last_undegraded': '2023-08-27T11:28:49.079866+0000', 'last_fullsized': '2023-08-27T11:28:49.079866+0000', 'mapping_epoch': 113, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 113, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'last_clean_scrub_stamp': '2023-08-27T11:28:49.069412+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-27T11:28:55.133 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-27T11:28:55.173 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:55.172+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:55.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.261+0000 7f964d8c6700 1 Processor -- start 2023-08-27T11:28:55.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.261+0000 7f964d8c6700 1 -- start start 2023-08-27T11:28:55.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f964d8c6700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f964806a8b0 0x7f96480ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f964d8c6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96480ce0d0 con 0x7f96480b23c0 2023-08-27T11:28:55.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f964d8c6700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96480b9040 con 0x7f96480b1e70 2023-08-27T11:28:55.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f964d8c6700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96480d2a10 con 0x7f964806a8b0 2023-08-27T11:28:55.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f96467fc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f96480b23c0 0x7f964806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47238/0 (socket says 172.21.15.45:47238) 2023-08-27T11:28:55.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f96467fc700 1 -- 172.21.15.45:0/910285075 learned_addr learned my addr 172.21.15.45:0/910285075 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:55.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f9646ffd700 1 --2- 172.21.15.45:0/910285075 >> v2:172.21.15.135:3300/0 conn(0x7f964806a8b0 0x7f96480ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f9645ffb700 1 -- 172.21.15.45:0/910285075 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870591093 0 0) 0x7f96480ce0d0 con 0x7f96480b23c0 2023-08-27T11:28:55.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f9645ffb700 1 -- 172.21.15.45:0/910285075 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96300035b0 con 0x7f96480b23c0 2023-08-27T11:28:55.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f9646ffd700 1 -- 172.21.15.45:0/910285075 >> v1:172.21.15.135:6789/0 conn(0x7f96480b1e70 legacy=0x7f96480b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f9646ffd700 1 -- 172.21.15.45:0/910285075 >> v1:172.21.15.45:6789/0 conn(0x7f96480b23c0 legacy=0x7f964806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.262+0000 7f9646ffd700 1 -- 172.21.15.45:0/910285075 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96480cd8d0 con 0x7f964806a8b0 2023-08-27T11:28:55.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f9646ffd700 1 --2- 172.21.15.45:0/910285075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964806a8b0 0x7f96480ce560 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9638009840 tx=0x7f9638009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13b592949c7eb24b server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f9645ffb700 1 -- 172.21.15.45:0/910285075 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638011830 con 0x7f964806a8b0 2023-08-27T11:28:55.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f9645ffb700 1 -- 172.21.15.45:0/910285075 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638011990 con 0x7f964806a8b0 2023-08-27T11:28:55.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f9645ffb700 1 -- 172.21.15.45:0/910285075 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638011d20 con 0x7f964806a8b0 2023-08-27T11:28:55.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f964d8c6700 1 -- 172.21.15.45:0/910285075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964806a8b0 msgr2=0x7f96480ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f964d8c6700 1 --2- 172.21.15.45:0/910285075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964806a8b0 0x7f96480ce560 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9638009840 tx=0x7f9638009b50 comp rx=0 tx=0).stop 2023-08-27T11:28:55.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f964d8c6700 1 -- 172.21.15.45:0/910285075 shutdown_connections 2023-08-27T11:28:55.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f964d8c6700 1 --2- 172.21.15.45:0/910285075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964806a8b0 0x7f96480ce560 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f964d8c6700 1 -- 172.21.15.45:0/910285075 >> 172.21.15.45:0/910285075 conn(0x7f96480d3070 msgr2=0x7f96480d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:55.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f964d8c6700 1 -- 172.21.15.45:0/910285075 shutdown_connections 2023-08-27T11:28:55.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.263+0000 7f964d8c6700 1 -- 172.21.15.45:0/910285075 wait complete. 2023-08-27T11:28:55.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.264+0000 7f964d8c6700 1 Processor -- start 2023-08-27T11:28:55.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.265+0000 7f964d8c6700 1 -- start start 2023-08-27T11:28:55.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.265+0000 7f964d8c6700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f964806a8b0 0x7f964815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.265+0000 7f964d8c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96480b1e70 0x7f9648158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.265+0000 7f964d8c6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96480b23c0 0x7f96480ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.265+0000 7f964d8c6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964815a870 con 0x7f96480b1e70 2023-08-27T11:28:55.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.265+0000 7f964d8c6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9648159bf0 con 0x7f96480b23c0 2023-08-27T11:28:55.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.265+0000 7f964d8c6700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9648159ed0 con 0x7f964806a8b0 2023-08-27T11:28:55.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96467fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96480b1e70 0x7f9648158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96467fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96480b1e70 0x7f9648158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60468/0 (socket says 172.21.15.45:60468) 2023-08-27T11:28:55.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96467fc700 1 -- 172.21.15.45:0/3498464903 learned_addr learned my addr 172.21.15.45:0/3498464903 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:55.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f9646ffd700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f964806a8b0 0x7f964815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96477fe700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96480b23c0 0x7f96480ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96467fc700 1 -- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f964806a8b0 msgr2=0x7f964815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96467fc700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f964806a8b0 0x7f964815ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96467fc700 1 -- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96480b23c0 msgr2=0x7f96480ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96467fc700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96480b23c0 0x7f96480ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96467fc700 1 -- 172.21.15.45:0/3498464903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380094a0 con 0x7f96480b1e70 2023-08-27T11:28:55.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f96467fc700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96480b1e70 0x7f9648158c90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f963c005910 tx=0x7f963c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f962ffff700 1 -- 172.21.15.45:0/3498464903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c005d70 con 0x7f96480b1e70 2023-08-27T11:28:55.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f962ffff700 1 -- 172.21.15.45:0/3498464903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c00ce80 con 0x7f96480b1e70 2023-08-27T11:28:55.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964815a1b0 con 0x7f96480b1e70 2023-08-27T11:28:55.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f962ffff700 1 -- 172.21.15.45:0/3498464903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c0072f0 con 0x7f96480b1e70 2023-08-27T11:28:55.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.266+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96480cbad0 con 0x7f96480b1e70 2023-08-27T11:28:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.267+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964815a9b0 con 0x7f96480b1e70 2023-08-27T11:28:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.268+0000 7f962ffff700 1 -- 172.21.15.45:0/3498464903 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f963c007450 con 0x7f96480b1e70 2023-08-27T11:28:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.268+0000 7f962ffff700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f963406a170 0x7f963406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.268+0000 7f962ffff700 1 -- 172.21.15.45:0/3498464903 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f963c087bd0 con 0x7f96480b1e70 2023-08-27T11:28:55.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.268+0000 7f9646ffd700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f963406a170 0x7f963406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.269+0000 7f9646ffd700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f963406a170 0x7f963406c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fd0 tx=0x7f9638008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.273+0000 7f962ffff700 1 -- 172.21.15.45:0/3498464903 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f964815a9b0 con 0x7f96480b1e70 2023-08-27T11:28:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.567+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f964815a9b0 con 0x7f96480b1e70 2023-08-27T11:28:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.568+0000 7f962ffff700 1 -- 172.21.15.45:0/3498464903 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1099 (secure 0 0 0) 0x7f964815a9b0 con 0x7f96480b1e70 2023-08-27T11:28:55.570 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:55.570 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1032,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":115,"num_osds":8,"num_up_osds":8,"osd_up_since":1693134714,"num_in_osds":8,"osd_in_since":1693134705,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":284786688,"bytes_avail":772809326592,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415,"read_bytes_sec":1089,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":49,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":8019}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-08-27T11:26:31.287744+0000","services":{}},"progress_events":{}} 2023-08-27T11:28:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f963406a170 msgr2=0x7f963406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f963406a170 0x7f963406c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fd0 tx=0x7f9638008030 comp rx=0 tx=0).stop 2023-08-27T11:28:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96480b1e70 msgr2=0x7f9648158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96480b1e70 0x7f9648158c90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f963c005910 tx=0x7f963c00c7b0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 shutdown_connections 2023-08-27T11:28:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f963406a170 0x7f963406c630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96480b23c0 0x7f96480ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f964806a8b0 0x7f964815ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 --2- 172.21.15.45:0/3498464903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96480b1e70 0x7f9648158c90 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 >> 172.21.15.45:0/3498464903 conn(0x7f96480d3070 msgr2=0x7f96480b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 shutdown_connections 2023-08-27T11:28:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.570+0000 7f964d8c6700 1 -- 172.21.15.45:0/3498464903 wait complete. 2023-08-27T11:28:55.584 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-27T11:28:55.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:55.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.710+0000 7f8ff59a3700 1 Processor -- start 2023-08-27T11:28:55.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.710+0000 7f8ff59a3700 1 -- start start 2023-08-27T11:28:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8ff59a3700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ff006a990 0x7f8ff00719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8ff59a3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff0073a10 0x7f8ff00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8ff59a3700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff00cfec0 con 0x7f8ff00ced20 2023-08-27T11:28:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8ff59a3700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff00af700 con 0x7f8ff006a990 2023-08-27T11:28:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8ff59a3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0072960 con 0x7f8ff0073a10 2023-08-27T11:28:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fee7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ff006a990 0x7f8ff00719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fee7fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ff006a990 0x7f8ff00719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60476/0 (socket says 172.21.15.45:60476) 2023-08-27T11:28:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fee7fc700 1 -- 172.21.15.45:0/3650640472 learned_addr learned my addr 172.21.15.45:0/3650640472 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fef7fe700 1 --2- 172.21.15.45:0/3650640472 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff0073a10 0x7f8ff00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fee7fc700 1 -- 172.21.15.45:0/3650640472 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff0073a10 msgr2=0x7f8ff00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fee7fc700 1 --2- 172.21.15.45:0/3650640472 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff0073a10 0x7f8ff00cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fee7fc700 1 -- 172.21.15.45:0/3650640472 >> v1:172.21.15.135:6789/0 conn(0x7f8ff00ced20 legacy=0x7f8ff006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fee7fc700 1 -- 172.21.15.45:0/3650640472 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff00af440 con 0x7f8ff006a990 2023-08-27T11:28:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fedffb700 1 -- 172.21.15.45:0/3650640472 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161345793 0 0) 0x7f8ff00cfec0 con 0x7f8ff00ced20 2023-08-27T11:28:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fee7fc700 1 --2- 172.21.15.45:0/3650640472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 0x7f8ff00719c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40098f0 tx=0x7f8fe4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0928d95e8d31f9c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fedffb700 1 -- 172.21.15.45:0/3650640472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4011830 con 0x7f8ff006a990 2023-08-27T11:28:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fedffb700 1 -- 172.21.15.45:0/3650640472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4011990 con 0x7f8ff006a990 2023-08-27T11:28:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.712+0000 7f8fedffb700 1 -- 172.21.15.45:0/3650640472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4011d20 con 0x7f8ff006a990 2023-08-27T11:28:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.713+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3650640472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 msgr2=0x7f8ff00719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.713+0000 7f8ff59a3700 1 --2- 172.21.15.45:0/3650640472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 0x7f8ff00719c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40098f0 tx=0x7f8fe4009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.713+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3650640472 shutdown_connections 2023-08-27T11:28:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.713+0000 7f8ff59a3700 1 --2- 172.21.15.45:0/3650640472 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff0073a10 0x7f8ff00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.713+0000 7f8ff59a3700 1 --2- 172.21.15.45:0/3650640472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 0x7f8ff00719c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.713+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3650640472 >> 172.21.15.45:0/3650640472 conn(0x7f8ff00b2470 msgr2=0x7f8ff00b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.713+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3650640472 shutdown_connections 2023-08-27T11:28:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.713+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3650640472 wait complete. 2023-08-27T11:28:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.714+0000 7f8ff59a3700 1 Processor -- start 2023-08-27T11:28:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8ff59a3700 1 -- start start 2023-08-27T11:28:55.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8ff59a3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 0x7f8ff01dc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8ff59a3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff0073a10 0x7f8ff01e7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8ff59a3700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff01e75a0 0x7f8ff01e3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8ff59a3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01dcd30 con 0x7f8ff006a990 2023-08-27T11:28:55.725 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8ff59a3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01e4d40 con 0x7f8ff0073a10 2023-08-27T11:28:55.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8ff59a3700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01e6030 con 0x7f8ff01e75a0 2023-08-27T11:28:55.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8feeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 0x7f8ff01dc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8feeffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 0x7f8ff01dc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60480/0 (socket says 172.21.15.45:60480) 2023-08-27T11:28:55.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.715+0000 7f8feeffd700 1 -- 172.21.15.45:0/3810819094 learned_addr learned my addr 172.21.15.45:0/3810819094 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:55.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8fef7fe700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff01e75a0 0x7f8ff01e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8fee7fc700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff0073a10 0x7f8ff01e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8feeffd700 1 -- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff01e75a0 msgr2=0x7f8ff01e3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8feeffd700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff01e75a0 0x7f8ff01e3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8feeffd700 1 -- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff0073a10 msgr2=0x7f8ff01e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8feeffd700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff0073a10 0x7f8ff01e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8feeffd700 1 -- 172.21.15.45:0/3810819094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe4009550 con 0x7f8ff006a990 2023-08-27T11:28:55.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8fee7fc700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff0073a10 0x7f8ff01e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:55.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8fef7fe700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff01e75a0 0x7f8ff01e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:28:55.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8feeffd700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 0x7f8ff01dc7f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0001eb0 tx=0x7f8fe000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8fd7fff700 1 -- 172.21.15.45:0/3810819094 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00054e0 con 0x7f8ff006a990 2023-08-27T11:28:55.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8fd7fff700 1 -- 172.21.15.45:0/3810819094 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe0003680 con 0x7f8ff006a990 2023-08-27T11:28:55.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8fd7fff700 1 -- 172.21.15.45:0/3810819094 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00039f0 con 0x7f8ff006a990 2023-08-27T11:28:55.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff01e4f90 con 0x7f8ff006a990 2023-08-27T11:28:55.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.716+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff01e5550 con 0x7f8ff006a990 2023-08-27T11:28:55.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.717+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff01e79b0 con 0x7f8ff006a990 2023-08-27T11:28:55.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.717+0000 7f8fd7fff700 1 -- 172.21.15.45:0/3810819094 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f8fe0003b50 con 0x7f8ff006a990 2023-08-27T11:28:55.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.718+0000 7f8fd7fff700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8fdc06a170 0x7f8fdc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:55.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.718+0000 7f8fd7fff700 1 -- 172.21.15.45:0/3810819094 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8fe0087ae0 con 0x7f8ff006a990 2023-08-27T11:28:55.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.718+0000 7f8fee7fc700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8fdc06a170 0x7f8fdc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:55.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.719+0000 7f8fee7fc700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8fdc06a170 0x7f8fdc06c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ff006e2a0 tx=0x7f8fe4008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.722+0000 7f8fd7fff700 1 -- 172.21.15.45:0/3810819094 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8fe0053120 con 0x7f8ff006a990 2023-08-27T11:28:55.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.929+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 --> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ff00723a0 con 0x7f8fdc06a170 2023-08-27T11:28:55.940 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.938+0000 7f8fd7fff700 1 -- 172.21.15.45:0/3810819094 <== mgr.7156 v2:172.21.15.135:6832/3759624346 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311968 (secure 0 0 0) 0x7f8fe401f070 con 0x7f8fdc06a170 2023-08-27T11:28:55.945 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:55.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8fdc06a170 msgr2=0x7f8fdc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8fdc06a170 0x7f8fdc06c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ff006e2a0 tx=0x7f8fe4008030 comp rx=0 tx=0).stop 2023-08-27T11:28:55.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 msgr2=0x7f8ff01dc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:55.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 0x7f8ff01dc7f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0001eb0 tx=0x7f8fe000c8b0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 shutdown_connections 2023-08-27T11:28:55.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f8fdc06a170 0x7f8fdc06c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ff006a990 0x7f8ff01dc7f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8ff01e75a0 0x7f8ff01e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 --2- 172.21.15.45:0/3810819094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ff0073a10 0x7f8ff01e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:55.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 >> 172.21.15.45:0/3810819094 conn(0x7f8ff00b2470 msgr2=0x7f8ff01e4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:55.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 shutdown_connections 2023-08-27T11:28:55.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:55.942+0000 7f8ff59a3700 1 -- 172.21.15.45:0/3810819094 wait complete. 2023-08-27T11:28:55.951 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:55.957 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-08-27T11:28:55.708210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":767,"num_read_kb":1473,"num_write":328,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5818,"ondisk_log_size":5818,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":304764,"kb_used_data":12296,"kb_used_omap":13,"kb_used_meta":292338,"kb_avail":754669956,"statfs":{"total":773094113280,"available":772782034944,"internally_reserved":0,"allocated":12591104,"data_stored":7155164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":299354200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.628540"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560654+0000","last_change":"2023-08-27T11:28:50.097122+0000","last_active":"2023-08-27T11:28:51.560654+0000","last_peered":"2023-08-27T11:28:51.560654+0000","last_clean":"2023-08-27T11:28:51.560654+0000","last_became_active":"2023-08-27T11:28:50.096823+0000","last_became_peered":"2023-08-27T11:28:50.096823+0000","last_unstale":"2023-08-27T11:28:51.560654+0000","last_undegraded":"2023-08-27T11:28:51.560654+0000","last_fullsized":"2023-08-27T11:28:51.560654+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:23:49.530921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232698+0000","last_change":"2023-08-27T11:28:50.090002+0000","last_active":"2023-08-27T11:28:51.232698+0000","last_peered":"2023-08-27T11:28:51.232698+0000","last_clean":"2023-08-27T11:28:51.232698+0000","last_became_active":"2023-08-27T11:28:50.089637+0000","last_became_peered":"2023-08-27T11:28:50.089637+0000","last_unstale":"2023-08-27T11:28:51.232698+0000","last_undegraded":"2023-08-27T11:28:51.232698+0000","last_fullsized":"2023-08-27T11:28:51.232698+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:41.115083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560378+0000","last_change":"2023-08-27T11:28:50.091610+0000","last_active":"2023-08-27T11:28:51.560378+0000","last_peered":"2023-08-27T11:28:51.560378+0000","last_clean":"2023-08-27T11:28:51.560378+0000","last_became_active":"2023-08-27T11:28:50.091310+0000","last_became_peered":"2023-08-27T11:28:50.091310+0000","last_unstale":"2023-08-27T11:28:51.560378+0000","last_undegraded":"2023-08-27T11:28:51.560378+0000","last_fullsized":"2023-08-27T11:28:51.560378+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:05:15.485701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560580+0000","last_change":"2023-08-27T11:28:50.099807+0000","last_active":"2023-08-27T11:28:51.560580+0000","last_peered":"2023-08-27T11:28:51.560580+0000","last_clean":"2023-08-27T11:28:51.560580+0000","last_became_active":"2023-08-27T11:28:50.099499+0000","last_became_peered":"2023-08-27T11:28:50.099499+0000","last_unstale":"2023-08-27T11:28:51.560580+0000","last_undegraded":"2023-08-27T11:28:51.560580+0000","last_fullsized":"2023-08-27T11:28:51.560580+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:05:28.545130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560130+0000","last_change":"2023-08-27T11:28:50.101108+0000","last_active":"2023-08-27T11:28:51.560130+0000","last_peered":"2023-08-27T11:28:51.560130+0000","last_clean":"2023-08-27T11:28:51.560130+0000","last_became_active":"2023-08-27T11:28:50.100852+0000","last_became_peered":"2023-08-27T11:28:50.100852+0000","last_unstale":"2023-08-27T11:28:51.560130+0000","last_undegraded":"2023-08-27T11:28:51.560130+0000","last_fullsized":"2023-08-27T11:28:51.560130+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:54.314223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230873+0000","last_change":"2023-08-27T11:28:50.091281+0000","last_active":"2023-08-27T11:28:51.230873+0000","last_peered":"2023-08-27T11:28:51.230873+0000","last_clean":"2023-08-27T11:28:51.230873+0000","last_became_active":"2023-08-27T11:28:50.091013+0000","last_became_peered":"2023-08-27T11:28:50.091013+0000","last_unstale":"2023-08-27T11:28:51.230873+0000","last_undegraded":"2023-08-27T11:28:51.230873+0000","last_fullsized":"2023-08-27T11:28:51.230873+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:15:25.273981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230921+0000","last_change":"2023-08-27T11:28:50.091458+0000","last_active":"2023-08-27T11:28:51.230921+0000","last_peered":"2023-08-27T11:28:51.230921+0000","last_clean":"2023-08-27T11:28:51.230921+0000","last_became_active":"2023-08-27T11:28:50.091225+0000","last_became_peered":"2023-08-27T11:28:50.091225+0000","last_unstale":"2023-08-27T11:28:51.230921+0000","last_undegraded":"2023-08-27T11:28:51.230921+0000","last_fullsized":"2023-08-27T11:28:51.230921+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:43:09.542275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232423+0000","last_change":"2023-08-27T11:28:50.092992+0000","last_active":"2023-08-27T11:28:51.232423+0000","last_peered":"2023-08-27T11:28:51.232423+0000","last_clean":"2023-08-27T11:28:51.232423+0000","last_became_active":"2023-08-27T11:28:50.092472+0000","last_became_peered":"2023-08-27T11:28:50.092472+0000","last_unstale":"2023-08-27T11:28:51.232423+0000","last_undegraded":"2023-08-27T11:28:51.232423+0000","last_fullsized":"2023-08-27T11:28:51.232423+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:05:23.180273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080522+0000","last_change":"2023-08-27T11:28:50.102638+0000","last_active":"2023-08-27T11:28:51.080522+0000","last_peered":"2023-08-27T11:28:51.080522+0000","last_clean":"2023-08-27T11:28:51.080522+0000","last_became_active":"2023-08-27T11:28:50.102375+0000","last_became_peered":"2023-08-27T11:28:50.102375+0000","last_unstale":"2023-08-27T11:28:51.080522+0000","last_undegraded":"2023-08-27T11:28:51.080522+0000","last_fullsized":"2023-08-27T11:28:51.080522+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:36:33.012903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560209+0000","last_change":"2023-08-27T11:28:50.102856+0000","last_active":"2023-08-27T11:28:51.560209+0000","last_peered":"2023-08-27T11:28:51.560209+0000","last_clean":"2023-08-27T11:28:51.560209+0000","last_became_active":"2023-08-27T11:28:50.102509+0000","last_became_peered":"2023-08-27T11:28:50.102509+0000","last_unstale":"2023-08-27T11:28:51.560209+0000","last_undegraded":"2023-08-27T11:28:51.560209+0000","last_fullsized":"2023-08-27T11:28:51.560209+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:20:32.689948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230962+0000","last_change":"2023-08-27T11:28:50.092360+0000","last_active":"2023-08-27T11:28:51.230962+0000","last_peered":"2023-08-27T11:28:51.230962+0000","last_clean":"2023-08-27T11:28:51.230962+0000","last_became_active":"2023-08-27T11:28:50.091909+0000","last_became_peered":"2023-08-27T11:28:50.091909+0000","last_unstale":"2023-08-27T11:28:51.230962+0000","last_undegraded":"2023-08-27T11:28:51.230962+0000","last_fullsized":"2023-08-27T11:28:51.230962+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:31.002100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080264+0000","last_change":"2023-08-27T11:28:50.104376+0000","last_active":"2023-08-27T11:28:51.080264+0000","last_peered":"2023-08-27T11:28:51.080264+0000","last_clean":"2023-08-27T11:28:51.080264+0000","last_became_active":"2023-08-27T11:28:50.104121+0000","last_became_peered":"2023-08-27T11:28:50.104121+0000","last_unstale":"2023-08-27T11:28:51.080264+0000","last_undegraded":"2023-08-27T11:28:51.080264+0000","last_fullsized":"2023-08-27T11:28:51.080264+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:14.712757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560408+0000","last_change":"2023-08-27T11:28:50.090400+0000","last_active":"2023-08-27T11:28:51.560408+0000","last_peered":"2023-08-27T11:28:51.560408+0000","last_clean":"2023-08-27T11:28:51.560408+0000","last_became_active":"2023-08-27T11:28:50.090097+0000","last_became_peered":"2023-08-27T11:28:50.090097+0000","last_unstale":"2023-08-27T11:28:51.560408+0000","last_undegraded":"2023-08-27T11:28:51.560408+0000","last_fullsized":"2023-08-27T11:28:51.560408+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:32:34.104328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078706+0000","last_change":"2023-08-27T11:28:50.090623+0000","last_active":"2023-08-27T11:28:51.078706+0000","last_peered":"2023-08-27T11:28:51.078706+0000","last_clean":"2023-08-27T11:28:51.078706+0000","last_became_active":"2023-08-27T11:28:50.090362+0000","last_became_peered":"2023-08-27T11:28:50.090362+0000","last_unstale":"2023-08-27T11:28:51.078706+0000","last_undegraded":"2023-08-27T11:28:51.078706+0000","last_fullsized":"2023-08-27T11:28:51.078706+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:03.877364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560088+0000","last_change":"2023-08-27T11:28:50.105514+0000","last_active":"2023-08-27T11:28:51.560088+0000","last_peered":"2023-08-27T11:28:51.560088+0000","last_clean":"2023-08-27T11:28:51.560088+0000","last_became_active":"2023-08-27T11:28:50.105134+0000","last_became_peered":"2023-08-27T11:28:50.105134+0000","last_unstale":"2023-08-27T11:28:51.560088+0000","last_undegraded":"2023-08-27T11:28:51.560088+0000","last_fullsized":"2023-08-27T11:28:51.560088+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:48:56.741867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230783+0000","last_change":"2023-08-27T11:28:50.089955+0000","last_active":"2023-08-27T11:28:51.230783+0000","last_peered":"2023-08-27T11:28:51.230783+0000","last_clean":"2023-08-27T11:28:51.230783+0000","last_became_active":"2023-08-27T11:28:50.089680+0000","last_became_peered":"2023-08-27T11:28:50.089680+0000","last_unstale":"2023-08-27T11:28:51.230783+0000","last_undegraded":"2023-08-27T11:28:51.230783+0000","last_fullsized":"2023-08-27T11:28:51.230783+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:35:33.140967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.233038+0000","last_change":"2023-08-27T11:28:50.101136+0000","last_active":"2023-08-27T11:28:51.233038+0000","last_peered":"2023-08-27T11:28:51.233038+0000","last_clean":"2023-08-27T11:28:51.233038+0000","last_became_active":"2023-08-27T11:28:50.100810+0000","last_became_peered":"2023-08-27T11:28:50.100810+0000","last_unstale":"2023-08-27T11:28:51.233038+0000","last_undegraded":"2023-08-27T11:28:51.233038+0000","last_fullsized":"2023-08-27T11:28:51.233038+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:07:38.715717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560940+0000","last_change":"2023-08-27T11:28:50.100982+0000","last_active":"2023-08-27T11:28:51.560940+0000","last_peered":"2023-08-27T11:28:51.560940+0000","last_clean":"2023-08-27T11:28:51.560940+0000","last_became_active":"2023-08-27T11:28:50.100676+0000","last_became_peered":"2023-08-27T11:28:50.100676+0000","last_unstale":"2023-08-27T11:28:51.560940+0000","last_undegraded":"2023-08-27T11:28:51.560940+0000","last_fullsized":"2023-08-27T11:28:51.560940+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:51.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078625+0000","last_change":"2023-08-27T11:28:50.094625+0000","last_active":"2023-08-27T11:28:51.078625+0000","last_peered":"2023-08-27T11:28:51.078625+0000","last_clean":"2023-08-27T11:28:51.078625+0000","last_became_active":"2023-08-27T11:28:50.094317+0000","last_became_peered":"2023-08-27T11:28:50.094317+0000","last_unstale":"2023-08-27T11:28:51.078625+0000","last_undegraded":"2023-08-27T11:28:51.078625+0000","last_fullsized":"2023-08-27T11:28:51.078625+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:44:04.674880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080485+0000","last_change":"2023-08-27T11:28:50.104689+0000","last_active":"2023-08-27T11:28:51.080485+0000","last_peered":"2023-08-27T11:28:51.080485+0000","last_clean":"2023-08-27T11:28:51.080485+0000","last_became_active":"2023-08-27T11:28:50.104424+0000","last_became_peered":"2023-08-27T11:28:50.104424+0000","last_unstale":"2023-08-27T11:28:51.080485+0000","last_undegraded":"2023-08-27T11:28:51.080485+0000","last_fullsized":"2023-08-27T11:28:51.080485+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:39.815694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078652+0000","last_change":"2023-08-27T11:28:50.084144+0000","last_active":"2023-08-27T11:28:51.078652+0000","last_peered":"2023-08-27T11:28:51.078652+0000","last_clean":"2023-08-27T11:28:51.078652+0000","last_became_active":"2023-08-27T11:28:50.083650+0000","last_became_peered":"2023-08-27T11:28:50.083650+0000","last_unstale":"2023-08-27T11:28:51.078652+0000","last_undegraded":"2023-08-27T11:28:51.078652+0000","last_fullsized":"2023-08-27T11:28:51.078652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:55.720567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232443+0000","last_change":"2023-08-27T11:28:50.091499+0000","last_active":"2023-08-27T11:28:51.232443+0000","last_peered":"2023-08-27T11:28:51.232443+0000","last_clean":"2023-08-27T11:28:51.232443+0000","last_became_active":"2023-08-27T11:28:50.091217+0000","last_became_peered":"2023-08-27T11:28:50.091217+0000","last_unstale":"2023-08-27T11:28:51.232443+0000","last_undegraded":"2023-08-27T11:28:51.232443+0000","last_fullsized":"2023-08-27T11:28:51.232443+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:26.655085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232433+0000","last_change":"2023-08-27T11:28:50.089444+0000","last_active":"2023-08-27T11:28:51.232433+0000","last_peered":"2023-08-27T11:28:51.232433+0000","last_clean":"2023-08-27T11:28:51.232433+0000","last_became_active":"2023-08-27T11:28:50.089137+0000","last_became_peered":"2023-08-27T11:28:50.089137+0000","last_unstale":"2023-08-27T11:28:51.232433+0000","last_undegraded":"2023-08-27T11:28:51.232433+0000","last_fullsized":"2023-08-27T11:28:51.232433+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:14.275788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230885+0000","last_change":"2023-08-27T11:28:50.110178+0000","last_active":"2023-08-27T11:28:51.230885+0000","last_peered":"2023-08-27T11:28:51.230885+0000","last_clean":"2023-08-27T11:28:51.230885+0000","last_became_active":"2023-08-27T11:28:50.109973+0000","last_became_peered":"2023-08-27T11:28:50.109973+0000","last_unstale":"2023-08-27T11:28:51.230885+0000","last_undegraded":"2023-08-27T11:28:51.230885+0000","last_fullsized":"2023-08-27T11:28:51.230885+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:45:23.272861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560011+0000","last_change":"2023-08-27T11:28:50.092156+0000","last_active":"2023-08-27T11:28:51.560011+0000","last_peered":"2023-08-27T11:28:51.560011+0000","last_clean":"2023-08-27T11:28:51.560011+0000","last_became_active":"2023-08-27T11:28:50.091924+0000","last_became_peered":"2023-08-27T11:28:50.091924+0000","last_unstale":"2023-08-27T11:28:51.560011+0000","last_undegraded":"2023-08-27T11:28:51.560011+0000","last_fullsized":"2023-08-27T11:28:51.560011+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:03:11.501424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232966+0000","last_change":"2023-08-27T11:28:50.098917+0000","last_active":"2023-08-27T11:28:51.232966+0000","last_peered":"2023-08-27T11:28:51.232966+0000","last_clean":"2023-08-27T11:28:51.232966+0000","last_became_active":"2023-08-27T11:28:50.098651+0000","last_became_peered":"2023-08-27T11:28:50.098651+0000","last_unstale":"2023-08-27T11:28:51.232966+0000","last_undegraded":"2023-08-27T11:28:51.232966+0000","last_fullsized":"2023-08-27T11:28:51.232966+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:30.076751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231217+0000","last_change":"2023-08-27T11:28:50.092166+0000","last_active":"2023-08-27T11:28:51.231217+0000","last_peered":"2023-08-27T11:28:51.231217+0000","last_clean":"2023-08-27T11:28:51.231217+0000","last_became_active":"2023-08-27T11:28:50.091564+0000","last_became_peered":"2023-08-27T11:28:50.091564+0000","last_unstale":"2023-08-27T11:28:51.231217+0000","last_undegraded":"2023-08-27T11:28:51.231217+0000","last_fullsized":"2023-08-27T11:28:51.231217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:35:12.257603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560029+0000","last_change":"2023-08-27T11:28:50.104554+0000","last_active":"2023-08-27T11:28:51.560029+0000","last_peered":"2023-08-27T11:28:51.560029+0000","last_clean":"2023-08-27T11:28:51.560029+0000","last_became_active":"2023-08-27T11:28:50.103887+0000","last_became_peered":"2023-08-27T11:28:50.103887+0000","last_unstale":"2023-08-27T11:28:51.560029+0000","last_undegraded":"2023-08-27T11:28:51.560029+0000","last_fullsized":"2023-08-27T11:28:51.560029+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:03:59.647986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232677+0000","last_change":"2023-08-27T11:28:50.104417+0000","last_active":"2023-08-27T11:28:51.232677+0000","last_peered":"2023-08-27T11:28:51.232677+0000","last_clean":"2023-08-27T11:28:51.232677+0000","last_became_active":"2023-08-27T11:28:50.104163+0000","last_became_peered":"2023-08-27T11:28:50.104163+0000","last_unstale":"2023-08-27T11:28:51.232677+0000","last_undegraded":"2023-08-27T11:28:51.232677+0000","last_fullsized":"2023-08-27T11:28:51.232677+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:10:52.138429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078238+0000","last_change":"2023-08-27T11:28:50.104134+0000","last_active":"2023-08-27T11:28:51.078238+0000","last_peered":"2023-08-27T11:28:51.078238+0000","last_clean":"2023-08-27T11:28:51.078238+0000","last_became_active":"2023-08-27T11:28:50.103919+0000","last_became_peered":"2023-08-27T11:28:50.103919+0000","last_unstale":"2023-08-27T11:28:51.078238+0000","last_undegraded":"2023-08-27T11:28:51.078238+0000","last_fullsized":"2023-08-27T11:28:51.078238+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:45:23.971858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560165+0000","last_change":"2023-08-27T11:28:50.096252+0000","last_active":"2023-08-27T11:28:51.560165+0000","last_peered":"2023-08-27T11:28:51.560165+0000","last_clean":"2023-08-27T11:28:51.560165+0000","last_became_active":"2023-08-27T11:28:50.095972+0000","last_became_peered":"2023-08-27T11:28:50.095972+0000","last_unstale":"2023-08-27T11:28:51.560165+0000","last_undegraded":"2023-08-27T11:28:51.560165+0000","last_fullsized":"2023-08-27T11:28:51.560165+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:13:26.132084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560067+0000","last_change":"2023-08-27T11:28:50.109894+0000","last_active":"2023-08-27T11:28:51.560067+0000","last_peered":"2023-08-27T11:28:51.560067+0000","last_clean":"2023-08-27T11:28:51.560067+0000","last_became_active":"2023-08-27T11:28:50.109380+0000","last_became_peered":"2023-08-27T11:28:50.109380+0000","last_unstale":"2023-08-27T11:28:51.560067+0000","last_undegraded":"2023-08-27T11:28:51.560067+0000","last_fullsized":"2023-08-27T11:28:51.560067+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:23:39.910873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078497+0000","last_change":"2023-08-27T11:28:50.086964+0000","last_active":"2023-08-27T11:28:51.078497+0000","last_peered":"2023-08-27T11:28:51.078497+0000","last_clean":"2023-08-27T11:28:51.078497+0000","last_became_active":"2023-08-27T11:28:50.086719+0000","last_became_peered":"2023-08-27T11:28:50.086719+0000","last_unstale":"2023-08-27T11:28:51.078497+0000","last_undegraded":"2023-08-27T11:28:51.078497+0000","last_fullsized":"2023-08-27T11:28:51.078497+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:39:26.224404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"114'4","reported_seq":17,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.233098+0000","last_change":"2023-08-27T11:28:49.093411+0000","last_active":"2023-08-27T11:28:51.233098+0000","last_peered":"2023-08-27T11:28:51.233098+0000","last_clean":"2023-08-27T11:28:51.233098+0000","last_became_active":"2023-08-27T11:28:49.092946+0000","last_became_peered":"2023-08-27T11:28:49.092946+0000","last_unstale":"2023-08-27T11:28:51.233098+0000","last_undegraded":"2023-08-27T11:28:51.233098+0000","last_fullsized":"2023-08-27T11:28:51.233098+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:21:34.770484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560629+0000","last_change":"2023-08-27T11:28:50.100223+0000","last_active":"2023-08-27T11:28:51.560629+0000","last_peered":"2023-08-27T11:28:51.560629+0000","last_clean":"2023-08-27T11:28:51.560629+0000","last_became_active":"2023-08-27T11:28:50.099858+0000","last_became_peered":"2023-08-27T11:28:50.099858+0000","last_unstale":"2023-08-27T11:28:51.560629+0000","last_undegraded":"2023-08-27T11:28:51.560629+0000","last_fullsized":"2023-08-27T11:28:51.560629+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:12.890450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231139+0000","last_change":"2023-08-27T11:28:49.095509+0000","last_active":"2023-08-27T11:28:51.231139+0000","last_peered":"2023-08-27T11:28:51.231139+0000","last_clean":"2023-08-27T11:28:51.231139+0000","last_became_active":"2023-08-27T11:28:49.095273+0000","last_became_peered":"2023-08-27T11:28:49.095273+0000","last_unstale":"2023-08-27T11:28:51.231139+0000","last_undegraded":"2023-08-27T11:28:51.231139+0000","last_fullsized":"2023-08-27T11:28:51.231139+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:15.759385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560481+0000","last_change":"2023-08-27T11:28:50.102328+0000","last_active":"2023-08-27T11:28:51.560481+0000","last_peered":"2023-08-27T11:28:51.560481+0000","last_clean":"2023-08-27T11:28:51.560481+0000","last_became_active":"2023-08-27T11:28:50.102071+0000","last_became_peered":"2023-08-27T11:28:50.102071+0000","last_unstale":"2023-08-27T11:28:51.560481+0000","last_undegraded":"2023-08-27T11:28:51.560481+0000","last_fullsized":"2023-08-27T11:28:51.560481+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:02:36.681114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232773+0000","last_change":"2023-08-27T11:28:49.093022+0000","last_active":"2023-08-27T11:28:51.232773+0000","last_peered":"2023-08-27T11:28:51.232773+0000","last_clean":"2023-08-27T11:28:51.232773+0000","last_became_active":"2023-08-27T11:28:49.092689+0000","last_became_peered":"2023-08-27T11:28:49.092689+0000","last_unstale":"2023-08-27T11:28:51.232773+0000","last_undegraded":"2023-08-27T11:28:51.232773+0000","last_fullsized":"2023-08-27T11:28:51.232773+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:54:24.228452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080750+0000","last_change":"2023-08-27T11:28:50.083219+0000","last_active":"2023-08-27T11:28:51.080750+0000","last_peered":"2023-08-27T11:28:51.080750+0000","last_clean":"2023-08-27T11:28:51.080750+0000","last_became_active":"2023-08-27T11:28:50.082971+0000","last_became_peered":"2023-08-27T11:28:50.082971+0000","last_unstale":"2023-08-27T11:28:51.080750+0000","last_undegraded":"2023-08-27T11:28:51.080750+0000","last_fullsized":"2023-08-27T11:28:51.080750+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:04.437174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232520+0000","last_change":"2023-08-27T11:28:49.098336+0000","last_active":"2023-08-27T11:28:51.232520+0000","last_peered":"2023-08-27T11:28:51.232520+0000","last_clean":"2023-08-27T11:28:51.232520+0000","last_became_active":"2023-08-27T11:28:49.098055+0000","last_became_peered":"2023-08-27T11:28:49.098055+0000","last_unstale":"2023-08-27T11:28:51.232520+0000","last_undegraded":"2023-08-27T11:28:51.232520+0000","last_fullsized":"2023-08-27T11:28:51.232520+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:43.143302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079721+0000","last_change":"2023-08-27T11:28:50.083624+0000","last_active":"2023-08-27T11:28:51.079721+0000","last_peered":"2023-08-27T11:28:51.079721+0000","last_clean":"2023-08-27T11:28:51.079721+0000","last_became_active":"2023-08-27T11:28:50.082821+0000","last_became_peered":"2023-08-27T11:28:50.082821+0000","last_unstale":"2023-08-27T11:28:51.079721+0000","last_undegraded":"2023-08-27T11:28:51.079721+0000","last_fullsized":"2023-08-27T11:28:51.079721+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:15.483275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560278+0000","last_change":"2023-08-27T11:28:49.088759+0000","last_active":"2023-08-27T11:28:51.560278+0000","last_peered":"2023-08-27T11:28:51.560278+0000","last_clean":"2023-08-27T11:28:51.560278+0000","last_became_active":"2023-08-27T11:28:49.088495+0000","last_became_peered":"2023-08-27T11:28:49.088495+0000","last_unstale":"2023-08-27T11:28:51.560278+0000","last_undegraded":"2023-08-27T11:28:51.560278+0000","last_fullsized":"2023-08-27T11:28:51.560278+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:16.706534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230960+0000","last_change":"2023-08-27T11:28:50.094281+0000","last_active":"2023-08-27T11:28:51.230960+0000","last_peered":"2023-08-27T11:28:51.230960+0000","last_clean":"2023-08-27T11:28:51.230960+0000","last_became_active":"2023-08-27T11:28:50.094033+0000","last_became_peered":"2023-08-27T11:28:50.094033+0000","last_unstale":"2023-08-27T11:28:51.230960+0000","last_undegraded":"2023-08-27T11:28:51.230960+0000","last_fullsized":"2023-08-27T11:28:51.230960+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:44:31.932038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232556+0000","last_change":"2023-08-27T11:28:49.093290+0000","last_active":"2023-08-27T11:28:51.232556+0000","last_peered":"2023-08-27T11:28:51.232556+0000","last_clean":"2023-08-27T11:28:51.232556+0000","last_became_active":"2023-08-27T11:28:49.092886+0000","last_became_peered":"2023-08-27T11:28:49.092886+0000","last_unstale":"2023-08-27T11:28:51.232556+0000","last_undegraded":"2023-08-27T11:28:51.232556+0000","last_fullsized":"2023-08-27T11:28:51.232556+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:49:44.466015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560257+0000","last_change":"2023-08-27T11:28:50.110407+0000","last_active":"2023-08-27T11:28:51.560257+0000","last_peered":"2023-08-27T11:28:51.560257+0000","last_clean":"2023-08-27T11:28:51.560257+0000","last_became_active":"2023-08-27T11:28:50.110097+0000","last_became_peered":"2023-08-27T11:28:50.110097+0000","last_unstale":"2023-08-27T11:28:51.560257+0000","last_undegraded":"2023-08-27T11:28:51.560257+0000","last_fullsized":"2023-08-27T11:28:51.560257+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:58.662545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560234+0000","last_change":"2023-08-27T11:28:49.093546+0000","last_active":"2023-08-27T11:28:51.560234+0000","last_peered":"2023-08-27T11:28:51.560234+0000","last_clean":"2023-08-27T11:28:51.560234+0000","last_became_active":"2023-08-27T11:28:49.093291+0000","last_became_peered":"2023-08-27T11:28:49.093291+0000","last_unstale":"2023-08-27T11:28:51.560234+0000","last_undegraded":"2023-08-27T11:28:51.560234+0000","last_fullsized":"2023-08-27T11:28:51.560234+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:41:52.496264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080021+0000","last_change":"2023-08-27T11:28:50.083832+0000","last_active":"2023-08-27T11:28:51.080021+0000","last_peered":"2023-08-27T11:28:51.080021+0000","last_clean":"2023-08-27T11:28:51.080021+0000","last_became_active":"2023-08-27T11:28:50.083581+0000","last_became_peered":"2023-08-27T11:28:50.083581+0000","last_unstale":"2023-08-27T11:28:51.080021+0000","last_undegraded":"2023-08-27T11:28:51.080021+0000","last_fullsized":"2023-08-27T11:28:51.080021+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:12.890450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.559914+0000","last_change":"2023-08-27T11:28:49.087457+0000","last_active":"2023-08-27T11:28:51.559914+0000","last_peered":"2023-08-27T11:28:51.559914+0000","last_clean":"2023-08-27T11:28:51.559914+0000","last_became_active":"2023-08-27T11:28:49.087210+0000","last_became_peered":"2023-08-27T11:28:49.087210+0000","last_unstale":"2023-08-27T11:28:51.559914+0000","last_undegraded":"2023-08-27T11:28:51.559914+0000","last_fullsized":"2023-08-27T11:28:51.559914+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:35:31.341715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.233132+0000","last_change":"2023-08-27T11:28:50.090070+0000","last_active":"2023-08-27T11:28:51.233132+0000","last_peered":"2023-08-27T11:28:51.233132+0000","last_clean":"2023-08-27T11:28:51.233132+0000","last_became_active":"2023-08-27T11:28:50.089748+0000","last_became_peered":"2023-08-27T11:28:50.089748+0000","last_unstale":"2023-08-27T11:28:51.233132+0000","last_undegraded":"2023-08-27T11:28:51.233132+0000","last_fullsized":"2023-08-27T11:28:51.233132+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:36:02.160998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560797+0000","last_change":"2023-08-27T11:28:49.089014+0000","last_active":"2023-08-27T11:28:51.560797+0000","last_peered":"2023-08-27T11:28:51.560797+0000","last_clean":"2023-08-27T11:28:51.560797+0000","last_became_active":"2023-08-27T11:28:49.088750+0000","last_became_peered":"2023-08-27T11:28:49.088750+0000","last_unstale":"2023-08-27T11:28:51.560797+0000","last_undegraded":"2023-08-27T11:28:51.560797+0000","last_fullsized":"2023-08-27T11:28:51.560797+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:57:17.423641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080659+0000","last_change":"2023-08-27T11:28:50.083843+0000","last_active":"2023-08-27T11:28:51.080659+0000","last_peered":"2023-08-27T11:28:51.080659+0000","last_clean":"2023-08-27T11:28:51.080659+0000","last_became_active":"2023-08-27T11:28:50.083567+0000","last_became_peered":"2023-08-27T11:28:50.083567+0000","last_unstale":"2023-08-27T11:28:51.080659+0000","last_undegraded":"2023-08-27T11:28:51.080659+0000","last_fullsized":"2023-08-27T11:28:51.080659+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:48.984397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.561006+0000","last_change":"2023-08-27T11:28:49.093294+0000","last_active":"2023-08-27T11:28:51.561006+0000","last_peered":"2023-08-27T11:28:51.561006+0000","last_clean":"2023-08-27T11:28:51.561006+0000","last_became_active":"2023-08-27T11:28:49.092967+0000","last_became_peered":"2023-08-27T11:28:49.092967+0000","last_unstale":"2023-08-27T11:28:51.561006+0000","last_undegraded":"2023-08-27T11:28:51.561006+0000","last_fullsized":"2023-08-27T11:28:51.561006+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:13.708814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560128+0000","last_change":"2023-08-27T11:28:50.092774+0000","last_active":"2023-08-27T11:28:51.560128+0000","last_peered":"2023-08-27T11:28:51.560128+0000","last_clean":"2023-08-27T11:28:51.560128+0000","last_became_active":"2023-08-27T11:28:50.092498+0000","last_became_peered":"2023-08-27T11:28:50.092498+0000","last_unstale":"2023-08-27T11:28:51.560128+0000","last_undegraded":"2023-08-27T11:28:51.560128+0000","last_fullsized":"2023-08-27T11:28:51.560128+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:47:35.903355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"114'3","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231133+0000","last_change":"2023-08-27T11:28:49.095274+0000","last_active":"2023-08-27T11:28:51.231133+0000","last_peered":"2023-08-27T11:28:51.231133+0000","last_clean":"2023-08-27T11:28:51.231133+0000","last_became_active":"2023-08-27T11:28:49.095022+0000","last_became_peered":"2023-08-27T11:28:49.095022+0000","last_unstale":"2023-08-27T11:28:51.231133+0000","last_undegraded":"2023-08-27T11:28:51.231133+0000","last_fullsized":"2023-08-27T11:28:51.231133+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:06:51.505347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230918+0000","last_change":"2023-08-27T11:28:50.090510+0000","last_active":"2023-08-27T11:28:51.230918+0000","last_peered":"2023-08-27T11:28:51.230918+0000","last_clean":"2023-08-27T11:28:51.230918+0000","last_became_active":"2023-08-27T11:28:50.089693+0000","last_became_peered":"2023-08-27T11:28:50.089693+0000","last_unstale":"2023-08-27T11:28:51.230918+0000","last_undegraded":"2023-08-27T11:28:51.230918+0000","last_fullsized":"2023-08-27T11:28:51.230918+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:03.638017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"114'6","reported_seq":19,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080494+0000","last_change":"2023-08-27T11:28:49.087250+0000","last_active":"2023-08-27T11:28:51.080494+0000","last_peered":"2023-08-27T11:28:51.080494+0000","last_clean":"2023-08-27T11:28:51.080494+0000","last_became_active":"2023-08-27T11:28:49.087002+0000","last_became_peered":"2023-08-27T11:28:49.087002+0000","last_unstale":"2023-08-27T11:28:51.080494+0000","last_undegraded":"2023-08-27T11:28:51.080494+0000","last_fullsized":"2023-08-27T11:28:51.080494+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:35:50.737206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560166+0000","last_change":"2023-08-27T11:28:50.099829+0000","last_active":"2023-08-27T11:28:51.560166+0000","last_peered":"2023-08-27T11:28:51.560166+0000","last_clean":"2023-08-27T11:28:51.560166+0000","last_became_active":"2023-08-27T11:28:50.099468+0000","last_became_peered":"2023-08-27T11:28:50.099468+0000","last_unstale":"2023-08-27T11:28:51.560166+0000","last_undegraded":"2023-08-27T11:28:51.560166+0000","last_fullsized":"2023-08-27T11:28:51.560166+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:13.974776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078672+0000","last_change":"2023-08-27T11:28:49.086306+0000","last_active":"2023-08-27T11:28:51.078672+0000","last_peered":"2023-08-27T11:28:51.078672+0000","last_clean":"2023-08-27T11:28:51.078672+0000","last_became_active":"2023-08-27T11:28:49.086033+0000","last_became_peered":"2023-08-27T11:28:49.086033+0000","last_unstale":"2023-08-27T11:28:51.078672+0000","last_undegraded":"2023-08-27T11:28:51.078672+0000","last_fullsized":"2023-08-27T11:28:51.078672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:28:54.969620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560136+0000","last_change":"2023-08-27T11:28:50.103342+0000","last_active":"2023-08-27T11:28:51.560136+0000","last_peered":"2023-08-27T11:28:51.560136+0000","last_clean":"2023-08-27T11:28:51.560136+0000","last_became_active":"2023-08-27T11:28:50.103001+0000","last_became_peered":"2023-08-27T11:28:50.103001+0000","last_unstale":"2023-08-27T11:28:51.560136+0000","last_undegraded":"2023-08-27T11:28:51.560136+0000","last_fullsized":"2023-08-27T11:28:51.560136+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:24:17.069937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078700+0000","last_change":"2023-08-27T11:28:49.088909+0000","last_active":"2023-08-27T11:28:51.078700+0000","last_peered":"2023-08-27T11:28:51.078700+0000","last_clean":"2023-08-27T11:28:51.078700+0000","last_became_active":"2023-08-27T11:28:49.088497+0000","last_became_peered":"2023-08-27T11:28:49.088497+0000","last_unstale":"2023-08-27T11:28:51.078700+0000","last_undegraded":"2023-08-27T11:28:51.078700+0000","last_fullsized":"2023-08-27T11:28:51.078700+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:13:02.667709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079962+0000","last_change":"2023-08-27T11:28:50.083177+0000","last_active":"2023-08-27T11:28:51.079962+0000","last_peered":"2023-08-27T11:28:51.079962+0000","last_clean":"2023-08-27T11:28:51.079962+0000","last_became_active":"2023-08-27T11:28:50.082919+0000","last_became_peered":"2023-08-27T11:28:50.082919+0000","last_unstale":"2023-08-27T11:28:51.079962+0000","last_undegraded":"2023-08-27T11:28:51.079962+0000","last_fullsized":"2023-08-27T11:28:51.079962+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:05:28.545130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080153+0000","last_change":"2023-08-27T11:28:49.087910+0000","last_active":"2023-08-27T11:28:51.080153+0000","last_peered":"2023-08-27T11:28:51.080153+0000","last_clean":"2023-08-27T11:28:51.080153+0000","last_became_active":"2023-08-27T11:28:49.087656+0000","last_became_peered":"2023-08-27T11:28:49.087656+0000","last_unstale":"2023-08-27T11:28:51.080153+0000","last_undegraded":"2023-08-27T11:28:51.080153+0000","last_fullsized":"2023-08-27T11:28:51.080153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:12.970832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078217+0000","last_change":"2023-08-27T11:28:50.082666+0000","last_active":"2023-08-27T11:28:51.078217+0000","last_peered":"2023-08-27T11:28:51.078217+0000","last_clean":"2023-08-27T11:28:51.078217+0000","last_became_active":"2023-08-27T11:28:50.082406+0000","last_became_peered":"2023-08-27T11:28:50.082406+0000","last_unstale":"2023-08-27T11:28:51.078217+0000","last_undegraded":"2023-08-27T11:28:51.078217+0000","last_fullsized":"2023-08-27T11:28:51.078217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:55:36.976832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"114'8","reported_seq":21,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560198+0000","last_change":"2023-08-27T11:28:49.092647+0000","last_active":"2023-08-27T11:28:51.560198+0000","last_peered":"2023-08-27T11:28:51.560198+0000","last_clean":"2023-08-27T11:28:51.560198+0000","last_became_active":"2023-08-27T11:28:49.092425+0000","last_became_peered":"2023-08-27T11:28:49.092425+0000","last_unstale":"2023-08-27T11:28:51.560198+0000","last_undegraded":"2023-08-27T11:28:51.560198+0000","last_fullsized":"2023-08-27T11:28:51.560198+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:45:23.352644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232958+0000","last_change":"2023-08-27T11:28:50.089902+0000","last_active":"2023-08-27T11:28:51.232958+0000","last_peered":"2023-08-27T11:28:51.232958+0000","last_clean":"2023-08-27T11:28:51.232958+0000","last_became_active":"2023-08-27T11:28:50.089251+0000","last_became_peered":"2023-08-27T11:28:50.089251+0000","last_unstale":"2023-08-27T11:28:51.232958+0000","last_undegraded":"2023-08-27T11:28:51.232958+0000","last_fullsized":"2023-08-27T11:28:51.232958+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:36:10.161734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079773+0000","last_change":"2023-08-27T11:28:49.090840+0000","last_active":"2023-08-27T11:28:51.079773+0000","last_peered":"2023-08-27T11:28:51.079773+0000","last_clean":"2023-08-27T11:28:51.079773+0000","last_became_active":"2023-08-27T11:28:49.090655+0000","last_became_peered":"2023-08-27T11:28:49.090655+0000","last_unstale":"2023-08-27T11:28:51.079773+0000","last_undegraded":"2023-08-27T11:28:51.079773+0000","last_fullsized":"2023-08-27T11:28:51.079773+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:05.557628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560691+0000","last_change":"2023-08-27T11:28:50.094733+0000","last_active":"2023-08-27T11:28:51.560691+0000","last_peered":"2023-08-27T11:28:51.560691+0000","last_clean":"2023-08-27T11:28:51.560691+0000","last_became_active":"2023-08-27T11:28:50.094287+0000","last_became_peered":"2023-08-27T11:28:50.094287+0000","last_unstale":"2023-08-27T11:28:51.560691+0000","last_undegraded":"2023-08-27T11:28:51.560691+0000","last_fullsized":"2023-08-27T11:28:51.560691+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:42:44.134686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"115'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.233179+0000","last_change":"2023-08-27T11:28:49.088886+0000","last_active":"2023-08-27T11:28:51.233179+0000","last_peered":"2023-08-27T11:28:51.233179+0000","last_clean":"2023-08-27T11:28:51.233179+0000","last_became_active":"2023-08-27T11:28:49.088618+0000","last_became_peered":"2023-08-27T11:28:49.088618+0000","last_unstale":"2023-08-27T11:28:51.233179+0000","last_undegraded":"2023-08-27T11:28:51.233179+0000","last_fullsized":"2023-08-27T11:28:51.233179+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:49:18.841766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560559+0000","last_change":"2023-08-27T11:28:50.101837+0000","last_active":"2023-08-27T11:28:51.560559+0000","last_peered":"2023-08-27T11:28:51.560559+0000","last_clean":"2023-08-27T11:28:51.560559+0000","last_became_active":"2023-08-27T11:28:50.099094+0000","last_became_peered":"2023-08-27T11:28:50.099094+0000","last_unstale":"2023-08-27T11:28:51.560559+0000","last_undegraded":"2023-08-27T11:28:51.560559+0000","last_fullsized":"2023-08-27T11:28:51.560559+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:03:46.748896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232807+0000","last_change":"2023-08-27T11:28:49.098450+0000","last_active":"2023-08-27T11:28:51.232807+0000","last_peered":"2023-08-27T11:28:51.232807+0000","last_clean":"2023-08-27T11:28:51.232807+0000","last_became_active":"2023-08-27T11:28:49.098071+0000","last_became_peered":"2023-08-27T11:28:49.098071+0000","last_unstale":"2023-08-27T11:28:51.232807+0000","last_undegraded":"2023-08-27T11:28:51.232807+0000","last_fullsized":"2023-08-27T11:28:51.232807+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:27.488608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560028+0000","last_change":"2023-08-27T11:28:50.101135+0000","last_active":"2023-08-27T11:28:51.560028+0000","last_peered":"2023-08-27T11:28:51.560028+0000","last_clean":"2023-08-27T11:28:51.560028+0000","last_became_active":"2023-08-27T11:28:50.100865+0000","last_became_peered":"2023-08-27T11:28:50.100865+0000","last_unstale":"2023-08-27T11:28:51.560028+0000","last_undegraded":"2023-08-27T11:28:51.560028+0000","last_fullsized":"2023-08-27T11:28:51.560028+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:09.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560249+0000","last_change":"2023-08-27T11:28:49.094459+0000","last_active":"2023-08-27T11:28:51.560249+0000","last_peered":"2023-08-27T11:28:51.560249+0000","last_clean":"2023-08-27T11:28:51.560249+0000","last_became_active":"2023-08-27T11:28:49.094215+0000","last_became_peered":"2023-08-27T11:28:49.094215+0000","last_unstale":"2023-08-27T11:28:51.560249+0000","last_undegraded":"2023-08-27T11:28:51.560249+0000","last_fullsized":"2023-08-27T11:28:51.560249+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:08:24.766786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078551+0000","last_change":"2023-08-27T11:28:50.085192+0000","last_active":"2023-08-27T11:28:51.078551+0000","last_peered":"2023-08-27T11:28:51.078551+0000","last_clean":"2023-08-27T11:28:51.078551+0000","last_became_active":"2023-08-27T11:28:50.084976+0000","last_became_peered":"2023-08-27T11:28:50.084976+0000","last_unstale":"2023-08-27T11:28:51.078551+0000","last_undegraded":"2023-08-27T11:28:51.078551+0000","last_fullsized":"2023-08-27T11:28:51.078551+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:43.535705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232804+0000","last_change":"2023-08-27T11:28:49.091063+0000","last_active":"2023-08-27T11:28:51.232804+0000","last_peered":"2023-08-27T11:28:51.232804+0000","last_clean":"2023-08-27T11:28:51.232804+0000","last_became_active":"2023-08-27T11:28:49.090005+0000","last_became_peered":"2023-08-27T11:28:49.090005+0000","last_unstale":"2023-08-27T11:28:51.232804+0000","last_undegraded":"2023-08-27T11:28:51.232804+0000","last_fullsized":"2023-08-27T11:28:51.232804+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:47:39.218852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080373+0000","last_change":"2023-08-27T11:28:50.091587+0000","last_active":"2023-08-27T11:28:51.080373+0000","last_peered":"2023-08-27T11:28:51.080373+0000","last_clean":"2023-08-27T11:28:51.080373+0000","last_became_active":"2023-08-27T11:28:50.091335+0000","last_became_peered":"2023-08-27T11:28:50.091335+0000","last_unstale":"2023-08-27T11:28:51.080373+0000","last_undegraded":"2023-08-27T11:28:51.080373+0000","last_fullsized":"2023-08-27T11:28:51.080373+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:18:54.384473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.559845+0000","last_change":"2023-08-27T11:28:49.089703+0000","last_active":"2023-08-27T11:28:51.559845+0000","last_peered":"2023-08-27T11:28:51.559845+0000","last_clean":"2023-08-27T11:28:51.559845+0000","last_became_active":"2023-08-27T11:28:49.089514+0000","last_became_peered":"2023-08-27T11:28:49.089514+0000","last_unstale":"2023-08-27T11:28:51.559845+0000","last_undegraded":"2023-08-27T11:28:51.559845+0000","last_fullsized":"2023-08-27T11:28:51.559845+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:03:21.390643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078216+0000","last_change":"2023-08-27T11:28:50.083531+0000","last_active":"2023-08-27T11:28:51.078216+0000","last_peered":"2023-08-27T11:28:51.078216+0000","last_clean":"2023-08-27T11:28:51.078216+0000","last_became_active":"2023-08-27T11:28:50.083264+0000","last_became_peered":"2023-08-27T11:28:50.083264+0000","last_unstale":"2023-08-27T11:28:51.078216+0000","last_undegraded":"2023-08-27T11:28:51.078216+0000","last_fullsized":"2023-08-27T11:28:51.078216+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:35:10.786473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560225+0000","last_change":"2023-08-27T11:28:49.087175+0000","last_active":"2023-08-27T11:28:51.560225+0000","last_peered":"2023-08-27T11:28:51.560225+0000","last_clean":"2023-08-27T11:28:51.560225+0000","last_became_active":"2023-08-27T11:28:49.086854+0000","last_became_peered":"2023-08-27T11:28:49.086854+0000","last_unstale":"2023-08-27T11:28:51.560225+0000","last_undegraded":"2023-08-27T11:28:51.560225+0000","last_fullsized":"2023-08-27T11:28:51.560225+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:54:04.403027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.559900+0000","last_change":"2023-08-27T11:28:50.101292+0000","last_active":"2023-08-27T11:28:51.559900+0000","last_peered":"2023-08-27T11:28:51.559900+0000","last_clean":"2023-08-27T11:28:51.559900+0000","last_became_active":"2023-08-27T11:28:50.101063+0000","last_became_peered":"2023-08-27T11:28:50.101063+0000","last_unstale":"2023-08-27T11:28:51.559900+0000","last_undegraded":"2023-08-27T11:28:51.559900+0000","last_fullsized":"2023-08-27T11:28:51.559900+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:59:54.095197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078386+0000","last_change":"2023-08-27T11:28:49.096229+0000","last_active":"2023-08-27T11:28:51.078386+0000","last_peered":"2023-08-27T11:28:51.078386+0000","last_clean":"2023-08-27T11:28:51.078386+0000","last_became_active":"2023-08-27T11:28:49.095951+0000","last_became_peered":"2023-08-27T11:28:49.095951+0000","last_unstale":"2023-08-27T11:28:51.078386+0000","last_undegraded":"2023-08-27T11:28:51.078386+0000","last_fullsized":"2023-08-27T11:28:51.078386+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:03.602034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.559816+0000","last_change":"2023-08-27T11:28:50.104220+0000","last_active":"2023-08-27T11:28:51.559816+0000","last_peered":"2023-08-27T11:28:51.559816+0000","last_clean":"2023-08-27T11:28:51.559816+0000","last_became_active":"2023-08-27T11:28:50.103932+0000","last_became_peered":"2023-08-27T11:28:50.103932+0000","last_unstale":"2023-08-27T11:28:51.559816+0000","last_undegraded":"2023-08-27T11:28:51.559816+0000","last_fullsized":"2023-08-27T11:28:51.559816+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:10:42.353419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232502+0000","last_change":"2023-08-27T11:28:49.095920+0000","last_active":"2023-08-27T11:28:51.232502+0000","last_peered":"2023-08-27T11:28:51.232502+0000","last_clean":"2023-08-27T11:28:51.232502+0000","last_became_active":"2023-08-27T11:28:49.095652+0000","last_became_peered":"2023-08-27T11:28:49.095652+0000","last_unstale":"2023-08-27T11:28:51.232502+0000","last_undegraded":"2023-08-27T11:28:51.232502+0000","last_fullsized":"2023-08-27T11:28:51.232502+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:39.685707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":197,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560654+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:51.560654+0000","last_peered":"2023-08-27T11:28:51.560654+0000","last_clean":"2023-08-27T11:28:51.560654+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:51.560654+0000","last_undegraded":"2023-08-27T11:28:51.560654+0000","last_fullsized":"2023-08-27T11:28:51.560654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:45:56.474276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560204+0000","last_change":"2023-08-27T11:28:50.091202+0000","last_active":"2023-08-27T11:28:51.560204+0000","last_peered":"2023-08-27T11:28:51.560204+0000","last_clean":"2023-08-27T11:28:51.560204+0000","last_became_active":"2023-08-27T11:28:50.090944+0000","last_became_peered":"2023-08-27T11:28:50.090944+0000","last_unstale":"2023-08-27T11:28:51.560204+0000","last_undegraded":"2023-08-27T11:28:51.560204+0000","last_fullsized":"2023-08-27T11:28:51.560204+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:10:09.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080201+0000","last_change":"2023-08-27T11:28:49.092776+0000","last_active":"2023-08-27T11:28:51.080201+0000","last_peered":"2023-08-27T11:28:51.080201+0000","last_clean":"2023-08-27T11:28:51.080201+0000","last_became_active":"2023-08-27T11:28:49.092503+0000","last_became_peered":"2023-08-27T11:28:49.092503+0000","last_unstale":"2023-08-27T11:28:51.080201+0000","last_undegraded":"2023-08-27T11:28:51.080201+0000","last_fullsized":"2023-08-27T11:28:51.080201+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:50.205324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079808+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:51.079808+0000","last_peered":"2023-08-27T11:28:51.079808+0000","last_clean":"2023-08-27T11:28:51.079808+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:51.079808+0000","last_undegraded":"2023-08-27T11:28:51.079808+0000","last_fullsized":"2023-08-27T11:28:51.079808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:19:17.795666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079492+0000","last_change":"2023-08-27T11:28:50.092211+0000","last_active":"2023-08-27T11:28:51.079492+0000","last_peered":"2023-08-27T11:28:51.079492+0000","last_clean":"2023-08-27T11:28:51.079492+0000","last_became_active":"2023-08-27T11:28:50.091948+0000","last_became_peered":"2023-08-27T11:28:50.091948+0000","last_unstale":"2023-08-27T11:28:51.079492+0000","last_undegraded":"2023-08-27T11:28:51.079492+0000","last_fullsized":"2023-08-27T11:28:51.079492+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:38.420624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560018+0000","last_change":"2023-08-27T11:28:49.088492+0000","last_active":"2023-08-27T11:28:51.560018+0000","last_peered":"2023-08-27T11:28:51.560018+0000","last_clean":"2023-08-27T11:28:51.560018+0000","last_became_active":"2023-08-27T11:28:49.088156+0000","last_became_peered":"2023-08-27T11:28:49.088156+0000","last_unstale":"2023-08-27T11:28:51.560018+0000","last_undegraded":"2023-08-27T11:28:51.560018+0000","last_fullsized":"2023-08-27T11:28:51.560018+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:53.858614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":187,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078629+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:51.078629+0000","last_peered":"2023-08-27T11:28:51.078629+0000","last_clean":"2023-08-27T11:28:51.078629+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:51.078629+0000","last_undegraded":"2023-08-27T11:28:51.078629+0000","last_fullsized":"2023-08-27T11:28:51.078629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:37.164055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078655+0000","last_change":"2023-08-27T11:28:50.102567+0000","last_active":"2023-08-27T11:28:51.078655+0000","last_peered":"2023-08-27T11:28:51.078655+0000","last_clean":"2023-08-27T11:28:51.078655+0000","last_became_active":"2023-08-27T11:28:50.102276+0000","last_became_peered":"2023-08-27T11:28:50.102276+0000","last_unstale":"2023-08-27T11:28:51.078655+0000","last_undegraded":"2023-08-27T11:28:51.078655+0000","last_fullsized":"2023-08-27T11:28:51.078655+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:01.245099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231149+0000","last_change":"2023-08-27T11:28:49.098322+0000","last_active":"2023-08-27T11:28:51.231149+0000","last_peered":"2023-08-27T11:28:51.231149+0000","last_clean":"2023-08-27T11:28:51.231149+0000","last_became_active":"2023-08-27T11:28:49.098035+0000","last_became_peered":"2023-08-27T11:28:49.098035+0000","last_unstale":"2023-08-27T11:28:51.231149+0000","last_undegraded":"2023-08-27T11:28:51.231149+0000","last_fullsized":"2023-08-27T11:28:51.231149+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:19.773653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079525+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:51.079525+0000","last_peered":"2023-08-27T11:28:51.079525+0000","last_clean":"2023-08-27T11:28:51.079525+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:51.079525+0000","last_undegraded":"2023-08-27T11:28:51.079525+0000","last_fullsized":"2023-08-27T11:28:51.079525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:23:03.008848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"115'5779","reported_seq":6248,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:54.698283+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:54.698283+0000","last_peered":"2023-08-27T11:28:54.698283+0000","last_clean":"2023-08-27T11:28:54.698283+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:54.698283+0000","last_undegraded":"2023-08-27T11:28:54.698283+0000","last_fullsized":"2023-08-27T11:28:54.698283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":5779,"log_dups_size":0,"ondisk_log_size":5779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:08.159263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":767,"num_read_kb":1473,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231086+0000","last_change":"2023-08-27T11:28:50.094601+0000","last_active":"2023-08-27T11:28:51.231086+0000","last_peered":"2023-08-27T11:28:51.231086+0000","last_clean":"2023-08-27T11:28:51.231086+0000","last_became_active":"2023-08-27T11:28:50.094358+0000","last_became_peered":"2023-08-27T11:28:50.094358+0000","last_unstale":"2023-08-27T11:28:51.231086+0000","last_undegraded":"2023-08-27T11:28:51.231086+0000","last_fullsized":"2023-08-27T11:28:51.231086+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:12:03.109372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232502+0000","last_change":"2023-08-27T11:28:49.098393+0000","last_active":"2023-08-27T11:28:51.232502+0000","last_peered":"2023-08-27T11:28:51.232502+0000","last_clean":"2023-08-27T11:28:51.232502+0000","last_became_active":"2023-08-27T11:28:49.098104+0000","last_became_peered":"2023-08-27T11:28:49.098104+0000","last_unstale":"2023-08-27T11:28:51.232502+0000","last_undegraded":"2023-08-27T11:28:51.232502+0000","last_fullsized":"2023-08-27T11:28:51.232502+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:28.524596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231045+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:51.231045+0000","last_peered":"2023-08-27T11:28:51.231045+0000","last_clean":"2023-08-27T11:28:51.231045+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:51.231045+0000","last_undegraded":"2023-08-27T11:28:51.231045+0000","last_fullsized":"2023-08-27T11:28:51.231045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:50.108951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078217+0000","last_change":"2023-08-27T11:28:50.085295+0000","last_active":"2023-08-27T11:28:51.078217+0000","last_peered":"2023-08-27T11:28:51.078217+0000","last_clean":"2023-08-27T11:28:51.078217+0000","last_became_active":"2023-08-27T11:28:50.085059+0000","last_became_peered":"2023-08-27T11:28:50.085059+0000","last_unstale":"2023-08-27T11:28:51.078217+0000","last_undegraded":"2023-08-27T11:28:51.078217+0000","last_fullsized":"2023-08-27T11:28:51.078217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:47:57.873044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079425+0000","last_change":"2023-08-27T11:28:49.090592+0000","last_active":"2023-08-27T11:28:51.079425+0000","last_peered":"2023-08-27T11:28:51.079425+0000","last_clean":"2023-08-27T11:28:51.079425+0000","last_became_active":"2023-08-27T11:28:49.090309+0000","last_became_peered":"2023-08-27T11:28:49.090309+0000","last_unstale":"2023-08-27T11:28:51.079425+0000","last_undegraded":"2023-08-27T11:28:51.079425+0000","last_fullsized":"2023-08-27T11:28:51.079425+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:59.514247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231298+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:51.231298+0000","last_peered":"2023-08-27T11:28:51.231298+0000","last_clean":"2023-08-27T11:28:51.231298+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:51.231298+0000","last_undegraded":"2023-08-27T11:28:51.231298+0000","last_fullsized":"2023-08-27T11:28:51.231298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:38:01.917581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078523+0000","last_change":"2023-08-27T11:28:50.087116+0000","last_active":"2023-08-27T11:28:51.078523+0000","last_peered":"2023-08-27T11:28:51.078523+0000","last_clean":"2023-08-27T11:28:51.078523+0000","last_became_active":"2023-08-27T11:28:50.086835+0000","last_became_peered":"2023-08-27T11:28:50.086835+0000","last_unstale":"2023-08-27T11:28:51.078523+0000","last_undegraded":"2023-08-27T11:28:51.078523+0000","last_fullsized":"2023-08-27T11:28:51.078523+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:57.288907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079690+0000","last_change":"2023-08-27T11:28:49.089867+0000","last_active":"2023-08-27T11:28:51.079690+0000","last_peered":"2023-08-27T11:28:51.079690+0000","last_clean":"2023-08-27T11:28:51.079690+0000","last_became_active":"2023-08-27T11:28:49.089621+0000","last_became_peered":"2023-08-27T11:28:49.089621+0000","last_unstale":"2023-08-27T11:28:51.079690+0000","last_undegraded":"2023-08-27T11:28:51.079690+0000","last_fullsized":"2023-08-27T11:28:51.079690+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:54.413756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560785+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:51.560785+0000","last_peered":"2023-08-27T11:28:51.560785+0000","last_clean":"2023-08-27T11:28:51.560785+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:51.560785+0000","last_undegraded":"2023-08-27T11:28:51.560785+0000","last_fullsized":"2023-08-27T11:28:51.560785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:05:14.617143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078391+0000","last_change":"2023-08-27T11:28:50.100699+0000","last_active":"2023-08-27T11:28:51.078391+0000","last_peered":"2023-08-27T11:28:51.078391+0000","last_clean":"2023-08-27T11:28:51.078391+0000","last_became_active":"2023-08-27T11:28:50.100447+0000","last_became_peered":"2023-08-27T11:28:50.100447+0000","last_unstale":"2023-08-27T11:28:51.078391+0000","last_undegraded":"2023-08-27T11:28:51.078391+0000","last_fullsized":"2023-08-27T11:28:51.078391+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:20:06.351594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232678+0000","last_change":"2023-08-27T11:28:49.089632+0000","last_active":"2023-08-27T11:28:51.232678+0000","last_peered":"2023-08-27T11:28:51.232678+0000","last_clean":"2023-08-27T11:28:51.232678+0000","last_became_active":"2023-08-27T11:28:49.089329+0000","last_became_peered":"2023-08-27T11:28:49.089329+0000","last_unstale":"2023-08-27T11:28:51.232678+0000","last_undegraded":"2023-08-27T11:28:51.232678+0000","last_fullsized":"2023-08-27T11:28:51.232678+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:29.634234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560315+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:51.560315+0000","last_peered":"2023-08-27T11:28:51.560315+0000","last_clean":"2023-08-27T11:28:51.560315+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:51.560315+0000","last_undegraded":"2023-08-27T11:28:51.560315+0000","last_fullsized":"2023-08-27T11:28:51.560315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:49.255856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":767,"num_read_kb":1473,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5779,"ondisk_log_size":5779,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41468,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330372,"statfs":{"total":96636764160,"available":96594300928,"internally_reserved":0,"allocated":2019328,"data_stored":1336681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.53900000000000003,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66000000000000003,"5min":1.601,"15min":1.601},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.72799999999999998,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.247}]},{"osd":1,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.53500000000000003,"15min":0.56899999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.55100000000000005,"15min":0.58299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.32100000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":0.60199999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59499999999999997,"15min":0.629},"min":{"1min":0.32000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":1.778,"15min":1.778},"last":0.46300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58199999999999996,"15min":0.62},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.75900000000000001,"5min":0.78100000000000003,"15min":0.82999999999999996},"last":0.432},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.61099999999999999,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82299999999999995,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.44400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.72899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.501}]},{"osd":5,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.47199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88500000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48499999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.38300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575049,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":746853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56799999999999995,"15min":0.60199999999999998},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74299999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":10.992000000000001,"5min":10.992000000000001,"15min":10.992000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.624,"5min":0.78800000000000003,"15min":0.82599999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.64100000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.42699999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.67600000000000005,"15min":0.624},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":11.019,"5min":11.019,"15min":11.019},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53900000000000003,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59199999999999997,"5min":0.77200000000000002,"15min":0.83399999999999996},"last":0.46700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54600000000000004,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60999999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":1.161,"5min":0.71099999999999997,"15min":0.64400000000000002},"min":{"1min":0.36899999999999999,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":10.956,"5min":10.956,"15min":10.956},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63800000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.622,"15min":0.53500000000000003},"min":{"1min":0.33400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.069000000000001,"5min":11.069000000000001,"15min":11.069000000000001},"last":0.45600000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.31900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56599999999999995,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.376},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.65500000000000003,"5min":1.157,"15min":1.157},"last":0.47599999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":0.64700000000000002,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":11.045,"5min":11.045,"15min":11.045},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.36899999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.44400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58940,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312900,"statfs":{"total":96636764160,"available":96576409600,"internally_reserved":0,"allocated":1417216,"data_stored":746201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58915083},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.371,"15min":0.38300000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.20399999999999999},"max":{"1min":0.5,"5min":0.53500000000000003,"15min":0.65600000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.45900000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.41099999999999998,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.437},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.32900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.63300000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60199999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45900000000000002,"15min":0.48199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67000000000000004,"5min":1.169,"15min":1.169},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.439,"15min":0.439},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.23400000000000001},"max":{"1min":0.53000000000000003,"5min":1.139,"15min":1.139},"last":0.499}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.621,"15min":0.623},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.88600000000000001,"15min":2.9380000000000002},"last":0.56299999999999994}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.625,"15min":0.63800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64300000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39100000000000001},"max":{"1min":0.77100000000000002,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.55100000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.64800000000000002,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69499999999999995,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.57899999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.629},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.752,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575049,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41464,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330376,"statfs":{"total":96636764160,"available":96594305024,"internally_reserved":0,"allocated":2015232,"data_stored":1337151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61099999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55400000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.34699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40699999999999997,"15min":0.41399999999999998},"min":{"1min":0.22900000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.495,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.315},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.23599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.46700000000000003,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.29899999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42099999999999999,"15min":0.42599999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63900000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.371},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.81100000000000005},"last":0.39300000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60899999999999999,"15min":0.63800000000000001},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.86599999999999999,"15min":1.202},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.91200000000000003},"last":0.41399999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.96199999999999997,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.63500000000000001,"15min":0.66700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.248},"last":0.46000000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63800000000000001,"15min":0.65400000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.98699999999999999},"last":0.438},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0189999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.47999999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76300000000000001,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.65500000000000003,"15min":0.69199999999999995},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86699999999999999,"5min":0.90700000000000003,"15min":1.2809999999999999},"last":0.49299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1421312,"data_stored":746235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36599999999999999,"15min":0.372},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.219},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.44600000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.26900000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47099999999999997,"5min":0.60899999999999999,"15min":0.753},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.55500000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52900000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.432,"15min":0.44900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.498,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.60599999999999998,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":1.163},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.36499999999999999},"max":{"1min":1.034,"5min":1.083,"15min":1.2470000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40699999999999997},"max":{"1min":0.879,"5min":0.96799999999999997,"15min":1.2330000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81599999999999995,"5min":0.99299999999999999,"15min":1.1319999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64600000000000002,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.65400000000000003,"15min":0.67800000000000005},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.68899999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.99299999999999999,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.54800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575051,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":747835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.39100000000000001,"15min":0.38800000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.22},"max":{"1min":0.52100000000000002,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.38900000000000001,"15min":0.40899999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.42299999999999999,"5min":0.61299999999999999,"15min":0.71399999999999997},"last":0.22900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.38500000000000001,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.71999999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.439,"15min":0.46000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.245},"max":{"1min":0.53800000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.252}]},{"osd":2,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44700000000000001,"15min":0.46400000000000002},"min":{"1min":0.33200000000000002,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.49399999999999999,"5min":0.71199999999999997,"15min":0.93100000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67600000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.29999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72899999999999998,"5min":0.99399999999999999,"15min":1.0329999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78800000000000003,"5min":0.91000000000000003,"15min":1.113},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.37,"15min":0.37},"max":{"1min":0.872,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":1.0620000000000001,"15min":1.218},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65300000000000002,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":0.97199999999999998,"15min":1.2010000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.68799999999999994,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":1.0269999999999999,"15min":1.0389999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65000000000000002,"15min":0.67900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.93799999999999994,"15min":1.006},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70899999999999996,"5min":1.0389999999999999,"15min":1.0640000000000001},"last":0.48999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575049,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28424,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343416,"statfs":{"total":96636764160,"available":96607657984,"internally_reserved":0,"allocated":1437696,"data_stored":747937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55900000000000005,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.37},{"interface":"front","average":{"1min":0.501,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.255},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.873},"last":0.42199999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.76200000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.85299999999999998},"last":0.48399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.497},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.38600000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77000000000000002,"5min":0.879,"15min":0.879},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.629,"15min":0.627},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.88},"last":0.47299999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.755,"15min":0.85699999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.26200000000000001},"max":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.82799999999999996},"last":0.34899999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.73299999999999998,"15min":1.0860000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.747,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.44400000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.874},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.1339999999999999},"last":0.433}]}]},{"osd":5,"up_from":13,"seq":55834575048,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36736,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335104,"statfs":{"total":96636764160,"available":96599146496,"internally_reserved":0,"allocated":1421312,"data_stored":746271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.58399999999999996},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57599999999999996,"15min":0.59999999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.436}]},{"osd":1,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.37}]},{"osd":2,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32000000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.23100000000000001},"max":{"1min":0.60699999999999998,"5min":1.157,"15min":1.157},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.61299999999999999,"5min":0.76700000000000002,"15min":0.86599999999999999},"last":0.33500000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70699999999999996,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44600000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.27800000000000002},"max":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:56.030 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:56.028+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:56.868 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:56.866+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:57.707 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:28:57.705+0000 7fede3b07700 -1 client.6584: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6479:ae20b3d2-2c38-4a7d-b5f6-f888f9d658ee:v1:172.21.15.135:0/626141668] 2023-08-27T11:28:58.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-27T11:28:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.092+0000 7fef02299700 1 Processor -- start 2023-08-27T11:28:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.092+0000 7fef02299700 1 -- start start 2023-08-27T11:28:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef02299700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7feefc0c9b00 0x7feefc0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef02299700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feefc0cdef0 0x7feefc0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef02299700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc0cd350 con 0x7feefc1599e0 2023-08-27T11:28:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef02299700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc06bdc0 con 0x7feefc0cdef0 2023-08-27T11:28:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef02299700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc0ce880 con 0x7feefc0c9b00 2023-08-27T11:28:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef00836700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feefc0cdef0 0x7feefc0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7feefbfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feefc1599e0 0x7feefc0700b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47264/0 (socket says 172.21.15.45:47264) 2023-08-27T11:28:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7feefbfff700 1 -- 172.21.15.45:0/2354204075 learned_addr learned my addr 172.21.15.45:0/2354204075 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7feefb7fe700 1 --2- 172.21.15.45:0/2354204075 >> v2:172.21.15.135:3300/0 conn(0x7feefc0c9b00 0x7feefc0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef00836700 1 -- 172.21.15.45:0/2354204075 >> v2:172.21.15.135:3300/0 conn(0x7feefc0c9b00 msgr2=0x7feefc0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef00836700 1 --2- 172.21.15.45:0/2354204075 >> v2:172.21.15.135:3300/0 conn(0x7feefc0c9b00 0x7feefc0c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef00836700 1 -- 172.21.15.45:0/2354204075 >> v1:172.21.15.45:6789/0 conn(0x7feefc1599e0 legacy=0x7feefc0700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7fef00836700 1 -- 172.21.15.45:0/2354204075 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0ca4a0 con 0x7feefc0cdef0 2023-08-27T11:28:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.093+0000 7feefaffd700 1 -- 172.21.15.45:0/2354204075 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376519029 0 0) 0x7feefc0cd350 con 0x7feefc1599e0 2023-08-27T11:28:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7fef00836700 1 --2- 172.21.15.45:0/2354204075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0cdef0 0x7feefc0ce340 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7feefc0b1600 tx=0x7feef400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13180f78d51b795 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7feefaffd700 1 -- 172.21.15.45:0/2354204075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400e070 con 0x7feefc0cdef0 2023-08-27T11:28:59.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7feefaffd700 1 -- 172.21.15.45:0/2354204075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef400bbd0 con 0x7feefc0cdef0 2023-08-27T11:28:59.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7feefaffd700 1 -- 172.21.15.45:0/2354204075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4012430 con 0x7feefc0cdef0 2023-08-27T11:28:59.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7fef02299700 1 -- 172.21.15.45:0/2354204075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0cdef0 msgr2=0x7feefc0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7fef02299700 1 --2- 172.21.15.45:0/2354204075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0cdef0 0x7feefc0ce340 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7feefc0b1600 tx=0x7feef400b330 comp rx=0 tx=0).stop 2023-08-27T11:28:59.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7fef02299700 1 -- 172.21.15.45:0/2354204075 shutdown_connections 2023-08-27T11:28:59.100 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7fef02299700 1 --2- 172.21.15.45:0/2354204075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0cdef0 0x7feefc0ce340 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7fef02299700 1 --2- 172.21.15.45:0/2354204075 >> v2:172.21.15.135:3300/0 conn(0x7feefc0c9b00 0x7feefc0c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7fef02299700 1 -- 172.21.15.45:0/2354204075 >> 172.21.15.45:0/2354204075 conn(0x7feefc0b07c0 msgr2=0x7feefc0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:59.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7fef02299700 1 -- 172.21.15.45:0/2354204075 shutdown_connections 2023-08-27T11:28:59.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.094+0000 7fef02299700 1 -- 172.21.15.45:0/2354204075 wait complete. 2023-08-27T11:28:59.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.095+0000 7fef02299700 1 Processor -- start 2023-08-27T11:28:59.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.096+0000 7fef02299700 1 -- start start 2023-08-27T11:28:59.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.096+0000 7fef02299700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0c9b00 0x7feefc15c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.096+0000 7fef02299700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feefc0cdef0 0x7feefc0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.096+0000 7fef02299700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feefc1599e0 0x7feefc0b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.096+0000 7fef02299700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b4250 con 0x7feefc0c9b00 2023-08-27T11:28:59.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.096+0000 7fef02299700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b4670 con 0x7feefc1599e0 2023-08-27T11:28:59.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.096+0000 7fef02299700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b3320 con 0x7feefc0cdef0 2023-08-27T11:28:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.096+0000 7feefbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0c9b00 0x7feefc15c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feefbfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0c9b00 0x7feefc15c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60494/0 (socket says 172.21.15.45:60494) 2023-08-27T11:28:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feefbfff700 1 -- 172.21.15.45:0/101322463 learned_addr learned my addr 172.21.15.45:0/101322463 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feefb7fe700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feefc0cdef0 0x7feefc0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7fef00836700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feefc1599e0 0x7feefc0b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feefbfff700 1 -- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feefc0cdef0 msgr2=0x7feefc0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feefbfff700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feefc0cdef0 0x7feefc0b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feefbfff700 1 -- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feefc1599e0 msgr2=0x7feefc0b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feefbfff700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feefc1599e0 0x7feefc0b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feefbfff700 1 -- 172.21.15.45:0/101322463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef4009a70 con 0x7feefc0c9b00 2023-08-27T11:28:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feefbfff700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0c9b00 0x7feefc15c940 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7feee400c550 tx=0x7feee400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feef8ff9700 1 -- 172.21.15.45:0/101322463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4004c80 con 0x7feefc0c9b00 2023-08-27T11:28:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feef8ff9700 1 -- 172.21.15.45:0/101322463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4004e00 con 0x7feefc0c9b00 2023-08-27T11:28:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc0b35d0 con 0x7feefc0c9b00 2023-08-27T11:28:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7feef8ff9700 1 -- 172.21.15.45:0/101322463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4007310 con 0x7feefc0c9b00 2023-08-27T11:28:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.097+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc0b3b90 con 0x7feefc0c9b00 2023-08-27T11:28:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.098+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc0b4390 con 0x7feefc0c9b00 2023-08-27T11:28:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.099+0000 7feef8ff9700 1 -- 172.21.15.45:0/101322463 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7feee4007470 con 0x7feefc0c9b00 2023-08-27T11:28:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.099+0000 7feef8ff9700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7feedc06a170 0x7feedc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.099+0000 7feef8ff9700 1 -- 172.21.15.45:0/101322463 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7feee4088400 con 0x7feefc0c9b00 2023-08-27T11:28:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.099+0000 7feefb7fe700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7feedc06a170 0x7feedc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.100+0000 7feefb7fe700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7feedc06a170 0x7feedc06c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feefc071180 tx=0x7feeec008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.104+0000 7feef8ff9700 1 -- 172.21.15.45:0/101322463 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feefc0b4390 con 0x7feefc0c9b00 2023-08-27T11:28:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.304+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 --> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feefc06c2b0 con 0x7feedc06a170 2023-08-27T11:28:59.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.313+0000 7feef8ff9700 1 -- 172.21.15.45:0/101322463 <== mgr.7156 v2:172.21.15.135:6832/3759624346 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311966 (secure 0 0 0) 0x7feeec009dd0 con 0x7feedc06a170 2023-08-27T11:28:59.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:28:59.321 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7feedc06a170 msgr2=0x7feedc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7feedc06a170 0x7feedc06c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feefc071180 tx=0x7feeec008040 comp rx=0 tx=0).stop 2023-08-27T11:28:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0c9b00 msgr2=0x7feefc15c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0c9b00 0x7feefc15c940 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7feee400c550 tx=0x7feee400c910 comp rx=0 tx=0).stop 2023-08-27T11:28:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 shutdown_connections 2023-08-27T11:28:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7feedc06a170 0x7feedc06c630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc0c9b00 0x7feefc15c940 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feefc1599e0 0x7feefc0b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 --2- 172.21.15.45:0/101322463 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7feefc0cdef0 0x7feefc0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 >> 172.21.15.45:0/101322463 conn(0x7feefc0b07c0 msgr2=0x7feefc0c7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 shutdown_connections 2023-08-27T11:28:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.316+0000 7fef02299700 1 -- 172.21.15.45:0/101322463 wait complete. 2023-08-27T11:28:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-08-27T11:28:59.330 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-08-27T11:28:59.122507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":767,"num_read_kb":1473,"num_write":328,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5818,"ondisk_log_size":5818,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":304764,"kb_used_data":12296,"kb_used_omap":13,"kb_used_meta":292338,"kb_avail":754669956,"statfs":{"total":773094113280,"available":772782034944,"internally_reserved":0,"allocated":12591104,"data_stored":7155164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":299354200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.045719"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560654+0000","last_change":"2023-08-27T11:28:50.097122+0000","last_active":"2023-08-27T11:28:51.560654+0000","last_peered":"2023-08-27T11:28:51.560654+0000","last_clean":"2023-08-27T11:28:51.560654+0000","last_became_active":"2023-08-27T11:28:50.096823+0000","last_became_peered":"2023-08-27T11:28:50.096823+0000","last_unstale":"2023-08-27T11:28:51.560654+0000","last_undegraded":"2023-08-27T11:28:51.560654+0000","last_fullsized":"2023-08-27T11:28:51.560654+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:23:49.530921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232698+0000","last_change":"2023-08-27T11:28:50.090002+0000","last_active":"2023-08-27T11:28:51.232698+0000","last_peered":"2023-08-27T11:28:51.232698+0000","last_clean":"2023-08-27T11:28:51.232698+0000","last_became_active":"2023-08-27T11:28:50.089637+0000","last_became_peered":"2023-08-27T11:28:50.089637+0000","last_unstale":"2023-08-27T11:28:51.232698+0000","last_undegraded":"2023-08-27T11:28:51.232698+0000","last_fullsized":"2023-08-27T11:28:51.232698+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:15:41.115083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560378+0000","last_change":"2023-08-27T11:28:50.091610+0000","last_active":"2023-08-27T11:28:51.560378+0000","last_peered":"2023-08-27T11:28:51.560378+0000","last_clean":"2023-08-27T11:28:51.560378+0000","last_became_active":"2023-08-27T11:28:50.091310+0000","last_became_peered":"2023-08-27T11:28:50.091310+0000","last_unstale":"2023-08-27T11:28:51.560378+0000","last_undegraded":"2023-08-27T11:28:51.560378+0000","last_fullsized":"2023-08-27T11:28:51.560378+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:05:15.485701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560580+0000","last_change":"2023-08-27T11:28:50.099807+0000","last_active":"2023-08-27T11:28:51.560580+0000","last_peered":"2023-08-27T11:28:51.560580+0000","last_clean":"2023-08-27T11:28:51.560580+0000","last_became_active":"2023-08-27T11:28:50.099499+0000","last_became_peered":"2023-08-27T11:28:50.099499+0000","last_unstale":"2023-08-27T11:28:51.560580+0000","last_undegraded":"2023-08-27T11:28:51.560580+0000","last_fullsized":"2023-08-27T11:28:51.560580+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:05:28.545130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560130+0000","last_change":"2023-08-27T11:28:50.101108+0000","last_active":"2023-08-27T11:28:51.560130+0000","last_peered":"2023-08-27T11:28:51.560130+0000","last_clean":"2023-08-27T11:28:51.560130+0000","last_became_active":"2023-08-27T11:28:50.100852+0000","last_became_peered":"2023-08-27T11:28:50.100852+0000","last_unstale":"2023-08-27T11:28:51.560130+0000","last_undegraded":"2023-08-27T11:28:51.560130+0000","last_fullsized":"2023-08-27T11:28:51.560130+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:48:54.314223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230873+0000","last_change":"2023-08-27T11:28:50.091281+0000","last_active":"2023-08-27T11:28:51.230873+0000","last_peered":"2023-08-27T11:28:51.230873+0000","last_clean":"2023-08-27T11:28:51.230873+0000","last_became_active":"2023-08-27T11:28:50.091013+0000","last_became_peered":"2023-08-27T11:28:50.091013+0000","last_unstale":"2023-08-27T11:28:51.230873+0000","last_undegraded":"2023-08-27T11:28:51.230873+0000","last_fullsized":"2023-08-27T11:28:51.230873+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:15:25.273981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230921+0000","last_change":"2023-08-27T11:28:50.091458+0000","last_active":"2023-08-27T11:28:51.230921+0000","last_peered":"2023-08-27T11:28:51.230921+0000","last_clean":"2023-08-27T11:28:51.230921+0000","last_became_active":"2023-08-27T11:28:50.091225+0000","last_became_peered":"2023-08-27T11:28:50.091225+0000","last_unstale":"2023-08-27T11:28:51.230921+0000","last_undegraded":"2023-08-27T11:28:51.230921+0000","last_fullsized":"2023-08-27T11:28:51.230921+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:43:09.542275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232423+0000","last_change":"2023-08-27T11:28:50.092992+0000","last_active":"2023-08-27T11:28:51.232423+0000","last_peered":"2023-08-27T11:28:51.232423+0000","last_clean":"2023-08-27T11:28:51.232423+0000","last_became_active":"2023-08-27T11:28:50.092472+0000","last_became_peered":"2023-08-27T11:28:50.092472+0000","last_unstale":"2023-08-27T11:28:51.232423+0000","last_undegraded":"2023-08-27T11:28:51.232423+0000","last_fullsized":"2023-08-27T11:28:51.232423+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:05:23.180273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080522+0000","last_change":"2023-08-27T11:28:50.102638+0000","last_active":"2023-08-27T11:28:51.080522+0000","last_peered":"2023-08-27T11:28:51.080522+0000","last_clean":"2023-08-27T11:28:51.080522+0000","last_became_active":"2023-08-27T11:28:50.102375+0000","last_became_peered":"2023-08-27T11:28:50.102375+0000","last_unstale":"2023-08-27T11:28:51.080522+0000","last_undegraded":"2023-08-27T11:28:51.080522+0000","last_fullsized":"2023-08-27T11:28:51.080522+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:36:33.012903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560209+0000","last_change":"2023-08-27T11:28:50.102856+0000","last_active":"2023-08-27T11:28:51.560209+0000","last_peered":"2023-08-27T11:28:51.560209+0000","last_clean":"2023-08-27T11:28:51.560209+0000","last_became_active":"2023-08-27T11:28:50.102509+0000","last_became_peered":"2023-08-27T11:28:50.102509+0000","last_unstale":"2023-08-27T11:28:51.560209+0000","last_undegraded":"2023-08-27T11:28:51.560209+0000","last_fullsized":"2023-08-27T11:28:51.560209+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:20:32.689948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230962+0000","last_change":"2023-08-27T11:28:50.092360+0000","last_active":"2023-08-27T11:28:51.230962+0000","last_peered":"2023-08-27T11:28:51.230962+0000","last_clean":"2023-08-27T11:28:51.230962+0000","last_became_active":"2023-08-27T11:28:50.091909+0000","last_became_peered":"2023-08-27T11:28:50.091909+0000","last_unstale":"2023-08-27T11:28:51.230962+0000","last_undegraded":"2023-08-27T11:28:51.230962+0000","last_fullsized":"2023-08-27T11:28:51.230962+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:17:31.002100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080264+0000","last_change":"2023-08-27T11:28:50.104376+0000","last_active":"2023-08-27T11:28:51.080264+0000","last_peered":"2023-08-27T11:28:51.080264+0000","last_clean":"2023-08-27T11:28:51.080264+0000","last_became_active":"2023-08-27T11:28:50.104121+0000","last_became_peered":"2023-08-27T11:28:50.104121+0000","last_unstale":"2023-08-27T11:28:51.080264+0000","last_undegraded":"2023-08-27T11:28:51.080264+0000","last_fullsized":"2023-08-27T11:28:51.080264+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:14.712757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560408+0000","last_change":"2023-08-27T11:28:50.090400+0000","last_active":"2023-08-27T11:28:51.560408+0000","last_peered":"2023-08-27T11:28:51.560408+0000","last_clean":"2023-08-27T11:28:51.560408+0000","last_became_active":"2023-08-27T11:28:50.090097+0000","last_became_peered":"2023-08-27T11:28:50.090097+0000","last_unstale":"2023-08-27T11:28:51.560408+0000","last_undegraded":"2023-08-27T11:28:51.560408+0000","last_fullsized":"2023-08-27T11:28:51.560408+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:32:34.104328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078706+0000","last_change":"2023-08-27T11:28:50.090623+0000","last_active":"2023-08-27T11:28:51.078706+0000","last_peered":"2023-08-27T11:28:51.078706+0000","last_clean":"2023-08-27T11:28:51.078706+0000","last_became_active":"2023-08-27T11:28:50.090362+0000","last_became_peered":"2023-08-27T11:28:50.090362+0000","last_unstale":"2023-08-27T11:28:51.078706+0000","last_undegraded":"2023-08-27T11:28:51.078706+0000","last_fullsized":"2023-08-27T11:28:51.078706+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:25:03.877364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560088+0000","last_change":"2023-08-27T11:28:50.105514+0000","last_active":"2023-08-27T11:28:51.560088+0000","last_peered":"2023-08-27T11:28:51.560088+0000","last_clean":"2023-08-27T11:28:51.560088+0000","last_became_active":"2023-08-27T11:28:50.105134+0000","last_became_peered":"2023-08-27T11:28:50.105134+0000","last_unstale":"2023-08-27T11:28:51.560088+0000","last_undegraded":"2023-08-27T11:28:51.560088+0000","last_fullsized":"2023-08-27T11:28:51.560088+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:48:56.741867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230783+0000","last_change":"2023-08-27T11:28:50.089955+0000","last_active":"2023-08-27T11:28:51.230783+0000","last_peered":"2023-08-27T11:28:51.230783+0000","last_clean":"2023-08-27T11:28:51.230783+0000","last_became_active":"2023-08-27T11:28:50.089680+0000","last_became_peered":"2023-08-27T11:28:50.089680+0000","last_unstale":"2023-08-27T11:28:51.230783+0000","last_undegraded":"2023-08-27T11:28:51.230783+0000","last_fullsized":"2023-08-27T11:28:51.230783+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:35:33.140967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.233038+0000","last_change":"2023-08-27T11:28:50.101136+0000","last_active":"2023-08-27T11:28:51.233038+0000","last_peered":"2023-08-27T11:28:51.233038+0000","last_clean":"2023-08-27T11:28:51.233038+0000","last_became_active":"2023-08-27T11:28:50.100810+0000","last_became_peered":"2023-08-27T11:28:50.100810+0000","last_unstale":"2023-08-27T11:28:51.233038+0000","last_undegraded":"2023-08-27T11:28:51.233038+0000","last_fullsized":"2023-08-27T11:28:51.233038+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:07:38.715717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560940+0000","last_change":"2023-08-27T11:28:50.100982+0000","last_active":"2023-08-27T11:28:51.560940+0000","last_peered":"2023-08-27T11:28:51.560940+0000","last_clean":"2023-08-27T11:28:51.560940+0000","last_became_active":"2023-08-27T11:28:50.100676+0000","last_became_peered":"2023-08-27T11:28:50.100676+0000","last_unstale":"2023-08-27T11:28:51.560940+0000","last_undegraded":"2023-08-27T11:28:51.560940+0000","last_fullsized":"2023-08-27T11:28:51.560940+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:51.209268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078625+0000","last_change":"2023-08-27T11:28:50.094625+0000","last_active":"2023-08-27T11:28:51.078625+0000","last_peered":"2023-08-27T11:28:51.078625+0000","last_clean":"2023-08-27T11:28:51.078625+0000","last_became_active":"2023-08-27T11:28:50.094317+0000","last_became_peered":"2023-08-27T11:28:50.094317+0000","last_unstale":"2023-08-27T11:28:51.078625+0000","last_undegraded":"2023-08-27T11:28:51.078625+0000","last_fullsized":"2023-08-27T11:28:51.078625+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:44:04.674880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080485+0000","last_change":"2023-08-27T11:28:50.104689+0000","last_active":"2023-08-27T11:28:51.080485+0000","last_peered":"2023-08-27T11:28:51.080485+0000","last_clean":"2023-08-27T11:28:51.080485+0000","last_became_active":"2023-08-27T11:28:50.104424+0000","last_became_peered":"2023-08-27T11:28:50.104424+0000","last_unstale":"2023-08-27T11:28:51.080485+0000","last_undegraded":"2023-08-27T11:28:51.080485+0000","last_fullsized":"2023-08-27T11:28:51.080485+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:37:39.815694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078652+0000","last_change":"2023-08-27T11:28:50.084144+0000","last_active":"2023-08-27T11:28:51.078652+0000","last_peered":"2023-08-27T11:28:51.078652+0000","last_clean":"2023-08-27T11:28:51.078652+0000","last_became_active":"2023-08-27T11:28:50.083650+0000","last_became_peered":"2023-08-27T11:28:50.083650+0000","last_unstale":"2023-08-27T11:28:51.078652+0000","last_undegraded":"2023-08-27T11:28:51.078652+0000","last_fullsized":"2023-08-27T11:28:51.078652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:07:55.720567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232443+0000","last_change":"2023-08-27T11:28:50.091499+0000","last_active":"2023-08-27T11:28:51.232443+0000","last_peered":"2023-08-27T11:28:51.232443+0000","last_clean":"2023-08-27T11:28:51.232443+0000","last_became_active":"2023-08-27T11:28:50.091217+0000","last_became_peered":"2023-08-27T11:28:50.091217+0000","last_unstale":"2023-08-27T11:28:51.232443+0000","last_undegraded":"2023-08-27T11:28:51.232443+0000","last_fullsized":"2023-08-27T11:28:51.232443+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:26:26.655085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232433+0000","last_change":"2023-08-27T11:28:50.089444+0000","last_active":"2023-08-27T11:28:51.232433+0000","last_peered":"2023-08-27T11:28:51.232433+0000","last_clean":"2023-08-27T11:28:51.232433+0000","last_became_active":"2023-08-27T11:28:50.089137+0000","last_became_peered":"2023-08-27T11:28:50.089137+0000","last_unstale":"2023-08-27T11:28:51.232433+0000","last_undegraded":"2023-08-27T11:28:51.232433+0000","last_fullsized":"2023-08-27T11:28:51.232433+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:40:14.275788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230885+0000","last_change":"2023-08-27T11:28:50.110178+0000","last_active":"2023-08-27T11:28:51.230885+0000","last_peered":"2023-08-27T11:28:51.230885+0000","last_clean":"2023-08-27T11:28:51.230885+0000","last_became_active":"2023-08-27T11:28:50.109973+0000","last_became_peered":"2023-08-27T11:28:50.109973+0000","last_unstale":"2023-08-27T11:28:51.230885+0000","last_undegraded":"2023-08-27T11:28:51.230885+0000","last_fullsized":"2023-08-27T11:28:51.230885+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:45:23.272861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560011+0000","last_change":"2023-08-27T11:28:50.092156+0000","last_active":"2023-08-27T11:28:51.560011+0000","last_peered":"2023-08-27T11:28:51.560011+0000","last_clean":"2023-08-27T11:28:51.560011+0000","last_became_active":"2023-08-27T11:28:50.091924+0000","last_became_peered":"2023-08-27T11:28:50.091924+0000","last_unstale":"2023-08-27T11:28:51.560011+0000","last_undegraded":"2023-08-27T11:28:51.560011+0000","last_fullsized":"2023-08-27T11:28:51.560011+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:03:11.501424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232966+0000","last_change":"2023-08-27T11:28:50.098917+0000","last_active":"2023-08-27T11:28:51.232966+0000","last_peered":"2023-08-27T11:28:51.232966+0000","last_clean":"2023-08-27T11:28:51.232966+0000","last_became_active":"2023-08-27T11:28:50.098651+0000","last_became_peered":"2023-08-27T11:28:50.098651+0000","last_unstale":"2023-08-27T11:28:51.232966+0000","last_undegraded":"2023-08-27T11:28:51.232966+0000","last_fullsized":"2023-08-27T11:28:51.232966+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:28:30.076751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231217+0000","last_change":"2023-08-27T11:28:50.092166+0000","last_active":"2023-08-27T11:28:51.231217+0000","last_peered":"2023-08-27T11:28:51.231217+0000","last_clean":"2023-08-27T11:28:51.231217+0000","last_became_active":"2023-08-27T11:28:50.091564+0000","last_became_peered":"2023-08-27T11:28:50.091564+0000","last_unstale":"2023-08-27T11:28:51.231217+0000","last_undegraded":"2023-08-27T11:28:51.231217+0000","last_fullsized":"2023-08-27T11:28:51.231217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:35:12.257603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560029+0000","last_change":"2023-08-27T11:28:50.104554+0000","last_active":"2023-08-27T11:28:51.560029+0000","last_peered":"2023-08-27T11:28:51.560029+0000","last_clean":"2023-08-27T11:28:51.560029+0000","last_became_active":"2023-08-27T11:28:50.103887+0000","last_became_peered":"2023-08-27T11:28:50.103887+0000","last_unstale":"2023-08-27T11:28:51.560029+0000","last_undegraded":"2023-08-27T11:28:51.560029+0000","last_fullsized":"2023-08-27T11:28:51.560029+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:03:59.647986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232677+0000","last_change":"2023-08-27T11:28:50.104417+0000","last_active":"2023-08-27T11:28:51.232677+0000","last_peered":"2023-08-27T11:28:51.232677+0000","last_clean":"2023-08-27T11:28:51.232677+0000","last_became_active":"2023-08-27T11:28:50.104163+0000","last_became_peered":"2023-08-27T11:28:50.104163+0000","last_unstale":"2023-08-27T11:28:51.232677+0000","last_undegraded":"2023-08-27T11:28:51.232677+0000","last_fullsized":"2023-08-27T11:28:51.232677+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:10:52.138429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078238+0000","last_change":"2023-08-27T11:28:50.104134+0000","last_active":"2023-08-27T11:28:51.078238+0000","last_peered":"2023-08-27T11:28:51.078238+0000","last_clean":"2023-08-27T11:28:51.078238+0000","last_became_active":"2023-08-27T11:28:50.103919+0000","last_became_peered":"2023-08-27T11:28:50.103919+0000","last_unstale":"2023-08-27T11:28:51.078238+0000","last_undegraded":"2023-08-27T11:28:51.078238+0000","last_fullsized":"2023-08-27T11:28:51.078238+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:45:23.971858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560165+0000","last_change":"2023-08-27T11:28:50.096252+0000","last_active":"2023-08-27T11:28:51.560165+0000","last_peered":"2023-08-27T11:28:51.560165+0000","last_clean":"2023-08-27T11:28:51.560165+0000","last_became_active":"2023-08-27T11:28:50.095972+0000","last_became_peered":"2023-08-27T11:28:50.095972+0000","last_unstale":"2023-08-27T11:28:51.560165+0000","last_undegraded":"2023-08-27T11:28:51.560165+0000","last_fullsized":"2023-08-27T11:28:51.560165+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:13:26.132084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560067+0000","last_change":"2023-08-27T11:28:50.109894+0000","last_active":"2023-08-27T11:28:51.560067+0000","last_peered":"2023-08-27T11:28:51.560067+0000","last_clean":"2023-08-27T11:28:51.560067+0000","last_became_active":"2023-08-27T11:28:50.109380+0000","last_became_peered":"2023-08-27T11:28:50.109380+0000","last_unstale":"2023-08-27T11:28:51.560067+0000","last_undegraded":"2023-08-27T11:28:51.560067+0000","last_fullsized":"2023-08-27T11:28:51.560067+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:23:39.910873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078497+0000","last_change":"2023-08-27T11:28:50.086964+0000","last_active":"2023-08-27T11:28:51.078497+0000","last_peered":"2023-08-27T11:28:51.078497+0000","last_clean":"2023-08-27T11:28:51.078497+0000","last_became_active":"2023-08-27T11:28:50.086719+0000","last_became_peered":"2023-08-27T11:28:50.086719+0000","last_unstale":"2023-08-27T11:28:51.078497+0000","last_undegraded":"2023-08-27T11:28:51.078497+0000","last_fullsized":"2023-08-27T11:28:51.078497+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:39:26.224404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"114'4","reported_seq":17,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.233098+0000","last_change":"2023-08-27T11:28:49.093411+0000","last_active":"2023-08-27T11:28:51.233098+0000","last_peered":"2023-08-27T11:28:51.233098+0000","last_clean":"2023-08-27T11:28:51.233098+0000","last_became_active":"2023-08-27T11:28:49.092946+0000","last_became_peered":"2023-08-27T11:28:49.092946+0000","last_unstale":"2023-08-27T11:28:51.233098+0000","last_undegraded":"2023-08-27T11:28:51.233098+0000","last_fullsized":"2023-08-27T11:28:51.233098+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:21:34.770484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560629+0000","last_change":"2023-08-27T11:28:50.100223+0000","last_active":"2023-08-27T11:28:51.560629+0000","last_peered":"2023-08-27T11:28:51.560629+0000","last_clean":"2023-08-27T11:28:51.560629+0000","last_became_active":"2023-08-27T11:28:50.099858+0000","last_became_peered":"2023-08-27T11:28:50.099858+0000","last_unstale":"2023-08-27T11:28:51.560629+0000","last_undegraded":"2023-08-27T11:28:51.560629+0000","last_fullsized":"2023-08-27T11:28:51.560629+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:12.890450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231139+0000","last_change":"2023-08-27T11:28:49.095509+0000","last_active":"2023-08-27T11:28:51.231139+0000","last_peered":"2023-08-27T11:28:51.231139+0000","last_clean":"2023-08-27T11:28:51.231139+0000","last_became_active":"2023-08-27T11:28:49.095273+0000","last_became_peered":"2023-08-27T11:28:49.095273+0000","last_unstale":"2023-08-27T11:28:51.231139+0000","last_undegraded":"2023-08-27T11:28:51.231139+0000","last_fullsized":"2023-08-27T11:28:51.231139+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:31:15.759385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560481+0000","last_change":"2023-08-27T11:28:50.102328+0000","last_active":"2023-08-27T11:28:51.560481+0000","last_peered":"2023-08-27T11:28:51.560481+0000","last_clean":"2023-08-27T11:28:51.560481+0000","last_became_active":"2023-08-27T11:28:50.102071+0000","last_became_peered":"2023-08-27T11:28:50.102071+0000","last_unstale":"2023-08-27T11:28:51.560481+0000","last_undegraded":"2023-08-27T11:28:51.560481+0000","last_fullsized":"2023-08-27T11:28:51.560481+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:02:36.681114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232773+0000","last_change":"2023-08-27T11:28:49.093022+0000","last_active":"2023-08-27T11:28:51.232773+0000","last_peered":"2023-08-27T11:28:51.232773+0000","last_clean":"2023-08-27T11:28:51.232773+0000","last_became_active":"2023-08-27T11:28:49.092689+0000","last_became_peered":"2023-08-27T11:28:49.092689+0000","last_unstale":"2023-08-27T11:28:51.232773+0000","last_undegraded":"2023-08-27T11:28:51.232773+0000","last_fullsized":"2023-08-27T11:28:51.232773+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:54:24.228452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080750+0000","last_change":"2023-08-27T11:28:50.083219+0000","last_active":"2023-08-27T11:28:51.080750+0000","last_peered":"2023-08-27T11:28:51.080750+0000","last_clean":"2023-08-27T11:28:51.080750+0000","last_became_active":"2023-08-27T11:28:50.082971+0000","last_became_peered":"2023-08-27T11:28:50.082971+0000","last_unstale":"2023-08-27T11:28:51.080750+0000","last_undegraded":"2023-08-27T11:28:51.080750+0000","last_fullsized":"2023-08-27T11:28:51.080750+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:47:04.437174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232520+0000","last_change":"2023-08-27T11:28:49.098336+0000","last_active":"2023-08-27T11:28:51.232520+0000","last_peered":"2023-08-27T11:28:51.232520+0000","last_clean":"2023-08-27T11:28:51.232520+0000","last_became_active":"2023-08-27T11:28:49.098055+0000","last_became_peered":"2023-08-27T11:28:49.098055+0000","last_unstale":"2023-08-27T11:28:51.232520+0000","last_undegraded":"2023-08-27T11:28:51.232520+0000","last_fullsized":"2023-08-27T11:28:51.232520+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:02:43.143302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079721+0000","last_change":"2023-08-27T11:28:50.083624+0000","last_active":"2023-08-27T11:28:51.079721+0000","last_peered":"2023-08-27T11:28:51.079721+0000","last_clean":"2023-08-27T11:28:51.079721+0000","last_became_active":"2023-08-27T11:28:50.082821+0000","last_became_peered":"2023-08-27T11:28:50.082821+0000","last_unstale":"2023-08-27T11:28:51.079721+0000","last_undegraded":"2023-08-27T11:28:51.079721+0000","last_fullsized":"2023-08-27T11:28:51.079721+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:12:15.483275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560278+0000","last_change":"2023-08-27T11:28:49.088759+0000","last_active":"2023-08-27T11:28:51.560278+0000","last_peered":"2023-08-27T11:28:51.560278+0000","last_clean":"2023-08-27T11:28:51.560278+0000","last_became_active":"2023-08-27T11:28:49.088495+0000","last_became_peered":"2023-08-27T11:28:49.088495+0000","last_unstale":"2023-08-27T11:28:51.560278+0000","last_undegraded":"2023-08-27T11:28:51.560278+0000","last_fullsized":"2023-08-27T11:28:51.560278+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:44:16.706534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230960+0000","last_change":"2023-08-27T11:28:50.094281+0000","last_active":"2023-08-27T11:28:51.230960+0000","last_peered":"2023-08-27T11:28:51.230960+0000","last_clean":"2023-08-27T11:28:51.230960+0000","last_became_active":"2023-08-27T11:28:50.094033+0000","last_became_peered":"2023-08-27T11:28:50.094033+0000","last_unstale":"2023-08-27T11:28:51.230960+0000","last_undegraded":"2023-08-27T11:28:51.230960+0000","last_fullsized":"2023-08-27T11:28:51.230960+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:44:31.932038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232556+0000","last_change":"2023-08-27T11:28:49.093290+0000","last_active":"2023-08-27T11:28:51.232556+0000","last_peered":"2023-08-27T11:28:51.232556+0000","last_clean":"2023-08-27T11:28:51.232556+0000","last_became_active":"2023-08-27T11:28:49.092886+0000","last_became_peered":"2023-08-27T11:28:49.092886+0000","last_unstale":"2023-08-27T11:28:51.232556+0000","last_undegraded":"2023-08-27T11:28:51.232556+0000","last_fullsized":"2023-08-27T11:28:51.232556+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:49:44.466015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560257+0000","last_change":"2023-08-27T11:28:50.110407+0000","last_active":"2023-08-27T11:28:51.560257+0000","last_peered":"2023-08-27T11:28:51.560257+0000","last_clean":"2023-08-27T11:28:51.560257+0000","last_became_active":"2023-08-27T11:28:50.110097+0000","last_became_peered":"2023-08-27T11:28:50.110097+0000","last_unstale":"2023-08-27T11:28:51.560257+0000","last_undegraded":"2023-08-27T11:28:51.560257+0000","last_fullsized":"2023-08-27T11:28:51.560257+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:58:58.662545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560234+0000","last_change":"2023-08-27T11:28:49.093546+0000","last_active":"2023-08-27T11:28:51.560234+0000","last_peered":"2023-08-27T11:28:51.560234+0000","last_clean":"2023-08-27T11:28:51.560234+0000","last_became_active":"2023-08-27T11:28:49.093291+0000","last_became_peered":"2023-08-27T11:28:49.093291+0000","last_unstale":"2023-08-27T11:28:51.560234+0000","last_undegraded":"2023-08-27T11:28:51.560234+0000","last_fullsized":"2023-08-27T11:28:51.560234+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:41:52.496264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080021+0000","last_change":"2023-08-27T11:28:50.083832+0000","last_active":"2023-08-27T11:28:51.080021+0000","last_peered":"2023-08-27T11:28:51.080021+0000","last_clean":"2023-08-27T11:28:51.080021+0000","last_became_active":"2023-08-27T11:28:50.083581+0000","last_became_peered":"2023-08-27T11:28:50.083581+0000","last_unstale":"2023-08-27T11:28:51.080021+0000","last_undegraded":"2023-08-27T11:28:51.080021+0000","last_fullsized":"2023-08-27T11:28:51.080021+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:26:12.890450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.559914+0000","last_change":"2023-08-27T11:28:49.087457+0000","last_active":"2023-08-27T11:28:51.559914+0000","last_peered":"2023-08-27T11:28:51.559914+0000","last_clean":"2023-08-27T11:28:51.559914+0000","last_became_active":"2023-08-27T11:28:49.087210+0000","last_became_peered":"2023-08-27T11:28:49.087210+0000","last_unstale":"2023-08-27T11:28:51.559914+0000","last_undegraded":"2023-08-27T11:28:51.559914+0000","last_fullsized":"2023-08-27T11:28:51.559914+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:35:31.341715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.233132+0000","last_change":"2023-08-27T11:28:50.090070+0000","last_active":"2023-08-27T11:28:51.233132+0000","last_peered":"2023-08-27T11:28:51.233132+0000","last_clean":"2023-08-27T11:28:51.233132+0000","last_became_active":"2023-08-27T11:28:50.089748+0000","last_became_peered":"2023-08-27T11:28:50.089748+0000","last_unstale":"2023-08-27T11:28:51.233132+0000","last_undegraded":"2023-08-27T11:28:51.233132+0000","last_fullsized":"2023-08-27T11:28:51.233132+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:36:02.160998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560797+0000","last_change":"2023-08-27T11:28:49.089014+0000","last_active":"2023-08-27T11:28:51.560797+0000","last_peered":"2023-08-27T11:28:51.560797+0000","last_clean":"2023-08-27T11:28:51.560797+0000","last_became_active":"2023-08-27T11:28:49.088750+0000","last_became_peered":"2023-08-27T11:28:49.088750+0000","last_unstale":"2023-08-27T11:28:51.560797+0000","last_undegraded":"2023-08-27T11:28:51.560797+0000","last_fullsized":"2023-08-27T11:28:51.560797+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:57:17.423641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080659+0000","last_change":"2023-08-27T11:28:50.083843+0000","last_active":"2023-08-27T11:28:51.080659+0000","last_peered":"2023-08-27T11:28:51.080659+0000","last_clean":"2023-08-27T11:28:51.080659+0000","last_became_active":"2023-08-27T11:28:50.083567+0000","last_became_peered":"2023-08-27T11:28:50.083567+0000","last_unstale":"2023-08-27T11:28:51.080659+0000","last_undegraded":"2023-08-27T11:28:51.080659+0000","last_fullsized":"2023-08-27T11:28:51.080659+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:08:48.984397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.561006+0000","last_change":"2023-08-27T11:28:49.093294+0000","last_active":"2023-08-27T11:28:51.561006+0000","last_peered":"2023-08-27T11:28:51.561006+0000","last_clean":"2023-08-27T11:28:51.561006+0000","last_became_active":"2023-08-27T11:28:49.092967+0000","last_became_peered":"2023-08-27T11:28:49.092967+0000","last_unstale":"2023-08-27T11:28:51.561006+0000","last_undegraded":"2023-08-27T11:28:51.561006+0000","last_fullsized":"2023-08-27T11:28:51.561006+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:27:13.708814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560128+0000","last_change":"2023-08-27T11:28:50.092774+0000","last_active":"2023-08-27T11:28:51.560128+0000","last_peered":"2023-08-27T11:28:51.560128+0000","last_clean":"2023-08-27T11:28:51.560128+0000","last_became_active":"2023-08-27T11:28:50.092498+0000","last_became_peered":"2023-08-27T11:28:50.092498+0000","last_unstale":"2023-08-27T11:28:51.560128+0000","last_undegraded":"2023-08-27T11:28:51.560128+0000","last_fullsized":"2023-08-27T11:28:51.560128+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:47:35.903355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"114'3","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231133+0000","last_change":"2023-08-27T11:28:49.095274+0000","last_active":"2023-08-27T11:28:51.231133+0000","last_peered":"2023-08-27T11:28:51.231133+0000","last_clean":"2023-08-27T11:28:51.231133+0000","last_became_active":"2023-08-27T11:28:49.095022+0000","last_became_peered":"2023-08-27T11:28:49.095022+0000","last_unstale":"2023-08-27T11:28:51.231133+0000","last_undegraded":"2023-08-27T11:28:51.231133+0000","last_fullsized":"2023-08-27T11:28:51.231133+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:06:51.505347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.230918+0000","last_change":"2023-08-27T11:28:50.090510+0000","last_active":"2023-08-27T11:28:51.230918+0000","last_peered":"2023-08-27T11:28:51.230918+0000","last_clean":"2023-08-27T11:28:51.230918+0000","last_became_active":"2023-08-27T11:28:50.089693+0000","last_became_peered":"2023-08-27T11:28:50.089693+0000","last_unstale":"2023-08-27T11:28:51.230918+0000","last_undegraded":"2023-08-27T11:28:51.230918+0000","last_fullsized":"2023-08-27T11:28:51.230918+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:08:03.638017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"114'6","reported_seq":19,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080494+0000","last_change":"2023-08-27T11:28:49.087250+0000","last_active":"2023-08-27T11:28:51.080494+0000","last_peered":"2023-08-27T11:28:51.080494+0000","last_clean":"2023-08-27T11:28:51.080494+0000","last_became_active":"2023-08-27T11:28:49.087002+0000","last_became_peered":"2023-08-27T11:28:49.087002+0000","last_unstale":"2023-08-27T11:28:51.080494+0000","last_undegraded":"2023-08-27T11:28:51.080494+0000","last_fullsized":"2023-08-27T11:28:51.080494+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:35:50.737206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560166+0000","last_change":"2023-08-27T11:28:50.099829+0000","last_active":"2023-08-27T11:28:51.560166+0000","last_peered":"2023-08-27T11:28:51.560166+0000","last_clean":"2023-08-27T11:28:51.560166+0000","last_became_active":"2023-08-27T11:28:50.099468+0000","last_became_peered":"2023-08-27T11:28:50.099468+0000","last_unstale":"2023-08-27T11:28:51.560166+0000","last_undegraded":"2023-08-27T11:28:51.560166+0000","last_fullsized":"2023-08-27T11:28:51.560166+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:13.974776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078672+0000","last_change":"2023-08-27T11:28:49.086306+0000","last_active":"2023-08-27T11:28:51.078672+0000","last_peered":"2023-08-27T11:28:51.078672+0000","last_clean":"2023-08-27T11:28:51.078672+0000","last_became_active":"2023-08-27T11:28:49.086033+0000","last_became_peered":"2023-08-27T11:28:49.086033+0000","last_unstale":"2023-08-27T11:28:51.078672+0000","last_undegraded":"2023-08-27T11:28:51.078672+0000","last_fullsized":"2023-08-27T11:28:51.078672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:28:54.969620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560136+0000","last_change":"2023-08-27T11:28:50.103342+0000","last_active":"2023-08-27T11:28:51.560136+0000","last_peered":"2023-08-27T11:28:51.560136+0000","last_clean":"2023-08-27T11:28:51.560136+0000","last_became_active":"2023-08-27T11:28:50.103001+0000","last_became_peered":"2023-08-27T11:28:50.103001+0000","last_unstale":"2023-08-27T11:28:51.560136+0000","last_undegraded":"2023-08-27T11:28:51.560136+0000","last_fullsized":"2023-08-27T11:28:51.560136+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:24:17.069937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078700+0000","last_change":"2023-08-27T11:28:49.088909+0000","last_active":"2023-08-27T11:28:51.078700+0000","last_peered":"2023-08-27T11:28:51.078700+0000","last_clean":"2023-08-27T11:28:51.078700+0000","last_became_active":"2023-08-27T11:28:49.088497+0000","last_became_peered":"2023-08-27T11:28:49.088497+0000","last_unstale":"2023-08-27T11:28:51.078700+0000","last_undegraded":"2023-08-27T11:28:51.078700+0000","last_fullsized":"2023-08-27T11:28:51.078700+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:13:02.667709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079962+0000","last_change":"2023-08-27T11:28:50.083177+0000","last_active":"2023-08-27T11:28:51.079962+0000","last_peered":"2023-08-27T11:28:51.079962+0000","last_clean":"2023-08-27T11:28:51.079962+0000","last_became_active":"2023-08-27T11:28:50.082919+0000","last_became_peered":"2023-08-27T11:28:50.082919+0000","last_unstale":"2023-08-27T11:28:51.079962+0000","last_undegraded":"2023-08-27T11:28:51.079962+0000","last_fullsized":"2023-08-27T11:28:51.079962+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:05:28.545130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080153+0000","last_change":"2023-08-27T11:28:49.087910+0000","last_active":"2023-08-27T11:28:51.080153+0000","last_peered":"2023-08-27T11:28:51.080153+0000","last_clean":"2023-08-27T11:28:51.080153+0000","last_became_active":"2023-08-27T11:28:49.087656+0000","last_became_peered":"2023-08-27T11:28:49.087656+0000","last_unstale":"2023-08-27T11:28:51.080153+0000","last_undegraded":"2023-08-27T11:28:51.080153+0000","last_fullsized":"2023-08-27T11:28:51.080153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:29:12.970832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078217+0000","last_change":"2023-08-27T11:28:50.082666+0000","last_active":"2023-08-27T11:28:51.078217+0000","last_peered":"2023-08-27T11:28:51.078217+0000","last_clean":"2023-08-27T11:28:51.078217+0000","last_became_active":"2023-08-27T11:28:50.082406+0000","last_became_peered":"2023-08-27T11:28:50.082406+0000","last_unstale":"2023-08-27T11:28:51.078217+0000","last_undegraded":"2023-08-27T11:28:51.078217+0000","last_fullsized":"2023-08-27T11:28:51.078217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:55:36.976832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"114'8","reported_seq":21,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560198+0000","last_change":"2023-08-27T11:28:49.092647+0000","last_active":"2023-08-27T11:28:51.560198+0000","last_peered":"2023-08-27T11:28:51.560198+0000","last_clean":"2023-08-27T11:28:51.560198+0000","last_became_active":"2023-08-27T11:28:49.092425+0000","last_became_peered":"2023-08-27T11:28:49.092425+0000","last_unstale":"2023-08-27T11:28:51.560198+0000","last_undegraded":"2023-08-27T11:28:51.560198+0000","last_fullsized":"2023-08-27T11:28:51.560198+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:45:23.352644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232958+0000","last_change":"2023-08-27T11:28:50.089902+0000","last_active":"2023-08-27T11:28:51.232958+0000","last_peered":"2023-08-27T11:28:51.232958+0000","last_clean":"2023-08-27T11:28:51.232958+0000","last_became_active":"2023-08-27T11:28:50.089251+0000","last_became_peered":"2023-08-27T11:28:50.089251+0000","last_unstale":"2023-08-27T11:28:51.232958+0000","last_undegraded":"2023-08-27T11:28:51.232958+0000","last_fullsized":"2023-08-27T11:28:51.232958+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:36:10.161734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079773+0000","last_change":"2023-08-27T11:28:49.090840+0000","last_active":"2023-08-27T11:28:51.079773+0000","last_peered":"2023-08-27T11:28:51.079773+0000","last_clean":"2023-08-27T11:28:51.079773+0000","last_became_active":"2023-08-27T11:28:49.090655+0000","last_became_peered":"2023-08-27T11:28:49.090655+0000","last_unstale":"2023-08-27T11:28:51.079773+0000","last_undegraded":"2023-08-27T11:28:51.079773+0000","last_fullsized":"2023-08-27T11:28:51.079773+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T20:05:05.557628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560691+0000","last_change":"2023-08-27T11:28:50.094733+0000","last_active":"2023-08-27T11:28:51.560691+0000","last_peered":"2023-08-27T11:28:51.560691+0000","last_clean":"2023-08-27T11:28:51.560691+0000","last_became_active":"2023-08-27T11:28:50.094287+0000","last_became_peered":"2023-08-27T11:28:50.094287+0000","last_unstale":"2023-08-27T11:28:51.560691+0000","last_undegraded":"2023-08-27T11:28:51.560691+0000","last_fullsized":"2023-08-27T11:28:51.560691+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:42:44.134686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"115'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.233179+0000","last_change":"2023-08-27T11:28:49.088886+0000","last_active":"2023-08-27T11:28:51.233179+0000","last_peered":"2023-08-27T11:28:51.233179+0000","last_clean":"2023-08-27T11:28:51.233179+0000","last_became_active":"2023-08-27T11:28:49.088618+0000","last_became_peered":"2023-08-27T11:28:49.088618+0000","last_unstale":"2023-08-27T11:28:51.233179+0000","last_undegraded":"2023-08-27T11:28:51.233179+0000","last_fullsized":"2023-08-27T11:28:51.233179+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:49:18.841766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560559+0000","last_change":"2023-08-27T11:28:50.101837+0000","last_active":"2023-08-27T11:28:51.560559+0000","last_peered":"2023-08-27T11:28:51.560559+0000","last_clean":"2023-08-27T11:28:51.560559+0000","last_became_active":"2023-08-27T11:28:50.099094+0000","last_became_peered":"2023-08-27T11:28:50.099094+0000","last_unstale":"2023-08-27T11:28:51.560559+0000","last_undegraded":"2023-08-27T11:28:51.560559+0000","last_fullsized":"2023-08-27T11:28:51.560559+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:03:46.748896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232807+0000","last_change":"2023-08-27T11:28:49.098450+0000","last_active":"2023-08-27T11:28:51.232807+0000","last_peered":"2023-08-27T11:28:51.232807+0000","last_clean":"2023-08-27T11:28:51.232807+0000","last_became_active":"2023-08-27T11:28:49.098071+0000","last_became_peered":"2023-08-27T11:28:49.098071+0000","last_unstale":"2023-08-27T11:28:51.232807+0000","last_undegraded":"2023-08-27T11:28:51.232807+0000","last_fullsized":"2023-08-27T11:28:51.232807+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:58:27.488608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560028+0000","last_change":"2023-08-27T11:28:50.101135+0000","last_active":"2023-08-27T11:28:51.560028+0000","last_peered":"2023-08-27T11:28:51.560028+0000","last_clean":"2023-08-27T11:28:51.560028+0000","last_became_active":"2023-08-27T11:28:50.100865+0000","last_became_peered":"2023-08-27T11:28:50.100865+0000","last_unstale":"2023-08-27T11:28:51.560028+0000","last_undegraded":"2023-08-27T11:28:51.560028+0000","last_fullsized":"2023-08-27T11:28:51.560028+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:35:09.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560249+0000","last_change":"2023-08-27T11:28:49.094459+0000","last_active":"2023-08-27T11:28:51.560249+0000","last_peered":"2023-08-27T11:28:51.560249+0000","last_clean":"2023-08-27T11:28:51.560249+0000","last_became_active":"2023-08-27T11:28:49.094215+0000","last_became_peered":"2023-08-27T11:28:49.094215+0000","last_unstale":"2023-08-27T11:28:51.560249+0000","last_undegraded":"2023-08-27T11:28:51.560249+0000","last_fullsized":"2023-08-27T11:28:51.560249+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:08:24.766786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078551+0000","last_change":"2023-08-27T11:28:50.085192+0000","last_active":"2023-08-27T11:28:51.078551+0000","last_peered":"2023-08-27T11:28:51.078551+0000","last_clean":"2023-08-27T11:28:51.078551+0000","last_became_active":"2023-08-27T11:28:50.084976+0000","last_became_peered":"2023-08-27T11:28:50.084976+0000","last_unstale":"2023-08-27T11:28:51.078551+0000","last_undegraded":"2023-08-27T11:28:51.078551+0000","last_fullsized":"2023-08-27T11:28:51.078551+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:12:43.535705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232804+0000","last_change":"2023-08-27T11:28:49.091063+0000","last_active":"2023-08-27T11:28:51.232804+0000","last_peered":"2023-08-27T11:28:51.232804+0000","last_clean":"2023-08-27T11:28:51.232804+0000","last_became_active":"2023-08-27T11:28:49.090005+0000","last_became_peered":"2023-08-27T11:28:49.090005+0000","last_unstale":"2023-08-27T11:28:51.232804+0000","last_undegraded":"2023-08-27T11:28:51.232804+0000","last_fullsized":"2023-08-27T11:28:51.232804+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:47:39.218852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080373+0000","last_change":"2023-08-27T11:28:50.091587+0000","last_active":"2023-08-27T11:28:51.080373+0000","last_peered":"2023-08-27T11:28:51.080373+0000","last_clean":"2023-08-27T11:28:51.080373+0000","last_became_active":"2023-08-27T11:28:50.091335+0000","last_became_peered":"2023-08-27T11:28:50.091335+0000","last_unstale":"2023-08-27T11:28:51.080373+0000","last_undegraded":"2023-08-27T11:28:51.080373+0000","last_fullsized":"2023-08-27T11:28:51.080373+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:18:54.384473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.559845+0000","last_change":"2023-08-27T11:28:49.089703+0000","last_active":"2023-08-27T11:28:51.559845+0000","last_peered":"2023-08-27T11:28:51.559845+0000","last_clean":"2023-08-27T11:28:51.559845+0000","last_became_active":"2023-08-27T11:28:49.089514+0000","last_became_peered":"2023-08-27T11:28:49.089514+0000","last_unstale":"2023-08-27T11:28:51.559845+0000","last_undegraded":"2023-08-27T11:28:51.559845+0000","last_fullsized":"2023-08-27T11:28:51.559845+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:03:21.390643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078216+0000","last_change":"2023-08-27T11:28:50.083531+0000","last_active":"2023-08-27T11:28:51.078216+0000","last_peered":"2023-08-27T11:28:51.078216+0000","last_clean":"2023-08-27T11:28:51.078216+0000","last_became_active":"2023-08-27T11:28:50.083264+0000","last_became_peered":"2023-08-27T11:28:50.083264+0000","last_unstale":"2023-08-27T11:28:51.078216+0000","last_undegraded":"2023-08-27T11:28:51.078216+0000","last_fullsized":"2023-08-27T11:28:51.078216+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:35:10.786473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560225+0000","last_change":"2023-08-27T11:28:49.087175+0000","last_active":"2023-08-27T11:28:51.560225+0000","last_peered":"2023-08-27T11:28:51.560225+0000","last_clean":"2023-08-27T11:28:51.560225+0000","last_became_active":"2023-08-27T11:28:49.086854+0000","last_became_peered":"2023-08-27T11:28:49.086854+0000","last_unstale":"2023-08-27T11:28:51.560225+0000","last_undegraded":"2023-08-27T11:28:51.560225+0000","last_fullsized":"2023-08-27T11:28:51.560225+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:54:04.403027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.559900+0000","last_change":"2023-08-27T11:28:50.101292+0000","last_active":"2023-08-27T11:28:51.559900+0000","last_peered":"2023-08-27T11:28:51.559900+0000","last_clean":"2023-08-27T11:28:51.559900+0000","last_became_active":"2023-08-27T11:28:50.101063+0000","last_became_peered":"2023-08-27T11:28:50.101063+0000","last_unstale":"2023-08-27T11:28:51.559900+0000","last_undegraded":"2023-08-27T11:28:51.559900+0000","last_fullsized":"2023-08-27T11:28:51.559900+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:59:54.095197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078386+0000","last_change":"2023-08-27T11:28:49.096229+0000","last_active":"2023-08-27T11:28:51.078386+0000","last_peered":"2023-08-27T11:28:51.078386+0000","last_clean":"2023-08-27T11:28:51.078386+0000","last_became_active":"2023-08-27T11:28:49.095951+0000","last_became_peered":"2023-08-27T11:28:49.095951+0000","last_unstale":"2023-08-27T11:28:51.078386+0000","last_undegraded":"2023-08-27T11:28:51.078386+0000","last_fullsized":"2023-08-27T11:28:51.078386+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:52:03.602034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.559816+0000","last_change":"2023-08-27T11:28:50.104220+0000","last_active":"2023-08-27T11:28:51.559816+0000","last_peered":"2023-08-27T11:28:51.559816+0000","last_clean":"2023-08-27T11:28:51.559816+0000","last_became_active":"2023-08-27T11:28:50.103932+0000","last_became_peered":"2023-08-27T11:28:50.103932+0000","last_unstale":"2023-08-27T11:28:51.559816+0000","last_undegraded":"2023-08-27T11:28:51.559816+0000","last_fullsized":"2023-08-27T11:28:51.559816+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:10:42.353419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232502+0000","last_change":"2023-08-27T11:28:49.095920+0000","last_active":"2023-08-27T11:28:51.232502+0000","last_peered":"2023-08-27T11:28:51.232502+0000","last_clean":"2023-08-27T11:28:51.232502+0000","last_became_active":"2023-08-27T11:28:49.095652+0000","last_became_peered":"2023-08-27T11:28:49.095652+0000","last_unstale":"2023-08-27T11:28:51.232502+0000","last_undegraded":"2023-08-27T11:28:51.232502+0000","last_fullsized":"2023-08-27T11:28:51.232502+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T18:23:39.685707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":197,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560654+0000","last_change":"2023-08-27T11:12:03.137413+0000","last_active":"2023-08-27T11:28:51.560654+0000","last_peered":"2023-08-27T11:28:51.560654+0000","last_clean":"2023-08-27T11:28:51.560654+0000","last_became_active":"2023-08-27T11:12:03.137117+0000","last_became_peered":"2023-08-27T11:12:03.137117+0000","last_unstale":"2023-08-27T11:28:51.560654+0000","last_undegraded":"2023-08-27T11:28:51.560654+0000","last_fullsized":"2023-08-27T11:28:51.560654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:45:56.474276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560204+0000","last_change":"2023-08-27T11:28:50.091202+0000","last_active":"2023-08-27T11:28:51.560204+0000","last_peered":"2023-08-27T11:28:51.560204+0000","last_clean":"2023-08-27T11:28:51.560204+0000","last_became_active":"2023-08-27T11:28:50.090944+0000","last_became_peered":"2023-08-27T11:28:50.090944+0000","last_unstale":"2023-08-27T11:28:51.560204+0000","last_undegraded":"2023-08-27T11:28:51.560204+0000","last_fullsized":"2023-08-27T11:28:51.560204+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:10:09.507716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.080201+0000","last_change":"2023-08-27T11:28:49.092776+0000","last_active":"2023-08-27T11:28:51.080201+0000","last_peered":"2023-08-27T11:28:51.080201+0000","last_clean":"2023-08-27T11:28:51.080201+0000","last_became_active":"2023-08-27T11:28:49.092503+0000","last_became_peered":"2023-08-27T11:28:49.092503+0000","last_unstale":"2023-08-27T11:28:51.080201+0000","last_undegraded":"2023-08-27T11:28:51.080201+0000","last_fullsized":"2023-08-27T11:28:51.080201+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:47:50.205324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079808+0000","last_change":"2023-08-27T11:12:02.871946+0000","last_active":"2023-08-27T11:28:51.079808+0000","last_peered":"2023-08-27T11:28:51.079808+0000","last_clean":"2023-08-27T11:28:51.079808+0000","last_became_active":"2023-08-27T11:12:02.871519+0000","last_became_peered":"2023-08-27T11:12:02.871519+0000","last_unstale":"2023-08-27T11:28:51.079808+0000","last_undegraded":"2023-08-27T11:28:51.079808+0000","last_fullsized":"2023-08-27T11:28:51.079808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:19:17.795666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079492+0000","last_change":"2023-08-27T11:28:50.092211+0000","last_active":"2023-08-27T11:28:51.079492+0000","last_peered":"2023-08-27T11:28:51.079492+0000","last_clean":"2023-08-27T11:28:51.079492+0000","last_became_active":"2023-08-27T11:28:50.091948+0000","last_became_peered":"2023-08-27T11:28:50.091948+0000","last_unstale":"2023-08-27T11:28:51.079492+0000","last_undegraded":"2023-08-27T11:28:51.079492+0000","last_fullsized":"2023-08-27T11:28:51.079492+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:38.420624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560018+0000","last_change":"2023-08-27T11:28:49.088492+0000","last_active":"2023-08-27T11:28:51.560018+0000","last_peered":"2023-08-27T11:28:51.560018+0000","last_clean":"2023-08-27T11:28:51.560018+0000","last_became_active":"2023-08-27T11:28:49.088156+0000","last_became_peered":"2023-08-27T11:28:49.088156+0000","last_unstale":"2023-08-27T11:28:51.560018+0000","last_undegraded":"2023-08-27T11:28:51.560018+0000","last_fullsized":"2023-08-27T11:28:51.560018+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:27:53.858614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":187,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078629+0000","last_change":"2023-08-27T11:12:03.137505+0000","last_active":"2023-08-27T11:28:51.078629+0000","last_peered":"2023-08-27T11:28:51.078629+0000","last_clean":"2023-08-27T11:28:51.078629+0000","last_became_active":"2023-08-27T11:12:03.137198+0000","last_became_peered":"2023-08-27T11:12:03.137198+0000","last_unstale":"2023-08-27T11:28:51.078629+0000","last_undegraded":"2023-08-27T11:28:51.078629+0000","last_fullsized":"2023-08-27T11:28:51.078629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T16:23:37.164055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078655+0000","last_change":"2023-08-27T11:28:50.102567+0000","last_active":"2023-08-27T11:28:51.078655+0000","last_peered":"2023-08-27T11:28:51.078655+0000","last_clean":"2023-08-27T11:28:51.078655+0000","last_became_active":"2023-08-27T11:28:50.102276+0000","last_became_peered":"2023-08-27T11:28:50.102276+0000","last_unstale":"2023-08-27T11:28:51.078655+0000","last_undegraded":"2023-08-27T11:28:51.078655+0000","last_fullsized":"2023-08-27T11:28:51.078655+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:21:01.245099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231149+0000","last_change":"2023-08-27T11:28:49.098322+0000","last_active":"2023-08-27T11:28:51.231149+0000","last_peered":"2023-08-27T11:28:51.231149+0000","last_clean":"2023-08-27T11:28:51.231149+0000","last_became_active":"2023-08-27T11:28:49.098035+0000","last_became_peered":"2023-08-27T11:28:49.098035+0000","last_unstale":"2023-08-27T11:28:51.231149+0000","last_undegraded":"2023-08-27T11:28:51.231149+0000","last_fullsized":"2023-08-27T11:28:51.231149+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:29:19.773653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079525+0000","last_change":"2023-08-27T11:12:02.873084+0000","last_active":"2023-08-27T11:28:51.079525+0000","last_peered":"2023-08-27T11:28:51.079525+0000","last_clean":"2023-08-27T11:28:51.079525+0000","last_became_active":"2023-08-27T11:12:02.872810+0000","last_became_peered":"2023-08-27T11:12:02.872810+0000","last_unstale":"2023-08-27T11:28:51.079525+0000","last_undegraded":"2023-08-27T11:28:51.079525+0000","last_fullsized":"2023-08-27T11:28:51.079525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:23:03.008848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"115'5779","reported_seq":6248,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:54.698283+0000","last_change":"2023-08-27T11:11:56.267560+0000","last_active":"2023-08-27T11:28:54.698283+0000","last_peered":"2023-08-27T11:28:54.698283+0000","last_clean":"2023-08-27T11:28:54.698283+0000","last_became_active":"2023-08-27T11:11:56.266603+0000","last_became_peered":"2023-08-27T11:11:56.266603+0000","last_unstale":"2023-08-27T11:28:54.698283+0000","last_undegraded":"2023-08-27T11:28:54.698283+0000","last_fullsized":"2023-08-27T11:28:54.698283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:11:55.211026+0000","last_clean_scrub_stamp":"2023-08-27T11:11:55.211026+0000","objects_scrubbed":0,"log_size":5779,"log_dups_size":0,"ondisk_log_size":5779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:36:08.159263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":767,"num_read_kb":1473,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231086+0000","last_change":"2023-08-27T11:28:50.094601+0000","last_active":"2023-08-27T11:28:51.231086+0000","last_peered":"2023-08-27T11:28:51.231086+0000","last_clean":"2023-08-27T11:28:51.231086+0000","last_became_active":"2023-08-27T11:28:50.094358+0000","last_became_peered":"2023-08-27T11:28:50.094358+0000","last_unstale":"2023-08-27T11:28:51.231086+0000","last_undegraded":"2023-08-27T11:28:51.231086+0000","last_fullsized":"2023-08-27T11:28:51.231086+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T21:12:03.109372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232502+0000","last_change":"2023-08-27T11:28:49.098393+0000","last_active":"2023-08-27T11:28:51.232502+0000","last_peered":"2023-08-27T11:28:51.232502+0000","last_clean":"2023-08-27T11:28:51.232502+0000","last_became_active":"2023-08-27T11:28:49.098104+0000","last_became_peered":"2023-08-27T11:28:49.098104+0000","last_unstale":"2023-08-27T11:28:51.232502+0000","last_undegraded":"2023-08-27T11:28:51.232502+0000","last_fullsized":"2023-08-27T11:28:51.232502+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T12:42:28.524596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231045+0000","last_change":"2023-08-27T11:12:02.874759+0000","last_active":"2023-08-27T11:28:51.231045+0000","last_peered":"2023-08-27T11:28:51.231045+0000","last_clean":"2023-08-27T11:28:51.231045+0000","last_became_active":"2023-08-27T11:12:02.874435+0000","last_became_peered":"2023-08-27T11:12:02.874435+0000","last_unstale":"2023-08-27T11:28:51.231045+0000","last_undegraded":"2023-08-27T11:28:51.231045+0000","last_fullsized":"2023-08-27T11:28:51.231045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:21:50.108951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078217+0000","last_change":"2023-08-27T11:28:50.085295+0000","last_active":"2023-08-27T11:28:51.078217+0000","last_peered":"2023-08-27T11:28:51.078217+0000","last_clean":"2023-08-27T11:28:51.078217+0000","last_became_active":"2023-08-27T11:28:50.085059+0000","last_became_peered":"2023-08-27T11:28:50.085059+0000","last_unstale":"2023-08-27T11:28:51.078217+0000","last_undegraded":"2023-08-27T11:28:51.078217+0000","last_fullsized":"2023-08-27T11:28:51.078217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:47:57.873044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079425+0000","last_change":"2023-08-27T11:28:49.090592+0000","last_active":"2023-08-27T11:28:51.079425+0000","last_peered":"2023-08-27T11:28:51.079425+0000","last_clean":"2023-08-27T11:28:51.079425+0000","last_became_active":"2023-08-27T11:28:49.090309+0000","last_became_peered":"2023-08-27T11:28:49.090309+0000","last_unstale":"2023-08-27T11:28:51.079425+0000","last_undegraded":"2023-08-27T11:28:51.079425+0000","last_fullsized":"2023-08-27T11:28:51.079425+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T22:15:59.514247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.231298+0000","last_change":"2023-08-27T11:12:03.137580+0000","last_active":"2023-08-27T11:28:51.231298+0000","last_peered":"2023-08-27T11:28:51.231298+0000","last_clean":"2023-08-27T11:28:51.231298+0000","last_became_active":"2023-08-27T11:12:03.137316+0000","last_became_peered":"2023-08-27T11:12:03.137316+0000","last_unstale":"2023-08-27T11:28:51.231298+0000","last_undegraded":"2023-08-27T11:28:51.231298+0000","last_fullsized":"2023-08-27T11:28:51.231298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T11:38:01.917581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078523+0000","last_change":"2023-08-27T11:28:50.087116+0000","last_active":"2023-08-27T11:28:51.078523+0000","last_peered":"2023-08-27T11:28:51.078523+0000","last_clean":"2023-08-27T11:28:51.078523+0000","last_became_active":"2023-08-27T11:28:50.086835+0000","last_became_peered":"2023-08-27T11:28:50.086835+0000","last_unstale":"2023-08-27T11:28:51.078523+0000","last_undegraded":"2023-08-27T11:28:51.078523+0000","last_fullsized":"2023-08-27T11:28:51.078523+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T14:56:57.288907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.079690+0000","last_change":"2023-08-27T11:28:49.089867+0000","last_active":"2023-08-27T11:28:51.079690+0000","last_peered":"2023-08-27T11:28:51.079690+0000","last_clean":"2023-08-27T11:28:51.079690+0000","last_became_active":"2023-08-27T11:28:49.089621+0000","last_became_peered":"2023-08-27T11:28:49.089621+0000","last_unstale":"2023-08-27T11:28:51.079690+0000","last_undegraded":"2023-08-27T11:28:51.079690+0000","last_fullsized":"2023-08-27T11:28:51.079690+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T23:05:54.413756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560785+0000","last_change":"2023-08-27T11:12:02.873570+0000","last_active":"2023-08-27T11:28:51.560785+0000","last_peered":"2023-08-27T11:28:51.560785+0000","last_clean":"2023-08-27T11:28:51.560785+0000","last_became_active":"2023-08-27T11:12:02.873316+0000","last_became_peered":"2023-08-27T11:12:02.873316+0000","last_unstale":"2023-08-27T11:28:51.560785+0000","last_undegraded":"2023-08-27T11:28:51.560785+0000","last_fullsized":"2023-08-27T11:28:51.560785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T13:05:14.617143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.078391+0000","last_change":"2023-08-27T11:28:50.100699+0000","last_active":"2023-08-27T11:28:51.078391+0000","last_peered":"2023-08-27T11:28:51.078391+0000","last_clean":"2023-08-27T11:28:51.078391+0000","last_became_active":"2023-08-27T11:28:50.100447+0000","last_became_peered":"2023-08-27T11:28:50.100447+0000","last_unstale":"2023-08-27T11:28:51.078391+0000","last_undegraded":"2023-08-27T11:28:51.078391+0000","last_fullsized":"2023-08-27T11:28:51.078391+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:49.069412+0000","last_clean_scrub_stamp":"2023-08-27T11:28:49.069412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T19:20:06.351594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.232678+0000","last_change":"2023-08-27T11:28:49.089632+0000","last_active":"2023-08-27T11:28:51.232678+0000","last_peered":"2023-08-27T11:28:51.232678+0000","last_clean":"2023-08-27T11:28:51.232678+0000","last_became_active":"2023-08-27T11:28:49.089329+0000","last_became_peered":"2023-08-27T11:28:49.089329+0000","last_unstale":"2023-08-27T11:28:51.232678+0000","last_undegraded":"2023-08-27T11:28:51.232678+0000","last_fullsized":"2023-08-27T11:28:51.232678+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:28:48.065469+0000","last_clean_scrub_stamp":"2023-08-27T11:28:48.065469+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T17:57:29.634234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-27T11:28:51.560315+0000","last_change":"2023-08-27T11:12:02.874521+0000","last_active":"2023-08-27T11:28:51.560315+0000","last_peered":"2023-08-27T11:28:51.560315+0000","last_clean":"2023-08-27T11:28:51.560315+0000","last_became_active":"2023-08-27T11:12:02.874242+0000","last_became_peered":"2023-08-27T11:12:02.874242+0000","last_unstale":"2023-08-27T11:28:51.560315+0000","last_undegraded":"2023-08-27T11:28:51.560315+0000","last_fullsized":"2023-08-27T11:28:51.560315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-27T11:12:01.854266+0000","last_clean_scrub_stamp":"2023-08-27T11:12:01.854266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-28T15:00:49.255856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":767,"num_read_kb":1473,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5779,"ondisk_log_size":5779,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575049,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41468,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330372,"statfs":{"total":96636764160,"available":96594300928,"internally_reserved":0,"allocated":2019328,"data_stored":1336681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.53900000000000003,"15min":0.57099999999999995},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66000000000000003,"5min":1.601,"15min":1.601},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.72799999999999998,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.247}]},{"osd":1,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.53500000000000003,"15min":0.56899999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.55100000000000005,"15min":0.58299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.32100000000000001}]},{"osd":2,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":0.60199999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.59499999999999997,"15min":0.629},"min":{"1min":0.32000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.77200000000000002,"5min":1.778,"15min":1.778},"last":0.46300000000000002}]},{"osd":3,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58199999999999996,"15min":0.62},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.75900000000000001,"5min":0.78100000000000003,"15min":0.82999999999999996},"last":0.432},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.61099999999999999,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82299999999999995,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.44400000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.72899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.501}]},{"osd":5,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47899999999999998,"15min":0.47199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88500000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48499999999999999}]},{"osd":6,"last update":"Sun Aug 27 11:28:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.38300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575049,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":746853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.56799999999999995,"15min":0.60199999999999998},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.74299999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":10.992000000000001,"5min":10.992000000000001,"15min":10.992000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56599999999999995,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.309,"15min":0.27600000000000002},"max":{"1min":0.624,"5min":0.78800000000000003,"15min":0.82599999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.64100000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.42699999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":0.67600000000000005,"15min":0.624},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":11.019,"5min":11.019,"15min":11.019},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53900000000000003,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.59199999999999997,"5min":0.77200000000000002,"15min":0.83399999999999996},"last":0.46700000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54600000000000004,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60999999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":1.161,"5min":0.71099999999999997,"15min":0.64400000000000002},"min":{"1min":0.36899999999999999,"5min":0.36099999999999999,"15min":0.32800000000000001},"max":{"1min":10.956,"5min":10.956,"15min":10.956},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63800000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.622,"15min":0.53500000000000003},"min":{"1min":0.33400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":11.069000000000001,"5min":11.069000000000001,"15min":11.069000000000001},"last":0.45600000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.31900000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56599999999999995,"5min":0.80400000000000005,"15min":0.82099999999999995},"last":0.376},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35099999999999998,"15min":0.32900000000000001},"max":{"1min":0.65500000000000003,"5min":1.157,"15min":1.157},"last":0.47599999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":0.64700000000000002,"15min":0.57699999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":11.045,"5min":11.045,"15min":11.045},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.54300000000000004},"min":{"1min":0.36899999999999999,"5min":0.36199999999999999,"15min":0.34899999999999998},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.86299999999999999},"last":0.44400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58940,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312900,"statfs":{"total":96636764160,"available":96576409600,"internally_reserved":0,"allocated":1417216,"data_stored":746201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58915083},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.371,"15min":0.38300000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.20399999999999999},"max":{"1min":0.5,"5min":0.53500000000000003,"15min":0.65600000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.254},"max":{"1min":0.61199999999999999,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.45900000000000002}]},{"osd":2,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.41099999999999998,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.73699999999999999},"last":0.437},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.32900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.63300000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60199999999999998}]},{"osd":3,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45900000000000002,"15min":0.48199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67000000000000004,"5min":1.169,"15min":1.169},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.439,"15min":0.439},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.23400000000000001},"max":{"1min":0.53000000000000003,"5min":1.139,"15min":1.139},"last":0.499}]},{"osd":4,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.621,"15min":0.623},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":0.97999999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.38900000000000001,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.88600000000000001,"15min":2.9380000000000002},"last":0.56299999999999994}]},{"osd":5,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.625,"15min":0.63800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64300000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39100000000000001},"max":{"1min":0.77100000000000002,"5min":0.94199999999999995,"15min":1.0529999999999999},"last":0.55100000000000005}]},{"osd":6,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.64800000000000002,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69499999999999995,"5min":1.1060000000000001,"15min":2.9740000000000002},"last":0.57899999999999996}]},{"osd":7,"last update":"Sun Aug 27 11:28:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.629},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.752,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575049,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41464,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330376,"statfs":{"total":96636764160,"available":96594305024,"internally_reserved":0,"allocated":2015232,"data_stored":1337151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40433931},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.39400000000000002,"15min":0.39800000000000002},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61099999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55400000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.34699999999999998}]},{"osd":2,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40699999999999997,"15min":0.41399999999999998},"min":{"1min":0.22900000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.495,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.315},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.42199999999999999,"15min":0.41299999999999998},"min":{"1min":0.23599999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.46700000000000003,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.29899999999999999}]},{"osd":3,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42099999999999999,"15min":0.42599999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63900000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.371},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.48899999999999999,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.316,"15min":0.29699999999999999},"max":{"1min":0.59799999999999998,"5min":0.64900000000000002,"15min":0.81100000000000005},"last":0.39300000000000002}]},{"osd":4,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60899999999999999,"15min":0.63800000000000001},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.86599999999999999,"15min":1.202},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.27900000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.91200000000000003},"last":0.41399999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.29899999999999999},"max":{"1min":0.96199999999999997,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.63500000000000001,"15min":0.66700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.248},"last":0.46000000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63800000000000001,"15min":0.65400000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.98699999999999999},"last":0.438},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66400000000000003,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0189999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.47999999999999998}]},{"osd":7,"last update":"Sun Aug 27 11:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76300000000000001,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.65500000000000003,"15min":0.69199999999999995},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86699999999999999,"5min":0.90700000000000003,"15min":1.2809999999999999},"last":0.49299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1421312,"data_stored":746235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.36599999999999999,"15min":0.372},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.497,"5min":0.63100000000000001,"15min":0.65400000000000003},"last":0.219},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.44600000000000001,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.26900000000000002}]},{"osd":1,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.38600000000000001,"15min":0.39300000000000002},"min":{"1min":0.22900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47099999999999997,"5min":0.60899999999999999,"15min":0.753},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.42999999999999999,"15min":0.44800000000000001},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.24099999999999999},"max":{"1min":0.55500000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.45000000000000001,"15min":0.46500000000000002},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52900000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.432,"15min":0.44900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.498,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.60599999999999998,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":1.163},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.57499999999999996}]},{"osd":5,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.36499999999999999},"max":{"1min":1.034,"5min":1.083,"15min":1.2470000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40699999999999997},"max":{"1min":0.879,"5min":0.96799999999999997,"15min":1.2330000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81599999999999995,"5min":0.99299999999999999,"15min":1.1319999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64600000000000002,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":1.0189999999999999,"15min":1.1910000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Sun Aug 27 11:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.65400000000000003,"15min":0.67800000000000005},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.2150000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.68899999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.42299999999999999},"max":{"1min":0.99299999999999999,"5min":1.0409999999999999,"15min":1.2789999999999999},"last":0.54800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575051,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":747835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31914251},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.39100000000000001,"15min":0.38800000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.22},"max":{"1min":0.52100000000000002,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.38900000000000001,"15min":0.40899999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.42299999999999999,"5min":0.61299999999999999,"15min":0.71399999999999997},"last":0.22900000000000001}]},{"osd":1,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.38500000000000001,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.71999999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.439,"15min":0.46000000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.245},"max":{"1min":0.53800000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.252}]},{"osd":2,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44700000000000001,"15min":0.46400000000000002},"min":{"1min":0.33200000000000002,"5min":0.29799999999999999,"15min":0.27100000000000002},"max":{"1min":0.49399999999999999,"5min":0.71199999999999997,"15min":0.93100000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.47599999999999998,"15min":0.497},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67600000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.29999999999999999}]},{"osd":4,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72899999999999998,"5min":0.99399999999999999,"15min":1.0329999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78800000000000003,"5min":0.91000000000000003,"15min":1.113},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.436,"5min":0.37,"15min":0.37},"max":{"1min":0.872,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.41399999999999998,"15min":0.40000000000000002},"max":{"1min":0.77800000000000002,"5min":1.0620000000000001,"15min":1.218},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65300000000000002,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76800000000000002,"5min":0.97199999999999998,"15min":1.2010000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.68799999999999994,"15min":0.71599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":1.0269999999999999,"15min":1.0389999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Sun Aug 27 11:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65000000000000002,"15min":0.67900000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69799999999999995,"5min":0.93799999999999994,"15min":1.006},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.67300000000000004,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.70899999999999996,"5min":1.0389999999999999,"15min":1.0640000000000001},"last":0.48999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575049,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28424,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343416,"statfs":{"total":96636764160,"available":96607657984,"internally_reserved":0,"allocated":1437696,"data_stored":747937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55900000000000005,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.23499999999999999},"max":{"1min":0.79000000000000004,"5min":0.878,"15min":0.89800000000000002},"last":0.37},{"interface":"front","average":{"1min":0.501,"5min":0.58899999999999997,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.255},"max":{"1min":0.77800000000000002,"5min":0.86499999999999999,"15min":0.873},"last":0.42199999999999999}]},{"osd":1,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.76200000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.85299999999999998},"last":0.48399999999999999}]},{"osd":2,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.497},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.38600000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77000000000000002,"5min":0.879,"15min":0.879},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.629,"15min":0.627},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.88},"last":0.47299999999999998}]},{"osd":5,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.48499999999999999,"15min":0.48799999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.64000000000000001,"5min":0.755,"15min":0.85699999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.26200000000000001},"max":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.82799999999999996},"last":0.34899999999999998}]},{"osd":6,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.73299999999999998,"15min":1.0860000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.747,"5min":0.83499999999999996,"15min":0.93899999999999995},"last":0.44400000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.874},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":1.1339999999999999},"last":0.433}]}]},{"osd":5,"up_from":13,"seq":55834575048,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36736,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335104,"statfs":{"total":96636764160,"available":96599146496,"internally_reserved":0,"allocated":1421312,"data_stored":746271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36174091},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.58399999999999996},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.57599999999999996,"15min":0.59999999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.26200000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.436}]},{"osd":1,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.248},"max":{"1min":0.84799999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.56000000000000005,"15min":0.58299999999999996},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.72499999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.37}]},{"osd":2,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.60199999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.59599999999999997,"15min":0.60999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":0.88300000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32000000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.23100000000000001},"max":{"1min":0.60699999999999998,"5min":1.157,"15min":1.157},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.61299999999999999,"5min":0.76700000000000002,"15min":0.86599999999999999},"last":0.33500000000000002}]},{"osd":6,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48599999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.84099999999999997,"15min":0.89000000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.42399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70699999999999996,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44600000000000001}]},{"osd":7,"last update":"Sun Aug 27 11:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.27800000000000002},"max":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.88100000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"max":{"1min":0.746,"5min":0.81200000000000006,"15min":0.84999999999999998},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-27T11:28:59.336 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-27T11:28:59.336 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-27T11:28:59.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.500+0000 7f17ebdea700 1 Processor -- start 2023-08-27T11:28:59.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.501+0000 7f17ebdea700 1 -- start start 2023-08-27T11:28:59.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.501+0000 7f17ebdea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e415d3a0 0x7f17e40d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.501+0000 7f17ebdea700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17e40d02c0 0x7f17e406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.501+0000 7f17ebdea700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e415d780 con 0x7f17e40b0320 2023-08-27T11:28:59.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.501+0000 7f17ebdea700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e40cfe30 con 0x7f17e40d02c0 2023-08-27T11:28:59.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.501+0000 7f17ebdea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40cf220 con 0x7f17e415d3a0 2023-08-27T11:28:59.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.501+0000 7f17e9385700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17e40d02c0 0x7f17e406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e9385700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17e40d02c0 0x7f17e406bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60502/0 (socket says 172.21.15.45:60502) 2023-08-27T11:28:59.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e9385700 1 -- 172.21.15.45:0/2034573132 learned_addr learned my addr 172.21.15.45:0/2034573132 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e9b86700 1 --2- 172.21.15.45:0/2034573132 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e415d3a0 0x7f17e40d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e9385700 1 -- 172.21.15.45:0/2034573132 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e415d3a0 msgr2=0x7f17e40d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e9385700 1 --2- 172.21.15.45:0/2034573132 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e415d3a0 0x7f17e40d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e9385700 1 -- 172.21.15.45:0/2034573132 >> v1:172.21.15.135:6789/0 conn(0x7f17e40b0320 legacy=0x7f17e40cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e9385700 1 -- 172.21.15.45:0/2034573132 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e40cf500 con 0x7f17e40d02c0 2023-08-27T11:28:59.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e9385700 1 --2- 172.21.15.45:0/2034573132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e40d02c0 0x7f17e406bda0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098f0 tx=0x7f17e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbe579535d93e0a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e8b84700 1 -- 172.21.15.45:0/2034573132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e000b8a0 con 0x7f17e40d02c0 2023-08-27T11:28:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e8b84700 1 -- 172.21.15.45:0/2034573132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e000ba00 con 0x7f17e40d02c0 2023-08-27T11:28:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.502+0000 7f17e8b84700 1 -- 172.21.15.45:0/2034573132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e000bd90 con 0x7f17e40d02c0 2023-08-27T11:28:59.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.503+0000 7f17ebdea700 1 -- 172.21.15.45:0/2034573132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e40d02c0 msgr2=0x7f17e406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.503+0000 7f17ebdea700 1 --2- 172.21.15.45:0/2034573132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e40d02c0 0x7f17e406bda0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098f0 tx=0x7f17e0009c00 comp rx=0 tx=0).stop 2023-08-27T11:28:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.503+0000 7f17ebdea700 1 -- 172.21.15.45:0/2034573132 shutdown_connections 2023-08-27T11:28:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.503+0000 7f17ebdea700 1 --2- 172.21.15.45:0/2034573132 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e415d3a0 0x7f17e40d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.503+0000 7f17ebdea700 1 --2- 172.21.15.45:0/2034573132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e40d02c0 0x7f17e406bda0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.503+0000 7f17ebdea700 1 -- 172.21.15.45:0/2034573132 >> 172.21.15.45:0/2034573132 conn(0x7f17e409a1a0 msgr2=0x7f17e40d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.503+0000 7f17ebdea700 1 -- 172.21.15.45:0/2034573132 shutdown_connections 2023-08-27T11:28:59.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.503+0000 7f17ebdea700 1 -- 172.21.15.45:0/2034573132 wait complete. 2023-08-27T11:28:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.504+0000 7f17ebdea700 1 Processor -- start 2023-08-27T11:28:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.504+0000 7f17ebdea700 1 -- start start 2023-08-27T11:28:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17ebdea700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e40b0320 0x7f17e41f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17ebdea700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e40d02c0 0x7f17e40b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17ebdea700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e415d3a0 0x7f17e40b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17ebdea700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40c49c0 con 0x7f17e415d3a0 2023-08-27T11:28:59.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17ebdea700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40c4ca0 con 0x7f17e40d02c0 2023-08-27T11:28:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17ebdea700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40c4f80 con 0x7f17e40b0320 2023-08-27T11:28:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17ea387700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e415d3a0 0x7f17e40b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17e9385700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e40d02c0 0x7f17e40b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17e9b86700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e40b0320 0x7f17e41f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17ea387700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e415d3a0 0x7f17e40b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60512/0 (socket says 172.21.15.45:60512) 2023-08-27T11:28:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.505+0000 7f17ea387700 1 -- 172.21.15.45:0/1336935977 learned_addr learned my addr 172.21.15.45:0/1336935977 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17ea387700 1 -- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e40b0320 msgr2=0x7f17e41f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17ea387700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e40b0320 0x7f17e41f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17ea387700 1 -- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e40d02c0 msgr2=0x7f17e40b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17ea387700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e40d02c0 0x7f17e40b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17ea387700 1 -- 172.21.15.45:0/1336935977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e0009550 con 0x7f17e415d3a0 2023-08-27T11:28:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17ea387700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e415d3a0 0x7f17e40b4690 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00cbc0 tx=0x7f17dc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17da7fc700 1 -- 172.21.15.45:0/1336935977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc003c00 con 0x7f17e415d3a0 2023-08-27T11:28:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17da7fc700 1 -- 172.21.15.45:0/1336935977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc003d60 con 0x7f17e415d3a0 2023-08-27T11:28:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e40c5260 con 0x7f17e415d3a0 2023-08-27T11:28:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17da7fc700 1 -- 172.21.15.45:0/1336935977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc00aae0 con 0x7f17e415d3a0 2023-08-27T11:28:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.506+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e40b2790 con 0x7f17e415d3a0 2023-08-27T11:28:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.507+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e40c5780 con 0x7f17e415d3a0 2023-08-27T11:28:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.508+0000 7f17da7fc700 1 -- 172.21.15.45:0/1336935977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f17dc00ac40 con 0x7f17e415d3a0 2023-08-27T11:28:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.508+0000 7f17da7fc700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f17c806a170 0x7f17c806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.509+0000 7f17da7fc700 1 -- 172.21.15.45:0/1336935977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f17dc088550 con 0x7f17e415d3a0 2023-08-27T11:28:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.509+0000 7f17e9b86700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f17c806a170 0x7f17c806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.509+0000 7f17e9b86700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f17c806a170 0x7f17c806c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b39b0 tx=0x7f17d4008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.513+0000 7f17da7fc700 1 -- 172.21.15.45:0/1336935977 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17e40c5780 con 0x7f17e415d3a0 2023-08-27T11:28:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.799+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f17e415d780 con 0x7f17e415d3a0 2023-08-27T11:28:59.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.802+0000 7f17da7fc700 1 -- 172.21.15.45:0/1336935977 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f17e415d780 con 0x7f17e415d3a0 2023-08-27T11:28:59.804 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-08-27T11:28:59.805 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBKL+tkQYX8HxAAX81cmGtbnkrVCZl7XXeveQ== 2023-08-27T11:28:59.805 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2023-08-27T11:28:59.805 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2023-08-27T11:28:59.805 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-27T11:28:59.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.804+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f17c806a170 msgr2=0x7f17c806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.804+0000 7f17ebdea700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f17c806a170 0x7f17c806c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b39b0 tx=0x7f17d4008040 comp rx=0 tx=0).stop 2023-08-27T11:28:59.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.804+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e415d3a0 msgr2=0x7f17e40b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.804+0000 7f17ebdea700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e415d3a0 0x7f17e40b4690 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00cbc0 tx=0x7f17dc00ced0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.805+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 shutdown_connections 2023-08-27T11:28:59.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.805+0000 7f17ebdea700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f17c806a170 0x7f17c806c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.805+0000 7f17ebdea700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17e415d3a0 0x7f17e40b4690 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.805+0000 7f17ebdea700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17e40b0320 0x7f17e41f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.805+0000 7f17ebdea700 1 --2- 172.21.15.45:0/1336935977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e40d02c0 0x7f17e40b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.805+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 >> 172.21.15.45:0/1336935977 conn(0x7f17e409a1a0 msgr2=0x7f17e4156450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:59.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.805+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 shutdown_connections 2023-08-27T11:28:59.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.805+0000 7f17ebdea700 1 -- 172.21.15.45:0/1336935977 wait complete. 2023-08-27T11:28:59.810 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2023-08-27T11:28:59.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:28:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.942+0000 7ff15e28d700 1 Processor -- start 2023-08-27T11:28:59.945 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.942+0000 7ff15e28d700 1 -- start start 2023-08-27T11:28:59.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff15e28d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff158070e80 0x7ff15806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff15e28d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff15806a990 0x7ff15815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff15e28d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1580cbf00 con 0x7ff158155660 2023-08-27T11:28:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff15e28d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15815c760 con 0x7ff15806a990 2023-08-27T11:28:59.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff15e28d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580cc180 con 0x7ff158070e80 2023-08-27T11:28:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff157fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff158155660 0x7ff15806a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47292/0 (socket says 172.21.15.45:47292) 2023-08-27T11:28:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff157fff700 1 -- 172.21.15.45:0/3019063793 learned_addr learned my addr 172.21.15.45:0/3019063793 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:28:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 872594426 0 0) 0x7ff1580cbf00 con 0x7ff158155660 2023-08-27T11:28:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff1577fe700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff158070e80 0x7ff15806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.943+0000 7ff15c82a700 1 --2- 172.21.15.45:0/3019063793 >> v2:172.21.15.135:3300/0 conn(0x7ff15806a990 0x7ff15815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1440035b0 con 0x7ff158155660 2023-08-27T11:28:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 149654409 0 0) 0x7ff1440035b0 con 0x7ff158155660 2023-08-27T11:28:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff158070e80 msgr2=0x7ff15806e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff158070e80 0x7ff15806e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 >> v2:172.21.15.135:3300/0 conn(0x7ff15806a990 msgr2=0x7ff15815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 --2- 172.21.15.45:0/3019063793 >> v2:172.21.15.135:3300/0 conn(0x7ff15806a990 0x7ff15815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580caa60 con 0x7ff158155660 2023-08-27T11:28:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7ff148001670 con 0x7ff158155660 2023-08-27T11:28:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 >> v1:172.21.15.45:6789/0 conn(0x7ff158155660 legacy=0x7ff15806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff144003a00 0x7ff144003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff158070e80 0x7ff144004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff144004d90 0x7ff144005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15815c760 con 0x7ff144003a00 2023-08-27T11:28:59.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580cc180 con 0x7ff144004d90 2023-08-27T11:28:59.953 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14400ba10 con 0x7ff158070e80 2023-08-27T11:28:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff157fff700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff144003a00 0x7ff144003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff1577fe700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff158070e80 0x7ff144004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff15c82a700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff144004d90 0x7ff144005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff157fff700 1 -- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff158070e80 msgr2=0x7ff144004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:59.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff157fff700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff158070e80 0x7ff144004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff157fff700 1 -- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff144004d90 msgr2=0x7ff144005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:28:59.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff157fff700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff144004d90 0x7ff144005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff157fff700 1 -- 172.21.15.45:0/3019063793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1440035b0 con 0x7ff144003a00 2023-08-27T11:28:59.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.944+0000 7ff1577fe700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff158070e80 0x7ff144004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:28:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff157fff700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff144003a00 0x7ff144003e20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff14800c370 tx=0x7ff144005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb4f093996aab73 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148007c60 con 0x7ff144003a00 2023-08-27T11:28:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff148005960 con 0x7ff144003a00 2023-08-27T11:28:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff156ffd700 1 -- 172.21.15.45:0/3019063793 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148005d10 con 0x7ff144003a00 2023-08-27T11:28:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff15e28d700 1 -- 172.21.15.45:0/3019063793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff144003a00 msgr2=0x7ff144003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff15e28d700 1 --2- 172.21.15.45:0/3019063793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff144003a00 0x7ff144003e20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff14800c370 tx=0x7ff144005880 comp rx=0 tx=0).stop 2023-08-27T11:28:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff15c82a700 1 -- 172.21.15.45:0/3019063793 reap_dead start 2023-08-27T11:28:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff15e28d700 1 -- 172.21.15.45:0/3019063793 shutdown_connections 2023-08-27T11:28:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff15e28d700 1 -- 172.21.15.45:0/3019063793 >> 172.21.15.45:0/3019063793 conn(0x7ff1580cfe50 msgr2=0x7ff1580d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:28:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff15e28d700 1 -- 172.21.15.45:0/3019063793 shutdown_connections 2023-08-27T11:28:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.945+0000 7ff15e28d700 1 -- 172.21.15.45:0/3019063793 wait complete. 2023-08-27T11:28:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.947+0000 7ff15e28d700 1 Processor -- start 2023-08-27T11:28:59.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.947+0000 7ff15e28d700 1 -- start start 2023-08-27T11:28:59.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.947+0000 7ff15e28d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff144003a00 0x7ff1581f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.947+0000 7ff15e28d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1581504c0 0x7ff1580aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.947+0000 7ff15e28d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff15814eb80 0x7ff158159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15e28d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1581f28c0 con 0x7ff15814eb80 2023-08-27T11:28:59.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15e28d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1581f6360 con 0x7ff144003a00 2023-08-27T11:28:59.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15e28d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1581f6640 con 0x7ff1581504c0 2023-08-27T11:28:59.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15c82a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff15814eb80 0x7ff158159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.962 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15c82a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff15814eb80 0x7ff158159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60532/0 (socket says 172.21.15.45:60532) 2023-08-27T11:28:59.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15c82a700 1 -- 172.21.15.45:0/3132426155 learned_addr learned my addr 172.21.15.45:0/3132426155 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:28:59.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff1577fe700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1581504c0 0x7ff1580aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff157fff700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff144003a00 0x7ff1581f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15c82a700 1 -- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1581504c0 msgr2=0x7ff1580aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.963 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15c82a700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1581504c0 0x7ff1580aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15c82a700 1 -- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff144003a00 msgr2=0x7ff1581f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:28:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15c82a700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff144003a00 0x7ff1581f5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:28:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15c82a700 1 -- 172.21.15.45:0/3132426155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1581f6920 con 0x7ff15814eb80 2023-08-27T11:28:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15c82a700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff15814eb80 0x7ff158159e50 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff158070a80 tx=0x7ff150011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff154ff9700 1 -- 172.21.15.45:0/3132426155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15000e040 con 0x7ff15814eb80 2023-08-27T11:28:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff154ff9700 1 -- 172.21.15.45:0/3132426155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15000f040 con 0x7ff15814eb80 2023-08-27T11:28:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1581f6c00 con 0x7ff15814eb80 2023-08-27T11:28:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff154ff9700 1 -- 172.21.15.45:0/3132426155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15000fdb0 con 0x7ff15814eb80 2023-08-27T11:28:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.948+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff158150940 con 0x7ff15814eb80 2023-08-27T11:28:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.949+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff158072810 con 0x7ff15814eb80 2023-08-27T11:28:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.950+0000 7ff154ff9700 1 -- 172.21.15.45:0/3132426155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7ff150022460 con 0x7ff15814eb80 2023-08-27T11:28:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.950+0000 7ff154ff9700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff13806a170 0x7ff13806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:28:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.950+0000 7ff154ff9700 1 -- 172.21.15.45:0/3132426155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7ff150088d60 con 0x7ff15814eb80 2023-08-27T11:28:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.950+0000 7ff157fff700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff13806a170 0x7ff13806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:28:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.951+0000 7ff157fff700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff13806a170 0x7ff13806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff1580cde60 tx=0x7ff14800a000 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:28:59.967 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:28:59.955+0000 7ff154ff9700 1 -- 172.21.15.45:0/3132426155 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff1500541b0 con 0x7ff15814eb80 2023-08-27T11:29:00.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.203+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1580734a0 con 0x7ff15814eb80 2023-08-27T11:29:00.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.203+0000 7ff154ff9700 1 -- 172.21.15.45:0/3132426155 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4874 (secure 0 0 0) 0x7ff1500577d0 con 0x7ff15814eb80 2023-08-27T11:29:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:29:00.206 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7402,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2571849064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2571849064},{"type":"v1","addr":"172.21.15.135:6836","nonce":2571849064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/509114843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":509114843},{"type":"v1","addr":"172.21.15.135:6837","nonce":509114843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/528899713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":528899713},{"type":"v1","addr":"172.21.15.45:6833","nonce":528899713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.511894+0000","modified":"2023-08-27T11:28:53.106978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8019":{"gid":8019,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/343785163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":343785163},{"type":"v1","addr":"172.21.15.45:6835","nonce":343785163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:29:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff13806a170 msgr2=0x7ff13806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff13806a170 0x7ff13806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff1580cde60 tx=0x7ff14800a000 comp rx=0 tx=0).stop 2023-08-27T11:29:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff15814eb80 msgr2=0x7ff158159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff15814eb80 0x7ff158159e50 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff158070a80 tx=0x7ff150011f80 comp rx=0 tx=0).stop 2023-08-27T11:29:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 shutdown_connections 2023-08-27T11:29:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7ff13806a170 0x7ff13806c630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff15814eb80 0x7ff158159e50 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff144003a00 0x7ff1581f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 --2- 172.21.15.45:0/3132426155 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff1581504c0 0x7ff1580aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 >> 172.21.15.45:0/3132426155 conn(0x7ff1580cfe50 msgr2=0x7ff1580b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 shutdown_connections 2023-08-27T11:29:00.210 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.205+0000 7ff15e28d700 1 -- 172.21.15.45:0/3132426155 wait complete. 2023-08-27T11:29:00.210 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 49 2023-08-27T11:29:00.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-27T11:28:49.511894+0000', 'modified': '2023-08-27T11:28:53.106978+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8019}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8019': {'gid': 8019, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/343785163', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 343785163}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 343785163}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-27T11:29:00.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-27T11:29:00.622 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-27T11:29:00.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.750+0000 7f45fa4ab700 1 Processor -- start 2023-08-27T11:29:00.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.751+0000 7f45fa4ab700 1 -- start start 2023-08-27T11:29:00.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.751+0000 7f45fa4ab700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40ca620 0x7f45f40cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.751+0000 7f45fa4ab700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f45f40af200 0x7f45f40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.751+0000 7f45fa4ab700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40ce340 con 0x7f45f40caa00 2023-08-27T11:29:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.751+0000 7f45fa4ab700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f40ceec0 con 0x7f45f40af200 2023-08-27T11:29:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.751+0000 7f45fa4ab700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f415d7c0 con 0x7f45f40ca620 2023-08-27T11:29:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f3fff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40ca620 0x7f45f40cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f37fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f45f40af200 0x7f45f40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f37fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f45f40af200 0x7f45f40ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.45:49392/0 (socket says 172.21.15.45:49392) 2023-08-27T11:29:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f8a48700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f45f40caa00 0x7f45f40acdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:42976/0 (socket says 172.21.15.45:42976) 2023-08-27T11:29:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f37fe700 1 -- 172.21.15.45:0/2727619917 learned_addr learned my addr 172.21.15.45:0/2727619917 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f37fe700 1 -- 172.21.15.45:0/2727619917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40ca620 msgr2=0x7f45f40cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f37fe700 1 --2- 172.21.15.45:0/2727619917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40ca620 0x7f45f40cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f37fe700 1 -- 172.21.15.45:0/2727619917 >> v1:172.21.15.135:6789/0 conn(0x7f45f40caa00 legacy=0x7f45f40acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f37fe700 1 -- 172.21.15.45:0/2727619917 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40cc570 con 0x7f45f40af200 2023-08-27T11:29:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f2ffd700 1 -- 172.21.15.45:0/2727619917 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920644782 0 0) 0x7f45f40ce340 con 0x7f45f40caa00 2023-08-27T11:29:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.752+0000 7f45f37fe700 1 --2- 172.21.15.45:0/2727619917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40af200 0x7f45f40ac8b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f45e80098f0 tx=0x7f45e8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb9fa172ebc551e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45f2ffd700 1 -- 172.21.15.45:0/2727619917 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8011830 con 0x7f45f40af200 2023-08-27T11:29:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45f2ffd700 1 -- 172.21.15.45:0/2727619917 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e8011990 con 0x7f45f40af200 2023-08-27T11:29:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45f2ffd700 1 -- 172.21.15.45:0/2727619917 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8011d20 con 0x7f45f40af200 2023-08-27T11:29:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45fa4ab700 1 -- 172.21.15.45:0/2727619917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40af200 msgr2=0x7f45f40ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45fa4ab700 1 --2- 172.21.15.45:0/2727619917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40af200 0x7f45f40ac8b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f45e80098f0 tx=0x7f45e8009c00 comp rx=0 tx=0).stop 2023-08-27T11:29:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45fa4ab700 1 -- 172.21.15.45:0/2727619917 shutdown_connections 2023-08-27T11:29:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45fa4ab700 1 --2- 172.21.15.45:0/2727619917 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40ca620 0x7f45f40cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45fa4ab700 1 --2- 172.21.15.45:0/2727619917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40af200 0x7f45f40ac8b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45fa4ab700 1 -- 172.21.15.45:0/2727619917 >> 172.21.15.45:0/2727619917 conn(0x7f45f40cc7e0 msgr2=0x7f45f40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45fa4ab700 1 -- 172.21.15.45:0/2727619917 shutdown_connections 2023-08-27T11:29:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.753+0000 7f45fa4ab700 1 -- 172.21.15.45:0/2727619917 wait complete. 2023-08-27T11:29:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.754+0000 7f45fa4ab700 1 Processor -- start 2023-08-27T11:29:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45fa4ab700 1 -- start start 2023-08-27T11:29:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45fa4ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40af200 0x7f45f40c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45fa4ab700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40ca620 0x7f45f40b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45fa4ab700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40caa00 0x7f45f4154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45fa4ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40c3510 con 0x7f45f40af200 2023-08-27T11:29:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45fa4ab700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b4670 con 0x7f45f40ca620 2023-08-27T11:29:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45fa4ab700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b48f0 con 0x7f45f40caa00 2023-08-27T11:29:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40af200 0x7f45f40c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40af200 0x7f45f40c3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60538/0 (socket says 172.21.15.45:60538) 2023-08-27T11:29:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.755+0000 7f45f3fff700 1 -- 172.21.15.45:0/927195359 learned_addr learned my addr 172.21.15.45:0/927195359 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f37fe700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40ca620 0x7f45f40b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f8a48700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40caa00 0x7f45f4154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f3fff700 1 -- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40caa00 msgr2=0x7f45f4154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f3fff700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40caa00 0x7f45f4154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f3fff700 1 -- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40ca620 msgr2=0x7f45f40b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f3fff700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40ca620 0x7f45f40b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f3fff700 1 -- 172.21.15.45:0/927195359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e8009550 con 0x7f45f40af200 2023-08-27T11:29:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f37fe700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40ca620 0x7f45f40b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f3fff700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40af200 0x7f45f40c3f60 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f45e400e9b0 tx=0x7f45e400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f0ff9700 1 -- 172.21.15.45:0/927195359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e4009710 con 0x7f45f40af200 2023-08-27T11:29:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f0ff9700 1 -- 172.21.15.45:0/927195359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e400c850 con 0x7f45f40af200 2023-08-27T11:29:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45fa4ab700 1 -- 172.21.15.45:0/927195359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f40b4bd0 con 0x7f45f40af200 2023-08-27T11:29:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45f0ff9700 1 -- 172.21.15.45:0/927195359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e400cbc0 con 0x7f45f40af200 2023-08-27T11:29:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.756+0000 7f45fa4ab700 1 -- 172.21.15.45:0/927195359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f40c3a40 con 0x7f45f40af200 2023-08-27T11:29:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.758+0000 7f45f0ff9700 1 -- 172.21.15.45:0/927195359 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 45) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f45e400cd20 con 0x7f45f40af200 2023-08-27T11:29:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.758+0000 7f45f0ff9700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f45e006a170 0x7f45e006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.758+0000 7f45f0ff9700 1 -- 172.21.15.45:0/927195359 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f45e4088930 con 0x7f45f40af200 2023-08-27T11:29:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.758+0000 7f45f37fe700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f45e006a170 0x7f45e006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.759+0000 7f45f37fe700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f45e006a170 0x7f45e006c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45e80098f0 tx=0x7f45e8008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.760+0000 7f460666db80 1 Processor -- start 2023-08-27T11:29:00.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.760+0000 7f460666db80 1 -- start start 2023-08-27T11:29:00.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f460666db80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c54d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.774 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f460666db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c549b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f460666db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c54a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f460666db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c47b730 con 0x55cc7c55e960 2023-08-27T11:29:00.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f460666db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c479050 con 0x55cc7c5203d0 2023-08-27T11:29:00.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f460666db80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c474940 con 0x55cc7c54cf40 2023-08-27T11:29:00.775 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c549b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c549b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60546/0 (socket says 172.21.15.45:60546) 2023-08-27T11:29:00.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f37fe700 1 -- 172.21.15.45:0/2156254847 learned_addr learned my addr 172.21.15.45:0/2156254847 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:00.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f8a48700 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c54a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f3fff700 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c54d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.776 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f37fe700 1 -- 172.21.15.45:0/2156254847 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 msgr2=0x55cc7c54d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f37fe700 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c54d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f37fe700 1 -- 172.21.15.45:0/2156254847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 msgr2=0x55cc7c54a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f37fe700 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c54a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f37fe700 1 -- 172.21.15.45:0/2156254847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc7c2f45e0 con 0x55cc7c55e960 2023-08-27T11:29:00.777 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.761+0000 7f45f3fff700 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c54d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f45f37fe700 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c549b70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f45e8011d60 tx=0x7f45e8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a147275407bf557 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f45d9ffb700 1 -- 172.21.15.45:0/2156254847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80038a0 con 0x55cc7c55e960 2023-08-27T11:29:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f45d9ffb700 1 -- 172.21.15.45:0/2156254847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e8003a00 con 0x55cc7c55e960 2023-08-27T11:29:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f45d9ffb700 1 -- 172.21.15.45:0/2156254847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8003d70 con 0x55cc7c55e960 2023-08-27T11:29:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f460666db80 1 -- 172.21.15.45:0/2156254847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 msgr2=0x55cc7c549b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f460666db80 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c549b70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f45e8011d60 tx=0x7f45e8010000 comp rx=0 tx=0).stop 2023-08-27T11:29:00.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f460666db80 1 -- 172.21.15.45:0/2156254847 shutdown_connections 2023-08-27T11:29:00.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f460666db80 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c54a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f460666db80 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c54d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f460666db80 1 --2- 172.21.15.45:0/2156254847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c549b70 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.779 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f460666db80 1 -- 172.21.15.45:0/2156254847 >> 172.21.15.45:0/2156254847 conn(0x55cc7c480c30 msgr2=0x55cc7c567d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f460666db80 1 -- 172.21.15.45:0/2156254847 shutdown_connections 2023-08-27T11:29:00.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.762+0000 7f460666db80 1 -- 172.21.15.45:0/2156254847 wait complete. 2023-08-27T11:29:00.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.763+0000 7f460666db80 1 -- start start 2023-08-27T11:29:00.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.763+0000 7f460666db80 1 Processor -- start 2023-08-27T11:29:00.780 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f460666db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c5203d0 0x55cc7c577d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f460666db80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c57a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f460666db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c55e960 0x55cc7c57adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f460666db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c412b60 con 0x55cc7c5203d0 2023-08-27T11:29:00.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f460666db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c412860 con 0x55cc7c55e960 2023-08-27T11:29:00.781 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f460666db80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c2f68d0 con 0x55cc7c54cf40 2023-08-27T11:29:00.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f460666db80 10 client.0 _ll_register_callbacks cb 0x55cc7c410630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:29:00.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f45f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c5203d0 0x55cc7c577d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f45f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c5203d0 0x55cc7c577d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60552/0 (socket says 172.21.15.45:60552) 2023-08-27T11:29:00.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f45f3fff700 1 -- 172.21.15.45:0/1991265265 learned_addr learned my addr 172.21.15.45:0/1991265265 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:00.782 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f45f37fe700 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c57a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f45f8a48700 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c55e960 0x55cc7c57adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f45f3fff700 1 -- 172.21.15.45:0/1991265265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 msgr2=0x55cc7c57a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f45f3fff700 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c57a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f45f3fff700 1 -- 172.21.15.45:0/1991265265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c55e960 msgr2=0x55cc7c57adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.783 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.764+0000 7f45f3fff700 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c55e960 0x55cc7c57adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f45f3fff700 1 -- 172.21.15.45:0/1991265265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e8022070 con 0x55cc7c5203d0 2023-08-27T11:29:00.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f45f3fff700 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c5203d0 0x55cc7c577d40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f45e4057a60 tx=0x7f45e405b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f53a826dd674f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f45d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:29:00.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f45d2ffd700 1 -- 172.21.15.45:0/1991265265 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e406d060 con 0x55cc7c5203d0 2023-08-27T11:29:00.784 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f45d2ffd700 1 -- 172.21.15.45:0/1991265265 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f45e406eb70 con 0x55cc7c5203d0 2023-08-27T11:29:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f460666db80 1 -- 172.21.15.45:0/1991265265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc7c5154f0 con 0x55cc7c5203d0 2023-08-27T11:29:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f45d2ffd700 1 -- 172.21.15.45:0/1991265265 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45e405f620 con 0x55cc7c5203d0 2023-08-27T11:29:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f460666db80 10 client.8193 fetch_fsmap learned FSMap version 49 2023-08-27T11:29:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f460666db80 1 -- 172.21.15.45:0/1991265265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55cc7c48c750 con 0x55cc7c5203d0 2023-08-27T11:29:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f45d2ffd700 1 -- 172.21.15.45:0/1991265265 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cc7c5154f0 con 0x55cc7c5203d0 2023-08-27T11:29:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f45d2ffd700 1 -- 172.21.15.45:0/1991265265 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f45e405bcf0 con 0x55cc7c5203d0 2023-08-27T11:29:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f460666db80 10 client.8193 fetch_fsmap finished waiting for FSMap version 49 2023-08-27T11:29:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f460666db80 10 client.8193 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-27T11:29:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f460666db80 20 client.8193 populate_metadata read hostname 'smithi045' 2023-08-27T11:29:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f460666db80 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] conn(0x55cc7c535390 0x55cc7c535770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f460666db80 4 client.8193 mds_command: new command op to 8019 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-27T11:29:00.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f460666db80 1 -- 172.21.15.45:0/1991265265 --> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc7c46d920 con 0x55cc7c535390 2023-08-27T11:29:00.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.765+0000 7f45f37fe700 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] conn(0x55cc7c535390 0x55cc7c535770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.766+0000 7f45d2ffd700 10 client.8193 ms_handle_connect on v2:172.21.15.45:6834/343785163 2023-08-27T11:29:00.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.766+0000 7f45f37fe700 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] conn(0x55cc7c535390 0x55cc7c535770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf0ead351d0e895 server_cookie=e32567e24cfc6875 in_seq=0 out_seq=0 2023-08-27T11:29:00.787 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.766+0000 7f45d2ffd700 1 -- 172.21.15.45:0/1991265265 <== mds.0 v2:172.21.15.45:6834/343785163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc7c46d920 con 0x55cc7c535390 2023-08-27T11:29:00.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.766+0000 7f45d2ffd700 10 client.8193 handle_command_reply: tid=0 2023-08-27T11:29:00.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.766+0000 7f460666db80 1 client.8193 shutdown 2023-08-27T11:29:00.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.766+0000 7f460666db80 1 -- 172.21.15.45:0/1991265265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c5203d0 msgr2=0x55cc7c577d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.766+0000 7f460666db80 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c5203d0 0x55cc7c577d40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f45e4057a60 tx=0x7f45e405b9d0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.788 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f460666db80 1 -- 172.21.15.45:0/1991265265 shutdown_connections 2023-08-27T11:29:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f460666db80 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] conn(0x55cc7c535390 0x55cc7c535770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f460666db80 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c5203d0 0x55cc7c577d40 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f45d2ffd700 0 client.8193 ms_handle_reset on v2:172.21.15.45:6834/343785163 2023-08-27T11:29:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f460666db80 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c55e960 0x55cc7c57adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f460666db80 1 --2- 172.21.15.45:0/1991265265 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c57a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f460666db80 1 -- 172.21.15.45:0/1991265265 >> 172.21.15.45:0/1991265265 conn(0x55cc7c480c30 msgr2=0x55cc7c546a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f460666db80 1 -- 172.21.15.45:0/1991265265 shutdown_connections 2023-08-27T11:29:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f460666db80 1 -- 172.21.15.45:0/1991265265 wait complete. 2023-08-27T11:29:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.767+0000 7f460666db80 20 client.8193 trim_cache size 0 max 16384 2023-08-27T11:29:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.784+0000 7f460666db80 1 Processor -- start 2023-08-27T11:29:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.784+0000 7f460666db80 1 -- start start 2023-08-27T11:29:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f460666db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c51bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f460666db80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c5209a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f460666db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c5814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f460666db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c566ec0 con 0x55cc7c55e960 2023-08-27T11:29:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f460666db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c5233f0 con 0x55cc7c5203d0 2023-08-27T11:29:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f460666db80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c5471b0 con 0x55cc7c54cf40 2023-08-27T11:29:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f8a48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c5814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f8a48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c5814b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60562/0 (socket says 172.21.15.45:60562) 2023-08-27T11:29:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f8a48700 1 -- 172.21.15.45:0/3966114471 learned_addr learned my addr 172.21.15.45:0/3966114471 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f3fff700 1 --2- 172.21.15.45:0/3966114471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c51bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3966114471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c5209a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f8a48700 1 -- 172.21.15.45:0/3966114471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 msgr2=0x55cc7c5209a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f8a48700 1 --2- 172.21.15.45:0/3966114471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c5209a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f8a48700 1 -- 172.21.15.45:0/3966114471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 msgr2=0x55cc7c51bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f8a48700 1 --2- 172.21.15.45:0/3966114471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c51bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.785+0000 7f45f8a48700 1 -- 172.21.15.45:0/3966114471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc7c2a0d90 con 0x55cc7c55e960 2023-08-27T11:29:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f45f8a48700 1 --2- 172.21.15.45:0/3966114471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c5814b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0021d0 tx=0x7f45ec007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec55b3182b97145 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f45da7fc700 1 -- 172.21.15.45:0/3966114471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec016420 con 0x55cc7c55e960 2023-08-27T11:29:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f45da7fc700 1 -- 172.21.15.45:0/3966114471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec016580 con 0x55cc7c55e960 2023-08-27T11:29:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f45da7fc700 1 -- 172.21.15.45:0/3966114471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0168f0 con 0x55cc7c55e960 2023-08-27T11:29:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f460666db80 1 -- 172.21.15.45:0/3966114471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 msgr2=0x55cc7c5814b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f460666db80 1 --2- 172.21.15.45:0/3966114471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c5814b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0021d0 tx=0x7f45ec007330 comp rx=0 tx=0).stop 2023-08-27T11:29:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f460666db80 1 -- 172.21.15.45:0/3966114471 shutdown_connections 2023-08-27T11:29:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f460666db80 1 --2- 172.21.15.45:0/3966114471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c5814b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f460666db80 1 --2- 172.21.15.45:0/3966114471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c51bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f460666db80 1 --2- 172.21.15.45:0/3966114471 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c5209a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f460666db80 1 -- 172.21.15.45:0/3966114471 >> 172.21.15.45:0/3966114471 conn(0x55cc7c480c30 msgr2=0x55cc7c546aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f460666db80 1 -- 172.21.15.45:0/3966114471 shutdown_connections 2023-08-27T11:29:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.786+0000 7f460666db80 1 -- 172.21.15.45:0/3966114471 wait complete. 2023-08-27T11:29:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.787+0000 7f460666db80 1 -- start start 2023-08-27T11:29:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.787+0000 7f460666db80 1 Processor -- start 2023-08-27T11:29:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f460666db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c545790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f460666db80 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c567b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f460666db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c568090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f460666db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c550d90 con 0x55cc7c55e960 2023-08-27T11:29:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f460666db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c5696f0 con 0x55cc7c5203d0 2023-08-27T11:29:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f460666db80 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc7c5d30f0 con 0x55cc7c54cf40 2023-08-27T11:29:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f460666db80 10 client.0 _ll_register_callbacks cb 0x55cc7c410630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-27T11:29:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f8a48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c568090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f8a48700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c568090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60570/0 (socket says 172.21.15.45:60570) 2023-08-27T11:29:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f8a48700 1 -- 172.21.15.45:0/3554246298 learned_addr learned my addr 172.21.15.45:0/3554246298 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f3fff700 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c545790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c567b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f8a48700 1 -- 172.21.15.45:0/3554246298 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 msgr2=0x55cc7c567b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f8a48700 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c567b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f8a48700 1 -- 172.21.15.45:0/3554246298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 msgr2=0x55cc7c545790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f8a48700 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c545790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f8a48700 1 -- 172.21.15.45:0/3554246298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ec00f040 con 0x55cc7c55e960 2023-08-27T11:29:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f37fe700 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c567b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45f8a48700 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c568090 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0021a0 tx=0x7f45ec001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6058b38b408f7a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2023-08-27T11:29:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.788+0000 7f45d2ffd700 1 -- 172.21.15.45:0/3554246298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec016420 con 0x55cc7c55e960 2023-08-27T11:29:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f45d2ffd700 1 -- 172.21.15.45:0/3554246298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f45ec01f660 con 0x55cc7c55e960 2023-08-27T11:29:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f460666db80 1 -- 172.21.15.45:0/3554246298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45e8003a00 con 0x55cc7c55e960 2023-08-27T11:29:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f45d2ffd700 1 -- 172.21.15.45:0/3554246298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45ec01f820 con 0x55cc7c55e960 2023-08-27T11:29:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f460666db80 10 client.8199 fetch_fsmap learned FSMap version 49 2023-08-27T11:29:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f460666db80 1 -- 172.21.15.45:0/3554246298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55cc7c289c10 con 0x55cc7c55e960 2023-08-27T11:29:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f45d2ffd700 1 -- 172.21.15.45:0/3554246298 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e8003a00 con 0x55cc7c55e960 2023-08-27T11:29:00.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f45d2ffd700 1 -- 172.21.15.45:0/3554246298 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f45ec01f980 con 0x55cc7c55e960 2023-08-27T11:29:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f460666db80 10 client.8199 fetch_fsmap finished waiting for FSMap version 49 2023-08-27T11:29:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f460666db80 10 client.8199 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-27T11:29:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f460666db80 20 client.8199 populate_metadata read hostname 'smithi045' 2023-08-27T11:29:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f460666db80 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] conn(0x55cc7c5ce5d0 0x55cc7c5ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f460666db80 4 client.8199 mds_command: new command op to 8019 tid=0[{"prefix": "status"}] 2023-08-27T11:29:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f460666db80 1 -- 172.21.15.45:0/3554246298 --> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cc7c42e000 con 0x55cc7c5ce5d0 2023-08-27T11:29:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f45f3fff700 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] conn(0x55cc7c5ce5d0 0x55cc7c5ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f45d2ffd700 10 client.8199 ms_handle_connect on v2:172.21.15.45:6834/343785163 2023-08-27T11:29:00.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.789+0000 7f45f3fff700 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] conn(0x55cc7c5ce5d0 0x55cc7c5ce9b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8e6902acea650 server_cookie=2ea31d5ef32ecd99 in_seq=0 out_seq=0 2023-08-27T11:29:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f45d2ffd700 1 -- 172.21.15.45:0/3554246298 <== mds.0 v2:172.21.15.45:6834/343785163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55cc7c42e000 con 0x55cc7c5ce5d0 2023-08-27T11:29:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f45d2ffd700 10 client.8199 handle_command_reply: tid=0 2023-08-27T11:29:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 client.8199 shutdown 2023-08-27T11:29:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 -- 172.21.15.45:0/3554246298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 msgr2=0x55cc7c568090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c568090 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0021a0 tx=0x7f45ec001250 comp rx=0 tx=0).stop 2023-08-27T11:29:00.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 -- 172.21.15.45:0/3554246298 shutdown_connections 2023-08-27T11:29:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.45:6834/343785163,v1:172.21.15.45:6835/343785163] conn(0x55cc7c5ce5d0 0x55cc7c5ce9b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc7c55e960 0x55cc7c568090 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc7c5203d0 0x55cc7c545790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f45d2ffd700 0 client.8199 ms_handle_reset on v2:172.21.15.45:6834/343785163 2023-08-27T11:29:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 --2- 172.21.15.45:0/3554246298 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x55cc7c54cf40 0x55cc7c567b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.811 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 -- 172.21.15.45:0/3554246298 >> 172.21.15.45:0/3554246298 conn(0x55cc7c480c30 msgr2=0x55cc7c569e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 -- 172.21.15.45:0/3554246298 shutdown_connections 2023-08-27T11:29:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 1 -- 172.21.15.45:0/3554246298 wait complete. 2023-08-27T11:29:00.812 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f460666db80 20 client.8199 trim_cache size 0 max 16384 2023-08-27T11:29:00.812 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-08-27T11:29:00.813 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "8ea1ae34-1697-4b79-b2ce-3f741cd3c828", 2023-08-27T11:29:00.813 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2023-08-27T11:29:00.813 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8019, 2023-08-27T11:29:00.813 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2023-08-27T11:29:00.813 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2023-08-27T11:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2023-08-27T11:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.715704218999999, 2023-08-27T11:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 49, 2023-08-27T11:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 116, 2023-08-27T11:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 115, 2023-08-27T11:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 16.506894911 2023-08-27T11:29:00.815 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-08-27T11:29:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f45fa4ab700 1 -- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f45e006a170 msgr2=0x7f45e006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f45fa4ab700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f45e006a170 0x7f45e006c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45e80098f0 tx=0x7f45e8008030 comp rx=0 tx=0).stop 2023-08-27T11:29:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f45fa4ab700 1 -- 172.21.15.45:0/927195359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40af200 msgr2=0x7f45f40c3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.790+0000 7f45fa4ab700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40af200 0x7f45f40c3f60 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f45e400e9b0 tx=0x7f45e400ed70 comp rx=0 tx=0).stop 2023-08-27T11:29:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.791+0000 7f45fa4ab700 1 -- 172.21.15.45:0/927195359 shutdown_connections 2023-08-27T11:29:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.791+0000 7f45fa4ab700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f45e006a170 0x7f45e006c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.791+0000 7f45fa4ab700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40af200 0x7f45f40c3f60 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.791+0000 7f45fa4ab700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f45f40caa00 0x7f45f4154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.791+0000 7f45fa4ab700 1 --2- 172.21.15.45:0/927195359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45f40ca620 0x7f45f40b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.791+0000 7f45fa4ab700 1 -- 172.21.15.45:0/927195359 >> 172.21.15.45:0/927195359 conn(0x7f45f40cc7e0 msgr2=0x7f45f40cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.791+0000 7f45fa4ab700 1 -- 172.21.15.45:0/927195359 shutdown_connections 2023-08-27T11:29:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.791+0000 7f45fa4ab700 1 -- 172.21.15.45:0/927195359 wait complete. 2023-08-27T11:29:00.819 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-27T11:29:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.942+0000 7f88a1d57700 1 Processor -- start 2023-08-27T11:29:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.942+0000 7f88a1d57700 1 -- start start 2023-08-27T11:29:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f88a1d57700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c15a240 0x7f889c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f88a1d57700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f889c0ceba0 0x7f889c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f88a1d57700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889c06d690 con 0x7f889c0cbe70 2023-08-27T11:29:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f88a1d57700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889c0cc440 con 0x7f889c0ceba0 2023-08-27T11:29:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f88a1d57700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c06d040 con 0x7f889c15a240 2023-08-27T11:29:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f889c0ceba0 0x7f889c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f889c0cbe70 0x7f889c0ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47350/0 (socket says 172.21.15.45:47350) 2023-08-27T11:29:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889affd700 1 -- 172.21.15.45:0/1687276124 learned_addr learned my addr 172.21.15.45:0/1687276124 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:29:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889bfff700 1 -- 172.21.15.45:0/1687276124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c15a240 msgr2=0x7f889c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889b7fe700 1 --2- 172.21.15.45:0/1687276124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c15a240 0x7f889c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889bfff700 1 --2- 172.21.15.45:0/1687276124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c15a240 0x7f889c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889bfff700 1 -- 172.21.15.45:0/1687276124 >> v1:172.21.15.45:6789/0 conn(0x7f889c0cbe70 legacy=0x7f889c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889bfff700 1 -- 172.21.15.45:0/1687276124 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c06d320 con 0x7f889c0ceba0 2023-08-27T11:29:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889a7fc700 1 -- 172.21.15.45:0/1687276124 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814055923 0 0) 0x7f889c06d690 con 0x7f889c0cbe70 2023-08-27T11:29:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889b7fe700 1 --2- 172.21.15.45:0/1687276124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c15a240 0x7f889c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:29:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889bfff700 1 --2- 172.21.15.45:0/1687276124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 0x7f889c06db20 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8894009d90 tx=0x7f889400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85673edbe504d254 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889a7fc700 1 -- 172.21.15.45:0/1687276124 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889400e070 con 0x7f889c0ceba0 2023-08-27T11:29:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889a7fc700 1 -- 172.21.15.45:0/1687276124 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889400bbd0 con 0x7f889c0ceba0 2023-08-27T11:29:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.943+0000 7f889a7fc700 1 -- 172.21.15.45:0/1687276124 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894012430 con 0x7f889c0ceba0 2023-08-27T11:29:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.944+0000 7f88a1d57700 1 -- 172.21.15.45:0/1687276124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 msgr2=0x7f889c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.944+0000 7f88a1d57700 1 --2- 172.21.15.45:0/1687276124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 0x7f889c06db20 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8894009d90 tx=0x7f889400b330 comp rx=0 tx=0).stop 2023-08-27T11:29:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.944+0000 7f88a1d57700 1 -- 172.21.15.45:0/1687276124 shutdown_connections 2023-08-27T11:29:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.944+0000 7f88a1d57700 1 --2- 172.21.15.45:0/1687276124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 0x7f889c06db20 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.944+0000 7f88a1d57700 1 --2- 172.21.15.45:0/1687276124 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c15a240 0x7f889c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.944+0000 7f88a1d57700 1 -- 172.21.15.45:0/1687276124 >> 172.21.15.45:0/1687276124 conn(0x7f889c0cfe50 msgr2=0x7f889c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:00.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.944+0000 7f88a1d57700 1 -- 172.21.15.45:0/1687276124 shutdown_connections 2023-08-27T11:29:00.951 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.944+0000 7f88a1d57700 1 -- 172.21.15.45:0/1687276124 wait complete. 2023-08-27T11:29:00.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.945+0000 7f88a1d57700 1 Processor -- start 2023-08-27T11:29:00.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.945+0000 7f88a1d57700 1 -- start start 2023-08-27T11:29:00.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f88a1d57700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c0cbe70 0x7f889c15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f88a1d57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 0x7f889c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.954 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f88a1d57700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f889c15a240 0x7f889c154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f88a1d57700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0b0aa0 con 0x7f889c0ceba0 2023-08-27T11:29:00.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f88a1d57700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c1f2c00 con 0x7f889c15a240 2023-08-27T11:29:00.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f88a1d57700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c1f2ee0 con 0x7f889c0cbe70 2023-08-27T11:29:00.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f889affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 0x7f889c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.955 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f889affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 0x7f889c0b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60582/0 (socket says 172.21.15.45:60582) 2023-08-27T11:29:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f889affd700 1 -- 172.21.15.45:0/2444672297 learned_addr learned my addr 172.21.15.45:0/2444672297 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f889b7fe700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c0cbe70 0x7f889c15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.946+0000 7f889bfff700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f889c15a240 0x7f889c154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f889affd700 1 -- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c0cbe70 msgr2=0x7f889c15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f889affd700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c0cbe70 0x7f889c15baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f889affd700 1 -- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f889c15a240 msgr2=0x7f889c154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f889affd700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f889c15a240 0x7f889c154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f889affd700 1 -- 172.21.15.45:0/2444672297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894009a70 con 0x7f889c0ceba0 2023-08-27T11:29:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f889affd700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 0x7f889c0b2090 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f88900011e0 tx=0x7f889000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f8883fff700 1 -- 172.21.15.45:0/2444672297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890004c80 con 0x7f889c0ceba0 2023-08-27T11:29:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f8883fff700 1 -- 172.21.15.45:0/2444672297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8890004e00 con 0x7f889c0ceba0 2023-08-27T11:29:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c1f31c0 con 0x7f889c0ceba0 2023-08-27T11:29:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f8883fff700 1 -- 172.21.15.45:0/2444672297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890007310 con 0x7f889c0ceba0 2023-08-27T11:29:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.947+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c15c030 con 0x7f889c0ceba0 2023-08-27T11:29:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.948+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c0b0c10 con 0x7f889c0ceba0 2023-08-27T11:29:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.949+0000 7f8883fff700 1 -- 172.21.15.45:0/2444672297 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 46) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8890007470 con 0x7f889c0ceba0 2023-08-27T11:29:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.949+0000 7f8883fff700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f888806a200 0x7f888806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.949+0000 7f8883fff700 1 -- 172.21.15.45:0/2444672297 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f8890088430 con 0x7f889c0ceba0 2023-08-27T11:29:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.949+0000 7f889b7fe700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f888806a200 0x7f888806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.950+0000 7f889b7fe700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f888806a200 0x7f888806c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f888c006fd0 tx=0x7f888c008040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:00.953+0000 7f8883fff700 1 -- 172.21.15.45:0/2444672297 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f889c0b0c10 con 0x7f889c0ceba0 2023-08-27T11:29:01.204 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.202+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f889c0b0c10 con 0x7f889c0ceba0 2023-08-27T11:29:01.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.203+0000 7f8883fff700 1 -- 172.21.15.45:0/2444672297 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f889c0b0c10 con 0x7f889c0ceba0 2023-08-27T11:29:01.205 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-27T11:29:01.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f888806a200 msgr2=0x7f888806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f888806a200 0x7f888806c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f888c006fd0 tx=0x7f888c008040 comp rx=0 tx=0).stop 2023-08-27T11:29:01.207 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 msgr2=0x7f889c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:01.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 0x7f889c0b2090 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f88900011e0 tx=0x7f889000ca30 comp rx=0 tx=0).stop 2023-08-27T11:29:01.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 shutdown_connections 2023-08-27T11:29:01.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f888806a200 0x7f888806c6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f889c15a240 0x7f889c154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.208 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f889c0cbe70 0x7f889c15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 --2- 172.21.15.45:0/2444672297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f889c0ceba0 0x7f889c0b2090 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:01.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 >> 172.21.15.45:0/2444672297 conn(0x7f889c0cfe50 msgr2=0x7f889c156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:01.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 shutdown_connections 2023-08-27T11:29:01.209 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:01.205+0000 7f88a1d57700 1 -- 172.21.15.45:0/2444672297 wait complete. 2023-08-27T11:29:01.219 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-27T11:29:01.219 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-27T11:29:01.220 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2023-08-27T11:29:01.220 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-27T11:29:01.220 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:01.221 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-27T11:29:01.221 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-27T11:29:01.221 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-27T11:29:01.222 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:29:01.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:01.223 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2023-08-27T11:29:01.240 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-27T11:29:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-27T11:29:01.240 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2023-08-27T11:29:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2023-08-27T11:29:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:01.241 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:29:01.241 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:01.242 INFO:teuthology.orchestra.run.smithi045.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-27T11:29:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-27T11:29:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41400sec preferred_lft 41400sec 2023-08-27T11:29:01.242 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-08-27T11:29:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:01.243 INFO:teuthology.orchestra.run.smithi045.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:29:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:01.243 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-27T11:29:01.243 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:01.244 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-27T11:29:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff 2023-08-27T11:29:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-27T11:29:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a41d:a1ff:fed9:3bf2/64 scope link 2023-08-27T11:29:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:01.245 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:29:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether a6:e4:29:97:ce:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-27T11:29:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::a4e4:29ff:fe97:ced1/64 scope link 2023-08-27T11:29:01.245 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:01.245 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-27T11:29:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:84:f3:1a:0a:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-27T11:29:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c84:f3ff:fe1a:a84/64 scope link 2023-08-27T11:29:01.246 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2023-08-27T11:29:01.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:01.247 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-27T11:29:01.307 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-08-27T11:29:01.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:01.309 DEBUG:teuthology.orchestra.run.smithi045:> route 2023-08-27T11:29:01.367 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2023-08-27T11:29:01.367 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-27T11:29:01.368 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-27T11:29:01.368 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-27T11:29:01.368 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-27T11:29:01.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:01.369 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2023-08-27T11:29:01.369 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2023-08-27T11:29:01.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-27T11:29:01.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-27T11:29:01.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-27T11:29:01.371 DEBUG:teuthology.orchestra.run.smithi045:> ') 2023-08-27T11:29:01.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:01.475 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2023-08-27T11:29:01.492 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-27T11:29:01.492 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-27T11:29:01.493 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:29:01.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:01.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2023-08-27T11:29:01.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:29:01.561 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:01.617 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-27T11:29:01.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-27T11:29:01.618 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:01.674 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-27T11:29:01.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:01.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:29:01.761 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:29:01.762 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:29:01.762 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:29:01.770 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:29:01.770 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:29:01.771 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:29:01.771 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:29:01.771 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:29:01.844 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-27T11:29:01.844 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-27T11:29:01.844 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2023-08-27T11:29:01.844 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.135:3300,172.21.15.135:3301" 2023-08-27T11:29:01.845 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-27T11:29:01.845 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790" 2023-08-27T11:29:01.845 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:6789,172.21.15.135:6789,172.21.15.135:6790:/ 2023-08-27T11:29:01.845 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8ea1ae34-1697-4b79-b2ce-3f741cd3c828" will pass to kernel 2023-08-27T11:29:01.846 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-27T11:29:01.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:01.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-27T11:29:01.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:29:01.875 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-27T11:29:01.960 INFO:teuthology.orchestra.run.smithi045.stdout:client7558 2023-08-27T11:29:01.960 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:29:01.961 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:29:01.978 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-27T11:29:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.142+0000 7f3553319700 1 Processor -- start 2023-08-27T11:29:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.142+0000 7f3553319700 1 -- start start 2023-08-27T11:29:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.142+0000 7f3553319700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f354c15d2c0 0x7f354c0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f3553319700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f354c0ce4b0 0x7f354c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f3553319700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c15d6a0 con 0x7f354c0b8c20 2023-08-27T11:29:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f3553319700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c0d2c90 con 0x7f354c15d2c0 2023-08-27T11:29:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f3553319700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0d1bf0 con 0x7f354c0ce4b0 2023-08-27T11:29:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f35510b5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f354c15d2c0 0x7f354c0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f35510b5700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f354c15d2c0 0x7f354c0d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60592/0 (socket says 172.21.15.45:60592) 2023-08-27T11:29:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f35510b5700 1 -- 172.21.15.45:0/2076390040 learned_addr learned my addr 172.21.15.45:0/2076390040 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f35510b5700 1 -- 172.21.15.45:0/2076390040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f354c0ce4b0 msgr2=0x7f354c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f35510b5700 1 --2- 172.21.15.45:0/2076390040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f354c0ce4b0 0x7f354c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f35510b5700 1 -- 172.21.15.45:0/2076390040 >> v1:172.21.15.135:6789/0 conn(0x7f354c0b8c20 legacy=0x7f354c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f35510b5700 1 -- 172.21.15.45:0/2076390040 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0d1ed0 con 0x7f354c15d2c0 2023-08-27T11:29:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f35510b5700 1 --2- 172.21.15.45:0/2076390040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c15d2c0 0x7f354c0d13a0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3548009920 tx=0x7f3548009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfcfecb7785815d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f3543fff700 1 -- 172.21.15.45:0/2076390040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548004230 con 0x7f354c15d2c0 2023-08-27T11:29:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f3543fff700 1 -- 172.21.15.45:0/2076390040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3548004390 con 0x7f354c15d2c0 2023-08-27T11:29:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.143+0000 7f3543fff700 1 -- 172.21.15.45:0/2076390040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548009e70 con 0x7f354c15d2c0 2023-08-27T11:29:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.144+0000 7f3553319700 1 -- 172.21.15.45:0/2076390040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c15d2c0 msgr2=0x7f354c0d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.144+0000 7f3553319700 1 --2- 172.21.15.45:0/2076390040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c15d2c0 0x7f354c0d13a0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3548009920 tx=0x7f3548009c30 comp rx=0 tx=0).stop 2023-08-27T11:29:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.144+0000 7f3553319700 1 -- 172.21.15.45:0/2076390040 shutdown_connections 2023-08-27T11:29:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.144+0000 7f3553319700 1 --2- 172.21.15.45:0/2076390040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f354c0ce4b0 0x7f354c0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.144+0000 7f3553319700 1 --2- 172.21.15.45:0/2076390040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c15d2c0 0x7f354c0d13a0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.144+0000 7f3553319700 1 -- 172.21.15.45:0/2076390040 >> 172.21.15.45:0/2076390040 conn(0x7f354c0d2fd0 msgr2=0x7f354c0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.144+0000 7f3553319700 1 -- 172.21.15.45:0/2076390040 shutdown_connections 2023-08-27T11:29:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.144+0000 7f3553319700 1 -- 172.21.15.45:0/2076390040 wait complete. 2023-08-27T11:29:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.145+0000 7f3553319700 1 Processor -- start 2023-08-27T11:29:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.145+0000 7f3553319700 1 -- start start 2023-08-27T11:29:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f3553319700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c0b8c20 0x7f354c0ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.153 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f3553319700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f354c0ce4b0 0x7f354c14c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f3553319700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f354c15d2c0 0x7f354c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f3553319700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0c9940 con 0x7f354c0b8c20 2023-08-27T11:29:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f3553319700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c14c560 con 0x7f354c0ce4b0 2023-08-27T11:29:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f3553319700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0ccba0 con 0x7f354c15d2c0 2023-08-27T11:29:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35510b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c0b8c20 0x7f354c0ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35510b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c0b8c20 0x7f354c0ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60598/0 (socket says 172.21.15.45:60598) 2023-08-27T11:29:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35510b5700 1 -- 172.21.15.45:0/1857786796 learned_addr learned my addr 172.21.15.45:0/1857786796 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35518b6700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f354c15d2c0 0x7f354c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35508b4700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f354c0ce4b0 0x7f354c14c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35510b5700 1 -- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f354c15d2c0 msgr2=0x7f354c15e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35510b5700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f354c15d2c0 0x7f354c15e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35510b5700 1 -- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f354c0ce4b0 msgr2=0x7f354c14c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35510b5700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f354c0ce4b0 0x7f354c14c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35510b5700 1 -- 172.21.15.45:0/1857786796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548009550 con 0x7f354c0b8c20 2023-08-27T11:29:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35508b4700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f354c0ce4b0 0x7f354c14c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.146+0000 7f35510b5700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c0b8c20 0x7f354c0ca0a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3548009130 tx=0x7f3548004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.147+0000 7f3541ffb700 1 -- 172.21.15.45:0/1857786796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35480036a0 con 0x7f354c0b8c20 2023-08-27T11:29:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.147+0000 7f3541ffb700 1 -- 172.21.15.45:0/1857786796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3548003800 con 0x7f354c0b8c20 2023-08-27T11:29:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.147+0000 7f3541ffb700 1 -- 172.21.15.45:0/1857786796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548003b90 con 0x7f354c0b8c20 2023-08-27T11:29:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.147+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c0cce80 con 0x7f354c0b8c20 2023-08-27T11:29:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.147+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c0c6dc0 con 0x7f354c0b8c20 2023-08-27T11:29:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.148+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c04e620 con 0x7f354c0b8c20 2023-08-27T11:29:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.148+0000 7f3541ffb700 1 -- 172.21.15.45:0/1857786796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 46) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3548003cf0 con 0x7f354c0b8c20 2023-08-27T11:29:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.148+0000 7f3541ffb700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f353006a200 0x7f353006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.149+0000 7f3541ffb700 1 -- 172.21.15.45:0/1857786796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f354808a0c0 con 0x7f354c0b8c20 2023-08-27T11:29:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.149+0000 7f35508b4700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f353006a200 0x7f353006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.149+0000 7f35508b4700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f353006a200 0x7f353006c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f354c15fff0 tx=0x7f353c009040 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.153+0000 7f3541ffb700 1 -- 172.21.15.45:0/1857786796 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3548055590 con 0x7f354c0b8c20 2023-08-27T11:29:02.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.383+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 --> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f354c0d2490 con 0x7f353006a200 2023-08-27T11:29:02.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.385+0000 7f3541ffb700 1 -- 172.21.15.45:0/1857786796 <== mgr.7156 v2:172.21.15.135:6832/3759624346 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f353c001460 con 0x7f353006a200 2023-08-27T11:29:02.387 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2023-08-27T11:29:02.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2023-08-27T11:29:02.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephfs" 2023-08-27T11:29:02.388 INFO:teuthology.orchestra.run.smithi045.stdout: } 2023-08-27T11:29:02.389 INFO:teuthology.orchestra.run.smithi045.stdout:] 2023-08-27T11:29:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f353006a200 msgr2=0x7f353006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f353006a200 0x7f353006c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f354c15fff0 tx=0x7f353c009040 comp rx=0 tx=0).stop 2023-08-27T11:29:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c0b8c20 msgr2=0x7f354c0ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c0b8c20 0x7f354c0ca0a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3548009130 tx=0x7f3548004750 comp rx=0 tx=0).stop 2023-08-27T11:29:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 shutdown_connections 2023-08-27T11:29:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f353006a200 0x7f353006c6c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354c0b8c20 0x7f354c0ca0a0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.392 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f354c15d2c0 0x7f354c15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 --2- 172.21.15.45:0/1857786796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f354c0ce4b0 0x7f354c14c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 >> 172.21.15.45:0/1857786796 conn(0x7f354c0d2fd0 msgr2=0x7f354c15c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 shutdown_connections 2023-08-27T11:29:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.387+0000 7f3553319700 1 -- 172.21.15.45:0/1857786796 wait complete. 2023-08-27T11:29:02.401 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-27T11:29:02.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.525+0000 7f5303613700 1 Processor -- start 2023-08-27T11:29:02.527 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.525+0000 7f5303613700 1 -- start start 2023-08-27T11:29:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f5303613700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52fc0cacd0 0x7f52fc0ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f5303613700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc06b010 con 0x7f52fc15d4f0 2023-08-27T11:29:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f5303613700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc0ad9f0 con 0x7f52fc0aefb0 2023-08-27T11:29:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f5303613700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc15d8d0 con 0x7f52fc0cacd0 2023-08-27T11:29:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f5300bae700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52fc0cacd0 0x7f52fc0ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f53013af700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f52fc15d4f0 0x7f52fc0ca5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47382/0 (socket says 172.21.15.45:47382) 2023-08-27T11:29:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f53013af700 1 -- 172.21.15.45:0/2495267156 learned_addr learned my addr 172.21.15.45:0/2495267156 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:29:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f52f3fff700 1 -- 172.21.15.45:0/2495267156 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008246298 0 0) 0x7f52fc06b010 con 0x7f52fc15d4f0 2023-08-27T11:29:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f52f3fff700 1 -- 172.21.15.45:0/2495267156 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e80035b0 con 0x7f52fc15d4f0 2023-08-27T11:29:02.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f5300bae700 1 -- 172.21.15.45:0/2495267156 >> v1:172.21.15.135:6789/0 conn(0x7f52fc0aefb0 legacy=0x7f52fc0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f5300bae700 1 -- 172.21.15.45:0/2495267156 >> v1:172.21.15.45:6789/0 conn(0x7f52fc15d4f0 legacy=0x7f52fc0ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.526+0000 7f5300bae700 1 -- 172.21.15.45:0/2495267156 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc06ad30 con 0x7f52fc0cacd0 2023-08-27T11:29:02.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f5300bae700 1 --2- 172.21.15.45:0/2495267156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0cacd0 0x7f52fc0ae490 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009840 tx=0x7f52f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4dd90250902094 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f52f3fff700 1 -- 172.21.15.45:0/2495267156 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8011830 con 0x7f52fc0cacd0 2023-08-27T11:29:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f52f3fff700 1 -- 172.21.15.45:0/2495267156 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f8011990 con 0x7f52fc0cacd0 2023-08-27T11:29:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f52f3fff700 1 -- 172.21.15.45:0/2495267156 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8011d20 con 0x7f52fc0cacd0 2023-08-27T11:29:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f5303613700 1 -- 172.21.15.45:0/2495267156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0cacd0 msgr2=0x7f52fc0ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f5303613700 1 --2- 172.21.15.45:0/2495267156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0cacd0 0x7f52fc0ae490 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009840 tx=0x7f52f8009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:02.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f5303613700 1 -- 172.21.15.45:0/2495267156 shutdown_connections 2023-08-27T11:29:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f5303613700 1 --2- 172.21.15.45:0/2495267156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0cacd0 0x7f52fc0ae490 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f5303613700 1 -- 172.21.15.45:0/2495267156 >> 172.21.15.45:0/2495267156 conn(0x7f52fc0cc7e0 msgr2=0x7f52fc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f5303613700 1 -- 172.21.15.45:0/2495267156 shutdown_connections 2023-08-27T11:29:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.527+0000 7f5303613700 1 -- 172.21.15.45:0/2495267156 wait complete. 2023-08-27T11:29:02.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.529+0000 7f5303613700 1 Processor -- start 2023-08-27T11:29:02.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.529+0000 7f5303613700 1 -- start start 2023-08-27T11:29:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.529+0000 7f5303613700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0aefb0 0x7f52fc146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.529+0000 7f5303613700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52fc0cacd0 0x7f52fc0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.529+0000 7f5303613700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52fc15d4f0 0x7f52fc147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.539 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.529+0000 7f5303613700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0c6e30 con 0x7f52fc0aefb0 2023-08-27T11:29:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.529+0000 7f5303613700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1473f0 con 0x7f52fc0cacd0 2023-08-27T11:29:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.529+0000 7f5303613700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0c7040 con 0x7f52fc15d4f0 2023-08-27T11:29:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.529+0000 7f53013af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0aefb0 0x7f52fc146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f53013af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0aefb0 0x7f52fc146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60610/0 (socket says 172.21.15.45:60610) 2023-08-27T11:29:02.540 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f53013af700 1 -- 172.21.15.45:0/2229054542 learned_addr learned my addr 172.21.15.45:0/2229054542 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f53013af700 1 -- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52fc15d4f0 msgr2=0x7f52fc147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f53013af700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52fc15d4f0 0x7f52fc147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f53013af700 1 -- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52fc0cacd0 msgr2=0x7f52fc0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f5300bae700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52fc0cacd0 0x7f52fc0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f53013af700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52fc0cacd0 0x7f52fc0b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f53013af700 1 -- 172.21.15.45:0/2229054542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80094a0 con 0x7f52fc0aefb0 2023-08-27T11:29:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f53013af700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0aefb0 0x7f52fc146eb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c700 tx=0x7f52ec00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f52f1ffb700 1 -- 172.21.15.45:0/2229054542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec005d70 con 0x7f52fc0aefb0 2023-08-27T11:29:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f52f1ffb700 1 -- 172.21.15.45:0/2229054542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec0070d0 con 0x7f52fc0aefb0 2023-08-27T11:29:02.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc0c7320 con 0x7f52fc0aefb0 2023-08-27T11:29:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f52f1ffb700 1 -- 172.21.15.45:0/2229054542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec007460 con 0x7f52fc0aefb0 2023-08-27T11:29:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.530+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc0b45f0 con 0x7f52fc0aefb0 2023-08-27T11:29:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.531+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc0b4180 con 0x7f52fc0aefb0 2023-08-27T11:29:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.532+0000 7f52f1ffb700 1 -- 172.21.15.45:0/2229054542 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 46) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f52ec0075c0 con 0x7f52fc0aefb0 2023-08-27T11:29:02.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.532+0000 7f52f1ffb700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f52e006a200 0x7f52e006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.532+0000 7f52f1ffb700 1 -- 172.21.15.45:0/2229054542 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f52ec087e40 con 0x7f52fc0aefb0 2023-08-27T11:29:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.532+0000 7f5300bae700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f52e006a200 0x7f52e006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.533+0000 7f5300bae700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f52e006a200 0x7f52e006c6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52fc06f180 tx=0x7f52f8008030 comp rx=0 tx=0).ready entity=mgr.7156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:02.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.537+0000 7f52f1ffb700 1 -- 172.21.15.45:0/2229054542 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f52ec053290 con 0x7f52fc0aefb0 2023-08-27T11:29:02.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:02.796+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f52fc0676b0 con 0x7f52fc0aefb0 2023-08-27T11:29:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.140+0000 7f52f1ffb700 1 -- 172.21.15.45:0/2229054542 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 47) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f52ec053660 con 0x7f52fc0aefb0 2023-08-27T11:29:03.143 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.141+0000 7f52f1ffb700 1 -- 172.21.15.45:0/2229054542 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v47) v1 ==== 92+0+0 (secure 0 0 0) 0x7f52ec0756b0 con 0x7f52fc0aefb0 2023-08-27T11:29:03.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f52e006a200 msgr2=0x7f52e006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f52e006a200 0x7f52e006c6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52fc06f180 tx=0x7f52f8008030 comp rx=0 tx=0).stop 2023-08-27T11:29:03.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0aefb0 msgr2=0x7f52fc146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.146 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0aefb0 0x7f52fc146eb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c700 tx=0x7f52ec00cac0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 shutdown_connections 2023-08-27T11:29:03.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f52e006a200 0x7f52e006c6c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52fc0aefb0 0x7f52fc146eb0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f52fc15d4f0 0x7f52fc147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.147 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 --2- 172.21.15.45:0/2229054542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52fc0cacd0 0x7f52fc0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 >> 172.21.15.45:0/2229054542 conn(0x7f52fc0cc7e0 msgr2=0x7f52fc0b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 shutdown_connections 2023-08-27T11:29:03.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.144+0000 7f5303613700 1 -- 172.21.15.45:0/2229054542 wait complete. 2023-08-27T11:29:03.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-27T11:29:03.288 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:03.287+0000 7f395cc26200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:29:03.292 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:03.291+0000 7fcf880b4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:29:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.293+0000 7f7505080700 1 Processor -- start 2023-08-27T11:29:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.293+0000 7f7505080700 1 -- start start 2023-08-27T11:29:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f7505080700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f750015af30 0x7f75000cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f7505080700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f75000cf960 0x7f75000cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f7505080700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75000cb8b0 con 0x7f750015b310 2023-08-27T11:29:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f7505080700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750006f9c0 con 0x7f750015af30 2023-08-27T11:29:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f7505080700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75000cecd0 con 0x7f75000cf960 2023-08-27T11:29:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f750015af30 0x7f75000cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74ff59e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f750015b310 0x7f750015ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47426/0 (socket says 172.21.15.45:47426) 2023-08-27T11:29:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fed9d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f750015af30 0x7f75000cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60646/0 (socket says 172.21.15.45:60646) 2023-08-27T11:29:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74ff59e700 1 -- 172.21.15.45:0/3705895807 learned_addr learned my addr 172.21.15.45:0/3705895807 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:29:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fe59c700 1 --2- 172.21.15.45:0/3705895807 >> v2:172.21.15.135:3300/0 conn(0x7f75000cf960 0x7f75000cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fed9d700 1 -- 172.21.15.45:0/3705895807 >> v2:172.21.15.135:3300/0 conn(0x7f75000cf960 msgr2=0x7f75000cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fed9d700 1 --2- 172.21.15.45:0/3705895807 >> v2:172.21.15.135:3300/0 conn(0x7f75000cf960 0x7f75000cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fed9d700 1 -- 172.21.15.45:0/3705895807 >> v1:172.21.15.45:6789/0 conn(0x7f750015b310 legacy=0x7f750015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fed9d700 1 -- 172.21.15.45:0/3705895807 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75000cbae0 con 0x7f750015af30 2023-08-27T11:29:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fdd9b700 1 -- 172.21.15.45:0/3705895807 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396237129 0 0) 0x7f75000cb8b0 con 0x7f750015b310 2023-08-27T11:29:03.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fed9d700 1 --2- 172.21.15.45:0/3705895807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 0x7f75000cf420 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f75000718f0 tx=0x7f74f0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b69151ff2650f2 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.295+0000 7f74fdd9b700 1 -- 172.21.15.45:0/3705895807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f0011830 con 0x7f750015af30 2023-08-27T11:29:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.296+0000 7f74fdd9b700 1 -- 172.21.15.45:0/3705895807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f0011990 con 0x7f750015af30 2023-08-27T11:29:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.296+0000 7f74fdd9b700 1 -- 172.21.15.45:0/3705895807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f0011d20 con 0x7f750015af30 2023-08-27T11:29:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.296+0000 7f7505080700 1 -- 172.21.15.45:0/3705895807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 msgr2=0x7f75000cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.296+0000 7f7505080700 1 --2- 172.21.15.45:0/3705895807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 0x7f75000cf420 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f75000718f0 tx=0x7f74f0009b20 comp rx=0 tx=0).stop 2023-08-27T11:29:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.296+0000 7f7505080700 1 -- 172.21.15.45:0/3705895807 shutdown_connections 2023-08-27T11:29:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.296+0000 7f7505080700 1 --2- 172.21.15.45:0/3705895807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 0x7f75000cf420 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.296+0000 7f7505080700 1 --2- 172.21.15.45:0/3705895807 >> v2:172.21.15.135:3300/0 conn(0x7f75000cf960 0x7f75000cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.296+0000 7f7505080700 1 -- 172.21.15.45:0/3705895807 >> 172.21.15.45:0/3705895807 conn(0x7f750009a1a0 msgr2=0x7f75000d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.297+0000 7f7505080700 1 -- 172.21.15.45:0/3705895807 shutdown_connections 2023-08-27T11:29:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.297+0000 7f7505080700 1 -- 172.21.15.45:0/3705895807 wait complete. 2023-08-27T11:29:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.298+0000 7f7505080700 1 Processor -- start 2023-08-27T11:29:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.298+0000 7f7505080700 1 -- start start 2023-08-27T11:29:03.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f7505080700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75000cf960 0x7f75000b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f7505080700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 0x7f7500154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f7505080700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f750015b310 0x7f750006b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f7505080700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001f6c90 con 0x7f750015af30 2023-08-27T11:29:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f7505080700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001f60a0 con 0x7f75000cf960 2023-08-27T11:29:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f7505080700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7500154a70 con 0x7f750015b310 2023-08-27T11:29:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fe59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 0x7f7500154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fe59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 0x7f7500154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60654/0 (socket says 172.21.15.45:60654) 2023-08-27T11:29:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fe59c700 1 -- 172.21.15.45:0/2105007331 learned_addr learned my addr 172.21.15.45:0/2105007331 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fed9d700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75000cf960 0x7f75000b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74ff59e700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f750015b310 0x7f750006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fe59c700 1 -- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f750015b310 msgr2=0x7f750006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fe59c700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f750015b310 0x7f750006b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fe59c700 1 -- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75000cf960 msgr2=0x7f75000b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fe59c700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75000cf960 0x7f75000b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fe59c700 1 -- 172.21.15.45:0/2105007331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f00094a0 con 0x7f750015af30 2023-08-27T11:29:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.299+0000 7f74fe59c700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 0x7f7500154530 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f74f400e980 tx=0x7f74f400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.300+0000 7f74e77fe700 1 -- 172.21.15.45:0/2105007331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4009710 con 0x7f750015af30 2023-08-27T11:29:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.300+0000 7f74e77fe700 1 -- 172.21.15.45:0/2105007331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f400c690 con 0x7f750015af30 2023-08-27T11:29:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.300+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75001f62f0 con 0x7f750015af30 2023-08-27T11:29:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.300+0000 7f74e77fe700 1 -- 172.21.15.45:0/2105007331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f400ca20 con 0x7f750015af30 2023-08-27T11:29:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.300+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75001f68b0 con 0x7f750015af30 2023-08-27T11:29:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.301+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75000734a0 con 0x7f750015af30 2023-08-27T11:29:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.301+0000 7f74e77fe700 1 -- 172.21.15.45:0/2105007331 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 47) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f74f400cb80 con 0x7f750015af30 2023-08-27T11:29:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.302+0000 7f74e77fe700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f74ec06a250 0x7f74ec06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.302+0000 7f74e77fe700 1 -- 172.21.15.45:0/2105007331 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f74f40888e0 con 0x7f750015af30 2023-08-27T11:29:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.302+0000 7f74fed9d700 1 -- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f74ec06a250 msgr2=0x7f74ec06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3759624346 2023-08-27T11:29:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.302+0000 7f74fed9d700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f74ec06a250 0x7f74ec06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:29:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.306+0000 7f74e77fe700 1 -- 172.21.15.45:0/2105007331 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f74f4053d20 con 0x7f750015af30 2023-08-27T11:29:03.354 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:03.353+0000 7f395cc26200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:29:03.364 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:03.363+0000 7fcf880b4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:29:03.427 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:03.425+0000 7f395cc26200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:29:03.439 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:03.437+0000 7fcf880b4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:29:03.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.503+0000 7f74fed9d700 1 -- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f74ec06a250 msgr2=0x7f74ec06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3759624346 2023-08-27T11:29:03.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.503+0000 7f74fed9d700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f74ec06a250 0x7f74ec06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:29:03.505 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:03.503+0000 7f395cc26200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:29:03.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.508+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f75000ce4c0 con 0x7f750015af30 2023-08-27T11:29:03.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.512+0000 7f74e77fe700 1 -- 172.21.15.45:0/2105007331 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f74f4057340 con 0x7f750015af30 2023-08-27T11:29:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f74ec06a250 msgr2=0x7f74ec06c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:29:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f74ec06a250 0x7f74ec06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 msgr2=0x7f7500154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 0x7f7500154530 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f74f400e980 tx=0x7f74f400ec90 comp rx=0 tx=0).stop 2023-08-27T11:29:03.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 shutdown_connections 2023-08-27T11:29:03.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f74ec06a250 0x7f74ec06c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f750015b310 0x7f750006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75000cf960 0x7f75000b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 --2- 172.21.15.45:0/2105007331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f750015af30 0x7f7500154530 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 >> 172.21.15.45:0/2105007331 conn(0x7f750009a1a0 msgr2=0x7f75000afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 shutdown_connections 2023-08-27T11:29:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.515+0000 7f7505080700 1 -- 172.21.15.45:0/2105007331 wait complete. 2023-08-27T11:29:03.522 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:03.521+0000 7fcf880b4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:29:03.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-27T11:29:03.646 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:03.645+0000 7f395cc26200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:29:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.657+0000 7f7311d1b700 1 Processor -- start 2023-08-27T11:29:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.657+0000 7f7311d1b700 1 -- start start 2023-08-27T11:29:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f7311d1b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f730c15d340 0x7f730c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f7311d1b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c0cd880 0x7f730c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f7311d1b700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f730c0cdd00 0x7f730c06a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f7311d1b700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730c072010 con 0x7f730c0cdd00 2023-08-27T11:29:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f7311d1b700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730c06b160 con 0x7f730c15d340 2023-08-27T11:29:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f7311d1b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c0af590 con 0x7f730c0cd880 2023-08-27T11:29:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f730bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f730c0cdd00 0x7f730c06a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f730bfff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f730c0cdd00 0x7f730c06a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60686/0 (socket says 172.21.15.45:60686) 2023-08-27T11:29:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f730b7fe700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f730c15d340 0x7f730c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f730affd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c0cd880 0x7f730c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.658+0000 7f730bfff700 1 -- 172.21.15.45:0/980196962 learned_addr learned my addr 172.21.15.45:0/980196962 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730bfff700 1 -- 172.21.15.45:0/980196962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c0cd880 msgr2=0x7f730c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730bfff700 1 --2- 172.21.15.45:0/980196962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c0cd880 0x7f730c06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730bfff700 1 -- 172.21.15.45:0/980196962 >> v2:172.21.15.135:3300/0 conn(0x7f730c15d340 msgr2=0x7f730c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730bfff700 1 --2- 172.21.15.45:0/980196962 >> v2:172.21.15.135:3300/0 conn(0x7f730c15d340 0x7f730c160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730bfff700 1 -- 172.21.15.45:0/980196962 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c0af2b0 con 0x7f730c0cdd00 2023-08-27T11:29:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730b7fe700 1 --2- 172.21.15.45:0/980196962 >> v2:172.21.15.135:3300/0 conn(0x7f730c15d340 0x7f730c160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730bfff700 1 --2- 172.21.15.45:0/980196962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 0x7f730c06a7c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f730c0732d0 tx=0x7f72fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9b55fd0deab272 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730a7fc700 1 -- 172.21.15.45:0/980196962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc011830 con 0x7f730c0cdd00 2023-08-27T11:29:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730a7fc700 1 -- 172.21.15.45:0/980196962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc011990 con 0x7f730c0cdd00 2023-08-27T11:29:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f730a7fc700 1 -- 172.21.15.45:0/980196962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc011d20 con 0x7f730c0cdd00 2023-08-27T11:29:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f7311d1b700 1 -- 172.21.15.45:0/980196962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 msgr2=0x7f730c06a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.659+0000 7f7311d1b700 1 --2- 172.21.15.45:0/980196962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 0x7f730c06a7c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f730c0732d0 tx=0x7f72fc009bd0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.660+0000 7f7311d1b700 1 -- 172.21.15.45:0/980196962 shutdown_connections 2023-08-27T11:29:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.660+0000 7f7311d1b700 1 --2- 172.21.15.45:0/980196962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 0x7f730c06a7c0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.660+0000 7f7311d1b700 1 --2- 172.21.15.45:0/980196962 >> v2:172.21.15.135:3300/0 conn(0x7f730c15d340 0x7f730c160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.660+0000 7f7311d1b700 1 --2- 172.21.15.45:0/980196962 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c0cd880 0x7f730c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.660+0000 7f7311d1b700 1 -- 172.21.15.45:0/980196962 >> 172.21.15.45:0/980196962 conn(0x7f730c0cc7c0 msgr2=0x7f730c0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.660+0000 7f7311d1b700 1 -- 172.21.15.45:0/980196962 shutdown_connections 2023-08-27T11:29:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.660+0000 7f7311d1b700 1 -- 172.21.15.45:0/980196962 wait complete. 2023-08-27T11:29:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.661+0000 7f7311d1b700 1 Processor -- start 2023-08-27T11:29:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.661+0000 7f7311d1b700 1 -- start start 2023-08-27T11:29:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f7311d1b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c0cd880 0x7f730c14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f7311d1b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 0x7f730c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f7311d1b700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c15d340 0x7f730c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f7311d1b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c0b0a60 con 0x7f730c0cdd00 2023-08-27T11:29:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f7311d1b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c14e5d0 con 0x7f730c0cd880 2023-08-27T11:29:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f7311d1b700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c0cc090 con 0x7f730c15d340 2023-08-27T11:29:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 0x7f730c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 0x7f730c0c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60690/0 (socket says 172.21.15.45:60690) 2023-08-27T11:29:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730affd700 1 -- 172.21.15.45:0/1665141856 learned_addr learned my addr 172.21.15.45:0/1665141856 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730b7fe700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c0cd880 0x7f730c14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730bfff700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c15d340 0x7f730c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730affd700 1 -- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c15d340 msgr2=0x7f730c0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730affd700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c15d340 0x7f730c0ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730affd700 1 -- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c0cd880 msgr2=0x7f730c14e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730affd700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c0cd880 0x7f730c14e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730affd700 1 -- 172.21.15.45:0/1665141856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7304009a70 con 0x7f730c0cdd00 2023-08-27T11:29:03.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730b7fe700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c0cd880 0x7f730c14e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:29:03.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f730affd700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 0x7f730c0c9fe0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f72f400e870 tx=0x7f72f400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:03.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.662+0000 7f7303fff700 1 -- 172.21.15.45:0/1665141856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4009590 con 0x7f730c0cdd00 2023-08-27T11:29:03.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.663+0000 7f7303fff700 1 -- 172.21.15.45:0/1665141856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f4013680 con 0x7f730c0cdd00 2023-08-27T11:29:03.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.663+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72fc009550 con 0x7f730c0cdd00 2023-08-27T11:29:03.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.663+0000 7f7303fff700 1 -- 172.21.15.45:0/1665141856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f40139f0 con 0x7f730c0cdd00 2023-08-27T11:29:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.663+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730c0b0fe0 con 0x7f730c0cdd00 2023-08-27T11:29:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.664+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730c0b11d0 con 0x7f730c0cdd00 2023-08-27T11:29:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.664+0000 7f7303fff700 1 -- 172.21.15.45:0/1665141856 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 47) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f72f4013b50 con 0x7f730c0cdd00 2023-08-27T11:29:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.664+0000 7f7303fff700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f72ec06a250 0x7f72ec06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:03.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.664+0000 7f730b7fe700 1 -- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f72ec06a250 msgr2=0x7f72ec06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3759624346 2023-08-27T11:29:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.664+0000 7f7303fff700 1 -- 172.21.15.45:0/1665141856 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f72f40887d0 con 0x7f730c0cdd00 2023-08-27T11:29:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.665+0000 7f730b7fe700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f72ec06a250 0x7f72ec06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:29:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.669+0000 7f7303fff700 1 -- 172.21.15.45:0/1665141856 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f72f4057620 con 0x7f730c0cdd00 2023-08-27T11:29:03.677 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:03.675+0000 7fcf880b4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:29:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.866+0000 7f730b7fe700 1 -- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f72ec06a250 msgr2=0x7f72ec06c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3759624346 2023-08-27T11:29:03.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.866+0000 7f730b7fe700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f72ec06a250 0x7f72ec06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:29:03.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.871+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f730c072690 con 0x7f730c0cdd00 2023-08-27T11:29:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.875+0000 7f7303fff700 1 -- 172.21.15.45:0/1665141856 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7f72f40571b0 con 0x7f730c0cdd00 2023-08-27T11:29:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f72ec06a250 msgr2=0x7f72ec06c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:29:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f72ec06a250 0x7f72ec06c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 msgr2=0x7f730c0c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 0x7f730c0c9fe0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f72f400e870 tx=0x7f72f400eb80 comp rx=0 tx=0).stop 2023-08-27T11:29:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 shutdown_connections 2023-08-27T11:29:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f72ec06a250 0x7f72ec06c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f730c15d340 0x7f730c0ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c0cd880 0x7f730c14e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 --2- 172.21.15.45:0/1665141856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730c0cdd00 0x7f730c0c9fe0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 >> 172.21.15.45:0/1665141856 conn(0x7f730c0cc7c0 msgr2=0x7f730c0af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 shutdown_connections 2023-08-27T11:29:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:03.878+0000 7f7311d1b700 1 -- 172.21.15.45:0/1665141856 wait complete. 2023-08-27T11:29:03.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2023-08-27T11:29:03.971 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:03.970+0000 7f395cc26200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:29:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.027+0000 7f0aa7ef2700 1 Processor -- start 2023-08-27T11:29:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.027+0000 7f0aa7ef2700 1 -- start start 2023-08-27T11:29:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.028+0000 7f0aa7ef2700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0aa0160d00 0x7f0aa006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.028+0000 7f0aa7ef2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa006b160 con 0x7f0aa015d340 2023-08-27T11:29:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.028+0000 7f0aa7ef2700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa00cdf90 con 0x7f0aa006a7c0 2023-08-27T11:29:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.028+0000 7f0aa7ef2700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa0072010 con 0x7f0aa0160d00 2023-08-27T11:29:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.028+0000 7f0aa548d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0aa0160d00 0x7f0aa006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.028+0000 7f0aa5c8e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0aa015d340 0x7f0aa00cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47508/0 (socket says 172.21.15.45:47508) 2023-08-27T11:29:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.028+0000 7f0aa5c8e700 1 -- 172.21.15.45:0/3655767543 learned_addr learned my addr 172.21.15.45:0/3655767543 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:29:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.029+0000 7f0aa548d700 1 -- 172.21.15.45:0/3655767543 >> v1:172.21.15.135:6789/0 conn(0x7f0aa006a7c0 legacy=0x7f0aa00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.029+0000 7f0aa548d700 1 -- 172.21.15.45:0/3655767543 >> v1:172.21.15.45:6789/0 conn(0x7f0aa015d340 legacy=0x7f0aa00cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.029+0000 7f0aa548d700 1 -- 172.21.15.45:0/3655767543 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa00af590 con 0x7f0aa0160d00 2023-08-27T11:29:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.029+0000 7f0aa4c8c700 1 -- 172.21.15.45:0/3655767543 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306338130 0 0) 0x7f0aa006b160 con 0x7f0aa015d340 2023-08-27T11:29:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.029+0000 7f0aa548d700 1 --2- 172.21.15.45:0/3655767543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa0160d00 0x7f0aa006a280 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009840 tx=0x7f0a9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3e2c5b2ead1ca1 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.029+0000 7f0aa4c8c700 1 -- 172.21.15.45:0/3655767543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c011830 con 0x7f0aa0160d00 2023-08-27T11:29:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.029+0000 7f0aa4c8c700 1 -- 172.21.15.45:0/3655767543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c011990 con 0x7f0aa0160d00 2023-08-27T11:29:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.029+0000 7f0aa4c8c700 1 -- 172.21.15.45:0/3655767543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c011d20 con 0x7f0aa0160d00 2023-08-27T11:29:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.030+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/3655767543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa0160d00 msgr2=0x7f0aa006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.030+0000 7f0aa7ef2700 1 --2- 172.21.15.45:0/3655767543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa0160d00 0x7f0aa006a280 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009840 tx=0x7f0a9c009b50 comp rx=0 tx=0).stop 2023-08-27T11:29:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.030+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/3655767543 shutdown_connections 2023-08-27T11:29:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.030+0000 7f0aa7ef2700 1 --2- 172.21.15.45:0/3655767543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa0160d00 0x7f0aa006a280 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.030+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/3655767543 >> 172.21.15.45:0/3655767543 conn(0x7f0aa00cc7c0 msgr2=0x7f0aa00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:04.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.030+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/3655767543 shutdown_connections 2023-08-27T11:29:04.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.030+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/3655767543 wait complete. 2023-08-27T11:29:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.031+0000 7f0aa7ef2700 1 Processor -- start 2023-08-27T11:29:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.031+0000 7f0aa7ef2700 1 -- start start 2023-08-27T11:29:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa7ef2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa006a7c0 0x7f0aa01f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa7ef2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0aa015d340 0x7f0aa00c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa7ef2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0aa0160d00 0x7f0aa00c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa7ef2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa014ad80 con 0x7f0aa006a7c0 2023-08-27T11:29:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa7ef2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa00c9a20 con 0x7f0aa0160d00 2023-08-27T11:29:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa7ef2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa014af90 con 0x7f0aa015d340 2023-08-27T11:29:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa5c8e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa006a7c0 0x7f0aa01f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa5c8e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa006a7c0 0x7f0aa01f3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60738/0 (socket says 172.21.15.45:60738) 2023-08-27T11:29:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa5c8e700 1 -- 172.21.15.45:0/1904821109 learned_addr learned my addr 172.21.15.45:0/1904821109 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa648f700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0aa0160d00 0x7f0aa00c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa548d700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0aa015d340 0x7f0aa00c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:04.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa5c8e700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0aa015d340 msgr2=0x7f0aa00c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa5c8e700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f0aa015d340 0x7f0aa00c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa5c8e700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0aa0160d00 msgr2=0x7f0aa00c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:04.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa5c8e700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0aa0160d00 0x7f0aa00c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:04.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa5c8e700 1 -- 172.21.15.45:0/1904821109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0094a0 con 0x7f0aa006a7c0 2023-08-27T11:29:04.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.032+0000 7f0aa5c8e700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa006a7c0 0x7f0aa01f3440 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000c520 tx=0x7f0a9000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:04.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.033+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90004c80 con 0x7f0aa006a7c0 2023-08-27T11:29:04.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.033+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a90004de0 con 0x7f0aa006a7c0 2023-08-27T11:29:04.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.033+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa014b270 con 0x7f0aa006a7c0 2023-08-27T11:29:04.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.033+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90007310 con 0x7f0aa006a7c0 2023-08-27T11:29:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.033+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa014ba70 con 0x7f0aa006a7c0 2023-08-27T11:29:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.034+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa0072690 con 0x7f0aa006a7c0 2023-08-27T11:29:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.034+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 47) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f0a90007470 con 0x7f0aa006a7c0 2023-08-27T11:29:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.034+0000 7f0a967fc700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 0x7f0a8406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.035+0000 7f0aa548d700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 msgr2=0x7f0a8406c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3759624346 2023-08-27T11:29:04.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.035+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f0a90088440 con 0x7f0aa006a7c0 2023-08-27T11:29:04.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.035+0000 7f0aa548d700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 0x7f0a8406c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:29:04.044 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:04.035+0000 7fcf880b4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:29:04.044 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.043+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0a90053880 con 0x7f0aa006a7c0 2023-08-27T11:29:04.087 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:04.086+0000 7f395cc26200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:29:04.161 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:04.160+0000 7f395cc26200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:29:04.162 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:04.160+0000 7fcf880b4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:29:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.236+0000 7f0aa548d700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 msgr2=0x7f0a8406c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3759624346 2023-08-27T11:29:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.236+0000 7f0aa548d700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 0x7f0a8406c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:29:04.240 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:04.238+0000 7fcf880b4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:29:04.241 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:04.240+0000 7f395cc26200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:29:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.269+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 --> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0aa0071950 con 0x7f0a8406a250 2023-08-27T11:29:04.325 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:04.324+0000 7fcf880b4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:29:04.611 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:04.610+0000 7f395cc26200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:29:04.638 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.637+0000 7f0aa548d700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 msgr2=0x7f0a8406c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3759624346 2023-08-27T11:29:04.639 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:04.637+0000 7f0aa548d700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 0x7f0a8406c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-27T11:29:04.715 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:04.714+0000 7fcf880b4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:29:05.181 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:05.180+0000 7f395cc26200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:29:05.251 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:05.250+0000 7f395cc26200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:29:05.327 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:05.325+0000 7fcf880b4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:29:05.400 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:05.399+0000 7fcf880b4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:29:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:05.438+0000 7f0aa548d700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 msgr2=0x7f0a8406c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3759624346 2023-08-27T11:29:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:05.438+0000 7f0aa548d700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 0x7f0a8406c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-27T11:29:05.886 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:05.885+0000 7f395cc26200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:29:05.964 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:05.963+0000 7f395cc26200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:29:06.066 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:06.064+0000 7fcf880b4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:29:06.109 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:06.108+0000 7f395cc26200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:29:06.150 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:06.149+0000 7fcf880b4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:29:06.302 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:06.301+0000 7fcf880b4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:29:06.338 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:06.337+0000 7f395cc26200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:29:06.411 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:06.410+0000 7f395cc26200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:29:06.541 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:06.539+0000 7fcf880b4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:29:06.618 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:06.617+0000 7fcf880b4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:29:06.691 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:06.690+0000 7f395cc26200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:29:06.897 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:06.895+0000 7fcf880b4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:29:06.912 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:06.911+0000 7f395cc26200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:29:07.041 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:07.040+0000 7f0aa548d700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 msgr2=0x7f0a8406c710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/3759624346 2023-08-27T11:29:07.042 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:07.040+0000 7f0aa548d700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 0x7f0a8406c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-27T11:29:07.062 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:07.060+0000 7f395cc26200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:29:07.130 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:07.128+0000 7fcf880b4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:29:07.140 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:29:07.143 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:29:07.177 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:07.176+0000 7f395cc26200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:29:07.291 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:07.289+0000 7fcf880b4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:29:07.333 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:07.332+0000 7f395cc26200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:29:07.410 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:07.409+0000 7fcf880b4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:29:07.573 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:07.571+0000 7fcf880b4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:29:07.595 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:07.594+0000 7f395cc26200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:29:07.755 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:07.754+0000 7f395cc26200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:29:07.847 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:07.845+0000 7fcf880b4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:29:08.016 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:08.015+0000 7fcf880b4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:29:08.281 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.280+0000 7f395cc26200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:29:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:08.302+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 48) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f0a90086660 con 0x7f0aa006a7c0 2023-08-27T11:29:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:08.302+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 msgr2=0x7f0a8406c710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:29:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:08.302+0000 7f0a967fc700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/3759624346,v1:172.21.15.135:6833/3759624346] conn(0x7f0a8406a250 0x7f0a8406c710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:08.406 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38d2c8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.406 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38d2c8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.407 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38d2c8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.407 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38d2c8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.407 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38d2c8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.407 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38cfc84700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.407 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38cfc84700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.408 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38cfc84700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.408 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38cfc84700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.408 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:29:08.405+0000 7f38cfc84700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:29:08.588 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:29:08.587+0000 7fcf880b4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:29:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.307+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 49) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f0a9008a540 con 0x7f0aa006a7c0 2023-08-27T11:29:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.307+0000 7f0a967fc700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f0a84072d90 0x7f0a84075180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.307+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 --> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a84003920 con 0x7f0a84072d90 2023-08-27T11:29:09.311 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.310+0000 7f0aa548d700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f0a84072d90 0x7f0a84075180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.314 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.312+0000 7f0aa548d700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f0a84072d90 0x7f0a84075180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0090b0 tx=0x7f0a9c008030 comp rx=0 tx=0).ready entity=mgr.6866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.374+0000 7f0a967fc700 1 -- 172.21.15.45:0/1904821109 <== mgr.6866 v2:172.21.15.135:6832/1129514681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f0a9c011b00 con 0x7f0a84072d90 2023-08-27T11:29:09.376 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path /bad-path 2023-08-27T11:29:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.376+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f0a84072d90 msgr2=0x7f0a84075180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.376+0000 7f0aa7ef2700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f0a84072d90 0x7f0a84075180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0090b0 tx=0x7f0a9c008030 comp rx=0 tx=0).stop 2023-08-27T11:29:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.376+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa006a7c0 msgr2=0x7f0aa01f3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.376+0000 7f0aa7ef2700 1 --2- 172.21.15.45:0/1904821109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0aa006a7c0 0x7f0aa01f3440 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000c520 tx=0x7f0a9000c830 comp rx=0 tx=0).stop 2023-08-27T11:29:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.376+0000 7f0aa648f700 1 -- 172.21.15.45:0/1904821109 reap_dead start 2023-08-27T11:29:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.376+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 shutdown_connections 2023-08-27T11:29:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.376+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 >> 172.21.15.45:0/1904821109 conn(0x7f0aa00cc7c0 msgr2=0x7f0aa00cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:09.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.376+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 shutdown_connections 2023-08-27T11:29:09.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.376+0000 7f0aa7ef2700 1 -- 172.21.15.45:0/1904821109 wait complete. 2023-08-27T11:29:09.392 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-08-27T11:29:09.392 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-08-27T11:29:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.562+0000 7f33b07a4700 1 Processor -- start 2023-08-27T11:29:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.562+0000 7f33b07a4700 1 -- start start 2023-08-27T11:29:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33b07a4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33a80aefb0 0x7f33a80ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33b07a4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a806b010 con 0x7f33a815d4f0 2023-08-27T11:29:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33b07a4700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a80ad9f0 con 0x7f33a80cacd0 2023-08-27T11:29:09.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33b07a4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a815d8d0 con 0x7f33a80aefb0 2023-08-27T11:29:09.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ae540700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f33a815d4f0 0x7f33a80ca5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47582/0 (socket says 172.21.15.45:47582) 2023-08-27T11:29:09.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ae540700 1 -- 172.21.15.45:0/3272975488 learned_addr learned my addr 172.21.15.45:0/3272975488 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:29:09.568 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553276746 0 0) 0x7f33a806b010 con 0x7f33a815d4f0 2023-08-27T11:29:09.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33940035b0 con 0x7f33a815d4f0 2023-08-27T11:29:09.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33aed41700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33a80aefb0 0x7f33a80ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3084805899 0 0) 0x7f33940035b0 con 0x7f33a815d4f0 2023-08-27T11:29:09.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33a80aefb0 msgr2=0x7f33a80ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:09.569 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33a80aefb0 0x7f33a80ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 >> v1:172.21.15.135:6789/0 conn(0x7f33a80cacd0 legacy=0x7f33a80cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a806ad30 con 0x7f33a815d4f0 2023-08-27T11:29:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f33a4001670 con 0x7f33a815d4f0 2023-08-27T11:29:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 >> v1:172.21.15.45:6789/0 conn(0x7f33a815d4f0 legacy=0x7f33a80ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3394003a30 0x7f3394003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.570 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3394004850 0x7f3394004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33a80aefb0 0x7f339400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a815d8d0 con 0x7f3394004850 2023-08-27T11:29:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33940055c0 con 0x7f3394003a30 2023-08-27T11:29:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.563+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339400bf50 con 0x7f33a80aefb0 2023-08-27T11:29:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33add3f700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3394004850 0x7f3394004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33ae540700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3394003a30 0x7f3394003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33aed41700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33a80aefb0 0x7f339400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33add3f700 1 -- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33a80aefb0 msgr2=0x7f339400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33add3f700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f33a80aefb0 0x7f339400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33add3f700 1 -- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3394003a30 msgr2=0x7f3394003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:29:09.573 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33add3f700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3394003a30 0x7f3394003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33add3f700 1 -- 172.21.15.45:0/3272975488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33940035b0 con 0x7f3394004850 2023-08-27T11:29:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33ae540700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3394003a30 0x7f3394003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33add3f700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3394004850 0x7f3394004cd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3398001800 tx=0x7f3394005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7204d965748a81 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980012f0 con 0x7f3394004850 2023-08-27T11:29:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980052f0 con 0x7f3394004850 2023-08-27T11:29:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.564+0000 7f33ad53e700 1 -- 172.21.15.45:0/3272975488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398006860 con 0x7f3394004850 2023-08-27T11:29:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.565+0000 7f33b07a4700 1 -- 172.21.15.45:0/3272975488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3394004850 msgr2=0x7f3394004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.565+0000 7f33b07a4700 1 --2- 172.21.15.45:0/3272975488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3394004850 0x7f3394004cd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3398001800 tx=0x7f3394005880 comp rx=0 tx=0).stop 2023-08-27T11:29:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.565+0000 7f33aed41700 1 -- 172.21.15.45:0/3272975488 reap_dead start 2023-08-27T11:29:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.565+0000 7f33b07a4700 1 -- 172.21.15.45:0/3272975488 shutdown_connections 2023-08-27T11:29:09.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.565+0000 7f33b07a4700 1 -- 172.21.15.45:0/3272975488 >> 172.21.15.45:0/3272975488 conn(0x7f33a80cc7e0 msgr2=0x7f33a80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:09.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.565+0000 7f33b07a4700 1 -- 172.21.15.45:0/3272975488 shutdown_connections 2023-08-27T11:29:09.578 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.565+0000 7f33b07a4700 1 -- 172.21.15.45:0/3272975488 wait complete. 2023-08-27T11:29:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.566+0000 7f33b07a4700 1 Processor -- start 2023-08-27T11:29:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.566+0000 7f33b07a4700 1 -- start start 2023-08-27T11:29:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33b07a4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3394004850 0x7f33a8071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33b07a4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33a8150b50 0x7f33a8150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33b07a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a815f060 0x7f33a815f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33b07a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a815fe20 con 0x7f33a815f060 2023-08-27T11:29:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33b07a4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8160100 con 0x7f33a8150b50 2023-08-27T11:29:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33b07a4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a81603e0 con 0x7f3394004850 2023-08-27T11:29:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33aed41700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a815f060 0x7f33a815f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33aed41700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a815f060 0x7f33a815f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60820/0 (socket says 172.21.15.45:60820) 2023-08-27T11:29:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33aed41700 1 -- 172.21.15.45:0/4292963271 learned_addr learned my addr 172.21.15.45:0/4292963271 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:29:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33add3f700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33a8150b50 0x7f33a8150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33ae540700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3394004850 0x7f33a8071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33aed41700 1 -- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3394004850 msgr2=0x7f33a8071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33aed41700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3394004850 0x7f33a8071080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33aed41700 1 -- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33a8150b50 msgr2=0x7f33a8150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33aed41700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33a8150b50 0x7f33a8150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33aed41700 1 -- 172.21.15.45:0/4292963271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a81606c0 con 0x7f33a815f060 2023-08-27T11:29:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33ae540700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3394004850 0x7f33a8071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:29:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f33aed41700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a815f060 0x7f33a815f4f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f33a8069790 tx=0x7f33a00048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.567+0000 7f339effd700 1 -- 172.21.15.45:0/4292963271 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a00079f0 con 0x7f33a815f060 2023-08-27T11:29:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.568+0000 7f339effd700 1 -- 172.21.15.45:0/4292963271 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a000f050 con 0x7f33a815f060 2023-08-27T11:29:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.568+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a81609a0 con 0x7f33a815f060 2023-08-27T11:29:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.568+0000 7f339effd700 1 -- 172.21.15.45:0/4292963271 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0007db0 con 0x7f33a815f060 2023-08-27T11:29:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.568+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a815fa30 con 0x7f33a815f060 2023-08-27T11:29:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.568+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a8160b90 con 0x7f33a815f060 2023-08-27T11:29:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.569+0000 7f339effd700 1 -- 172.21.15.45:0/4292963271 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 49) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f33a001e460 con 0x7f33a815f060 2023-08-27T11:29:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.569+0000 7f339effd700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f338c069f80 0x7f338c06c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:29:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.569+0000 7f339effd700 1 -- 172.21.15.45:0/4292963271 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f33a0088710 con 0x7f33a815f060 2023-08-27T11:29:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.570+0000 7f33ae540700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f338c069f80 0x7f338c06c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:29:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.570+0000 7f33ae540700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f338c069f80 0x7f338c06c440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000fb0 tx=0x7f33a4009040 comp rx=0 tx=0).ready entity=mgr.6866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:29:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.574+0000 7f339effd700 1 -- 172.21.15.45:0/4292963271 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f33a0053e50 con 0x7f33a815f060 2023-08-27T11:29:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.798+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 --> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33a80adf70 con 0x7f338c069f80 2023-08-27T11:29:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.808+0000 7f339effd700 1 -- 172.21.15.45:0/4292963271 <== mgr.6866 v2:172.21.15.135:6832/1129514681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7f33a4004870 con 0x7f338c069f80 2023-08-27T11:29:09.810 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:29:09.811 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:29:09", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-08-27T11:29:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f338c069f80 msgr2=0x7f338c06c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f338c069f80 0x7f338c06c440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000fb0 tx=0x7f33a4009040 comp rx=0 tx=0).stop 2023-08-27T11:29:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a815f060 msgr2=0x7f33a815f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:29:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a815f060 0x7f33a815f4f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f33a8069790 tx=0x7f33a00048c0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 shutdown_connections 2023-08-27T11:29:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f338c069f80 0x7f338c06c440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a815f060 0x7f33a815f4f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3394004850 0x7f33a8071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 --2- 172.21.15.45:0/4292963271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33a8150b50 0x7f33a8150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:29:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 >> 172.21.15.45:0/4292963271 conn(0x7f33a80cc7e0 msgr2=0x7f33a80b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:29:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 shutdown_connections 2023-08-27T11:29:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:29:09.811+0000 7f33b07a4700 1 -- 172.21.15.45:0/4292963271 wait complete. 2023-08-27T11:29:09.825 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:29:09", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-08-27T11:29:37.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:29:37.290 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:30:00.387 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:00.385+0000 7f38c2469700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:00.387 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:00.385+0000 7f38c2469700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:00.387 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:00.385+0000 7f38c2469700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:00.388 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:00.385+0000 7f38c2469700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:00.388 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:00.385+0000 7f38c2469700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:07.377 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:30:07.381 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-27T11:30:09.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-08-27T11:30:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.970+0000 7f59a562e700 1 Processor -- start 2023-08-27T11:30:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.970+0000 7f59a562e700 1 -- start start 2023-08-27T11:30:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.971+0000 7f59a562e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59a00b8c20 0x7f59a00cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.971+0000 7f59a562e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a015d6a0 con 0x7f59a00ce2e0 2023-08-27T11:30:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.971+0000 7f59a562e700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a00d2c90 con 0x7f59a015d2c0 2023-08-27T11:30:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.971+0000 7f59a562e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00ce770 con 0x7f59a00b8c20 2023-08-27T11:30:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.971+0000 7f599f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f59a00ce2e0 0x7f59a00b6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47606/0 (socket says 172.21.15.45:47606) 2023-08-27T11:30:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.971+0000 7f599f7fe700 1 -- 172.21.15.45:0/4016883376 learned_addr learned my addr 172.21.15.45:0/4016883376 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.971+0000 7f59967fc700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59a00b8c20 0x7f59a00cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.971+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235892438 0 0) 0x7f5998001490 con 0x7f59a00ce2e0 2023-08-27T11:30:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.971+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f598c0035b0 con 0x7f59a00ce2e0 2023-08-27T11:30:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721764052 0 0) 0x7f59a00d2c90 con 0x7f59a015d2c0 2023-08-27T11:30:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5998001490 con 0x7f59a015d2c0 2023-08-27T11:30:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4101434747 0 0) 0x7f598c0035b0 con 0x7f59a00ce2e0 2023-08-27T11:30:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59a00b8c20 msgr2=0x7f59a00cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59a00b8c20 0x7f59a00cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 >> v1:172.21.15.135:6789/0 conn(0x7f59a015d2c0 legacy=0x7f59a00d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00d1bf0 con 0x7f59a00ce2e0 2023-08-27T11:30:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f5998001b80 con 0x7f59a00ce2e0 2023-08-27T11:30:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 >> v1:172.21.15.45:6789/0 conn(0x7f59a00ce2e0 legacy=0x7f59a00b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59a00b8c20 0x7f598c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f598c004120 0x7f598c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f598c004d90 0x7f598c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00ce770 con 0x7f598c004d90 2023-08-27T11:30:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c00baf0 con 0x7f598c004120 2023-08-27T11:30:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c00bcb0 con 0x7f59a00b8c20 2023-08-27T11:30:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599f7fe700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f598c004d90 0x7f598c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599effd700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59a00b8c20 0x7f598c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f59967fc700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f598c004120 0x7f598c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599f7fe700 1 -- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59a00b8c20 msgr2=0x7f598c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:09.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599f7fe700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f59a00b8c20 0x7f598c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.982 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599f7fe700 1 -- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f598c004120 msgr2=0x7f598c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:09.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599f7fe700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f598c004120 0x7f598c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599f7fe700 1 -- 172.21.15.45:0/4016883376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0035b0 con 0x7f598c004d90 2023-08-27T11:30:09.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599f7fe700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f598c004d90 0x7f598c005210 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f599800a790 tx=0x7f598c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279cec4f404967c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:09.983 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998012600 con 0x7f598c004d90 2023-08-27T11:30:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998012760 con 0x7f598c004d90 2023-08-27T11:30:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.972+0000 7f599e7fc700 1 -- 172.21.15.45:0/4016883376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998012b10 con 0x7f598c004d90 2023-08-27T11:30:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.973+0000 7f59a562e700 1 -- 172.21.15.45:0/4016883376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f598c004d90 msgr2=0x7f598c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.973+0000 7f59a562e700 1 --2- 172.21.15.45:0/4016883376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f598c004d90 0x7f598c005210 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f599800a790 tx=0x7f598c005880 comp rx=0 tx=0).stop 2023-08-27T11:30:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.973+0000 7f599f7fe700 1 -- 172.21.15.45:0/4016883376 reap_dead start 2023-08-27T11:30:09.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.973+0000 7f59a562e700 1 -- 172.21.15.45:0/4016883376 shutdown_connections 2023-08-27T11:30:09.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.973+0000 7f59a562e700 1 -- 172.21.15.45:0/4016883376 >> 172.21.15.45:0/4016883376 conn(0x7f59a00d2fd0 msgr2=0x7f59a00d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:09.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.973+0000 7f59a562e700 1 -- 172.21.15.45:0/4016883376 shutdown_connections 2023-08-27T11:30:09.985 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.973+0000 7f59a562e700 1 -- 172.21.15.45:0/4016883376 wait complete. 2023-08-27T11:30:09.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.974+0000 7f59a562e700 1 Processor -- start 2023-08-27T11:30:09.986 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59a562e700 1 -- start start 2023-08-27T11:30:09.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59a562e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f598c004d90 0x7f59a0159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59a562e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59a00cbbc0 0x7f59a00cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59a562e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59a00c8cf0 0x7f59a00c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.989 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59a562e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01f5d00 con 0x7f59a00cbbc0 2023-08-27T11:30:09.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59a562e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00cc580 con 0x7f59a00c8cf0 2023-08-27T11:30:09.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59a562e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a01f6810 con 0x7f598c004d90 2023-08-27T11:30:09.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59967fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59a00cbbc0 0x7f59a00cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.990 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59967fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59a00cbbc0 0x7f59a00cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60840/0 (socket says 172.21.15.45:60840) 2023-08-27T11:30:09.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.975+0000 7f59967fc700 1 -- 172.21.15.45:0/2642895791 learned_addr learned my addr 172.21.15.45:0/2642895791 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:09.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f599f7fe700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59a00c8cf0 0x7f59a00c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f599effd700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f598c004d90 0x7f59a0159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.991 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f59967fc700 1 -- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f598c004d90 msgr2=0x7f59a0159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f59967fc700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f598c004d90 0x7f59a0159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f59967fc700 1 -- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59a00c8cf0 msgr2=0x7f59a00c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:09.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f59967fc700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59a00c8cf0 0x7f59a00c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:09.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f59967fc700 1 -- 172.21.15.45:0/2642895791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a01f5ee0 con 0x7f59a00cbbc0 2023-08-27T11:30:09.992 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f599f7fe700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59a00c8cf0 0x7f59a00c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:09.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f599effd700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f598c004d90 0x7f59a0159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:09.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f59967fc700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59a00cbbc0 0x7f59a00cc040 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5988011ad0 tx=0x7f5988011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:09.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f5997fff700 1 -- 172.21.15.45:0/2642895791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598800fa00 con 0x7f59a00cbbc0 2023-08-27T11:30:09.993 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f5997fff700 1 -- 172.21.15.45:0/2642895791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598800fb60 con 0x7f59a00cbbc0 2023-08-27T11:30:09.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f5997fff700 1 -- 172.21.15.45:0/2642895791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988014440 con 0x7f59a00cbbc0 2023-08-27T11:30:09.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a01f61c0 con 0x7f59a00cbbc0 2023-08-27T11:30:09.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.976+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a00c96e0 con 0x7f59a00cbbc0 2023-08-27T11:30:09.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.977+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a004e620 con 0x7f59a00cbbc0 2023-08-27T11:30:09.994 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.977+0000 7f5997fff700 1 -- 172.21.15.45:0/2642895791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 52) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f59880145a0 con 0x7f59a00cbbc0 2023-08-27T11:30:09.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.978+0000 7f5997fff700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f598006a370 0x7f598006c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:09.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.978+0000 7f5997fff700 1 -- 172.21.15.45:0/2642895791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5988089490 con 0x7f59a00cbbc0 2023-08-27T11:30:09.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.978+0000 7f599effd700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f598006a370 0x7f598006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:09.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.979+0000 7f599effd700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f598006a370 0x7f598006c830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5990007fd0 tx=0x7f5990009040 comp rx=0 tx=0).ready entity=mgr.6866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:09.995 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:09.982+0000 7f5997fff700 1 -- 172.21.15.45:0/2642895791 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59880545f0 con 0x7f59a00cbbc0 2023-08-27T11:30:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.216+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 --> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59a00d21b0 con 0x7f598006a370 2023-08-27T11:30:10.225 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.223+0000 7f5997fff700 1 -- 172.21.15.45:0/2642895791 <== mgr.6866 v2:172.21.15.135:6832/1129514681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f5990005320 con 0x7f598006a370 2023-08-27T11:30:10.225 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:30:10.226 INFO:teuthology.orchestra.run.smithi045.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:29:09", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-08-27T11:30:10.227 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f598006a370 msgr2=0x7f598006c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f598006a370 0x7f598006c830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5990007fd0 tx=0x7f5990009040 comp rx=0 tx=0).stop 2023-08-27T11:30:10.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59a00cbbc0 msgr2=0x7f59a00cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59a00cbbc0 0x7f59a00cc040 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5988011ad0 tx=0x7f5988011e90 comp rx=0 tx=0).stop 2023-08-27T11:30:10.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 shutdown_connections 2023-08-27T11:30:10.228 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59a00c8cf0 0x7f59a00c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f598c004d90 0x7f59a0159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f598006a370 0x7f598006c830 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 --2- 172.21.15.45:0/2642895791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59a00cbbc0 0x7f59a00cc040 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.225+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 >> 172.21.15.45:0/2642895791 conn(0x7f59a00d2fd0 msgr2=0x7f59a00cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:10.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.226+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 shutdown_connections 2023-08-27T11:30:10.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.226+0000 7f59a562e700 1 -- 172.21.15.45:0/2642895791 wait complete. 2023-08-27T11:30:10.239 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-08-27T00:00:00", "created": "2023-08-27T11:29:09", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-08-27T11:30:10.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2023-08-27T11:30:10.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.367+0000 7f8f40f7a700 1 Processor -- start 2023-08-27T11:30:10.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.367+0000 7f8f40f7a700 1 -- start start 2023-08-27T11:30:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.368+0000 7f8f40f7a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8f3c15d600 0x7f8f3c0cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.368+0000 7f8f40f7a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0aed20 0x7f8f3c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.368+0000 7f8f40f7a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c0af270 con 0x7f8f3c0ca3b0 2023-08-27T11:30:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.368+0000 7f8f40f7a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c0af550 con 0x7f8f3c15d600 2023-08-27T11:30:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.368+0000 7f8f40f7a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c06ac90 con 0x7f8f3c0aed20 2023-08-27T11:30:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3ad9d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0aed20 0x7f8f3c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f39d9b700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8f3c15d600 0x7f8f3c0cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3a59c700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f8f3c0ca3b0 0x7f8f3c0ca790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:43286/0 (socket says 172.21.15.45:43286) 2023-08-27T11:30:10.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3a59c700 1 -- 172.21.15.45:0/4040136190 learned_addr learned my addr 172.21.15.45:0/4040136190 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:10.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3959a700 1 -- 172.21.15.45:0/4040136190 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683910095 0 0) 0x7f8f3c0af270 con 0x7f8f3c0ca3b0 2023-08-27T11:30:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3959a700 1 -- 172.21.15.45:0/4040136190 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f240035b0 con 0x7f8f3c0ca3b0 2023-08-27T11:30:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3ad9d700 1 -- 172.21.15.45:0/4040136190 >> v2:172.21.15.135:3300/0 conn(0x7f8f3c15d600 msgr2=0x7f8f3c0cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3ad9d700 1 --2- 172.21.15.45:0/4040136190 >> v2:172.21.15.135:3300/0 conn(0x7f8f3c15d600 0x7f8f3c0cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3ad9d700 1 -- 172.21.15.45:0/4040136190 >> v1:172.21.15.135:6789/0 conn(0x7f8f3c0ca3b0 legacy=0x7f8f3c0ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3ad9d700 1 -- 172.21.15.45:0/4040136190 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c06af70 con 0x7f8f3c0aed20 2023-08-27T11:30:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f39d9b700 1 --2- 172.21.15.45:0/4040136190 >> v2:172.21.15.135:3300/0 conn(0x7f8f3c15d600 0x7f8f3c0cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.369+0000 7f8f3ad9d700 1 --2- 172.21.15.45:0/4040136190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0aed20 0x7f8f3c0ce250 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009d90 tx=0x7f8f3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b93c3f6ddfe26afe server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f3959a700 1 -- 172.21.15.45:0/4040136190 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400e070 con 0x7f8f3c0aed20 2023-08-27T11:30:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f3959a700 1 -- 172.21.15.45:0/4040136190 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400bb20 con 0x7f8f3c0aed20 2023-08-27T11:30:10.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f3959a700 1 -- 172.21.15.45:0/4040136190 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34012430 con 0x7f8f3c0aed20 2023-08-27T11:30:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f40f7a700 1 -- 172.21.15.45:0/4040136190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0aed20 msgr2=0x7f8f3c0ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f40f7a700 1 --2- 172.21.15.45:0/4040136190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0aed20 0x7f8f3c0ce250 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009d90 tx=0x7f8f3400b330 comp rx=0 tx=0).stop 2023-08-27T11:30:10.380 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f40f7a700 1 -- 172.21.15.45:0/4040136190 shutdown_connections 2023-08-27T11:30:10.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f40f7a700 1 --2- 172.21.15.45:0/4040136190 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0aed20 0x7f8f3c0ce250 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f40f7a700 1 --2- 172.21.15.45:0/4040136190 >> v2:172.21.15.135:3300/0 conn(0x7f8f3c15d600 0x7f8f3c0cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.381 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f40f7a700 1 -- 172.21.15.45:0/4040136190 >> 172.21.15.45:0/4040136190 conn(0x7f8f3c0cc7e0 msgr2=0x7f8f3c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:10.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f40f7a700 1 -- 172.21.15.45:0/4040136190 shutdown_connections 2023-08-27T11:30:10.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.370+0000 7f8f40f7a700 1 -- 172.21.15.45:0/4040136190 wait complete. 2023-08-27T11:30:10.382 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.372+0000 7f8f40f7a700 1 Processor -- start 2023-08-27T11:30:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.372+0000 7f8f40f7a700 1 -- start start 2023-08-27T11:30:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f40f7a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3c0aed20 0x7f8f3c160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.383 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f40f7a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0ca3b0 0x7f8f3c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f40f7a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f3c0b0920 0x7f8f3c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.384 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f40f7a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c160cc0 con 0x7f8f3c0b0920 2023-08-27T11:30:10.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f40f7a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0b12e0 con 0x7f8f3c0aed20 2023-08-27T11:30:10.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f40f7a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0b3920 con 0x7f8f3c0ca3b0 2023-08-27T11:30:10.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f3ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f3c0b0920 0x7f8f3c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f3ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f3c0b0920 0x7f8f3c0b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60856/0 (socket says 172.21.15.45:60856) 2023-08-27T11:30:10.385 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f3ad9d700 1 -- 172.21.15.45:0/622658617 learned_addr learned my addr 172.21.15.45:0/622658617 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:10.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f3a59c700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3c0aed20 0x7f8f3c160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.373+0000 7f8f39d9b700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0ca3b0 0x7f8f3c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f3ad9d700 1 -- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0ca3b0 msgr2=0x7f8f3c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f3ad9d700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0ca3b0 0x7f8f3c159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.386 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f3ad9d700 1 -- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3c0aed20 msgr2=0x7f8f3c160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f3ad9d700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3c0aed20 0x7f8f3c160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f3ad9d700 1 -- 172.21.15.45:0/622658617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300094a0 con 0x7f8f3c0b0920 2023-08-27T11:30:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f3a59c700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3c0aed20 0x7f8f3c160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f3ad9d700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f3c0b0920 0x7f8f3c0b0da0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8f34014040 tx=0x7f8f34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:10.387 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f2affd700 1 -- 172.21.15.45:0/622658617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34012430 con 0x7f8f3c0b0920 2023-08-27T11:30:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f2affd700 1 -- 172.21.15.45:0/622658617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f34020070 con 0x7f8f3c0b0920 2023-08-27T11:30:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f34009a70 con 0x7f8f3c0b0920 2023-08-27T11:30:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f2affd700 1 -- 172.21.15.45:0/622658617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34004280 con 0x7f8f3c0b0920 2023-08-27T11:30:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.374+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c0b3080 con 0x7f8f3c0b0920 2023-08-27T11:30:10.388 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.375+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c0726d0 con 0x7f8f3c0b0920 2023-08-27T11:30:10.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.375+0000 7f8f2affd700 1 -- 172.21.15.45:0/622658617 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 52) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f8f340043e0 con 0x7f8f3c0b0920 2023-08-27T11:30:10.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.376+0000 7f8f2affd700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f8f1c06a370 0x7f8f1c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.376+0000 7f8f2affd700 1 -- 172.21.15.45:0/622658617 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8f34022070 con 0x7f8f3c0b0920 2023-08-27T11:30:10.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.376+0000 7f8f3a59c700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f8f1c06a370 0x7f8f1c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.389 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.377+0000 7f8f3a59c700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f8f1c06a370 0x7f8f1c06c830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0011e0 tx=0x7f8f2c009040 comp rx=0 tx=0).ready entity=mgr.6866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:10.390 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.380+0000 7f8f2affd700 1 -- 172.21.15.45:0/622658617 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8f34055a40 con 0x7f8f3c0b0920 2023-08-27T11:30:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.610+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 --> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f3c06f400 con 0x7f8f1c06a370 2023-08-27T11:30:10.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.675+0000 7f8f2affd700 1 -- 172.21.15.45:0/622658617 <== mgr.6866 v2:172.21.15.135:6832/1129514681 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f8f2c0014f0 con 0x7f8f1c06a370 2023-08-27T11:30:10.677 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule removed for path /bad-path 2023-08-27T11:30:10.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f8f1c06a370 msgr2=0x7f8f1c06c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f8f1c06a370 0x7f8f1c06c830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0011e0 tx=0x7f8f2c009040 comp rx=0 tx=0).stop 2023-08-27T11:30:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f3c0b0920 msgr2=0x7f8f3c0b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f3c0b0920 0x7f8f3c0b0da0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8f34014040 tx=0x7f8f34004ad0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 shutdown_connections 2023-08-27T11:30:10.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f8f1c06a370 0x7f8f1c06c830 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f3c0b0920 0x7f8f3c0b0da0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f3c0aed20 0x7f8f3c160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 --2- 172.21.15.45:0/622658617 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f8f3c0ca3b0 0x7f8f3c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 >> 172.21.15.45:0/622658617 conn(0x7f8f3c0cc7e0 msgr2=0x7f8f3c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:10.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 shutdown_connections 2023-08-27T11:30:10.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.677+0000 7f8f40f7a700 1 -- 172.21.15.45:0/622658617 wait complete. 2023-08-27T11:30:10.691 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-08-27T11:30:10.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-27T11:30:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.816+0000 7f6c4805a700 1 Processor -- start 2023-08-27T11:30:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.817+0000 7f6c4805a700 1 -- start start 2023-08-27T11:30:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.817+0000 7f6c4805a700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6c40160890 0x7f6c400cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.817+0000 7f6c4805a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c400b62c0 0x7f6c400b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.817+0000 7f6c4805a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c400cdf40 con 0x7f6c40160c70 2023-08-27T11:30:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.817+0000 7f6c4805a700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c400d0fd0 con 0x7f6c40160890 2023-08-27T11:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.817+0000 7f6c4805a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400ce190 con 0x7f6c400b62c0 2023-08-27T11:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.817+0000 7f6c45df6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6c40160890 0x7f6c400cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c45df6700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6c40160890 0x7f6c400cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60860/0 (socket says 172.21.15.45:60860) 2023-08-27T11:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c45df6700 1 -- 172.21.15.45:0/1669606241 learned_addr learned my addr 172.21.15.45:0/1669606241 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c455f5700 1 --2- 172.21.15.45:0/1669606241 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c400b62c0 0x7f6c400b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c45df6700 1 -- 172.21.15.45:0/1669606241 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c400b62c0 msgr2=0x7f6c400b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c45df6700 1 --2- 172.21.15.45:0/1669606241 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c400b62c0 0x7f6c400b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c45df6700 1 -- 172.21.15.45:0/1669606241 >> v1:172.21.15.135:6789/0 conn(0x7f6c40160c70 legacy=0x7f6c40161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c45df6700 1 -- 172.21.15.45:0/1669606241 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400ce470 con 0x7f6c40160890 2023-08-27T11:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c455f5700 1 --2- 172.21.15.45:0/1669606241 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c400b62c0 0x7f6c400b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c45df6700 1 --2- 172.21.15.45:0/1669606241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160890 0x7f6c400cd8f0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098f0 tx=0x7f6c3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f165d2dc625992fd server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c44df4700 1 -- 172.21.15.45:0/1669606241 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00b8a0 con 0x7f6c40160890 2023-08-27T11:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c44df4700 1 -- 172.21.15.45:0/1669606241 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c00ba00 con 0x7f6c40160890 2023-08-27T11:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.818+0000 7f6c44df4700 1 -- 172.21.15.45:0/1669606241 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00bd90 con 0x7f6c40160890 2023-08-27T11:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.819+0000 7f6c4805a700 1 -- 172.21.15.45:0/1669606241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160890 msgr2=0x7f6c400cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.819+0000 7f6c4805a700 1 --2- 172.21.15.45:0/1669606241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160890 0x7f6c400cd8f0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098f0 tx=0x7f6c3c009c00 comp rx=0 tx=0).stop 2023-08-27T11:30:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.819+0000 7f6c4805a700 1 -- 172.21.15.45:0/1669606241 shutdown_connections 2023-08-27T11:30:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.819+0000 7f6c4805a700 1 --2- 172.21.15.45:0/1669606241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160890 0x7f6c400cd8f0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.819+0000 7f6c4805a700 1 --2- 172.21.15.45:0/1669606241 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c400b62c0 0x7f6c400b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.819+0000 7f6c4805a700 1 -- 172.21.15.45:0/1669606241 >> 172.21.15.45:0/1669606241 conn(0x7f6c400cfb10 msgr2=0x7f6c400cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:10.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.819+0000 7f6c4805a700 1 -- 172.21.15.45:0/1669606241 shutdown_connections 2023-08-27T11:30:10.826 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.819+0000 7f6c4805a700 1 -- 172.21.15.45:0/1669606241 wait complete. 2023-08-27T11:30:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.820+0000 7f6c4805a700 1 Processor -- start 2023-08-27T11:30:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.821+0000 7f6c4805a700 1 -- start start 2023-08-27T11:30:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.821+0000 7f6c4805a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c400b62c0 0x7f6c400a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.821+0000 7f6c4805a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c40160890 0x7f6c4015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.821+0000 7f6c4805a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160c70 0x7f6c400aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c4805a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4015d380 con 0x7f6c40160c70 2023-08-27T11:30:10.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c4805a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4015c880 con 0x7f6c400b62c0 2023-08-27T11:30:10.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c4805a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c401506b0 con 0x7f6c40160890 2023-08-27T11:30:10.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c465f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160c70 0x7f6c400aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c465f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160c70 0x7f6c400aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60870/0 (socket says 172.21.15.45:60870) 2023-08-27T11:30:10.830 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c465f7700 1 -- 172.21.15.45:0/3012548754 learned_addr learned my addr 172.21.15.45:0/3012548754 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c455f5700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c40160890 0x7f6c4015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c45df6700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c400b62c0 0x7f6c400a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c465f7700 1 -- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c40160890 msgr2=0x7f6c4015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c465f7700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c40160890 0x7f6c4015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c465f7700 1 -- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c400b62c0 msgr2=0x7f6c400a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c465f7700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c400b62c0 0x7f6c400a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c465f7700 1 -- 172.21.15.45:0/3012548754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c009550 con 0x7f6c40160c70 2023-08-27T11:30:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c455f5700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c40160890 0x7f6c4015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c45df6700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c400b62c0 0x7f6c400a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c465f7700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160c70 0x7f6c400aa8c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800cbc0 tx=0x7f6c3800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c327fc700 1 -- 172.21.15.45:0/3012548754 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38003c00 con 0x7f6c40160c70 2023-08-27T11:30:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c327fc700 1 -- 172.21.15.45:0/3012548754 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c38003d60 con 0x7f6c40160c70 2023-08-27T11:30:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4015d830 con 0x7f6c40160c70 2023-08-27T11:30:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c327fc700 1 -- 172.21.15.45:0/3012548754 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800ab90 con 0x7f6c40160c70 2023-08-27T11:30:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.822+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4015cdc0 con 0x7f6c40160c70 2023-08-27T11:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.823+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4015da50 con 0x7f6c40160c70 2023-08-27T11:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.824+0000 7f6c327fc700 1 -- 172.21.15.45:0/3012548754 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 52) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f6c3800acf0 con 0x7f6c40160c70 2023-08-27T11:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.824+0000 7f6c327fc700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f6c2406a370 0x7f6c2406c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.824+0000 7f6c327fc700 1 -- 172.21.15.45:0/3012548754 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6c38088cd0 con 0x7f6c40160c70 2023-08-27T11:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.824+0000 7f6c45df6700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f6c2406a370 0x7f6c2406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.825+0000 7f6c45df6700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f6c2406a370 0x7f6c2406c830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098f0 tx=0x7f6c3c008030 comp rx=0 tx=0).ready entity=mgr.6866 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:10.829+0000 7f6c327fc700 1 -- 172.21.15.45:0/3012548754 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6c4015da50 con 0x7f6c40160c70 2023-08-27T11:30:11.089 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.088+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f6c4015da20 con 0x7f6c40160c70 2023-08-27T11:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.164+0000 7f6c327fc700 1 -- 172.21.15.45:0/3012548754 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f6c380541d0 con 0x7f6c40160c70 2023-08-27T11:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.166+0000 7f6c327fc700 1 -- 172.21.15.45:0/3012548754 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v53) v1 ==== 93+0+0 (secure 0 0 0) 0x7f6c38057890 con 0x7f6c40160c70 2023-08-27T11:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f6c2406a370 msgr2=0x7f6c2406c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f6c2406a370 0x7f6c2406c830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098f0 tx=0x7f6c3c008030 comp rx=0 tx=0).stop 2023-08-27T11:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160c70 msgr2=0x7f6c400aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160c70 0x7f6c400aa8c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800cbc0 tx=0x7f6c3800cf80 comp rx=0 tx=0).stop 2023-08-27T11:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 shutdown_connections 2023-08-27T11:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f6c2406a370 0x7f6c2406c830 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40160c70 0x7f6c400aa8c0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c400b62c0 0x7f6c400a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 --2- 172.21.15.45:0/3012548754 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f6c40160890 0x7f6c4015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 >> 172.21.15.45:0/3012548754 conn(0x7f6c400cfb10 msgr2=0x7f6c400b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 shutdown_connections 2023-08-27T11:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.168+0000 7f6c4805a700 1 -- 172.21.15.45:0/3012548754 wait complete. 2023-08-27T11:30:11.182 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-27T11:30:11.183 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:30:11.222 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-27T11:30:11.267 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-27T11:30:11.268 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:30:11.268 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:30:11.286 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:30:11.287 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:30:11.312 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:11.311+0000 7fcb8ec68200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:30:11.313 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:11.311+0000 7fb732698200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-27T11:30:11.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:30:11.377 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:11.376+0000 7fcb8ec68200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:30:11.385 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:11.384+0000 7fb732698200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-27T11:30:11.449 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:11.448+0000 7fcb8ec68200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:30:11.461 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:11.460+0000 7fb732698200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-27T11:30:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.497+0000 7fba75d5d700 1 Processor -- start 2023-08-27T11:30:11.499 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.497+0000 7fba75d5d700 1 -- start start 2023-08-27T11:30:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba75d5d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba7015d2c0 0x7fba700d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.501 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba75d5d700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fba700b8c20 0x7fba700cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba75d5d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700ce2e0 0x7fba7006c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba75d5d700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba700ce770 con 0x7fba7015d2c0 2023-08-27T11:30:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba75d5d700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7015d6a0 con 0x7fba700b8c20 2023-08-27T11:30:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba75d5d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba700d2c90 con 0x7fba700ce2e0 2023-08-27T11:30:11.502 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba7015d2c0 0x7fba700d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6f7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba7015d2c0 0x7fba700d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60944/0 (socket says 172.21.15.45:60944) 2023-08-27T11:30:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6f7fe700 1 -- 172.21.15.45:0/3523029250 learned_addr learned my addr 172.21.15.45:0/3523029250 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6effd700 1 --2- 172.21.15.45:0/3523029250 >> v2:172.21.15.135:3300/0 conn(0x7fba700b8c20 0x7fba700cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6ffff700 1 --2- 172.21.15.45:0/3523029250 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700ce2e0 0x7fba7006c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6f7fe700 1 -- 172.21.15.45:0/3523029250 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700ce2e0 msgr2=0x7fba7006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:11.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6f7fe700 1 --2- 172.21.15.45:0/3523029250 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700ce2e0 0x7fba7006c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6f7fe700 1 -- 172.21.15.45:0/3523029250 >> v2:172.21.15.135:3300/0 conn(0x7fba700b8c20 msgr2=0x7fba700cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6f7fe700 1 --2- 172.21.15.45:0/3523029250 >> v2:172.21.15.135:3300/0 conn(0x7fba700b8c20 0x7fba700cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6f7fe700 1 -- 172.21.15.45:0/3523029250 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700d1bf0 con 0x7fba7015d2c0 2023-08-27T11:30:11.504 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.498+0000 7fba6f7fe700 1 --2- 172.21.15.45:0/3523029250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7015d2c0 0x7fba700d13a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fba600098f0 tx=0x7fba60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0be8159136c70c server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba6e7fc700 1 -- 172.21.15.45:0/3523029250 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60011830 con 0x7fba7015d2c0 2023-08-27T11:30:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba6e7fc700 1 -- 172.21.15.45:0/3523029250 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba60011990 con 0x7fba7015d2c0 2023-08-27T11:30:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba6e7fc700 1 -- 172.21.15.45:0/3523029250 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60011d20 con 0x7fba7015d2c0 2023-08-27T11:30:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba75d5d700 1 -- 172.21.15.45:0/3523029250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7015d2c0 msgr2=0x7fba700d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba75d5d700 1 --2- 172.21.15.45:0/3523029250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7015d2c0 0x7fba700d13a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fba600098f0 tx=0x7fba60009c00 comp rx=0 tx=0).stop 2023-08-27T11:30:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba75d5d700 1 -- 172.21.15.45:0/3523029250 shutdown_connections 2023-08-27T11:30:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba75d5d700 1 --2- 172.21.15.45:0/3523029250 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700ce2e0 0x7fba7006c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba75d5d700 1 --2- 172.21.15.45:0/3523029250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7015d2c0 0x7fba700d13a0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba75d5d700 1 --2- 172.21.15.45:0/3523029250 >> v2:172.21.15.135:3300/0 conn(0x7fba700b8c20 0x7fba700cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba75d5d700 1 -- 172.21.15.45:0/3523029250 >> 172.21.15.45:0/3523029250 conn(0x7fba700d2fd0 msgr2=0x7fba700d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba75d5d700 1 -- 172.21.15.45:0/3523029250 shutdown_connections 2023-08-27T11:30:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.499+0000 7fba75d5d700 1 -- 172.21.15.45:0/3523029250 wait complete. 2023-08-27T11:30:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.501+0000 7fba75d5d700 1 Processor -- start 2023-08-27T11:30:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.501+0000 7fba75d5d700 1 -- start start 2023-08-27T11:30:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.501+0000 7fba75d5d700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700b8c20 0x7fba701f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.501+0000 7fba75d5d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba700ce2e0 0x7fba7015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.501+0000 7fba75d5d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba7015d2c0 0x7fba7013f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.501+0000 7fba75d5d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba70160140 con 0x7fba700ce2e0 2023-08-27T11:30:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.501+0000 7fba75d5d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7013fec0 con 0x7fba7015d2c0 2023-08-27T11:30:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.501+0000 7fba75d5d700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba701f2d40 con 0x7fba700b8c20 2023-08-27T11:30:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba700ce2e0 0x7fba7015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.511 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba700ce2e0 0x7fba7015fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60952/0 (socket says 172.21.15.45:60952) 2023-08-27T11:30:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6effd700 1 -- 172.21.15.45:0/2083707158 learned_addr learned my addr 172.21.15.45:0/2083707158 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6f7fe700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700b8c20 0x7fba701f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6ffff700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba7015d2c0 0x7fba7013f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6f7fe700 1 -- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba7015d2c0 msgr2=0x7fba7013f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6f7fe700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba7015d2c0 0x7fba7013f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6f7fe700 1 -- 172.21.15.45:0/2083707158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba700ce2e0 msgr2=0x7fba7015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6f7fe700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba700ce2e0 0x7fba7015fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6f7fe700 1 -- 172.21.15.45:0/2083707158 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba60009550 con 0x7fba700b8c20 2023-08-27T11:30:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6effd700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba700ce2e0 0x7fba7015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba6f7fe700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700b8c20 0x7fba701f2710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fba60016000 tx=0x7fba60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba57fff700 1 -- 172.21.15.45:0/2083707158 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60004450 con 0x7fba700b8c20 2023-08-27T11:30:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba57fff700 1 -- 172.21.15.45:0/2083707158 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6001e070 con 0x7fba700b8c20 2023-08-27T11:30:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba701f3020 con 0x7fba700b8c20 2023-08-27T11:30:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.502+0000 7fba57fff700 1 -- 172.21.15.45:0/2083707158 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba600038e0 con 0x7fba700b8c20 2023-08-27T11:30:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.503+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba701f35e0 con 0x7fba700b8c20 2023-08-27T11:30:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.503+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7004e620 con 0x7fba700b8c20 2023-08-27T11:30:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.504+0000 7fba57fff700 1 -- 172.21.15.45:0/2083707158 <== mon.2 v2:172.21.15.135:3301/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fba60044ab0 con 0x7fba700b8c20 2023-08-27T11:30:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.504+0000 7fba57fff700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fba5c06a320 0x7fba5c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.504+0000 7fba6effd700 1 -- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fba5c06a320 msgr2=0x7fba5c06c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.504+0000 7fba57fff700 1 -- 172.21.15.45:0/2083707158 <== mon.2 v2:172.21.15.135:3301/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fba60089900 con 0x7fba700b8c20 2023-08-27T11:30:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.505+0000 7fba6effd700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fba5c06a320 0x7fba5c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.508+0000 7fba57fff700 1 -- 172.21.15.45:0/2083707158 <== mon.2 v2:172.21.15.135:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fba60054be0 con 0x7fba700b8c20 2023-08-27T11:30:11.527 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:11.526+0000 7fcb8ec68200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:30:11.547 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:11.546+0000 7fb732698200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-27T11:30:11.667 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:11.666+0000 7fcb8ec68200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:30:11.696 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:11.695+0000 7fb732698200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-27T11:30:11.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.706+0000 7fba6effd700 1 -- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fba5c06a320 msgr2=0x7fba5c06c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:11.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.706+0000 7fba6effd700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fba5c06a320 0x7fba5c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:11.712 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.710+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba700d0480 con 0x7fba700b8c20 2023-08-27T11:30:11.713 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.712+0000 7fba57fff700 1 -- 172.21.15.45:0/2083707158 <== mon.2 v2:172.21.15.135:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+19157 (secure 0 0 0) 0x7fba60015090 con 0x7fba700b8c20 2023-08-27T11:30:11.714 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:30:11.714 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":117,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:29:08.294477+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.834012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.480260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/3391794165":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/2740998365":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/1494860999":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/91399608":"2023-08-28T11:29:08.294424+0000","172.21.15.135:6833/3759624346":"2023-08-28T11:29:08.294424+0000","172.21.15.135:6832/3759624346":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/4148146534":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/3025460570":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/626141668":"2023-08-27T12:28:58.549630+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:30:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fba5c06a320 msgr2=0x7fba5c06c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fba5c06a320 0x7fba5c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700b8c20 msgr2=0x7fba701f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700b8c20 0x7fba701f2710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fba60016000 tx=0x7fba60004750 comp rx=0 tx=0).stop 2023-08-27T11:30:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 shutdown_connections 2023-08-27T11:30:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba7015d2c0 0x7fba7013f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fba700b8c20 0x7fba701f2710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fba5c06a320 0x7fba5c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 --2- 172.21.15.45:0/2083707158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba700ce2e0 0x7fba7015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 >> 172.21.15.45:0/2083707158 conn(0x7fba700d2fd0 msgr2=0x7fba7015c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 shutdown_connections 2023-08-27T11:30:11.718 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.714+0000 7fba75d5d700 1 -- 172.21.15.45:0/2083707158 wait complete. 2023-08-27T11:30:11.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:28:47.834012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:28:48.480260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:30:11.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:30:11.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.897+0000 7fa069159700 1 Processor -- start 2023-08-27T11:30:11.899 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.897+0000 7fa069159700 1 -- start start 2023-08-27T11:30:11.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.898+0000 7fa069159700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 0x7fa0640cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.898+0000 7fa069159700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0640cdf40 con 0x7fa0640b5220 2023-08-27T11:30:11.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.898+0000 7fa069159700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0640d0fd0 con 0x7fa064160890 2023-08-27T11:30:11.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.898+0000 7fa069159700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640ce190 con 0x7fa0640b62c0 2023-08-27T11:30:11.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.898+0000 7fa06259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa0640b5220 0x7fa0640d1460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47780/0 (socket says 172.21.15.45:47780) 2023-08-27T11:30:11.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.898+0000 7fa06259c700 1 -- 172.21.15.45:0/2331758948 learned_addr learned my addr 172.21.15.45:0/2331758948 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:11.903 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.898+0000 7fa06359e700 1 --2- 172.21.15.45:0/2331758948 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 0x7fa0640cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.898+0000 7fa061d9b700 1 -- 172.21.15.45:0/2331758948 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081702683 0 0) 0x7fa0640cdf40 con 0x7fa0640b5220 2023-08-27T11:30:11.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa061d9b700 1 -- 172.21.15.45:0/2331758948 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa04c0035b0 con 0x7fa0640b5220 2023-08-27T11:30:11.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa061d9b700 1 -- 172.21.15.45:0/2331758948 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235848034 0 0) 0x7fa0640d0fd0 con 0x7fa064160890 2023-08-27T11:30:11.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa061d9b700 1 -- 172.21.15.45:0/2331758948 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0640cdf40 con 0x7fa064160890 2023-08-27T11:30:11.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa06359e700 1 -- 172.21.15.45:0/2331758948 >> v1:172.21.15.135:6789/0 conn(0x7fa064160890 legacy=0x7fa0640b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa06359e700 1 -- 172.21.15.45:0/2331758948 >> v1:172.21.15.45:6789/0 conn(0x7fa0640b5220 legacy=0x7fa0640d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa06359e700 1 -- 172.21.15.45:0/2331758948 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0640ce470 con 0x7fa0640b62c0 2023-08-27T11:30:11.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa06359e700 1 --2- 172.21.15.45:0/2331758948 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 0x7fa0640cd8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009dc0 tx=0x7fa05c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dad7cd71b08629ec server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa061d9b700 1 -- 172.21.15.45:0/2331758948 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00e070 con 0x7fa0640b62c0 2023-08-27T11:30:11.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa061d9b700 1 -- 172.21.15.45:0/2331758948 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c00bb20 con 0x7fa0640b62c0 2023-08-27T11:30:11.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.899+0000 7fa061d9b700 1 -- 172.21.15.45:0/2331758948 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c012430 con 0x7fa0640b62c0 2023-08-27T11:30:11.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.900+0000 7fa069159700 1 -- 172.21.15.45:0/2331758948 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 msgr2=0x7fa0640cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.900+0000 7fa069159700 1 --2- 172.21.15.45:0/2331758948 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 0x7fa0640cd8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009dc0 tx=0x7fa05c00b330 comp rx=0 tx=0).stop 2023-08-27T11:30:11.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.900+0000 7fa069159700 1 -- 172.21.15.45:0/2331758948 shutdown_connections 2023-08-27T11:30:11.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.900+0000 7fa069159700 1 --2- 172.21.15.45:0/2331758948 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 0x7fa0640cd8f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.900+0000 7fa069159700 1 -- 172.21.15.45:0/2331758948 >> 172.21.15.45:0/2331758948 conn(0x7fa0640cfb10 msgr2=0x7fa0640cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:11.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.900+0000 7fa069159700 1 -- 172.21.15.45:0/2331758948 shutdown_connections 2023-08-27T11:30:11.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.900+0000 7fa069159700 1 -- 172.21.15.45:0/2331758948 wait complete. 2023-08-27T11:30:11.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.901+0000 7fa069159700 1 Processor -- start 2023-08-27T11:30:11.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.901+0000 7fa069159700 1 -- start start 2023-08-27T11:30:11.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa069159700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0640b5220 0x7fa06415cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa069159700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 0x7fa0640ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa069159700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa064160890 0x7fa0640abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa069159700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640cb1b0 con 0x7fa0640b5220 2023-08-27T11:30:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa069159700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640aae60 con 0x7fa064160890 2023-08-27T11:30:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa069159700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640ab140 con 0x7fa0640b62c0 2023-08-27T11:30:11.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa062d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0640b5220 0x7fa06415cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa062d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0640b5220 0x7fa06415cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32778/0 (socket says 172.21.15.45:32778) 2023-08-27T11:30:11.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa062d9d700 1 -- 172.21.15.45:0/4022308149 learned_addr learned my addr 172.21.15.45:0/4022308149 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:11.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa06259c700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 0x7fa0640ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.902+0000 7fa06359e700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa064160890 0x7fa0640abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:11.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa062d9d700 1 -- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 msgr2=0x7fa0640ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa062d9d700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 0x7fa0640ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa062d9d700 1 -- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa064160890 msgr2=0x7fa0640abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:11.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa062d9d700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa064160890 0x7fa0640abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa062d9d700 1 -- 172.21.15.45:0/4022308149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c009a70 con 0x7fa0640b5220 2023-08-27T11:30:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa06359e700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa064160890 0x7fa0640abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa062d9d700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0640b5220 0x7fa06415cbf0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c500 tx=0x7fa05400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa04b7fe700 1 -- 172.21.15.45:0/4022308149 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540052d0 con 0x7fa0640b5220 2023-08-27T11:30:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa04b7fe700 1 -- 172.21.15.45:0/4022308149 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa054005450 con 0x7fa0640b5220 2023-08-27T11:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa04b7fe700 1 -- 172.21.15.45:0/4022308149 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540038c0 con 0x7fa0640b5220 2023-08-27T11:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0640cb400 con 0x7fa0640b5220 2023-08-27T11:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.903+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0640cd470 con 0x7fa0640b5220 2023-08-27T11:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.904+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06406c120 con 0x7fa0640b5220 2023-08-27T11:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.905+0000 7fa04b7fe700 1 -- 172.21.15.45:0/4022308149 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fa054003a20 con 0x7fa0640b5220 2023-08-27T11:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.905+0000 7fa04b7fe700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa05006a320 0x7fa05006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.905+0000 7fa06259c700 1 -- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa05006a320 msgr2=0x7fa05006c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.905+0000 7fa04b7fe700 1 -- 172.21.15.45:0/4022308149 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa054087ed0 con 0x7fa0640b5220 2023-08-27T11:30:11.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.905+0000 7fa06259c700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa05006a320 0x7fa05006c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:11.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:11.909+0000 7fa04b7fe700 1 -- 172.21.15.45:0/4022308149 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa0540530d0 con 0x7fa0640b5220 2023-08-27T11:30:11.990 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:11.989+0000 7fcb8ec68200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:30:12.046 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:12.044+0000 7fb732698200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-27T11:30:12.105 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:12.104+0000 7fcb8ec68200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:30:12.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.106+0000 7fa06259c700 1 -- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa05006a320 msgr2=0x7fa05006c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.106+0000 7fa06259c700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa05006a320 0x7fa05006c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.112+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0640734a0 con 0x7fa0640b5220 2023-08-27T11:30:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.112+0000 7fa04b7fe700 1 -- 172.21.15.45:0/4022308149 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa054017020 con 0x7fa0640b5220 2023-08-27T11:30:12.115 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:30:12.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa05006a320 msgr2=0x7fa05006c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:12.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa05006a320 0x7fa05006c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0640b5220 msgr2=0x7fa06415cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0640b5220 0x7fa06415cbf0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c500 tx=0x7fa05400c810 comp rx=0 tx=0).stop 2023-08-27T11:30:12.118 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 shutdown_connections 2023-08-27T11:30:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa05006a320 0x7fa05006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0640b5220 0x7fa06415cbf0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa064160890 0x7fa0640abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 --2- 172.21.15.45:0/4022308149 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa0640b62c0 0x7fa0640ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 >> 172.21.15.45:0/4022308149 conn(0x7fa0640cfb10 msgr2=0x7fa0640b7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 shutdown_connections 2023-08-27T11:30:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.115+0000 7fa069159700 1 -- 172.21.15.45:0/4022308149 wait complete. 2023-08-27T11:30:12.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:30:12.169 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:12.168+0000 7fb732698200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-27T11:30:12.178 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:12.177+0000 7fcb8ec68200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:30:12.248 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:12.246+0000 7fb732698200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-27T11:30:12.258 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:12.257+0000 7fcb8ec68200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:30:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.261+0000 7f683d45e700 1 Processor -- start 2023-08-27T11:30:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.261+0000 7f683d45e700 1 -- start start 2023-08-27T11:30:12.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f683d45e700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68380b5220 0x7f68380cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f683d45e700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f68380b62c0 0x7f68380d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f683d45e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68380cdf40 con 0x7f6838160890 2023-08-27T11:30:12.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f683d45e700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68380d0fd0 con 0x7f68380b5220 2023-08-27T11:30:12.265 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f683d45e700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68380ce190 con 0x7f68380b62c0 2023-08-27T11:30:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f68367fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68380b5220 0x7f68380cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f6836ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6838160890 0x7f68380b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47796/0 (socket says 172.21.15.45:47796) 2023-08-27T11:30:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f68367fc700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68380b5220 0x7f68380cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32790/0 (socket says 172.21.15.45:32790) 2023-08-27T11:30:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f6836ffd700 1 -- 172.21.15.45:0/2265948264 learned_addr learned my addr 172.21.15.45:0/2265948264 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:12.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f68377fe700 1 --2- 172.21.15.45:0/2265948264 >> v2:172.21.15.135:3300/0 conn(0x7f68380b62c0 0x7f68380d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f68367fc700 1 -- 172.21.15.45:0/2265948264 >> v2:172.21.15.135:3300/0 conn(0x7f68380b62c0 msgr2=0x7f68380d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f68367fc700 1 --2- 172.21.15.45:0/2265948264 >> v2:172.21.15.135:3300/0 conn(0x7f68380b62c0 0x7f68380d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f68367fc700 1 -- 172.21.15.45:0/2265948264 >> v1:172.21.15.45:6789/0 conn(0x7f6838160890 legacy=0x7f68380b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f6835ffb700 1 -- 172.21.15.45:0/2265948264 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383569395 0 0) 0x7f68380cdf40 con 0x7f6838160890 2023-08-27T11:30:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.262+0000 7f68367fc700 1 -- 172.21.15.45:0/2265948264 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380ce470 con 0x7f68380b5220 2023-08-27T11:30:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f68367fc700 1 --2- 172.21.15.45:0/2265948264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68380b5220 0x7f68380cd8f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6838162220 tx=0x7f682c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62ee3a92811dd8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f6835ffb700 1 -- 172.21.15.45:0/2265948264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c011830 con 0x7f68380b5220 2023-08-27T11:30:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f6835ffb700 1 -- 172.21.15.45:0/2265948264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c011990 con 0x7f68380b5220 2023-08-27T11:30:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f6835ffb700 1 -- 172.21.15.45:0/2265948264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c011d20 con 0x7f68380b5220 2023-08-27T11:30:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f683d45e700 1 -- 172.21.15.45:0/2265948264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68380b5220 msgr2=0x7f68380cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f683d45e700 1 --2- 172.21.15.45:0/2265948264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68380b5220 0x7f68380cd8f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6838162220 tx=0x7f682c009b20 comp rx=0 tx=0).stop 2023-08-27T11:30:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f683d45e700 1 -- 172.21.15.45:0/2265948264 shutdown_connections 2023-08-27T11:30:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f683d45e700 1 --2- 172.21.15.45:0/2265948264 >> v2:172.21.15.135:3300/0 conn(0x7f68380b62c0 0x7f68380d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f683d45e700 1 --2- 172.21.15.45:0/2265948264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68380b5220 0x7f68380cd8f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.263+0000 7f683d45e700 1 -- 172.21.15.45:0/2265948264 >> 172.21.15.45:0/2265948264 conn(0x7f68380cfb10 msgr2=0x7f68380cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.264+0000 7f683d45e700 1 -- 172.21.15.45:0/2265948264 shutdown_connections 2023-08-27T11:30:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.264+0000 7f683d45e700 1 -- 172.21.15.45:0/2265948264 wait complete. 2023-08-27T11:30:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.265+0000 7f683d45e700 1 Processor -- start 2023-08-27T11:30:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.265+0000 7f683d45e700 1 -- start start 2023-08-27T11:30:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f683d45e700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68380b5220 0x7f68381f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f683d45e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68380b62c0 0x7f68381529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f683d45e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6838160890 0x7f68381560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f683d45e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380ca160 con 0x7f6838160890 2023-08-27T11:30:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f683d45e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68381f2e40 con 0x7f68380b62c0 2023-08-27T11:30:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f683d45e700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380ca3b0 con 0x7f68380b5220 2023-08-27T11:30:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6838160890 0x7f68381560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68377fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6838160890 0x7f68381560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32798/0 (socket says 172.21.15.45:32798) 2023-08-27T11:30:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68377fe700 1 -- 172.21.15.45:0/1782798512 learned_addr learned my addr 172.21.15.45:0/1782798512 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68367fc700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68380b62c0 0x7f68381529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f6836ffd700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68380b5220 0x7f68381f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68377fe700 1 -- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68380b5220 msgr2=0x7f68381f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68377fe700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68380b5220 0x7f68381f2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68377fe700 1 -- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68380b62c0 msgr2=0x7f68381529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68377fe700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68380b62c0 0x7f68381529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68377fe700 1 -- 172.21.15.45:0/1782798512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c0094a0 con 0x7f6838160890 2023-08-27T11:30:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68367fc700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68380b62c0 0x7f68381529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.266+0000 7f68377fe700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6838160890 0x7f68381560a0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f683000efc0 tx=0x7f683000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.267+0000 7f681f7fe700 1 -- 172.21.15.45:0/1782798512 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683000cc90 con 0x7f6838160890 2023-08-27T11:30:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.267+0000 7f681f7fe700 1 -- 172.21.15.45:0/1782798512 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683000f040 con 0x7f6838160890 2023-08-27T11:30:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.267+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68380ca690 con 0x7f6838160890 2023-08-27T11:30:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.267+0000 7f681f7fe700 1 -- 172.21.15.45:0/1782798512 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68300146c0 con 0x7f6838160890 2023-08-27T11:30:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.267+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68380cac50 con 0x7f6838160890 2023-08-27T11:30:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.268+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683806c120 con 0x7f6838160890 2023-08-27T11:30:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.269+0000 7f681f7fe700 1 -- 172.21.15.45:0/1782798512 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f6830014820 con 0x7f6838160890 2023-08-27T11:30:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.270+0000 7f681f7fe700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f682406a320 0x7f682406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.270+0000 7f681f7fe700 1 -- 172.21.15.45:0/1782798512 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6830089da0 con 0x7f6838160890 2023-08-27T11:30:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.270+0000 7f6836ffd700 1 -- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f682406a320 msgr2=0x7f682406c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.270+0000 7f6836ffd700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f682406a320 0x7f682406c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.273+0000 7f681f7fe700 1 -- 172.21.15.45:0/1782798512 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6830054ea0 con 0x7f6838160890 2023-08-27T11:30:12.336 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:12.335+0000 7fb732698200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-27T11:30:12.472 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.471+0000 7f6836ffd700 1 -- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f682406a320 msgr2=0x7f682406c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:12.473 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.471+0000 7f6836ffd700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f682406a320 0x7f682406c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.472+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f68380734a0 con 0x7f6838160890 2023-08-27T11:30:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.473+0000 7f681f7fe700 1 -- 172.21.15.45:0/1782798512 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v117) v1 ==== 92+0+10 (secure 0 0 0) 0x7f68300180a0 con 0x7f6838160890 2023-08-27T11:30:12.474 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:30:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f682406a320 msgr2=0x7f682406c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f682406a320 0x7f682406c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6838160890 msgr2=0x7f68381560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6838160890 0x7f68381560a0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f683000efc0 tx=0x7f683000c3a0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 shutdown_connections 2023-08-27T11:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f682406a320 0x7f682406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6838160890 0x7f68381560a0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f68380b5220 0x7f68381f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 --2- 172.21.15.45:0/1782798512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68380b62c0 0x7f68381529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 >> 172.21.15.45:0/1782798512 conn(0x7f68380cfb10 msgr2=0x7f683815b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 shutdown_connections 2023-08-27T11:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.475+0000 7f683d45e700 1 -- 172.21.15.45:0/1782798512 wait complete. 2023-08-27T11:30:12.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:30:12.626 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:12.625+0000 7fcb8ec68200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:30:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.657+0000 7f594b9e7700 1 Processor -- start 2023-08-27T11:30:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.657+0000 7f594b9e7700 1 -- start start 2023-08-27T11:30:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.658+0000 7f594b9e7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 0x7f59440d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.658+0000 7f594b9e7700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59440d02c0 0x7f594406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.658+0000 7f594b9e7700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f594415d780 con 0x7f59440b0320 2023-08-27T11:30:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.658+0000 7f594b9e7700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59440cfe30 con 0x7f59440d02c0 2023-08-27T11:30:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.658+0000 7f594b9e7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440cf220 con 0x7f594415d3a0 2023-08-27T11:30:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.658+0000 7f5948f82700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59440d02c0 0x7f594406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.658+0000 7f5948f82700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59440d02c0 0x7f594406bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32826/0 (socket says 172.21.15.45:32826) 2023-08-27T11:30:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.658+0000 7f5948f82700 1 -- 172.21.15.45:0/2662775460 learned_addr learned my addr 172.21.15.45:0/2662775460 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.658+0000 7f5949783700 1 --2- 172.21.15.45:0/2662775460 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 0x7f59440d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.659+0000 7f5948f82700 1 -- 172.21.15.45:0/2662775460 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 msgr2=0x7f59440d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.659+0000 7f5948f82700 1 --2- 172.21.15.45:0/2662775460 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 0x7f59440d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.659+0000 7f5948f82700 1 -- 172.21.15.45:0/2662775460 >> v1:172.21.15.135:6789/0 conn(0x7f59440b0320 legacy=0x7f59440cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.659+0000 7f5948f82700 1 -- 172.21.15.45:0/2662775460 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440cf500 con 0x7f59440d02c0 2023-08-27T11:30:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.659+0000 7f5948f82700 1 --2- 172.21.15.45:0/2662775460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 0x7f594406bda0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f59340098f0 tx=0x7f5934009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b56a0989b4cc38 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.659+0000 7f593bfff700 1 -- 172.21.15.45:0/2662775460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934011830 con 0x7f59440d02c0 2023-08-27T11:30:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.659+0000 7f593bfff700 1 -- 172.21.15.45:0/2662775460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5934011990 con 0x7f59440d02c0 2023-08-27T11:30:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.659+0000 7f593bfff700 1 -- 172.21.15.45:0/2662775460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934011d20 con 0x7f59440d02c0 2023-08-27T11:30:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.660+0000 7f594b9e7700 1 -- 172.21.15.45:0/2662775460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 msgr2=0x7f594406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.660+0000 7f594b9e7700 1 --2- 172.21.15.45:0/2662775460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 0x7f594406bda0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f59340098f0 tx=0x7f5934009c00 comp rx=0 tx=0).stop 2023-08-27T11:30:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.660+0000 7f594b9e7700 1 -- 172.21.15.45:0/2662775460 shutdown_connections 2023-08-27T11:30:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.660+0000 7f594b9e7700 1 --2- 172.21.15.45:0/2662775460 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 0x7f59440d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.660+0000 7f594b9e7700 1 --2- 172.21.15.45:0/2662775460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 0x7f594406bda0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.660+0000 7f594b9e7700 1 -- 172.21.15.45:0/2662775460 >> 172.21.15.45:0/2662775460 conn(0x7f594409a1a0 msgr2=0x7f59440d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.660+0000 7f594b9e7700 1 -- 172.21.15.45:0/2662775460 shutdown_connections 2023-08-27T11:30:12.666 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.660+0000 7f594b9e7700 1 -- 172.21.15.45:0/2662775460 wait complete. 2023-08-27T11:30:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.661+0000 7f594b9e7700 1 Processor -- start 2023-08-27T11:30:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.661+0000 7f594b9e7700 1 -- start start 2023-08-27T11:30:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f594b9e7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59440b0320 0x7f59440b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f594b9e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 0x7f5944151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f594b9e7700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 0x7f59440b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f594b9e7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440b5d60 con 0x7f59440d02c0 2023-08-27T11:30:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f594b9e7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440c7440 con 0x7f59440b0320 2023-08-27T11:30:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f594b9e7700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440b3be0 con 0x7f594415d3a0 2023-08-27T11:30:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5948f82700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 0x7f5944151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5948f82700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 0x7f5944151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32832/0 (socket says 172.21.15.45:32832) 2023-08-27T11:30:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5948f82700 1 -- 172.21.15.45:0/2889322710 learned_addr learned my addr 172.21.15.45:0/2889322710 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5949783700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59440b0320 0x7f59440b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5949f84700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 0x7f59440b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5948f82700 1 -- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 msgr2=0x7f59440b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5948f82700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 0x7f59440b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5948f82700 1 -- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59440b0320 msgr2=0x7f59440b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5948f82700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59440b0320 0x7f59440b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.662+0000 7f5948f82700 1 -- 172.21.15.45:0/2889322710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5934009550 con 0x7f59440d02c0 2023-08-27T11:30:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.663+0000 7f5948f82700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 0x7f5944151470 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f5934000c00 tx=0x7f5934004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.663+0000 7f5939ffb700 1 -- 172.21.15.45:0/2889322710 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934004400 con 0x7f59440d02c0 2023-08-27T11:30:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.663+0000 7f5939ffb700 1 -- 172.21.15.45:0/2889322710 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593401f070 con 0x7f59440d02c0 2023-08-27T11:30:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.663+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59440b3ec0 con 0x7f59440d02c0 2023-08-27T11:30:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.663+0000 7f5939ffb700 1 -- 172.21.15.45:0/2889322710 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59340038e0 con 0x7f59440d02c0 2023-08-27T11:30:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.663+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5944151cb0 con 0x7f59440d02c0 2023-08-27T11:30:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.664+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5944066650 con 0x7f59440d02c0 2023-08-27T11:30:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.664+0000 7f5939ffb700 1 -- 172.21.15.45:0/2889322710 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f5934021030 con 0x7f59440d02c0 2023-08-27T11:30:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.665+0000 7f5939ffb700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f592806a320 0x7f592806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.665+0000 7f5939ffb700 1 -- 172.21.15.45:0/2889322710 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f593408a040 con 0x7f59440d02c0 2023-08-27T11:30:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.665+0000 7f5949783700 1 -- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f592806a320 msgr2=0x7f592806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.665+0000 7f5949783700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f592806a320 0x7f592806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:12.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.669+0000 7f5939ffb700 1 -- 172.21.15.45:0/2889322710 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59340550f0 con 0x7f59440d02c0 2023-08-27T11:30:12.736 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:12.734+0000 7fb732698200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-27T11:30:12.867 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.866+0000 7f5949783700 1 -- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f592806a320 msgr2=0x7f592806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:12.868 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.866+0000 7f5949783700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f592806a320 0x7f592806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:12.872 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.871+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59440734a0 con 0x7f59440d02c0 2023-08-27T11:30:12.873 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.871+0000 7f5939ffb700 1 -- 172.21.15.45:0/2889322710 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5934058710 con 0x7f59440d02c0 2023-08-27T11:30:12.873 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:30:12.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f592806a320 msgr2=0x7f592806c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:12.875 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f592806a320 0x7f592806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 msgr2=0x7f5944151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:12.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 0x7f5944151470 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f5934000c00 tx=0x7f5934004750 comp rx=0 tx=0).stop 2023-08-27T11:30:12.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 shutdown_connections 2023-08-27T11:30:12.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f594415d3a0 0x7f59440b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.876 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59440b0320 0x7f59440b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f592806a320 0x7f592806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 --2- 172.21.15.45:0/2889322710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59440d02c0 0x7f5944151470 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:12.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 >> 172.21.15.45:0/2889322710 conn(0x7f594409a1a0 msgr2=0x7f594406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:12.877 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 shutdown_connections 2023-08-27T11:30:12.878 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:12.873+0000 7f594b9e7700 1 -- 172.21.15.45:0/2889322710 wait complete. 2023-08-27T11:30:12.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:30:13.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.018+0000 7fa24f343700 1 Processor -- start 2023-08-27T11:30:13.020 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.018+0000 7fa24f343700 1 -- start start 2023-08-27T11:30:13.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.019+0000 7fa24f343700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 0x7fa2480cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.019+0000 7fa24f343700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2480cdf40 con 0x7fa248160890 2023-08-27T11:30:13.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.019+0000 7fa24f343700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2480d0fd0 con 0x7fa2480b5220 2023-08-27T11:30:13.023 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.019+0000 7fa24f343700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2480ce190 con 0x7fa2480b62c0 2023-08-27T11:30:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.019+0000 7fa24d0df700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa248160890 0x7fa2480b4b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47848/0 (socket says 172.21.15.45:47848) 2023-08-27T11:30:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.019+0000 7fa24d0df700 1 -- 172.21.15.45:0/1861870648 learned_addr learned my addr 172.21.15.45:0/1861870648 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa24d8e0700 1 --2- 172.21.15.45:0/1861870648 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 0x7fa2480cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa23ffff700 1 -- 172.21.15.45:0/1861870648 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988692589 0 0) 0x7fa2480cdf40 con 0x7fa248160890 2023-08-27T11:30:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa23ffff700 1 -- 172.21.15.45:0/1861870648 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2340035b0 con 0x7fa248160890 2023-08-27T11:30:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa23ffff700 1 -- 172.21.15.45:0/1861870648 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765266484 0 0) 0x7fa2480d0fd0 con 0x7fa2480b5220 2023-08-27T11:30:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa23ffff700 1 -- 172.21.15.45:0/1861870648 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2480cdf40 con 0x7fa2480b5220 2023-08-27T11:30:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa24d8e0700 1 -- 172.21.15.45:0/1861870648 >> v1:172.21.15.135:6789/0 conn(0x7fa2480b5220 legacy=0x7fa2480d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa24d8e0700 1 -- 172.21.15.45:0/1861870648 >> v1:172.21.15.45:6789/0 conn(0x7fa248160890 legacy=0x7fa2480b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa24d8e0700 1 -- 172.21.15.45:0/1861870648 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2480ce470 con 0x7fa2480b62c0 2023-08-27T11:30:13.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa24d8e0700 1 --2- 172.21.15.45:0/1861870648 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 0x7fa2480cd8f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa240009dc0 tx=0x7fa24000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49003373b4644b5d server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:13.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa23ffff700 1 -- 172.21.15.45:0/1861870648 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000e070 con 0x7fa2480b62c0 2023-08-27T11:30:13.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa23ffff700 1 -- 172.21.15.45:0/1861870648 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24000bb20 con 0x7fa2480b62c0 2023-08-27T11:30:13.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.020+0000 7fa23ffff700 1 -- 172.21.15.45:0/1861870648 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240012430 con 0x7fa2480b62c0 2023-08-27T11:30:13.026 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.021+0000 7fa24f343700 1 -- 172.21.15.45:0/1861870648 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 msgr2=0x7fa2480cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.021+0000 7fa24f343700 1 --2- 172.21.15.45:0/1861870648 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 0x7fa2480cd8f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa240009dc0 tx=0x7fa24000b330 comp rx=0 tx=0).stop 2023-08-27T11:30:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.021+0000 7fa24f343700 1 -- 172.21.15.45:0/1861870648 shutdown_connections 2023-08-27T11:30:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.021+0000 7fa24f343700 1 --2- 172.21.15.45:0/1861870648 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 0x7fa2480cd8f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.021+0000 7fa24f343700 1 -- 172.21.15.45:0/1861870648 >> 172.21.15.45:0/1861870648 conn(0x7fa2480cfb10 msgr2=0x7fa2480cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.021+0000 7fa24f343700 1 -- 172.21.15.45:0/1861870648 shutdown_connections 2023-08-27T11:30:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.021+0000 7fa24f343700 1 -- 172.21.15.45:0/1861870648 wait complete. 2023-08-27T11:30:13.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.023+0000 7fa24f343700 1 Processor -- start 2023-08-27T11:30:13.030 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.023+0000 7fa24f343700 1 -- start start 2023-08-27T11:30:13.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24f343700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2480b5220 0x7fa248157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24f343700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 0x7fa2480a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24f343700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa248160890 0x7fa2480cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.031 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24f343700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2481f6b50 con 0x7fa2480b5220 2023-08-27T11:30:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24f343700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2481f6e30 con 0x7fa248160890 2023-08-27T11:30:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24f343700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2481f7110 con 0x7fa2480b62c0 2023-08-27T11:30:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d0df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2480b5220 0x7fa248157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d0df700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2480b5220 0x7fa248157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32844/0 (socket says 172.21.15.45:32844) 2023-08-27T11:30:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d0df700 1 -- 172.21.15.45:0/1129834177 learned_addr learned my addr 172.21.15.45:0/1129834177 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d8e0700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa248160890 0x7fa2480cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24c8de700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 0x7fa2480a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d0df700 1 -- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 msgr2=0x7fa2480a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d0df700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 0x7fa2480a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d0df700 1 -- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa248160890 msgr2=0x7fa2480cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d0df700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa248160890 0x7fa2480cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d0df700 1 -- 172.21.15.45:0/1129834177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa240009a70 con 0x7fa2480b5220 2023-08-27T11:30:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.024+0000 7fa24d0df700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2480b5220 0x7fa248157d70 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa244001eb0 tx=0x7fa24400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.025+0000 7fa23dffb700 1 -- 172.21.15.45:0/1129834177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244005d70 con 0x7fa2480b5220 2023-08-27T11:30:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.025+0000 7fa23dffb700 1 -- 172.21.15.45:0/1129834177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2440070d0 con 0x7fa2480b5220 2023-08-27T11:30:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.025+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2481f73f0 con 0x7fa2480b5220 2023-08-27T11:30:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.025+0000 7fa23dffb700 1 -- 172.21.15.45:0/1129834177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244007440 con 0x7fa2480b5220 2023-08-27T11:30:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.025+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2480aa280 con 0x7fa2480b5220 2023-08-27T11:30:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.026+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2480aa4a0 con 0x7fa2480b5220 2023-08-27T11:30:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.026+0000 7fa23dffb700 1 -- 172.21.15.45:0/1129834177 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fa2440075a0 con 0x7fa2480b5220 2023-08-27T11:30:13.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.027+0000 7fa23dffb700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa22c06a320 0x7fa22c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.027+0000 7fa24c8de700 1 -- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa22c06a320 msgr2=0x7fa22c06c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:13.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.027+0000 7fa23dffb700 1 -- 172.21.15.45:0/1129834177 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa244087e70 con 0x7fa2480b5220 2023-08-27T11:30:13.036 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.027+0000 7fa24c8de700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa22c06a320 0x7fa22c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:13.037 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.031+0000 7fa23dffb700 1 -- 172.21.15.45:0/1129834177 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa244053050 con 0x7fa2480b5220 2023-08-27T11:30:13.196 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:13.195+0000 7fcb8ec68200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:30:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.228+0000 7fa24c8de700 1 -- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa22c06a320 msgr2=0x7fa22c06c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.228+0000 7fa24c8de700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa22c06a320 0x7fa22c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:13.266 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:13.265+0000 7fcb8ec68200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:30:13.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.301+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa24806c120 con 0x7fa2480b5220 2023-08-27T11:30:13.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.302+0000 7fa23dffb700 1 -- 172.21.15.45:0/1129834177 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa244056670 con 0x7fa2480b5220 2023-08-27T11:30:13.304 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:30:13.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa22c06a320 msgr2=0x7fa22c06c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:13.306 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa22c06a320 0x7fa22c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2480b5220 msgr2=0x7fa248157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.307 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2480b5220 0x7fa248157d70 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa244001eb0 tx=0x7fa24400c7b0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 shutdown_connections 2023-08-27T11:30:13.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fa22c06a320 0x7fa22c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2480b5220 0x7fa248157d70 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa248160890 0x7fa2480cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 --2- 172.21.15.45:0/1129834177 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fa2480b62c0 0x7fa2480a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 >> 172.21.15.45:0/1129834177 conn(0x7fa2480cfb10 msgr2=0x7fa2480c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:13.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 shutdown_connections 2023-08-27T11:30:13.309 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.304+0000 7fa24f343700 1 -- 172.21.15.45:0/1129834177 wait complete. 2023-08-27T11:30:13.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-27T11:30:13.372 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:13.370+0000 7fb732698200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-27T11:30:13.448 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:13.446+0000 7fb732698200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-27T11:30:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.447+0000 7f2590596700 1 Processor -- start 2023-08-27T11:30:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.447+0000 7f2590596700 1 -- start start 2023-08-27T11:30:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.448+0000 7f2590596700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25880b5220 0x7f25880cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.448+0000 7f2590596700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25880b62c0 0x7f25880d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.448+0000 7f2590596700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25880cdf40 con 0x7f2588160890 2023-08-27T11:30:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.448+0000 7f2590596700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25880d0fd0 con 0x7f25880b5220 2023-08-27T11:30:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.448+0000 7f2590596700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25880ce190 con 0x7f25880b62c0 2023-08-27T11:30:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258db31700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25880b5220 0x7f25880cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258db31700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25880b5220 0x7f25880cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32856/0 (socket says 172.21.15.45:32856) 2023-08-27T11:30:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258db31700 1 -- 172.21.15.45:0/612347278 learned_addr learned my addr 172.21.15.45:0/612347278 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258eb33700 1 --2- 172.21.15.45:0/612347278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25880b62c0 0x7f25880d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258db31700 1 -- 172.21.15.45:0/612347278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25880b62c0 msgr2=0x7f25880d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258db31700 1 --2- 172.21.15.45:0/612347278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25880b62c0 0x7f25880d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258db31700 1 -- 172.21.15.45:0/612347278 >> v1:172.21.15.135:6789/0 conn(0x7f2588160890 legacy=0x7f25880b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258db31700 1 -- 172.21.15.45:0/612347278 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25880ce470 con 0x7f25880b5220 2023-08-27T11:30:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258eb33700 1 --2- 172.21.15.45:0/612347278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25880b62c0 0x7f25880d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258db31700 1 --2- 172.21.15.45:0/612347278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b5220 0x7f25880cd8f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f25780098f0 tx=0x7f2578009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1259d30b45b1ef server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258d330700 1 -- 172.21.15.45:0/612347278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800b8a0 con 0x7f25880b5220 2023-08-27T11:30:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258d330700 1 -- 172.21.15.45:0/612347278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257800ba00 con 0x7f25880b5220 2023-08-27T11:30:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.449+0000 7f258d330700 1 -- 172.21.15.45:0/612347278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800bd90 con 0x7f25880b5220 2023-08-27T11:30:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.450+0000 7f2590596700 1 -- 172.21.15.45:0/612347278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b5220 msgr2=0x7f25880cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.450+0000 7f2590596700 1 --2- 172.21.15.45:0/612347278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b5220 0x7f25880cd8f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f25780098f0 tx=0x7f2578009c00 comp rx=0 tx=0).stop 2023-08-27T11:30:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.450+0000 7f2590596700 1 -- 172.21.15.45:0/612347278 shutdown_connections 2023-08-27T11:30:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.450+0000 7f2590596700 1 --2- 172.21.15.45:0/612347278 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f25880b62c0 0x7f25880d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.450+0000 7f2590596700 1 --2- 172.21.15.45:0/612347278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b5220 0x7f25880cd8f0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.450+0000 7f2590596700 1 -- 172.21.15.45:0/612347278 >> 172.21.15.45:0/612347278 conn(0x7f25880cfb10 msgr2=0x7f25880cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.450+0000 7f2590596700 1 -- 172.21.15.45:0/612347278 shutdown_connections 2023-08-27T11:30:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.450+0000 7f2590596700 1 -- 172.21.15.45:0/612347278 wait complete. 2023-08-27T11:30:13.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.451+0000 7f2590596700 1 Processor -- start 2023-08-27T11:30:13.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.451+0000 7f2590596700 1 -- start start 2023-08-27T11:30:13.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f2590596700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25880b5220 0x7f258815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f2590596700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b62c0 0x7f25881f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.462 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f2590596700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2588160890 0x7f25881f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f2590596700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258815f080 con 0x7f25880b62c0 2023-08-27T11:30:13.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f2590596700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25881f3570 con 0x7f25880b5220 2023-08-27T11:30:13.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f2590596700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25881f3850 con 0x7f2588160890 2023-08-27T11:30:13.463 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258db31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b62c0 0x7f25881f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:13.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258db31700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b62c0 0x7f25881f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32862/0 (socket says 172.21.15.45:32862) 2023-08-27T11:30:13.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258e332700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25880b5220 0x7f258815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:13.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258db31700 1 -- 172.21.15.45:0/3816574386 learned_addr learned my addr 172.21.15.45:0/3816574386 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:13.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258eb33700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2588160890 0x7f25881f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:13.464 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258db31700 1 -- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2588160890 msgr2=0x7f25881f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258db31700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2588160890 0x7f25881f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258db31700 1 -- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25880b5220 msgr2=0x7f258815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258db31700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25880b5220 0x7f258815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.452+0000 7f258db31700 1 -- 172.21.15.45:0/3816574386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578009550 con 0x7f25880b62c0 2023-08-27T11:30:13.465 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.453+0000 7f258db31700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b62c0 0x7f25881f2c80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2578000c00 tx=0x7f2578004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:13.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.453+0000 7f257effd700 1 -- 172.21.15.45:0/3816574386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578004300 con 0x7f25880b62c0 2023-08-27T11:30:13.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.453+0000 7f257effd700 1 -- 172.21.15.45:0/3816574386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257801f070 con 0x7f25880b62c0 2023-08-27T11:30:13.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.453+0000 7f257effd700 1 -- 172.21.15.45:0/3816574386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780036a0 con 0x7f25880b62c0 2023-08-27T11:30:13.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.453+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25881f3b30 con 0x7f25880b62c0 2023-08-27T11:30:13.466 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.453+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25881f31c0 con 0x7f25880b62c0 2023-08-27T11:30:13.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.454+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2588158c50 con 0x7f25880b62c0 2023-08-27T11:30:13.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.454+0000 7f257effd700 1 -- 172.21.15.45:0/3816574386 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f2578021030 con 0x7f25880b62c0 2023-08-27T11:30:13.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.455+0000 7f257effd700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f256c06a320 0x7f256c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.455+0000 7f258e332700 1 -- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f256c06a320 msgr2=0x7f256c06c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:13.467 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.455+0000 7f257effd700 1 -- 172.21.15.45:0/3816574386 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2578089f80 con 0x7f25880b62c0 2023-08-27T11:30:13.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.455+0000 7f258e332700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f256c06a320 0x7f256c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:13.468 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.459+0000 7f257effd700 1 -- 172.21.15.45:0/3816574386 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f25780bc990 con 0x7f25880b62c0 2023-08-27T11:30:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.656+0000 7f258e332700 1 -- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f256c06a320 msgr2=0x7f256c06c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.656+0000 7f258e332700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f256c06a320 0x7f256c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:13.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.680+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f258806c120 con 0x7f25880b62c0 2023-08-27T11:30:13.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.680+0000 7f257effd700 1 -- 172.21.15.45:0/3816574386 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 9 entries v117) v1 ==== 80+0+534 (secure 0 0 0) 0x7f2588158c50 con 0x7f25880b62c0 2023-08-27T11:30:13.682 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/3391794165 2023-08-28T11:29:08.294424+0000 2023-08-27T11:30:13.683 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/2740998365 2023-08-28T11:29:08.294424+0000 2023-08-27T11:30:13.683 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/1494860999 2023-08-28T11:29:08.294424+0000 2023-08-27T11:30:13.683 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/91399608 2023-08-28T11:29:08.294424+0000 2023-08-27T11:30:13.683 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6833/3759624346 2023-08-28T11:29:08.294424+0000 2023-08-27T11:30:13.683 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:6832/3759624346 2023-08-28T11:29:08.294424+0000 2023-08-27T11:30:13.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/4148146534 2023-08-28T11:29:08.294424+0000 2023-08-27T11:30:13.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/3025460570 2023-08-28T11:29:08.294424+0000 2023-08-27T11:30:13.684 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.135:0/626141668 2023-08-27T12:28:58.549630+0000 2023-08-27T11:30:13.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f256c06a320 msgr2=0x7f256c06c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:13.685 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f256c06a320 0x7f256c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b62c0 msgr2=0x7f25881f2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b62c0 0x7f25881f2c80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2578000c00 tx=0x7f2578004750 comp rx=0 tx=0).stop 2023-08-27T11:30:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 shutdown_connections 2023-08-27T11:30:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f2588160890 0x7f25881f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.686 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25880b5220 0x7f258815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f256c06a320 0x7f256c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 --2- 172.21.15.45:0/3816574386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25880b62c0 0x7f25881f2c80 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 >> 172.21.15.45:0/3816574386 conn(0x7f25880cfb10 msgr2=0x7f25880d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 shutdown_connections 2023-08-27T11:30:13.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.683+0000 7f2590596700 1 -- 172.21.15.45:0/3816574386 wait complete. 2023-08-27T11:30:13.688 INFO:teuthology.orchestra.run.smithi045.stderr:listed 9 entries 2023-08-27T11:30:13.697 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-27T11:30:13.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:30:13.697 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-27T11:30:13.751 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2023-08-27T11:30:13.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-27T11:30:13.752 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-27T11:30:13.769 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:30:13.770 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:30:13.788 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:30:13.788 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-08-27T11:30:13.788 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:30:13.827 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16889 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-27T11:30:13.827 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-27T11:30:13.828 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-27T11:30:13.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-27T11:30:13.848 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:30:13.898 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:13.897+0000 7fcb8ec68200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:30:13.976 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:13.975+0000 7fcb8ec68200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:30:13.996 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.995+0000 7ff94f486700 1 Processor -- start 2023-08-27T11:30:13.997 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.995+0000 7ff94f486700 1 -- start start 2023-08-27T11:30:13.998 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94f486700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff94806ac60 0x7ff94806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:13.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94f486700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9480cc5c0 con 0x7ff94806d3a0 2023-08-27T11:30:13.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94f486700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9480cbd80 con 0x7ff948158cb0 2023-08-27T11:30:13.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94f486700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94806b230 con 0x7ff94806ac60 2023-08-27T11:30:13.999 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94ca21700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff94806ac60 0x7ff94806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94da23700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff94806d3a0 0x7ff94815be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47882/0 (socket says 172.21.15.45:47882) 2023-08-27T11:30:14.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94ca21700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff94806ac60 0x7ff94806ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32870/0 (socket says 172.21.15.45:32870) 2023-08-27T11:30:14.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94da23700 1 -- 172.21.15.45:0/2414128704 learned_addr learned my addr 172.21.15.45:0/2414128704 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:14.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94ca21700 1 -- 172.21.15.45:0/2414128704 >> v1:172.21.15.135:6789/0 conn(0x7ff948158cb0 legacy=0x7ff9480ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.000 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94ca21700 1 -- 172.21.15.45:0/2414128704 >> v1:172.21.15.45:6789/0 conn(0x7ff94806d3a0 legacy=0x7ff94815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff94ca21700 1 -- 172.21.15.45:0/2414128704 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480cc000 con 0x7ff94806ac60 2023-08-27T11:30:14.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.996+0000 7ff93ffff700 1 -- 172.21.15.45:0/2414128704 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061651091 0 0) 0x7ff9480cc5c0 con 0x7ff94806d3a0 2023-08-27T11:30:14.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.997+0000 7ff94ca21700 1 --2- 172.21.15.45:0/2414128704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806ac60 0x7ff94806ce60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff938009840 tx=0x7ff938009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738f4e013c590fb7 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:14.001 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.997+0000 7ff93ffff700 1 -- 172.21.15.45:0/2414128704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938011830 con 0x7ff94806ac60 2023-08-27T11:30:14.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.997+0000 7ff93ffff700 1 -- 172.21.15.45:0/2414128704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff938011990 con 0x7ff94806ac60 2023-08-27T11:30:14.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.997+0000 7ff93ffff700 1 -- 172.21.15.45:0/2414128704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938011d20 con 0x7ff94806ac60 2023-08-27T11:30:14.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.997+0000 7ff94f486700 1 -- 172.21.15.45:0/2414128704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806ac60 msgr2=0x7ff94806ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.997+0000 7ff94f486700 1 --2- 172.21.15.45:0/2414128704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806ac60 0x7ff94806ce60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff938009840 tx=0x7ff938009b50 comp rx=0 tx=0).stop 2023-08-27T11:30:14.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.998+0000 7ff94f486700 1 -- 172.21.15.45:0/2414128704 shutdown_connections 2023-08-27T11:30:14.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.998+0000 7ff94f486700 1 --2- 172.21.15.45:0/2414128704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806ac60 0x7ff94806ce60 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.005 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.998+0000 7ff94f486700 1 -- 172.21.15.45:0/2414128704 >> 172.21.15.45:0/2414128704 conn(0x7ff9480cfd60 msgr2=0x7ff9480d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:14.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.998+0000 7ff94f486700 1 -- 172.21.15.45:0/2414128704 shutdown_connections 2023-08-27T11:30:14.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.998+0000 7ff94f486700 1 -- 172.21.15.45:0/2414128704 wait complete. 2023-08-27T11:30:14.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.999+0000 7ff94f486700 1 Processor -- start 2023-08-27T11:30:14.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:13.999+0000 7ff94f486700 1 -- start start 2023-08-27T11:30:14.006 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94f486700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff94806ac60 0x7ff9480c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94f486700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806d3a0 0x7ff9481f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94f486700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff948158cb0 0x7ff9480ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94f486700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480adfc0 con 0x7ff94806d3a0 2023-08-27T11:30:14.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94f486700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480cdaf0 con 0x7ff948158cb0 2023-08-27T11:30:14.007 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94f486700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480ad760 con 0x7ff94806ac60 2023-08-27T11:30:14.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94ca21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806d3a0 0x7ff9481f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94ca21700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806d3a0 0x7ff9481f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32878/0 (socket says 172.21.15.45:32878) 2023-08-27T11:30:14.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94ca21700 1 -- 172.21.15.45:0/1492473987 learned_addr learned my addr 172.21.15.45:0/1492473987 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:14.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94da23700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff948158cb0 0x7ff9480ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.008 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94d222700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff94806ac60 0x7ff9480c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94ca21700 1 -- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff94806ac60 msgr2=0x7ff9480c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94ca21700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff94806ac60 0x7ff9480c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94ca21700 1 -- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff948158cb0 msgr2=0x7ff9480ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94ca21700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff948158cb0 0x7ff9480ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.009 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94ca21700 1 -- 172.21.15.45:0/1492473987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380094a0 con 0x7ff94806d3a0 2023-08-27T11:30:14.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff94ca21700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806d3a0 0x7ff9481f2340 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff9380090b0 tx=0x7ff938004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:14.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.000+0000 7ff93dffb700 1 -- 172.21.15.45:0/1492473987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93801a7e0 con 0x7ff94806d3a0 2023-08-27T11:30:14.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.001+0000 7ff93dffb700 1 -- 172.21.15.45:0/1492473987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93801a940 con 0x7ff94806d3a0 2023-08-27T11:30:14.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.001+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff948159210 con 0x7ff94806d3a0 2023-08-27T11:30:14.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.001+0000 7ff93dffb700 1 -- 172.21.15.45:0/1492473987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93801acb0 con 0x7ff94806d3a0 2023-08-27T11:30:14.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.001+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9480adce0 con 0x7ff94806d3a0 2023-08-27T11:30:14.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.002+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff948072780 con 0x7ff94806d3a0 2023-08-27T11:30:14.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.002+0000 7ff93dffb700 1 -- 172.21.15.45:0/1492473987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7ff938022070 con 0x7ff94806d3a0 2023-08-27T11:30:14.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.003+0000 7ff93dffb700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7ff92c06a320 0x7ff92c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.003+0000 7ff94d222700 1 -- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7ff92c06a320 msgr2=0x7ff92c06c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:14.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.003+0000 7ff93dffb700 1 -- 172.21.15.45:0/1492473987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff938089e80 con 0x7ff94806d3a0 2023-08-27T11:30:14.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.003+0000 7ff94d222700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7ff92c06a320 0x7ff92c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:14.012 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.007+0000 7ff93dffb700 1 -- 172.21.15.45:0/1492473987 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff938055180 con 0x7ff94806d3a0 2023-08-27T11:30:14.121 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:14.120+0000 7fcb8ec68200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:30:14.128 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:14.127+0000 7fb732698200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:30:14.205 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.204+0000 7ff94d222700 1 -- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7ff92c06a320 msgr2=0x7ff92c06c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:14.206 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.204+0000 7ff94d222700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7ff92c06a320 0x7ff92c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:14.211 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:14.209+0000 7fb732698200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-27T11:30:14.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.257+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9480734a0 con 0x7ff94806d3a0 2023-08-27T11:30:14.259 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.257+0000 7ff93dffb700 1 -- 172.21.15.45:0/1492473987 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4874 (secure 0 0 0) 0x7ff938015090 con 0x7ff94806d3a0 2023-08-27T11:30:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:30:14.260 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7402,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2571849064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2571849064},{"type":"v1","addr":"172.21.15.135:6836","nonce":2571849064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/509114843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":509114843},{"type":"v1","addr":"172.21.15.135:6837","nonce":509114843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/528899713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":528899713},{"type":"v1","addr":"172.21.15.45:6833","nonce":528899713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.511894+0000","modified":"2023-08-27T11:28:53.106978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8019":{"gid":8019,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/343785163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":343785163},{"type":"v1","addr":"172.21.15.45:6835","nonce":343785163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:30:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7ff92c06a320 msgr2=0x7ff92c06c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7ff92c06a320 0x7ff92c06c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806d3a0 msgr2=0x7ff9481f2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806d3a0 0x7ff9481f2340 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff9380090b0 tx=0x7ff938004750 comp rx=0 tx=0).stop 2023-08-27T11:30:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 shutdown_connections 2023-08-27T11:30:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff948158cb0 0x7ff9480ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff94806ac60 0x7ff9480c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7ff92c06a320 0x7ff92c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 --2- 172.21.15.45:0/1492473987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff94806d3a0 0x7ff9481f2340 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 >> 172.21.15.45:0/1492473987 conn(0x7ff9480cfd60 msgr2=0x7ff94806d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 shutdown_connections 2023-08-27T11:30:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.260+0000 7ff94f486700 1 -- 172.21.15.45:0/1492473987 wait complete. 2023-08-27T11:30:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 49 2023-08-27T11:30:14.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:30:14.350 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:14.349+0000 7fcb8ec68200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:30:14.369 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:14.368+0000 7fb732698200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-27T11:30:14.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.402+0000 7fe706fad700 1 Processor -- start 2023-08-27T11:30:14.404 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.402+0000 7fe706fad700 1 -- start start 2023-08-27T11:30:14.412 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.403+0000 7fe706fad700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe7000d1a20 0x7fe7000cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.403+0000 7fe706fad700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7000cd9b0 0x7fe7000b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.413 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.403+0000 7fe706fad700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7000ada50 con 0x7fe70015b120 2023-08-27T11:30:14.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.403+0000 7fe706fad700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7000cde40 con 0x7fe7000d1a20 2023-08-27T11:30:14.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.403+0000 7fe706fad700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000ccb40 con 0x7fe7000cd9b0 2023-08-27T11:30:14.414 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.403+0000 7fe6fffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe7000d1a20 0x7fe7000cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6fffff700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe7000d1a20 0x7fe7000cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32908/0 (socket says 172.21.15.45:32908) 2023-08-27T11:30:14.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6fffff700 1 -- 172.21.15.45:0/3490263389 learned_addr learned my addr 172.21.15.45:0/3490263389 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:14.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe704d49700 1 --1- 172.21.15.45:0/3490263389 >> v1:172.21.15.45:6789/0 conn(0x7fe70015b120 0x7fe7000cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47914/0 (socket says 172.21.15.45:47914) 2023-08-27T11:30:14.415 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe70554a700 1 --2- 172.21.15.45:0/3490263389 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7000cd9b0 0x7fe7000b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6fffff700 1 -- 172.21.15.45:0/3490263389 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7000cd9b0 msgr2=0x7fe7000b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6fffff700 1 --2- 172.21.15.45:0/3490263389 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7000cd9b0 0x7fe7000b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6fffff700 1 -- 172.21.15.45:0/3490263389 >> v1:172.21.15.45:6789/0 conn(0x7fe70015b120 legacy=0x7fe7000cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6ff7fe700 1 -- 172.21.15.45:0/3490263389 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173147135 0 0) 0x7fe7000ada50 con 0x7fe70015b120 2023-08-27T11:30:14.416 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6fffff700 1 -- 172.21.15.45:0/3490263389 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000ccd60 con 0x7fe7000d1a20 2023-08-27T11:30:14.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe70554a700 1 --2- 172.21.15.45:0/3490263389 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7000cd9b0 0x7fe7000b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:14.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6fffff700 1 --2- 172.21.15.45:0/3490263389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 0x7fe7000cd470 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009920 tx=0x7fe6ec009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5b2d4030859175 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:14.417 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6ff7fe700 1 -- 172.21.15.45:0/3490263389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec00ba20 con 0x7fe7000d1a20 2023-08-27T11:30:14.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6ff7fe700 1 -- 172.21.15.45:0/3490263389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec00bb80 con 0x7fe7000d1a20 2023-08-27T11:30:14.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.404+0000 7fe6ff7fe700 1 -- 172.21.15.45:0/3490263389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec009e70 con 0x7fe7000d1a20 2023-08-27T11:30:14.418 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.405+0000 7fe706fad700 1 -- 172.21.15.45:0/3490263389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 msgr2=0x7fe7000cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.405+0000 7fe706fad700 1 --2- 172.21.15.45:0/3490263389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 0x7fe7000cd470 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009920 tx=0x7fe6ec009c30 comp rx=0 tx=0).stop 2023-08-27T11:30:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.405+0000 7fe706fad700 1 -- 172.21.15.45:0/3490263389 shutdown_connections 2023-08-27T11:30:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.405+0000 7fe706fad700 1 --2- 172.21.15.45:0/3490263389 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe7000cd9b0 0x7fe7000b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.405+0000 7fe706fad700 1 --2- 172.21.15.45:0/3490263389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 0x7fe7000cd470 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.419 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.405+0000 7fe706fad700 1 -- 172.21.15.45:0/3490263389 >> 172.21.15.45:0/3490263389 conn(0x7fe70009a160 msgr2=0x7fe7000d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.405+0000 7fe706fad700 1 -- 172.21.15.45:0/3490263389 shutdown_connections 2023-08-27T11:30:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.405+0000 7fe706fad700 1 -- 172.21.15.45:0/3490263389 wait complete. 2023-08-27T11:30:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.406+0000 7fe706fad700 1 Processor -- start 2023-08-27T11:30:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.406+0000 7fe706fad700 1 -- start start 2023-08-27T11:30:14.420 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe706fad700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7000cd9b0 0x7fe700159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe706fad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 0x7fe70015b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe706fad700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe70015b120 0x7fe70014ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe706fad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70015c8f0 con 0x7fe7000d1a20 2023-08-27T11:30:14.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe706fad700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe700159b60 con 0x7fe7000cd9b0 2023-08-27T11:30:14.421 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe706fad700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70015ba40 con 0x7fe70015b120 2023-08-27T11:30:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 0x7fe70015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe6fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 0x7fe70015b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32914/0 (socket says 172.21.15.45:32914) 2023-08-27T11:30:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe6fffff700 1 -- 172.21.15.45:0/599609748 learned_addr learned my addr 172.21.15.45:0/599609748 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe70554a700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe70015b120 0x7fe70014ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.407+0000 7fe704d49700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7000cd9b0 0x7fe700159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.422 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe6fffff700 1 -- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe70015b120 msgr2=0x7fe70014ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe6fffff700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe70015b120 0x7fe70014ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe6fffff700 1 -- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7000cd9b0 msgr2=0x7fe700159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe6fffff700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7000cd9b0 0x7fe700159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe6fffff700 1 -- 172.21.15.45:0/599609748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec009550 con 0x7fe7000d1a20 2023-08-27T11:30:14.423 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe6fffff700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 0x7fe70015b500 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec000c00 tx=0x7fe6ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe6fd7fa700 1 -- 172.21.15.45:0/599609748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec004300 con 0x7fe7000d1a20 2023-08-27T11:30:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe6fd7fa700 1 -- 172.21.15.45:0/599609748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec01f070 con 0x7fe7000d1a20 2023-08-27T11:30:14.424 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70014f460 con 0x7fe7000d1a20 2023-08-27T11:30:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe6fd7fa700 1 -- 172.21.15.45:0/599609748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0036a0 con 0x7fe7000d1a20 2023-08-27T11:30:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.408+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70015bf50 con 0x7fe7000d1a20 2023-08-27T11:30:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.409+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70015ca30 con 0x7fe7000d1a20 2023-08-27T11:30:14.425 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.409+0000 7fe6fd7fa700 1 -- 172.21.15.45:0/599609748 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fe6ec021030 con 0x7fe7000d1a20 2023-08-27T11:30:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.410+0000 7fe6fd7fa700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fe6e406a320 0x7fe6e406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.410+0000 7fe704d49700 1 -- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fe6e406a320 msgr2=0x7fe6e406c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.410+0000 7fe6fd7fa700 1 -- 172.21.15.45:0/599609748 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe6ec08a0f0 con 0x7fe7000d1a20 2023-08-27T11:30:14.426 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.410+0000 7fe704d49700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fe6e406a320 0x7fe6e406c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:14.427 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:14.423+0000 7fcb8ec68200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:30:14.427 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.414+0000 7fe6fd7fa700 1 -- 172.21.15.45:0/599609748 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe70015ca30 con 0x7fe7000d1a20 2023-08-27T11:30:14.612 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.611+0000 7fe704d49700 1 -- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fe6e406a320 msgr2=0x7fe6e406c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:14.613 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.611+0000 7fe704d49700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fe6e406a320 0x7fe6e406c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:14.634 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:14.633+0000 7fb732698200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-27T11:30:14.645 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.644+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe70015ca30 con 0x7fe7000d1a20 2023-08-27T11:30:14.646 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.645+0000 7fe6fd7fa700 1 -- 172.21.15.45:0/599609748 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+19157 (secure 0 0 0) 0x7fe70015ca30 con 0x7fe7000d1a20 2023-08-27T11:30:14.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:30:14.647 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":117,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:29:08.294477+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.834012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.480260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/3391794165":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/2740998365":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/1494860999":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/91399608":"2023-08-28T11:29:08.294424+0000","172.21.15.135:6833/3759624346":"2023-08-28T11:29:08.294424+0000","172.21.15.135:6832/3759624346":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/4148146534":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/3025460570":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/626141668":"2023-08-27T12:28:58.549630+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fe6e406a320 msgr2=0x7fe6e406c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fe6e406a320 0x7fe6e406c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 msgr2=0x7fe70015b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.649 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 0x7fe70015b500 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec000c00 tx=0x7fe6ec004750 comp rx=0 tx=0).stop 2023-08-27T11:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 shutdown_connections 2023-08-27T11:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe70015b120 0x7fe70014ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7000cd9b0 0x7fe700159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.650 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fe6e406a320 0x7fe6e406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 --2- 172.21.15.45:0/599609748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7000d1a20 0x7fe70015b500 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 >> 172.21.15.45:0/599609748 conn(0x7fe70009a160 msgr2=0x7fe7000ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 shutdown_connections 2023-08-27T11:30:14.651 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.647+0000 7fe706fad700 1 -- 172.21.15.45:0/599609748 wait complete. 2023-08-27T11:30:14.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-27T11:11:54.862440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-27T11:12:01.658644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-27T11:28:47.834012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-08-27T11:28:48.480260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-27T11:30:14.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-27T11:30:14.687 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:14.686+0000 7fcb8ec68200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:30:14.716 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:14.715+0000 7fb732698200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-27T11:30:14.793 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.792+0000 7fc9eb2d1700 1 Processor -- start 2023-08-27T11:30:14.794 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.792+0000 7fc9eb2d1700 1 -- start start 2023-08-27T11:30:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9eb2d1700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc9e415d3a0 0x7fc9e40d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.795 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9eb2d1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9e40d02c0 0x7fc9e406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9eb2d1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e415d780 con 0x7fc9e40b0320 2023-08-27T11:30:14.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9eb2d1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e40cfe30 con 0x7fc9e40d02c0 2023-08-27T11:30:14.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9eb2d1700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e40cf220 con 0x7fc9e415d3a0 2023-08-27T11:30:14.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9e886c700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9e40d02c0 0x7fc9e406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.796 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9e986e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc9e40b0320 0x7fc9e40cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47932/0 (socket says 172.21.15.45:47932) 2023-08-27T11:30:14.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9e986e700 1 -- 172.21.15.45:0/54546000 learned_addr learned my addr 172.21.15.45:0/54546000 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:14.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9e906d700 1 --2- 172.21.15.45:0/54546000 >> v2:172.21.15.135:3300/0 conn(0x7fc9e415d3a0 0x7fc9e40d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9e886c700 1 -- 172.21.15.45:0/54546000 >> v2:172.21.15.135:3300/0 conn(0x7fc9e415d3a0 msgr2=0x7fc9e40d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:14.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9e886c700 1 --2- 172.21.15.45:0/54546000 >> v2:172.21.15.135:3300/0 conn(0x7fc9e415d3a0 0x7fc9e40d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.797 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9e886c700 1 -- 172.21.15.45:0/54546000 >> v1:172.21.15.45:6789/0 conn(0x7fc9e40b0320 legacy=0x7fc9e40cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9e886c700 1 -- 172.21.15.45:0/54546000 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40cf500 con 0x7fc9e40d02c0 2023-08-27T11:30:14.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9dbfff700 1 -- 172.21.15.45:0/54546000 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542307742 0 0) 0x7fc9dc001490 con 0x7fc9e40b0320 2023-08-27T11:30:14.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.793+0000 7fc9e886c700 1 --2- 172.21.15.45:0/54546000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e40d02c0 0x7fc9e406bda0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40701d0 tx=0x7fc9d4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b69973654fdfb0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:14.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.794+0000 7fc9dbfff700 1 -- 172.21.15.45:0/54546000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4011830 con 0x7fc9e40d02c0 2023-08-27T11:30:14.798 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.794+0000 7fc9dbfff700 1 -- 172.21.15.45:0/54546000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d4011990 con 0x7fc9e40d02c0 2023-08-27T11:30:14.799 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.794+0000 7fc9dbfff700 1 -- 172.21.15.45:0/54546000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4011d20 con 0x7fc9e40d02c0 2023-08-27T11:30:14.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.794+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/54546000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e40d02c0 msgr2=0x7fc9e406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.794+0000 7fc9eb2d1700 1 --2- 172.21.15.45:0/54546000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e40d02c0 0x7fc9e406bda0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40701d0 tx=0x7fc9d4009b20 comp rx=0 tx=0).stop 2023-08-27T11:30:14.802 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.794+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/54546000 shutdown_connections 2023-08-27T11:30:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.794+0000 7fc9eb2d1700 1 --2- 172.21.15.45:0/54546000 >> v2:172.21.15.135:3300/0 conn(0x7fc9e415d3a0 0x7fc9e40d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.794+0000 7fc9eb2d1700 1 --2- 172.21.15.45:0/54546000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e40d02c0 0x7fc9e406bda0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.794+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/54546000 >> 172.21.15.45:0/54546000 conn(0x7fc9e409a1a0 msgr2=0x7fc9e40d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.795+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/54546000 shutdown_connections 2023-08-27T11:30:14.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.795+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/54546000 wait complete. 2023-08-27T11:30:14.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.796+0000 7fc9eb2d1700 1 Processor -- start 2023-08-27T11:30:14.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.796+0000 7fc9eb2d1700 1 -- start start 2023-08-27T11:30:14.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9eb2d1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e40b0320 0x7fc9e40b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.804 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9eb2d1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9e40d02c0 0x7fc9e4142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9eb2d1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e415d3a0 0x7fc9e40b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9eb2d1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40b23a0 con 0x7fc9e415d3a0 2023-08-27T11:30:14.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9eb2d1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4145fb0 con 0x7fc9e40b0320 2023-08-27T11:30:14.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9eb2d1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4156c80 con 0x7fc9e40d02c0 2023-08-27T11:30:14.805 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9e986e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e415d3a0 0x7fc9e40b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9e986e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e415d3a0 0x7fc9e40b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32930/0 (socket says 172.21.15.45:32930) 2023-08-27T11:30:14.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9e986e700 1 -- 172.21.15.45:0/146131967 learned_addr learned my addr 172.21.15.45:0/146131967 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:14.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.797+0000 7fc9e886c700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9e40d02c0 0x7fc9e4142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9e906d700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e40b0320 0x7fc9e40b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:14.806 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9e986e700 1 -- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9e40d02c0 msgr2=0x7fc9e4142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9e986e700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9e40d02c0 0x7fc9e4142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9e986e700 1 -- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e40b0320 msgr2=0x7fc9e40b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:14.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9e986e700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e40b0320 0x7fc9e40b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:14.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9e986e700 1 -- 172.21.15.45:0/146131967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d40094a0 con 0x7fc9e415d3a0 2023-08-27T11:30:14.807 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9e986e700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e415d3a0 0x7fc9e40b27f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00cca0 tx=0x7fc9dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:14.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9d9ffb700 1 -- 172.21.15.45:0/146131967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00ab10 con 0x7fc9e415d3a0 2023-08-27T11:30:14.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9d9ffb700 1 -- 172.21.15.45:0/146131967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc00ac90 con 0x7fc9e415d3a0 2023-08-27T11:30:14.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e4156f60 con 0x7fc9e415d3a0 2023-08-27T11:30:14.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9d9ffb700 1 -- 172.21.15.45:0/146131967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0070d0 con 0x7fc9e415d3a0 2023-08-27T11:30:14.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.798+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e4156260 con 0x7fc9e415d3a0 2023-08-27T11:30:14.808 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.799+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e40b2510 con 0x7fc9e415d3a0 2023-08-27T11:30:14.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.800+0000 7fc9d9ffb700 1 -- 172.21.15.45:0/146131967 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fc9dc007250 con 0x7fc9e415d3a0 2023-08-27T11:30:14.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.800+0000 7fc9d9ffb700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fc9c806a320 0x7fc9c806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:14.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.800+0000 7fc9e906d700 1 -- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fc9c806a320 msgr2=0x7fc9c806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:14.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.800+0000 7fc9d9ffb700 1 -- 172.21.15.45:0/146131967 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc9dc0888f0 con 0x7fc9e415d3a0 2023-08-27T11:30:14.809 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.800+0000 7fc9e906d700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fc9c806a320 0x7fc9c806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:14.810 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:14.805+0000 7fc9d9ffb700 1 -- 172.21.15.45:0/146131967 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc9e40b2510 con 0x7fc9e415d3a0 2023-08-27T11:30:14.908 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:14.907+0000 7fcb8ec68200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:30:14.998 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:14.996+0000 7fb732698200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-27T11:30:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.001+0000 7fc9e906d700 1 -- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fc9c806a320 msgr2=0x7fc9c806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.001+0000 7fc9e906d700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fc9c806a320 0x7fc9c806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:15.010 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.009+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9e40b2510 con 0x7fc9e415d3a0 2023-08-27T11:30:15.011 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.009+0000 7fc9d9ffb700 1 -- 172.21.15.45:0/146131967 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9dc018020 con 0x7fc9e415d3a0 2023-08-27T11:30:15.011 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-08-27T11:30:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fc9c806a320 msgr2=0x7fc9c806c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fc9c806a320 0x7fc9c806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e415d3a0 msgr2=0x7fc9e40b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e415d3a0 0x7fc9e40b27f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00cca0 tx=0x7fc9dc00a060 comp rx=0 tx=0).stop 2023-08-27T11:30:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 shutdown_connections 2023-08-27T11:30:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fc9c806a320 0x7fc9c806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9e415d3a0 0x7fc9e40b27f0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e40b0320 0x7fc9e40b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 --2- 172.21.15.45:0/146131967 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fc9e40d02c0 0x7fc9e4142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 >> 172.21.15.45:0/146131967 conn(0x7fc9e409a1a0 msgr2=0x7fc9e40afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:15.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 shutdown_connections 2023-08-27T11:30:15.016 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.011+0000 7fc9eb2d1700 1 -- 172.21.15.45:0/146131967 wait complete. 2023-08-27T11:30:15.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-27T11:30:15.057 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:15.056+0000 7fcb8ec68200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:30:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.154+0000 7f291ba62700 1 Processor -- start 2023-08-27T11:30:15.156 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.154+0000 7f291ba62700 1 -- start start 2023-08-27T11:30:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f291ba62700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f291415d3a0 0x7f29140d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.158 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f291ba62700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291406c4b0 0x7f29140cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f291ba62700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291415d780 con 0x7f29140d02c0 2023-08-27T11:30:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f291ba62700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29140cfe30 con 0x7f291415d3a0 2023-08-27T11:30:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f291ba62700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29140cf220 con 0x7f291406c4b0 2023-08-27T11:30:15.159 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f29197fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f291415d3a0 0x7f29140d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f29197fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f291415d3a0 0x7f29140d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32936/0 (socket says 172.21.15.45:32936) 2023-08-27T11:30:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f29197fe700 1 -- 172.21.15.45:0/1899460186 learned_addr learned my addr 172.21.15.45:0/1899460186 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f2919fff700 1 --2- 172.21.15.45:0/1899460186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291406c4b0 0x7f29140cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f29197fe700 1 -- 172.21.15.45:0/1899460186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291406c4b0 msgr2=0x7f29140cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f29197fe700 1 --2- 172.21.15.45:0/1899460186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291406c4b0 0x7f29140cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.160 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f29197fe700 1 -- 172.21.15.45:0/1899460186 >> v1:172.21.15.135:6789/0 conn(0x7f29140d02c0 legacy=0x7f291406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.155+0000 7f29197fe700 1 -- 172.21.15.45:0/1899460186 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29140cf500 con 0x7f291415d3a0 2023-08-27T11:30:15.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f29197fe700 1 --2- 172.21.15.45:0/1899460186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291415d3a0 0x7f29140d1550 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f29100098f0 tx=0x7f2910009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44569c981356670 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:15.161 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f290bfff700 1 -- 172.21.15.45:0/1899460186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910011830 con 0x7f291415d3a0 2023-08-27T11:30:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f290bfff700 1 -- 172.21.15.45:0/1899460186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2910011990 con 0x7f291415d3a0 2023-08-27T11:30:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f290bfff700 1 -- 172.21.15.45:0/1899460186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910011d20 con 0x7f291415d3a0 2023-08-27T11:30:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f291ba62700 1 -- 172.21.15.45:0/1899460186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291415d3a0 msgr2=0x7f29140d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f291ba62700 1 --2- 172.21.15.45:0/1899460186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291415d3a0 0x7f29140d1550 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f29100098f0 tx=0x7f2910009c00 comp rx=0 tx=0).stop 2023-08-27T11:30:15.162 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f291ba62700 1 -- 172.21.15.45:0/1899460186 shutdown_connections 2023-08-27T11:30:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f291ba62700 1 --2- 172.21.15.45:0/1899460186 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291406c4b0 0x7f29140cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f291ba62700 1 --2- 172.21.15.45:0/1899460186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291415d3a0 0x7f29140d1550 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.156+0000 7f291ba62700 1 -- 172.21.15.45:0/1899460186 >> 172.21.15.45:0/1899460186 conn(0x7f291409a1a0 msgr2=0x7f29140d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:15.163 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.157+0000 7f291ba62700 1 -- 172.21.15.45:0/1899460186 shutdown_connections 2023-08-27T11:30:15.164 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.157+0000 7f291ba62700 1 -- 172.21.15.45:0/1899460186 wait complete. 2023-08-27T11:30:15.166 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.158+0000 7f291ba62700 1 Processor -- start 2023-08-27T11:30:15.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.158+0000 7f291ba62700 1 -- start start 2023-08-27T11:30:15.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.158+0000 7f291ba62700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291406c4b0 0x7f2914158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.167 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.158+0000 7f291ba62700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29140d02c0 0x7f29141558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.158+0000 7f291ba62700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291415d3a0 0x7f29141f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f291ba62700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141569e0 con 0x7f291406c4b0 2023-08-27T11:30:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f291ba62700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141591d0 con 0x7f29140d02c0 2023-08-27T11:30:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f291ba62700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29141594b0 con 0x7f291415d3a0 2023-08-27T11:30:15.168 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f29197fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291406c4b0 0x7f2914158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f29197fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291406c4b0 0x7f2914158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32942/0 (socket says 172.21.15.45:32942) 2023-08-27T11:30:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f29197fe700 1 -- 172.21.15.45:0/3328450377 learned_addr learned my addr 172.21.15.45:0/3328450377 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f2918ffd700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29140d02c0 0x7f29141558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f2919fff700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291415d3a0 0x7f29141f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.169 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f29197fe700 1 -- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291415d3a0 msgr2=0x7f29141f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f29197fe700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291415d3a0 0x7f29141f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f29197fe700 1 -- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29140d02c0 msgr2=0x7f29141558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f29197fe700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29140d02c0 0x7f29141558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f29197fe700 1 -- 172.21.15.45:0/3328450377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910009550 con 0x7f291406c4b0 2023-08-27T11:30:15.170 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f29197fe700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291406c4b0 0x7f2914158b40 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2910009c40 tx=0x7f2910004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f2909ffb700 1 -- 172.21.15.45:0/3328450377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910004510 con 0x7f291406c4b0 2023-08-27T11:30:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f2909ffb700 1 -- 172.21.15.45:0/3328450377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291001e070 con 0x7f291406c4b0 2023-08-27T11:30:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29141e6dc0 con 0x7f291406c4b0 2023-08-27T11:30:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.159+0000 7f2909ffb700 1 -- 172.21.15.45:0/3328450377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29100038e0 con 0x7f291406c4b0 2023-08-27T11:30:15.171 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.160+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2914155e20 con 0x7f291406c4b0 2023-08-27T11:30:15.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.161+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29141e6c20 con 0x7f291406c4b0 2023-08-27T11:30:15.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.161+0000 7f2909ffb700 1 -- 172.21.15.45:0/3328450377 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f2910003a40 con 0x7f291406c4b0 2023-08-27T11:30:15.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.162+0000 7f2909ffb700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f28f806a320 0x7f28f806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.172 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.162+0000 7f2918ffd700 1 -- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f28f806a320 msgr2=0x7f28f806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.162+0000 7f2909ffb700 1 -- 172.21.15.45:0/3328450377 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f29100899b0 con 0x7f291406c4b0 2023-08-27T11:30:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.162+0000 7f2918ffd700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f28f806a320 0x7f28f806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:15.173 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:15.171+0000 7fcb8ec68200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:30:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.166+0000 7f2909ffb700 1 -- 172.21.15.45:0/3328450377 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29141e6c20 con 0x7f291406c4b0 2023-08-27T11:30:15.240 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:15.238+0000 7fb732698200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-27T11:30:15.328 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:15.327+0000 7fcb8ec68200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:30:15.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.363+0000 7f2918ffd700 1 -- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f28f806a320 msgr2=0x7f28f806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:15.364 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.363+0000 7f2918ffd700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f28f806a320 0x7f28f806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:15.369 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.368+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f29141e6c20 con 0x7f291406c4b0 2023-08-27T11:30:15.370 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.368+0000 7f2909ffb700 1 -- 172.21.15.45:0/3328450377 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v117) v1 ==== 92+0+10 (secure 0 0 0) 0x7f29141e6c20 con 0x7f291406c4b0 2023-08-27T11:30:15.370 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 8 2023-08-27T11:30:15.372 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f28f806a320 msgr2=0x7f28f806c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:15.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f28f806a320 0x7f28f806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291406c4b0 msgr2=0x7f2914158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291406c4b0 0x7f2914158b40 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2910009c40 tx=0x7f2910004750 comp rx=0 tx=0).stop 2023-08-27T11:30:15.373 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 shutdown_connections 2023-08-27T11:30:15.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f28f806a320 0x7f28f806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f291406c4b0 0x7f2914158b40 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f291415d3a0 0x7f29141f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 --2- 172.21.15.45:0/3328450377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29140d02c0 0x7f29141558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.374 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 >> 172.21.15.45:0/3328450377 conn(0x7f291409a1a0 msgr2=0x7f291415d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:15.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 shutdown_connections 2023-08-27T11:30:15.375 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.371+0000 7f291ba62700 1 -- 172.21.15.45:0/3328450377 wait complete. 2023-08-27T11:30:15.388 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-27T11:30:15.402 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:15.401+0000 7fb732698200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-27T11:30:15.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.516+0000 7f304ab11700 1 Processor -- start 2023-08-27T11:30:15.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.516+0000 7f304ab11700 1 -- start start 2023-08-27T11:30:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f304ab11700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30440b17c0 0x7f30440b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f304ab11700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f304406ced0 0x7f304406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f304ab11700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304406d860 con 0x7f3044157be0 2023-08-27T11:30:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f304ab11700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3044157fc0 con 0x7f304406ced0 2023-08-27T11:30:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f304ab11700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3044071dc0 con 0x7f30440b17c0 2023-08-27T11:30:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f30490ae700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f304406ced0 0x7f304406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f3043fff700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30440b17c0 0x7f30440b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f30488ad700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3044157be0 0x7f30440716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:43624/0 (socket says 172.21.15.45:43624) 2023-08-27T11:30:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f30488ad700 1 -- 172.21.15.45:0/381652600 learned_addr learned my addr 172.21.15.45:0/381652600 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f30490ae700 1 -- 172.21.15.45:0/381652600 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30440b17c0 msgr2=0x7f30440b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f30490ae700 1 --2- 172.21.15.45:0/381652600 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30440b17c0 0x7f30440b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f30490ae700 1 -- 172.21.15.45:0/381652600 >> v1:172.21.15.135:6789/0 conn(0x7f3044157be0 legacy=0x7f30440716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f30490ae700 1 -- 172.21.15.45:0/381652600 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304415b460 con 0x7f304406ced0 2023-08-27T11:30:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.517+0000 7f3043fff700 1 --2- 172.21.15.45:0/381652600 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30440b17c0 0x7f30440b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f30490ae700 1 --2- 172.21.15.45:0/381652600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f304406ced0 0x7f304406d320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f303c009dc0 tx=0x7f303c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f186abe63ae86e03 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f30437fe700 1 -- 172.21.15.45:0/381652600 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00e070 con 0x7f304406ced0 2023-08-27T11:30:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f30437fe700 1 -- 172.21.15.45:0/381652600 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c00bae0 con 0x7f304406ced0 2023-08-27T11:30:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f30437fe700 1 -- 172.21.15.45:0/381652600 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00be90 con 0x7f304406ced0 2023-08-27T11:30:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f304ab11700 1 -- 172.21.15.45:0/381652600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f304406ced0 msgr2=0x7f304406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f304ab11700 1 --2- 172.21.15.45:0/381652600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f304406ced0 0x7f304406d320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f303c009dc0 tx=0x7f303c00b330 comp rx=0 tx=0).stop 2023-08-27T11:30:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f304ab11700 1 -- 172.21.15.45:0/381652600 shutdown_connections 2023-08-27T11:30:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f304ab11700 1 --2- 172.21.15.45:0/381652600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f304406ced0 0x7f304406d320 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f304ab11700 1 --2- 172.21.15.45:0/381652600 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f30440b17c0 0x7f30440b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f304ab11700 1 -- 172.21.15.45:0/381652600 >> 172.21.15.45:0/381652600 conn(0x7f30440cc7c0 msgr2=0x7f30440ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:15.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f304ab11700 1 -- 172.21.15.45:0/381652600 shutdown_connections 2023-08-27T11:30:15.525 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.518+0000 7f304ab11700 1 -- 172.21.15.45:0/381652600 wait complete. 2023-08-27T11:30:15.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.520+0000 7f304ab11700 1 Processor -- start 2023-08-27T11:30:15.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.520+0000 7f304ab11700 1 -- start start 2023-08-27T11:30:15.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f304ab11700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f304406ced0 0x7f304414c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f304ab11700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30440b17c0 0x7f304415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.528 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f304ab11700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3044157be0 0x7f30440b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f304ab11700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3044155e30 con 0x7f304406ced0 2023-08-27T11:30:15.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f304ab11700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304414d790 con 0x7f30440b17c0 2023-08-27T11:30:15.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f304ab11700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30440c84b0 con 0x7f3044157be0 2023-08-27T11:30:15.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30488ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f304406ced0 0x7f304414c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.529 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30488ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f304406ced0 0x7f304414c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32958/0 (socket says 172.21.15.45:32958) 2023-08-27T11:30:15.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30488ad700 1 -- 172.21.15.45:0/2600812360 learned_addr learned my addr 172.21.15.45:0/2600812360 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:15.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f3043fff700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30440b17c0 0x7f304415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30490ae700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3044157be0 0x7f30440b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30488ad700 1 -- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3044157be0 msgr2=0x7f30440b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.530 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30488ad700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3044157be0 0x7f30440b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30488ad700 1 -- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30440b17c0 msgr2=0x7f304415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30488ad700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30440b17c0 0x7f304415ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30488ad700 1 -- 172.21.15.45:0/2600812360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300094a0 con 0x7f304406ced0 2023-08-27T11:30:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f3043fff700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30440b17c0 0x7f304415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30488ad700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f304406ced0 0x7f304414c410 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f303800c520 tx=0x7f303800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.521+0000 7f30417fa700 1 -- 172.21.15.45:0/2600812360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3038005d70 con 0x7f304406ced0 2023-08-27T11:30:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.522+0000 7f30417fa700 1 -- 172.21.15.45:0/2600812360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30380070d0 con 0x7f304406ced0 2023-08-27T11:30:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.522+0000 7f30417fa700 1 -- 172.21.15.45:0/2600812360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3038007440 con 0x7f304406ced0 2023-08-27T11:30:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.522+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c009a70 con 0x7f304406ced0 2023-08-27T11:30:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.522+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30440c7ef0 con 0x7f304406ced0 2023-08-27T11:30:15.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.523+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304415af30 con 0x7f304406ced0 2023-08-27T11:30:15.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.523+0000 7f30417fa700 1 -- 172.21.15.45:0/2600812360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f30380075a0 con 0x7f304406ced0 2023-08-27T11:30:15.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.524+0000 7f30417fa700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f302806a320 0x7f302806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.524+0000 7f30417fa700 1 -- 172.21.15.45:0/2600812360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3038087cf0 con 0x7f304406ced0 2023-08-27T11:30:15.533 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.524+0000 7f3043fff700 1 -- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f302806a320 msgr2=0x7f302806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.524+0000 7f3043fff700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f302806a320 0x7f302806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.528+0000 7f30417fa700 1 -- 172.21.15.45:0/2600812360 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f303808c070 con 0x7f304406ced0 2023-08-27T11:30:15.535 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:15.528+0000 7fb732698200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-27T11:30:15.592 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:15.591+0000 7fcb8ec68200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:30:15.697 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:15.696+0000 7fb732698200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-27T11:30:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.725+0000 7f3043fff700 1 -- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f302806a320 msgr2=0x7f302806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:15.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.725+0000 7f3043fff700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f302806a320 0x7f302806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:15.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.727+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f304406d860 con 0x7f304406ced0 2023-08-27T11:30:15.729 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.728+0000 7f30417fa700 1 -- 172.21.15.45:0/2600812360 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f304406d860 con 0x7f304406ced0 2023-08-27T11:30:15.730 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2023-08-27T11:30:15.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f302806a320 msgr2=0x7f302806c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:15.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f302806a320 0x7f302806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f304406ced0 msgr2=0x7f304414c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f304406ced0 0x7f304414c410 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f303800c520 tx=0x7f303800c830 comp rx=0 tx=0).stop 2023-08-27T11:30:15.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 shutdown_connections 2023-08-27T11:30:15.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f302806a320 0x7f302806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f304406ced0 0x7f304414c410 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3044157be0 0x7f30440b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 --2- 172.21.15.45:0/2600812360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30440b17c0 0x7f304415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 >> 172.21.15.45:0/2600812360 conn(0x7f30440cc7c0 msgr2=0x7f30440af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:15.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 shutdown_connections 2023-08-27T11:30:15.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.730+0000 7f304ab11700 1 -- 172.21.15.45:0/2600812360 wait complete. 2023-08-27T11:30:15.744 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-27T11:30:15.751 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:15.750+0000 7fcb8ec68200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:30:15.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.916+0000 7fd4acfd0700 1 Processor -- start 2023-08-27T11:30:15.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.916+0000 7fd4acfd0700 1 -- start start 2023-08-27T11:30:15.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.917+0000 7fd4acfd0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a8073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.917+0000 7fd4acfd0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd4a80b2f60 0x7fd4a806b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.917+0000 7fd4acfd0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a8073d40 con 0x7fd4a806b890 2023-08-27T11:30:15.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.917+0000 7fd4acfd0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a80cf360 con 0x7fd4a80b2f60 2023-08-27T11:30:15.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.917+0000 7fd4acfd0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a815a6a0 con 0x7fd4a815a2c0 2023-08-27T11:30:15.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.917+0000 7fd4a6d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd4a806b890 0x7fd4a806bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47980/0 (socket says 172.21.15.45:47980) 2023-08-27T11:30:15.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.917+0000 7fd4a6d9d700 1 -- 172.21.15.45:0/1472328540 learned_addr learned my addr 172.21.15.45:0/1472328540 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:15.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.917+0000 7fd4a659c700 1 --2- 172.21.15.45:0/1472328540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a8073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.917+0000 7fd4a5d9b700 1 --2- 172.21.15.45:0/1472328540 >> v2:172.21.15.135:3300/0 conn(0x7fd4a80b2f60 0x7fd4a806b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a559a700 1 -- 172.21.15.45:0/1472328540 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636755096 0 0) 0x7fd4a8073d40 con 0x7fd4a806b890 2023-08-27T11:30:15.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a559a700 1 -- 172.21.15.45:0/1472328540 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4900035b0 con 0x7fd4a806b890 2023-08-27T11:30:15.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a659c700 1 -- 172.21.15.45:0/1472328540 >> v2:172.21.15.135:3300/0 conn(0x7fd4a80b2f60 msgr2=0x7fd4a806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:15.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a659c700 1 --2- 172.21.15.45:0/1472328540 >> v2:172.21.15.135:3300/0 conn(0x7fd4a80b2f60 0x7fd4a806b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a659c700 1 -- 172.21.15.45:0/1472328540 >> v1:172.21.15.45:6789/0 conn(0x7fd4a806b890 legacy=0x7fd4a806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.923 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a659c700 1 -- 172.21.15.45:0/1472328540 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a80ceda0 con 0x7fd4a815a2c0 2023-08-27T11:30:15.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a659c700 1 --2- 172.21.15.45:0/1472328540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a8073800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009820 tx=0x7fd49c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99230a07b24566f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:15.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a559a700 1 -- 172.21.15.45:0/1472328540 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c005e30 con 0x7fd4a815a2c0 2023-08-27T11:30:15.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a559a700 1 -- 172.21.15.45:0/1472328540 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c006030 con 0x7fd4a815a2c0 2023-08-27T11:30:15.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.918+0000 7fd4a559a700 1 -- 172.21.15.45:0/1472328540 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0063c0 con 0x7fd4a815a2c0 2023-08-27T11:30:15.924 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.919+0000 7fd4acfd0700 1 -- 172.21.15.45:0/1472328540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 msgr2=0x7fd4a8073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.919+0000 7fd4acfd0700 1 --2- 172.21.15.45:0/1472328540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a8073800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009820 tx=0x7fd49c009b30 comp rx=0 tx=0).stop 2023-08-27T11:30:15.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.919+0000 7fd4acfd0700 1 -- 172.21.15.45:0/1472328540 shutdown_connections 2023-08-27T11:30:15.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.919+0000 7fd4acfd0700 1 --2- 172.21.15.45:0/1472328540 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a8073800 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.919+0000 7fd4acfd0700 1 --2- 172.21.15.45:0/1472328540 >> v2:172.21.15.135:3300/0 conn(0x7fd4a80b2f60 0x7fd4a806b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.919+0000 7fd4acfd0700 1 -- 172.21.15.45:0/1472328540 >> 172.21.15.45:0/1472328540 conn(0x7fd4a80b2440 msgr2=0x7fd4a80b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:15.925 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.919+0000 7fd4acfd0700 1 -- 172.21.15.45:0/1472328540 shutdown_connections 2023-08-27T11:30:15.926 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.919+0000 7fd4acfd0700 1 -- 172.21.15.45:0/1472328540 wait complete. 2023-08-27T11:30:15.928 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.920+0000 7fd4acfd0700 1 Processor -- start 2023-08-27T11:30:15.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.920+0000 7fd4acfd0700 1 -- start start 2023-08-27T11:30:15.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4acfd0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a806b890 0x7fd4a81f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4acfd0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80b2f60 0x7fd4a80b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4acfd0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a80aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.929 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4acfd0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a80c7e10 con 0x7fd4a806b890 2023-08-27T11:30:15.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4acfd0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a81f2910 con 0x7fd4a80b2f60 2023-08-27T11:30:15.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4acfd0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a80b15d0 con 0x7fd4a815a2c0 2023-08-27T11:30:15.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a806b890 0x7fd4a81f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a806b890 0x7fd4a81f23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32974/0 (socket says 172.21.15.45:32974) 2023-08-27T11:30:15.930 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4a659c700 1 -- 172.21.15.45:0/713892768 learned_addr learned my addr 172.21.15.45:0/713892768 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:15.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4a6d9d700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a80aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.921+0000 7fd4a5d9b700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80b2f60 0x7fd4a80b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:15.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4a659c700 1 -- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 msgr2=0x7fd4a80aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4a659c700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a80aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.931 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4a659c700 1 -- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80b2f60 msgr2=0x7fd4a80b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4a659c700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80b2f60 0x7fd4a80b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4a659c700 1 -- 172.21.15.45:0/713892768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c009430 con 0x7fd4a806b890 2023-08-27T11:30:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4a6d9d700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a80aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4a5d9b700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80b2f60 0x7fd4a80b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4a659c700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a806b890 0x7fd4a81f23d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd49c000c00 tx=0x7fd49c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd496ffd700 1 -- 172.21.15.45:0/713892768 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0061a0 con 0x7fd4a806b890 2023-08-27T11:30:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd496ffd700 1 -- 172.21.15.45:0/713892768 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c020070 con 0x7fd4a806b890 2023-08-27T11:30:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd496ffd700 1 -- 172.21.15.45:0/713892768 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c006560 con 0x7fd4a806b890 2023-08-27T11:30:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a80b18b0 con 0x7fd4a806b890 2023-08-27T11:30:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.922+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a80c7a30 con 0x7fd4a806b890 2023-08-27T11:30:15.933 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.923+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a80c7f50 con 0x7fd4a806b890 2023-08-27T11:30:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.924+0000 7fd496ffd700 1 -- 172.21.15.45:0/713892768 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fd49c022030 con 0x7fd4a806b890 2023-08-27T11:30:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.925+0000 7fd496ffd700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fd48806a320 0x7fd48806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.925+0000 7fd496ffd700 1 -- 172.21.15.45:0/713892768 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd49c089fe0 con 0x7fd4a806b890 2023-08-27T11:30:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.925+0000 7fd4a5d9b700 1 -- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fd48806a320 msgr2=0x7fd48806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.925+0000 7fd4a5d9b700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fd48806a320 0x7fd48806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:15.929+0000 7fd496ffd700 1 -- 172.21.15.45:0/713892768 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd4a80c7f50 con 0x7fd4a806b890 2023-08-27T11:30:15.982 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:15.980+0000 7fb732698200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-27T11:30:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.126+0000 7fd4a5d9b700 1 -- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fd48806a320 msgr2=0x7fd48806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.126+0000 7fd4a5d9b700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fd48806a320 0x7fd48806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-27T11:30:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.143+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4a80c7f50 con 0x7fd4a806b890 2023-08-27T11:30:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.144+0000 7fd496ffd700 1 -- 172.21.15.45:0/713892768 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4a80c7f50 con 0x7fd4a806b890 2023-08-27T11:30:16.146 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2023-08-27T11:30:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fd48806a320 msgr2=0x7fd48806c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fd48806a320 0x7fd48806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a806b890 msgr2=0x7fd4a81f23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a806b890 0x7fd4a81f23d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd49c000c00 tx=0x7fd49c004750 comp rx=0 tx=0).stop 2023-08-27T11:30:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 shutdown_connections 2023-08-27T11:30:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7fd48806a320 0x7fd48806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a806b890 0x7fd4a81f23d0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fd4a815a2c0 0x7fd4a80aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 --2- 172.21.15.45:0/713892768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80b2f60 0x7fd4a80b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 >> 172.21.15.45:0/713892768 conn(0x7fd4a80b2440 msgr2=0x7fd4a80cad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 shutdown_connections 2023-08-27T11:30:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.146+0000 7fd4acfd0700 1 -- 172.21.15.45:0/713892768 wait complete. 2023-08-27T11:30:16.161 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:30:16.165 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:16.163+0000 7fb732698200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-27T11:30:16.280 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.280+0000 7fcb8ec68200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:30:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.287+0000 7f36cb253700 1 Processor -- start 2023-08-27T11:30:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.287+0000 7f36cb253700 1 -- start start 2023-08-27T11:30:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.288+0000 7f36cb253700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cf980 0x7f36c40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.288+0000 7f36cb253700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f36c40b45d0 0x7f36c40d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.288+0000 7f36cb253700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c40ceae0 con 0x7f36c40cfd60 2023-08-27T11:30:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.288+0000 7f36cb253700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c40cf0a0 con 0x7f36c40b45d0 2023-08-27T11:30:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.288+0000 7f36cb253700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40cedc0 con 0x7f36c40cf980 2023-08-27T11:30:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c3fff700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f36c40b45d0 0x7f36c40d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c8fef700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cf980 0x7f36c40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c97f0700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f36c40cfd60 0x7f36c4161470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:43658/0 (socket says 172.21.15.45:43658) 2023-08-27T11:30:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c97f0700 1 -- 172.21.15.45:0/4075594384 learned_addr learned my addr 172.21.15.45:0/4075594384 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c3fff700 1 -- 172.21.15.45:0/4075594384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cf980 msgr2=0x7f36c40d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c3fff700 1 --2- 172.21.15.45:0/4075594384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cf980 0x7f36c40d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c3fff700 1 -- 172.21.15.45:0/4075594384 >> v1:172.21.15.135:6789/0 conn(0x7f36c40cfd60 legacy=0x7f36c4161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c3fff700 1 -- 172.21.15.45:0/4075594384 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40b22c0 con 0x7f36c40b45d0 2023-08-27T11:30:16.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c37fe700 1 -- 172.21.15.45:0/4075594384 <== mon.1 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74813002 0 0) 0x7f36c40ceae0 con 0x7f36c40cfd60 2023-08-27T11:30:16.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c8fef700 1 --2- 172.21.15.45:0/4075594384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cf980 0x7f36c40d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:16.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c3fff700 1 --2- 172.21.15.45:0/4075594384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c40b45d0 0x7f36c40d1460 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009870 tx=0x7f36b0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2911252ba4faae7f server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:16.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c37fe700 1 -- 172.21.15.45:0/4075594384 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0011830 con 0x7f36c40b45d0 2023-08-27T11:30:16.294 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c37fe700 1 -- 172.21.15.45:0/4075594384 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b0011990 con 0x7f36c40b45d0 2023-08-27T11:30:16.295 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.289+0000 7f36c37fe700 1 -- 172.21.15.45:0/4075594384 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0011d20 con 0x7f36c40b45d0 2023-08-27T11:30:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.290+0000 7f36cb253700 1 -- 172.21.15.45:0/4075594384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c40b45d0 msgr2=0x7f36c40d1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.290+0000 7f36cb253700 1 --2- 172.21.15.45:0/4075594384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c40b45d0 0x7f36c40d1460 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009870 tx=0x7f36b0009b80 comp rx=0 tx=0).stop 2023-08-27T11:30:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.290+0000 7f36cb253700 1 -- 172.21.15.45:0/4075594384 shutdown_connections 2023-08-27T11:30:16.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.290+0000 7f36cb253700 1 --2- 172.21.15.45:0/4075594384 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cf980 0x7f36c40d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.290+0000 7f36cb253700 1 --2- 172.21.15.45:0/4075594384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c40b45d0 0x7f36c40d1460 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.290+0000 7f36cb253700 1 -- 172.21.15.45:0/4075594384 >> 172.21.15.45:0/4075594384 conn(0x7f36c40d3020 msgr2=0x7f36c40d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:16.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.290+0000 7f36cb253700 1 -- 172.21.15.45:0/4075594384 shutdown_connections 2023-08-27T11:30:16.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.290+0000 7f36cb253700 1 -- 172.21.15.45:0/4075594384 wait complete. 2023-08-27T11:30:16.298 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.291+0000 7f36cb253700 1 Processor -- start 2023-08-27T11:30:16.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.291+0000 7f36cb253700 1 -- start start 2023-08-27T11:30:16.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.292+0000 7f36cb253700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c40b45d0 0x7f36c415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.292+0000 7f36cb253700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c40cf980 0x7f36c40cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.292+0000 7f36cb253700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cfd60 0x7f36c40b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.299 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.292+0000 7f36cb253700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40c6b50 con 0x7f36c40cf980 2023-08-27T11:30:16.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.292+0000 7f36cb253700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40c6da0 con 0x7f36c40b45d0 2023-08-27T11:30:16.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.292+0000 7f36cb253700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40c7080 con 0x7f36c40cfd60 2023-08-27T11:30:16.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.292+0000 7f36c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c40cf980 0x7f36c40cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.292+0000 7f36c8fef700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c40b45d0 0x7f36c415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.300 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.292+0000 7f36c3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c40cf980 0x7f36c40cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32990/0 (socket says 172.21.15.45:32990) 2023-08-27T11:30:16.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.293+0000 7f36c3fff700 1 -- 172.21.15.45:0/3465491864 learned_addr learned my addr 172.21.15.45:0/3465491864 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:16.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.293+0000 7f36c97f0700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cfd60 0x7f36c40b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.293+0000 7f36c3fff700 1 -- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cfd60 msgr2=0x7f36c40b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.293+0000 7f36c3fff700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cfd60 0x7f36c40b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.301 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.293+0000 7f36c3fff700 1 -- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c40b45d0 msgr2=0x7f36c415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.293+0000 7f36c3fff700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c40b45d0 0x7f36c415c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.293+0000 7f36c3fff700 1 -- 172.21.15.45:0/3465491864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b80094a0 con 0x7f36c40cf980 2023-08-27T11:30:16.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.293+0000 7f36c3fff700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c40cf980 0x7f36c40cb3c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f36b0000c00 tx=0x7f36b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:16.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.294+0000 7f36c17fa700 1 -- 172.21.15.45:0/3465491864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b00036a0 con 0x7f36c40cf980 2023-08-27T11:30:16.302 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.294+0000 7f36c17fa700 1 -- 172.21.15.45:0/3465491864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b001e070 con 0x7f36c40cf980 2023-08-27T11:30:16.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.294+0000 7f36c17fa700 1 -- 172.21.15.45:0/3465491864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0003a20 con 0x7f36c40cf980 2023-08-27T11:30:16.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.294+0000 7f36cb253700 1 -- 172.21.15.45:0/3465491864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b00094a0 con 0x7f36c40cf980 2023-08-27T11:30:16.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.294+0000 7f36cb253700 1 -- 172.21.15.45:0/3465491864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c40c7640 con 0x7f36c40cf980 2023-08-27T11:30:16.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.295+0000 7f36cb253700 1 -- 172.21.15.45:0/3465491864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c40c8840 con 0x7f36c40cf980 2023-08-27T11:30:16.303 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.296+0000 7f36c17fa700 1 -- 172.21.15.45:0/3465491864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 53) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f36b0003b80 con 0x7f36c40cf980 2023-08-27T11:30:16.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.296+0000 7f36c17fa700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f36a806a320 0x7f36a806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.296+0000 7f36c17fa700 1 -- 172.21.15.45:0/3465491864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f36b0089b60 con 0x7f36c40cf980 2023-08-27T11:30:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.296+0000 7f36c8fef700 1 -- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f36a806a320 msgr2=0x7f36a806c7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6832/1129514681 2023-08-27T11:30:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.296+0000 7f36c8fef700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f36a806a320 0x7f36a806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-27T11:30:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.300+0000 7f36c17fa700 1 -- 172.21.15.45:0/3465491864 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 54) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f36b0054fc0 con 0x7f36c40cf980 2023-08-27T11:30:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.300+0000 7f36c17fa700 1 -- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f36a806a320 msgr2=0x7f36a806c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-27T11:30:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.300+0000 7f36c17fa700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f36a806a320 0x7f36a806c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.307+0000 7f36c17fa700 1 -- 172.21.15.45:0/3465491864 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f36b005c400 con 0x7f36c40cf980 2023-08-27T11:30:16.362 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb084d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.363 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb084d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.363 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb084d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.363 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb0b4d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.363 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb084d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.363 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb084d3700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.364 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb0b4d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.364 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb0b4d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.364 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb0b4d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.364 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-27T11:30:16.361+0000 7fcb0b4d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-27T11:30:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.557+0000 7f36cb253700 1 -- 172.21.15.45:0/3465491864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36c40a1030 con 0x7f36c40cf980 2023-08-27T11:30:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.557+0000 7f36c17fa700 1 -- 172.21.15.45:0/3465491864 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4874 (secure 0 0 0) 0x7f36b00583f0 con 0x7f36c40cf980 2023-08-27T11:30:16.560 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:30:16.560 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7402,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2571849064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2571849064},{"type":"v1","addr":"172.21.15.135:6836","nonce":2571849064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/509114843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":509114843},{"type":"v1","addr":"172.21.15.135:6837","nonce":509114843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/528899713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":528899713},{"type":"v1","addr":"172.21.15.45:6833","nonce":528899713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.511894+0000","modified":"2023-08-27T11:28:53.106978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8019":{"gid":8019,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/343785163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":343785163},{"type":"v1","addr":"172.21.15.45:6835","nonce":343785163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:30:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 -- 172.21.15.45:0/3465491864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c40cf980 msgr2=0x7f36c40cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c40cf980 0x7f36c40cb3c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f36b0000c00 tx=0x7f36b0004750 comp rx=0 tx=0).stop 2023-08-27T11:30:16.562 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 -- 172.21.15.45:0/3465491864 shutdown_connections 2023-08-27T11:30:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f36c40cfd60 0x7f36c40b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36c40b45d0 0x7f36c415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.135:6832/1129514681,v1:172.21.15.135:6833/1129514681] conn(0x7f36a806a320 0x7f36a806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 --2- 172.21.15.45:0/3465491864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36c40cf980 0x7f36c40cb3c0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 -- 172.21.15.45:0/3465491864 >> 172.21.15.45:0/3465491864 conn(0x7f36c40d3020 msgr2=0x7f36c4160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:16.564 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 -- 172.21.15.45:0/3465491864 shutdown_connections 2023-08-27T11:30:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.559+0000 7f36cb253700 1 -- 172.21.15.45:0/3465491864 wait complete. 2023-08-27T11:30:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 49 2023-08-27T11:30:16.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:30:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.740+0000 7f3b27a60700 1 Processor -- start 2023-08-27T11:30:16.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.740+0000 7f3b27a60700 1 -- start start 2023-08-27T11:30:16.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.741+0000 7f3b27a60700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3b200b45d0 0x7f3b200d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.741+0000 7f3b27a60700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 0x7f3b200b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.741+0000 7f3b27a60700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200ceae0 con 0x7f3b200cf980 2023-08-27T11:30:16.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.741+0000 7f3b27a60700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b200cf0a0 con 0x7f3b200b45d0 2023-08-27T11:30:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.741+0000 7f3b27a60700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200cedc0 con 0x7f3b200cfd60 2023-08-27T11:30:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.741+0000 7f3b24ffb700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3b200b45d0 0x7f3b200d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.741+0000 7f3b25ffd700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 0x7f3b200b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.741+0000 7f3b257fc700 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3b200cf980 0x7f3b200d0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.45:43668/0 (socket says 172.21.15.45:43668) 2023-08-27T11:30:16.745 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.741+0000 7f3b257fc700 1 -- 172.21.15.45:0/618610179 learned_addr learned my addr 172.21.15.45:0/618610179 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.742+0000 7f3b25ffd700 1 -- 172.21.15.45:0/618610179 >> v2:172.21.15.135:3300/0 conn(0x7f3b200b45d0 msgr2=0x7f3b200d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.742+0000 7f3b25ffd700 1 --2- 172.21.15.45:0/618610179 >> v2:172.21.15.135:3300/0 conn(0x7f3b200b45d0 0x7f3b200d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.742+0000 7f3b25ffd700 1 -- 172.21.15.45:0/618610179 >> v1:172.21.15.135:6789/0 conn(0x7f3b200cf980 legacy=0x7f3b200d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.742+0000 7f3b25ffd700 1 -- 172.21.15.45:0/618610179 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200b22c0 con 0x7f3b200cfd60 2023-08-27T11:30:16.746 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.742+0000 7f3b24ffb700 1 --2- 172.21.15.45:0/618610179 >> v2:172.21.15.135:3300/0 conn(0x7f3b200b45d0 0x7f3b200d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-27T11:30:16.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.742+0000 7f3b25ffd700 1 --2- 172.21.15.45:0/618610179 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 0x7f3b200b2ac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009d90 tx=0x7f3b1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24efa2954d77e741 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:16.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.742+0000 7f3b17fff700 1 -- 172.21.15.45:0/618610179 <== mon.2 v2:172.21.15.135:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800e070 con 0x7f3b200cfd60 2023-08-27T11:30:16.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.742+0000 7f3b17fff700 1 -- 172.21.15.45:0/618610179 <== mon.2 v2:172.21.15.135:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1800bb20 con 0x7f3b200cfd60 2023-08-27T11:30:16.747 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.742+0000 7f3b17fff700 1 -- 172.21.15.45:0/618610179 <== mon.2 v2:172.21.15.135:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18012430 con 0x7f3b200cfd60 2023-08-27T11:30:16.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.743+0000 7f3b27a60700 1 -- 172.21.15.45:0/618610179 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 msgr2=0x7f3b200b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.749 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.743+0000 7f3b27a60700 1 --2- 172.21.15.45:0/618610179 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 0x7f3b200b2ac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009d90 tx=0x7f3b1800b330 comp rx=0 tx=0).stop 2023-08-27T11:30:16.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.743+0000 7f3b27a60700 1 -- 172.21.15.45:0/618610179 shutdown_connections 2023-08-27T11:30:16.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.743+0000 7f3b27a60700 1 --2- 172.21.15.45:0/618610179 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 0x7f3b200b2ac0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.743+0000 7f3b27a60700 1 --2- 172.21.15.45:0/618610179 >> v2:172.21.15.135:3300/0 conn(0x7f3b200b45d0 0x7f3b200d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.743+0000 7f3b27a60700 1 -- 172.21.15.45:0/618610179 >> 172.21.15.45:0/618610179 conn(0x7f3b200d3020 msgr2=0x7f3b200d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:16.750 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.743+0000 7f3b27a60700 1 -- 172.21.15.45:0/618610179 shutdown_connections 2023-08-27T11:30:16.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.743+0000 7f3b27a60700 1 -- 172.21.15.45:0/618610179 wait complete. 2023-08-27T11:30:16.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.744+0000 7f3b27a60700 1 Processor -- start 2023-08-27T11:30:16.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.744+0000 7f3b27a60700 1 -- start start 2023-08-27T11:30:16.751 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b27a60700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200b45d0 0x7f3b2015d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b27a60700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b200cf980 0x7f3b201f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b27a60700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 0x7f3b2015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b27a60700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201f3d30 con 0x7f3b200b45d0 2023-08-27T11:30:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b27a60700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2015ba80 con 0x7f3b200cf980 2023-08-27T11:30:16.752 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b27a60700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200ca0f0 con 0x7f3b200cfd60 2023-08-27T11:30:16.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b257fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200b45d0 0x7f3b2015d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b257fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200b45d0 0x7f3b2015d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33002/0 (socket says 172.21.15.45:33002) 2023-08-27T11:30:16.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b257fc700 1 -- 172.21.15.45:0/1848590135 learned_addr learned my addr 172.21.15.45:0/1848590135 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:16.753 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b25ffd700 1 --2- 172.21.15.45:0/1848590135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 0x7f3b2015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.754 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b24ffb700 1 --2- 172.21.15.45:0/1848590135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b200cf980 0x7f3b201f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:16.754 INFO:tasks.ceph.mgr.y.smithi045.stderr:2023-08-27T11:30:16.745+0000 7fb732698200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-27T11:30:16.755 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b257fc700 1 -- 172.21.15.45:0/1848590135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 msgr2=0x7f3b2015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b257fc700 1 --2- 172.21.15.45:0/1848590135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 0x7f3b2015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b257fc700 1 -- 172.21.15.45:0/1848590135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b200cf980 msgr2=0x7f3b201f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b257fc700 1 --2- 172.21.15.45:0/1848590135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b200cf980 0x7f3b201f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.756 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.745+0000 7f3b257fc700 1 -- 172.21.15.45:0/1848590135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b100094a0 con 0x7f3b200b45d0 2023-08-27T11:30:16.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.746+0000 7f3b257fc700 1 --2- 172.21.15.45:0/1848590135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200b45d0 0x7f3b2015d470 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c520 tx=0x7f3b1c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:16.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.746+0000 7f3b15ffb700 1 -- 172.21.15.45:0/1848590135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c005d70 con 0x7f3b200b45d0 2023-08-27T11:30:16.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.746+0000 7f3b15ffb700 1 -- 172.21.15.45:0/1848590135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c0070d0 con 0x7f3b200b45d0 2023-08-27T11:30:16.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.746+0000 7f3b27a60700 1 -- 172.21.15.45:0/1848590135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b18009a70 con 0x7f3b200b45d0 2023-08-27T11:30:16.757 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.746+0000 7f3b15ffb700 1 -- 172.21.15.45:0/1848590135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c007440 con 0x7f3b200b45d0 2023-08-27T11:30:16.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.746+0000 7f3b27a60700 1 -- 172.21.15.45:0/1848590135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2015dde0 con 0x7f3b200b45d0 2023-08-27T11:30:16.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.747+0000 7f3b27a60700 1 -- 172.21.15.45:0/1848590135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b200a1030 con 0x7f3b200b45d0 2023-08-27T11:30:16.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.747+0000 7f3b15ffb700 1 -- 172.21.15.45:0/1848590135 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 54) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f3b1c0075a0 con 0x7f3b200b45d0 2023-08-27T11:30:16.758 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.748+0000 7f3b15ffb700 1 -- 172.21.15.45:0/1848590135 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7243+0+0 (secure 0 0 0) 0x7f3b1c087960 con 0x7f3b200b45d0 2023-08-27T11:30:16.759 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.752+0000 7f3b15ffb700 1 -- 172.21.15.45:0/1848590135 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3b1c052fd0 con 0x7f3b200b45d0 2023-08-27T11:30:16.956 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.954+0000 7f3b27a60700 1 -- 172.21.15.45:0/1848590135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b200734a0 con 0x7f3b200b45d0 2023-08-27T11:30:16.957 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.955+0000 7f3b15ffb700 1 -- 172.21.15.45:0/1848590135 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+19792 (secure 0 0 0) 0x7f3b1c0565f0 con 0x7f3b200b45d0 2023-08-27T11:30:16.957 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:30:16.958 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":118,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:30:16.294284+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.834012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.480260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/1522410493":"2023-08-28T11:30:16.294242+0000","172.21.15.135:6833/1129514681":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/2952730327":"2023-08-28T11:30:16.294242+0000","172.21.15.135:6832/1129514681":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/3201130321":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/1606231370":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/3548314165":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/3391794165":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/2740998365":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/1494860999":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/91399608":"2023-08-28T11:29:08.294424+0000","172.21.15.135:6833/3759624346":"2023-08-28T11:29:08.294424+0000","172.21.15.135:6832/3759624346":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/3287118442":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/927635461":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/4148146534":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/3025460570":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/4144122828":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/626141668":"2023-08-27T12:28:58.549630+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:30:16.959 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.958+0000 7f3b27a60700 1 -- 172.21.15.45:0/1848590135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200b45d0 msgr2=0x7f3b2015d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:16.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.958+0000 7f3b27a60700 1 --2- 172.21.15.45:0/1848590135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200b45d0 0x7f3b2015d470 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c520 tx=0x7f3b1c00c8e0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.958+0000 7f3b27a60700 1 -- 172.21.15.45:0/1848590135 shutdown_connections 2023-08-27T11:30:16.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.958+0000 7f3b27a60700 1 --2- 172.21.15.45:0/1848590135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200b45d0 0x7f3b2015d470 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.958+0000 7f3b27a60700 1 --2- 172.21.15.45:0/1848590135 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3b200cfd60 0x7f3b2015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.960 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.958+0000 7f3b27a60700 1 --2- 172.21.15.45:0/1848590135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b200cf980 0x7f3b201f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:16.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.958+0000 7f3b27a60700 1 -- 172.21.15.45:0/1848590135 >> 172.21.15.45:0/1848590135 conn(0x7f3b200d3020 msgr2=0x7f3b200b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:16.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.958+0000 7f3b27a60700 1 -- 172.21.15.45:0/1848590135 shutdown_connections 2023-08-27T11:30:16.961 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:16.958+0000 7f3b27a60700 1 -- 172.21.15.45:0/1848590135 wait complete. 2023-08-27T11:30:16.972 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-27T11:30:16.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:30:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.097+0000 7ff7805b1700 1 Processor -- start 2023-08-27T11:30:17.099 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.097+0000 7ff7805b1700 1 -- start start 2023-08-27T11:30:17.101 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff7805b1700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff77815d3d0 0x7ff7781619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff7805b1700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7780af040 0x7ff7780cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff7805b1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff778161550 con 0x7ff77806e3a0 2023-08-27T11:30:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff7805b1700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7780cdf90 con 0x7ff7780af040 2023-08-27T11:30:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff7805b1700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff778161f20 con 0x7ff77815d3d0 2023-08-27T11:30:17.102 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff77eb4e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff77806e3a0 0x7ff778070e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48022/0 (socket says 172.21.15.45:48022) 2023-08-27T11:30:17.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff77eb4e700 1 -- 172.21.15.45:0/1402751530 learned_addr learned my addr 172.21.15.45:0/1402751530 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:17.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff77db4c700 1 --2- 172.21.15.45:0/1402751530 >> v2:172.21.15.45:3300/0 conn(0x7ff7780af040 0x7ff7780cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff77e34d700 1 --2- 172.21.15.45:0/1402751530 >> v2:172.21.15.135:3300/0 conn(0x7ff77815d3d0 0x7ff7781619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.098+0000 7ff77d34b700 1 -- 172.21.15.45:0/1402751530 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322022721 0 0) 0x7ff778161550 con 0x7ff77806e3a0 2023-08-27T11:30:17.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff77d34b700 1 -- 172.21.15.45:0/1402751530 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7640035b0 con 0x7ff77806e3a0 2023-08-27T11:30:17.103 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff77db4c700 1 -- 172.21.15.45:0/1402751530 >> v2:172.21.15.135:3300/0 conn(0x7ff77815d3d0 msgr2=0x7ff7781619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:17.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff77db4c700 1 --2- 172.21.15.45:0/1402751530 >> v2:172.21.15.135:3300/0 conn(0x7ff77815d3d0 0x7ff7781619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff77db4c700 1 -- 172.21.15.45:0/1402751530 >> v1:172.21.15.45:6789/0 conn(0x7ff77806e3a0 legacy=0x7ff778070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff77db4c700 1 -- 172.21.15.45:0/1402751530 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7780cd9d0 con 0x7ff7780af040 2023-08-27T11:30:17.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff77db4c700 1 --2- 172.21.15.45:0/1402751530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 0x7ff7780cf420 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff778067880 tx=0x7ff768009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63e1600f337a76a server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:17.104 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff77d34b700 1 -- 172.21.15.45:0/1402751530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768011830 con 0x7ff7780af040 2023-08-27T11:30:17.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff77d34b700 1 -- 172.21.15.45:0/1402751530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff768011990 con 0x7ff7780af040 2023-08-27T11:30:17.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff77d34b700 1 -- 172.21.15.45:0/1402751530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768011d20 con 0x7ff7780af040 2023-08-27T11:30:17.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff7805b1700 1 -- 172.21.15.45:0/1402751530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 msgr2=0x7ff7780cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.099+0000 7ff7805b1700 1 --2- 172.21.15.45:0/1402751530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 0x7ff7780cf420 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff778067880 tx=0x7ff768009b20 comp rx=0 tx=0).stop 2023-08-27T11:30:17.105 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.100+0000 7ff7805b1700 1 -- 172.21.15.45:0/1402751530 shutdown_connections 2023-08-27T11:30:17.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.100+0000 7ff7805b1700 1 --2- 172.21.15.45:0/1402751530 >> v2:172.21.15.135:3300/0 conn(0x7ff77815d3d0 0x7ff7781619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.100+0000 7ff7805b1700 1 --2- 172.21.15.45:0/1402751530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 0x7ff7780cf420 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.106 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.100+0000 7ff7805b1700 1 -- 172.21.15.45:0/1402751530 >> 172.21.15.45:0/1402751530 conn(0x7ff7780cfd60 msgr2=0x7ff7780d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:17.107 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.100+0000 7ff7805b1700 1 -- 172.21.15.45:0/1402751530 shutdown_connections 2023-08-27T11:30:17.108 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.100+0000 7ff7805b1700 1 -- 172.21.15.45:0/1402751530 wait complete. 2023-08-27T11:30:17.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.101+0000 7ff7805b1700 1 Processor -- start 2023-08-27T11:30:17.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.101+0000 7ff7805b1700 1 -- start start 2023-08-27T11:30:17.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff7805b1700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff77806e3a0 0x7ff7780b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.110 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff7805b1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 0x7ff77815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff7805b1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff77815d3d0 0x7ff7781510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.111 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff7805b1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77815e700 con 0x7ff7780af040 2023-08-27T11:30:17.112 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff7805b1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77815ebd0 con 0x7ff77815d3d0 2023-08-27T11:30:17.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff7805b1700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77815e8f0 con 0x7ff77806e3a0 2023-08-27T11:30:17.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77db4c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 0x7ff77815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77db4c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 0x7ff77815e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33018/0 (socket says 172.21.15.45:33018) 2023-08-27T11:30:17.113 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77db4c700 1 -- 172.21.15.45:0/1113276687 learned_addr learned my addr 172.21.15.45:0/1113276687 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77e34d700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff77806e3a0 0x7ff7780b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77eb4e700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff77815d3d0 0x7ff7781510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77db4c700 1 -- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff77806e3a0 msgr2=0x7ff7780b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77db4c700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff77806e3a0 0x7ff7780b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.114 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77db4c700 1 -- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff77815d3d0 msgr2=0x7ff7781510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77db4c700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff77815d3d0 0x7ff7781510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77db4c700 1 -- 172.21.15.45:0/1113276687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680094a0 con 0x7ff7780af040 2023-08-27T11:30:17.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff77db4c700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 0x7ff77815e1c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff768009b60 tx=0x7ff768004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:17.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff76effd700 1 -- 172.21.15.45:0/1113276687 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7680036a0 con 0x7ff7780af040 2023-08-27T11:30:17.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff76effd700 1 -- 172.21.15.45:0/1113276687 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76801e070 con 0x7ff7780af040 2023-08-27T11:30:17.115 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7780b2a90 con 0x7ff7780af040 2023-08-27T11:30:17.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff76effd700 1 -- 172.21.15.45:0/1113276687 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768003a60 con 0x7ff7780af040 2023-08-27T11:30:17.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.102+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7780b3050 con 0x7ff7780af040 2023-08-27T11:30:17.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.103+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7780b3240 con 0x7ff7780af040 2023-08-27T11:30:17.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.104+0000 7ff76effd700 1 -- 172.21.15.45:0/1113276687 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 54) v1 ==== 88052+0+0 (secure 0 0 0) 0x7ff768003bc0 con 0x7ff7780af040 2023-08-27T11:30:17.116 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.104+0000 7ff76effd700 1 -- 172.21.15.45:0/1113276687 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7243+0+0 (secure 0 0 0) 0x7ff7680893e0 con 0x7ff7780af040 2023-08-27T11:30:17.117 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.108+0000 7ff76effd700 1 -- 172.21.15.45:0/1113276687 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff768054a80 con 0x7ff7780af040 2023-08-27T11:30:17.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.302+0000 7ff76effd700 1 -- 172.21.15.45:0/1113276687 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 55) v1 ==== 88179+0+0 (secure 0 0 0) 0x7ff768054530 con 0x7ff7780af040 2023-08-27T11:30:17.304 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.302+0000 7ff76effd700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7ff75c0705f0 0x7ff75c0729e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.303+0000 7ff77e34d700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7ff75c0705f0 0x7ff75c0729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.305 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.304+0000 7ff77e34d700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7ff75c0705f0 0x7ff75c0729e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff774009ab0 tx=0x7ff774008040 comp rx=0 tx=0).ready entity=mgr.7681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:17.362 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.361+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7780734a0 con 0x7ff7780af040 2023-08-27T11:30:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.361+0000 7ff76effd700 1 -- 172.21.15.45:0/1113276687 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4874 (secure 0 0 0) 0x7ff76802fa80 con 0x7ff7780af040 2023-08-27T11:30:17.363 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:30:17.364 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7402,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2571849064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2571849064},{"type":"v1","addr":"172.21.15.135:6836","nonce":2571849064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/509114843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":509114843},{"type":"v1","addr":"172.21.15.135:6837","nonce":509114843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/528899713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":528899713},{"type":"v1","addr":"172.21.15.45:6833","nonce":528899713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.511894+0000","modified":"2023-08-27T11:28:53.106978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8019":{"gid":8019,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/343785163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":343785163},{"type":"v1","addr":"172.21.15.45:6835","nonce":343785163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:30:17.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7ff75c0705f0 msgr2=0x7ff75c0729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7ff75c0705f0 0x7ff75c0729e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff774009ab0 tx=0x7ff774008040 comp rx=0 tx=0).stop 2023-08-27T11:30:17.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 msgr2=0x7ff77815e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 0x7ff77815e1c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff768009b60 tx=0x7ff768004750 comp rx=0 tx=0).stop 2023-08-27T11:30:17.366 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 shutdown_connections 2023-08-27T11:30:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7ff75c0705f0 0x7ff75c0729e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff77815d3d0 0x7ff7781510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7ff77806e3a0 0x7ff7780b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 --2- 172.21.15.45:0/1113276687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7780af040 0x7ff77815e1c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.367 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 >> 172.21.15.45:0/1113276687 conn(0x7ff7780cfd60 msgr2=0x7ff7780b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 shutdown_connections 2023-08-27T11:30:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.364+0000 7ff7805b1700 1 -- 172.21.15.45:0/1113276687 wait complete. 2023-08-27T11:30:17.368 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 49 2023-08-27T11:30:17.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-27T11:30:17.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.501+0000 7fea71dde700 1 Processor -- start 2023-08-27T11:30:17.503 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.501+0000 7fea71dde700 1 -- start start 2023-08-27T11:30:17.505 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea71dde700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea6c15b070 0x7fea6c0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea71dde700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c0cdf60 0x7fea6c06e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea71dde700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6c15dad0 con 0x7fea6c06cec0 2023-08-27T11:30:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea71dde700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6c15d8e0 con 0x7fea6c15b070 2023-08-27T11:30:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea71dde700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c15ddb0 con 0x7fea6c0cdf60 2023-08-27T11:30:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea6b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea6c15b070 0x7fea6c0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea6b7fe700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea6c15b070 0x7fea6c0cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33038/0 (socket says 172.21.15.45:33038) 2023-08-27T11:30:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea6b7fe700 1 -- 172.21.15.45:0/837246110 learned_addr learned my addr 172.21.15.45:0/837246110 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea6affd700 1 --2- 172.21.15.45:0/837246110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c0cdf60 0x7fea6c06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea6b7fe700 1 -- 172.21.15.45:0/837246110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c0cdf60 msgr2=0x7fea6c06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea6b7fe700 1 --2- 172.21.15.45:0/837246110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c0cdf60 0x7fea6c06e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea6b7fe700 1 -- 172.21.15.45:0/837246110 >> v1:172.21.15.135:6789/0 conn(0x7fea6c06cec0 legacy=0x7fea6c0ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea6b7fe700 1 -- 172.21.15.45:0/837246110 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c15e090 con 0x7fea6c15b070 2023-08-27T11:30:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.502+0000 7fea6affd700 1 --2- 172.21.15.45:0/837246110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c0cdf60 0x7fea6c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea6b7fe700 1 --2- 172.21.15.45:0/837246110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c15b070 0x7fea6c0cda20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0098f0 tx=0x7fea5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adcba94974a4668 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea6a7fc700 1 -- 172.21.15.45:0/837246110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c004230 con 0x7fea6c15b070 2023-08-27T11:30:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea6a7fc700 1 -- 172.21.15.45:0/837246110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5c004390 con 0x7fea6c15b070 2023-08-27T11:30:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea6a7fc700 1 -- 172.21.15.45:0/837246110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0036a0 con 0x7fea6c15b070 2023-08-27T11:30:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea71dde700 1 -- 172.21.15.45:0/837246110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c15b070 msgr2=0x7fea6c0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea71dde700 1 --2- 172.21.15.45:0/837246110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c15b070 0x7fea6c0cda20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0098f0 tx=0x7fea5c009c00 comp rx=0 tx=0).stop 2023-08-27T11:30:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea71dde700 1 -- 172.21.15.45:0/837246110 shutdown_connections 2023-08-27T11:30:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea71dde700 1 --2- 172.21.15.45:0/837246110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c15b070 0x7fea6c0cda20 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea71dde700 1 --2- 172.21.15.45:0/837246110 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c0cdf60 0x7fea6c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea71dde700 1 -- 172.21.15.45:0/837246110 >> 172.21.15.45:0/837246110 conn(0x7fea6c0cfe50 msgr2=0x7fea6c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea71dde700 1 -- 172.21.15.45:0/837246110 shutdown_connections 2023-08-27T11:30:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.503+0000 7fea71dde700 1 -- 172.21.15.45:0/837246110 wait complete. 2023-08-27T11:30:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.505+0000 7fea71dde700 1 Processor -- start 2023-08-27T11:30:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.505+0000 7fea71dde700 1 -- start start 2023-08-27T11:30:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea71dde700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c06cec0 0x7fea6c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea71dde700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c0cdf60 0x7fea6c141770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea71dde700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c15b070 0x7fea6c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea71dde700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c0c6920 con 0x7fea6c0cdf60 2023-08-27T11:30:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea71dde700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c0c5d10 con 0x7fea6c06cec0 2023-08-27T11:30:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea71dde700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c0c5ff0 con 0x7fea6c15b070 2023-08-27T11:30:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c0cdf60 0x7fea6c141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c0cdf60 0x7fea6c141770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33046/0 (socket says 172.21.15.45:33046) 2023-08-27T11:30:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6affd700 1 -- 172.21.15.45:0/2559537633 learned_addr learned my addr 172.21.15.45:0/2559537633 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6b7fe700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c06cec0 0x7fea6c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6bfff700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c15b070 0x7fea6c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6affd700 1 -- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c15b070 msgr2=0x7fea6c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6affd700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c15b070 0x7fea6c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6affd700 1 -- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c06cec0 msgr2=0x7fea6c0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6affd700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c06cec0 0x7fea6c0c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6affd700 1 -- 172.21.15.45:0/2559537633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c009550 con 0x7fea6c0cdf60 2023-08-27T11:30:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6bfff700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c15b070 0x7fea6c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6b7fe700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c06cec0 0x7fea6c0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea6affd700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c0cdf60 0x7fea6c141770 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fea5400c740 tx=0x7fea5400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea63fff700 1 -- 172.21.15.45:0/2559537633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54004220 con 0x7fea6c0cdf60 2023-08-27T11:30:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea63fff700 1 -- 172.21.15.45:0/2559537633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea54004380 con 0x7fea6c0cdf60 2023-08-27T11:30:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c0c62d0 con 0x7fea6c0cdf60 2023-08-27T11:30:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea63fff700 1 -- 172.21.15.45:0/2559537633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea540036a0 con 0x7fea6c0cdf60 2023-08-27T11:30:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.506+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c141cb0 con 0x7fea6c0cdf60 2023-08-27T11:30:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.507+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6c15b570 con 0x7fea6c0cdf60 2023-08-27T11:30:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.508+0000 7fea63fff700 1 -- 172.21.15.45:0/2559537633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 55) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fea54003800 con 0x7fea6c0cdf60 2023-08-27T11:30:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.508+0000 7fea63fff700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fea4c069f30 0x7fea4c06c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.508+0000 7fea63fff700 1 -- 172.21.15.45:0/2559537633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7243+0+0 (secure 0 0 0) 0x7fea54088a20 con 0x7fea6c0cdf60 2023-08-27T11:30:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.509+0000 7fea6b7fe700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fea4c069f30 0x7fea4c06c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.509+0000 7fea6b7fe700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fea4c069f30 0x7fea4c06c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009130 tx=0x7fea5c008030 comp rx=0 tx=0).ready entity=mgr.7681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:17.520 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.513+0000 7fea63fff700 1 -- 172.21.15.45:0/2559537633 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fea6c15b570 con 0x7fea6c0cdf60 2023-08-27T11:30:17.760 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.759+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea6c15b570 con 0x7fea6c0cdf60 2023-08-27T11:30:17.762 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.759+0000 7fea63fff700 1 -- 172.21.15.45:0/2559537633 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4874 (secure 0 0 0) 0x7fea6c15b570 con 0x7fea6c0cdf60 2023-08-27T11:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7402,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/2571849064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2571849064},{"type":"v1","addr":"172.21.15.135:6836","nonce":2571849064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/509114843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":509114843},{"type":"v1","addr":"172.21.15.135:6837","nonce":509114843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":8016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/528899713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":528899713},{"type":"v1","addr":"172.21.15.45:6833","nonce":528899713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-27T11:28:49.511894+0000","modified":"2023-08-27T11:28:53.106978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8019":{"gid":8019,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/343785163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":343785163},{"type":"v1","addr":"172.21.15.45:6835","nonce":343785163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-27T11:30:17.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fea4c069f30 msgr2=0x7fea4c06c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fea4c069f30 0x7fea4c06c3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009130 tx=0x7fea5c008030 comp rx=0 tx=0).stop 2023-08-27T11:30:17.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c0cdf60 msgr2=0x7fea6c141770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.764 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c0cdf60 0x7fea6c141770 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fea5400c740 tx=0x7fea5400cb00 comp rx=0 tx=0).stop 2023-08-27T11:30:17.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 shutdown_connections 2023-08-27T11:30:17.766 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fea4c069f30 0x7fea4c06c3f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fea6c15b070 0x7fea6c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea6c06cec0 0x7fea6c0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.767 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 --2- 172.21.15.45:0/2559537633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6c0cdf60 0x7fea6c141770 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 >> 172.21.15.45:0/2559537633 conn(0x7fea6c0cfe50 msgr2=0x7fea6c159130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:17.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 shutdown_connections 2023-08-27T11:30:17.768 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.762+0000 7fea71dde700 1 -- 172.21.15.45:0/2559537633 wait complete. 2023-08-27T11:30:17.769 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 49 2023-08-27T11:30:17.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-27T11:30:17.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.899+0000 7f17b5cc2700 1 Processor -- start 2023-08-27T11:30:17.901 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.899+0000 7f17b5cc2700 1 -- start start 2023-08-27T11:30:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17b5cc2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17b00cad40 0x7f17b015f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17b5cc2700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f17b006a990 0x7f17b015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17b5cc2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b015ed90 con 0x7f17b015b5c0 2023-08-27T11:30:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17b5cc2700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b015c0e0 con 0x7f17b006a990 2023-08-27T11:30:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17b5cc2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b015c3c0 con 0x7f17b00cad40 2023-08-27T11:30:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17aeffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f17b015b5c0 0x7f17b006a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48062/0 (socket says 172.21.15.45:48062) 2023-08-27T11:30:17.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17aeffd700 1 -- 172.21.15.45:0/1982298630 learned_addr learned my addr 172.21.15.45:0/1982298630 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:17.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17af7fe700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17b00cad40 0x7f17b015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087893667 0 0) 0x7f17b015ed90 con 0x7f17b015b5c0 2023-08-27T11:30:17.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f179c0035b0 con 0x7f17b015b5c0 2023-08-27T11:30:17.906 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.900+0000 7f17affff700 1 --2- 172.21.15.45:0/1982298630 >> v2:172.21.15.135:3300/0 conn(0x7f17b006a990 0x7f17b015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1573417999 0 0) 0x7f179c0035b0 con 0x7f17b015b5c0 2023-08-27T11:30:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17b00cad40 msgr2=0x7f17b015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17b00cad40 0x7f17b015f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 >> v2:172.21.15.135:3300/0 conn(0x7f17b006a990 msgr2=0x7f17b015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 --2- 172.21.15.45:0/1982298630 >> v2:172.21.15.135:3300/0 conn(0x7f17b006a990 0x7f17b015ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b015f050 con 0x7f17b015b5c0 2023-08-27T11:30:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f1798001670 con 0x7f17b015b5c0 2023-08-27T11:30:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 >> v1:172.21.15.45:6789/0 conn(0x7f17b015b5c0 legacy=0x7f17b006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c003a00 0x7f179c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f179c004850 0x7f179c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17b00cad40 0x7f179c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b015c0e0 con 0x7f179c003a00 2023-08-27T11:30:17.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b015c3c0 con 0x7f179c004850 2023-08-27T11:30:17.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c00ba10 con 0x7f17b00cad40 2023-08-27T11:30:17.909 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17af7fe700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c003a00 0x7f179c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17aeffd700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f179c004850 0x7f179c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17affff700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17b00cad40 0x7f179c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17af7fe700 1 -- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17b00cad40 msgr2=0x7f179c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:17.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17af7fe700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17b00cad40 0x7f179c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.910 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17af7fe700 1 -- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f179c004850 msgr2=0x7f179c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:17.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17af7fe700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f179c004850 0x7f179c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17af7fe700 1 -- 172.21.15.45:0/1982298630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c0035b0 con 0x7f179c003a00 2023-08-27T11:30:17.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17affff700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f17b00cad40 0x7f179c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:17.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17af7fe700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c003a00 0x7f179c003e20 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009b60 tx=0x7f179c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8de09cc6f6b50e server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:17.911 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a000e6c0 con 0x7f179c003a00 2023-08-27T11:30:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a000e820 con 0x7f179c003a00 2023-08-27T11:30:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.901+0000 7f17ae7fc700 1 -- 172.21.15.45:0/1982298630 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a000eb90 con 0x7f179c003a00 2023-08-27T11:30:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.902+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1982298630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c003a00 msgr2=0x7f179c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.902+0000 7f17b5cc2700 1 --2- 172.21.15.45:0/1982298630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c003a00 0x7f179c003e20 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009b60 tx=0x7f179c005880 comp rx=0 tx=0).stop 2023-08-27T11:30:17.912 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.902+0000 7f17affff700 1 -- 172.21.15.45:0/1982298630 reap_dead start 2023-08-27T11:30:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.902+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1982298630 shutdown_connections 2023-08-27T11:30:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.902+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1982298630 >> 172.21.15.45:0/1982298630 conn(0x7f17b00cfd70 msgr2=0x7f17b00d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.902+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1982298630 shutdown_connections 2023-08-27T11:30:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.902+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1982298630 wait complete. 2023-08-27T11:30:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.903+0000 7f17b5cc2700 1 Processor -- start 2023-08-27T11:30:17.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.903+0000 7f17b5cc2700 1 -- start start 2023-08-27T11:30:17.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17b5cc2700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f179c003a00 0x7f17b0154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.916 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17b5cc2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17b0153790 0x7f17b00b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17b5cc2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b0150770 0x7f17b00b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17b5cc2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0154830 con 0x7f17b0150770 2023-08-27T11:30:17.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17b5cc2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0153c20 con 0x7f17b0153790 2023-08-27T11:30:17.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17b5cc2700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0153f00 con 0x7f179c003a00 2023-08-27T11:30:17.917 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17affff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b0150770 0x7f17b00b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17affff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b0150770 0x7f17b00b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33068/0 (socket says 172.21.15.45:33068) 2023-08-27T11:30:17.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17affff700 1 -- 172.21.15.45:0/1314747491 learned_addr learned my addr 172.21.15.45:0/1314747491 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:17.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17af7fe700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f179c003a00 0x7f17b0154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17aeffd700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17b0153790 0x7f17b00b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.918 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17affff700 1 -- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f179c003a00 msgr2=0x7f17b0154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17affff700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f179c003a00 0x7f17b0154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17affff700 1 -- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17b0153790 msgr2=0x7f17b00b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:17.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17affff700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17b0153790 0x7f17b00b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:17.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17affff700 1 -- 172.21.15.45:0/1314747491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b01541e0 con 0x7f17b0150770 2023-08-27T11:30:17.919 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.904+0000 7f17aeffd700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17b0153790 0x7f17b00b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:17.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.905+0000 7f17af7fe700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f179c003a00 0x7f17b0154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:17.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.905+0000 7f17affff700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b0150770 0x7f17b00b3040 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f17b00c6d70 tx=0x7f17a80104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:17.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.905+0000 7f17a7fff700 1 -- 172.21.15.45:0/1314747491 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a801b070 con 0x7f17b0150770 2023-08-27T11:30:17.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.905+0000 7f17a7fff700 1 -- 172.21.15.45:0/1314747491 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a800f040 con 0x7f17b0150770 2023-08-27T11:30:17.920 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.905+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b01544c0 con 0x7f17b0150770 2023-08-27T11:30:17.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.905+0000 7f17a7fff700 1 -- 172.21.15.45:0/1314747491 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800e050 con 0x7f17b0150770 2023-08-27T11:30:17.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.905+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b00b3580 con 0x7f17b0150770 2023-08-27T11:30:17.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.906+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b00727c0 con 0x7f17b0150770 2023-08-27T11:30:17.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.906+0000 7f17a7fff700 1 -- 172.21.15.45:0/1314747491 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 55) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f17a8015440 con 0x7f17b0150770 2023-08-27T11:30:17.921 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.907+0000 7f17a7fff700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f1790069f60 0x7f179006c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.907+0000 7f17a7fff700 1 -- 172.21.15.45:0/1314747491 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7243+0+0 (secure 0 0 0) 0x7f17a80886b0 con 0x7f17b0150770 2023-08-27T11:30:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.907+0000 7f17af7fe700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f1790069f60 0x7f179006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.908+0000 7f17af7fe700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f1790069f60 0x7f179006c420 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17b00d0830 tx=0x7f17a0008030 comp rx=0 tx=0).ready entity=mgr.7681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:17.911+0000 7f17a7fff700 1 -- 172.21.15.45:0/1314747491 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17a8053c40 con 0x7f17b0150770 2023-08-27T11:30:18.120 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.118+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17b00734a0 con 0x7f17b0150770 2023-08-27T11:30:18.121 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.119+0000 7f17a7fff700 1 -- 172.21.15.45:0/1314747491 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+19792 (secure 0 0 0) 0x7f17a8057260 con 0x7f17b0150770 2023-08-27T11:30:18.121 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-08-27T11:30:18.122 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":118,"fsid":"8ea1ae34-1697-4b79-b2ce-3f741cd3c828","created":"2023-08-27T11:11:43.090516+0000","modified":"2023-08-27T11:30:16.294284+0000","last_up_change":"2023-08-27T11:11:54.209212+0000","last_in_change":"2023-08-27T11:11:45.636261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-27T11:11:54.862440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-27T11:12:01.658644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-27T11:28:47.834012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-27T11:28:48.480260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ee29c4c-1bd5-47d1-9f3b-0aff9522a2fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6801","nonce":1741844211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6803","nonce":1741844211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6807","nonce":1741844211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1741844211},{"type":"v1","addr":"172.21.15.45:6805","nonce":1741844211}]},"public_addr":"172.21.15.45:6801/1741844211","cluster_addr":"172.21.15.45:6803/1741844211","heartbeat_back_addr":"172.21.15.45:6807/1741844211","heartbeat_front_addr":"172.21.15.45:6805/1741844211","state":["exists","up"]},{"osd":1,"uuid":"1d89c17a-9f89-47a3-a818-ad5c27e9b2e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6809","nonce":2953012420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6811","nonce":2953012420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6815","nonce":2953012420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2953012420},{"type":"v1","addr":"172.21.15.45:6813","nonce":2953012420}]},"public_addr":"172.21.15.45:6809/2953012420","cluster_addr":"172.21.15.45:6811/2953012420","heartbeat_back_addr":"172.21.15.45:6815/2953012420","heartbeat_front_addr":"172.21.15.45:6813/2953012420","state":["exists","up"]},{"osd":2,"uuid":"b98fcbad-8b7b-4487-bbc8-ba2548e82f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6825","nonce":1621076447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6827","nonce":1621076447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6831","nonce":1621076447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1621076447},{"type":"v1","addr":"172.21.15.45:6829","nonce":1621076447}]},"public_addr":"172.21.15.45:6825/1621076447","cluster_addr":"172.21.15.45:6827/1621076447","heartbeat_back_addr":"172.21.15.45:6831/1621076447","heartbeat_front_addr":"172.21.15.45:6829/1621076447","state":["exists","up"]},{"osd":3,"uuid":"279ba826-637d-47c4-ac4f-1b31f3185aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6817","nonce":1641214045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6819","nonce":1641214045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6823","nonce":1641214045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1641214045},{"type":"v1","addr":"172.21.15.45:6821","nonce":1641214045}]},"public_addr":"172.21.15.45:6817/1641214045","cluster_addr":"172.21.15.45:6819/1641214045","heartbeat_back_addr":"172.21.15.45:6823/1641214045","heartbeat_front_addr":"172.21.15.45:6821/1641214045","state":["exists","up"]},{"osd":4,"uuid":"23ee4406-9cb4-4559-acc4-d2f25a487f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6801","nonce":1785499536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6803","nonce":1785499536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6807","nonce":1785499536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1785499536},{"type":"v1","addr":"172.21.15.135:6805","nonce":1785499536}]},"public_addr":"172.21.15.135:6801/1785499536","cluster_addr":"172.21.15.135:6803/1785499536","heartbeat_back_addr":"172.21.15.135:6807/1785499536","heartbeat_front_addr":"172.21.15.135:6805/1785499536","state":["exists","up"]},{"osd":5,"uuid":"996b9bc1-3eab-4765-b323-f2fe8edbd353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6825","nonce":3277146230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6827","nonce":3277146230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6831","nonce":3277146230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3277146230},{"type":"v1","addr":"172.21.15.135:6829","nonce":3277146230}]},"public_addr":"172.21.15.135:6825/3277146230","cluster_addr":"172.21.15.135:6827/3277146230","heartbeat_back_addr":"172.21.15.135:6831/3277146230","heartbeat_front_addr":"172.21.15.135:6829/3277146230","state":["exists","up"]},{"osd":6,"uuid":"d9bd54f7-ae70-4fab-9b76-afbf0e040d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6816","nonce":2334385273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6817","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6819","nonce":2334385273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6823","nonce":2334385273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":2334385273},{"type":"v1","addr":"172.21.15.135:6821","nonce":2334385273}]},"public_addr":"172.21.15.135:6816/2334385273","cluster_addr":"172.21.15.135:6819/2334385273","heartbeat_back_addr":"172.21.15.135:6823/2334385273","heartbeat_front_addr":"172.21.15.135:6821/2334385273","state":["exists","up"]},{"osd":7,"uuid":"d1119d04-f9a6-40e1-a79b-bab5eb998f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6809","nonce":3357661340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6811","nonce":3357661340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6818","nonce":3357661340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3357661340},{"type":"v1","addr":"172.21.15.135:6813","nonce":3357661340}]},"public_addr":"172.21.15.135:6809/3357661340","cluster_addr":"172.21.15.135:6811/3357661340","heartbeat_back_addr":"172.21.15.135:6818/3357661340","heartbeat_front_addr":"172.21.15.135:6813/3357661340","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.830500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.798064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.803159+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.867981+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.797785+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.907078+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-27T11:11:50.833954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.135:0/1522410493":"2023-08-28T11:30:16.294242+0000","172.21.15.135:6833/1129514681":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/2952730327":"2023-08-28T11:30:16.294242+0000","172.21.15.135:6832/1129514681":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/3201130321":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/1606231370":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/3548314165":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/3391794165":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/2740998365":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/1494860999":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/91399608":"2023-08-28T11:29:08.294424+0000","172.21.15.135:6833/3759624346":"2023-08-28T11:29:08.294424+0000","172.21.15.135:6832/3759624346":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/3287118442":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/927635461":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/4148146534":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/3025460570":"2023-08-28T11:29:08.294424+0000","172.21.15.135:0/4144122828":"2023-08-28T11:30:16.294242+0000","172.21.15.135:0/626141668":"2023-08-27T12:28:58.549630+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":106,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-27T11:30:18.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f1790069f60 msgr2=0x7f179006c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.123 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f1790069f60 0x7f179006c420 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17b00d0830 tx=0x7f17a0008030 comp rx=0 tx=0).stop 2023-08-27T11:30:18.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b0150770 msgr2=0x7f17b00b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b0150770 0x7f17b00b3040 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f17b00c6d70 tx=0x7f17a80104a0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 shutdown_connections 2023-08-27T11:30:18.124 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f1790069f60 0x7f179006c420 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b0150770 0x7f17b00b3040 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f179c003a00 0x7f17b0154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 --2- 172.21.15.45:0/1314747491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17b0153790 0x7f17b00b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.125 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 >> 172.21.15.45:0/1314747491 conn(0x7f17b00cfd70 msgr2=0x7f17b006a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:18.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 shutdown_connections 2023-08-27T11:30:18.126 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.121+0000 7f17b5cc2700 1 -- 172.21.15.45:0/1314747491 wait complete. 2023-08-27T11:30:18.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-27T11:30:18.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.260+0000 7f3de5d84700 1 Processor -- start 2023-08-27T11:30:18.262 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.260+0000 7f3de5d84700 1 -- start start 2023-08-27T11:30:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3de5d84700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3de015af30 0x7f3de00cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3de5d84700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de00cb8b0 con 0x7f3de00cde80 2023-08-27T11:30:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3de5d84700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de00cecd0 con 0x7f3de00cf960 2023-08-27T11:30:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3de5d84700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de006f9c0 con 0x7f3de015af30 2023-08-27T11:30:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3ddffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3de00cde80 0x7f3de0164fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48084/0 (socket says 172.21.15.45:48084) 2023-08-27T11:30:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3ddffff700 1 -- 172.21.15.45:0/129287462 learned_addr learned my addr 172.21.15.45:0/129287462 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3ddf7fe700 1 --2- 172.21.15.45:0/129287462 >> v2:172.21.15.135:3300/0 conn(0x7f3de015af30 0x7f3de00cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343274232 0 0) 0x7f3de00cb8b0 con 0x7f3de00cde80 2023-08-27T11:30:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dcc0035b0 con 0x7f3de00cde80 2023-08-27T11:30:18.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2152295881 0 0) 0x7f3dcc0035b0 con 0x7f3de00cde80 2023-08-27T11:30:18.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 >> v2:172.21.15.135:3300/0 conn(0x7f3de015af30 msgr2=0x7f3de00cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:18.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3dde7fc700 1 --2- 172.21.15.45:0/129287462 >> v2:172.21.15.135:3300/0 conn(0x7f3de015af30 0x7f3de00cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 >> v1:172.21.15.135:6789/0 conn(0x7f3de00cf960 legacy=0x7f3de00cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.268 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de00cbae0 con 0x7f3de00cde80 2023-08-27T11:30:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.261+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f3dd80019f0 con 0x7f3de00cde80 2023-08-27T11:30:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 >> v1:172.21.15.45:6789/0 conn(0x7f3de00cde80 legacy=0x7f3de0164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dcc003a00 0x7f3dcc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3dcc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dcc005210 0x7f3dcc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc00c090 con 0x7f3dcc004850 2023-08-27T11:30:18.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc00c200 con 0x7f3dcc003a00 2023-08-27T11:30:18.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc00c3c0 con 0x7f3dcc005210 2023-08-27T11:30:18.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddeffd700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3dcc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.270 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddf7fe700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dcc003a00 0x7f3dcc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddffff700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dcc005210 0x7f3dcc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddeffd700 1 -- 172.21.15.45:0/129287462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dcc005210 msgr2=0x7f3dcc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:18.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddeffd700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3dcc005210 0x7f3dcc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddeffd700 1 -- 172.21.15.45:0/129287462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dcc003a00 msgr2=0x7f3dcc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:18.271 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddeffd700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dcc003a00 0x7f3dcc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddeffd700 1 -- 172.21.15.45:0/129287462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc0035b0 con 0x7f3dcc004850 2023-08-27T11:30:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddffff700 1 -- 172.21.15.45:0/129287462 reap_dead start 2023-08-27T11:30:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3ddeffd700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3dcc004cd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3de00cfdc0 tx=0x7f3dcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3313194c0977eba8 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00052d0 con 0x7f3dcc004850 2023-08-27T11:30:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0005450 con 0x7f3dcc004850 2023-08-27T11:30:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.262+0000 7f3dde7fc700 1 -- 172.21.15.45:0/129287462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00038c0 con 0x7f3dcc004850 2023-08-27T11:30:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.263+0000 7f3de5d84700 1 -- 172.21.15.45:0/129287462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 msgr2=0x7f3dcc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.263+0000 7f3de5d84700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3dcc004cd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3de00cfdc0 tx=0x7f3dcc005880 comp rx=0 tx=0).stop 2023-08-27T11:30:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.263+0000 7f3de5d84700 1 -- 172.21.15.45:0/129287462 shutdown_connections 2023-08-27T11:30:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.263+0000 7f3de5d84700 1 --2- 172.21.15.45:0/129287462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3dcc004cd0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.263+0000 7f3de5d84700 1 -- 172.21.15.45:0/129287462 >> 172.21.15.45:0/129287462 conn(0x7f3de009a1a0 msgr2=0x7f3de00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.263+0000 7f3de5d84700 1 -- 172.21.15.45:0/129287462 shutdown_connections 2023-08-27T11:30:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.263+0000 7f3de5d84700 1 -- 172.21.15.45:0/129287462 wait complete. 2023-08-27T11:30:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.264+0000 7f3de5d84700 1 Processor -- start 2023-08-27T11:30:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.264+0000 7f3de5d84700 1 -- start start 2023-08-27T11:30:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3de5d84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3de00c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3de5d84700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de006ce60 0x7f3de0158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3de5d84700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de00b1ac0 0x7f3de00b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3de5d84700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00b2480 con 0x7f3dcc004850 2023-08-27T11:30:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3de5d84700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0158660 con 0x7f3de006ce60 2023-08-27T11:30:18.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3de5d84700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00c6e20 con 0x7f3de00b1ac0 2023-08-27T11:30:18.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3de00c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3de00c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33084/0 (socket says 172.21.15.45:33084) 2023-08-27T11:30:18.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddf7fe700 1 -- 172.21.15.45:0/2928027598 learned_addr learned my addr 172.21.15.45:0/2928027598 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:18.278 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddeffd700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de006ce60 0x7f3de0158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddffff700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de00b1ac0 0x7f3de00b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddf7fe700 1 -- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de00b1ac0 msgr2=0x7f3de00b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddf7fe700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de00b1ac0 0x7f3de00b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddf7fe700 1 -- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de006ce60 msgr2=0x7f3de0158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.279 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddf7fe700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de006ce60 0x7f3de0158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.265+0000 7f3ddf7fe700 1 -- 172.21.15.45:0/2928027598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de015c360 con 0x7f3dcc004850 2023-08-27T11:30:18.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.266+0000 7f3ddf7fe700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3de00c6840 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8011ad0 tx=0x7f3dc8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:18.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.266+0000 7f3dd7fff700 1 -- 172.21.15.45:0/2928027598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc800fa00 con 0x7f3dcc004850 2023-08-27T11:30:18.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.266+0000 7f3dd7fff700 1 -- 172.21.15.45:0/2928027598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc800fb60 con 0x7f3dcc004850 2023-08-27T11:30:18.280 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.266+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de015c640 con 0x7f3dcc004850 2023-08-27T11:30:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.266+0000 7f3dd7fff700 1 -- 172.21.15.45:0/2928027598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8014440 con 0x7f3dcc004850 2023-08-27T11:30:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.266+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de015cc00 con 0x7f3dcc004850 2023-08-27T11:30:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.267+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de00734a0 con 0x7f3dcc004850 2023-08-27T11:30:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.268+0000 7f3dd7fff700 1 -- 172.21.15.45:0/2928027598 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 55) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f3dc80145a0 con 0x7f3dcc004850 2023-08-27T11:30:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.268+0000 7f3dd7fff700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f3dc0069f00 0x7f3dc006c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.268+0000 7f3dd7fff700 1 -- 172.21.15.45:0/2928027598 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7243+0+0 (secure 0 0 0) 0x7f3dc8089400 con 0x7f3dcc004850 2023-08-27T11:30:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.268+0000 7f3ddeffd700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f3dc0069f00 0x7f3dc006c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.269+0000 7f3ddeffd700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f3dc0069f00 0x7f3dc006c3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0005040 tx=0x7f3dd0009030 comp rx=0 tx=0).ready entity=mgr.7681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.272+0000 7f3dd7fff700 1 -- 172.21.15.45:0/2928027598 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3dc8054a50 con 0x7f3dcc004850 2023-08-27T11:30:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.525+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3de00ce4c0 con 0x7f3dcc004850 2023-08-27T11:30:18.527 INFO:tasks.ceph.mon.a.smithi045.stderr:2023-08-27T11:30:18.525+0000 7f87a1c48700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-27T11:30:18.531 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.529+0000 7f3dd7fff700 1 -- 172.21.15.45:0/2928027598 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3dc8058070 con 0x7f3dcc004850 2023-08-27T11:30:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.532+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f3dc0069f00 msgr2=0x7f3dc006c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.532+0000 7f3de5d84700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f3dc0069f00 0x7f3dc006c3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0005040 tx=0x7f3dd0009030 comp rx=0 tx=0).stop 2023-08-27T11:30:18.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.532+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 msgr2=0x7f3de00c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.532+0000 7f3de5d84700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3de00c6840 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8011ad0 tx=0x7f3dc8011e90 comp rx=0 tx=0).stop 2023-08-27T11:30:18.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.533+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 shutdown_connections 2023-08-27T11:30:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.533+0000 7f3de5d84700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f3dc0069f00 0x7f3dc006c3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.533+0000 7f3de5d84700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dcc004850 0x7f3de00c6840 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.533+0000 7f3de5d84700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f3de00b1ac0 0x7f3de00b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.533+0000 7f3de5d84700 1 --2- 172.21.15.45:0/2928027598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3de006ce60 0x7f3de0158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.533+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 >> 172.21.15.45:0/2928027598 conn(0x7f3de009a1a0 msgr2=0x7f3de00afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.533+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 shutdown_connections 2023-08-27T11:30:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.533+0000 7f3de5d84700 1 -- 172.21.15.45:0/2928027598 wait complete. 2023-08-27T11:30:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-27T11:30:18.538 INFO:tasks.ceph.mds.a.smithi045.stderr: -8167> 2023-08-27T11:28:55.068+0000 7f4f3933f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-27T11:30:18.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-27T11:30:18.570 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2023-08-27T11:30:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.674+0000 7fe1747a4700 1 Processor -- start 2023-08-27T11:30:18.677 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.674+0000 7fe1747a4700 1 -- start start 2023-08-27T11:30:18.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.675+0000 7fe1747a4700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe16c0ca620 0x7fe16c0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.679 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.675+0000 7fe1747a4700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe16c0caa00 0x7fe16c0acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.675+0000 7fe1747a4700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c0ce340 con 0x7fe16c0af200 2023-08-27T11:30:18.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.675+0000 7fe1747a4700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c0ceec0 con 0x7fe16c0ca620 2023-08-27T11:30:18.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.675+0000 7fe1747a4700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c15d7c0 con 0x7fe16c0caa00 2023-08-27T11:30:18.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172540700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe16c0ca620 0x7fe16c0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.680 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172540700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe16c0ca620 0x7fe16c0cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33104/0 (socket says 172.21.15.45:33104) 2023-08-27T11:30:18.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172540700 1 -- 172.21.15.45:0/92122415 learned_addr learned my addr 172.21.15.45:0/92122415 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:18.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172d41700 1 --2- 172.21.15.45:0/92122415 >> v2:172.21.15.135:3300/0 conn(0x7fe16c0caa00 0x7fe16c0acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172540700 1 -- 172.21.15.45:0/92122415 >> v2:172.21.15.135:3300/0 conn(0x7fe16c0caa00 msgr2=0x7fe16c0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:18.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172540700 1 --2- 172.21.15.45:0/92122415 >> v2:172.21.15.135:3300/0 conn(0x7fe16c0caa00 0x7fe16c0acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.681 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172540700 1 -- 172.21.15.45:0/92122415 >> v1:172.21.15.135:6789/0 conn(0x7fe16c0af200 legacy=0x7fe16c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172540700 1 -- 172.21.15.45:0/92122415 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0cc570 con 0x7fe16c0ca620 2023-08-27T11:30:18.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172d41700 1 --2- 172.21.15.45:0/92122415 >> v2:172.21.15.135:3300/0 conn(0x7fe16c0caa00 0x7fe16c0acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:18.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe172540700 1 --2- 172.21.15.45:0/92122415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0ca620 0x7fe16c0cf350 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fe16c06ccd0 tx=0x7fe168009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b8e5c45b07b846 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:18.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe17153e700 1 -- 172.21.15.45:0/92122415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16800b870 con 0x7fe16c0ca620 2023-08-27T11:30:18.682 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe17153e700 1 -- 172.21.15.45:0/92122415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16800b9d0 con 0x7fe16c0ca620 2023-08-27T11:30:18.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.676+0000 7fe17153e700 1 -- 172.21.15.45:0/92122415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16800bd60 con 0x7fe16c0ca620 2023-08-27T11:30:18.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.677+0000 7fe1747a4700 1 -- 172.21.15.45:0/92122415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0ca620 msgr2=0x7fe16c0cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.677+0000 7fe1747a4700 1 --2- 172.21.15.45:0/92122415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0ca620 0x7fe16c0cf350 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fe16c06ccd0 tx=0x7fe168009bd0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.677+0000 7fe1747a4700 1 -- 172.21.15.45:0/92122415 shutdown_connections 2023-08-27T11:30:18.683 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.677+0000 7fe1747a4700 1 --2- 172.21.15.45:0/92122415 >> v2:172.21.15.135:3300/0 conn(0x7fe16c0caa00 0x7fe16c0acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.677+0000 7fe1747a4700 1 --2- 172.21.15.45:0/92122415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0ca620 0x7fe16c0cf350 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.677+0000 7fe1747a4700 1 -- 172.21.15.45:0/92122415 >> 172.21.15.45:0/92122415 conn(0x7fe16c0cc7e0 msgr2=0x7fe16c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:18.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.677+0000 7fe1747a4700 1 -- 172.21.15.45:0/92122415 shutdown_connections 2023-08-27T11:30:18.684 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.677+0000 7fe1747a4700 1 -- 172.21.15.45:0/92122415 wait complete. 2023-08-27T11:30:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.678+0000 7fe1747a4700 1 Processor -- start 2023-08-27T11:30:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.678+0000 7fe1747a4700 1 -- start start 2023-08-27T11:30:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe1747a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0af200 0x7fe16c0b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe1747a4700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe16c0ca620 0x7fe16c144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe1747a4700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe16c0caa00 0x7fe16c06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe1747a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c157a60 con 0x7fe16c0af200 2023-08-27T11:30:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe1747a4700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c06da80 con 0x7fe16c0caa00 2023-08-27T11:30:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe1747a4700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c15b260 con 0x7fe16c0ca620 2023-08-27T11:30:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172540700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0af200 0x7fe16c0b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172540700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0af200 0x7fe16c0b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33112/0 (socket says 172.21.15.45:33112) 2023-08-27T11:30:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172540700 1 -- 172.21.15.45:0/2720886828 learned_addr learned my addr 172.21.15.45:0/2720886828 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172d41700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe16c0caa00 0x7fe16c06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe171d3f700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe16c0ca620 0x7fe16c144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172540700 1 -- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe16c0ca620 msgr2=0x7fe16c144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172540700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe16c0ca620 0x7fe16c144060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172540700 1 -- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe16c0caa00 msgr2=0x7fe16c06ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172540700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe16c0caa00 0x7fe16c06ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172540700 1 -- 172.21.15.45:0/2720886828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe168009550 con 0x7fe16c0af200 2023-08-27T11:30:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe171d3f700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe16c0ca620 0x7fe16c144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.679+0000 7fe172540700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0af200 0x7fe16c0b53c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fe168000c00 tx=0x7fe168004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.680+0000 7fe15effd700 1 -- 172.21.15.45:0/2720886828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168004310 con 0x7fe16c0af200 2023-08-27T11:30:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.680+0000 7fe15effd700 1 -- 172.21.15.45:0/2720886828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16801f070 con 0x7fe16c0af200 2023-08-27T11:30:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.680+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c157ff0 con 0x7fe16c0af200 2023-08-27T11:30:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.680+0000 7fe15effd700 1 -- 172.21.15.45:0/2720886828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1680036a0 con 0x7fe16c0af200 2023-08-27T11:30:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.680+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c0b5900 con 0x7fe16c0af200 2023-08-27T11:30:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.681+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c0cbce0 con 0x7fe16c0af200 2023-08-27T11:30:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.681+0000 7fe15effd700 1 -- 172.21.15.45:0/2720886828 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 55) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fe168021030 con 0x7fe16c0af200 2023-08-27T11:30:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.681+0000 7fe15effd700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fe150069f30 0x7fe15006c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.681+0000 7fe15effd700 1 -- 172.21.15.45:0/2720886828 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7329+0+0 (secure 0 0 0) 0x7fe168089930 con 0x7fe16c0af200 2023-08-27T11:30:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.682+0000 7fe171d3f700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fe150069f30 0x7fe15006c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.682+0000 7fe171d3f700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fe150069f30 0x7fe15006c3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe160000f60 tx=0x7fe160009040 comp rx=0 tx=0).ready entity=mgr.7681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.686+0000 7fe15effd700 1 -- 172.21.15.45:0/2720886828 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe168054e60 con 0x7fe16c0af200 2023-08-27T11:30:18.936 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:18.934+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe16c0734a0 con 0x7fe16c0af200 2023-08-27T11:30:19.315 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.313+0000 7fe15effd700 1 -- 172.21.15.45:0/2720886828 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 56) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fe168055230 con 0x7fe16c0af200 2023-08-27T11:30:19.532 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.530+0000 7fe15effd700 1 -- 172.21.15.45:0/2720886828 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1680588f0 con 0x7fe16c0af200 2023-08-27T11:30:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fe150069f30 msgr2=0x7fe15006c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fe150069f30 0x7fe15006c3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe160000f60 tx=0x7fe160009040 comp rx=0 tx=0).stop 2023-08-27T11:30:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0af200 msgr2=0x7fe16c0b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0af200 0x7fe16c0b53c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fe168000c00 tx=0x7fe168004750 comp rx=0 tx=0).stop 2023-08-27T11:30:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 shutdown_connections 2023-08-27T11:30:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7fe150069f30 0x7fe15006c3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c0af200 0x7fe16c0b53c0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe16c0caa00 0x7fe16c06ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 --2- 172.21.15.45:0/2720886828 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7fe16c0ca620 0x7fe16c144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 >> 172.21.15.45:0/2720886828 conn(0x7fe16c0cc7e0 msgr2=0x7fe16c0ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 shutdown_connections 2023-08-27T11:30:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.533+0000 7fe1747a4700 1 -- 172.21.15.45:0/2720886828 wait complete. 2023-08-27T11:30:19.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-27T11:30:19.727 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.726+0000 7f32538c0700 1 Processor -- start 2023-08-27T11:30:19.728 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.726+0000 7f32538c0700 1 -- start start 2023-08-27T11:30:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f32538c0700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f324c15d4f0 0x7f324c0ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f32538c0700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f324c0cacd0 0x7f324c0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f32538c0700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324c06b010 con 0x7f324c0aefb0 2023-08-27T11:30:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f32538c0700 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324c15d8d0 con 0x7f324c0cacd0 2023-08-27T11:30:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f32538c0700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324c0ad9f0 con 0x7f324c15d4f0 2023-08-27T11:30:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f3251e5d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f324c0cacd0 0x7f324c0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f3251e5d700 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f324c0cacd0 0x7f324c0cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33124/0 (socket says 172.21.15.45:33124) 2023-08-27T11:30:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f3251e5d700 1 -- 172.21.15.45:0/1308557960 learned_addr learned my addr 172.21.15.45:0/1308557960 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f325165c700 1 --2- 172.21.15.45:0/1308557960 >> v2:172.21.15.135:3300/0 conn(0x7f324c15d4f0 0x7f324c0ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f3251e5d700 1 -- 172.21.15.45:0/1308557960 >> v2:172.21.15.135:3300/0 conn(0x7f324c15d4f0 msgr2=0x7f324c0ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f3251e5d700 1 --2- 172.21.15.45:0/1308557960 >> v2:172.21.15.135:3300/0 conn(0x7f324c15d4f0 0x7f324c0ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f3251e5d700 1 -- 172.21.15.45:0/1308557960 >> v1:172.21.15.135:6789/0 conn(0x7f324c0aefb0 legacy=0x7f324c0ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f3251e5d700 1 -- 172.21.15.45:0/1308557960 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c06ad30 con 0x7f324c0cacd0 2023-08-27T11:30:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f325165c700 1 --2- 172.21.15.45:0/1308557960 >> v2:172.21.15.135:3300/0 conn(0x7f324c15d4f0 0x7f324c0ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.727+0000 7f3251e5d700 1 --2- 172.21.15.45:0/1308557960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c0cacd0 0x7f324c0cd930 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f324c079e70 tx=0x7f324400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7df6efd5c644584 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f3243fff700 1 -- 172.21.15.45:0/1308557960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324400e070 con 0x7f324c0cacd0 2023-08-27T11:30:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f3243fff700 1 -- 172.21.15.45:0/1308557960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3244003a20 con 0x7f324c0cacd0 2023-08-27T11:30:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f3243fff700 1 -- 172.21.15.45:0/1308557960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244003dd0 con 0x7f324c0cacd0 2023-08-27T11:30:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f32538c0700 1 -- 172.21.15.45:0/1308557960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c0cacd0 msgr2=0x7f324c0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f32538c0700 1 --2- 172.21.15.45:0/1308557960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c0cacd0 0x7f324c0cd930 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f324c079e70 tx=0x7f324400b330 comp rx=0 tx=0).stop 2023-08-27T11:30:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f32538c0700 1 -- 172.21.15.45:0/1308557960 shutdown_connections 2023-08-27T11:30:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f32538c0700 1 --2- 172.21.15.45:0/1308557960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c0cacd0 0x7f324c0cd930 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f32538c0700 1 --2- 172.21.15.45:0/1308557960 >> v2:172.21.15.135:3300/0 conn(0x7f324c15d4f0 0x7f324c0ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:19.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f32538c0700 1 -- 172.21.15.45:0/1308557960 >> 172.21.15.45:0/1308557960 conn(0x7f324c0cc7e0 msgr2=0x7f324c0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:19.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f32538c0700 1 -- 172.21.15.45:0/1308557960 shutdown_connections 2023-08-27T11:30:19.735 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.728+0000 7f32538c0700 1 -- 172.21.15.45:0/1308557960 wait complete. 2023-08-27T11:30:19.737 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.729+0000 7f32538c0700 1 Processor -- start 2023-08-27T11:30:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.729+0000 7f32538c0700 1 -- start start 2023-08-27T11:30:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f32538c0700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324c0aefb0 0x7f324c0c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f32538c0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f324c0cacd0 0x7f324c151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f32538c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c15d4f0 0x7f324c14d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f32538c0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c14e340 con 0x7f324c15d4f0 2023-08-27T11:30:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f32538c0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c14e620 con 0x7f324c0cacd0 2023-08-27T11:30:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f32538c0700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324c14e900 con 0x7f324c0aefb0 2023-08-27T11:30:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f3251e5d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c15d4f0 0x7f324c14d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f3251e5d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c15d4f0 0x7f324c14d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33130/0 (socket says 172.21.15.45:33130) 2023-08-27T11:30:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f3251e5d700 1 -- 172.21.15.45:0/1428112837 learned_addr learned my addr 172.21.15.45:0/1428112837 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f325165c700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324c0aefb0 0x7f324c0c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.730+0000 7f3250e5b700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f324c0cacd0 0x7f324c151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f3251e5d700 1 -- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324c0aefb0 msgr2=0x7f324c0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f3251e5d700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324c0aefb0 0x7f324c0c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f3251e5d700 1 -- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f324c0cacd0 msgr2=0x7f324c151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f3251e5d700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f324c0cacd0 0x7f324c151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f3251e5d700 1 -- 172.21.15.45:0/1428112837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3244009b30 con 0x7f324c15d4f0 2023-08-27T11:30:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f3251e5d700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c15d4f0 0x7f324c14d8f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f324400b370 tx=0x7f3244004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f3241ffb700 1 -- 172.21.15.45:0/1428112837 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244004500 con 0x7f324c15d4f0 2023-08-27T11:30:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f3241ffb700 1 -- 172.21.15.45:0/1428112837 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3244004680 con 0x7f324c15d4f0 2023-08-27T11:30:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c14ebe0 con 0x7f324c15d4f0 2023-08-27T11:30:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f3241ffb700 1 -- 172.21.15.45:0/1428112837 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244012430 con 0x7f324c15d4f0 2023-08-27T11:30:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.731+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c14f1a0 con 0x7f324c15d4f0 2023-08-27T11:30:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.732+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324c0726d0 con 0x7f324c15d4f0 2023-08-27T11:30:19.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.733+0000 7f3241ffb700 1 -- 172.21.15.45:0/1428112837 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 56) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f3244019030 con 0x7f324c15d4f0 2023-08-27T11:30:19.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.733+0000 7f3241ffb700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f323006a170 0x7f323006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:19.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.733+0000 7f3241ffb700 1 -- 172.21.15.45:0/1428112837 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7329+0+0 (secure 0 0 0) 0x7f3244089f80 con 0x7f324c15d4f0 2023-08-27T11:30:19.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.733+0000 7f325165c700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f323006a170 0x7f323006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:19.743 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.734+0000 7f325165c700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f323006a170 0x7f323006c630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3248006fd0 tx=0x7f3248008040 comp rx=0 tx=0).ready entity=mgr.7681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:19.744 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.737+0000 7f3241ffb700 1 -- 172.21.15.45:0/1428112837 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3244058a90 con 0x7f324c15d4f0 2023-08-27T11:30:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:19.937+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f324c14f390 con 0x7f324c15d4f0 2023-08-27T11:30:20.536 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.534+0000 7f3241ffb700 1 -- 172.21.15.45:0/1428112837 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3244058620 con 0x7f324c15d4f0 2023-08-27T11:30:20.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f323006a170 msgr2=0x7f323006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:20.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f323006a170 0x7f323006c630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3248006fd0 tx=0x7f3248008040 comp rx=0 tx=0).stop 2023-08-27T11:30:20.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c15d4f0 msgr2=0x7f324c14d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:20.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c15d4f0 0x7f324c14d8f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f324400b370 tx=0x7f3244004ad0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.542 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 shutdown_connections 2023-08-27T11:30:20.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f323006a170 0x7f323006c630 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f324c15d4f0 0x7f324c14d8f0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f324c0aefb0 0x7f324c0c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 --2- 172.21.15.45:0/1428112837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f324c0cacd0 0x7f324c151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.543 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 >> 172.21.15.45:0/1428112837 conn(0x7f324c0cc7e0 msgr2=0x7f324c070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:20.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 shutdown_connections 2023-08-27T11:30:20.544 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.539+0000 7f32538c0700 1 -- 172.21.15.45:0/1428112837 wait complete. 2023-08-27T11:30:20.544 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2023-08-27T11:30:20.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-27T11:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.687+0000 7f14bae5a700 1 Processor -- start 2023-08-27T11:30:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.687+0000 7f14bae5a700 1 -- start start 2023-08-27T11:30:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.688+0000 7f14bae5a700 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f14b40b5220 0x7f14b40cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.688+0000 7f14bae5a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b40cdf40 con 0x7f14b40b62c0 2023-08-27T11:30:20.692 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.688+0000 7f14bae5a700 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b40d0fd0 con 0x7f14b4160890 2023-08-27T11:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.688+0000 7f14bae5a700 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b40ce190 con 0x7f14b40b5220 2023-08-27T11:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.688+0000 7f14b93f7700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f14b40b62c0 0x7f14b40d1460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48146/0 (socket says 172.21.15.45:48146) 2023-08-27T11:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.688+0000 7f14b93f7700 1 -- 172.21.15.45:0/403369040 learned_addr learned my addr 172.21.15.45:0/403369040 (peer_addr_for_me v1:172.21.15.45:0/0) 2023-08-27T11:30:20.693 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.688+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924522829 0 0) 0x7f14b40cdf40 con 0x7f14b40b62c0 2023-08-27T11:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.688+0000 7f14abfff700 1 --2- 172.21.15.45:0/403369040 >> v2:172.21.15.135:3300/0 conn(0x7f14b40b5220 0x7f14b40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.688+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f149c0035b0 con 0x7f14b40b62c0 2023-08-27T11:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2723602168 0 0) 0x7f149c0035b0 con 0x7f14b40b62c0 2023-08-27T11:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 >> v2:172.21.15.135:3300/0 conn(0x7f14b40b5220 msgr2=0x7f14b40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:20.694 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 --2- 172.21.15.45:0/403369040 >> v2:172.21.15.135:3300/0 conn(0x7f14b40b5220 0x7f14b40cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 >> v1:172.21.15.135:6789/0 conn(0x7f14b4160890 legacy=0x7f14b40b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40ce470 con 0x7f14b40b62c0 2023-08-27T11:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1257659694 0 0) 0x7f14ac0019f0 con 0x7f14b40b62c0 2023-08-27T11:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 >> v1:172.21.15.45:6789/0 conn(0x7f14b40b62c0 legacy=0x7f14b40d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:20.695 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f149c003a00 0x7f149c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:20.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c004850 0x7f149c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:20.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c005210 0x7f149c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:20.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40ce190 con 0x7f149c005210 2023-08-27T11:30:20.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c00c1b0 con 0x7f149c004850 2023-08-27T11:30:20.696 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c00c370 con 0x7f149c003a00 2023-08-27T11:30:20.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14b93f7700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c005210 0x7f149c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:20.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14abfff700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c004850 0x7f149c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:20.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14b8bf6700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f149c003a00 0x7f149c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:20.697 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14b93f7700 1 -- 172.21.15.45:0/403369040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f149c003a00 msgr2=0x7f149c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14b93f7700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f149c003a00 0x7f149c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14b93f7700 1 -- 172.21.15.45:0/403369040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c004850 msgr2=0x7f149c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-27T11:30:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14b93f7700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c004850 0x7f149c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14b93f7700 1 -- 172.21.15.45:0/403369040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0035b0 con 0x7f149c005210 2023-08-27T11:30:20.698 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14abfff700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c004850 0x7f149c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-27T11:30:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14b93f7700 1 -- 172.21.15.45:0/403369040 reap_dead start 2023-08-27T11:30:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14b93f7700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c005210 0x7f149c00ba10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00a800 tx=0x7f149c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe3cd2ec9d8bef3 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00a2f0 con 0x7f149c005210 2023-08-27T11:30:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac012620 con 0x7f149c005210 2023-08-27T11:30:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.689+0000 7f14ab7fe700 1 -- 172.21.15.45:0/403369040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0129d0 con 0x7f149c005210 2023-08-27T11:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.690+0000 7f14bae5a700 1 -- 172.21.15.45:0/403369040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c005210 msgr2=0x7f149c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.690+0000 7f14bae5a700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c005210 0x7f149c00ba10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00a800 tx=0x7f149c005880 comp rx=0 tx=0).stop 2023-08-27T11:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.690+0000 7f14bae5a700 1 -- 172.21.15.45:0/403369040 shutdown_connections 2023-08-27T11:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.690+0000 7f14bae5a700 1 --2- 172.21.15.45:0/403369040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c005210 0x7f149c00ba10 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.690+0000 7f14bae5a700 1 -- 172.21.15.45:0/403369040 >> 172.21.15.45:0/403369040 conn(0x7f14b40cfb10 msgr2=0x7f14b40cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.690+0000 7f14bae5a700 1 -- 172.21.15.45:0/403369040 shutdown_connections 2023-08-27T11:30:20.701 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.690+0000 7f14bae5a700 1 -- 172.21.15.45:0/403369040 wait complete. 2023-08-27T11:30:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.691+0000 7f14bae5a700 1 Processor -- start 2023-08-27T11:30:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.691+0000 7f14bae5a700 1 -- start start 2023-08-27T11:30:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14bae5a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c005210 0x7f14b41f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14bae5a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b414cc50 0x7f14b4149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14bae5a700 1 --2- >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14b40a9a10 0x7f14b40a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14bae5a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40aa400 con 0x7f14b414cc50 2023-08-27T11:30:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14bae5a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b414a2e0 con 0x7f149c005210 2023-08-27T11:30:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14bae5a700 1 -- --> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b4149eb0 con 0x7f14b40a9a10 2023-08-27T11:30:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14abfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b414cc50 0x7f14b4149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14abfff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b414cc50 0x7f14b4149970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33148/0 (socket says 172.21.15.45:33148) 2023-08-27T11:30:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14abfff700 1 -- 172.21.15.45:0/1978886007 learned_addr learned my addr 172.21.15.45:0/1978886007 (peer_addr_for_me v2:172.21.15.45:0/0) 2023-08-27T11:30:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14b8bf6700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c005210 0x7f14b41f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.692+0000 7f14b93f7700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14b40a9a10 0x7f14b40a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14abfff700 1 -- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14b40a9a10 msgr2=0x7f14b40a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14abfff700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14b40a9a10 0x7f14b40a9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14abfff700 1 -- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c005210 msgr2=0x7f14b41f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14abfff700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c005210 0x7f14b41f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14abfff700 1 -- 172.21.15.45:0/1978886007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b41f34e0 con 0x7f14b414cc50 2023-08-27T11:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14b93f7700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14b40a9a10 0x7f14b40a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-27T11:30:20.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14abfff700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b414cc50 0x7f14b4149970 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f14a4010850 tx=0x7f14a4010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:20.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14a97fa700 1 -- 172.21.15.45:0/1978886007 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4010e50 con 0x7f14b414cc50 2023-08-27T11:30:20.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14a97fa700 1 -- 172.21.15.45:0/1978886007 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a4004020 con 0x7f14b414cc50 2023-08-27T11:30:20.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14a97fa700 1 -- 172.21.15.45:0/1978886007 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a40043b0 con 0x7f14b414cc50 2023-08-27T11:30:20.708 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b41f37c0 con 0x7f14b414cc50 2023-08-27T11:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.693+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b41f3d80 con 0x7f14b414cc50 2023-08-27T11:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.694+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b414a470 con 0x7f14b414cc50 2023-08-27T11:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.694+0000 7f14a97fa700 1 -- 172.21.15.45:0/1978886007 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 56) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f14a400e8d0 con 0x7f14b414cc50 2023-08-27T11:30:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.695+0000 7f14a97fa700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f14a006a170 0x7f14a006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-27T11:30:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.695+0000 7f14a97fa700 1 -- 172.21.15.45:0/1978886007 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6891+0+0 (secure 0 0 0) 0x7f14a4088be0 con 0x7f14b414cc50 2023-08-27T11:30:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.695+0000 7f14b8bf6700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f14a006a170 0x7f14a006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-27T11:30:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.695+0000 7f14b8bf6700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f14a006a170 0x7f14a006c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f14b40c8fa0 tx=0x7f14b0009040 comp rx=0 tx=0).ready entity=mgr.7681 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-27T11:30:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.699+0000 7f14a97fa700 1 -- 172.21.15.45:0/1978886007 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f14a4057a30 con 0x7f14b414cc50 2023-08-27T11:30:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:20.900+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f14b40724e0 con 0x7f14b414cc50 2023-08-27T11:30:21.319 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.317+0000 7f14a97fa700 1 -- 172.21.15.45:0/1978886007 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 57) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f14a4054370 con 0x7f14b414cc50 2023-08-27T11:30:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.539+0000 7f14a97fa700 1 -- 172.21.15.45:0/1978886007 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) v1 ==== 169+0+0 (secure 0 0 0) 0x7f14a40577a0 con 0x7f14b414cc50 2023-08-27T11:30:21.545 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.543+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f14a006a170 msgr2=0x7f14a006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.543+0000 7f14bae5a700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f14a006a170 0x7f14a006c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f14b40c8fa0 tx=0x7f14b0009040 comp rx=0 tx=0).stop 2023-08-27T11:30:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.543+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b414cc50 msgr2=0x7f14b4149970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-27T11:30:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.543+0000 7f14bae5a700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b414cc50 0x7f14b4149970 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f14a4010850 tx=0x7f14a4010c10 comp rx=0 tx=0).stop 2023-08-27T11:30:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.544+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 shutdown_connections 2023-08-27T11:30:21.546 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.544+0000 7f14bae5a700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:6832/3791167869,v1:172.21.15.135:6833/3791167869] conn(0x7f14a006a170 0x7f14a006c630 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.544+0000 7f14bae5a700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:3301/0,v1:172.21.15.135:6790/0] conn(0x7f14b40a9a10 0x7f14b40a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.544+0000 7f14bae5a700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f149c005210 0x7f14b41f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.544+0000 7f14bae5a700 1 --2- 172.21.15.45:0/1978886007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b414cc50 0x7f14b4149970 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-27T11:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.544+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 >> 172.21.15.45:0/1978886007 conn(0x7f14b40cfb10 msgr2=0x7f14b40c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-27T11:30:21.547 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.544+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 shutdown_connections 2023-08-27T11:30:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:2023-08-27T11:30:21.544+0000 7f14bae5a700 1 -- 172.21.15.45:0/1978886007 wait complete. 2023-08-27T11:30:21.548 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2023-08-27T11:30:21.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete